antlr-3.2/ 0000755 0001750 0001750 00000000000 11410174110 012422 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/ 0000755 0001750 0001750 00000000000 11410174107 016233 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/ 0000755 0001750 0001750 00000000000 11410174107 017022 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/site/ 0000755 0001750 0001750 00000000000 11410174107 017766 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/site/site.xml 0000644 0001750 0001750 00000001561 11256465172 021474 0 ustar twerner twerner
antlr-3.2/antlr3-maven-plugin/src/site/apt/ 0000755 0001750 0001750 00000000000 11410174107 020552 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/site/apt/examples/ 0000755 0001750 0001750 00000000000 11410174107 022370 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/site/apt/examples/simple.apt 0000644 0001750 0001750 00000002331 11256465172 024403 0 ustar twerner twerner Simple configuration
If your grammar files are organized into the default locations as described in the {{{../index.html}introduction}},
then configuring the pom.xml file for your project is as simple as adding this to it
+--
org.antlr
antlr3-maven-plugin
3.1.3-1
antlr
...
+--
When the mvn command is executed all grammar files under <<>>, except any
import grammars under <<>> will be analyzed and converted to
java source code in the output directory <<>>.
Your input files under <<>> should be stored in sub directories that
reflect the package structure of your java parsers. If your grammar file parser.g contains:
+---
@header {
package org.jimi.themuss;
}
+---
Then the .g file should be stored in: <<>>. THis way
the generated .java files will correctly reflect the package structure in which they will
finally rest as classes.
antlr-3.2/antlr3-maven-plugin/src/site/apt/examples/import.apt 0000644 0001750 0001750 00000000634 11256465172 024430 0 ustar twerner twerner Imported Grammar Files
In order to have the ANTLR plugin automatically locate and use grammars used
as imports in your main .g files, you need to place the imported grammar
files in the imports directory beneath the root directory of your grammar
files (which is <<>> by default of course).
For a default layout, place your import grammars in the directory: <<>>
antlr-3.2/antlr3-maven-plugin/src/site/apt/examples/libraries.apt 0000644 0001750 0001750 00000003624 11256465172 025074 0 ustar twerner twerner Libraries
The introduction of the import directive in a grammar allows reuse of common grammar files
as well as the ability to divide up functional components of large grammars. However it has
caused some confusion in regard to the fact that generated vocab files (<<>>) can also
be searched for with the <<<>>> directive.
This has confused two separate functions and imposes a structure upon the layout of
your grammar files in certain cases. If you have grammars that both use the import
directive and also require the use of a vocab file then you will need to locate
the grammar that generates the .tokens file alongside the grammar that uses it. This
is because you will need to use the <<<>>> directive to specify the
location of your imported grammars and ANTLR will not find any vocab files in
this directory.
The .tokens files for any grammars are generated within the same output directory structure
as the .java files. So, whereever the .java files are generated, you will also find the .tokens
files. ANTLR looks for .tokens files in both the <<<>>> and the output directory
where it is placing the geenrated .java files. Hence when you locate the grammars that generate
.tokens files in the same source directory as the ones that use the .tokens files, then
the Maven plugin will find the expected .tokens files.
The <<<>>> is specified like any other directory parameter in Maven. Here is an
example:
+--
org.antlr
antlr3-maven-plugin
3.1.3-1
antlr
src/main/antlr_imports
+--
antlr-3.2/antlr3-maven-plugin/src/site/apt/usage.apt.vm 0000644 0001750 0001750 00000015172 11256465172 023030 0 ustar twerner twerner Usage
The Maven plugin for antlr is simple to use but is at its simplest when you use the default
layouts for your grammars, as so:
+--
src/main/
|
+--- antlr3/... .g files organized in the required package structure
|
+--- imports/ .g files that are imported by other grammars.
+--
However, if you are not able to use this structure for whatever reason, you
can configure the locations of the grammar files, where library/import files
are located and where the output files should be generated.
* Plugin Descriptor
The current version of the plugin is shown at the top of this page after the <> date.
The full layout of the descriptor (at least, those parts that are not standard Maven things),
showing the default values of the configuration options, is as follows:
+--
org.antlr
antlr3-maven-plugin
3.1.3-1
antlr
10000
false
false
false
src/main/antlr3/imports
antlr
target/generated-sources/antlr3
false
false
false
src/main/antlr3
false
true
+--
Note that you can create multiple executions, and thus build some grammars with different
options to others (such as setting the debug option for instance).
** Configuration parameters
*** report
If set to true, then after the tool has processed an input grammar file
it will report variaous statistics about the parser, such as information
on cyclic DFAs, which rules may use backtracking, and so on.
default-value="false"
*** printGrammar
If set to true, then the ANTLR tool will print a version of the input
grammar which is devoid of any actions that may be present in the input file.
default-value = "false"
*** debug
If set to true, then the code generated by the ANTLR code generator will
be set to debug mode. This means that when run, the code will 'hang' and
wait for a debug connection on a TCP port (49100 by default).
default-value="false"
*** profile
If set to true, then then the generated parser will compute and report on
profile information at runtime.
default-value="false"
*** nfa
If set to true then the ANTLR tool will generate a description of the nfa
for each rule in Dot format
default-value="false"
protected boolean nfa;
*** dfa
If set to true then the ANTLR tool will generate a description of the DFA
for each decision in the grammar in Dot format
default-value="false"
*** trace
If set to true, the generated parser code will log rule entry and exit points
to stdout as an aid to debugging.
default-value="false"
*** messageFormat
If this parameter is set, it indicates that any warning or error messages returned
by ANLTR, shoould be formatted in the specified way. Currently, ANTLR supports the
built-in formats of antlr, gnu and vs2005.
default-value="antlr"
*** verbose
If this parameter is set to true, then ANTLR will report all sorts of things
about what it is doing such as the names of files and the version of ANTLR and so on.
default-value="true"
*** conversionTimeout
The number of milliseconds ANTLR will wait for analysis of each
alternative in the grammar to complete before giving up. You may raise
this value if ANTLR gives up on a complicated alt and tells you that
there are lots of ambiguties, but you know that it just needed to spend
more time on it. Note that this is an absolute time and not CPU time.
default-value="10000"
*** includes
Provides an explicit list of all the grammars that should
be included in the generate phase of the plugin. Note that the plugin
is smart enough to realize that imported grammars should be included but
not acted upon directly by the ANTLR Tool.
Unless otherwise specified, the include list scans for and includes all
files that end in ".g" in any directory beneath src/main/antlr3. Note that
this version of the plugin looks for the directory antlr3 and not the directory
antlr, so as to avoid clashes and confusion for projects that use both v2 and v3 grammars
such as ANTLR itself.
*** excludes
Provides an explicit list of any grammars that should be excluded from
the generate phase of the plugin. Files listed here will not be sent for
processing by the ANTLR tool.
*** sourceDirectory
Specifies the Antlr directory containing grammar files. For
antlr version 3.x we default this to a directory in the tree
called antlr3 because the antlr directory is occupied by version
2.x grammars.
<> Take careful note that the default location for antlr grammars
is now <> and NOT <>
default-value="<<<${basedir}/src/main/antlr3>>>"
*** outputDirectory
Location for generated Java files. For antlr version 3.x we default
this to a directory in the tree called antlr3 because the antlr
directory is occupied by version 2.x grammars.
default-value="<<<${project.build.directory}/generated-sources/antlr3>>>"
*** libDirectory
Location for imported token files, e.g. .tokens and imported grammars.
Note that ANTLR will not try to process grammars that it finds in this directory, but
will include this directory in the search for .tokens files and import grammars.
<> If you change the lib directory from the default but the directory is
still under<<<${basedir}/src/main/antlr3>>>, then you will need to exclude
the grammars from processing specifically, using the <<<>>> option.
default-value="<<<${basedir}/src/main/antlr3/imports>>>"
antlr-3.2/antlr3-maven-plugin/src/site/apt/index.apt 0000644 0001750 0001750 00000005476 11256465172 022420 0 ustar twerner twerner -------------
ANTLR v3 Maven Plugin
-------------
Jim Idle
-------------
March 2009
-------------
ANTLR v3 Maven plugin
The ANTLR v3 Maven plugin is completely re-written as of version 3.1.3; if you are familiar
with prior versions, you should note that there are some behavioral differences that make
it worthwhile reading this documentation.
The job of the plugin is essentially to tell the standard ANTLR parser generator where the
input grammar files are and where the output files should be generated. As with all Maven
plugins, there are defaults, which you are advised to comply to, but are not forced to
comply to.
This version of the plugin allows full control over ANTLR and allows configuration of all
options that are useful for a build system. The code required to calculate dependencies,
check the build order, and otherwise work with your grammar files is built into the ANTLR
tool as of version 3.1.3 of ANTLR and this plugin.
* Plugin Versioning
The plugin version tracks the version of the ANTLR tool that it controls. Hence if you
use version 3.1.3 of the plugin, you will build your grammars using version 3.1.3 of the
ANTLR tool, version 3.2 of the plugin will use version 3.2 of the ANTLR tool and so on.
You may also find that there are patch versions of the plugin suchas 3.1.3-1 3.1.3-2 and
so on. Use the latest patch release of the plugin.
The current version of the plugin is shown at the top of this page after the <> date.
* Default directories
As with all Maven plugins, this plugin will automatically default to standard locations
for your grammar and import files. Organizing your source code to reflect this standard
layout will greatly reduce the configuration effort required. The standard layout lookd
like this:
+--
src/main/
|
+--- antlr3/... .g files organized in the required package structure
|
+--- imports/ .g files that are imported by other grammars.
+--
If your grammar is intended to be part of a package called org.foo.bar then you would
place it in the directory <<>>. The plugin will then produce
.java and .tokens files in the output directory <<>>
When the Java files are compiled they will be in the correct location for the javac
compiler without any special configuration. The generated java files are automatically
submitted for compilation by the plugin.
The <<>> directory is treated in a special way. It should contain
any grammar files that are imported by other grammar files (do not make subdirectories here.)
Such files are never built on their own, but the plugin will automatically tell the ANTLR
tool to look in this directory for library files.
antlr-3.2/antlr3-maven-plugin/src/main/ 0000755 0001750 0001750 00000000000 11410174107 017746 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/main/java/ 0000755 0001750 0001750 00000000000 11410174107 020667 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/main/java/org/ 0000755 0001750 0001750 00000000000 11410174107 021456 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/main/java/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 022576 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/main/java/org/antlr/mojo/ 0000755 0001750 0001750 00000000000 11410174107 023542 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/main/java/org/antlr/mojo/antlr3/ 0000755 0001750 0001750 00000000000 11410174107 024745 5 ustar twerner twerner antlr-3.2/antlr3-maven-plugin/src/main/java/org/antlr/mojo/antlr3/Antlr3ErrorLog.java 0000644 0001750 0001750 00000005713 11256465172 030452 0 ustar twerner twerner /**
[The "BSD licence"]
ANTLR - Copyright (c) 2005-2008 Terence Parr
Maven Plugin - Copyright (c) 2009 Jim Idle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.mojo.antlr3;
import org.antlr.tool.ANTLRErrorListener;
import org.antlr.tool.Message;
import org.antlr.tool.ToolMessage;
import org.apache.maven.plugin.logging.Log;
/**
* The Maven plexus container gives us a Log logging provider
* which we can use to install an error listener for the ANTLR
* tool to report errors by.
*/
public class Antlr3ErrorLog implements ANTLRErrorListener {
private Log log;
/**
* Instantiate an ANTLR ErrorListner that communicates any messages
* it receives to the Maven error sink.
*
* @param log The Maven Error Log
*/
public Antlr3ErrorLog(Log log) {
this.log = log;
}
/**
* Sends an informational message to the Maven log sink.
* @param s The message to send to Maven
*/
public void info(String message) {
log.info(message);
}
/**
* Sends an error message from ANTLR analysis to the Maven Log sink.
*
* @param message The message to send to Maven.
*/
public void error(Message message) {
log.error(message.toString());
}
/**
* Sends a warning message to the Maven log sink.
*
* @param message
*/
public void warning(Message message) {
log.warn(message.toString());
}
/**
* Sends an error message from the ANTLR tool to the Maven Log sink.
* @param toolMessage
*/
public void error(ToolMessage toolMessage) {
log.error(toolMessage.toString());
}
}
antlr-3.2/antlr3-maven-plugin/src/main/java/org/antlr/mojo/antlr3/Antlr3Mojo.java 0000644 0001750 0001750 00000044160 11256465172 027622 0 ustar twerner twerner /**
[The "BSD licence"]
ANTLR - Copyright (c) 2005-2008 Terence Parr
Maven Plugin - Copyright (c) 2009 Jim Idle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* ========================================================================
* This is the definitive ANTLR3 Mojo set. All other sets are belong to us.
*/
package org.antlr.mojo.antlr3;
import antlr.RecognitionException;
import antlr.TokenStreamException;
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugin.MojoExecutionException;
import org.apache.maven.plugin.MojoFailureException;
import org.apache.maven.project.MavenProject;
import java.io.File;
import java.io.IOException;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
import org.antlr.Tool;
import org.apache.maven.plugin.logging.Log;
import org.codehaus.plexus.compiler.util.scan.InclusionScanException;
import org.codehaus.plexus.compiler.util.scan.SimpleSourceInclusionScanner;
import org.codehaus.plexus.compiler.util.scan.SourceInclusionScanner;
import org.codehaus.plexus.compiler.util.scan.mapping.SourceMapping;
import org.codehaus.plexus.compiler.util.scan.mapping.SuffixMapping;
/**
* Goal that picks up all the ANTLR grammars in a project and moves those that
* are required for generation of the compilable sources into the location
* that we use to compile them, such as target/generated-sources/antlr3 ...
*
* @goal antlr
*
* @phase process-sources
* @requiresDependencyResolution compile
* @requiresProject true
*
* @author Jim Idle
*/
public class Antlr3Mojo
extends AbstractMojo {
// First, let's deal with the options that the ANTLR tool itself
// can be configured by.
//
/**
* If set to true, then after the tool has processed an input grammar file
* it will report variaous statistics about the parser, such as information
* on cyclic DFAs, which rules may use backtracking, and so on.
*
* @parameter default-value="false"
*/
protected boolean report;
/**
* If set to true, then the ANTLR tool will print a version of the input
* grammar which is devoid of any actions that may be present in the input file.
*
* @parameter default-value="false"
*/
protected boolean printGrammar;
/**
* If set to true, then the code generated by the ANTLR code generator will
* be set to debug mode. This means that when run, the code will 'hang' and
* wait for a debug connection on a TCP port (49100 by default).
*
* @parameter default-value="false"
*/
protected boolean debug;
/**
* If set to true, then then the generated parser will compute and report on
* profile information at runtime.
*
* @parameter default-value="false"
*/
protected boolean profile;
/**
* If set to true then the ANTLR tool will generate a description of the nfa
* for each rule in Dot format
*
* @parameter default-value="false"
*/
protected boolean nfa;
/**
* If set to true then the ANTLR tool will generate a description of the DFA
* for each decision in the grammar in Dot format
*
* @parameter default-value="false"
*/
protected boolean dfa;
/**
* If set to true, the generated parser code will log rule entry and exit points
* to stdout as an aid to debugging.
*
* @parameter default-value="false"
*/
protected boolean trace;
/**
* If this parameter is set, it indicates that any warning or error messages returned
* by ANLTR, shoould be formatted in the specified way. Currently, ANTLR suports the
* built-in formats of antlr, gnu and vs2005.
*
* @parameter default-value="antlr"
*/
protected String messageFormat;
/**
* If this parameter is set to true, then ANTLR will report all sorts of things
* about what it is doing such as the names of files and the version of ANTLR and so on.
*
* @parameter default-value="true"
*/
protected boolean verbose;
/**
* The number of milliseconds ANTLR will wait for analysis of each
* alternative in the grammar to complete before giving up. You may raise
* this value if ANTLR gives up on a complicated alt and tells you that
* there are lots of ambiguties, but you know that it just needed to spend
* more time on it. Note that this is an absolute time and not CPU time.
*
* @parameter default-value="10000"
*/
private int conversionTimeout;
/**
* The number of alts, beyond which ANTLR will not generate a switch statement
* for the DFA.
*
* @parameter default-value="300"
*/
private int maxSwitchCaseLabels;
/**
* The number of alts, below which ANTLR will not choose to generate a switch
* statement over an if statement.
*/
private int minSwitchAlts;
/* --------------------------------------------------------------------
* The following are Maven specific parameters, rather than specificlly
* options that the ANTLR tool can use.
*/
/**
* Provides an explicit list of all the grammars that should
* be included in the generate phase of the plugin. Note that the plugin
* is smart enough to realize that imported grammars should be included but
* not acted upon directly by the ANTLR Tool.
*
* Unless otherwise specified, the include list scans for and includes all
* files that end in ".g" in any directory beneath src/main/antlr3. Note that
* this version of the plugin looks for the directory antlr3 and not the directory
* antlr, so as to avoid clashes and confusion for projects that use both v2 and v3 grammars
* such as ANTLR itself.
*
* @parameter
*/
protected Set includes = new HashSet();
/**
* Provides an explicit list of any grammars that should be excluded from
* the generate phase of the plugin. Files listed here will not be sent for
* processing by the ANTLR tool.
*
* @parameter
*/
protected Set excludes = new HashSet();
/**
* @parameter expression="${project}"
* @required
* @readonly
*/
protected MavenProject project;
/**
* Specifies the Antlr directory containing grammar files. For
* antlr version 3.x we default this to a directory in the tree
* called antlr3 because the antlr directory is occupied by version
* 2.x grammars.
*
* @parameter default-value="${basedir}/src/main/antlr3"
* @required
*/
private File sourceDirectory;
/**
* Location for generated Java files. For antlr version 3.x we default
* this to a directory in the tree called antlr3 because the antlr
* directory is occupied by version 2.x grammars.
*
* @parameter default-value="${project.build.directory}/generated-sources/antlr3"
* @required
*/
private File outputDirectory;
/**
* Location for imported token files, e.g. .tokens and imported grammars.
* Note that ANTLR will not try to process grammars that it finds to be imported
* into other grammars (in the same processing session).
*
* @parameter default-value="${basedir}/src/main/antlr3/imports"
*/
private File libDirectory;
public File getSourceDirectory() {
return sourceDirectory;
}
public File getOutputDirectory() {
return outputDirectory;
}
public File getLibDirectory() {
return libDirectory;
}
void addSourceRoot(File outputDir) {
project.addCompileSourceRoot(outputDir.getPath());
}
/**
* An instance of the ANTLR tool build
*/
protected Tool tool;
/**
* The main entry point for this Mojo, it is responsible for converting
* ANTLR 3.x grammars into the target language specified by the grammar.
*
* @throws org.apache.maven.plugin.MojoExecutionException When something is disvocered such as a missing source
* @throws org.apache.maven.plugin.MojoFailureException When something really bad happesn such as not being able to create the ANTLR Tool
*/
public void execute()
throws MojoExecutionException, MojoFailureException {
Log log = getLog();
// Check to see if the user asked for debug information, then dump all the
// parameters we have picked up if they did.
//
if (log.isDebugEnabled()) {
// Excludes
//
for (String e : (Set) excludes) {
log.debug("ANTLR: Exclude: " + e);
}
// Includes
//
for (String e : (Set) includes) {
log.debug("ANTLR: Include: " + e);
}
// Output location
//
log.debug("ANTLR: Output: " + outputDirectory);
// Library directory
//
log.debug("ANTLR: Library: " + libDirectory);
// Flags
//
log.debug("ANTLR: report : " + report);
log.debug("ANTLR: printGrammar : " + printGrammar);
log.debug("ANTLR: debug : " + debug);
log.debug("ANTLR: profile : " + profile);
log.debug("ANTLR: nfa : " + nfa);
log.debug("ANTLR: dfa : " + dfa);
log.debug("ANTLR: trace : " + trace);
log.debug("ANTLR: messageFormat : " + messageFormat);
log.debug("ANTLR: conversionTimeout : " + conversionTimeout);
log.debug("ANTLR: maxSwitchCaseLabels : " + maxSwitchCaseLabels);
log.debug("ANTLR: minSwitchAlts : " + minSwitchAlts);
log.debug("ANTLR: verbose : " + verbose);
}
// Ensure that the output directory path is all in tact so that
// ANTLR can just write into it.
//
File outputDir = getOutputDirectory();
if (!outputDir.exists()) {
outputDir.mkdirs();
}
// First thing we need is an instance of the ANTLR 3.1 build tool
//
try {
// ANTLR Tool buld interface
//
tool = new Tool();
} catch (Exception e) {
log.error("The attempt to create the ANTLR build tool failed, see exception report for details");
throw new MojoFailureException("Jim failed you!");
}
// Next we need to set the options given to us in the pom into the
// tool instance we have created.
//
tool.setConversionTimeout(conversionTimeout);
tool.setDebug(debug);
tool.setGenerate_DFA_dot(dfa);
tool.setGenerate_NFA_dot(nfa);
tool.setProfile(profile);
tool.setReport(report);
tool.setPrintGrammar(printGrammar);
tool.setTrace(trace);
tool.setVerbose(verbose);
tool.setMessageFormat(messageFormat);
tool.setMaxSwitchCaseLabels(maxSwitchCaseLabels);
tool.setMinSwitchAlts(minSwitchAlts);
// Where do we want ANTLR to produce its output? (Base directory)
//
if (log.isDebugEnabled())
{
log.debug("Output directory base will be " + outputDirectory.getAbsolutePath());
}
tool.setOutputDirectory(outputDirectory.getAbsolutePath());
// Tell ANTLR that we always want the output files to be produced in the output directory
// using the same relative path as the input file was to the input directory.
//
tool.setForceRelativeOutput(true);
// Where do we want ANTLR to look for .tokens and import grammars?
//
tool.setLibDirectory(libDirectory.getAbsolutePath());
if (!sourceDirectory.exists()) {
if (log.isInfoEnabled()) {
log.info("No ANTLR grammars to compile in " + sourceDirectory.getAbsolutePath());
}
return;
} else {
if (log.isInfoEnabled()) {
log.info("ANTLR: Processing source directory " + sourceDirectory.getAbsolutePath());
}
}
// Set working directory for ANTLR to be the base source directory
//
tool.setInputDirectory(sourceDirectory.getAbsolutePath());
try {
// Now pick up all the files and process them with the Tool
//
processGrammarFiles(sourceDirectory, outputDirectory);
} catch (InclusionScanException ie) {
log.error(ie);
throw new MojoExecutionException("Fatal error occured while evaluating the names of the grammar files to analyze");
} catch (Exception e) {
getLog().error(e);
throw new MojoExecutionException(e.getMessage());
}
tool.process();
// If any of the grammar files caused errors but did nto throw exceptions
// then we should have accumulated errors in the counts
//
if (tool.getNumErrors() > 0) {
throw new MojoExecutionException("ANTLR caught " + tool.getNumErrors() + " build errors.");
}
// All looks good, so we need to tel Maven about the sources that
// we just created.
//
if (project != null) {
// Tell Maven that there are some new source files underneath
// the output directory.
//
addSourceRoot(this.getOutputDirectory());
}
}
/**
*
* @param sourceDirectory
* @param outputDirectory
* @throws antlr.TokenStreamException
* @throws antlr.RecognitionException
* @throws java.io.IOException
* @throws org.codehaus.plexus.compiler.util.scan.InclusionScanException
*/
private void processGrammarFiles(File sourceDirectory, File outputDirectory)
throws TokenStreamException, RecognitionException, IOException, InclusionScanException {
// Which files under the source set should we be looking for as grammar files
//
SourceMapping mapping = new SuffixMapping("g", Collections.EMPTY_SET);
// What are the sets of includes (defaulted or otherwise).
//
Set includes = getIncludesPatterns();
// Now, to the excludes, we need to add the imports directory
// as this is autoscanned for importd grammars and so is auto-excluded from the
// set of gramamr fiels we shuold be analyzing.
//
excludes.add("imports/**");
SourceInclusionScanner scan = new SimpleSourceInclusionScanner(includes, excludes);
scan.addSourceMapping(mapping);
Set grammarFiles = scan.getIncludedSources(sourceDirectory, null);
if (grammarFiles.isEmpty()) {
if (getLog().isInfoEnabled()) {
getLog().info("No grammars to process");
}
} else {
// Tell the ANTLR tool that we want sorted build mode
//
tool.setMake(true);
// Iterate each grammar file we were given and add it into the tool's list of
// grammars to process.
//
for (File grammar : (Set) grammarFiles) {
if (getLog().isDebugEnabled()) {
getLog().debug("Grammar file '" + grammar.getPath() + "' detected.");
}
String relPath = findSourceSubdir(sourceDirectory, grammar.getPath()) + grammar.getName();
if (getLog().isDebugEnabled()) {
getLog().debug(" ... relative path is: " + relPath);
}
tool.addGrammarFile(relPath);
}
}
}
public Set getIncludesPatterns() {
if (includes == null || includes.isEmpty()) {
return Collections.singleton("**/*.g");
}
return includes;
}
/**
* Given the source directory File object and the full PATH to a
* grammar, produce the path to the named grammar file in relative
* terms to the sourceDirectory. This will then allow ANTLR to
* produce output relative to the base of the output directory and
* reflect the input organization of the grammar files.
*
* @param sourceDirectory The source directory File object
* @param grammarFileName The full path to the input grammar file
* @return The path to the grammar file relative to the source directory
*/
private String findSourceSubdir(File sourceDirectory, String grammarFileName) {
String srcPath = sourceDirectory.getPath() + File.separator;
if (!grammarFileName.startsWith(srcPath)) {
throw new IllegalArgumentException("expected " + grammarFileName + " to be prefixed with " + sourceDirectory);
}
File unprefixedGrammarFileName = new File(grammarFileName.substring(srcPath.length()));
return unprefixedGrammarFileName.getParent() + File.separator;
}
}
antlr-3.2/antlr3-maven-plugin/pom.xml 0000644 0001750 0001750 00000027322 11256466522 017573 0 ustar twerner twerner
4.0.0
org.antlr
antlr3-maven-plugin
maven-plugin
3.2
Maven plugin for ANTLR V3
2.0
http://antlr.org
UTF-8
This is the brand new, re-written from scratch plugin for ANTLR v3.
Previous valiant efforts all suffered from being unable to modify the ANTLR Tool
itself to provide support not just for Maven oriented things but any other tool
that might wish to invoke ANTLR without resorting to the command line interface.
Rather than try to shoe-horn new code into the existing Mojo (in fact I think that
by incorporating a patch supplied by someone I ended up with tow versions of the
Mojo, I elected to rewrite everything from scratch, including the documentation, so
that we might end up with a perfect Mojo that can do everything that ANTLR v3 supports
such as imported grammar processing, proper support for library directories and
locating token files from generated sources, and so on.
In the end I decided to also change the the ANTLR Tool.java code so that it
would be the provider of all the things that a build tool needs, rather than
delegating things to 5 different tools. So, things like dependencies, dependency
sorting, option tracking, generating sources and so on are all folded back
in to ANTLR's Tool.java code, where they belong, and they now provide a
public interface to anyone that might want to interface with them.
One other goal of this rewrite was to completely document the whole thing
to death. Hence even this pom has more comments than funcitonal elements,
in case I get run over by a bus or fall off a cliff while skiing.
Jim Idle - March 2009
Jim Idle
http://www.temporal-wave.com
Originator, version 3.1.3
Terence Parr
http://antlr.org/wiki/display/~admin/Home
Project lead - ANTLR
David Holroyd
http://david.holroyd.me.uk/
Originator - prior version
Kenny MacDermid
mailto:kenny "at" kmdconsulting.ca
Contributor - prior versions
hudson
http://antlr.org/hudson/job/Maven_Plugin/lastSuccessfulBuild/
rss
http://antlr.org/hudson/job/Maven_Plugin/rssAll
JIRA
http://antlr.org/jira/browse/ANTLR
repo
The BSD License
http://www.antlr.org/LICENSE.txt
antlr-repo
ANTLR Testing repository
scpexe://antlr.org/home/mavensync/antlr-repo
antlr-snapshot
ANTLR Testing Snapshot Repository
scpexe://antlr.org/home/mavensync/antlr-snapshot
antlr-repo
ANTLR Maven Plugin Web Site
scpexe://antlr.org/home/mavensync/antlr-maven-webs/antlr3-maven-plugin
antlr-snapshot
ANTLR Testing Snapshot Repository
http://antlr.org/antlr-snapshot
true
always
false
2009
http://antlr.markmail.org/
http://www.antlr.org/pipermail/antlr-interest/
ANTLR Users
http://www.antlr.org/mailman/listinfo/antlr-interest/
http://www.antlr.org/mailman/options/antlr-interest/
antlr-interest@antlr.org
ANTLR.org
http://www.antlr.org
org.apache.maven
maven-plugin-api
2.0
compile
org.apache.maven
maven-project
2.0
org.codehaus.plexus
plexus-compiler-api
1.5.3
org.antlr
antlr
3.2
junit
junit
4.5
test
org.apache.maven.shared
maven-plugin-testing-harness
1.0
test
install
org.apache.maven.wagon
wagon-ssh-external
1.0-beta-2
maven-compiler-plugin
2.0.2
1.5
jsr14
org.apache.maven.plugins
maven-site-plugin
2.0
org.apache.maven.plugins
maven-project-info-reports-plugin
2.1.1
false
antlr-3.2/BUILD.txt 0000644 0001750 0001750 00000052145 11256465172 014054 0 ustar twerner twerner [The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
Maven Plugin - Copyright (c) 2009 Jim Idle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
============================================================================
This file contains the build instructions for the ANTLR toolset as
of version 3.1.3 and beyond.
The ANTLR toolset must be built using the Maven build system as
this build system updates the version numbers and controls the
whole build process. However, if you just want the latest build
and do not care to learn anything about Maven, then visit:
http://antlr.org/hudson
And download the current complete jar from the Tool_Daily
Hudson project (just follow the links for last successful build).
At the time of writing, the link for the last successful
snapshot build is:
http://antlr.org/hudson/job/ANTLR_Tool_Daily/lastSuccessfulBuild/org.antlr$antlr/
If you are looking for the latest released version of ANTLR, then
visit the downloads page on the main antlr.org website.
These instructions are mainly for the ANTLR development team,
though you are free to build ANTLR yourself of course.
Source code Structure
-----------------------
The main development branch of ANTLR is stored within the Perforce SCM at:
//depot/code/antlr/main/...
release branches are stored in Perforce like so:
//depot/code/antlr/release-3.1.3/...
In this top level directory, you will find a master build file for Maven called pom.xml and
you will also note that there are a number of subdirectories:
tool - The ANTLR tool itself
runtime/Java - The ANTLR Java runtime
runtime/X - The runtime for language target X
gunit - The grammar test tool
antlr3-maven-plugin - The plugin tool for Maven that allows Maven projects to process
ANTLR grammars.
Each of these sub-directories also contains a file pom.xml that controls the build of each
sub-component (or module in Maven parlance).
Build Parameters
-----------------
Alongside each pom.xml (other than for the antlr3-maven-plugin), you will see that there
is a file called antlr.config. This file is called a filter and should contain a set of key/value
pairs in the same manner as Java properties files:
antlr.something="Some config thang!"
When the build of any component happens, any values in the antlr.config for the master
build file and any values in the antlr.config file for each component are made available
to the build. This is mainly used by the resource processor, which will filter any file it
finds under: src/main/resources/** and replace any references such as ${antlr.something}
with the actual value at the time of the build.
Building
--------
Building ANTLR is trivial, assuming that you have loaded Maven version 2.0.9 or
better on to your build system and installed it as explained here:
http://maven.apache.org/download.html
If you are unfamiliar with Maven (and even if you are), the best resource for learning
about it is The Definitive Guide:
http://www.sonatype.com/books/maven-book/reference/public-book.html
The instructions here assume that Maven is installed and working correctly.
If this is the first time you have built the ANTLR toolset, you will possibly
need to install the master pom in your local repository (however the build
may be able to locate this in the ANTLR snapshot or release repository). If you try
to build sub-modules on their own (as in run the mvn command in the sub directory
for that tool, such as runtime/Java), and you receive a message that
maven cannot find the master pom, then execute this in the main (or release)
directory:
mvn -N install
This command will install the master build pom in your local maven repository
(it's ~/.m2 on UNIX) and individual builds of sub-modules will now work correctly.
To build then, simply cd into the master build directory (e.g. $P4ROOT//code/antlr/main)
and type:
mvn -Dmaven.test.skip=true
Assuming that everything is correctly installed and synchronized, then ANTLR will build
and skip any unit tests in the modules (the ANTLR tool tests can take a long time).
This command will build each of the tools in the correct order and will create the jar
artifacts of all the components in your local development Maven repository (which
takes precedence over remote repositories by default). At the end of the build you
should see:
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] ANTLR Master build control POM ........................ SUCCESS [1.373s]
[INFO] Antlr 3 Runtime ....................................... SUCCESS [0.879s]
[INFO] ANTLR Grammar Tool .................................... SUCCESS [5.431s]
[INFO] Maven plugin for ANTLR V3 ............................. SUCCESS [1.277s]
[INFO] ANTLR gUnit ........................................... SUCCESS [1.566s]
[INFO] Maven plugin for gUnit ANTLR V3 ....................... SUCCESS [0.079s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11 seconds
However, unless you are using Maven exclusively in your projects, you will most
likely want to build the ANTLR Uber Jar, which is an executable jar containing
all the components that ANTLR needs to build and run parsers (note that at
runtime, you need only the runtime components you use, such as the Java
runtime and say stringtemplate).
Because the Uber jar is not something we want to deploy to Maven repositories
it is built with a special invocation of Maven:
mvn -Dmaven.test.skip=true package assembly:assembly
Note that Maven will appear to build everything twice, which is a quirk of how
it calculates the dependencies and makes sure it has everything packaged up
so it can build the uber-jar assembly.
Somewhere in the build output (towards the end), you will find a line like this:
[INFO] Building jar: /home/jimi/antlrsrc/code/antlr/main/target/antlr-master-3.1.3-SNAPSHOT-completejar.jar
This is the executable jar that you need and you can either copy it somewhere or,
like me, you can create this script (assuming UNIX) somewhere in your PATH:
#! /bin/bash
java -jar ~/antlrsrc/code/antlr/main/target/antlr-master-3.1.3-SNAPSHOT-completejar.jar $*
Version Numbering
-------------------
The first and Golden rule is that any pom files stored under the main branch of the toolset
should never be modified to contain a release version number. They should always contain
a.b.c-SNAPSHOT (e.g. 3.1.3-SNAPSHOT). Only release branches should have their
pom version numbers set to a release version. You can release as many SNAPSHOTS
as you like, but only one release version. However, release versions may be updated
with a patch level: 3.1.3-1, 3.1.3-2 and so on.
Fortunately, Maven helps us with the version numbering in a number of ways. Firstly,
the pom.xml files for the various modules do not specify a version of the
artifacts themselves. They pick up their version number from the master build pom.
However, there is a catch, because they need to know what version of the parent pom
they inherit from and so they DO mention the version number. However, this does
prevent accidentally releasing different versions of sub-modules than the master pom
describes.
Fortunately once again, Maven has a neat way of helping us with change the version.
All you need do is check out all the pom.xml files from perforce, then modify the
a.b.c-SNAPSHOT in the master pom. When the version number
is correct in the master pom, you make sure your working directory is the location
of the master pom and type:
mvn versions:update-child-modules
This command will then update the child pom.xml files to reflect the version number
defined in the master pom.xml.
There is unfortunately one last catch here though and that is that the antlr3-maven-plugin
and the gunit-maven-plugin are not able to use the parent pom. The reason for
this is subtle but makes sense as doing so would create a circular dependency
between the ANTLR tool (which uses the plugin to build its own grammar files),
and the plugins (which uses the tool to build grammar files and gunit to test).
This catch-22 situation means that the pom.xml file in the antlr3-maven-plugin directory
and the one in the gunit-maven-plugin directory MUST be updated manually (or we
must write a script to do this).
Finally, we need to remember that because the tool is dependent on the antlr3-maven-plugin
and the plugin is itself dependent on the the tool, that we must manually update
the versions of each that they reference. So, when we bump the version of the toolset
to say 3.1.4-SNAPSHOT, we need to change the antlr3-maven-plugin pom.xml and the
gunit-maven-plugin pom.xml to reference that version of the antlr tool. The tool
itself is always built with the prior released version of the plugin, so when we
release we must change the main branch of the plugin to use the newly released
version of the plugin. This is covered in the release checklist.
Deploying
----------
Deploying the tools at the current version is relatively easy, but to deploy to the
ANTLR repositories (snapshot or release) you must have been granted access
to the antlr.org server and supplied an ssh key. Few people will have this access of
course.
Assuming that you have ssh access to antlr.org, then you will need to do the following
before deployment will authorize and work correctly (UNIX assumed here):
$ eval `ssh-agent`
Agent PID nnnnn
$ ssh-add
Enter passphrase for /home/you/.ssh/id_rsa:
Identity added....
Next, because we do not publish access information for antlr.org, you will need
to configure the repository server names locally. You do this by creating (or
adding to) the file:
~/.m2/settings.xml
Which should look like this:
antlr-snapshot
mavensync
passphrase for your private key
/home/youruserlogin/.ssh/id_rsa
antlr-repo
mavensync
passphrase for your private key
/home/youruserlogin/.ssh/id_rsa
When this configuration is in place, you will be able to deploy the components,
either individually or from the master directory:
mvn -Dmaven.test.skip=true deploy
You will then see lots of information about checking existing version information
and so on, and the components will be deployed.
Note that so long as the artifacts are versioned with a.b.c-SNAPSHOT then
deployment will always be to the development snapshot directory. When the
artifacts are versioned with a release version then deployment will be to the
antlr.org release repository, which will then be mirrored around the world. It
is important not to deploy a release until you have built and tested it to your
satisfaction.
Release Checklist
------------------
Here is the procedure to use to make a release of ANTLR. Note that we should
really use the mvn release:release command, but the perforce plugin for Maven is
not commercial quality and I want to rewrite it.
For this checklist, let's assume that the current development version of ANTLR
is 3.1.3-SNAPSHOT. This means that it will probably (but not necessarily)
become release version 3.1.3 and that the development version will bump
to 3.1.4-SNAPSHOT.
0) Run a build of the main branch and check that it is builds and passes as many
tests as you want it to.
1) First make a branch from main into the target release directory. Then submit
this to perforce. You could change versions numbers before submitting, but
doing that in separate stages will keep things sane;
--- Use main development branch from here ---
2) Before we deploy the release, we want to update the versions of the development
branch, so we don't deploy what is now the new release as an older snapshot (this
is not super important, but procedure is good right?).
Check out all the pom.xml files (and if you are using any antlr.config parameters
that must change, then do that too).
3) Edit the master pom.xml in the main directory and change the version from
3.1.3-SNAPSHOT to 3.1.4-SNAPSHOT.
4) Edit the pom.xml file for antlr3-maven-plugin under the main directory and
change the version from 3.1.3-SNAPSHOT to 3.1.4-SNAPSHOT. Do the same for the
pom.xml in the gunit-maven-plugin directory.
5) Now (from the main directory), run the command:
mvn versions:update-child-modules
You should see:
[INFO] [versions:update-child-modules]
[INFO] Module: gunit
[INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
[INFO] Module: runtime/Java
[INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
[INFO] Module: tool
[INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
6) Run a build of the main branch:
mvn -Dmaven.test.skip=true
All should be good.
7) Submit the pom changes of the main branch to perforce.
8) Deploy the new snapshot as a placeholder for the next release. It
will go to the snapshot repository of course:
mvn -N deploy
mvn -Dmaven.test.skip=true deploy
9) You are now finished with the main development branch and should change
working directories to the release branch you made earlier.
--- Use release branch from here ---
10) Check out all the pom.xml files in the release branch (and if you are
using any antlr.config parameters that must change, then do that too).
11) Edit the master pom.xml in the release-3.1.3 directory and change the version from
3.1.3-SNAPSHOT to 3.1.3.
12) Edit the pom.xml file for antlr3-maven-plugin under the release-3.1.3 directory and
change the version from 3.1.3-SNAPSHOT to 3.1.3. Also change the version of
the tool that the this pom.xml references from 3.1.3-SNAPSHOT to 3.1.3 as we
are now releasing the plugin of course and it needs to reference the version
we are about to release. You will find this reference in the dependencies
section of the antlr3-maven-plugin pom.xml. Also change the version references
in the pom for gunit-maven-plugin.
13) Now (from the release-3.1.3 directory), run the command:
mvn versions:update-child-modules
You should see:
[INFO] [versions:update-child-modules]
[INFO] Module: gunit
[INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
now org.antlr:antlr-master:3.1.3
[INFO] Module: runtime/Java
[INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
now org.antlr:antlr-master:3.1.3
[INFO] Module: tool
[INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
now org.antlr:antlr-master:3.1.3
14) Run a build of the release-3.1.3 branch:
mvn # Note I am letting unit tests run here!
All should be good, or as good as it gets ;-)
15) Submit the pom changes of the release-3.1.3 branch to perforce.
16) Deploy the new release (this is it guys, make sure you are happy):
mvn -N deploy
mvn -Dmaven.test.skip=true deploy
Note that we must skip the tests as Maven will not let you deploy releases
that fail any junit tests.
17) The final step is that we must update the main branch pom.xml for the
tool to reference the newly release version of the antlr3-maven-plugin. This is
because each release of ANTLR is built with the prior release of ANTLR, and
we have just released a new version. Edit the pom.xml for the tool (main/tool/pom.xml)
under the main (that's the MAIN branch, not the release branch) and find
the dependency reference to the antlr plugin. If you just released say
3.1.3, then the tool should now reference version 3.1.3 of the plugin. Having done
this, you should probably rebuild the main branch and let it run the junit
tests. Later, I will automate this dependency update as mvn can do this
for us.
18) Having deployed the release to maven, you will want to create the
uber jar for the new release, to make it downloadable from the
antlr.org website. This is a repeat of the earlier described step
to build the uber jar:
mvn =Dmaven.test.skip=true package assembly:assembly
MAven will produce the uber jar in the target directory:
antlr-master-3.1.3-completejar.jar
And this is the complete jar that can be downloaded from the web site. You
may wish to produce an md5 checksum to go with the jar:
md5sum target/antlr-master-3.1.3-completejar.jar
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx target/antlr-master-3.1.4-SNAPSHOT-completejar.jar
The command you just ran will also produce a second jar:
antlr-master-3.1.3-src.jar
This is the source code for everythign you just deployed and can be unjarred
and built from scratch using the very procedures described here, which means
you will now be reading this BUILD.txt file for ever.
19) Reward anyone around you with good beer.
Miscellany
-----------
It was a little tricky to get all the interdependencies correct because ANTLR builds
itself using itself and the maven plugin references the ANTLR Tool as well. Hence
the maven tool is not a child project of the master pom.xml file, even though it is
built by it.
An observant person will not that when the assembly:assembly phase is run, that
it invokes the build of the ANTLR tool using the version of the Maven plugin that
it has just built, and this results in the plugin using the version of ANTLR tool that
it has just built. This is safe because everything will already be up to date and so
we package up the version of the tool that we expect, but the Maven plugin we
deploy will use the correct version of ANTLR, even though there is technically
a circular dependency.
The master pom.xml does give us a way to cause the build of the ANTLR tool
to use itself to build itself. This is because in dependencyManagement in the
master pom.xml, we can reference the current version of the Tool and the
Maven plugin, even though in the pom.xml for the tool itself refers to the previous
version of the plugin.
What happens is that if we first cd into the tool and maven directories and build ANTLR, it will
build itself with the prior version and this will deploy locally (.m2). We can then
clean build from the master pom and when ANTLR asks for the prior version of the tool,
the master pom.xml will override it and build with the interim versions we just built manually.
However, strictly speaking, we need a third build where we rebuild the tool again with
the version of the tool that was built with itself and not deploy the version that was
built by the version of itself that was built by a prior version of itself. I decided that this
was not particularly useful and complicates things too much. Building with a prior
version of the tool is fine and if there was ever a need to, we could release twice
in quick succession.
I have occasionally seen the MAven reactor screw up (or perhaps it is the ANTLR tool) when
building. If this happens you will see an ANTLR Panic - cannot find en.stg message. If this
happens to you, then just rerun the build and it will eventually work.
Jim Idle - March 2009
antlr-3.2/antlrsources.xml 0000644 0001750 0001750 00000030065 11256465172 015717 0 ustar twerner twerner
src
jar
true
org.antlr:gunit
src
src
pom.xml
CHANGES.txt
LICENSE.txt
README.txt
antlr.config
org.antlr:antlr-runtime
runtime/Java
src
src
pom.xml
doxyfile
antlr.config
org.antlr:antlr
tool
src
src
pom.xml
CHANGES.txt
LICENSE.txt
README.txt
antlr.config
org.antlr:antlr3-maven-plugin
src
src
pom.xml
org.antlr:maven-gunit-plugin
gunit-maven-plugin
src
src
pom.xml
pom.xml
antlrjar.xml
antlrsources.xml
BUILD.txt
antlr-3.2/runtime/ 0000755 0001750 0001750 00000000000 11410174107 014113 5 ustar twerner twerner antlr-3.2/runtime/Java/ 0000755 0001750 0001750 00000000000 11410174107 014774 5 ustar twerner twerner antlr-3.2/runtime/Java/antlr.config 0000644 0001750 0001750 00000000000 11256465214 017303 0 ustar twerner twerner antlr-3.2/runtime/Java/src/ 0000755 0001750 0001750 00000000000 11410174107 015563 5 ustar twerner twerner antlr-3.2/runtime/Java/src/main/ 0000755 0001750 0001750 00000000000 11410174107 016507 5 ustar twerner twerner antlr-3.2/runtime/Java/src/main/java/ 0000755 0001750 0001750 00000000000 11410174107 017430 5 ustar twerner twerner antlr-3.2/runtime/Java/src/main/java/org/ 0000755 0001750 0001750 00000000000 11410174107 020217 5 ustar twerner twerner antlr-3.2/runtime/Java/src/main/java/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 021337 5 ustar twerner twerner antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/ 0000755 0001750 0001750 00000000000 11410174107 023022 5 ustar twerner twerner antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/ClassicToken.java 0000644 0001750 0001750 00000007470 11256465214 026271 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A Token object like we'd use in ANTLR 2.x; has an actual string created
* and associated with this object. These objects are needed for imaginary
* tree nodes that have payload objects. We need to create a Token object
* that has a string; the tree node will point at this token. CommonToken
* has indexes into a char stream and hence cannot be used to introduce
* new strings.
*/
public class ClassicToken implements Token {
protected String text;
protected int type;
protected int line;
protected int charPositionInLine;
protected int channel=DEFAULT_CHANNEL;
/** What token number is this from 0..n-1 tokens */
protected int index;
public ClassicToken(int type) {
this.type = type;
}
public ClassicToken(Token oldToken) {
text = oldToken.getText();
type = oldToken.getType();
line = oldToken.getLine();
charPositionInLine = oldToken.getCharPositionInLine();
channel = oldToken.getChannel();
}
public ClassicToken(int type, String text) {
this.type = type;
this.text = text;
}
public ClassicToken(int type, String text, int channel) {
this.type = type;
this.text = text;
this.channel = channel;
}
public int getType() {
return type;
}
public void setLine(int line) {
this.line = line;
}
public String getText() {
return text;
}
public void setText(String text) {
this.text = text;
}
public int getLine() {
return line;
}
public int getCharPositionInLine() {
return charPositionInLine;
}
public void setCharPositionInLine(int charPositionInLine) {
this.charPositionInLine = charPositionInLine;
}
public int getChannel() {
return channel;
}
public void setChannel(int channel) {
this.channel = channel;
}
public void setType(int type) {
this.type = type;
}
public int getTokenIndex() {
return index;
}
public void setTokenIndex(int index) {
this.index = index;
}
public CharStream getInputStream() {
return null;
}
public void setInputStream(CharStream input) {
}
public String toString() {
String channelStr = "";
if ( channel>0 ) {
channelStr=",channel="+channel;
}
String txt = getText();
if ( txt!=null ) {
txt = txt.replaceAll("\n","\\\\n");
txt = txt.replaceAll("\r","\\\\r");
txt = txt.replaceAll("\t","\\\\t");
}
else {
txt = "";
}
return "[@"+getTokenIndex()+",'"+txt+"',<"+type+">"+channelStr+","+line+":"+getCharPositionInLine()+"]";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/DFA.java 0000644 0001750 0001750 00000016605 11256465214 024301 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A DFA implemented as a set of transition tables.
*
* Any state that has a semantic predicate edge is special; those states
* are generated with if-then-else structures in a specialStateTransition()
* which is generated by cyclicDFA template.
*
* There are at most 32767 states (16-bit signed short).
* Could get away with byte sometimes but would have to generate different
* types and the simulation code too. For a point of reference, the Java
* lexer's Tokens rule DFA has 326 states roughly.
*/
public class DFA {
protected short[] eot;
protected short[] eof;
protected char[] min;
protected char[] max;
protected short[] accept;
protected short[] special;
protected short[][] transition;
protected int decisionNumber;
/** Which recognizer encloses this DFA? Needed to check backtracking */
protected BaseRecognizer recognizer;
public static final boolean debug = false;
/** From the input stream, predict what alternative will succeed
* using this DFA (representing the covering regular approximation
* to the underlying CFL). Return an alternative number 1..n. Throw
* an exception upon error.
*/
public int predict(IntStream input)
throws RecognitionException
{
if ( debug ) {
System.err.println("Enter DFA.predict for decision "+decisionNumber);
}
int mark = input.mark(); // remember where decision started in input
int s = 0; // we always start at s0
try {
while ( true ) {
if ( debug ) System.err.println("DFA "+decisionNumber+" state "+s+" LA(1)="+(char)input.LA(1)+"("+input.LA(1)+
"), index="+input.index());
int specialState = special[s];
if ( specialState>=0 ) {
if ( debug ) {
System.err.println("DFA "+decisionNumber+
" state "+s+" is special state "+specialState);
}
s = specialStateTransition(specialState,input);
if ( debug ) {
System.err.println("DFA "+decisionNumber+
" returns from special state "+specialState+" to "+s);
}
if ( s==-1 ) {
noViableAlt(s,input);
return 0;
}
input.consume();
continue;
}
if ( accept[s] >= 1 ) {
if ( debug ) System.err.println("accept; predict "+accept[s]+" from state "+s);
return accept[s];
}
// look for a normal char transition
char c = (char)input.LA(1); // -1 == \uFFFF, all tokens fit in 65000 space
if (c>=min[s] && c<=max[s]) {
int snext = transition[s][c-min[s]]; // move to next state
if ( snext < 0 ) {
// was in range but not a normal transition
// must check EOT, which is like the else clause.
// eot[s]>=0 indicates that an EOT edge goes to another
// state.
if ( eot[s]>=0 ) { // EOT Transition to accept state?
if ( debug ) System.err.println("EOT transition");
s = eot[s];
input.consume();
// TODO: I had this as return accept[eot[s]]
// which assumed here that the EOT edge always
// went to an accept...faster to do this, but
// what about predicated edges coming from EOT
// target?
continue;
}
noViableAlt(s,input);
return 0;
}
s = snext;
input.consume();
continue;
}
if ( eot[s]>=0 ) { // EOT Transition?
if ( debug ) System.err.println("EOT transition");
s = eot[s];
input.consume();
continue;
}
if ( c==(char)Token.EOF && eof[s]>=0 ) { // EOF Transition to accept state?
if ( debug ) System.err.println("accept via EOF; predict "+accept[eof[s]]+" from "+eof[s]);
return accept[eof[s]];
}
// not in range and not EOF/EOT, must be invalid symbol
if ( debug ) {
System.err.println("min["+s+"]="+min[s]);
System.err.println("max["+s+"]="+max[s]);
System.err.println("eot["+s+"]="+eot[s]);
System.err.println("eof["+s+"]="+eof[s]);
for (int p=0; p0) {
recognizer.state.failed=true;
return;
}
NoViableAltException nvae =
new NoViableAltException(getDescription(),
decisionNumber,
s,
input);
error(nvae);
throw nvae;
}
/** A hook for debugging interface */
protected void error(NoViableAltException nvae) { ; }
public int specialStateTransition(int s, IntStream input)
throws NoViableAltException
{
return -1;
}
public String getDescription() {
return "n/a";
}
/** Given a String that has a run-length-encoding of some unsigned shorts
* like "\1\2\3\9", convert to short[] {2,9,9,9}. We do this to avoid
* static short[] which generates so much init code that the class won't
* compile. :(
*/
public static short[] unpackEncodedString(String encodedString) {
// walk first to find how big it is.
int size = 0;
for (int i=0; i= n ) {
//System.out.println("char LA("+i+")=EOF; p="+p);
return CharStream.EOF;
}
//System.out.println("char LA("+i+")="+(char)data[p+i-1]+"; p="+p);
//System.out.println("LA("+i+"); p="+p+" n="+n+" data.length="+data.length);
return data[p+i-1];
}
public int LT(int i) {
return LA(i);
}
/** Return the current input symbol index 0..n where n indicates the
* last symbol has been read. The index is the index of char to
* be returned from LA(1).
*/
public int index() {
return p;
}
public int size() {
return n;
}
public int mark() {
if ( markers==null ) {
markers = new ArrayList();
markers.add(null); // depth 0 means no backtracking, leave blank
}
markDepth++;
CharStreamState state = null;
if ( markDepth>=markers.size() ) {
state = new CharStreamState();
markers.add(state);
}
else {
state = (CharStreamState)markers.get(markDepth);
}
state.p = p;
state.line = line;
state.charPositionInLine = charPositionInLine;
lastMarker = markDepth;
return markDepth;
}
public void rewind(int m) {
CharStreamState state = (CharStreamState)markers.get(m);
// restore stream state
seek(state.p);
line = state.line;
charPositionInLine = state.charPositionInLine;
release(m);
}
public void rewind() {
rewind(lastMarker);
}
public void release(int marker) {
// unwind any other markers made after m and release m
markDepth = marker;
// release this marker
markDepth--;
}
/** consume() ahead until p==index; can't just set p=index as we must
* update line and charPositionInLine.
*/
public void seek(int index) {
if ( index<=p ) {
p = index; // just jump; don't update stream state (line, ...)
return;
}
// seek forward, consume until p hits index
while ( p {
/** dynamically-sized buffer of elements */
protected List data = new ArrayList();
/** index of next element to fill */
protected int p = 0;
public void reset() { p = 0; data.clear(); }
/** Get and remove first element in queue */
public T remove() {
T o = get(0);
p++;
// have we hit end of buffer?
if ( p == data.size() ) {
// if so, it's an opportunity to start filling at index 0 again
clear(); // size goes to 0, but retains memory
}
return o;
}
public void add(T o) { data.add(o); }
public int size() { return data.size() - p; }
public T head() { return get(0); }
/** Return element i elements ahead of current element. i==0 gets
* current element. This is not an absolute index into the data list
* since p defines the start of the real list.
*/
public T get(int i) {
if ( p+i >= data.size() ) {
throw new NoSuchElementException("queue index "+(p+i)+" > size "+data.size());
}
return data.get(p+i);
}
public void clear() { p = 0; data.clear(); }
/** Return string of current buffer contents; non-destructive */
public String toString() {
StringBuffer buf = new StringBuffer();
int n = size();
for (int i=0; i extends FastQueue {
public static final int UNINITIALIZED_EOF_ELEMENT_INDEX = Integer.MAX_VALUE;
/** Set to buffer index of eof when nextElement returns eof */
protected int eofElementIndex = UNINITIALIZED_EOF_ELEMENT_INDEX;
/** Returned by nextElement upon end of stream; we add to buffer also */
public T eof = null;
/** Track the last mark() call result value for use in rewind(). */
protected int lastMarker;
/** tracks how deep mark() calls are nested */
protected int markDepth = 0;
public LookaheadStream(T eof) {
this.eof = eof;
}
public void reset() {
eofElementIndex = UNINITIALIZED_EOF_ELEMENT_INDEX;
super.reset();
}
/** Implement nextElement to supply a stream of elements to this
* lookahead buffer. Return eof upon end of the stream we're pulling from.
*/
public abstract T nextElement();
/** Get and remove first element in queue; override FastQueue.remove() */
public T remove() {
T o = get(0);
p++;
// have we hit end of buffer and not backtracking?
if ( p == data.size() && markDepth==0 ) {
// if so, it's an opportunity to start filling at index 0 again
clear(); // size goes to 0, but retains memory
}
return o;
}
/** Make sure we have at least one element to remove, even if EOF */
public void consume() { sync(1); remove(); }
/** Make sure we have 'need' elements from current position p. Last valid
* p index is data.size()-1. p+need-1 is the data index 'need' elements
* ahead. If we need 1 element, (p+1-1)==p must be < data.size().
*/
public void sync(int need) {
int n = (p+need-1) - data.size() + 1; // how many more elements we need?
if ( n > 0 ) fill(n); // out of elements?
}
/** add n elements to buffer */
public void fill(int n) {
for (int i=1; i<=n; i++) {
T o = nextElement();
if ( o==eof ) {
data.add(eof);
eofElementIndex = data.size()-1;
}
else data.add(o);
}
}
//public boolean hasNext() { return eofElementIndex!=UNINITIALIZED_EOF_ELEMENT_INDEX; }
/** Size of entire stream is unknown; we only know buffer size from FastQueue */
public int size() { throw new UnsupportedOperationException("streams are of unknown size"); }
public Object LT(int k) {
if ( k==0 ) {
return null;
}
if ( k<0 ) {
return LB(-k);
}
//System.out.print("LT(p="+p+","+k+")=");
if ( (p+k-1) >= eofElementIndex ) { // move to super.LT
return eof;
}
sync(k);
return get(k-1);
}
/** Look backwards k nodes */
protected Object LB(int k) {
if ( k==0 ) {
return null;
}
if ( (p-k)<0 ) {
return null;
}
return get(-k);
}
public Object getCurrentSymbol() { return LT(1); }
public int index() { return p; }
public int mark() {
markDepth++;
lastMarker = index();
return lastMarker;
}
public void release(int marker) {
// no resources to release
}
public void rewind(int marker) {
markDepth--;
seek(marker); // assume marker is top
// release(marker); // waste of call; it does nothing in this class
}
public void rewind() {
seek(lastMarker); // rewind but do not release marker
}
/** Seek to a 0-indexed position within data buffer. Can't handle
* case where you seek beyond end of existing buffer. Normally used
* to seek backwards in the buffer. Does not force loading of nodes.
*/
public void seek(int index) { p = index; }
} antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/misc/IntArray.java 0000644 0001750 0001750 00000005500 11256465214 026363 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.misc;
/** A dynamic array that uses int not Integer objects. In principle this
* is more efficient in time, but certainly in space.
*
* This is simple enough that you can access the data array directly,
* but make sure that you append elements only with add() so that you
* get dynamic sizing. Make sure to call ensureCapacity() when you are
* manually adding new elements.
*
* Doesn't impl List because it doesn't return objects and I mean this
* really as just an array not a List per se. Manipulate the elements
* at will. This has stack methods too.
*
* When runtime can be 1.5, I'll make this generic.
*/
public class IntArray {
public static final int INITIAL_SIZE = 10;
public int[] data;
protected int p = -1;
public void add(int v) {
ensureCapacity(p+1);
data[++p] = v;
}
public void push(int v) {
add(v);
}
public int pop() {
int v = data[p];
p--;
return v;
}
/** This only tracks elements added via push/add. */
public int size() {
return p;
}
public void clear() {
p = -1;
}
public void ensureCapacity(int index) {
if ( data==null ) {
data = new int[INITIAL_SIZE];
}
else if ( (index+1)>=data.length ) {
int newSize = data.length*2;
if ( index>newSize ) {
newSize = index+1;
}
int[] newData = new int[newSize];
System.arraycopy(data, 0, newData, 0, data.length);
data = newData;
}
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/misc/Stats.java 0000644 0001750 0001750 00000007721 11256465214 025737 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.misc;
import java.io.*;
/** Stats routines needed by profiler etc...
// note that these routines return 0.0 if no values exist in the X[]
// which is not "correct", but it is useful so I don't generate NaN
// in my output
*/
public class Stats {
public static final String ANTLRWORKS_DIR = "antlrworks";
/** Compute the sample (unbiased estimator) standard deviation following:
*
* Computing Deviations: Standard Accuracy
* Tony F. Chan and John Gregg Lewis
* Stanford University
* Communications of ACM September 1979 of Volume 22 the ACM Number 9
*
* The "two-pass" method from the paper; supposed to have better
* numerical properties than the textbook summation/sqrt. To me
* this looks like the textbook method, but I ain't no numerical
* methods guy.
*/
public static double stddev(int[] X) {
int m = X.length;
if ( m<=1 ) {
return 0;
}
double xbar = avg(X);
double s2 = 0.0;
for (int i=0; i=0.0 ) {
return xbar / m;
}
return 0.0;
}
public static int min(int[] X) {
int min = Integer.MAX_VALUE;
int m = X.length;
if ( m==0 ) {
return 0;
}
for (int i=0; i max ) {
max = X[i];
}
}
return max;
}
public static int sum(int[] X) {
int s = 0;
int m = X.length;
if ( m==0 ) {
return 0;
}
for (int i=0; i";
}
}
class InsertBeforeOp extends RewriteOperation {
public InsertBeforeOp(int index, Object text) {
super(index,text);
}
public int execute(StringBuffer buf) {
buf.append(text);
buf.append(((Token)tokens.get(index)).getText());
return index+1;
}
}
/** I'm going to try replacing range from x..y with (y-x)+1 ReplaceOp
* instructions.
*/
class ReplaceOp extends RewriteOperation {
protected int lastIndex;
public ReplaceOp(int from, int to, Object text) {
super(from,text);
lastIndex = to;
}
public int execute(StringBuffer buf) {
if ( text!=null ) {
buf.append(text);
}
return lastIndex+1;
}
public String toString() {
return "";
}
}
class DeleteOp extends ReplaceOp {
public DeleteOp(int from, int to) {
super(from, to, null);
}
public String toString() {
return "";
}
}
/** You may have multiple, named streams of rewrite operations.
* I'm calling these things "programs."
* Maps String (name) -> rewrite (List)
*/
protected Map programs = null;
/** Map String (program name) -> Integer index */
protected Map lastRewriteTokenIndexes = null;
public TokenRewriteStream() {
init();
}
protected void init() {
programs = new HashMap();
programs.put(DEFAULT_PROGRAM_NAME, new ArrayList(PROGRAM_INIT_SIZE));
lastRewriteTokenIndexes = new HashMap();
}
public TokenRewriteStream(TokenSource tokenSource) {
super(tokenSource);
init();
}
public TokenRewriteStream(TokenSource tokenSource, int channel) {
super(tokenSource, channel);
init();
}
public void rollback(int instructionIndex) {
rollback(DEFAULT_PROGRAM_NAME, instructionIndex);
}
/** Rollback the instruction stream for a program so that
* the indicated instruction (via instructionIndex) is no
* longer in the stream. UNTESTED!
*/
public void rollback(String programName, int instructionIndex) {
List is = (List)programs.get(programName);
if ( is!=null ) {
programs.put(programName, is.subList(MIN_TOKEN_INDEX,instructionIndex));
}
}
public void deleteProgram() {
deleteProgram(DEFAULT_PROGRAM_NAME);
}
/** Reset the program so that no instructions exist */
public void deleteProgram(String programName) {
rollback(programName, MIN_TOKEN_INDEX);
}
public void insertAfter(Token t, Object text) {
insertAfter(DEFAULT_PROGRAM_NAME, t, text);
}
public void insertAfter(int index, Object text) {
insertAfter(DEFAULT_PROGRAM_NAME, index, text);
}
public void insertAfter(String programName, Token t, Object text) {
insertAfter(programName,t.getTokenIndex(), text);
}
public void insertAfter(String programName, int index, Object text) {
// to insert after, just insert before next index (even if past end)
insertBefore(programName,index+1, text);
//addToSortedRewriteList(programName, new InsertAfterOp(index,text));
}
public void insertBefore(Token t, Object text) {
insertBefore(DEFAULT_PROGRAM_NAME, t, text);
}
public void insertBefore(int index, Object text) {
insertBefore(DEFAULT_PROGRAM_NAME, index, text);
}
public void insertBefore(String programName, Token t, Object text) {
insertBefore(programName, t.getTokenIndex(), text);
}
public void insertBefore(String programName, int index, Object text) {
//addToSortedRewriteList(programName, new InsertBeforeOp(index,text));
RewriteOperation op = new InsertBeforeOp(index,text);
List rewrites = getProgram(programName);
op.instructionIndex = rewrites.size();
rewrites.add(op);
}
public void replace(int index, Object text) {
replace(DEFAULT_PROGRAM_NAME, index, index, text);
}
public void replace(int from, int to, Object text) {
replace(DEFAULT_PROGRAM_NAME, from, to, text);
}
public void replace(Token indexT, Object text) {
replace(DEFAULT_PROGRAM_NAME, indexT, indexT, text);
}
public void replace(Token from, Token to, Object text) {
replace(DEFAULT_PROGRAM_NAME, from, to, text);
}
public void replace(String programName, int from, int to, Object text) {
if ( from > to || from<0 || to<0 || to >= tokens.size() ) {
throw new IllegalArgumentException("replace: range invalid: "+from+".."+to+"(size="+tokens.size()+")");
}
RewriteOperation op = new ReplaceOp(from, to, text);
List rewrites = getProgram(programName);
op.instructionIndex = rewrites.size();
rewrites.add(op);
}
public void replace(String programName, Token from, Token to, Object text) {
replace(programName,
from.getTokenIndex(),
to.getTokenIndex(),
text);
}
public void delete(int index) {
delete(DEFAULT_PROGRAM_NAME, index, index);
}
public void delete(int from, int to) {
delete(DEFAULT_PROGRAM_NAME, from, to);
}
public void delete(Token indexT) {
delete(DEFAULT_PROGRAM_NAME, indexT, indexT);
}
public void delete(Token from, Token to) {
delete(DEFAULT_PROGRAM_NAME, from, to);
}
public void delete(String programName, int from, int to) {
replace(programName,from,to,null);
}
public void delete(String programName, Token from, Token to) {
replace(programName,from,to,null);
}
public int getLastRewriteTokenIndex() {
return getLastRewriteTokenIndex(DEFAULT_PROGRAM_NAME);
}
protected int getLastRewriteTokenIndex(String programName) {
Integer I = (Integer)lastRewriteTokenIndexes.get(programName);
if ( I==null ) {
return -1;
}
return I.intValue();
}
protected void setLastRewriteTokenIndex(String programName, int i) {
lastRewriteTokenIndexes.put(programName, new Integer(i));
}
protected List getProgram(String name) {
List is = (List)programs.get(name);
if ( is==null ) {
is = initializeProgram(name);
}
return is;
}
private List initializeProgram(String name) {
List is = new ArrayList(PROGRAM_INIT_SIZE);
programs.put(name, is);
return is;
}
public String toOriginalString() {
return toOriginalString(MIN_TOKEN_INDEX, size()-1);
}
public String toOriginalString(int start, int end) {
StringBuffer buf = new StringBuffer();
for (int i=start; i>=MIN_TOKEN_INDEX && i<=end && itokens.size()-1 ) end = tokens.size()-1;
if ( start<0 ) start = 0;
if ( rewrites==null || rewrites.size()==0 ) {
return toOriginalString(start,end); // no instructions to execute
}
StringBuffer buf = new StringBuffer();
// First, optimize instruction stream
Map indexToOp = reduceToSingleOperationPerIndex(rewrites);
// Walk buffer, executing instructions and emitting tokens
int i = start;
while ( i <= end && i < tokens.size() ) {
RewriteOperation op = (RewriteOperation)indexToOp.get(new Integer(i));
indexToOp.remove(new Integer(i)); // remove so any left have index size-1
Token t = (Token) tokens.get(i);
if ( op==null ) {
// no operation at that index, just dump token
buf.append(t.getText());
i++; // move to next token
}
else {
i = op.execute(buf); // execute operation and skip
}
}
// include stuff after end if it's last index in buffer
// So, if they did an insertAfter(lastValidIndex, "foo"), include
// foo if end==lastValidIndex.
if ( end==tokens.size()-1 ) {
// Scan any remaining operations after last token
// should be included (they will be inserts).
Iterator it = indexToOp.values().iterator();
while (it.hasNext()) {
RewriteOperation op = (RewriteOperation)it.next();
if ( op.index >= tokens.size()-1 ) buf.append(op.text);
}
}
return buf.toString();
}
/** We need to combine operations and report invalid operations (like
* overlapping replaces that are not completed nested). Inserts to
* same index need to be combined etc... Here are the cases:
*
* I.i.u I.j.v leave alone, nonoverlapping
* I.i.u I.i.v combine: Iivu
*
* R.i-j.u R.x-y.v | i-j in x-y delete first R
* R.i-j.u R.i-j.v delete first R
* R.i-j.u R.x-y.v | x-y in i-j ERROR
* R.i-j.u R.x-y.v | boundaries overlap ERROR
*
* I.i.u R.x-y.v | i in x-y delete I
* I.i.u R.x-y.v | i not in x-y leave alone, nonoverlapping
* R.x-y.v I.i.u | i in x-y ERROR
* R.x-y.v I.x.u R.x-y.uv (combine, delete I)
* R.x-y.v I.i.u | i not in x-y leave alone, nonoverlapping
*
* I.i.u = insert u before op @ index i
* R.x-y.u = replace x-y indexed tokens with u
*
* First we need to examine replaces. For any replace op:
*
* 1. wipe out any insertions before op within that range.
* 2. Drop any replace op before that is contained completely within
* that range.
* 3. Throw exception upon boundary overlap with any previous replace.
*
* Then we can deal with inserts:
*
* 1. for any inserts to same index, combine even if not adjacent.
* 2. for any prior replace with same left boundary, combine this
* insert with replace and delete this replace.
* 3. throw exception if index in same range as previous replace
*
* Don't actually delete; make op null in list. Easier to walk list.
* Later we can throw as we add to index -> op map.
*
* Note that I.2 R.2-2 will wipe out I.2 even though, technically, the
* inserted stuff would be before the replace range. But, if you
* add tokens in front of a method body '{' and then delete the method
* body, I think the stuff before the '{' you added should disappear too.
*
* Return a map from token index to operation.
*/
protected Map reduceToSingleOperationPerIndex(List rewrites) {
//System.out.println("rewrites="+rewrites);
// WALK REPLACES
for (int i = 0; i < rewrites.size(); i++) {
RewriteOperation op = (RewriteOperation)rewrites.get(i);
if ( op==null ) continue;
if ( !(op instanceof ReplaceOp) ) continue;
ReplaceOp rop = (ReplaceOp)rewrites.get(i);
// Wipe prior inserts within range
List inserts = getKindOfOps(rewrites, InsertBeforeOp.class, i);
for (int j = 0; j < inserts.size(); j++) {
InsertBeforeOp iop = (InsertBeforeOp) inserts.get(j);
if ( iop.index >= rop.index && iop.index <= rop.lastIndex ) {
// delete insert as it's a no-op.
rewrites.set(iop.instructionIndex, null);
}
}
// Drop any prior replaces contained within
List prevReplaces = getKindOfOps(rewrites, ReplaceOp.class, i);
for (int j = 0; j < prevReplaces.size(); j++) {
ReplaceOp prevRop = (ReplaceOp) prevReplaces.get(j);
if ( prevRop.index>=rop.index && prevRop.lastIndex <= rop.lastIndex ) {
// delete replace as it's a no-op.
rewrites.set(prevRop.instructionIndex, null);
continue;
}
// throw exception unless disjoint or identical
boolean disjoint =
prevRop.lastIndex rop.lastIndex;
boolean same =
prevRop.index==rop.index && prevRop.lastIndex==rop.lastIndex;
if ( !disjoint && !same ) {
throw new IllegalArgumentException("replace op boundaries of "+rop+
" overlap with previous "+prevRop);
}
}
}
// WALK INSERTS
for (int i = 0; i < rewrites.size(); i++) {
RewriteOperation op = (RewriteOperation)rewrites.get(i);
if ( op==null ) continue;
if ( !(op instanceof InsertBeforeOp) ) continue;
InsertBeforeOp iop = (InsertBeforeOp)rewrites.get(i);
// combine current insert with prior if any at same index
List prevInserts = getKindOfOps(rewrites, InsertBeforeOp.class, i);
for (int j = 0; j < prevInserts.size(); j++) {
InsertBeforeOp prevIop = (InsertBeforeOp) prevInserts.get(j);
if ( prevIop.index == iop.index ) { // combine objects
// convert to strings...we're in process of toString'ing
// whole token buffer so no lazy eval issue with any templates
iop.text = catOpText(iop.text,prevIop.text);
// delete redundant prior insert
rewrites.set(prevIop.instructionIndex, null);
}
}
// look for replaces where iop.index is in range; error
List prevReplaces = getKindOfOps(rewrites, ReplaceOp.class, i);
for (int j = 0; j < prevReplaces.size(); j++) {
ReplaceOp rop = (ReplaceOp) prevReplaces.get(j);
if ( iop.index == rop.index ) {
rop.text = catOpText(iop.text,rop.text);
rewrites.set(i, null); // delete current insert
continue;
}
if ( iop.index >= rop.index && iop.index <= rop.lastIndex ) {
throw new IllegalArgumentException("insert op "+iop+
" within boundaries of previous "+rop);
}
}
}
// System.out.println("rewrites after="+rewrites);
Map m = new HashMap();
for (int i = 0; i < rewrites.size(); i++) {
RewriteOperation op = (RewriteOperation)rewrites.get(i);
if ( op==null ) continue; // ignore deleted ops
if ( m.get(new Integer(op.index))!=null ) {
throw new Error("should only be one op per index");
}
m.put(new Integer(op.index), op);
}
//System.out.println("index to op: "+m);
return m;
}
protected String catOpText(Object a, Object b) {
String x = "";
String y = "";
if ( a!=null ) x = a.toString();
if ( b!=null ) y = b.toString();
return x+y;
}
protected List getKindOfOps(List rewrites, Class kind) {
return getKindOfOps(rewrites, kind, rewrites.size());
}
/** Get all operations before an index of a particular kind */
protected List getKindOfOps(List rewrites, Class kind, int before) {
List ops = new ArrayList();
for (int i=0; i=MIN_TOKEN_INDEX && i<=end && imaxRuleInvocationDepth ) {
maxRuleInvocationDepth = ruleLevel;
}
}
/** Track memoization; this is not part of standard debug interface
* but is triggered by profiling. Code gen inserts an override
* for this method in the recognizer, which triggers this method.
*/
public void examineRuleMemoization(IntStream input,
int ruleIndex,
String ruleName)
{
//System.out.println("examine memo "+ruleName);
int stopIndex = parser.getRuleMemoization(ruleIndex, input.index());
if ( stopIndex==BaseRecognizer.MEMO_RULE_UNKNOWN ) {
//System.out.println("rule "+ruleIndex+" missed @ "+input.index());
numMemoizationCacheMisses++;
numGuessingRuleInvocations++; // we'll have to enter
}
else {
// regardless of rule success/failure, if in cache, we have a cache hit
//System.out.println("rule "+ruleIndex+" hit @ "+input.index());
numMemoizationCacheHits++;
}
}
public void memoize(IntStream input,
int ruleIndex,
int ruleStartIndex,
String ruleName)
{
// count how many entries go into table
//System.out.println("memoize "+ruleName);
numMemoizationCacheEntries++;
}
public void exitRule(String grammarFileName, String ruleName) {
ruleLevel--;
}
public void enterDecision(int decisionNumber) {
decisionLevel++;
int startingLookaheadIndex = parser.getTokenStream().index();
//System.out.println("enterDecision "+decisionNumber+" @ index "+startingLookaheadIndex);
lookaheadStack.add(new Integer(startingLookaheadIndex));
}
public void exitDecision(int decisionNumber) {
//System.out.println("exitDecision "+decisionNumber);
// track how many of acyclic, cyclic here as we don't know what kind
// yet in enterDecision event.
if ( parser.isCyclicDecision ) {
numCyclicDecisions++;
}
else {
numFixedDecisions++;
}
lookaheadStack.remove(lookaheadStack.size()-1); // pop lookahead depth counter
decisionLevel--;
if ( parser.isCyclicDecision ) {
if ( numCyclicDecisions>=decisionMaxCyclicLookaheads.length ) {
int[] bigger = new int[decisionMaxCyclicLookaheads.length*2];
System.arraycopy(decisionMaxCyclicLookaheads,0,bigger,0,decisionMaxCyclicLookaheads.length);
decisionMaxCyclicLookaheads = bigger;
}
decisionMaxCyclicLookaheads[numCyclicDecisions-1] = maxLookaheadInCurrentDecision;
}
else {
if ( numFixedDecisions>=decisionMaxFixedLookaheads.length ) {
int[] bigger = new int[decisionMaxFixedLookaheads.length*2];
System.arraycopy(decisionMaxFixedLookaheads,0,bigger,0,decisionMaxFixedLookaheads.length);
decisionMaxFixedLookaheads = bigger;
}
decisionMaxFixedLookaheads[numFixedDecisions-1] = maxLookaheadInCurrentDecision;
}
parser.isCyclicDecision = false; // can't nest so just reset to false
maxLookaheadInCurrentDecision = 0;
}
public void consumeToken(Token token) {
//System.out.println("consume token "+token);
lastTokenConsumed = (CommonToken)token;
}
/** The parser is in a decision if the decision depth > 0. This
* works for backtracking also, which can have nested decisions.
*/
public boolean inDecision() {
return decisionLevel>0;
}
public void consumeHiddenToken(Token token) {
//System.out.println("consume hidden token "+token);
lastTokenConsumed = (CommonToken)token;
}
/** Track refs to lookahead if in a fixed/nonfixed decision.
*/
public void LT(int i, Token t) {
if ( inDecision() ) {
// get starting index off stack
int stackTop = lookaheadStack.size()-1;
Integer startingIndex = (Integer)lookaheadStack.get(stackTop);
// compute lookahead depth
int thisRefIndex = parser.getTokenStream().index();
int numHidden =
getNumberOfHiddenTokens(startingIndex.intValue(), thisRefIndex);
int depth = i + thisRefIndex - startingIndex.intValue() - numHidden;
/*
System.out.println("LT("+i+") @ index "+thisRefIndex+" is depth "+depth+
" max is "+maxLookaheadInCurrentDecision);
*/
if ( depth>maxLookaheadInCurrentDecision ) {
maxLookaheadInCurrentDecision = depth;
}
}
}
/** Track backtracking decisions. You'll see a fixed or cyclic decision
* and then a backtrack.
*
* enter rule
* ...
* enter decision
* LA and possibly consumes (for cyclic DFAs)
* begin backtrack level
* mark m
* rewind m
* end backtrack level, success
* exit decision
* ...
* exit rule
*/
public void beginBacktrack(int level) {
//System.out.println("enter backtrack "+level);
numBacktrackDecisions++;
}
/** Successful or not, track how much lookahead synpreds use */
public void endBacktrack(int level, boolean successful) {
//System.out.println("exit backtrack "+level+": "+successful);
decisionMaxSynPredLookaheads.add(
new Integer(maxLookaheadInCurrentDecision)
);
}
/*
public void mark(int marker) {
int i = parser.getTokenStream().index();
System.out.println("mark @ index "+i);
synPredLookaheadStack.add(new Integer(i));
}
public void rewind(int marker) {
// pop starting index off stack
int stackTop = synPredLookaheadStack.size()-1;
Integer startingIndex = (Integer)synPredLookaheadStack.get(stackTop);
synPredLookaheadStack.remove(synPredLookaheadStack.size()-1);
// compute lookahead depth
int stopIndex = parser.getTokenStream().index();
System.out.println("rewind @ index "+stopIndex);
int depth = stopIndex - startingIndex.intValue();
System.out.println("depth of lookahead for synpred: "+depth);
decisionMaxSynPredLookaheads.add(
new Integer(depth)
);
}
*/
public void recognitionException(RecognitionException e) {
numberReportedErrors++;
}
public void semanticPredicate(boolean result, String predicate) {
if ( inDecision() ) {
numSemanticPredicates++;
}
}
public void terminate() {
String stats = toNotifyString();
try {
Stats.writeReport(RUNTIME_STATS_FILENAME,stats);
}
catch (IOException ioe) {
System.err.println(ioe);
ioe.printStackTrace(System.err);
}
System.out.println(toString(stats));
}
public void setParser(DebugParser parser) {
this.parser = parser;
}
// R E P O R T I N G
public String toNotifyString() {
TokenStream input = parser.getTokenStream();
for (int i=0; i"+channelStr+","+line+":"+getCharPositionInLine()+",@"+index+"]";
}
}
public static class ProxyTree extends BaseTree {
public int ID;
public int type;
public int line = 0;
public int charPos = -1;
public int tokenIndex = -1;
public String text;
public ProxyTree(int ID, int type, int line, int charPos, int tokenIndex, String text) {
this.ID = ID;
this.type = type;
this.line = line;
this.charPos = charPos;
this.tokenIndex = tokenIndex;
this.text = text;
}
public ProxyTree(int ID) { this.ID = ID; }
public int getTokenStartIndex() { return tokenIndex; }
public void setTokenStartIndex(int index) { }
public int getTokenStopIndex() { return 0; }
public void setTokenStopIndex(int index) { }
public Tree dupNode() { return null; }
public int getType() { return type; }
public String getText() { return text; }
public String toString() {
return "fix this";
}
}
public RemoteDebugEventSocketListener(DebugEventListener listener,
String machine,
int port) throws IOException
{
this.listener = listener;
this.machine = machine;
this.port = port;
if( !openConnection() ) {
throw new ConnectException();
}
}
protected void eventHandler() {
try {
handshake();
event = in.readLine();
while ( event!=null ) {
dispatch(event);
ack();
event = in.readLine();
}
}
catch (Exception e) {
System.err.println(e);
e.printStackTrace(System.err);
}
finally {
closeConnection();
}
}
protected boolean openConnection() {
boolean success = false;
try {
channel = new Socket(machine, port);
channel.setTcpNoDelay(true);
OutputStream os = channel.getOutputStream();
OutputStreamWriter osw = new OutputStreamWriter(os, "UTF8");
out = new PrintWriter(new BufferedWriter(osw));
InputStream is = channel.getInputStream();
InputStreamReader isr = new InputStreamReader(is, "UTF8");
in = new BufferedReader(isr);
success = true;
} catch(Exception e) {
System.err.println(e);
}
return success;
}
protected void closeConnection() {
try {
in.close(); in = null;
out.close(); out = null;
channel.close(); channel=null;
}
catch (Exception e) {
System.err.println(e);
e.printStackTrace(System.err);
}
finally {
if ( in!=null ) {
try {in.close();} catch (IOException ioe) {
System.err.println(ioe);
}
}
if ( out!=null ) {
out.close();
}
if ( channel!=null ) {
try {channel.close();} catch (IOException ioe) {
System.err.println(ioe);
}
}
}
}
protected void handshake() throws IOException {
String antlrLine = in.readLine();
String[] antlrElements = getEventElements(antlrLine);
version = antlrElements[1];
String grammarLine = in.readLine();
String[] grammarElements = getEventElements(grammarLine);
grammarFileName = grammarElements[1];
ack();
listener.commence(); // inform listener after handshake
}
protected void ack() {
out.println("ack");
out.flush();
}
protected void dispatch(String line) {
//System.out.println("event: "+line);
String[] elements = getEventElements(line);
if ( elements==null || elements[0]==null ) {
System.err.println("unknown debug event: "+line);
return;
}
if ( elements[0].equals("enterRule") ) {
listener.enterRule(elements[1], elements[2]);
}
else if ( elements[0].equals("exitRule") ) {
listener.exitRule(elements[1], elements[2]);
}
else if ( elements[0].equals("enterAlt") ) {
listener.enterAlt(Integer.parseInt(elements[1]));
}
else if ( elements[0].equals("enterSubRule") ) {
listener.enterSubRule(Integer.parseInt(elements[1]));
}
else if ( elements[0].equals("exitSubRule") ) {
listener.exitSubRule(Integer.parseInt(elements[1]));
}
else if ( elements[0].equals("enterDecision") ) {
listener.enterDecision(Integer.parseInt(elements[1]));
}
else if ( elements[0].equals("exitDecision") ) {
listener.exitDecision(Integer.parseInt(elements[1]));
}
else if ( elements[0].equals("location") ) {
listener.location(Integer.parseInt(elements[1]),
Integer.parseInt(elements[2]));
}
else if ( elements[0].equals("consumeToken") ) {
ProxyToken t = deserializeToken(elements, 1);
if ( t.getTokenIndex() == previousTokenIndex ) {
tokenIndexesInvalid = true;
}
previousTokenIndex = t.getTokenIndex();
listener.consumeToken(t);
}
else if ( elements[0].equals("consumeHiddenToken") ) {
ProxyToken t = deserializeToken(elements, 1);
if ( t.getTokenIndex() == previousTokenIndex ) {
tokenIndexesInvalid = true;
}
previousTokenIndex = t.getTokenIndex();
listener.consumeHiddenToken(t);
}
else if ( elements[0].equals("LT") ) {
Token t = deserializeToken(elements, 2);
listener.LT(Integer.parseInt(elements[1]), t);
}
else if ( elements[0].equals("mark") ) {
listener.mark(Integer.parseInt(elements[1]));
}
else if ( elements[0].equals("rewind") ) {
if ( elements[1]!=null ) {
listener.rewind(Integer.parseInt(elements[1]));
}
else {
listener.rewind();
}
}
else if ( elements[0].equals("beginBacktrack") ) {
listener.beginBacktrack(Integer.parseInt(elements[1]));
}
else if ( elements[0].equals("endBacktrack") ) {
int level = Integer.parseInt(elements[1]);
int successI = Integer.parseInt(elements[2]);
listener.endBacktrack(level, successI==DebugEventListener.TRUE);
}
else if ( elements[0].equals("exception") ) {
String excName = elements[1];
String indexS = elements[2];
String lineS = elements[3];
String posS = elements[4];
Class excClass = null;
try {
excClass = Class.forName(excName);
RecognitionException e =
(RecognitionException)excClass.newInstance();
e.index = Integer.parseInt(indexS);
e.line = Integer.parseInt(lineS);
e.charPositionInLine = Integer.parseInt(posS);
listener.recognitionException(e);
}
catch (ClassNotFoundException cnfe) {
System.err.println("can't find class "+cnfe);
cnfe.printStackTrace(System.err);
}
catch (InstantiationException ie) {
System.err.println("can't instantiate class "+ie);
ie.printStackTrace(System.err);
}
catch (IllegalAccessException iae) {
System.err.println("can't access class "+iae);
iae.printStackTrace(System.err);
}
}
else if ( elements[0].equals("beginResync") ) {
listener.beginResync();
}
else if ( elements[0].equals("endResync") ) {
listener.endResync();
}
else if ( elements[0].equals("terminate") ) {
listener.terminate();
}
else if ( elements[0].equals("semanticPredicate") ) {
Boolean result = Boolean.valueOf(elements[1]);
String predicateText = elements[2];
predicateText = unEscapeNewlines(predicateText);
listener.semanticPredicate(result.booleanValue(),
predicateText);
}
else if ( elements[0].equals("consumeNode") ) {
ProxyTree node = deserializeNode(elements, 1);
listener.consumeNode(node);
}
else if ( elements[0].equals("LN") ) {
int i = Integer.parseInt(elements[1]);
ProxyTree node = deserializeNode(elements, 2);
listener.LT(i, node);
}
else if ( elements[0].equals("createNodeFromTokenElements") ) {
int ID = Integer.parseInt(elements[1]);
int type = Integer.parseInt(elements[2]);
String text = elements[3];
text = unEscapeNewlines(text);
ProxyTree node = new ProxyTree(ID, type, -1, -1, -1, text);
listener.createNode(node);
}
else if ( elements[0].equals("createNode") ) {
int ID = Integer.parseInt(elements[1]);
int tokenIndex = Integer.parseInt(elements[2]);
// create dummy node/token filled with ID, tokenIndex
ProxyTree node = new ProxyTree(ID);
ProxyToken token = new ProxyToken(tokenIndex);
listener.createNode(node, token);
}
else if ( elements[0].equals("nilNode") ) {
int ID = Integer.parseInt(elements[1]);
ProxyTree node = new ProxyTree(ID);
listener.nilNode(node);
}
else if ( elements[0].equals("errorNode") ) {
// TODO: do we need a special tree here?
int ID = Integer.parseInt(elements[1]);
int type = Integer.parseInt(elements[2]);
String text = elements[3];
text = unEscapeNewlines(text);
ProxyTree node = new ProxyTree(ID, type, -1, -1, -1, text);
listener.errorNode(node);
}
else if ( elements[0].equals("becomeRoot") ) {
int newRootID = Integer.parseInt(elements[1]);
int oldRootID = Integer.parseInt(elements[2]);
ProxyTree newRoot = new ProxyTree(newRootID);
ProxyTree oldRoot = new ProxyTree(oldRootID);
listener.becomeRoot(newRoot, oldRoot);
}
else if ( elements[0].equals("addChild") ) {
int rootID = Integer.parseInt(elements[1]);
int childID = Integer.parseInt(elements[2]);
ProxyTree root = new ProxyTree(rootID);
ProxyTree child = new ProxyTree(childID);
listener.addChild(root, child);
}
else if ( elements[0].equals("setTokenBoundaries") ) {
int ID = Integer.parseInt(elements[1]);
ProxyTree node = new ProxyTree(ID);
listener.setTokenBoundaries(
node,
Integer.parseInt(elements[2]),
Integer.parseInt(elements[3]));
}
else {
System.err.println("unknown debug event: "+line);
}
}
protected ProxyTree deserializeNode(String[] elements, int offset) {
int ID = Integer.parseInt(elements[offset+0]);
int type = Integer.parseInt(elements[offset+1]);
int tokenLine = Integer.parseInt(elements[offset+2]);
int charPositionInLine = Integer.parseInt(elements[offset+3]);
int tokenIndex = Integer.parseInt(elements[offset+4]);
String text = elements[offset+5];
text = unEscapeNewlines(text);
return new ProxyTree(ID, type, tokenLine, charPositionInLine, tokenIndex, text);
}
protected ProxyToken deserializeToken(String[] elements,
int offset)
{
String indexS = elements[offset+0];
String typeS = elements[offset+1];
String channelS = elements[offset+2];
String lineS = elements[offset+3];
String posS = elements[offset+4];
String text = elements[offset+5];
text = unEscapeNewlines(text);
int index = Integer.parseInt(indexS);
ProxyToken t =
new ProxyToken(index,
Integer.parseInt(typeS),
Integer.parseInt(channelS),
Integer.parseInt(lineS),
Integer.parseInt(posS),
text);
return t;
}
/** Create a thread to listen to the remote running recognizer */
public void start() {
Thread t = new Thread(this);
t.start();
}
public void run() {
eventHandler();
}
// M i s c
public String[] getEventElements(String event) {
if ( event==null ) {
return null;
}
String[] elements = new String[MAX_EVENT_ELEMENTS];
String str = null; // a string element if present (must be last)
try {
int firstQuoteIndex = event.indexOf('"');
if ( firstQuoteIndex>=0 ) {
// treat specially; has a string argument like "a comment\n
// Note that the string is terminated by \n not end quote.
// Easier to parse that way.
String eventWithoutString = event.substring(0,firstQuoteIndex);
str = event.substring(firstQuoteIndex+1,event.length());
event = eventWithoutString;
}
StringTokenizer st = new StringTokenizer(event, "\t", false);
int i = 0;
while ( st.hasMoreTokens() ) {
if ( i>=MAX_EVENT_ELEMENTS ) {
// ErrorManager.internalError("event has more than "+MAX_EVENT_ELEMENTS+" args: "+event);
return elements;
}
elements[i] = st.nextToken();
i++;
}
if ( str!=null ) {
elements[i] = str;
}
}
catch (Exception e) {
e.printStackTrace(System.err);
}
return elements;
}
protected String unEscapeNewlines(String txt) {
// this unescape is slow but easy to understand
txt = txt.replaceAll("%0A","\n"); // unescape \n
txt = txt.replaceAll("%0D","\r"); // unescape \r
txt = txt.replaceAll("%25","%"); // undo escaped escape chars
return txt;
}
public boolean tokenIndexesAreInvalid() {
return false;
//return tokenIndexesInvalid;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/DebugEventHub.java 0000644 0001750 0001750 00000021602 11256465214 027455 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.Token;
import org.antlr.runtime.RecognitionException;
import java.util.List;
import java.util.ArrayList;
/** Broadcast debug events to multiple listeners. Lets you debug and still
* use the event mechanism to build parse trees etc... Not thread-safe.
* Don't add events in one thread while parser fires events in another.
*
* @see also DebugEventRepeater
*/
public class DebugEventHub implements DebugEventListener {
protected List listeners = new ArrayList();
public DebugEventHub(DebugEventListener listener) {
listeners.add(listener);
}
public DebugEventHub(DebugEventListener a, DebugEventListener b) {
listeners.add(a);
listeners.add(b);
}
/** Add another listener to broadcast events too. Not thread-safe.
* Don't add events in one thread while parser fires events in another.
*/
public void addListener(DebugEventListener listener) {
listeners.add(listener);
}
/* To avoid a mess like this:
public void enterRule(final String ruleName) {
broadcast(new Code(){
public void exec(DebugEventListener listener) {listener.enterRule(ruleName);}}
);
}
I am dup'ing the for-loop in each. Where are Java closures!? blech!
*/
public void enterRule(String grammarFileName, String ruleName) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.enterRule(grammarFileName,ruleName);
}
}
public void exitRule(String grammarFileName, String ruleName) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.exitRule(grammarFileName, ruleName);
}
}
public void enterAlt(int alt) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.enterAlt(alt);
}
}
public void enterSubRule(int decisionNumber) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.enterSubRule(decisionNumber);
}
}
public void exitSubRule(int decisionNumber) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.exitSubRule(decisionNumber);
}
}
public void enterDecision(int decisionNumber) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.enterDecision(decisionNumber);
}
}
public void exitDecision(int decisionNumber) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.exitDecision(decisionNumber);
}
}
public void location(int line, int pos) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.location(line, pos);
}
}
public void consumeToken(Token token) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.consumeToken(token);
}
}
public void consumeHiddenToken(Token token) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.consumeHiddenToken(token);
}
}
public void LT(int index, Token t) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.LT(index, t);
}
}
public void mark(int index) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.mark(index);
}
}
public void rewind(int index) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.rewind(index);
}
}
public void rewind() {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.rewind();
}
}
public void beginBacktrack(int level) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.beginBacktrack(level);
}
}
public void endBacktrack(int level, boolean successful) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.endBacktrack(level, successful);
}
}
public void recognitionException(RecognitionException e) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.recognitionException(e);
}
}
public void beginResync() {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.beginResync();
}
}
public void endResync() {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.endResync();
}
}
public void semanticPredicate(boolean result, String predicate) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.semanticPredicate(result, predicate);
}
}
public void commence() {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.commence();
}
}
public void terminate() {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.terminate();
}
}
// Tree parsing stuff
public void consumeNode(Object t) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.consumeNode(t);
}
}
public void LT(int index, Object t) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.LT(index, t);
}
}
// AST Stuff
public void nilNode(Object t) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.nilNode(t);
}
}
public void errorNode(Object t) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.errorNode(t);
}
}
public void createNode(Object t) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.createNode(t);
}
}
public void createNode(Object node, Token token) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.createNode(node, token);
}
}
public void becomeRoot(Object newRoot, Object oldRoot) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.becomeRoot(newRoot, oldRoot);
}
}
public void addChild(Object root, Object child) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.addChild(root, child);
}
}
public void setTokenBoundaries(Object t, int tokenStartIndex, int tokenStopIndex) {
for (int i = 0; i < listeners.size(); i++) {
DebugEventListener listener = (DebugEventListener)listeners.get(i);
listener.setTokenBoundaries(t, tokenStartIndex, tokenStopIndex);
}
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/ParseTreeBuilder.java 0000644 0001750 0001750 00000007314 11256465214 030173 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.RecognitionException;
import org.antlr.runtime.Token;
import org.antlr.runtime.tree.ParseTree;
import java.util.Stack;
import java.util.ArrayList;
import java.util.List;
/** This parser listener tracks rule entry/exit and token matches
* to build a simple parse tree using ParseTree nodes.
*/
public class ParseTreeBuilder extends BlankDebugEventListener {
public static final String EPSILON_PAYLOAD = "";
Stack callStack = new Stack();
List hiddenTokens = new ArrayList();
int backtracking = 0;
public ParseTreeBuilder(String grammarName) {
ParseTree root = create("");
callStack.push(root);
}
public ParseTree getTree() {
return (ParseTree)callStack.elementAt(0);
}
/** What kind of node to create. You might want to override
* so I factored out creation here.
*/
public ParseTree create(Object payload) {
return new ParseTree(payload);
}
public ParseTree epsilonNode() {
return create(EPSILON_PAYLOAD);
}
/** Backtracking or cyclic DFA, don't want to add nodes to tree */
public void enterDecision(int d) { backtracking++; }
public void exitDecision(int i) { backtracking--; }
public void enterRule(String filename, String ruleName) {
if ( backtracking>0 ) return;
ParseTree parentRuleNode = (ParseTree)callStack.peek();
ParseTree ruleNode = create(ruleName);
parentRuleNode.addChild(ruleNode);
callStack.push(ruleNode);
}
public void exitRule(String filename, String ruleName) {
if ( backtracking>0 ) return;
ParseTree ruleNode = (ParseTree)callStack.peek();
if ( ruleNode.getChildCount()==0 ) {
ruleNode.addChild(epsilonNode());
}
callStack.pop();
}
public void consumeToken(Token token) {
if ( backtracking>0 ) return;
ParseTree ruleNode = (ParseTree)callStack.peek();
ParseTree elementNode = create(token);
elementNode.hiddenTokens = this.hiddenTokens;
this.hiddenTokens = new ArrayList();
ruleNode.addChild(elementNode);
}
public void consumeHiddenToken(Token token) {
if ( backtracking>0 ) return;
hiddenTokens.add(token);
}
public void recognitionException(RecognitionException e) {
if ( backtracking>0 ) return;
ParseTree ruleNode = (ParseTree)callStack.peek();
ParseTree errorNode = create(e);
ruleNode.addChild(errorNode);
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/DebugTreeAdaptor.java 0000644 0001750 0001750 00000015624 11256465214 030156 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.Token;
import org.antlr.runtime.TokenStream;
import org.antlr.runtime.RecognitionException;
import org.antlr.runtime.tree.TreeAdaptor;
/** A TreeAdaptor proxy that fires debugging events to a DebugEventListener
* delegate and uses the TreeAdaptor delegate to do the actual work. All
* AST events are triggered by this adaptor; no code gen changes are needed
* in generated rules. Debugging events are triggered *after* invoking
* tree adaptor routines.
*
* Trees created with actions in rewrite actions like "-> ^(ADD {foo} {bar})"
* cannot be tracked as they might not use the adaptor to create foo, bar.
* The debug listener has to deal with tree node IDs for which it did
* not see a createNode event. A single node is sufficient even
* if it represents a whole tree.
*/
public class DebugTreeAdaptor implements TreeAdaptor {
protected DebugEventListener dbg;
protected TreeAdaptor adaptor;
public DebugTreeAdaptor(DebugEventListener dbg, TreeAdaptor adaptor) {
this.dbg = dbg;
this.adaptor = adaptor;
}
public Object create(Token payload) {
if ( payload.getTokenIndex() < 0 ) {
// could be token conjured up during error recovery
return create(payload.getType(), payload.getText());
}
Object node = adaptor.create(payload);
dbg.createNode(node, payload);
return node;
}
public Object errorNode(TokenStream input, Token start, Token stop,
RecognitionException e)
{
Object node = adaptor.errorNode(input, start, stop, e);
if ( node!=null ) {
dbg.errorNode(node);
}
return node;
}
public Object dupTree(Object tree) {
Object t = adaptor.dupTree(tree);
// walk the tree and emit create and add child events
// to simulate what dupTree has done. dupTree does not call this debug
// adapter so I must simulate.
simulateTreeConstruction(t);
return t;
}
/** ^(A B C): emit create A, create B, add child, ...*/
protected void simulateTreeConstruction(Object t) {
dbg.createNode(t);
int n = adaptor.getChildCount(t);
for (int i=0; i,1:0]
* location 7 7
* enterSubRule 2
* enter decision 2
* LT(1)
* LT(1)
* recognitionException NoViableAltException 2 1 2
* exit decision 2
* exitSubRule 2
* beginResync
* LT(1)
* consumeToken [c/<4>,1:1]
* LT(1)
* endResync
* LT(-1)
* exitRule b
* terminate
*/
public void recognitionException(RecognitionException e);
/** Indicates the recognizer is about to consume tokens to resynchronize
* the parser. Any consume events from here until the recovered event
* are not part of the parse--they are dead tokens.
*/
public void beginResync();
/** Indicates that the recognizer has finished consuming tokens in order
* to resychronize. There may be multiple beginResync/endResync pairs
* before the recognizer comes out of errorRecovery mode (in which
* multiple errors are suppressed). This will be useful
* in a gui where you want to probably grey out tokens that are consumed
* but not matched to anything in grammar. Anything between
* a beginResync/endResync pair was tossed out by the parser.
*/
public void endResync();
/** A semantic predicate was evaluate with this result and action text */
public void semanticPredicate(boolean result, String predicate);
/** Announce that parsing has begun. Not technically useful except for
* sending events over a socket. A GUI for example will launch a thread
* to connect and communicate with a remote parser. The thread will want
* to notify the GUI when a connection is made. ANTLR parsers
* trigger this upon entry to the first rule (the ruleLevel is used to
* figure this out).
*/
public void commence();
/** Parsing is over; successfully or not. Mostly useful for telling
* remote debugging listeners that it's time to quit. When the rule
* invocation level goes to zero at the end of a rule, we are done
* parsing.
*/
public void terminate();
// T r e e P a r s i n g
/** Input for a tree parser is an AST, but we know nothing for sure
* about a node except its type and text (obtained from the adaptor).
* This is the analog of the consumeToken method. Again, the ID is
* the hashCode usually of the node so it only works if hashCode is
* not implemented. If the type is UP or DOWN, then
* the ID is not really meaningful as it's fixed--there is
* just one UP node and one DOWN navigation node.
* @param t
*/
public void consumeNode(Object t);
/** The tree parser lookedahead. If the type is UP or DOWN,
* then the ID is not really meaningful as it's fixed--there is
* just one UP node and one DOWN navigation node.
*/
public void LT(int i, Object t);
// A S T E v e n t s
/** A nil was created (even nil nodes have a unique ID...
* they are not "null" per se). As of 4/28/2006, this
* seems to be uniquely triggered when starting a new subtree
* such as when entering a subrule in automatic mode and when
* building a tree in rewrite mode.
*
* If you are receiving this event over a socket via
* RemoteDebugEventSocketListener then only t.ID is set.
*/
public void nilNode(Object t);
/** Upon syntax error, recognizers bracket the error with an error node
* if they are building ASTs.
* @param t
*/
public void errorNode(Object t);
/** Announce a new node built from token elements such as type etc...
*
* If you are receiving this event over a socket via
* RemoteDebugEventSocketListener then only t.ID, type, text are
* set.
*/
public void createNode(Object t);
/** Announce a new node built from an existing token.
*
* If you are receiving this event over a socket via
* RemoteDebugEventSocketListener then only node.ID and token.tokenIndex
* are set.
*/
public void createNode(Object node, Token token);
/** Make a node the new root of an existing root. See
*
* Note: the newRootID parameter is possibly different
* than the TreeAdaptor.becomeRoot() newRoot parameter.
* In our case, it will always be the result of calling
* TreeAdaptor.becomeRoot() and not root_n or whatever.
*
* The listener should assume that this event occurs
* only when the current subrule (or rule) subtree is
* being reset to newRootID.
*
* If you are receiving this event over a socket via
* RemoteDebugEventSocketListener then only IDs are set.
*
* @see org.antlr.runtime.tree.TreeAdaptor.becomeRoot()
*/
public void becomeRoot(Object newRoot, Object oldRoot);
/** Make childID a child of rootID.
*
* If you are receiving this event over a socket via
* RemoteDebugEventSocketListener then only IDs are set.
*
* @see org.antlr.runtime.tree.TreeAdaptor.addChild()
*/
public void addChild(Object root, Object child);
/** Set the token start/stop token index for a subtree root or node.
*
* If you are receiving this event over a socket via
* RemoteDebugEventSocketListener then only t.ID is set.
*/
public void setTokenBoundaries(Object t, int tokenStartIndex, int tokenStopIndex);
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/DebugTreeNodeStream.java 0000644 0001750 0001750 00000010475 11256465214 030624 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.tree.TreeAdaptor;
import org.antlr.runtime.tree.TreeNodeStream;
import org.antlr.runtime.TokenStream;
/** Debug any tree node stream. The constructor accepts the stream
* and a debug listener. As node stream calls come in, debug events
* are triggered.
*/
public class DebugTreeNodeStream implements TreeNodeStream {
protected DebugEventListener dbg;
protected TreeAdaptor adaptor;
protected TreeNodeStream input;
protected boolean initialStreamState = true;
/** Track the last mark() call result value for use in rewind(). */
protected int lastMarker;
public DebugTreeNodeStream(TreeNodeStream input,
DebugEventListener dbg)
{
this.input = input;
this.adaptor = input.getTreeAdaptor();
this.input.setUniqueNavigationNodes(true);
setDebugListener(dbg);
}
public void setDebugListener(DebugEventListener dbg) {
this.dbg = dbg;
}
public TreeAdaptor getTreeAdaptor() {
return adaptor;
}
public void consume() {
Object node = input.LT(1);
input.consume();
dbg.consumeNode(node);
}
public Object get(int i) {
return input.get(i);
}
public Object LT(int i) {
Object node = input.LT(i);
int ID = adaptor.getUniqueID(node);
String text = adaptor.getText(node);
int type = adaptor.getType(node);
dbg.LT(i, node);
return node;
}
public int LA(int i) {
Object node = input.LT(i);
int ID = adaptor.getUniqueID(node);
String text = adaptor.getText(node);
int type = adaptor.getType(node);
dbg.LT(i, node);
return type;
}
public int mark() {
lastMarker = input.mark();
dbg.mark(lastMarker);
return lastMarker;
}
public int index() {
return input.index();
}
public void rewind(int marker) {
dbg.rewind(marker);
input.rewind(marker);
}
public void rewind() {
dbg.rewind();
input.rewind(lastMarker);
}
public void release(int marker) {
}
public void seek(int index) {
// TODO: implement seek in dbg interface
// db.seek(index);
input.seek(index);
}
public int size() {
return input.size();
}
public void reset() { ; }
public Object getTreeSource() {
return input;
}
public String getSourceName() {
return getTokenStream().getSourceName();
}
public TokenStream getTokenStream() {
return input.getTokenStream();
}
/** It is normally this object that instructs the node stream to
* create unique nav nodes, but to satisfy interface, we have to
* define it. It might be better to ignore the parameter but
* there might be a use for it later, so I'll leave.
*/
public void setUniqueNavigationNodes(boolean uniqueNavigationNodes) {
input.setUniqueNavigationNodes(uniqueNavigationNodes);
}
public void replaceChildren(Object parent, int startChildIndex, int stopChildIndex, Object t) {
input.replaceChildren(parent, startChildIndex, stopChildIndex, t);
}
public String toString(Object start, Object stop) {
return input.toString(start,stop);
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/DebugEventSocketProxy.java 0000644 0001750 0001750 00000024023 11256465214 031231 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.RecognitionException;
import org.antlr.runtime.Token;
import org.antlr.runtime.BaseRecognizer;
import org.antlr.runtime.tree.TreeAdaptor;
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
/** A proxy debug event listener that forwards events over a socket to
* a debugger (or any other listener) using a simple text-based protocol;
* one event per line. ANTLRWorks listens on server socket with a
* RemoteDebugEventSocketListener instance. These two objects must therefore
* be kept in sync. New events must be handled on both sides of socket.
*/
public class DebugEventSocketProxy extends BlankDebugEventListener {
public static final int DEFAULT_DEBUGGER_PORT = 49100; // was 49153
protected int port = DEFAULT_DEBUGGER_PORT;
protected ServerSocket serverSocket;
protected Socket socket;
protected String grammarFileName;
protected PrintWriter out;
protected BufferedReader in;
/** Who am i debugging? */
protected BaseRecognizer recognizer;
/** Almost certainly the recognizer will have adaptor set, but
* we don't know how to cast it (Parser or TreeParser) to get
* the adaptor field. Must be set with a constructor. :(
*/
protected TreeAdaptor adaptor;
public DebugEventSocketProxy(BaseRecognizer recognizer, TreeAdaptor adaptor) {
this(recognizer, DEFAULT_DEBUGGER_PORT, adaptor);
}
public DebugEventSocketProxy(BaseRecognizer recognizer, int port, TreeAdaptor adaptor) {
this.grammarFileName = recognizer.getGrammarFileName();
this.adaptor = adaptor;
this.port = port;
}
public void handshake() throws IOException {
if ( serverSocket==null ) {
serverSocket = new ServerSocket(port);
socket = serverSocket.accept();
socket.setTcpNoDelay(true);
OutputStream os = socket.getOutputStream();
OutputStreamWriter osw = new OutputStreamWriter(os, "UTF8");
out = new PrintWriter(new BufferedWriter(osw));
InputStream is = socket.getInputStream();
InputStreamReader isr = new InputStreamReader(is, "UTF8");
in = new BufferedReader(isr);
out.println("ANTLR "+ DebugEventListener.PROTOCOL_VERSION);
out.println("grammar \""+ grammarFileName);
out.flush();
ack();
}
}
public void commence() {
// don't bother sending event; listener will trigger upon connection
}
public void terminate() {
transmit("terminate");
out.close();
try {
socket.close();
}
catch (IOException ioe) {
ioe.printStackTrace(System.err);
}
}
protected void ack() {
try {
in.readLine();
}
catch (IOException ioe) {
ioe.printStackTrace(System.err);
}
}
protected void transmit(String event) {
out.println(event);
out.flush();
ack();
}
public void enterRule(String grammarFileName, String ruleName) {
transmit("enterRule\t"+grammarFileName+"\t"+ruleName);
}
public void enterAlt(int alt) {
transmit("enterAlt\t"+alt);
}
public void exitRule(String grammarFileName, String ruleName) {
transmit("exitRule\t"+grammarFileName+"\t"+ruleName);
}
public void enterSubRule(int decisionNumber) {
transmit("enterSubRule\t"+decisionNumber);
}
public void exitSubRule(int decisionNumber) {
transmit("exitSubRule\t"+decisionNumber);
}
public void enterDecision(int decisionNumber) {
transmit("enterDecision\t"+decisionNumber);
}
public void exitDecision(int decisionNumber) {
transmit("exitDecision\t"+decisionNumber);
}
public void consumeToken(Token t) {
String buf = serializeToken(t);
transmit("consumeToken\t"+buf);
}
public void consumeHiddenToken(Token t) {
String buf = serializeToken(t);
transmit("consumeHiddenToken\t"+buf);
}
public void LT(int i, Token t) {
if(t != null)
transmit("LT\t"+i+"\t"+serializeToken(t));
}
public void mark(int i) {
transmit("mark\t"+i);
}
public void rewind(int i) {
transmit("rewind\t"+i);
}
public void rewind() {
transmit("rewind");
}
public void beginBacktrack(int level) {
transmit("beginBacktrack\t"+level);
}
public void endBacktrack(int level, boolean successful) {
transmit("endBacktrack\t"+level+"\t"+(successful?TRUE:FALSE));
}
public void location(int line, int pos) {
transmit("location\t"+line+"\t"+pos);
}
public void recognitionException(RecognitionException e) {
StringBuffer buf = new StringBuffer(50);
buf.append("exception\t");
buf.append(e.getClass().getName());
// dump only the data common to all exceptions for now
buf.append("\t");
buf.append(e.index);
buf.append("\t");
buf.append(e.line);
buf.append("\t");
buf.append(e.charPositionInLine);
transmit(buf.toString());
}
public void beginResync() {
transmit("beginResync");
}
public void endResync() {
transmit("endResync");
}
public void semanticPredicate(boolean result, String predicate) {
StringBuffer buf = new StringBuffer(50);
buf.append("semanticPredicate\t");
buf.append(result);
serializeText(buf, predicate);
transmit(buf.toString());
}
// A S T P a r s i n g E v e n t s
public void consumeNode(Object t) {
StringBuffer buf = new StringBuffer(50);
buf.append("consumeNode");
serializeNode(buf, t);
transmit(buf.toString());
}
public void LT(int i, Object t) {
int ID = adaptor.getUniqueID(t);
String text = adaptor.getText(t);
int type = adaptor.getType(t);
StringBuffer buf = new StringBuffer(50);
buf.append("LN\t"); // lookahead node; distinguish from LT in protocol
buf.append(i);
serializeNode(buf, t);
transmit(buf.toString());
}
protected void serializeNode(StringBuffer buf, Object t) {
int ID = adaptor.getUniqueID(t);
String text = adaptor.getText(t);
int type = adaptor.getType(t);
buf.append("\t");
buf.append(ID);
buf.append("\t");
buf.append(type);
Token token = adaptor.getToken(t);
int line = -1;
int pos = -1;
if ( token!=null ) {
line = token.getLine();
pos = token.getCharPositionInLine();
}
buf.append("\t");
buf.append(line);
buf.append("\t");
buf.append(pos);
int tokenIndex = adaptor.getTokenStartIndex(t);
buf.append("\t");
buf.append(tokenIndex);
serializeText(buf, text);
}
// A S T E v e n t s
public void nilNode(Object t) {
int ID = adaptor.getUniqueID(t);
transmit("nilNode\t"+ID);
}
public void errorNode(Object t) {
int ID = adaptor.getUniqueID(t);
String text = t.toString();
StringBuffer buf = new StringBuffer(50);
buf.append("errorNode\t");
buf.append(ID);
buf.append("\t");
buf.append(Token.INVALID_TOKEN_TYPE);
serializeText(buf, text);
transmit(buf.toString());
}
public void createNode(Object t) {
int ID = adaptor.getUniqueID(t);
String text = adaptor.getText(t);
int type = adaptor.getType(t);
StringBuffer buf = new StringBuffer(50);
buf.append("createNodeFromTokenElements\t");
buf.append(ID);
buf.append("\t");
buf.append(type);
serializeText(buf, text);
transmit(buf.toString());
}
public void createNode(Object node, Token token) {
int ID = adaptor.getUniqueID(node);
int tokenIndex = token.getTokenIndex();
transmit("createNode\t"+ID+"\t"+tokenIndex);
}
public void becomeRoot(Object newRoot, Object oldRoot) {
int newRootID = adaptor.getUniqueID(newRoot);
int oldRootID = adaptor.getUniqueID(oldRoot);
transmit("becomeRoot\t"+newRootID+"\t"+oldRootID);
}
public void addChild(Object root, Object child) {
int rootID = adaptor.getUniqueID(root);
int childID = adaptor.getUniqueID(child);
transmit("addChild\t"+rootID+"\t"+childID);
}
public void setTokenBoundaries(Object t, int tokenStartIndex, int tokenStopIndex) {
int ID = adaptor.getUniqueID(t);
transmit("setTokenBoundaries\t"+ID+"\t"+tokenStartIndex+"\t"+tokenStopIndex);
}
// support
public void setTreeAdaptor(TreeAdaptor adaptor) { this.adaptor = adaptor; }
public TreeAdaptor getTreeAdaptor() { return adaptor; }
protected String serializeToken(Token t) {
StringBuffer buf = new StringBuffer(50);
buf.append(t.getTokenIndex()); buf.append('\t');
buf.append(t.getType()); buf.append('\t');
buf.append(t.getChannel()); buf.append('\t');
buf.append(t.getLine()); buf.append('\t');
buf.append(t.getCharPositionInLine());
serializeText(buf, t.getText());
return buf.toString();
}
protected void serializeText(StringBuffer buf, String text) {
buf.append("\t\"");
if ( text==null ) {
text = "";
}
// escape \n and \r all text for token appears to exist on one line
// this escape is slow but easy to understand
text = escapeNewlines(text);
buf.append(text);
}
protected String escapeNewlines(String txt) {
txt = txt.replaceAll("%","%25"); // escape all escape char ;)
txt = txt.replaceAll("\n","%0A"); // escape \n
txt = txt.replaceAll("\r","%0D"); // escape \r
return txt;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/DebugParser.java 0000644 0001750 0001750 00000006402 11256465214 027172 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.*;
import java.io.IOException;
public class DebugParser extends Parser {
/** Who to notify when events in the parser occur. */
protected DebugEventListener dbg = null;
/** Used to differentiate between fixed lookahead and cyclic DFA decisions
* while profiling.
*/
public boolean isCyclicDecision = false;
/** Create a normal parser except wrap the token stream in a debug
* proxy that fires consume events.
*/
public DebugParser(TokenStream input, DebugEventListener dbg, RecognizerSharedState state) {
super(input instanceof DebugTokenStream?input:new DebugTokenStream(input,dbg), state);
setDebugListener(dbg);
}
public DebugParser(TokenStream input, RecognizerSharedState state) {
super(input instanceof DebugTokenStream?input:new DebugTokenStream(input,null), state);
}
public DebugParser(TokenStream input, DebugEventListener dbg) {
this(input instanceof DebugTokenStream?input:new DebugTokenStream(input,dbg), dbg, null);
}
/** Provide a new debug event listener for this parser. Notify the
* input stream too that it should send events to this listener.
*/
public void setDebugListener(DebugEventListener dbg) {
if ( input instanceof DebugTokenStream ) {
((DebugTokenStream)input).setDebugListener(dbg);
}
this.dbg = dbg;
}
public DebugEventListener getDebugListener() {
return dbg;
}
public void reportError(IOException e) {
System.err.println(e);
e.printStackTrace(System.err);
}
public void beginResync() {
dbg.beginResync();
}
public void endResync() {
dbg.endResync();
}
public void beginBacktrack(int level) {
dbg.beginBacktrack(level);
}
public void endBacktrack(int level, boolean successful) {
dbg.endBacktrack(level,successful);
}
public void reportError(RecognitionException e) {
dbg.recognitionException(e);
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/DebugEventRepeater.java 0000644 0001750 0001750 00000011005 11256465214 030502 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.Token;
import org.antlr.runtime.RecognitionException;
/** A simple event repeater (proxy) that delegates all functionality to the
* listener sent into the ctor. Useful if you want to listen in on a few
* debug events w/o interrupting the debugger. Just subclass the repeater
* and override the methods you want to listen in on. Remember to call
* the method in this class so the event will continue on to the original
* recipient.
*
* @see DebugEventHub
*/
public class DebugEventRepeater implements DebugEventListener {
protected DebugEventListener listener;
public DebugEventRepeater(DebugEventListener listener) {
this.listener = listener;
}
public void enterRule(String grammarFileName, String ruleName) { listener.enterRule(grammarFileName, ruleName); }
public void exitRule(String grammarFileName, String ruleName) { listener.exitRule(grammarFileName, ruleName); }
public void enterAlt(int alt) { listener.enterAlt(alt); }
public void enterSubRule(int decisionNumber) { listener.enterSubRule(decisionNumber); }
public void exitSubRule(int decisionNumber) { listener.exitSubRule(decisionNumber); }
public void enterDecision(int decisionNumber) { listener.enterDecision(decisionNumber); }
public void exitDecision(int decisionNumber) { listener.exitDecision(decisionNumber); }
public void location(int line, int pos) { listener.location(line, pos); }
public void consumeToken(Token token) { listener.consumeToken(token); }
public void consumeHiddenToken(Token token) { listener.consumeHiddenToken(token); }
public void LT(int i, Token t) { listener.LT(i, t); }
public void mark(int i) { listener.mark(i); }
public void rewind(int i) { listener.rewind(i); }
public void rewind() { listener.rewind(); }
public void beginBacktrack(int level) { listener.beginBacktrack(level); }
public void endBacktrack(int level, boolean successful) { listener.endBacktrack(level, successful); }
public void recognitionException(RecognitionException e) { listener.recognitionException(e); }
public void beginResync() { listener.beginResync(); }
public void endResync() { listener.endResync(); }
public void semanticPredicate(boolean result, String predicate) { listener.semanticPredicate(result, predicate); }
public void commence() { listener.commence(); }
public void terminate() { listener.terminate(); }
// Tree parsing stuff
public void consumeNode(Object t) { listener.consumeNode(t); }
public void LT(int i, Object t) { listener.LT(i, t); }
// AST Stuff
public void nilNode(Object t) { listener.nilNode(t); }
public void errorNode(Object t) { listener.errorNode(t); }
public void createNode(Object t) { listener.createNode(t); }
public void createNode(Object node, Token token) { listener.createNode(node, token); }
public void becomeRoot(Object newRoot, Object oldRoot) { listener.becomeRoot(newRoot, oldRoot); }
public void addChild(Object root, Object child) { listener.addChild(root, child); }
public void setTokenBoundaries(Object t, int tokenStartIndex, int tokenStopIndex) {
listener.setTokenBoundaries(t, tokenStartIndex, tokenStopIndex);
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/Tracer.java 0000644 0001750 0001750 00000004625 11256465214 026214 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.IntStream;
import org.antlr.runtime.TokenStream;
/** The default tracer mimics the traceParser behavior of ANTLR 2.x.
* This listens for debugging events from the parser and implies
* that you cannot debug and trace at the same time.
*/
public class Tracer extends BlankDebugEventListener {
public IntStream input;
protected int level = 0;
public Tracer(IntStream input) {
this.input = input;
}
public void enterRule(String ruleName) {
for (int i=1; i<=level; i++) {System.out.print(" ");}
System.out.println("> "+ruleName+" lookahead(1)="+getInputSymbol(1));
level++;
}
public void exitRule(String ruleName) {
level--;
for (int i=1; i<=level; i++) {System.out.print(" ");}
System.out.println("< "+ruleName+" lookahead(1)="+getInputSymbol(1));
}
public Object getInputSymbol(int k) {
if ( input instanceof TokenStream ) {
return ((TokenStream)input).LT(k);
}
return new Character((char)input.LA(k));
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/debug/DebugTokenStream.java 0000644 0001750 0001750 00000007653 11256465214 030203 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.debug;
import org.antlr.runtime.*;
public class DebugTokenStream implements TokenStream {
protected DebugEventListener dbg;
public TokenStream input;
protected boolean initialStreamState = true;
/** Track the last mark() call result value for use in rewind(). */
protected int lastMarker;
public DebugTokenStream(TokenStream input, DebugEventListener dbg) {
this.input = input;
setDebugListener(dbg);
// force TokenStream to get at least first valid token
// so we know if there are any hidden tokens first in the stream
input.LT(1);
}
public void setDebugListener(DebugEventListener dbg) {
this.dbg = dbg;
}
public void consume() {
if ( initialStreamState ) {
consumeInitialHiddenTokens();
}
int a = input.index();
Token t = input.LT(1);
input.consume();
int b = input.index();
dbg.consumeToken(t);
if ( b>a+1 ) {
// then we consumed more than one token; must be off channel tokens
for (int i=a+1; i0 ) {
// we found the most recent real line / pos info
this.line = priorPayload.getLine();
this.charPositionInLine = priorPayload.getCharPositionInLine();
this.approximateLineInfo = true;
break;
}
--i;
priorNode = nodes.LT(i);
}
}
else { // node created from real token
this.line = payload.getLine();
this.charPositionInLine = payload.getCharPositionInLine();
}
}
else if ( this.node instanceof Tree) {
this.line = ((Tree)this.node).getLine();
this.charPositionInLine = ((Tree)this.node).getCharPositionInLine();
if ( this.node instanceof CommonTree) {
this.token = ((CommonTree)this.node).token;
}
}
else {
int type = adaptor.getType(this.node);
String text = adaptor.getText(this.node);
this.token = new CommonToken(type, text);
}
}
/** Return the token type or char of the unexpected input element */
public int getUnexpectedType() {
if ( input instanceof TokenStream ) {
return token.getType();
}
else if ( input instanceof TreeNodeStream ) {
TreeNodeStream nodes = (TreeNodeStream)input;
TreeAdaptor adaptor = nodes.getTreeAdaptor();
return adaptor.getType(node);
}
else {
return c;
}
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/ 0000755 0001750 0001750 00000000000 11410174107 023761 5 ustar twerner twerner antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeParser.java 0000644 0001750 0001750 00000013436 11256465214 026721 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.*;
import java.util.regex.Pattern;
import java.util.regex.Matcher;
/** A parser for a stream of tree nodes. "tree grammars" result in a subclass
* of this. All the error reporting and recovery is shared with Parser via
* the BaseRecognizer superclass.
*/
public class TreeParser extends BaseRecognizer {
public static final int DOWN = Token.DOWN;
public static final int UP = Token.UP;
// precompiled regex used by inContext
static String dotdot = ".*[^.]\\.\\.[^.].*";
static String doubleEtc = ".*\\.\\.\\.\\s+\\.\\.\\..*";
static Pattern dotdotPattern = Pattern.compile(dotdot);
static Pattern doubleEtcPattern = Pattern.compile(doubleEtc);
protected TreeNodeStream input;
public TreeParser(TreeNodeStream input) {
super(); // highlight that we go to super to set state object
setTreeNodeStream(input);
}
public TreeParser(TreeNodeStream input, RecognizerSharedState state) {
super(state); // share the state object with another parser
setTreeNodeStream(input);
}
public void reset() {
super.reset(); // reset all recognizer state variables
if ( input!=null ) {
input.seek(0); // rewind the input
}
}
/** Set the input stream */
public void setTreeNodeStream(TreeNodeStream input) {
this.input = input;
}
public TreeNodeStream getTreeNodeStream() {
return input;
}
public String getSourceName() {
return input.getSourceName();
}
protected Object getCurrentInputSymbol(IntStream input) {
return ((TreeNodeStream)input).LT(1);
}
protected Object getMissingSymbol(IntStream input,
RecognitionException e,
int expectedTokenType,
BitSet follow)
{
String tokenText =
"";
return new CommonTree(new CommonToken(expectedTokenType, tokenText));
}
/** Match '.' in tree parser has special meaning. Skip node or
* entire tree if node has children. If children, scan until
* corresponding UP node.
*/
public void matchAny(IntStream ignore) { // ignore stream, copy of input
state.errorRecovery = false;
state.failed = false;
Object look = input.LT(1);
if ( input.getTreeAdaptor().getChildCount(look)==0 ) {
input.consume(); // not subtree, consume 1 node and return
return;
}
// current node is a subtree, skip to corresponding UP.
// must count nesting level to get right UP
int level=0;
int tokenType = input.getTreeAdaptor().getType(look);
while ( tokenType!=Token.EOF && !(tokenType==UP && level==0) ) {
input.consume();
look = input.LT(1);
tokenType = input.getTreeAdaptor().getType(look);
if ( tokenType == DOWN ) {
level++;
}
else if ( tokenType == UP ) {
level--;
}
}
input.consume(); // consume UP
}
/** We have DOWN/UP nodes in the stream that have no line info; override.
* plus we want to alter the exception type. Don't try to recover
* from tree parser errors inline...
*/
protected Object recoverFromMismatchedToken(IntStream input,
int ttype,
BitSet follow)
throws RecognitionException
{
throw new MismatchedTreeNodeException(ttype, (TreeNodeStream)input);
}
/** Prefix error message with the grammar name because message is
* always intended for the programmer because the parser built
* the input tree not the user.
*/
public String getErrorHeader(RecognitionException e) {
return getGrammarFileName()+": node from "+
(e.approximateLineInfo?"after ":"")+"line "+e.line+":"+e.charPositionInLine;
}
/** Tree parsers parse nodes they usually have a token object as
* payload. Set the exception token and do the default behavior.
*/
public String getErrorMessage(RecognitionException e, String[] tokenNames) {
if ( this instanceof TreeParser ) {
TreeAdaptor adaptor = ((TreeNodeStream)e.input).getTreeAdaptor();
e.token = adaptor.getToken(e.node);
if ( e.token==null ) { // could be an UP/DOWN node
e.token = new CommonToken(adaptor.getType(e.node),
adaptor.getText(e.node));
}
}
return super.getErrorMessage(e, tokenNames);
}
public void traceIn(String ruleName, int ruleIndex) {
super.traceIn(ruleName, ruleIndex, input.LT(1));
}
public void traceOut(String ruleName, int ruleIndex) {
super.traceOut(ruleName, ruleIndex, input.LT(1));
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/RewriteEmptyStreamException.java 0000644 0001750 0001750 00000003307 11256465214 032334 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
/** Ref to ID or expr but no tokens in ID stream or subtrees in expr stream */
public class RewriteEmptyStreamException extends RewriteCardinalityException {
public RewriteEmptyStreamException(String elementDescription) {
super(elementDescription);
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/BaseTreeAdaptor.java 0000644 0001750 0001750 00000022516 11256465214 027651 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import org.antlr.runtime.TokenStream;
import org.antlr.runtime.RecognitionException;
import java.util.HashMap;
import java.util.Map;
/** A TreeAdaptor that works with any Tree implementation. */
public abstract class BaseTreeAdaptor implements TreeAdaptor {
/** System.identityHashCode() is not always unique; we have to
* track ourselves. That's ok, it's only for debugging, though it's
* expensive: we have to create a hashtable with all tree nodes in it.
*/
protected Map treeToUniqueIDMap;
protected int uniqueNodeID = 1;
public Object nil() {
return create(null);
}
/** create tree node that holds the start and stop tokens associated
* with an error.
*
* If you specify your own kind of tree nodes, you will likely have to
* override this method. CommonTree returns Token.INVALID_TOKEN_TYPE
* if no token payload but you might have to set token type for diff
* node type.
*
* You don't have to subclass CommonErrorNode; you will likely need to
* subclass your own tree node class to avoid class cast exception.
*/
public Object errorNode(TokenStream input, Token start, Token stop,
RecognitionException e)
{
CommonErrorNode t = new CommonErrorNode(input, start, stop, e);
//System.out.println("returning error node '"+t+"' @index="+input.index());
return t;
}
public boolean isNil(Object tree) {
return ((Tree)tree).isNil();
}
public Object dupTree(Object tree) {
return dupTree(tree, null);
}
/** This is generic in the sense that it will work with any kind of
* tree (not just Tree interface). It invokes the adaptor routines
* not the tree node routines to do the construction.
*/
public Object dupTree(Object t, Object parent) {
if ( t==null ) {
return null;
}
Object newTree = dupNode(t);
// ensure new subtree root has parent/child index set
setChildIndex(newTree, getChildIndex(t)); // same index in new tree
setParent(newTree, parent);
int n = getChildCount(t);
for (int i = 0; i < n; i++) {
Object child = getChild(t, i);
Object newSubTree = dupTree(child, t);
addChild(newTree, newSubTree);
}
return newTree;
}
/** Add a child to the tree t. If child is a flat tree (a list), make all
* in list children of t. Warning: if t has no children, but child does
* and child isNil then you can decide it is ok to move children to t via
* t.children = child.children; i.e., without copying the array. Just
* make sure that this is consistent with have the user will build
* ASTs.
*/
public void addChild(Object t, Object child) {
if ( t!=null && child!=null ) {
((Tree)t).addChild((Tree)child);
}
}
/** If oldRoot is a nil root, just copy or move the children to newRoot.
* If not a nil root, make oldRoot a child of newRoot.
*
* old=^(nil a b c), new=r yields ^(r a b c)
* old=^(a b c), new=r yields ^(r ^(a b c))
*
* If newRoot is a nil-rooted single child tree, use the single
* child as the new root node.
*
* old=^(nil a b c), new=^(nil r) yields ^(r a b c)
* old=^(a b c), new=^(nil r) yields ^(r ^(a b c))
*
* If oldRoot was null, it's ok, just return newRoot (even if isNil).
*
* old=null, new=r yields r
* old=null, new=^(nil r) yields ^(nil r)
*
* Return newRoot. Throw an exception if newRoot is not a
* simple node or nil root with a single child node--it must be a root
* node. If newRoot is ^(nil x) return x as newRoot.
*
* Be advised that it's ok for newRoot to point at oldRoot's
* children; i.e., you don't have to copy the list. We are
* constructing these nodes so we should have this control for
* efficiency.
*/
public Object becomeRoot(Object newRoot, Object oldRoot) {
//System.out.println("becomeroot new "+newRoot.toString()+" old "+oldRoot);
Tree newRootTree = (Tree)newRoot;
Tree oldRootTree = (Tree)oldRoot;
if ( oldRoot==null ) {
return newRoot;
}
// handle ^(nil real-node)
if ( newRootTree.isNil() ) {
int nc = newRootTree.getChildCount();
if ( nc==1 ) newRootTree = (Tree)newRootTree.getChild(0);
else if ( nc >1 ) {
// TODO: make tree run time exceptions hierarchy
throw new RuntimeException("more than one node as root (TODO: make exception hierarchy)");
}
}
// add oldRoot to newRoot; addChild takes care of case where oldRoot
// is a flat list (i.e., nil-rooted tree). All children of oldRoot
// are added to newRoot.
newRootTree.addChild(oldRootTree);
return newRootTree;
}
/** Transform ^(nil x) to x and nil to null */
public Object rulePostProcessing(Object root) {
//System.out.println("rulePostProcessing: "+((Tree)root).toStringTree());
Tree r = (Tree)root;
if ( r!=null && r.isNil() ) {
if ( r.getChildCount()==0 ) {
r = null;
}
else if ( r.getChildCount()==1 ) {
r = (Tree)r.getChild(0);
// whoever invokes rule will set parent and child index
r.setParent(null);
r.setChildIndex(-1);
}
}
return r;
}
public Object becomeRoot(Token newRoot, Object oldRoot) {
return becomeRoot(create(newRoot), oldRoot);
}
public Object create(int tokenType, Token fromToken) {
fromToken = createToken(fromToken);
//((ClassicToken)fromToken).setType(tokenType);
fromToken.setType(tokenType);
Tree t = (Tree)create(fromToken);
return t;
}
public Object create(int tokenType, Token fromToken, String text) {
fromToken = createToken(fromToken);
fromToken.setType(tokenType);
fromToken.setText(text);
Tree t = (Tree)create(fromToken);
return t;
}
public Object create(int tokenType, String text) {
Token fromToken = createToken(tokenType, text);
Tree t = (Tree)create(fromToken);
return t;
}
public int getType(Object t) {
return ((Tree)t).getType();
}
public void setType(Object t, int type) {
throw new NoSuchMethodError("don't know enough about Tree node");
}
public String getText(Object t) {
return ((Tree)t).getText();
}
public void setText(Object t, String text) {
throw new NoSuchMethodError("don't know enough about Tree node");
}
public Object getChild(Object t, int i) {
return ((Tree)t).getChild(i);
}
public void setChild(Object t, int i, Object child) {
((Tree)t).setChild(i, (Tree)child);
}
public Object deleteChild(Object t, int i) {
return ((Tree)t).deleteChild(i);
}
public int getChildCount(Object t) {
return ((Tree)t).getChildCount();
}
public int getUniqueID(Object node) {
if ( treeToUniqueIDMap==null ) {
treeToUniqueIDMap = new HashMap();
}
Integer prevID = (Integer)treeToUniqueIDMap.get(node);
if ( prevID!=null ) {
return prevID.intValue();
}
int ID = uniqueNodeID;
treeToUniqueIDMap.put(node, new Integer(ID));
uniqueNodeID++;
return ID;
// GC makes these nonunique:
// return System.identityHashCode(node);
}
/** Tell me how to create a token for use with imaginary token nodes.
* For example, there is probably no input symbol associated with imaginary
* token DECL, but you need to create it as a payload or whatever for
* the DECL node as in ^(DECL type ID).
*
* If you care what the token payload objects' type is, you should
* override this method and any other createToken variant.
*/
public abstract Token createToken(int tokenType, String text);
/** Tell me how to create a token for use with imaginary token nodes.
* For example, there is probably no input symbol associated with imaginary
* token DECL, but you need to create it as a payload or whatever for
* the DECL node as in ^(DECL type ID).
*
* This is a variant of createToken where the new token is derived from
* an actual real input token. Typically this is for converting '{'
* tokens to BLOCK etc... You'll see
*
* r : lc='{' ID+ '}' -> ^(BLOCK[$lc] ID+) ;
*
* If you care what the token payload objects' type is, you should
* override this method and any other createToken variant.
*/
public abstract Token createToken(Token fromToken);
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/RewriteRuleNodeStream.java 0000644 0001750 0001750 00000005106 11256465214 031073 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import java.util.List;
/** Queues up nodes matched on left side of -> in a tree parser. This is
* the analog of RewriteRuleTokenStream for normal parsers.
*/
public class RewriteRuleNodeStream extends RewriteRuleElementStream {
public RewriteRuleNodeStream(TreeAdaptor adaptor, String elementDescription) {
super(adaptor, elementDescription);
}
/** Create a stream with one element */
public RewriteRuleNodeStream(TreeAdaptor adaptor,
String elementDescription,
Object oneElement)
{
super(adaptor, elementDescription, oneElement);
}
/** Create a stream, but feed off an existing list */
public RewriteRuleNodeStream(TreeAdaptor adaptor,
String elementDescription,
List elements)
{
super(adaptor, elementDescription, elements);
}
public Object nextNode() {
return _next();
}
protected Object toTree(Object el) {
return adaptor.dupNode(el);
}
protected Object dup(Object el) {
// we dup every node, so don't have to worry about calling dup; short-
// circuited next() so it doesn't call.
throw new UnsupportedOperationException("dup can't be called for a node stream.");
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/RewriteEarlyExitException.java 0000644 0001750 0001750 00000003342 11256465214 031767 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
/** No elements within a (...)+ in a rewrite rule */
public class RewriteEarlyExitException extends RewriteCardinalityException {
public RewriteEarlyExitException() {
super(null);
}
public RewriteEarlyExitException(String elementDescription) {
super(elementDescription);
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/Tree.java 0000644 0001750 0001750 00000010335 11256465214 025537 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import java.util.List;
/** What does a tree look like? ANTLR has a number of support classes
* such as CommonTreeNodeStream that work on these kinds of trees. You
* don't have to make your trees implement this interface, but if you do,
* you'll be able to use more support code.
*
* NOTE: When constructing trees, ANTLR can build any kind of tree; it can
* even use Token objects as trees if you add a child list to your tokens.
*
* This is a tree node without any payload; just navigation and factory stuff.
*/
public interface Tree {
public static final Tree INVALID_NODE = new CommonTree(Token.INVALID_TOKEN);
Tree getChild(int i);
int getChildCount();
// Tree tracks parent and child index now > 3.0
public Tree getParent();
public void setParent(Tree t);
/** Is there is a node above with token type ttype? */
public boolean hasAncestor(int ttype);
/** Walk upwards and get first ancestor with this token type. */
public Tree getAncestor(int ttype);
/** Return a list of all ancestors of this node. The first node of
* list is the root and the last is the parent of this node.
*/
public List getAncestors();
/** This node is what child index? 0..n-1 */
public int getChildIndex();
public void setChildIndex(int index);
/** Set the parent and child index values for all children */
public void freshenParentAndChildIndexes();
/** Add t as a child to this node. If t is null, do nothing. If t
* is nil, add all children of t to this' children.
*/
void addChild(Tree t);
/** Set ith child (0..n-1) to t; t must be non-null and non-nil node */
public void setChild(int i, Tree t);
public Object deleteChild(int i);
/** Delete children from start to stop and replace with t even if t is
* a list (nil-root tree). num of children can increase or decrease.
* For huge child lists, inserting children can force walking rest of
* children to set their childindex; could be slow.
*/
public void replaceChildren(int startChildIndex, int stopChildIndex, Object t);
/** Indicates the node is a nil node but may still have children, meaning
* the tree is a flat list.
*/
boolean isNil();
/** What is the smallest token index (indexing from 0) for this node
* and its children?
*/
int getTokenStartIndex();
void setTokenStartIndex(int index);
/** What is the largest token index (indexing from 0) for this node
* and its children?
*/
int getTokenStopIndex();
void setTokenStopIndex(int index);
Tree dupNode();
/** Return a token type; needed for tree parsing */
int getType();
String getText();
/** In case we don't have a token payload, what is the line for errors? */
int getLine();
int getCharPositionInLine();
String toStringTree();
String toString();
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/BaseTree.java 0000644 0001750 0001750 00000025213 11256465214 026333 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import java.util.ArrayList;
import java.util.List;
/** A generic tree implementation with no payload. You must subclass to
* actually have any user data. ANTLR v3 uses a list of children approach
* instead of the child-sibling approach in v2. A flat tree (a list) is
* an empty node whose children represent the list. An empty, but
* non-null node is called "nil".
*/
public abstract class BaseTree implements Tree {
protected List children;
public BaseTree() {
}
/** Create a new node from an existing node does nothing for BaseTree
* as there are no fields other than the children list, which cannot
* be copied as the children are not considered part of this node.
*/
public BaseTree(Tree node) {
}
public Tree getChild(int i) {
if ( children==null || i>=children.size() ) {
return null;
}
return (Tree)children.get(i);
}
/** Get the children internal List; note that if you directly mess with
* the list, do so at your own risk.
*/
public List getChildren() {
return children;
}
public Tree getFirstChildWithType(int type) {
for (int i = 0; children!=null && i < children.size(); i++) {
Tree t = (Tree) children.get(i);
if ( t.getType()==type ) {
return t;
}
}
return null;
}
public int getChildCount() {
if ( children==null ) {
return 0;
}
return children.size();
}
/** Add t as child of this node.
*
* Warning: if t has no children, but child does
* and child isNil then this routine moves children to t via
* t.children = child.children; i.e., without copying the array.
*/
public void addChild(Tree t) {
//System.out.println("add child "+t.toStringTree()+" "+this.toStringTree());
//System.out.println("existing children: "+children);
if ( t==null ) {
return; // do nothing upon addChild(null)
}
BaseTree childTree = (BaseTree)t;
if ( childTree.isNil() ) { // t is an empty node possibly with children
if ( this.children!=null && this.children == childTree.children ) {
throw new RuntimeException("attempt to add child list to itself");
}
// just add all of childTree's children to this
if ( childTree.children!=null ) {
if ( this.children!=null ) { // must copy, this has children already
int n = childTree.children.size();
for (int i = 0; i < n; i++) {
Tree c = (Tree)childTree.children.get(i);
this.children.add(c);
// handle double-link stuff for each child of nil root
c.setParent(this);
c.setChildIndex(children.size()-1);
}
}
else {
// no children for this but t has children; just set pointer
// call general freshener routine
this.children = childTree.children;
this.freshenParentAndChildIndexes();
}
}
}
else { // child is not nil (don't care about children)
if ( children==null ) {
children = createChildrenList(); // create children list on demand
}
children.add(t);
childTree.setParent(this);
childTree.setChildIndex(children.size()-1);
}
// System.out.println("now children are: "+children);
}
/** Add all elements of kids list as children of this node */
public void addChildren(List kids) {
for (int i = 0; i < kids.size(); i++) {
Tree t = (Tree) kids.get(i);
addChild(t);
}
}
public void setChild(int i, Tree t) {
if ( t==null ) {
return;
}
if ( t.isNil() ) {
throw new IllegalArgumentException("Can't set single child to a list");
}
if ( children==null ) {
children = createChildrenList();
}
children.set(i, t);
t.setParent(this);
t.setChildIndex(i);
}
public Object deleteChild(int i) {
if ( children==null ) {
return null;
}
Tree killed = (Tree)children.remove(i);
// walk rest and decrement their child indexes
this.freshenParentAndChildIndexes(i);
return killed;
}
/** Delete children from start to stop and replace with t even if t is
* a list (nil-root tree). num of children can increase or decrease.
* For huge child lists, inserting children can force walking rest of
* children to set their childindex; could be slow.
*/
public void replaceChildren(int startChildIndex, int stopChildIndex, Object t) {
/*
System.out.println("replaceChildren "+startChildIndex+", "+stopChildIndex+
" with "+((BaseTree)t).toStringTree());
System.out.println("in="+toStringTree());
*/
if ( children==null ) {
throw new IllegalArgumentException("indexes invalid; no children in list");
}
int replacingHowMany = stopChildIndex - startChildIndex + 1;
int replacingWithHowMany;
BaseTree newTree = (BaseTree)t;
List newChildren = null;
// normalize to a list of children to add: newChildren
if ( newTree.isNil() ) {
newChildren = newTree.children;
}
else {
newChildren = new ArrayList(1);
newChildren.add(newTree);
}
replacingWithHowMany = newChildren.size();
int numNewChildren = newChildren.size();
int delta = replacingHowMany - replacingWithHowMany;
// if same number of nodes, do direct replace
if ( delta == 0 ) {
int j = 0; // index into new children
for (int i=startChildIndex; i<=stopChildIndex; i++) {
BaseTree child = (BaseTree)newChildren.get(j);
children.set(i, child);
child.setParent(this);
child.setChildIndex(i);
j++;
}
}
else if ( delta > 0 ) { // fewer new nodes than there were
// set children and then delete extra
for (int j=0; j0 ) {
buf.append(' ');
}
buf.append(t.toStringTree());
}
if ( !isNil() ) {
buf.append(")");
}
return buf.toString();
}
public int getLine() {
return 0;
}
public int getCharPositionInLine() {
return 0;
}
/** Override to say how a node (not a tree) should look as text */
public abstract String toString();
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeNodeStream.java 0000644 0001750 0001750 00000010744 11256465214 027525 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.IntStream;
import org.antlr.runtime.TokenStream;
/** A stream of tree nodes, accessing nodes from a tree of some kind */
public interface TreeNodeStream extends IntStream {
/** Get a tree node at an absolute index i; 0..n-1.
* If you don't want to buffer up nodes, then this method makes no
* sense for you.
*/
public Object get(int i);
/** Get tree node at current input pointer + i ahead where i=1 is next node.
* i<0 indicates nodes in the past. So LT(-1) is previous node, but
* implementations are not required to provide results for k < -1.
* LT(0) is undefined. For i>=n, return null.
* Return null for LT(0) and any index that results in an absolute address
* that is negative.
*
* This is analogus to the LT() method of the TokenStream, but this
* returns a tree node instead of a token. Makes code gen identical
* for both parser and tree grammars. :)
*/
public Object LT(int k);
/** Where is this stream pulling nodes from? This is not the name, but
* the object that provides node objects.
*/
public Object getTreeSource();
/** If the tree associated with this stream was created from a TokenStream,
* you can specify it here. Used to do rule $text attribute in tree
* parser. Optional unless you use tree parser rule text attribute
* or output=template and rewrite=true options.
*/
public TokenStream getTokenStream();
/** What adaptor can tell me how to interpret/navigate nodes and
* trees. E.g., get text of a node.
*/
public TreeAdaptor getTreeAdaptor();
/** As we flatten the tree, we use UP, DOWN nodes to represent
* the tree structure. When debugging we need unique nodes
* so we have to instantiate new ones. When doing normal tree
* parsing, it's slow and a waste of memory to create unique
* navigation nodes. Default should be false;
*/
public void setUniqueNavigationNodes(boolean uniqueNavigationNodes);
/** Reset the tree node stream in such a way that it acts like
* a freshly constructed stream.
*/
public void reset();
/** Return the text of all nodes from start to stop, inclusive.
* If the stream does not buffer all the nodes then it can still
* walk recursively from start until stop. You can always return
* null or "" too, but users should not access $ruleLabel.text in
* an action of course in that case.
*/
public String toString(Object start, Object stop);
// REWRITING TREES (used by tree parser)
/** Replace from start to stop child index of parent with t, which might
* be a list. Number of children may be different
* after this call. The stream is notified because it is walking the
* tree and might need to know you are monkeying with the underlying
* tree. Also, it might be able to modify the node stream to avoid
* restreaming for future phases.
*
* If parent is null, don't do anything; must be at root of overall tree.
* Can't replace whatever points to the parent externally. Do nothing.
*/
public void replaceChildren(Object parent, int startChildIndex, int stopChildIndex, Object t);
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeRewriter.java 0000644 0001750 0001750 00000011436 11256465214 027266 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.RecognizerSharedState;
import org.antlr.runtime.RecognitionException;
import org.antlr.runtime.TokenStream;
public class TreeRewriter extends TreeParser {
public interface fptr {
public Object rule() throws RecognitionException;
}
protected boolean showTransformations = false;
protected TokenStream originalTokenStream;
protected TreeAdaptor originalAdaptor;
public TreeRewriter(TreeNodeStream input) {
this(input, new RecognizerSharedState());
}
public TreeRewriter(TreeNodeStream input, RecognizerSharedState state) {
super(input, state);
originalAdaptor = input.getTreeAdaptor();
originalTokenStream = input.getTokenStream();
}
public Object applyOnce(Object t, fptr whichRule) {
if ( t==null ) return null;
try {
// share TreeParser object but not parsing-related state
state = new RecognizerSharedState();
input = new CommonTreeNodeStream(originalAdaptor, t);
((CommonTreeNodeStream)input).setTokenStream(originalTokenStream);
setBacktrackingLevel(1);
TreeRuleReturnScope r = (TreeRuleReturnScope)whichRule.rule();
setBacktrackingLevel(0);
if ( failed() ) return t;
if ( showTransformations &&
r!=null && !t.equals(r.getTree()) && r.getTree()!=null )
{
reportTransformation(t, r.getTree());
}
if ( r!=null && r.getTree()!=null ) return r.getTree();
else return t;
}
catch (RecognitionException e) { ; }
return t;
}
public Object applyRepeatedly(Object t, fptr whichRule) {
boolean treeChanged = true;
while ( treeChanged ) {
Object u = applyOnce(t, whichRule);
treeChanged = !t.equals(u);
t = u;
}
return t;
}
public Object downup(Object t) { return downup(t, false); }
public Object downup(Object t, boolean showTransformations) {
this.showTransformations = showTransformations;
TreeVisitor v = new TreeVisitor(new CommonTreeAdaptor());
TreeVisitorAction actions = new TreeVisitorAction() {
public Object pre(Object t) { return applyOnce(t, topdown_fptr); }
public Object post(Object t) { return applyRepeatedly(t, bottomup_ftpr); }
};
t = v.visit(t, actions);
return t;
}
/** Override this if you need transformation tracing to go somewhere
* other than stdout or if you're not using Tree-derived trees.
*/
public void reportTransformation(Object oldTree, Object newTree) {
System.out.println(((Tree)oldTree).toStringTree()+" -> "+
((Tree)newTree).toStringTree());
}
fptr topdown_fptr = new fptr() {
public Object rule() throws RecognitionException { return topdown(); }
};
fptr bottomup_ftpr = new fptr() {
public Object rule() throws RecognitionException { return bottomup(); }
};
// methods the downup strategy uses to do the up and down rules.
// to override, just define tree grammar rule topdown and turn on
// filter=true.
public Object topdown() throws RecognitionException { return null; }
public Object bottomup() throws RecognitionException { return null; }
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/CommonErrorNode.java 0000644 0001750 0001750 00000007362 11256465214 027716 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.*;
/** A node representing erroneous token range in token stream */
public class CommonErrorNode extends CommonTree {
public IntStream input;
public Token start;
public Token stop;
public RecognitionException trappedException;
public CommonErrorNode(TokenStream input, Token start, Token stop,
RecognitionException e)
{
//System.out.println("start: "+start+", stop: "+stop);
if ( stop==null ||
(stop.getTokenIndex() < start.getTokenIndex() &&
stop.getType()!=Token.EOF) )
{
// sometimes resync does not consume a token (when LT(1) is
// in follow set. So, stop will be 1 to left to start. adjust.
// Also handle case where start is the first token and no token
// is consumed during recovery; LT(-1) will return null.
stop = start;
}
this.input = input;
this.start = start;
this.stop = stop;
this.trappedException = e;
}
public boolean isNil() {
return false;
}
public int getType() {
return Token.INVALID_TOKEN_TYPE;
}
public String getText() {
String badText = null;
if ( start instanceof Token ) {
int i = ((Token)start).getTokenIndex();
int j = ((Token)stop).getTokenIndex();
if ( ((Token)stop).getType() == Token.EOF ) {
j = ((TokenStream)input).size();
}
badText = ((TokenStream)input).toString(i, j);
}
else if ( start instanceof Tree ) {
badText = ((TreeNodeStream)input).toString(start, stop);
}
else {
// people should subclass if they alter the tree type so this
// next one is for sure correct.
badText = "";
}
return badText;
}
public String toString() {
if ( trappedException instanceof MissingTokenException ) {
return "";
}
else if ( trappedException instanceof UnwantedTokenException ) {
return "";
}
else if ( trappedException instanceof MismatchedTokenException ) {
return "";
}
else if ( trappedException instanceof NoViableAltException ) {
return "";
}
return "";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/BufferedTreeNodeStream.java 0000644 0001750 0001750 00000027541 11256465214 031173 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import org.antlr.runtime.TokenStream;
import org.antlr.runtime.misc.IntArray;
import java.util.*;
/** A buffered stream of tree nodes. Nodes can be from a tree of ANY kind.
*
* This node stream sucks all nodes out of the tree specified in
* the constructor during construction and makes pointers into
* the tree using an array of Object pointers. The stream necessarily
* includes pointers to DOWN and UP and EOF nodes.
*
* This stream knows how to mark/release for backtracking.
*
* This stream is most suitable for tree interpreters that need to
* jump around a lot or for tree parsers requiring speed (at cost of memory).
* There is some duplicated functionality here with UnBufferedTreeNodeStream
* but just in bookkeeping, not tree walking etc...
*
* TARGET DEVELOPERS:
*
* This is the old CommonTreeNodeStream that buffered up entire node stream.
* No need to implement really as new CommonTreeNodeStream is much better
* and covers what we need.
*
* @see CommonTreeNodeStream
*/
public class BufferedTreeNodeStream implements TreeNodeStream {
public static final int DEFAULT_INITIAL_BUFFER_SIZE = 100;
public static final int INITIAL_CALL_STACK_SIZE = 10;
protected class StreamIterator implements Iterator {
int i = 0;
public boolean hasNext() {
return i=0 &&
(adaptor.getType(get(i))==Token.EOF ||
adaptor.getType(get(i))==Token.UP ||
adaptor.getType(get(i))==Token.DOWN) )
{
i--;
}
System.out.println("stop at node: "+i+" "+nodes.get(i));
return nodes.get(i);
}
*/
/** Look backwards k nodes */
protected Object LB(int k) {
if ( k==0 ) {
return null;
}
if ( (p-k)<0 ) {
return null;
}
return nodes.get(p-k);
}
public Object getTreeSource() {
return root;
}
public String getSourceName() {
return getTokenStream().getSourceName();
}
public TokenStream getTokenStream() {
return tokens;
}
public void setTokenStream(TokenStream tokens) {
this.tokens = tokens;
}
public TreeAdaptor getTreeAdaptor() {
return adaptor;
}
public void setTreeAdaptor(TreeAdaptor adaptor) {
this.adaptor = adaptor;
}
public boolean hasUniqueNavigationNodes() {
return uniqueNavigationNodes;
}
public void setUniqueNavigationNodes(boolean uniqueNavigationNodes) {
this.uniqueNavigationNodes = uniqueNavigationNodes;
}
public void consume() {
if ( p==-1 ) {
fillBuffer();
}
p++;
}
public int LA(int i) {
return adaptor.getType(LT(i));
}
public int mark() {
if ( p==-1 ) {
fillBuffer();
}
lastMarker = index();
return lastMarker;
}
public void release(int marker) {
// no resources to release
}
public int index() {
return p;
}
public void rewind(int marker) {
seek(marker);
}
public void rewind() {
seek(lastMarker);
}
public void seek(int index) {
if ( p==-1 ) {
fillBuffer();
}
p = index;
}
/** Make stream jump to a new location, saving old location.
* Switch back with pop().
*/
public void push(int index) {
if ( calls==null ) {
calls = new IntArray();
}
calls.push(p); // save current index
seek(index);
}
/** Seek back to previous index saved during last push() call.
* Return top of stack (return index).
*/
public int pop() {
int ret = calls.pop();
seek(ret);
return ret;
}
public void reset() {
p = 0;
lastMarker = 0;
if (calls != null) {
calls.clear();
}
}
public int size() {
if ( p==-1 ) {
fillBuffer();
}
return nodes.size();
}
public Iterator iterator() {
if ( p==-1 ) {
fillBuffer();
}
return new StreamIterator();
}
// TREE REWRITE INTERFACE
public void replaceChildren(Object parent, int startChildIndex, int stopChildIndex, Object t) {
if ( parent!=null ) {
adaptor.replaceChildren(parent, startChildIndex, stopChildIndex, t);
}
}
/** Used for testing, just return the token type stream */
public String toTokenTypeString() {
if ( p==-1 ) {
fillBuffer();
}
StringBuffer buf = new StringBuffer();
for (int i = 0; i < nodes.size(); i++) {
Object t = (Object) nodes.get(i);
buf.append(" ");
buf.append(adaptor.getType(t));
}
return buf.toString();
}
/** Debugging */
public String toTokenString(int start, int stop) {
if ( p==-1 ) {
fillBuffer();
}
StringBuffer buf = new StringBuffer();
for (int i = start; i < nodes.size() && i <= stop; i++) {
Object t = (Object) nodes.get(i);
buf.append(" ");
buf.append(adaptor.getToken(t));
}
return buf.toString();
}
public String toString(Object start, Object stop) {
System.out.println("toString");
if ( start==null || stop==null ) {
return null;
}
if ( p==-1 ) {
fillBuffer();
}
//System.out.println("stop: "+stop);
if ( start instanceof CommonTree )
System.out.print("toString: "+((CommonTree)start).getToken()+", ");
else
System.out.println(start);
if ( stop instanceof CommonTree )
System.out.println(((CommonTree)stop).getToken());
else
System.out.println(stop);
// if we have the token stream, use that to dump text in order
if ( tokens!=null ) {
int beginTokenIndex = adaptor.getTokenStartIndex(start);
int endTokenIndex = adaptor.getTokenStopIndex(stop);
// if it's a tree, use start/stop index from start node
// else use token range from start/stop nodes
if ( adaptor.getType(stop)==Token.UP ) {
endTokenIndex = adaptor.getTokenStopIndex(start);
}
else if ( adaptor.getType(stop)==Token.EOF ) {
endTokenIndex = size()-2; // don't use EOF
}
return tokens.toString(beginTokenIndex, endTokenIndex);
}
// walk nodes looking for start
Object t = null;
int i = 0;
for (; i < nodes.size(); i++) {
t = nodes.get(i);
if ( t==start ) {
break;
}
}
// now walk until we see stop, filling string buffer with text
StringBuffer buf = new StringBuffer();
t = nodes.get(i);
while ( t!=stop ) {
String text = adaptor.getText(t);
if ( text==null ) {
text = " "+String.valueOf(adaptor.getType(t));
}
buf.append(text);
i++;
t = nodes.get(i);
}
// include stop node too
String text = adaptor.getText(stop);
if ( text==null ) {
text = " "+String.valueOf(adaptor.getType(stop));
}
buf.append(text);
return buf.toString();
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/CommonTreeAdaptor.java 0000644 0001750 0001750 00000013545 11256465214 030231 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.CommonToken;
import org.antlr.runtime.Token;
/** A TreeAdaptor that works with any Tree implementation. It provides
* really just factory methods; all the work is done by BaseTreeAdaptor.
* If you would like to have different tokens created than ClassicToken
* objects, you need to override this and then set the parser tree adaptor to
* use your subclass.
*
* To get your parser to build nodes of a different type, override
* create(Token), errorNode(), and to be safe, YourTreeClass.dupNode().
* dupNode is called to duplicate nodes during rewrite operations.
*/
public class CommonTreeAdaptor extends BaseTreeAdaptor {
/** Duplicate a node. This is part of the factory;
* override if you want another kind of node to be built.
*
* I could use reflection to prevent having to override this
* but reflection is slow.
*/
public Object dupNode(Object t) {
if ( t==null ) return null;
return ((Tree)t).dupNode();
}
public Object create(Token payload) {
return new CommonTree(payload);
}
/** Tell me how to create a token for use with imaginary token nodes.
* For example, there is probably no input symbol associated with imaginary
* token DECL, but you need to create it as a payload or whatever for
* the DECL node as in ^(DECL type ID).
*
* If you care what the token payload objects' type is, you should
* override this method and any other createToken variant.
*/
public Token createToken(int tokenType, String text) {
return new CommonToken(tokenType, text);
}
/** Tell me how to create a token for use with imaginary token nodes.
* For example, there is probably no input symbol associated with imaginary
* token DECL, but you need to create it as a payload or whatever for
* the DECL node as in ^(DECL type ID).
*
* This is a variant of createToken where the new token is derived from
* an actual real input token. Typically this is for converting '{'
* tokens to BLOCK etc... You'll see
*
* r : lc='{' ID+ '}' -> ^(BLOCK[$lc] ID+) ;
*
* If you care what the token payload objects' type is, you should
* override this method and any other createToken variant.
*/
public Token createToken(Token fromToken) {
return new CommonToken(fromToken);
}
/** Track start/stop token for subtree root created for a rule.
* Only works with Tree nodes. For rules that match nothing,
* seems like this will yield start=i and stop=i-1 in a nil node.
* Might be useful info so I'll not force to be i..i.
*/
public void setTokenBoundaries(Object t, Token startToken, Token stopToken) {
if ( t==null ) return;
int start = 0;
int stop = 0;
if ( startToken!=null ) start = startToken.getTokenIndex();
if ( stopToken!=null ) stop = stopToken.getTokenIndex();
((Tree)t).setTokenStartIndex(start);
((Tree)t).setTokenStopIndex(stop);
}
public int getTokenStartIndex(Object t) {
if ( t==null ) return -1;
return ((Tree)t).getTokenStartIndex();
}
public int getTokenStopIndex(Object t) {
if ( t==null ) return -1;
return ((Tree)t).getTokenStopIndex();
}
public String getText(Object t) {
if ( t==null ) return null;
return ((Tree)t).getText();
}
public int getType(Object t) {
if ( t==null ) return Token.INVALID_TOKEN_TYPE;
return ((Tree)t).getType();
}
/** What is the Token associated with this node? If
* you are not using CommonTree, then you must
* override this in your own adaptor.
*/
public Token getToken(Object t) {
if ( t instanceof CommonTree ) {
return ((CommonTree)t).getToken();
}
return null; // no idea what to do
}
public Object getChild(Object t, int i) {
if ( t==null ) return null;
return ((Tree)t).getChild(i);
}
public int getChildCount(Object t) {
if ( t==null ) return 0;
return ((Tree)t).getChildCount();
}
public Object getParent(Object t) {
if ( t==null ) return null;
return ((Tree)t).getParent();
}
public void setParent(Object t, Object parent) {
if ( t!=null ) ((Tree)t).setParent((Tree)parent);
}
public int getChildIndex(Object t) {
if ( t==null ) return 0;
return ((Tree)t).getChildIndex();
}
public void setChildIndex(Object t, int index) {
if ( t!=null ) ((Tree)t).setChildIndex(index);
}
public void replaceChildren(Object parent, int startChildIndex, int stopChildIndex, Object t) {
if ( parent!=null ) {
((Tree)parent).replaceChildren(startChildIndex, stopChildIndex, t);
}
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/CommonTree.java 0000644 0001750 0001750 00000012045 11256465214 026710 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
/** A tree node that is wrapper for a Token object. After 3.0 release
* while building tree rewrite stuff, it became clear that computing
* parent and child index is very difficult and cumbersome. Better to
* spend the space in every tree node. If you don't want these extra
* fields, it's easy to cut them out in your own BaseTree subclass.
*/
public class CommonTree extends BaseTree {
/** A single token is the payload */
public Token token;
/** What token indexes bracket all tokens associated with this node
* and below?
*/
protected int startIndex=-1, stopIndex=-1;
/** Who is the parent node of this node; if null, implies node is root */
public CommonTree parent;
/** What index is this node in the child list? Range: 0..n-1 */
public int childIndex = -1;
public CommonTree() { }
public CommonTree(CommonTree node) {
super(node);
this.token = node.token;
this.startIndex = node.startIndex;
this.stopIndex = node.stopIndex;
}
public CommonTree(Token t) {
this.token = t;
}
public Token getToken() {
return token;
}
public Tree dupNode() {
return new CommonTree(this);
}
public boolean isNil() {
return token==null;
}
public int getType() {
if ( token==null ) {
return Token.INVALID_TOKEN_TYPE;
}
return token.getType();
}
public String getText() {
if ( token==null ) {
return null;
}
return token.getText();
}
public int getLine() {
if ( token==null || token.getLine()==0 ) {
if ( getChildCount()>0 ) {
return getChild(0).getLine();
}
return 0;
}
return token.getLine();
}
public int getCharPositionInLine() {
if ( token==null || token.getCharPositionInLine()==-1 ) {
if ( getChildCount()>0 ) {
return getChild(0).getCharPositionInLine();
}
return 0;
}
return token.getCharPositionInLine();
}
public int getTokenStartIndex() {
if ( startIndex==-1 && token!=null ) {
return token.getTokenIndex();
}
return startIndex;
}
public void setTokenStartIndex(int index) {
startIndex = index;
}
public int getTokenStopIndex() {
if ( stopIndex==-1 && token!=null ) {
return token.getTokenIndex();
}
return stopIndex;
}
public void setTokenStopIndex(int index) {
stopIndex = index;
}
/** For every node in this subtree, make sure it's start/stop token's
* are set. Walk depth first, visit bottom up. Only updates nodes
* with at least one token index < 0.
*/
public void setUnknownTokenBoundaries() {
if ( children==null ) {
if ( startIndex<0 || stopIndex<0 ) {
startIndex = stopIndex = token.getTokenIndex();
}
return;
}
for (int i=0; i=0 && stopIndex>=0 ) return; // already set
if ( children.size() > 0 ) {
CommonTree firstChild = (CommonTree)children.get(0);
CommonTree lastChild = (CommonTree)children.get(children.size()-1);
startIndex = firstChild.getTokenStartIndex();
stopIndex = lastChild.getTokenStopIndex();
}
}
public int getChildIndex() {
return childIndex;
}
public Tree getParent() {
return parent;
}
public void setParent(Tree t) {
this.parent = (CommonTree)t;
}
public void setChildIndex(int index) {
this.childIndex = index;
}
public String toString() {
if ( isNil() ) {
return "nil";
}
if ( getType()==Token.INVALID_TOKEN_TYPE ) {
return "";
}
if ( token==null ) {
return null;
}
return token.getText();
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeVisitor.java 0000644 0001750 0001750 00000003134 11256465214 027116 0 ustar twerner twerner package org.antlr.runtime.tree;
/** Do a depth first walk of a tree, applying pre() and post() actions
* as we discover and finish nodes.
*/
public class TreeVisitor {
protected TreeAdaptor adaptor;
public TreeVisitor(TreeAdaptor adaptor) {
this.adaptor = adaptor;
}
public TreeVisitor() { this(new CommonTreeAdaptor()); }
/** Visit every node in tree t and trigger an action for each node
* before/after having visited all of its children.
* Execute both actions even if t has no children.
* If a child visit yields a new child, it can update its
* parent's child list or just return the new child. The
* child update code works even if the child visit alters its parent
* and returns the new tree.
*
* Return result of applying post action to this node.
*/
public Object visit(Object t, TreeVisitorAction action) {
// System.out.println("visit "+((Tree)t).toStringTree());
boolean isNil = adaptor.isNil(t);
if ( action!=null && !isNil ) {
t = action.pre(t); // if rewritten, walk children of new t
}
int n = adaptor.getChildCount(t);
for (int i=0; i";
}
return t.getText();
}
return payload.toString();
}
/** Emit a token and all hidden nodes before. EOF node holds all
* hidden tokens after last real token.
*/
public String toStringWithHiddenTokens() {
StringBuffer buf = new StringBuffer();
if ( hiddenTokens!=null ) {
for (int i = 0; i < hiddenTokens.size(); i++) {
Token hidden = (Token) hiddenTokens.get(i);
buf.append(hidden.getText());
}
}
String nodeText = this.toString();
if ( !nodeText.equals("") ) buf.append(nodeText);
return buf.toString();
}
/** Print out the leaves of this tree, which means printing original
* input back out.
*/
public String toInputString() {
StringBuffer buf = new StringBuffer();
_toStringLeaves(buf);
return buf.toString();
}
public void _toStringLeaves(StringBuffer buf) {
if ( payload instanceof Token ) { // leaf node token?
buf.append(this.toStringWithHiddenTokens());
return;
}
for (int i = 0; children!=null && i < children.size(); i++) {
ParseTree t = (ParseTree)children.get(i);
t._toStringLeaves(buf);
}
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/RewriteRuleTokenStream.java 0000644 0001750 0001750 00000005170 11256465214 031267 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import java.util.List;
public class RewriteRuleTokenStream extends RewriteRuleElementStream {
public RewriteRuleTokenStream(TreeAdaptor adaptor, String elementDescription) {
super(adaptor, elementDescription);
}
/** Create a stream with one element */
public RewriteRuleTokenStream(TreeAdaptor adaptor,
String elementDescription,
Object oneElement)
{
super(adaptor, elementDescription, oneElement);
}
/** Create a stream, but feed off an existing list */
public RewriteRuleTokenStream(TreeAdaptor adaptor,
String elementDescription,
List elements)
{
super(adaptor, elementDescription, elements);
}
/** Get next token from stream and make a node for it */
public Object nextNode() {
Token t = (Token)_next();
return adaptor.create(t);
}
public Token nextToken() {
return (Token)_next();
}
/** Don't convert to a tree unless they explicitly call nextTree.
* This way we can do hetero tree nodes in rewrite.
*/
protected Object toTree(Object el) {
return el;
}
protected Object dup(Object el) {
throw new UnsupportedOperationException("dup can't be called for a token stream.");
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreePatternParser.java 0000644 0001750 0001750 00000011033 11256465214 030246 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import org.antlr.runtime.CommonToken;
public class TreePatternParser {
protected TreePatternLexer tokenizer;
protected int ttype;
protected TreeWizard wizard;
protected TreeAdaptor adaptor;
public TreePatternParser(TreePatternLexer tokenizer, TreeWizard wizard, TreeAdaptor adaptor) {
this.tokenizer = tokenizer;
this.wizard = wizard;
this.adaptor = adaptor;
ttype = tokenizer.nextToken(); // kickstart
}
public Object pattern() {
if ( ttype==TreePatternLexer.BEGIN ) {
return parseTree();
}
else if ( ttype==TreePatternLexer.ID ) {
Object node = parseNode();
if ( ttype==TreePatternLexer.EOF ) {
return node;
}
return null; // extra junk on end
}
return null;
}
public Object parseTree() {
if ( ttype != TreePatternLexer.BEGIN ) {
System.out.println("no BEGIN");
return null;
}
ttype = tokenizer.nextToken();
Object root = parseNode();
if ( root==null ) {
return null;
}
while ( ttype==TreePatternLexer.BEGIN ||
ttype==TreePatternLexer.ID ||
ttype==TreePatternLexer.PERCENT ||
ttype==TreePatternLexer.DOT )
{
if ( ttype==TreePatternLexer.BEGIN ) {
Object subtree = parseTree();
adaptor.addChild(root, subtree);
}
else {
Object child = parseNode();
if ( child==null ) {
return null;
}
adaptor.addChild(root, child);
}
}
if ( ttype != TreePatternLexer.END ) {
System.out.println("no END");
return null;
}
ttype = tokenizer.nextToken();
return root;
}
public Object parseNode() {
// "%label:" prefix
String label = null;
if ( ttype == TreePatternLexer.PERCENT ) {
ttype = tokenizer.nextToken();
if ( ttype != TreePatternLexer.ID ) {
return null;
}
label = tokenizer.sval.toString();
ttype = tokenizer.nextToken();
if ( ttype != TreePatternLexer.COLON ) {
return null;
}
ttype = tokenizer.nextToken(); // move to ID following colon
}
// Wildcard?
if ( ttype == TreePatternLexer.DOT ) {
ttype = tokenizer.nextToken();
Token wildcardPayload = new CommonToken(0, ".");
TreeWizard.TreePattern node =
new TreeWizard.WildcardTreePattern(wildcardPayload);
if ( label!=null ) {
node.label = label;
}
return node;
}
// "ID" or "ID[arg]"
if ( ttype != TreePatternLexer.ID ) {
return null;
}
String tokenName = tokenizer.sval.toString();
ttype = tokenizer.nextToken();
if ( tokenName.equals("nil") ) {
return adaptor.nil();
}
String text = tokenName;
// check for arg
String arg = null;
if ( ttype == TreePatternLexer.ARG ) {
arg = tokenizer.sval.toString();
text = arg;
ttype = tokenizer.nextToken();
}
// create node
int treeNodeType = wizard.getTokenType(tokenName);
if ( treeNodeType==Token.INVALID_TOKEN_TYPE ) {
return null;
}
Object node;
node = adaptor.create(treeNodeType, text);
if ( label!=null && node.getClass()==TreeWizard.TreePattern.class ) {
((TreeWizard.TreePattern)node).label = label;
}
if ( arg!=null && node.getClass()==TreeWizard.TreePattern.class ) {
((TreeWizard.TreePattern)node).hasTextArg = true;
}
return node;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/RewriteRuleSubtreeStream.java 0000644 0001750 0001750 00000006203 11256465214 031616 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import java.util.List;
public class RewriteRuleSubtreeStream extends RewriteRuleElementStream {
public RewriteRuleSubtreeStream(TreeAdaptor adaptor, String elementDescription) {
super(adaptor, elementDescription);
}
/** Create a stream with one element */
public RewriteRuleSubtreeStream(TreeAdaptor adaptor,
String elementDescription,
Object oneElement)
{
super(adaptor, elementDescription, oneElement);
}
/** Create a stream, but feed off an existing list */
public RewriteRuleSubtreeStream(TreeAdaptor adaptor,
String elementDescription,
List elements)
{
super(adaptor, elementDescription, elements);
}
/** Treat next element as a single node even if it's a subtree.
* This is used instead of next() when the result has to be a
* tree root node. Also prevents us from duplicating recently-added
* children; e.g., ^(type ID)+ adds ID to type and then 2nd iteration
* must dup the type node, but ID has been added.
*
* Referencing a rule result twice is ok; dup entire tree as
* we can't be adding trees as root; e.g., expr expr.
*
* Hideous code duplication here with super.next(). Can't think of
* a proper way to refactor. This needs to always call dup node
* and super.next() doesn't know which to call: dup node or dup tree.
*/
public Object nextNode() {
int n = size();
if ( dirty || (cursor>=n && n==1) ) {
// if out of elements and size is 1, dup (at most a single node
// since this is for making root nodes).
Object el = _next();
return adaptor.dupNode(el);
}
// test size above then fetch
Object el = _next();
return el;
}
protected Object dup(Object el) {
return adaptor.dupTree(el);
}
} antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/RewriteRuleElementStream.java 0000644 0001750 0001750 00000015616 11256465214 031606 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import org.antlr.runtime.CommonToken;
import java.util.List;
import java.util.ArrayList;
/** A generic list of elements tracked in an alternative to be used in
* a -> rewrite rule. We need to subclass to fill in the next() method,
* which returns either an AST node wrapped around a token payload or
* an existing subtree.
*
* Once you start next()ing, do not try to add more elements. It will
* break the cursor tracking I believe.
*
* @see org.antlr.runtime.tree.RewriteRuleSubtreeStream
* @see org.antlr.runtime.tree.RewriteRuleTokenStream
*
* TODO: add mechanism to detect/puke on modification after reading from stream
*/
public abstract class RewriteRuleElementStream {
/** Cursor 0..n-1. If singleElement!=null, cursor is 0 until you next(),
* which bumps it to 1 meaning no more elements.
*/
protected int cursor = 0;
/** Track single elements w/o creating a list. Upon 2nd add, alloc list */
protected Object singleElement;
/** The list of tokens or subtrees we are tracking */
protected List elements;
/** Once a node / subtree has been used in a stream, it must be dup'd
* from then on. Streams are reset after subrules so that the streams
* can be reused in future subrules. So, reset must set a dirty bit.
* If dirty, then next() always returns a dup.
*
* I wanted to use "naughty bit" here, but couldn't think of a way
* to use "naughty".
*/
protected boolean dirty = false;
/** The element or stream description; usually has name of the token or
* rule reference that this list tracks. Can include rulename too, but
* the exception would track that info.
*/
protected String elementDescription;
protected TreeAdaptor adaptor;
public RewriteRuleElementStream(TreeAdaptor adaptor, String elementDescription) {
this.elementDescription = elementDescription;
this.adaptor = adaptor;
}
/** Create a stream with one element */
public RewriteRuleElementStream(TreeAdaptor adaptor,
String elementDescription,
Object oneElement)
{
this(adaptor, elementDescription);
add(oneElement);
}
/** Create a stream, but feed off an existing list */
public RewriteRuleElementStream(TreeAdaptor adaptor,
String elementDescription,
List elements)
{
this(adaptor, elementDescription);
this.singleElement = null;
this.elements = elements;
}
/** Reset the condition of this stream so that it appears we have
* not consumed any of its elements. Elements themselves are untouched.
* Once we reset the stream, any future use will need duplicates. Set
* the dirty bit.
*/
public void reset() {
cursor = 0;
dirty = true;
}
public void add(Object el) {
//System.out.println("add '"+elementDescription+"' is "+el);
if ( el==null ) {
return;
}
if ( elements!=null ) { // if in list, just add
elements.add(el);
return;
}
if ( singleElement == null ) { // no elements yet, track w/o list
singleElement = el;
return;
}
// adding 2nd element, move to list
elements = new ArrayList(5);
elements.add(singleElement);
singleElement = null;
elements.add(el);
}
/** Return the next element in the stream. If out of elements, throw
* an exception unless size()==1. If size is 1, then return elements[0].
* Return a duplicate node/subtree if stream is out of elements and
* size==1. If we've already used the element, dup (dirty bit set).
*/
public Object nextTree() {
int n = size();
if ( dirty || (cursor>=n && n==1) ) {
// if out of elements and size is 1, dup
Object el = _next();
return dup(el);
}
// test size above then fetch
Object el = _next();
return el;
}
/** do the work of getting the next element, making sure that it's
* a tree node or subtree. Deal with the optimization of single-
* element list versus list of size > 1. Throw an exception
* if the stream is empty or we're out of elements and size>1.
* protected so you can override in a subclass if necessary.
*/
protected Object _next() {
int n = size();
if ( n ==0 ) {
throw new RewriteEmptyStreamException(elementDescription);
}
if ( cursor>= n) { // out of elements?
if ( n ==1 ) { // if size is 1, it's ok; return and we'll dup
return toTree(singleElement);
}
// out of elements and size was not 1, so we can't dup
throw new RewriteCardinalityException(elementDescription);
}
// we have elements
if ( singleElement!=null ) {
cursor++; // move cursor even for single element list
return toTree(singleElement);
}
// must have more than one in list, pull from elements
Object o = toTree(elements.get(cursor));
cursor++;
return o;
}
/** When constructing trees, sometimes we need to dup a token or AST
* subtree. Dup'ing a token means just creating another AST node
* around it. For trees, you must call the adaptor.dupTree() unless
* the element is for a tree root; then it must be a node dup.
*/
protected abstract Object dup(Object el);
/** Ensure stream emits trees; tokens must be converted to AST nodes.
* AST nodes can be passed through unmolested.
*/
protected Object toTree(Object el) {
return el;
}
public boolean hasNext() {
return (singleElement != null && cursor < 1) ||
(elements!=null && cursor < elements.size());
}
public int size() {
int n = 0;
if ( singleElement != null ) {
n = 1;
}
if ( elements!=null ) {
return elements.size();
}
return n;
}
public String getDescription() {
return elementDescription;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/CommonTreeNodeStream.java 0000644 0001750 0001750 00000012776 11256465214 030705 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import org.antlr.runtime.TokenStream;
import org.antlr.runtime.misc.LookaheadStream;
import org.antlr.runtime.misc.IntArray;
import java.util.*;
public class CommonTreeNodeStream extends LookaheadStream implements TreeNodeStream {
public static final int DEFAULT_INITIAL_BUFFER_SIZE = 100;
public static final int INITIAL_CALL_STACK_SIZE = 10;
/** Pull nodes from which tree? */
protected Object root;
/** If this tree (root) was created from a token stream, track it. */
protected TokenStream tokens;
/** What tree adaptor was used to build these trees */
TreeAdaptor adaptor;
/** The tree iterator we using */
protected TreeIterator it;
/** Stack of indexes used for push/pop calls */
protected IntArray calls;
/** Tree (nil A B C) trees like flat A B C streams */
protected boolean hasNilRoot = false;
/** Tracks tree depth. Level=0 means we're at root node level. */
protected int level = 0;
public CommonTreeNodeStream(Object tree) {
this(new CommonTreeAdaptor(), tree);
}
public CommonTreeNodeStream(TreeAdaptor adaptor, Object tree) {
super(adaptor.create(Token.EOF, "EOF")); // set EOF
this.root = tree;
this.adaptor = adaptor;
it = new TreeIterator(root);
it.eof = this.eof; // make sure tree iterator returns the EOF we want
}
public void reset() {
super.reset();
it.reset();
hasNilRoot = false;
level = 0;
if ( calls != null ) calls.clear();
}
/** Pull elements from tree iterator. Track tree level 0..max_level.
* If nil rooted tree, don't give initial nil and DOWN nor final UP.
*/
public Object nextElement() {
Object t = it.next();
//System.out.println("pulled "+adaptor.getType(t));
if ( t == it.up ) {
level--;
if ( level==0 && hasNilRoot ) return it.next(); // don't give last UP; get EOF
}
else if ( t == it.down ) level++;
if ( level==0 && adaptor.isNil(t) ) { // if nil root, scarf nil, DOWN
hasNilRoot = true;
t = it.next(); // t is now DOWN, so get first real node next
level++;
t = it.next();
}
return t;
}
public void setUniqueNavigationNodes(boolean uniqueNavigationNodes) { }
public Object getTreeSource() { return root; }
public String getSourceName() { return getTokenStream().getSourceName(); }
public TokenStream getTokenStream() { return tokens; }
public void setTokenStream(TokenStream tokens) { this.tokens = tokens; }
public TreeAdaptor getTreeAdaptor() { return adaptor; }
public void setTreeAdaptor(TreeAdaptor adaptor) { this.adaptor = adaptor; }
public int LA(int i) { return adaptor.getType(LT(i)); }
/** Make stream jump to a new location, saving old location.
* Switch back with pop().
*/
public void push(int index) {
if ( calls==null ) {
calls = new IntArray();
}
calls.push(p); // save current index
seek(index);
}
/** Seek back to previous index saved during last push() call.
* Return top of stack (return index).
*/
public int pop() {
int ret = calls.pop();
seek(ret);
return ret;
}
// TREE REWRITE INTERFACE
public void replaceChildren(Object parent, int startChildIndex, int stopChildIndex, Object t) {
if ( parent!=null ) {
adaptor.replaceChildren(parent, startChildIndex, stopChildIndex, t);
}
}
public String toString(Object start, Object stop) {
// we'll have to walk from start to stop in tree; we're not keeping
// a complete node stream buffer
return "n/a";
}
/** For debugging; destructive: moves tree iterator to end. */
public String toTokenTypeString() {
reset();
StringBuffer buf = new StringBuffer();
Object o = LT(1);
int type = adaptor.getType(o);
while ( type!=Token.EOF ) {
buf.append(" ");
buf.append(type);
consume();
o = LT(1);
type = adaptor.getType(o);
}
return buf.toString();
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreePatternLexer.java 0000644 0001750 0001750 00000006725 11256465214 030105 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
public class TreePatternLexer {
public static final int EOF = -1;
public static final int BEGIN = 1;
public static final int END = 2;
public static final int ID = 3;
public static final int ARG = 4;
public static final int PERCENT = 5;
public static final int COLON = 6;
public static final int DOT = 7;
/** The tree pattern to lex like "(A B C)" */
protected String pattern;
/** Index into input string */
protected int p = -1;
/** Current char */
protected int c;
/** How long is the pattern in char? */
protected int n;
/** Set when token type is ID or ARG (name mimics Java's StreamTokenizer) */
public StringBuffer sval = new StringBuffer();
public boolean error = false;
public TreePatternLexer(String pattern) {
this.pattern = pattern;
this.n = pattern.length();
consume();
}
public int nextToken() {
sval.setLength(0); // reset, but reuse buffer
while ( c != EOF ) {
if ( c==' ' || c=='\n' || c=='\r' || c=='\t' ) {
consume();
continue;
}
if ( (c>='a' && c<='z') || (c>='A' && c<='Z') || c=='_' ) {
sval.append((char)c);
consume();
while ( (c>='a' && c<='z') || (c>='A' && c<='Z') ||
(c>='0' && c<='9') || c=='_' )
{
sval.append((char)c);
consume();
}
return ID;
}
if ( c=='(' ) {
consume();
return BEGIN;
}
if ( c==')' ) {
consume();
return END;
}
if ( c=='%' ) {
consume();
return PERCENT;
}
if ( c==':' ) {
consume();
return COLON;
}
if ( c=='.' ) {
consume();
return DOT;
}
if ( c=='[' ) { // grab [x] as a string, returning x
consume();
while ( c!=']' ) {
if ( c=='\\' ) {
consume();
if ( c!=']' ) {
sval.append('\\');
}
sval.append((char)c);
}
else {
sval.append((char)c);
}
consume();
}
consume();
return ARG;
}
consume();
error = true;
return EOF;
}
return EOF;
}
protected void consume() {
p++;
if ( p>=n ) {
c = EOF;
}
else {
c = pattern.charAt(p);
}
}
} antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeRuleReturnScope.java 0000644 0001750 0001750 00000003605 11256465214 030563 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.RuleReturnScope;
/** This is identical to the ParserRuleReturnScope except that
* the start property is a tree nodes not Token object
* when you are parsing trees. To be generic the tree node types
* have to be Object.
*/
public class TreeRuleReturnScope extends RuleReturnScope {
/** First node or root node of tree matched for this rule. */
public Object start;
public Object getStart() { return start; }
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeAdaptor.java 0000644 0001750 0001750 00000024365 11256465214 027062 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import org.antlr.runtime.TokenStream;
import org.antlr.runtime.RecognitionException;
/** How to create and navigate trees. Rather than have a separate factory
* and adaptor, I've merged them. Makes sense to encapsulate.
*
* This takes the place of the tree construction code generated in the
* generated code in 2.x and the ASTFactory.
*
* I do not need to know the type of a tree at all so they are all
* generic Objects. This may increase the amount of typecasting needed. :(
*/
public interface TreeAdaptor {
// C o n s t r u c t i o n
/** Create a tree node from Token object; for CommonTree type trees,
* then the token just becomes the payload. This is the most
* common create call.
*
* Override if you want another kind of node to be built.
*/
public Object create(Token payload);
/** Duplicate a single tree node.
* Override if you want another kind of node to be built.
*/
public Object dupNode(Object treeNode);
/** Duplicate tree recursively, using dupNode() for each node */
public Object dupTree(Object tree);
/** Return a nil node (an empty but non-null node) that can hold
* a list of element as the children. If you want a flat tree (a list)
* use "t=adaptor.nil(); t.addChild(x); t.addChild(y);"
*/
public Object nil();
/** Return a tree node representing an error. This node records the
* tokens consumed during error recovery. The start token indicates the
* input symbol at which the error was detected. The stop token indicates
* the last symbol consumed during recovery.
*
* You must specify the input stream so that the erroneous text can
* be packaged up in the error node. The exception could be useful
* to some applications; default implementation stores ptr to it in
* the CommonErrorNode.
*
* This only makes sense during token parsing, not tree parsing.
* Tree parsing should happen only when parsing and tree construction
* succeed.
*/
public Object errorNode(TokenStream input, Token start, Token stop, RecognitionException e);
/** Is tree considered a nil node used to make lists of child nodes? */
public boolean isNil(Object tree);
/** Add a child to the tree t. If child is a flat tree (a list), make all
* in list children of t. Warning: if t has no children, but child does
* and child isNil then you can decide it is ok to move children to t via
* t.children = child.children; i.e., without copying the array. Just
* make sure that this is consistent with have the user will build
* ASTs. Do nothing if t or child is null.
*/
public void addChild(Object t, Object child);
/** If oldRoot is a nil root, just copy or move the children to newRoot.
* If not a nil root, make oldRoot a child of newRoot.
*
* old=^(nil a b c), new=r yields ^(r a b c)
* old=^(a b c), new=r yields ^(r ^(a b c))
*
* If newRoot is a nil-rooted single child tree, use the single
* child as the new root node.
*
* old=^(nil a b c), new=^(nil r) yields ^(r a b c)
* old=^(a b c), new=^(nil r) yields ^(r ^(a b c))
*
* If oldRoot was null, it's ok, just return newRoot (even if isNil).
*
* old=null, new=r yields r
* old=null, new=^(nil r) yields ^(nil r)
*
* Return newRoot. Throw an exception if newRoot is not a
* simple node or nil root with a single child node--it must be a root
* node. If newRoot is ^(nil x) return x as newRoot.
*
* Be advised that it's ok for newRoot to point at oldRoot's
* children; i.e., you don't have to copy the list. We are
* constructing these nodes so we should have this control for
* efficiency.
*/
public Object becomeRoot(Object newRoot, Object oldRoot);
/** Given the root of the subtree created for this rule, post process
* it to do any simplifications or whatever you want. A required
* behavior is to convert ^(nil singleSubtree) to singleSubtree
* as the setting of start/stop indexes relies on a single non-nil root
* for non-flat trees.
*
* Flat trees such as for lists like "idlist : ID+ ;" are left alone
* unless there is only one ID. For a list, the start/stop indexes
* are set in the nil node.
*
* This method is executed after all rule tree construction and right
* before setTokenBoundaries().
*/
public Object rulePostProcessing(Object root);
/** For identifying trees.
*
* How to identify nodes so we can say "add node to a prior node"?
* Even becomeRoot is an issue. Use System.identityHashCode(node)
* usually.
*/
public int getUniqueID(Object node);
// R e w r i t e R u l e s
/** Create a node for newRoot make it the root of oldRoot.
* If oldRoot is a nil root, just copy or move the children to newRoot.
* If not a nil root, make oldRoot a child of newRoot.
*
* Return node created for newRoot.
*
* Be advised: when debugging ASTs, the DebugTreeAdaptor manually
* calls create(Token child) and then plain becomeRoot(node, node)
* because it needs to trap calls to create, but it can't since it delegates
* to not inherits from the TreeAdaptor.
*/
public Object becomeRoot(Token newRoot, Object oldRoot);
/** Create a new node derived from a token, with a new token type.
* This is invoked from an imaginary node ref on right side of a
* rewrite rule as IMAG[$tokenLabel].
*
* This should invoke createToken(Token).
*/
public Object create(int tokenType, Token fromToken);
/** Same as create(tokenType,fromToken) except set the text too.
* This is invoked from an imaginary node ref on right side of a
* rewrite rule as IMAG[$tokenLabel, "IMAG"].
*
* This should invoke createToken(Token).
*/
public Object create(int tokenType, Token fromToken, String text);
/** Create a new node derived from a token, with a new token type.
* This is invoked from an imaginary node ref on right side of a
* rewrite rule as IMAG["IMAG"].
*
* This should invoke createToken(int,String).
*/
public Object create(int tokenType, String text);
// C o n t e n t
/** For tree parsing, I need to know the token type of a node */
public int getType(Object t);
/** Node constructors can set the type of a node */
public void setType(Object t, int type);
public String getText(Object t);
/** Node constructors can set the text of a node */
public void setText(Object t, String text);
/** Return the token object from which this node was created.
* Currently used only for printing an error message.
* The error display routine in BaseRecognizer needs to
* display where the input the error occurred. If your
* tree of limitation does not store information that can
* lead you to the token, you can create a token filled with
* the appropriate information and pass that back. See
* BaseRecognizer.getErrorMessage().
*/
public Token getToken(Object t);
/** Where are the bounds in the input token stream for this node and
* all children? Each rule that creates AST nodes will call this
* method right before returning. Flat trees (i.e., lists) will
* still usually have a nil root node just to hold the children list.
* That node would contain the start/stop indexes then.
*/
public void setTokenBoundaries(Object t, Token startToken, Token stopToken);
/** Get the token start index for this subtree; return -1 if no such index */
public int getTokenStartIndex(Object t);
/** Get the token stop index for this subtree; return -1 if no such index */
public int getTokenStopIndex(Object t);
// N a v i g a t i o n / T r e e P a r s i n g
/** Get a child 0..n-1 node */
public Object getChild(Object t, int i);
/** Set ith child (0..n-1) to t; t must be non-null and non-nil node */
public void setChild(Object t, int i, Object child);
/** Remove ith child and shift children down from right. */
public Object deleteChild(Object t, int i);
/** How many children? If 0, then this is a leaf node */
public int getChildCount(Object t);
/** Who is the parent node of this node; if null, implies node is root.
* If your node type doesn't handle this, it's ok but the tree rewrites
* in tree parsers need this functionality.
*/
public Object getParent(Object t);
public void setParent(Object t, Object parent);
/** What index is this node in the child list? Range: 0..n-1
* If your node type doesn't handle this, it's ok but the tree rewrites
* in tree parsers need this functionality.
*/
public int getChildIndex(Object t);
public void setChildIndex(Object t, int index);
/** Replace from start to stop child index of parent with t, which might
* be a list. Number of children may be different
* after this call.
*
* If parent is null, don't do anything; must be at root of overall tree.
* Can't replace whatever points to the parent externally. Do nothing.
*/
public void replaceChildren(Object parent, int startChildIndex, int stopChildIndex, Object t);
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeIterator.java 0000644 0001750 0001750 00000011323 11256465214 027247 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import org.antlr.runtime.misc.FastQueue;
import java.util.Iterator;
/** Return a node stream from a doubly-linked tree whose nodes
* know what child index they are. No remove() is supported.
*
* Emit navigation nodes (DOWN, UP, and EOF) to let show tree structure.
*/
public class TreeIterator implements Iterator {
protected TreeAdaptor adaptor;
protected Object root;
protected Object tree;
protected boolean firstTime = true;
// navigation nodes to return during walk and at end
public Object up;
public Object down;
public Object eof;
/** If we emit UP/DOWN nodes, we need to spit out multiple nodes per
* next() call.
*/
protected FastQueue nodes;
public TreeIterator(Object tree) {
this(new CommonTreeAdaptor(),tree);
}
public TreeIterator(TreeAdaptor adaptor, Object tree) {
this.adaptor = adaptor;
this.tree = tree;
this.root = tree;
nodes = new FastQueue();
down = adaptor.create(Token.DOWN, "DOWN");
up = adaptor.create(Token.UP, "UP");
eof = adaptor.create(Token.EOF, "EOF");
}
public void reset() {
firstTime = true;
tree = root;
nodes.clear();
}
public boolean hasNext() {
if ( firstTime ) return root!=null;
if ( nodes!=null && nodes.size()>0 ) return true;
if ( tree==null ) return false;
if ( adaptor.getChildCount(tree)>0 ) return true;
return adaptor.getParent(tree)!=null; // back at root?
}
public Object next() {
if ( firstTime ) { // initial condition
firstTime = false;
if ( adaptor.getChildCount(tree)==0 ) { // single node tree (special)
nodes.add(eof);
return tree;
}
return tree;
}
// if any queued up, use those first
if ( nodes!=null && nodes.size()>0 ) return nodes.remove();
// no nodes left?
if ( tree==null ) return eof;
// next node will be child 0 if any children
if ( adaptor.getChildCount(tree)>0 ) {
tree = adaptor.getChild(tree, 0);
nodes.add(tree); // real node is next after DOWN
return down;
}
// if no children, look for next sibling of tree or ancestor
Object parent = adaptor.getParent(tree);
// while we're out of siblings, keep popping back up towards root
while ( parent!=null &&
adaptor.getChildIndex(tree)+1 >= adaptor.getChildCount(parent) )
{
nodes.add(up); // we're moving back up
tree = parent;
parent = adaptor.getParent(tree);
}
// no nodes left?
if ( parent==null ) {
tree = null; // back at root? nothing left then
nodes.add(eof); // add to queue, might have UP nodes in there
return nodes.remove();
}
// must have found a node with an unvisited sibling
// move to it and return it
int nextSiblingIndex = adaptor.getChildIndex(tree) + 1;
tree = adaptor.getChild(parent, nextSiblingIndex);
nodes.add(tree); // add to queue, might have UP nodes in there
return nodes.remove();
}
public void remove() { throw new UnsupportedOperationException(); }
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeVisitorAction.java 0000644 0001750 0001750 00000001336 11256465214 030256 0 ustar twerner twerner package org.antlr.runtime.tree;
/** How to execute code for node t when a visitor visits node t. Execute
* pre() before visiting children and execute post() after visiting children.
*/
public interface TreeVisitorAction {
/** Execute an action before visiting children of t. Return t or
* a rewritten t. It is up to the visitor to decide what to do
* with the return value. Children of returned value will be
* visited if using TreeVisitor.visit().
*/
public Object pre(Object t);
/** Execute an action after visiting children of t. Return t or
* a rewritten t. It is up to the visitor to decide what to do
* with the return value.
*/
public Object post(Object t);
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeWizard.java 0000644 0001750 0001750 00000043064 11256465216 026727 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.Token;
import java.util.*;
/** Build and navigate trees with this object. Must know about the names
* of tokens so you have to pass in a map or array of token names (from which
* this class can build the map). I.e., Token DECL means nothing unless the
* class can translate it to a token type.
*
* In order to create nodes and navigate, this class needs a TreeAdaptor.
*
* This class can build a token type -> node index for repeated use or for
* iterating over the various nodes with a particular type.
*
* This class works in conjunction with the TreeAdaptor rather than moving
* all this functionality into the adaptor. An adaptor helps build and
* navigate trees using methods. This class helps you do it with string
* patterns like "(A B C)". You can create a tree from that pattern or
* match subtrees against it.
*/
public class TreeWizard {
protected TreeAdaptor adaptor;
protected Map tokenNameToTypeMap;
public interface ContextVisitor {
// TODO: should this be called visit or something else?
public void visit(Object t, Object parent, int childIndex, Map labels);
}
public static abstract class Visitor implements ContextVisitor {
public void visit(Object t, Object parent, int childIndex, Map labels) {
visit(t);
}
public abstract void visit(Object t);
}
/** When using %label:TOKENNAME in a tree for parse(), we must
* track the label.
*/
public static class TreePattern extends CommonTree {
public String label;
public boolean hasTextArg;
public TreePattern(Token payload) {
super(payload);
}
public String toString() {
if ( label!=null ) {
return "%"+label+":"+super.toString();
}
else {
return super.toString();
}
}
}
public static class WildcardTreePattern extends TreePattern {
public WildcardTreePattern(Token payload) {
super(payload);
}
}
/** This adaptor creates TreePattern objects for use during scan() */
public static class TreePatternTreeAdaptor extends CommonTreeAdaptor {
public Object create(Token payload) {
return new TreePattern(payload);
}
}
// TODO: build indexes for the wizard
/** During fillBuffer(), we can make a reverse index from a set
* of token types of interest to the list of indexes into the
* node stream. This lets us convert a node pointer to a
* stream index semi-efficiently for a list of interesting
* nodes such as function definition nodes (you'll want to seek
* to their bodies for an interpreter). Also useful for doing
* dynamic searches; i.e., go find me all PLUS nodes.
protected Map tokenTypeToStreamIndexesMap;
/** If tokenTypesToReverseIndex set to INDEX_ALL then indexing
* occurs for all token types.
public static final Set INDEX_ALL = new HashSet();
/** A set of token types user would like to index for faster lookup.
* If this is INDEX_ALL, then all token types are tracked. If null,
* then none are indexed.
protected Set tokenTypesToReverseIndex = null;
*/
public TreeWizard(TreeAdaptor adaptor) {
this.adaptor = adaptor;
}
public TreeWizard(TreeAdaptor adaptor, Map tokenNameToTypeMap) {
this.adaptor = adaptor;
this.tokenNameToTypeMap = tokenNameToTypeMap;
}
public TreeWizard(TreeAdaptor adaptor, String[] tokenNames) {
this.adaptor = adaptor;
this.tokenNameToTypeMap = computeTokenTypes(tokenNames);
}
public TreeWizard(String[] tokenNames) {
this(null, tokenNames);
}
/** Compute a Map that is an inverted index of
* tokenNames (which maps int token types to names).
*/
public Map computeTokenTypes(String[] tokenNames) {
Map m = new HashMap();
if ( tokenNames==null ) {
return m;
}
for (int ttype = Token.MIN_TOKEN_TYPE; ttype < tokenNames.length; ttype++) {
String name = tokenNames[ttype];
m.put(name, new Integer(ttype));
}
return m;
}
/** Using the map of token names to token types, return the type. */
public int getTokenType(String tokenName) {
if ( tokenNameToTypeMap==null ) {
return Token.INVALID_TOKEN_TYPE;
}
Integer ttypeI = (Integer)tokenNameToTypeMap.get(tokenName);
if ( ttypeI!=null ) {
return ttypeI.intValue();
}
return Token.INVALID_TOKEN_TYPE;
}
/** Walk the entire tree and make a node name to nodes mapping.
* For now, use recursion but later nonrecursive version may be
* more efficient. Returns Map where the List is
* of your AST node type. The Integer is the token type of the node.
*
* TODO: save this index so that find and visit are faster
*/
public Map index(Object t) {
Map m = new HashMap();
_index(t, m);
return m;
}
/** Do the work for index */
protected void _index(Object t, Map m) {
if ( t==null ) {
return;
}
int ttype = adaptor.getType(t);
List elements = (List)m.get(new Integer(ttype));
if ( elements==null ) {
elements = new ArrayList();
m.put(new Integer(ttype), elements);
}
elements.add(t);
int n = adaptor.getChildCount(t);
for (int i=0; i>
*
* This data structure allows you to find all nodes with type INT in order.
*
* If you really need to find a node of type, say, FUNC quickly then perhaps
*
* Map>
*
* would be better for you. The interior maps map a tree node to
* the index so you don't have to search linearly for a specific node.
*
* If you change this method, you will likely need to change
* getNodeIndex(), which extracts information.
protected void fillReverseIndex(Object node, int streamIndex) {
//System.out.println("revIndex "+node+"@"+streamIndex);
if ( tokenTypesToReverseIndex==null ) {
return; // no indexing if this is empty (nothing of interest)
}
if ( tokenTypeToStreamIndexesMap==null ) {
tokenTypeToStreamIndexesMap = new HashMap(); // first indexing op
}
int tokenType = adaptor.getType(node);
Integer tokenTypeI = new Integer(tokenType);
if ( !(tokenTypesToReverseIndex==INDEX_ALL ||
tokenTypesToReverseIndex.contains(tokenTypeI)) )
{
return; // tokenType not of interest
}
Integer streamIndexI = new Integer(streamIndex);
ArrayList indexes = (ArrayList)tokenTypeToStreamIndexesMap.get(tokenTypeI);
if ( indexes==null ) {
indexes = new ArrayList(); // no list yet for this token type
indexes.add(streamIndexI); // not there yet, add
tokenTypeToStreamIndexesMap.put(tokenTypeI, indexes);
}
else {
if ( !indexes.contains(streamIndexI) ) {
indexes.add(streamIndexI); // not there yet, add
}
}
}
/** Track the indicated token type in the reverse index. Call this
* repeatedly for each type or use variant with Set argument to
* set all at once.
* @param tokenType
public void reverseIndex(int tokenType) {
if ( tokenTypesToReverseIndex==null ) {
tokenTypesToReverseIndex = new HashSet();
}
else if ( tokenTypesToReverseIndex==INDEX_ALL ) {
return;
}
tokenTypesToReverseIndex.add(new Integer(tokenType));
}
/** Track the indicated token types in the reverse index. Set
* to INDEX_ALL to track all token types.
public void reverseIndex(Set tokenTypes) {
tokenTypesToReverseIndex = tokenTypes;
}
/** Given a node pointer, return its index into the node stream.
* This is not its Token stream index. If there is no reverse map
* from node to stream index or the map does not contain entries
* for node's token type, a linear search of entire stream is used.
*
* Return -1 if exact node pointer not in stream.
public int getNodeIndex(Object node) {
//System.out.println("get "+node);
if ( tokenTypeToStreamIndexesMap==null ) {
return getNodeIndexLinearly(node);
}
int tokenType = adaptor.getType(node);
Integer tokenTypeI = new Integer(tokenType);
ArrayList indexes = (ArrayList)tokenTypeToStreamIndexesMap.get(tokenTypeI);
if ( indexes==null ) {
//System.out.println("found linearly; stream index = "+getNodeIndexLinearly(node));
return getNodeIndexLinearly(node);
}
for (int i = 0; i < indexes.size(); i++) {
Integer streamIndexI = (Integer)indexes.get(i);
Object n = get(streamIndexI.intValue());
if ( n==node ) {
//System.out.println("found in index; stream index = "+streamIndexI);
return streamIndexI.intValue(); // found it!
}
}
return -1;
}
*/
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/DOTTreeGenerator.java 0000644 0001750 0001750 00000016176 11256465214 027766 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.stringtemplate.StringTemplate;
import java.util.HashMap;
/** A utility class to generate DOT diagrams (graphviz) from
* arbitrary trees. You can pass in your own templates and
* can pass in any kind of tree or use Tree interface method.
* I wanted this separator so that you don't have to include
* ST just to use the org.antlr.runtime.tree.* package.
* This is a set of non-static methods so you can subclass
* to override. For example, here is an invocation:
*
* CharStream input = new ANTLRInputStream(System.in);
* TLexer lex = new TLexer(input);
* CommonTokenStream tokens = new CommonTokenStream(lex);
* TParser parser = new TParser(tokens);
* TParser.e_return r = parser.e();
* Tree t = (Tree)r.tree;
* System.out.println(t.toStringTree());
* DOTTreeGenerator gen = new DOTTreeGenerator();
* StringTemplate st = gen.toDOT(t);
* System.out.println(st);
*/
public class DOTTreeGenerator {
public static StringTemplate _treeST =
new StringTemplate(
"digraph {\n\n" +
"\tordering=out;\n" +
"\tranksep=.4;\n" +
"\tbgcolor=\"lightgrey\"; node [shape=box, fixedsize=false, fontsize=12, fontname=\"Helvetica-bold\", fontcolor=\"blue\"\n" +
"\t\twidth=.25, height=.25, color=\"black\", fillcolor=\"white\", style=\"filled, solid, bold\"];\n" +
"\tedge [arrowsize=.5, color=\"black\", style=\"bold\"]\n\n" +
" $nodes$\n" +
" $edges$\n" +
"}\n");
public static StringTemplate _nodeST =
new StringTemplate("$name$ [label=\"$text$\"];\n");
public static StringTemplate _edgeST =
new StringTemplate("$parent$ -> $child$ // \"$parentText$\" -> \"$childText$\"\n");
/** Track node to number mapping so we can get proper node name back */
HashMap nodeToNumberMap = new HashMap();
/** Track node number so we can get unique node names */
int nodeNumber = 0;
public StringTemplate toDOT(Object tree,
TreeAdaptor adaptor,
StringTemplate _treeST,
StringTemplate _edgeST)
{
StringTemplate treeST = _treeST.getInstanceOf();
nodeNumber = 0;
toDOTDefineNodes(tree, adaptor, treeST);
nodeNumber = 0;
toDOTDefineEdges(tree, adaptor, treeST);
/*
if ( adaptor.getChildCount(tree)==0 ) {
// single node, don't do edge.
treeST.setAttribute("nodes", adaptor.getText(tree));
}
*/
return treeST;
}
public StringTemplate toDOT(Object tree,
TreeAdaptor adaptor)
{
return toDOT(tree, adaptor, _treeST, _edgeST);
}
/** Generate DOT (graphviz) for a whole tree not just a node.
* For example, 3+4*5 should generate:
*
* digraph {
* node [shape=plaintext, fixedsize=true, fontsize=11, fontname="Courier",
* width=.4, height=.2];
* edge [arrowsize=.7]
* "+"->3
* "+"->"*"
* "*"->4
* "*"->5
* }
*
* Return the ST not a string in case people want to alter.
*
* Takes a Tree interface object.
*/
public StringTemplate toDOT(Tree tree) {
return toDOT(tree, new CommonTreeAdaptor());
}
protected void toDOTDefineNodes(Object tree,
TreeAdaptor adaptor,
StringTemplate treeST)
{
if ( tree==null ) {
return;
}
int n = adaptor.getChildCount(tree);
if ( n==0 ) {
// must have already dumped as child from previous
// invocation; do nothing
return;
}
// define parent node
StringTemplate parentNodeST = getNodeST(adaptor, tree);
treeST.setAttribute("nodes", parentNodeST);
// for each child, do a " [label=text]" node def
for (int i = 0; i < n; i++) {
Object child = adaptor.getChild(tree, i);
StringTemplate nodeST = getNodeST(adaptor, child);
treeST.setAttribute("nodes", nodeST);
toDOTDefineNodes(child, adaptor, treeST);
}
}
protected void toDOTDefineEdges(Object tree,
TreeAdaptor adaptor,
StringTemplate treeST)
{
if ( tree==null ) {
return;
}
int n = adaptor.getChildCount(tree);
if ( n==0 ) {
// must have already dumped as child from previous
// invocation; do nothing
return;
}
String parentName = "n"+getNodeNumber(tree);
// for each child, do a parent -> child edge using unique node names
String parentText = adaptor.getText(tree);
for (int i = 0; i < n; i++) {
Object child = adaptor.getChild(tree, i);
String childText = adaptor.getText(child);
String childName = "n"+getNodeNumber(child);
StringTemplate edgeST = _edgeST.getInstanceOf();
edgeST.setAttribute("parent", parentName);
edgeST.setAttribute("child", childName);
edgeST.setAttribute("parentText", fixString(parentText));
edgeST.setAttribute("childText", fixString(childText));
treeST.setAttribute("edges", edgeST);
toDOTDefineEdges(child, adaptor, treeST);
}
}
protected StringTemplate getNodeST(TreeAdaptor adaptor, Object t) {
String text = adaptor.getText(t);
StringTemplate nodeST = _nodeST.getInstanceOf();
String uniqueName = "n"+getNodeNumber(t);
nodeST.setAttribute("name", uniqueName);
nodeST.setAttribute("text", fixString(text));
return nodeST;
}
protected int getNodeNumber(Object t) {
Integer nI = (Integer)nodeToNumberMap.get(t);
if ( nI!=null ) {
return nI.intValue();
}
else {
nodeToNumberMap.put(t, new Integer(nodeNumber));
nodeNumber++;
return nodeNumber-1;
}
}
protected String fixString(String in)
{
String text = in;
if (text!=null) {
text = text.replaceAll("\"", "\\\\\"");
text = text.replaceAll("\\t", " ");
text = text.replaceAll("\\n", "\\\\n");
text = text.replaceAll("\\r", "\\\\r");
if (text.length() > 20) {
text = text.substring(0, 8) + "..." + text.substring(text.length()-8);
}
}
return text;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/tree/TreeFilter.java 0000644 0001750 0001750 00000013471 11256465214 026711 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime.tree;
import org.antlr.runtime.RecognizerSharedState;
import org.antlr.runtime.RecognitionException;
import org.antlr.runtime.TokenStream;
/**
Cut-n-paste from material I'm not using in the book anymore (edit later
to make sense):
Now, how are we going to test these tree patterns against every
subtree in our original tree? In what order should we visit nodes?
For this application, it turns out we need a simple ``apply once''
rule application strategy and a ``down then up'' tree traversal
strategy. Let's look at rule application first.
As we visit each node, we need to see if any of our patterns match. If
a pattern matches, we execute the associated tree rewrite and move on
to the next node. In other words, we only look for a single rule
application opportunity (we'll see below that we sometimes need to
repeatedly apply rules). The following method applies a rule in a @cl
TreeParser (derived from a tree grammar) to a tree:
here is where weReferenced code/walking/patterns/TreePatternMatcher.java
It uses reflection to lookup the appropriate rule within the generated
tree parser class (@cl Simplify in this case). Most of the time, the
rule will not match the tree. To avoid issuing syntax errors and
attempting error recovery, it bumps up the backtracking level. Upon
failure, the invoked rule immediately returns. If you don't plan on
using this technique in your own ANTLR-based application, don't sweat
the details. This method boils down to ``call a rule to match a tree,
executing any embedded actions and rewrite rules.''
At this point, we know how to define tree grammar rules and how to
apply them to a particular subtree. The final piece of the tree
pattern matcher is the actual tree traversal. We have to get the
correct node visitation order. In particular, we need to perform the
scalar-vector multiply transformation on the way down (preorder) and
we need to reduce multiply-by-zero subtrees on the way up (postorder).
To implement a top-down visitor, we do a depth first walk of the tree,
executing an action in the preorder position. To get a bottom-up
visitor, we execute an action in the postorder position. ANTLR
provides a standard @cl TreeVisitor class with a depth first search @v
visit method. That method executes either a @m pre or @m post method
or both. In our case, we need to call @m applyOnce in both. On the way
down, we'll look for @r vmult patterns. On the way up,
we'll look for @r mult0 patterns.
*/
public class TreeFilter extends TreeParser {
public interface fptr {
public void rule() throws RecognitionException;
}
protected TokenStream originalTokenStream;
protected TreeAdaptor originalAdaptor;
public TreeFilter(TreeNodeStream input) {
this(input, new RecognizerSharedState());
}
public TreeFilter(TreeNodeStream input, RecognizerSharedState state) {
super(input, state);
originalAdaptor = input.getTreeAdaptor();
originalTokenStream = input.getTokenStream();
}
public void applyOnce(Object t, fptr whichRule) {
if ( t==null ) return;
try {
// share TreeParser object but not parsing-related state
state = new RecognizerSharedState();
input = new CommonTreeNodeStream(originalAdaptor, t);
((CommonTreeNodeStream)input).setTokenStream(originalTokenStream);
setBacktrackingLevel(1);
whichRule.rule();
setBacktrackingLevel(0);
}
catch (RecognitionException e) { ; }
}
public void downup(Object t) {
TreeVisitor v = new TreeVisitor(new CommonTreeAdaptor());
TreeVisitorAction actions = new TreeVisitorAction() {
public Object pre(Object t) { applyOnce(t, topdown_fptr); return t; }
public Object post(Object t) { applyOnce(t, bottomup_fptr); return t; }
};
v.visit(t, actions);
}
fptr topdown_fptr = new fptr() {
public void rule() throws RecognitionException {
topdown();
}
};
fptr bottomup_fptr = new fptr() {
public void rule() throws RecognitionException {
bottomup();
}
};
// methods the downup strategy uses to do the up and down rules.
// to override, just define tree grammar rule topdown and turn on
// filter=true.
public void topdown() throws RecognitionException {;}
public void bottomup() throws RecognitionException {;}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/MismatchedTreeNodeException.java 0000644 0001750 0001750 00000003626 11256465214 031271 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
import org.antlr.runtime.tree.TreeNodeStream;
import org.antlr.runtime.tree.Tree;
/**
*/
public class MismatchedTreeNodeException extends RecognitionException {
public int expecting;
public MismatchedTreeNodeException() {
}
public MismatchedTreeNodeException(int expecting, TreeNodeStream input) {
super(input);
this.expecting = expecting;
}
public String toString() {
return "MismatchedTreeNodeException("+getUnexpectedType()+"!="+expecting+")";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/CharStreamState.java 0000644 0001750 0001750 00000003745 11256465214 026742 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** When walking ahead with cyclic DFA or for syntactic predicates,
* we need to record the state of the input stream (char index,
* line, etc...) so that we can rewind the state after scanning ahead.
*
* This is the complete state of a stream.
*/
public class CharStreamState {
/** Index into the char stream of next lookahead char */
int p;
/** What line number is the scanner at before processing buffer[p]? */
int line;
/** What char position 0..n-1 in line is scanner before processing buffer[p]? */
int charPositionInLine;
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/ANTLRReaderStream.java 0000644 0001750 0001750 00000006433 11256465214 027064 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
import java.io.*;
/** Vacuum all input from a Reader and then treat it like a StringStream.
* Manage the buffer manually to avoid unnecessary data copying.
*
* If you need encoding, use ANTLRInputStream.
*/
public class ANTLRReaderStream extends ANTLRStringStream {
public static final int READ_BUFFER_SIZE = 1024;
public static final int INITIAL_BUFFER_SIZE = 1024;
public ANTLRReaderStream() {
}
public ANTLRReaderStream(Reader r) throws IOException {
this(r, INITIAL_BUFFER_SIZE, READ_BUFFER_SIZE);
}
public ANTLRReaderStream(Reader r, int size) throws IOException {
this(r, size, READ_BUFFER_SIZE);
}
public ANTLRReaderStream(Reader r, int size, int readChunkSize) throws IOException {
load(r, size, readChunkSize);
}
public void load(Reader r, int size, int readChunkSize)
throws IOException
{
if ( r==null ) {
return;
}
if ( size<=0 ) {
size = INITIAL_BUFFER_SIZE;
}
if ( readChunkSize<=0 ) {
readChunkSize = READ_BUFFER_SIZE;
}
// System.out.println("load "+size+" in chunks of "+readChunkSize);
try {
// alloc initial buffer size.
data = new char[size];
// read all the data in chunks of readChunkSize
int numRead=0;
int p = 0;
do {
if ( p+readChunkSize > data.length ) { // overflow?
// System.out.println("### overflow p="+p+", data.length="+data.length);
char[] newdata = new char[data.length*2]; // resize
System.arraycopy(data, 0, newdata, 0, data.length);
data = newdata;
}
numRead = r.read(data, p, readChunkSize);
// System.out.println("read "+numRead+" chars; p was "+p+" is now "+(p+numRead));
p += numRead;
} while (numRead!=-1); // while not EOF
// set the actual size of the data available;
// EOF subtracted one above in p+=numRead; add one back
super.n = p+1;
//System.out.println("n="+n);
}
finally {
r.close();
}
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/CharStream.java 0000644 0001750 0001750 00000004654 11256465214 025741 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A source of characters for an ANTLR lexer */
public interface CharStream extends IntStream {
public static final int EOF = -1;
/** For infinite streams, you don't need this; primarily I'm providing
* a useful interface for action code. Just make sure actions don't
* use this on streams that don't support it.
*/
public String substring(int start, int stop);
/** Get the ith character of lookahead. This is the same usually as
* LA(i). This will be used for labels in the generated
* lexer code. I'd prefer to return a char here type-wise, but it's
* probably better to be 32-bit clean and be consistent with LA.
*/
public int LT(int i);
/** ANTLR tracks the line information automatically */
int getLine();
/** Because this stream can rewind, we need to be able to reset the line */
void setLine(int line);
void setCharPositionInLine(int pos);
/** The index of the character relative to the beginning of the line 0..n-1 */
int getCharPositionInLine();
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/MismatchedRangeException.java 0000644 0001750 0001750 00000003530 11256465214 030612 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
public class MismatchedRangeException extends RecognitionException {
public int a,b;
/** Used for remote debugger deserialization */
public MismatchedRangeException() {;}
public MismatchedRangeException(int a, int b, IntStream input) {
super(input);
this.a = a;
this.b = b;
}
public String toString() {
return "MismatchedNotSetException("+getUnexpectedType()+" not in ["+a+","+b+"])";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/ParserRuleReturnScope.java 0000644 0001750 0001750 00000005145 11256465214 030162 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** Rules that return more than a single value must return an object
* containing all the values. Besides the properties defined in
* RuleLabelScope.predefinedRulePropertiesScope there may be user-defined
* return values. This class simply defines the minimum properties that
* are always defined and methods to access the others that might be
* available depending on output option such as template and tree.
*
* Note text is not an actual property of the return value, it is computed
* from start and stop using the input stream's toString() method. I
* could add a ctor to this so that we can pass in and store the input
* stream, but I'm not sure we want to do that. It would seem to be undefined
* to get the .text property anyway if the rule matches tokens from multiple
* input streams.
*
* I do not use getters for fields of objects that are used simply to
* group values such as this aggregate. The getters/setters are there to
* satisfy the superclass interface.
*/
public class ParserRuleReturnScope extends RuleReturnScope {
public Token start, stop;
public Object getStart() { return start; }
public Object getStop() { return stop; }
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/MismatchedNotSetException.java 0000644 0001750 0001750 00000003470 11256465214 030775 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
public class MismatchedNotSetException extends MismatchedSetException {
/** Used for remote debugger deserialization */
public MismatchedNotSetException() {;}
public MismatchedNotSetException(BitSet expecting, IntStream input) {
super(expecting, input);
}
public String toString() {
return "MismatchedNotSetException("+getUnexpectedType()+"!="+expecting+")";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/TokenStream.java 0000644 0001750 0001750 00000006214 11256465214 026136 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A stream of tokens accessing tokens from a TokenSource */
public interface TokenStream extends IntStream {
/** Get Token at current input pointer + i ahead where i=1 is next Token.
* i<0 indicates tokens in the past. So -1 is previous token and -2 is
* two tokens ago. LT(0) is undefined. For i>=n, return Token.EOFToken.
* Return null for LT(0) and any index that results in an absolute address
* that is negative.
*/
public Token LT(int k);
/** Get a token at an absolute index i; 0..n-1. This is really only
* needed for profiling and debugging and token stream rewriting.
* If you don't want to buffer up tokens, then this method makes no
* sense for you. Naturally you can't use the rewrite stream feature.
* I believe DebugTokenStream can easily be altered to not use
* this method, removing the dependency.
*/
public Token get(int i);
/** Where is this stream pulling tokens from? This is not the name, but
* the object that provides Token objects.
*/
public TokenSource getTokenSource();
/** Return the text of all tokens from start to stop, inclusive.
* If the stream does not buffer all the tokens then it can just
* return "" or null; Users should not access $ruleLabel.text in
* an action of course in that case.
*/
public String toString(int start, int stop);
/** Because the user is not required to use a token with an index stored
* in it, we must provide a means for two token objects themselves to
* indicate the start/end location. Most often this will just delegate
* to the other toString(int,int). This is also parallel with
* the TreeNodeStream.toString(Object,Object).
*/
public String toString(Token start, Token stop);
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/CommonTokenStream.java 0000644 0001750 0001750 00000023255 11256465214 027313 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
import java.util.*;
/** The most common stream of tokens is one where every token is buffered up
* and tokens are prefiltered for a certain channel (the parser will only
* see these tokens and cannot change the filter channel number during the
* parse).
*
* TODO: how to access the full token stream? How to track all tokens matched per rule?
*/
public class CommonTokenStream implements TokenStream {
protected TokenSource tokenSource;
/** Record every single token pulled from the source so we can reproduce
* chunks of it later.
*/
protected List tokens;
/** Map to override some Tokens' channel numbers */
protected Map channelOverrideMap;
/** Set; discard any tokens with this type */
protected Set discardSet;
/** Skip tokens on any channel but this one; this is how we skip whitespace... */
protected int channel = Token.DEFAULT_CHANNEL;
/** By default, track all incoming tokens */
protected boolean discardOffChannelTokens = false;
/** Track the last mark() call result value for use in rewind(). */
protected int lastMarker;
/** The index into the tokens list of the current token (next token
* to consume). p==-1 indicates that the tokens list is empty
*/
protected int p = -1;
public CommonTokenStream() {
tokens = new ArrayList(500);
}
public CommonTokenStream(TokenSource tokenSource) {
this();
this.tokenSource = tokenSource;
}
public CommonTokenStream(TokenSource tokenSource, int channel) {
this(tokenSource);
this.channel = channel;
}
/** Reset this token stream by setting its token source. */
public void setTokenSource(TokenSource tokenSource) {
this.tokenSource = tokenSource;
tokens.clear();
p = -1;
channel = Token.DEFAULT_CHANNEL;
}
/** Load all tokens from the token source and put in tokens.
* This is done upon first LT request because you might want to
* set some token type / channel overrides before filling buffer.
*/
protected void fillBuffer() {
int index = 0;
Token t = tokenSource.nextToken();
while ( t!=null && t.getType()!=CharStream.EOF ) {
boolean discard = false;
// is there a channel override for token type?
if ( channelOverrideMap!=null ) {
Integer channelI = (Integer)
channelOverrideMap.get(new Integer(t.getType()));
if ( channelI!=null ) {
t.setChannel(channelI.intValue());
}
}
if ( discardSet!=null &&
discardSet.contains(new Integer(t.getType())) )
{
discard = true;
}
else if ( discardOffChannelTokens && t.getChannel()!=this.channel ) {
discard = true;
}
if ( !discard ) {
t.setTokenIndex(index);
tokens.add(t);
index++;
}
t = tokenSource.nextToken();
}
// leave p pointing at first token on channel
p = 0;
p = skipOffTokenChannels(p);
}
/** Move the input pointer to the next incoming token. The stream
* must become active with LT(1) available. consume() simply
* moves the input pointer so that LT(1) points at the next
* input symbol. Consume at least one token.
*
* Walk past any token not on the channel the parser is listening to.
*/
public void consume() {
if ( p=0 && ((Token)tokens.get(i)).getChannel()!=channel ) {
i--;
}
return i;
}
/** A simple filter mechanism whereby you can tell this token stream
* to force all tokens of type ttype to be on channel. For example,
* when interpreting, we cannot exec actions so we need to tell
* the stream to force all WS and NEWLINE to be a different, ignored
* channel.
*/
public void setTokenTypeChannel(int ttype, int channel) {
if ( channelOverrideMap==null ) {
channelOverrideMap = new HashMap();
}
channelOverrideMap.put(new Integer(ttype), new Integer(channel));
}
public void discardTokenType(int ttype) {
if ( discardSet==null ) {
discardSet = new HashSet();
}
discardSet.add(new Integer(ttype));
}
public void discardOffChannelTokens(boolean discardOffChannelTokens) {
this.discardOffChannelTokens = discardOffChannelTokens;
}
public List getTokens() {
if ( p == -1 ) {
fillBuffer();
}
return tokens;
}
public List getTokens(int start, int stop) {
return getTokens(start, stop, (BitSet)null);
}
/** Given a start and stop index, return a List of all tokens in
* the token type BitSet. Return null if no tokens were found. This
* method looks at both on and off channel tokens.
*/
public List getTokens(int start, int stop, BitSet types) {
if ( p == -1 ) {
fillBuffer();
}
if ( stop>=tokens.size() ) {
stop=tokens.size()-1;
}
if ( start<0 ) {
start=0;
}
if ( start>stop ) {
return null;
}
// list = tokens[start:stop]:{Token t, t.getType() in types}
List filteredTokens = new ArrayList();
for (int i=start; i<=stop; i++) {
Token t = (Token)tokens.get(i);
if ( types==null || types.member(t.getType()) ) {
filteredTokens.add(t);
}
}
if ( filteredTokens.size()==0 ) {
filteredTokens = null;
}
return filteredTokens;
}
public List getTokens(int start, int stop, List types) {
return getTokens(start,stop,new BitSet(types));
}
public List getTokens(int start, int stop, int ttype) {
return getTokens(start,stop,BitSet.of(ttype));
}
/** Get the ith token from the current position 1..n where k=1 is the
* first symbol of lookahead.
*/
public Token LT(int k) {
if ( p == -1 ) {
fillBuffer();
}
if ( k==0 ) {
return null;
}
if ( k<0 ) {
return LB(-k);
}
//System.out.print("LT(p="+p+","+k+")=");
if ( (p+k-1) >= tokens.size() ) {
return Token.EOF_TOKEN;
}
//System.out.println(tokens.get(p+k-1));
int i = p;
int n = 1;
// find k good tokens
while ( n=tokens.size() ) {
return Token.EOF_TOKEN;
}
return (Token)tokens.get(i);
}
/** Look backwards k tokens on-channel tokens */
protected Token LB(int k) {
//System.out.print("LB(p="+p+","+k+") ");
if ( p == -1 ) {
fillBuffer();
}
if ( k==0 ) {
return null;
}
if ( (p-k)<0 ) {
return null;
}
int i = p;
int n = 1;
// find k good tokens looking backwards
while ( n<=k ) {
// skip off-channel tokens
i = skipOffTokenChannelsReverse(i-1); // leave p on valid token
n++;
}
if ( i<0 ) {
return null;
}
return (Token)tokens.get(i);
}
/** Return absolute token i; ignore which channel the tokens are on;
* that is, count all tokens not just on-channel tokens.
*/
public Token get(int i) {
return (Token)tokens.get(i);
}
public int LA(int i) {
return LT(i).getType();
}
public int mark() {
if ( p == -1 ) {
fillBuffer();
}
lastMarker = index();
return lastMarker;
}
public void release(int marker) {
// no resources to release
}
public int size() {
return tokens.size();
}
public int index() {
return p;
}
public void rewind(int marker) {
seek(marker);
}
public void rewind() {
seek(lastMarker);
}
public void reset() {
p = 0;
lastMarker = 0;
}
public void seek(int index) {
p = index;
}
public TokenSource getTokenSource() {
return tokenSource;
}
public String getSourceName() {
return getTokenSource().getSourceName();
}
public String toString() {
if ( p == -1 ) {
fillBuffer();
}
return toString(0, tokens.size()-1);
}
public String toString(int start, int stop) {
if ( start<0 || stop<0 ) {
return null;
}
if ( p == -1 ) {
fillBuffer();
}
if ( stop>=tokens.size() ) {
stop = tokens.size()-1;
}
StringBuffer buf = new StringBuffer();
for (int i = start; i <= stop; i++) {
Token t = (Token)tokens.get(i);
buf.append(t.getText());
}
return buf.toString();
}
public String toString(Token start, Token stop) {
if ( start!=null && stop!=null ) {
return toString(start.getTokenIndex(), stop.getTokenIndex());
}
return null;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/SerializedGrammar.java 0000644 0001750 0001750 00000012745 11256465214 027312 0 ustar twerner twerner package org.antlr.runtime;
import java.io.IOException;
import java.io.FileInputStream;
import java.io.BufferedInputStream;
import java.io.DataInputStream;
import java.util.List;
import java.util.ArrayList;
public class SerializedGrammar {
public static final String COOKIE = "$ANTLR";
public static final int FORMAT_VERSION = 1;
//public static org.antlr.tool.Grammar gr; // TESTING ONLY; remove later
public String name;
public char type; // in {l, p, t, c}
public List rules;
class Rule {
String name;
Block block;
public Rule(String name, Block block) {
this.name = name;
this.block = block;
}
public String toString() {
return name+":"+block;
}
}
class Block {
List[] alts;
public Block(List[] alts) {
this.alts = alts;
}
public String toString() {
StringBuffer buf = new StringBuffer();
buf.append("(");
for (int i = 0; i < alts.length; i++) {
List alt = alts[i];
if ( i>0 ) buf.append("|");
buf.append(alt.toString());
}
buf.append(")");
return buf.toString();
}
}
class TokenRef {
int ttype;
public TokenRef(int ttype) { this.ttype = ttype; }
public String toString() { return String.valueOf(ttype); }
}
class RuleRef {
int ruleIndex;
public RuleRef(int ruleIndex) { this.ruleIndex = ruleIndex; }
public String toString() { return String.valueOf(ruleIndex); }
}
public SerializedGrammar(String filename) throws IOException {
System.out.println("loading "+filename);
FileInputStream fis = new FileInputStream(filename);
BufferedInputStream bos = new BufferedInputStream(fis);
DataInputStream in = new DataInputStream(bos);
readFile(in);
in.close();
}
protected void readFile(DataInputStream in) throws IOException {
String cookie = readString(in); // get $ANTLR
if ( !cookie.equals(COOKIE) ) throw new IOException("not a serialized grammar file");
int version = in.readByte();
char grammarType = (char)in.readByte();
this.type = grammarType;
String grammarName = readString(in);
this.name = grammarName;
System.out.println(grammarType+" grammar "+grammarName);
int numRules = in.readShort();
System.out.println("num rules = "+numRules);
rules = readRules(in, numRules);
}
protected List readRules(DataInputStream in, int numRules) throws IOException {
List rules = new ArrayList();
for (int i=0; i0 ) {
state.failed = true;
return;
}
MismatchedTokenException mte =
new MismatchedTokenException(s.charAt(i), input);
recover(mte);
throw mte;
}
i++;
input.consume();
state.failed = false;
}
}
public void matchAny() {
input.consume();
}
public void match(int c) throws MismatchedTokenException {
if ( input.LA(1)!=c ) {
if ( state.backtracking>0 ) {
state.failed = true;
return;
}
MismatchedTokenException mte =
new MismatchedTokenException(c, input);
recover(mte); // don't really recover; just consume in lexer
throw mte;
}
input.consume();
state.failed = false;
}
public void matchRange(int a, int b)
throws MismatchedRangeException
{
if ( input.LA(1)b ) {
if ( state.backtracking>0 ) {
state.failed = true;
return;
}
MismatchedRangeException mre =
new MismatchedRangeException(a,b,input);
recover(mre);
throw mre;
}
input.consume();
state.failed = false;
}
public int getLine() {
return input.getLine();
}
public int getCharPositionInLine() {
return input.getCharPositionInLine();
}
/** What is the index of the current character of lookahead? */
public int getCharIndex() {
return input.index();
}
/** Return the text matched so far for the current token or any
* text override.
*/
public String getText() {
if ( state.text!=null ) {
return state.text;
}
return input.substring(state.tokenStartCharIndex,getCharIndex()-1);
}
/** Set the complete text of this token; it wipes any previous
* changes to the text.
*/
public void setText(String text) {
state.text = text;
}
public void reportError(RecognitionException e) {
/** TODO: not thought about recovery in lexer yet.
*
// if we've already reported an error and have not matched a token
// yet successfully, don't report any errors.
if ( errorRecovery ) {
//System.err.print("[SPURIOUS] ");
return;
}
errorRecovery = true;
*/
displayRecognitionError(this.getTokenNames(), e);
}
public String getErrorMessage(RecognitionException e, String[] tokenNames) {
String msg = null;
if ( e instanceof MismatchedTokenException ) {
MismatchedTokenException mte = (MismatchedTokenException)e;
msg = "mismatched character "+getCharErrorDisplay(e.c)+" expecting "+getCharErrorDisplay(mte.expecting);
}
else if ( e instanceof NoViableAltException ) {
NoViableAltException nvae = (NoViableAltException)e;
// for development, can add "decision=<<"+nvae.grammarDecisionDescription+">>"
// and "(decision="+nvae.decisionNumber+") and
// "state "+nvae.stateNumber
msg = "no viable alternative at character "+getCharErrorDisplay(e.c);
}
else if ( e instanceof EarlyExitException ) {
EarlyExitException eee = (EarlyExitException)e;
// for development, can add "(decision="+eee.decisionNumber+")"
msg = "required (...)+ loop did not match anything at character "+getCharErrorDisplay(e.c);
}
else if ( e instanceof MismatchedNotSetException ) {
MismatchedNotSetException mse = (MismatchedNotSetException)e;
msg = "mismatched character "+getCharErrorDisplay(e.c)+" expecting set "+mse.expecting;
}
else if ( e instanceof MismatchedSetException ) {
MismatchedSetException mse = (MismatchedSetException)e;
msg = "mismatched character "+getCharErrorDisplay(e.c)+" expecting set "+mse.expecting;
}
else if ( e instanceof MismatchedRangeException ) {
MismatchedRangeException mre = (MismatchedRangeException)e;
msg = "mismatched character "+getCharErrorDisplay(e.c)+" expecting set "+
getCharErrorDisplay(mre.a)+".."+getCharErrorDisplay(mre.b);
}
else {
msg = super.getErrorMessage(e, tokenNames);
}
return msg;
}
public String getCharErrorDisplay(int c) {
String s = String.valueOf((char)c);
switch ( c ) {
case Token.EOF :
s = "";
break;
case '\n' :
s = "\\n";
break;
case '\t' :
s = "\\t";
break;
case '\r' :
s = "\\r";
break;
}
return "'"+s+"'";
}
/** Lexers can normally match any char in it's vocabulary after matching
* a token, so do the easy thing and just kill a character and hope
* it all works out. You can instead use the rule invocation stack
* to do sophisticated error recovery if you are in a fragment rule.
*/
public void recover(RecognitionException re) {
//System.out.println("consuming char "+(char)input.LA(1)+" during recovery");
//re.printStackTrace();
input.consume();
}
public void traceIn(String ruleName, int ruleIndex) {
String inputSymbol = ((char)input.LT(1))+" line="+getLine()+":"+getCharPositionInLine();
super.traceIn(ruleName, ruleIndex, inputSymbol);
}
public void traceOut(String ruleName, int ruleIndex) {
String inputSymbol = ((char)input.LT(1))+" line="+getLine()+":"+getCharPositionInLine();
super.traceOut(ruleName, ruleIndex, inputSymbol);
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/NoViableAltException.java 0000644 0001750 0001750 00000004401 11256465214 027715 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
public class NoViableAltException extends RecognitionException {
public String grammarDecisionDescription;
public int decisionNumber;
public int stateNumber;
/** Used for remote debugger deserialization */
public NoViableAltException() {;}
public NoViableAltException(String grammarDecisionDescription,
int decisionNumber,
int stateNumber,
IntStream input)
{
super(input);
this.grammarDecisionDescription = grammarDecisionDescription;
this.decisionNumber = decisionNumber;
this.stateNumber = stateNumber;
}
public String toString() {
if ( input instanceof CharStream ) {
return "NoViableAltException('"+(char)getUnexpectedType()+"'@["+grammarDecisionDescription+"])";
}
else {
return "NoViableAltException("+getUnexpectedType()+"@["+grammarDecisionDescription+"])";
}
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/BitSet.java 0000644 0001750 0001750 00000020526 11256465214 025076 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
import java.util.List;
/**A stripped-down version of org.antlr.misc.BitSet that is just
* good enough to handle runtime requirements such as FOLLOW sets
* for automatic error recovery.
*/
public class BitSet implements Cloneable {
protected final static int BITS = 64; // number of bits / long
protected final static int LOG_BITS = 6; // 2^6 == 64
/* We will often need to do a mod operator (i mod nbits). Its
* turns out that, for powers of two, this mod operation is
* same as (i & (nbits-1)). Since mod is slow, we use a
* precomputed mod mask to do the mod instead.
*/
protected final static int MOD_MASK = BITS - 1;
/** The actual data bits */
protected long bits[];
/** Construct a bitset of size one word (64 bits) */
public BitSet() {
this(BITS);
}
/** Construction from a static array of longs */
public BitSet(long[] bits_) {
bits = bits_;
}
/** Construction from a list of integers */
public BitSet(List items) {
this();
for (int i = 0; i < items.size(); i++) {
Integer v = (Integer) items.get(i);
add(v.intValue());
}
}
/** Construct a bitset given the size
* @param nbits The size of the bitset in bits
*/
public BitSet(int nbits) {
bits = new long[((nbits - 1) >> LOG_BITS) + 1];
}
public static BitSet of(int el) {
BitSet s = new BitSet(el + 1);
s.add(el);
return s;
}
public static BitSet of(int a, int b) {
BitSet s = new BitSet(Math.max(a,b)+1);
s.add(a);
s.add(b);
return s;
}
public static BitSet of(int a, int b, int c) {
BitSet s = new BitSet();
s.add(a);
s.add(b);
s.add(c);
return s;
}
public static BitSet of(int a, int b, int c, int d) {
BitSet s = new BitSet();
s.add(a);
s.add(b);
s.add(c);
s.add(d);
return s;
}
/** return this | a in a new set */
public BitSet or(BitSet a) {
if ( a==null ) {
return this;
}
BitSet s = (BitSet)this.clone();
s.orInPlace(a);
return s;
}
/** or this element into this set (grow as necessary to accommodate) */
public void add(int el) {
int n = wordNumber(el);
if (n >= bits.length) {
growToInclude(el);
}
bits[n] |= bitMask(el);
}
/**
* Grows the set to a larger number of bits.
* @param bit element that must fit in set
*/
public void growToInclude(int bit) {
int newSize = Math.max(bits.length << 1, numWordsToHold(bit));
long newbits[] = new long[newSize];
System.arraycopy(bits, 0, newbits, 0, bits.length);
bits = newbits;
}
public void orInPlace(BitSet a) {
if ( a==null ) {
return;
}
// If this is smaller than a, grow this first
if (a.bits.length > bits.length) {
setSize(a.bits.length);
}
int min = Math.min(bits.length, a.bits.length);
for (int i = min - 1; i >= 0; i--) {
bits[i] |= a.bits[i];
}
}
/**
* Sets the size of a set.
* @param nwords how many words the new set should be
*/
private void setSize(int nwords) {
long newbits[] = new long[nwords];
int n = Math.min(nwords, bits.length);
System.arraycopy(bits, 0, newbits, 0, n);
bits = newbits;
}
private final static long bitMask(int bitNumber) {
int bitPosition = bitNumber & MOD_MASK; // bitNumber mod BITS
return 1L << bitPosition;
}
public Object clone() {
BitSet s;
try {
s = (BitSet)super.clone();
s.bits = new long[bits.length];
System.arraycopy(bits, 0, s.bits, 0, bits.length);
}
catch (CloneNotSupportedException e) {
throw new InternalError();
}
return s;
}
public int size() {
int deg = 0;
for (int i = bits.length - 1; i >= 0; i--) {
long word = bits[i];
if (word != 0L) {
for (int bit = BITS - 1; bit >= 0; bit--) {
if ((word & (1L << bit)) != 0) {
deg++;
}
}
}
}
return deg;
}
public boolean equals(Object other) {
if ( other == null || !(other instanceof BitSet) ) {
return false;
}
BitSet otherSet = (BitSet)other;
int n = Math.min(this.bits.length, otherSet.bits.length);
// for any bits in common, compare
for (int i=0; i n) {
for (int i = n+1; i n) {
for (int i = n+1; i= bits.length) return false;
return (bits[n] & bitMask(el)) != 0;
}
// remove this element from this set
public void remove(int el) {
int n = wordNumber(el);
if (n < bits.length) {
bits[n] &= ~bitMask(el);
}
}
public boolean isNil() {
for (int i = bits.length - 1; i >= 0; i--) {
if (bits[i] != 0) return false;
}
return true;
}
private final int numWordsToHold(int el) {
return (el >> LOG_BITS) + 1;
}
public int numBits() {
return bits.length << LOG_BITS; // num words * bits per word
}
/** return how much space is being used by the bits array not
* how many actually have member bits on.
*/
public int lengthInLongWords() {
return bits.length;
}
/**Is this contained within a? */
/*
public boolean subset(BitSet a) {
if (a == null || !(a instanceof BitSet)) return false;
return this.and(a).equals(this);
}
*/
public int[] toArray() {
int[] elems = new int[size()];
int en = 0;
for (int i = 0; i < (bits.length << LOG_BITS); i++) {
if (member(i)) {
elems[en++] = i;
}
}
return elems;
}
public long[] toPackedArray() {
return bits;
}
private final static int wordNumber(int bit) {
return bit >> LOG_BITS; // bit / BITS
}
public String toString() {
return toString(null);
}
public String toString(String[] tokenNames) {
StringBuffer buf = new StringBuffer();
String separator = ",";
boolean havePrintedAnElement = false;
buf.append('{');
for (int i = 0; i < (bits.length << LOG_BITS); i++) {
if (member(i)) {
if (i > 0 && havePrintedAnElement ) {
buf.append(separator);
}
if ( tokenNames!=null ) {
buf.append(tokenNames[i]);
}
else {
buf.append(i);
}
havePrintedAnElement = true;
}
}
buf.append('}');
return buf.toString();
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/FailedPredicateException.java 0000644 0001750 0001750 00000004313 11256465214 030564 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A semantic predicate failed during validation. Validation of predicates
* occurs when normally parsing the alternative just like matching a token.
* Disambiguating predicate evaluation occurs when we hoist a predicate into
* a prediction decision.
*/
public class FailedPredicateException extends RecognitionException {
public String ruleName;
public String predicateText;
/** Used for remote debugger deserialization */
public FailedPredicateException() {;}
public FailedPredicateException(IntStream input,
String ruleName,
String predicateText)
{
super(input);
this.ruleName = ruleName;
this.predicateText = predicateText;
}
public String toString() {
return "FailedPredicateException("+ruleName+",{"+predicateText+"}?)";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/RecognizerSharedState.java 0000644 0001750 0001750 00000013417 11256465214 030144 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/package org.antlr.runtime;
import java.util.Map;
/** The set of fields needed by an abstract recognizer to recognize input
* and recover from errors etc... As a separate state object, it can be
* shared among multiple grammars; e.g., when one grammar imports another.
*
* These fields are publically visible but the actual state pointer per
* parser is protected.
*/
public class RecognizerSharedState {
/** Track the set of token types that can follow any rule invocation.
* Stack grows upwards. When it hits the max, it grows 2x in size
* and keeps going.
*/
public BitSet[] following = new BitSet[BaseRecognizer.INITIAL_FOLLOW_STACK_SIZE];
public int _fsp = -1;
/** This is true when we see an error and before having successfully
* matched a token. Prevents generation of more than one error message
* per error.
*/
public boolean errorRecovery = false;
/** The index into the input stream where the last error occurred.
* This is used to prevent infinite loops where an error is found
* but no token is consumed during recovery...another error is found,
* ad naseum. This is a failsafe mechanism to guarantee that at least
* one token/tree node is consumed for two errors.
*/
public int lastErrorIndex = -1;
/** In lieu of a return value, this indicates that a rule or token
* has failed to match. Reset to false upon valid token match.
*/
public boolean failed = false;
/** Did the recognizer encounter a syntax error? Track how many. */
public int syntaxErrors = 0;
/** If 0, no backtracking is going on. Safe to exec actions etc...
* If >0 then it's the level of backtracking.
*/
public int backtracking = 0;
/** An array[size num rules] of Map that tracks
* the stop token index for each rule. ruleMemo[ruleIndex] is
* the memoization table for ruleIndex. For key ruleStartIndex, you
* get back the stop token for associated rule or MEMO_RULE_FAILED.
*
* This is only used if rule memoization is on (which it is by default).
*/
public Map[] ruleMemo;
// LEXER FIELDS (must be in same state object to avoid casting
// constantly in generated code and Lexer object) :(
/** The goal of all lexer rules/methods is to create a token object.
* This is an instance variable as multiple rules may collaborate to
* create a single token. nextToken will return this object after
* matching lexer rule(s). If you subclass to allow multiple token
* emissions, then set this to the last token to be matched or
* something nonnull so that the auto token emit mechanism will not
* emit another token.
*/
public Token token;
/** What character index in the stream did the current token start at?
* Needed, for example, to get the text for current token. Set at
* the start of nextToken.
*/
public int tokenStartCharIndex = -1;
/** The line on which the first character of the token resides */
public int tokenStartLine;
/** The character position of first character within the line */
public int tokenStartCharPositionInLine;
/** The channel number for the current token */
public int channel;
/** The token type for the current token */
public int type;
/** You can set the text for the current token to override what is in
* the input char buffer. Use setText() or can set this instance var.
*/
public String text;
public RecognizerSharedState() {;}
public RecognizerSharedState(RecognizerSharedState state) {
if ( this.following.length < state.following.length ) {
this.following = new BitSet[state.following.length];
}
System.arraycopy(state.following, 0, this.following, 0, state.following.length);
this._fsp = state._fsp;
this.errorRecovery = state.errorRecovery;
this.lastErrorIndex = state.lastErrorIndex;
this.failed = state.failed;
this.syntaxErrors = state.syntaxErrors;
this.backtracking = state.backtracking;
if ( state.ruleMemo!=null ) {
this.ruleMemo = new Map[state.ruleMemo.length];
System.arraycopy(state.ruleMemo, 0, this.ruleMemo, 0, state.ruleMemo.length);
}
this.token = state.token;
this.tokenStartCharIndex = state.tokenStartCharIndex;
this.tokenStartCharPositionInLine = state.tokenStartCharPositionInLine;
this.channel = state.channel;
this.type = state.type;
this.text = state.text;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/IntStream.java 0000644 0001750 0001750 00000012524 11256465214 025611 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A simple stream of integers used when all I care about is the char
* or token type sequence (such as interpretation).
*/
public interface IntStream {
void consume();
/** Get int at current input pointer + i ahead where i=1 is next int.
* Negative indexes are allowed. LA(-1) is previous token (token
* just matched). LA(-i) where i is before first token should
* yield -1, invalid char / EOF.
*/
int LA(int i);
/** Tell the stream to start buffering if it hasn't already. Return
* current input position, index(), or some other marker so that
* when passed to rewind() you get back to the same spot.
* rewind(mark()) should not affect the input cursor. The Lexer
* track line/col info as well as input index so its markers are
* not pure input indexes. Same for tree node streams.
*/
int mark();
/** Return the current input symbol index 0..n where n indicates the
* last symbol has been read. The index is the symbol about to be
* read not the most recently read symbol.
*/
int index();
/** Reset the stream so that next call to index would return marker.
* The marker will usually be index() but it doesn't have to be. It's
* just a marker to indicate what state the stream was in. This is
* essentially calling release() and seek(). If there are markers
* created after this marker argument, this routine must unroll them
* like a stack. Assume the state the stream was in when this marker
* was created.
*/
void rewind(int marker);
/** Rewind to the input position of the last marker.
* Used currently only after a cyclic DFA and just
* before starting a sem/syn predicate to get the
* input position back to the start of the decision.
* Do not "pop" the marker off the state. mark(i)
* and rewind(i) should balance still. It is
* like invoking rewind(last marker) but it should not "pop"
* the marker off. It's like seek(last marker's input position).
*/
void rewind();
/** You may want to commit to a backtrack but don't want to force the
* stream to keep bookkeeping objects around for a marker that is
* no longer necessary. This will have the same behavior as
* rewind() except it releases resources without the backward seek.
* This must throw away resources for all markers back to the marker
* argument. So if you're nested 5 levels of mark(), and then release(2)
* you have to release resources for depths 2..5.
*/
void release(int marker);
/** Set the input cursor to the position indicated by index. This is
* normally used to seek ahead in the input stream. No buffering is
* required to do this unless you know your stream will use seek to
* move backwards such as when backtracking.
*
* This is different from rewind in its multi-directional
* requirement and in that its argument is strictly an input cursor (index).
*
* For char streams, seeking forward must update the stream state such
* as line number. For seeking backwards, you will be presumably
* backtracking using the mark/rewind mechanism that restores state and
* so this method does not need to update state when seeking backwards.
*
* Currently, this method is only used for efficient backtracking using
* memoization, but in the future it may be used for incremental parsing.
*
* The index is 0..n-1. A seek to position i means that LA(1) will
* return the ith symbol. So, seeking to 0 means LA(1) will return the
* first element in the stream.
*/
void seek(int index);
/** Only makes sense for streams that buffer everything up probably, but
* might be useful to display the entire stream or for testing. This
* value includes a single EOF.
*/
int size();
/** Where are you getting symbols from? Normally, implementations will
* pass the buck all the way to the lexer who can ask its input stream
* for the file name or whatever.
*/
public String getSourceName();
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/MissingTokenException.java 0000644 0001750 0001750 00000004336 11256465214 030176 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** We were expecting a token but it's not found. The current token
* is actually what we wanted next. Used for tree node errors too.
*/
public class MissingTokenException extends MismatchedTokenException {
public Object inserted;
/** Used for remote debugger deserialization */
public MissingTokenException() {;}
public MissingTokenException(int expecting, IntStream input, Object inserted) {
super(expecting, input);
this.inserted = inserted;
}
public int getMissingType() {
return expecting;
}
public String toString() {
if ( inserted!=null && token!=null ) {
return "MissingTokenException(inserted "+inserted+" at "+token.getText()+")";
}
if ( token!=null ) {
return "MissingTokenException(at "+token.getText()+")";
}
return "MissingTokenException";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/RuleReturnScope.java 0000644 0001750 0001750 00000003740 11256465214 027004 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** Rules can return start/stop info as well as possible trees and templates */
public class RuleReturnScope {
/** Return the start token or tree */
public Object getStart() { return null; }
/** Return the stop token or tree */
public Object getStop() { return null; }
/** Has a value potentially if output=AST; */
public Object getTree() { return null; }
/** Has a value potentially if output=template; Don't use StringTemplate
* type as it then causes a dependency with ST lib.
*/
public Object getTemplate() { return null; }
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/CommonToken.java 0000644 0001750 0001750 00000011526 11256465214 026135 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
import java.io.Serializable;
public class CommonToken implements Token, Serializable {
protected int type;
protected int line;
protected int charPositionInLine = -1; // set to invalid position
protected int channel=DEFAULT_CHANNEL;
protected transient CharStream input;
/** We need to be able to change the text once in a while. If
* this is non-null, then getText should return this. Note that
* start/stop are not affected by changing this.
*/
protected String text;
/** What token number is this from 0..n-1 tokens; < 0 implies invalid index */
protected int index = -1;
/** The char position into the input buffer where this token starts */
protected int start;
/** The char position into the input buffer where this token stops */
protected int stop;
public CommonToken(int type) {
this.type = type;
}
public CommonToken(CharStream input, int type, int channel, int start, int stop) {
this.input = input;
this.type = type;
this.channel = channel;
this.start = start;
this.stop = stop;
}
public CommonToken(int type, String text) {
this.type = type;
this.channel = DEFAULT_CHANNEL;
this.text = text;
}
public CommonToken(Token oldToken) {
text = oldToken.getText();
type = oldToken.getType();
line = oldToken.getLine();
index = oldToken.getTokenIndex();
charPositionInLine = oldToken.getCharPositionInLine();
channel = oldToken.getChannel();
if ( oldToken instanceof CommonToken ) {
start = ((CommonToken)oldToken).start;
stop = ((CommonToken)oldToken).stop;
}
}
public int getType() {
return type;
}
public void setLine(int line) {
this.line = line;
}
public String getText() {
if ( text!=null ) {
return text;
}
if ( input==null ) {
return null;
}
text = input.substring(start,stop);
return text;
}
/** Override the text for this token. getText() will return this text
* rather than pulling from the buffer. Note that this does not mean
* that start/stop indexes are not valid. It means that that input
* was converted to a new string in the token object.
*/
public void setText(String text) {
this.text = text;
}
public int getLine() {
return line;
}
public int getCharPositionInLine() {
return charPositionInLine;
}
public void setCharPositionInLine(int charPositionInLine) {
this.charPositionInLine = charPositionInLine;
}
public int getChannel() {
return channel;
}
public void setChannel(int channel) {
this.channel = channel;
}
public void setType(int type) {
this.type = type;
}
public int getStartIndex() {
return start;
}
public void setStartIndex(int start) {
this.start = start;
}
public int getStopIndex() {
return stop;
}
public void setStopIndex(int stop) {
this.stop = stop;
}
public int getTokenIndex() {
return index;
}
public void setTokenIndex(int index) {
this.index = index;
}
public CharStream getInputStream() {
return input;
}
public void setInputStream(CharStream input) {
this.input = input;
}
public String toString() {
String channelStr = "";
if ( channel>0 ) {
channelStr=",channel="+channel;
}
String txt = getText();
if ( txt!=null ) {
txt = txt.replaceAll("\n","\\\\n");
txt = txt.replaceAll("\r","\\\\r");
txt = txt.replaceAll("\t","\\\\t");
}
else {
txt = "";
}
return "[@"+getTokenIndex()+","+start+":"+stop+"='"+txt+"',<"+type+">"+channelStr+","+line+":"+getCharPositionInLine()+"]";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/BaseRecognizer.java 0000644 0001750 0001750 00000077210 11256465214 026610 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/** A generic recognizer that can handle recognizers generated from
* lexer, parser, and tree grammars. This is all the parsing
* support code essentially; most of it is error recovery stuff and
* backtracking.
*/
public abstract class BaseRecognizer {
public static final int MEMO_RULE_FAILED = -2;
public static final int MEMO_RULE_UNKNOWN = -1;
public static final int INITIAL_FOLLOW_STACK_SIZE = 100;
// copies from Token object for convenience in actions
public static final int DEFAULT_TOKEN_CHANNEL = Token.DEFAULT_CHANNEL;
public static final int HIDDEN = Token.HIDDEN_CHANNEL;
public static final String NEXT_TOKEN_RULE_NAME = "nextToken";
/** State of a lexer, parser, or tree parser are collected into a state
* object so the state can be shared. This sharing is needed to
* have one grammar import others and share same error variables
* and other state variables. It's a kind of explicit multiple
* inheritance via delegation of methods and shared state.
*/
protected RecognizerSharedState state;
public BaseRecognizer() {
state = new RecognizerSharedState();
}
public BaseRecognizer(RecognizerSharedState state) {
if ( state==null ) {
state = new RecognizerSharedState();
}
this.state = state;
}
/** reset the parser's state; subclasses must rewinds the input stream */
public void reset() {
// wack everything related to error recovery
if ( state==null ) {
return; // no shared state work to do
}
state._fsp = -1;
state.errorRecovery = false;
state.lastErrorIndex = -1;
state.failed = false;
state.syntaxErrors = 0;
// wack everything related to backtracking and memoization
state.backtracking = 0;
for (int i = 0; state.ruleMemo!=null && i < state.ruleMemo.length; i++) { // wipe cache
state.ruleMemo[i] = null;
}
}
/** Match current input symbol against ttype. Attempt
* single token insertion or deletion error recovery. If
* that fails, throw MismatchedTokenException.
*
* To turn off single token insertion or deletion error
* recovery, override recoverFromMismatchedToken() and have it
* throw an exception. See TreeParser.recoverFromMismatchedToken().
* This way any error in a rule will cause an exception and
* immediate exit from rule. Rule would recover by resynchronizing
* to the set of symbols that can follow rule ref.
*/
public Object match(IntStream input, int ttype, BitSet follow)
throws RecognitionException
{
//System.out.println("match "+((TokenStream)input).LT(1));
Object matchedSymbol = getCurrentInputSymbol(input);
if ( input.LA(1)==ttype ) {
input.consume();
state.errorRecovery = false;
state.failed = false;
return matchedSymbol;
}
if ( state.backtracking>0 ) {
state.failed = true;
return matchedSymbol;
}
matchedSymbol = recoverFromMismatchedToken(input, ttype, follow);
return matchedSymbol;
}
/** Match the wildcard: in a symbol */
public void matchAny(IntStream input) {
state.errorRecovery = false;
state.failed = false;
input.consume();
}
public boolean mismatchIsUnwantedToken(IntStream input, int ttype) {
return input.LA(2)==ttype;
}
public boolean mismatchIsMissingToken(IntStream input, BitSet follow) {
if ( follow==null ) {
// we have no information about the follow; we can only consume
// a single token and hope for the best
return false;
}
// compute what can follow this grammar element reference
if ( follow.member(Token.EOR_TOKEN_TYPE) ) {
BitSet viableTokensFollowingThisRule = computeContextSensitiveRuleFOLLOW();
follow = follow.or(viableTokensFollowingThisRule);
if ( state._fsp>=0 ) { // remove EOR if we're not the start symbol
follow.remove(Token.EOR_TOKEN_TYPE);
}
}
// if current token is consistent with what could come after set
// then we know we're missing a token; error recovery is free to
// "insert" the missing token
//System.out.println("viable tokens="+follow.toString(getTokenNames()));
//System.out.println("LT(1)="+((TokenStream)input).LT(1));
// BitSet cannot handle negative numbers like -1 (EOF) so I leave EOR
// in follow set to indicate that the fall of the start symbol is
// in the set (EOF can follow).
if ( follow.member(input.LA(1)) || follow.member(Token.EOR_TOKEN_TYPE) ) {
//System.out.println("LT(1)=="+((TokenStream)input).LT(1)+" is consistent with what follows; inserting...");
return true;
}
return false;
}
/** Report a recognition problem.
*
* This method sets errorRecovery to indicate the parser is recovering
* not parsing. Once in recovery mode, no errors are generated.
* To get out of recovery mode, the parser must successfully match
* a token (after a resync). So it will go:
*
* 1. error occurs
* 2. enter recovery mode, report error
* 3. consume until token found in resynch set
* 4. try to resume parsing
* 5. next match() will reset errorRecovery mode
*
* If you override, make sure to update syntaxErrors if you care about that.
*/
public void reportError(RecognitionException e) {
// if we've already reported an error and have not matched a token
// yet successfully, don't report any errors.
if ( state.errorRecovery ) {
//System.err.print("[SPURIOUS] ");
return;
}
state.syntaxErrors++; // don't count spurious
state.errorRecovery = true;
displayRecognitionError(this.getTokenNames(), e);
}
public void displayRecognitionError(String[] tokenNames,
RecognitionException e)
{
String hdr = getErrorHeader(e);
String msg = getErrorMessage(e, tokenNames);
emitErrorMessage(hdr+" "+msg);
}
/** What error message should be generated for the various
* exception types?
*
* Not very object-oriented code, but I like having all error message
* generation within one method rather than spread among all of the
* exception classes. This also makes it much easier for the exception
* handling because the exception classes do not have to have pointers back
* to this object to access utility routines and so on. Also, changing
* the message for an exception type would be difficult because you
* would have to subclassing exception, but then somehow get ANTLR
* to make those kinds of exception objects instead of the default.
* This looks weird, but trust me--it makes the most sense in terms
* of flexibility.
*
* For grammar debugging, you will want to override this to add
* more information such as the stack frame with
* getRuleInvocationStack(e, this.getClass().getName()) and,
* for no viable alts, the decision description and state etc...
*
* Override this to change the message generated for one or more
* exception types.
*/
public String getErrorMessage(RecognitionException e, String[] tokenNames) {
String msg = e.getMessage();
if ( e instanceof UnwantedTokenException ) {
UnwantedTokenException ute = (UnwantedTokenException)e;
String tokenName="";
if ( ute.expecting== Token.EOF ) {
tokenName = "EOF";
}
else {
tokenName = tokenNames[ute.expecting];
}
msg = "extraneous input "+getTokenErrorDisplay(ute.getUnexpectedToken())+
" expecting "+tokenName;
}
else if ( e instanceof MissingTokenException ) {
MissingTokenException mte = (MissingTokenException)e;
String tokenName="";
if ( mte.expecting== Token.EOF ) {
tokenName = "EOF";
}
else {
tokenName = tokenNames[mte.expecting];
}
msg = "missing "+tokenName+" at "+getTokenErrorDisplay(e.token);
}
else if ( e instanceof MismatchedTokenException ) {
MismatchedTokenException mte = (MismatchedTokenException)e;
String tokenName="";
if ( mte.expecting== Token.EOF ) {
tokenName = "EOF";
}
else {
tokenName = tokenNames[mte.expecting];
}
msg = "mismatched input "+getTokenErrorDisplay(e.token)+
" expecting "+tokenName;
}
else if ( e instanceof MismatchedTreeNodeException ) {
MismatchedTreeNodeException mtne = (MismatchedTreeNodeException)e;
String tokenName="";
if ( mtne.expecting==Token.EOF ) {
tokenName = "EOF";
}
else {
tokenName = tokenNames[mtne.expecting];
}
msg = "mismatched tree node: "+mtne.node+
" expecting "+tokenName;
}
else if ( e instanceof NoViableAltException ) {
//NoViableAltException nvae = (NoViableAltException)e;
// for development, can add "decision=<<"+nvae.grammarDecisionDescription+">>"
// and "(decision="+nvae.decisionNumber+") and
// "state "+nvae.stateNumber
msg = "no viable alternative at input "+getTokenErrorDisplay(e.token);
}
else if ( e instanceof EarlyExitException ) {
//EarlyExitException eee = (EarlyExitException)e;
// for development, can add "(decision="+eee.decisionNumber+")"
msg = "required (...)+ loop did not match anything at input "+
getTokenErrorDisplay(e.token);
}
else if ( e instanceof MismatchedSetException ) {
MismatchedSetException mse = (MismatchedSetException)e;
msg = "mismatched input "+getTokenErrorDisplay(e.token)+
" expecting set "+mse.expecting;
}
else if ( e instanceof MismatchedNotSetException ) {
MismatchedNotSetException mse = (MismatchedNotSetException)e;
msg = "mismatched input "+getTokenErrorDisplay(e.token)+
" expecting set "+mse.expecting;
}
else if ( e instanceof FailedPredicateException ) {
FailedPredicateException fpe = (FailedPredicateException)e;
msg = "rule "+fpe.ruleName+" failed predicate: {"+
fpe.predicateText+"}?";
}
return msg;
}
/** Get number of recognition errors (lexer, parser, tree parser). Each
* recognizer tracks its own number. So parser and lexer each have
* separate count. Does not count the spurious errors found between
* an error and next valid token match
*
* See also reportError()
*/
public int getNumberOfSyntaxErrors() {
return state.syntaxErrors;
}
/** What is the error header, normally line/character position information? */
public String getErrorHeader(RecognitionException e) {
return "line "+e.line+":"+e.charPositionInLine;
}
/** How should a token be displayed in an error message? The default
* is to display just the text, but during development you might
* want to have a lot of information spit out. Override in that case
* to use t.toString() (which, for CommonToken, dumps everything about
* the token). This is better than forcing you to override a method in
* your token objects because you don't have to go modify your lexer
* so that it creates a new Java type.
*/
public String getTokenErrorDisplay(Token t) {
String s = t.getText();
if ( s==null ) {
if ( t.getType()==Token.EOF ) {
s = "";
}
else {
s = "<"+t.getType()+">";
}
}
s = s.replaceAll("\n","\\\\n");
s = s.replaceAll("\r","\\\\r");
s = s.replaceAll("\t","\\\\t");
return "'"+s+"'";
}
/** Override this method to change where error messages go */
public void emitErrorMessage(String msg) {
System.err.println(msg);
}
/** Recover from an error found on the input stream. This is
* for NoViableAlt and mismatched symbol exceptions. If you enable
* single token insertion and deletion, this will usually not
* handle mismatched symbol exceptions but there could be a mismatched
* token that the match() routine could not recover from.
*/
public void recover(IntStream input, RecognitionException re) {
if ( state.lastErrorIndex==input.index() ) {
// uh oh, another error at same token index; must be a case
// where LT(1) is in the recovery token set so nothing is
// consumed; consume a single token so at least to prevent
// an infinite loop; this is a failsafe.
input.consume();
}
state.lastErrorIndex = input.index();
BitSet followSet = computeErrorRecoverySet();
beginResync();
consumeUntil(input, followSet);
endResync();
}
/** A hook to listen in on the token consumption during error recovery.
* The DebugParser subclasses this to fire events to the listenter.
*/
public void beginResync() {
}
public void endResync() {
}
/* Compute the error recovery set for the current rule. During
* rule invocation, the parser pushes the set of tokens that can
* follow that rule reference on the stack; this amounts to
* computing FIRST of what follows the rule reference in the
* enclosing rule. This local follow set only includes tokens
* from within the rule; i.e., the FIRST computation done by
* ANTLR stops at the end of a rule.
*
* EXAMPLE
*
* When you find a "no viable alt exception", the input is not
* consistent with any of the alternatives for rule r. The best
* thing to do is to consume tokens until you see something that
* can legally follow a call to r *or* any rule that called r.
* You don't want the exact set of viable next tokens because the
* input might just be missing a token--you might consume the
* rest of the input looking for one of the missing tokens.
*
* Consider grammar:
*
* a : '[' b ']'
* | '(' b ')'
* ;
* b : c '^' INT ;
* c : ID
* | INT
* ;
*
* At each rule invocation, the set of tokens that could follow
* that rule is pushed on a stack. Here are the various "local"
* follow sets:
*
* FOLLOW(b1_in_a) = FIRST(']') = ']'
* FOLLOW(b2_in_a) = FIRST(')') = ')'
* FOLLOW(c_in_b) = FIRST('^') = '^'
*
* Upon erroneous input "[]", the call chain is
*
* a -> b -> c
*
* and, hence, the follow context stack is:
*
* depth local follow set after call to rule
* 0 a (from main())
* 1 ']' b
* 3 '^' c
*
* Notice that ')' is not included, because b would have to have
* been called from a different context in rule a for ')' to be
* included.
*
* For error recovery, we cannot consider FOLLOW(c)
* (context-sensitive or otherwise). We need the combined set of
* all context-sensitive FOLLOW sets--the set of all tokens that
* could follow any reference in the call chain. We need to
* resync to one of those tokens. Note that FOLLOW(c)='^' and if
* we resync'd to that token, we'd consume until EOF. We need to
* sync to context-sensitive FOLLOWs for a, b, and c: {']','^'}.
* In this case, for input "[]", LA(1) is in this set so we would
* not consume anything and after printing an error rule c would
* return normally. It would not find the required '^' though.
* At this point, it gets a mismatched token error and throws an
* exception (since LA(1) is not in the viable following token
* set). The rule exception handler tries to recover, but finds
* the same recovery set and doesn't consume anything. Rule b
* exits normally returning to rule a. Now it finds the ']' (and
* with the successful match exits errorRecovery mode).
*
* So, you cna see that the parser walks up call chain looking
* for the token that was a member of the recovery set.
*
* Errors are not generated in errorRecovery mode.
*
* ANTLR's error recovery mechanism is based upon original ideas:
*
* "Algorithms + Data Structures = Programs" by Niklaus Wirth
*
* and
*
* "A note on error recovery in recursive descent parsers":
* http://portal.acm.org/citation.cfm?id=947902.947905
*
* Later, Josef Grosch had some good ideas:
*
* "Efficient and Comfortable Error Recovery in Recursive Descent
* Parsers":
* ftp://www.cocolab.com/products/cocktail/doca4.ps/ell.ps.zip
*
* Like Grosch I implemented local FOLLOW sets that are combined
* at run-time upon error to avoid overhead during parsing.
*/
protected BitSet computeErrorRecoverySet() {
return combineFollows(false);
}
/** Compute the context-sensitive FOLLOW set for current rule.
* This is set of token types that can follow a specific rule
* reference given a specific call chain. You get the set of
* viable tokens that can possibly come next (lookahead depth 1)
* given the current call chain. Contrast this with the
* definition of plain FOLLOW for rule r:
*
* FOLLOW(r)={x | S=>*alpha r beta in G and x in FIRST(beta)}
*
* where x in T* and alpha, beta in V*; T is set of terminals and
* V is the set of terminals and nonterminals. In other words,
* FOLLOW(r) is the set of all tokens that can possibly follow
* references to r in *any* sentential form (context). At
* runtime, however, we know precisely which context applies as
* we have the call chain. We may compute the exact (rather
* than covering superset) set of following tokens.
*
* For example, consider grammar:
*
* stat : ID '=' expr ';' // FOLLOW(stat)=={EOF}
* | "return" expr '.'
* ;
* expr : atom ('+' atom)* ; // FOLLOW(expr)=={';','.',')'}
* atom : INT // FOLLOW(atom)=={'+',')',';','.'}
* | '(' expr ')'
* ;
*
* The FOLLOW sets are all inclusive whereas context-sensitive
* FOLLOW sets are precisely what could follow a rule reference.
* For input input "i=(3);", here is the derivation:
*
* stat => ID '=' expr ';'
* => ID '=' atom ('+' atom)* ';'
* => ID '=' '(' expr ')' ('+' atom)* ';'
* => ID '=' '(' atom ')' ('+' atom)* ';'
* => ID '=' '(' INT ')' ('+' atom)* ';'
* => ID '=' '(' INT ')' ';'
*
* At the "3" token, you'd have a call chain of
*
* stat -> expr -> atom -> expr -> atom
*
* What can follow that specific nested ref to atom? Exactly ')'
* as you can see by looking at the derivation of this specific
* input. Contrast this with the FOLLOW(atom)={'+',')',';','.'}.
*
* You want the exact viable token set when recovering from a
* token mismatch. Upon token mismatch, if LA(1) is member of
* the viable next token set, then you know there is most likely
* a missing token in the input stream. "Insert" one by just not
* throwing an exception.
*/
protected BitSet computeContextSensitiveRuleFOLLOW() {
return combineFollows(true);
}
protected BitSet combineFollows(boolean exact) {
int top = state._fsp;
BitSet followSet = new BitSet();
for (int i=top; i>=0; i--) {
BitSet localFollowSet = (BitSet)state.following[i];
/*
System.out.println("local follow depth "+i+"="+
localFollowSet.toString(getTokenNames())+")");
*/
followSet.orInPlace(localFollowSet);
if ( exact ) {
// can we see end of rule?
if ( localFollowSet.member(Token.EOR_TOKEN_TYPE) ) {
// Only leave EOR in set if at top (start rule); this lets
// us know if have to include follow(start rule); i.e., EOF
if ( i>0 ) {
followSet.remove(Token.EOR_TOKEN_TYPE);
}
}
else { // can't see end of rule, quit
break;
}
}
}
return followSet;
}
/** Attempt to recover from a single missing or extra token.
*
* EXTRA TOKEN
*
* LA(1) is not what we are looking for. If LA(2) has the right token,
* however, then assume LA(1) is some extra spurious token. Delete it
* and LA(2) as if we were doing a normal match(), which advances the
* input.
*
* MISSING TOKEN
*
* If current token is consistent with what could come after
* ttype then it is ok to "insert" the missing token, else throw
* exception For example, Input "i=(3;" is clearly missing the
* ')'. When the parser returns from the nested call to expr, it
* will have call chain:
*
* stat -> expr -> atom
*
* and it will be trying to match the ')' at this point in the
* derivation:
*
* => ID '=' '(' INT ')' ('+' atom)* ';'
* ^
* match() will see that ';' doesn't match ')' and report a
* mismatched token error. To recover, it sees that LA(1)==';'
* is in the set of tokens that can follow the ')' token
* reference in rule atom. It can assume that you forgot the ')'.
*/
protected Object recoverFromMismatchedToken(IntStream input, int ttype, BitSet follow)
throws RecognitionException
{
RecognitionException e = null;
// if next token is what we are looking for then "delete" this token
if ( mismatchIsUnwantedToken(input, ttype) ) {
e = new UnwantedTokenException(ttype, input);
/*
System.err.println("recoverFromMismatchedToken deleting "+
((TokenStream)input).LT(1)+
" since "+((TokenStream)input).LT(2)+" is what we want");
*/
beginResync();
input.consume(); // simply delete extra token
endResync();
reportError(e); // report after consuming so AW sees the token in the exception
// we want to return the token we're actually matching
Object matchedSymbol = getCurrentInputSymbol(input);
input.consume(); // move past ttype token as if all were ok
return matchedSymbol;
}
// can't recover with single token deletion, try insertion
if ( mismatchIsMissingToken(input, follow) ) {
Object inserted = getMissingSymbol(input, e, ttype, follow);
e = new MissingTokenException(ttype, input, inserted);
reportError(e); // report after inserting so AW sees the token in the exception
return inserted;
}
// even that didn't work; must throw the exception
e = new MismatchedTokenException(ttype, input);
throw e;
}
/** Not currently used */
public Object recoverFromMismatchedSet(IntStream input,
RecognitionException e,
BitSet follow)
throws RecognitionException
{
if ( mismatchIsMissingToken(input, follow) ) {
// System.out.println("missing token");
reportError(e);
// we don't know how to conjure up a token for sets yet
return getMissingSymbol(input, e, Token.INVALID_TOKEN_TYPE, follow);
}
// TODO do single token deletion like above for Token mismatch
throw e;
}
/** Match needs to return the current input symbol, which gets put
* into the label for the associated token ref; e.g., x=ID. Token
* and tree parsers need to return different objects. Rather than test
* for input stream type or change the IntStream interface, I use
* a simple method to ask the recognizer to tell me what the current
* input symbol is.
*
* This is ignored for lexers.
*/
protected Object getCurrentInputSymbol(IntStream input) { return null; }
/** Conjure up a missing token during error recovery.
*
* The recognizer attempts to recover from single missing
* symbols. But, actions might refer to that missing symbol.
* For example, x=ID {f($x);}. The action clearly assumes
* that there has been an identifier matched previously and that
* $x points at that token. If that token is missing, but
* the next token in the stream is what we want we assume that
* this token is missing and we keep going. Because we
* have to return some token to replace the missing token,
* we have to conjure one up. This method gives the user control
* over the tokens returned for missing tokens. Mostly,
* you will want to create something special for identifier
* tokens. For literals such as '{' and ',', the default
* action in the parser or tree parser works. It simply creates
* a CommonToken of the appropriate type. The text will be the token.
* If you change what tokens must be created by the lexer,
* override this method to create the appropriate tokens.
*/
protected Object getMissingSymbol(IntStream input,
RecognitionException e,
int expectedTokenType,
BitSet follow)
{
return null;
}
public void consumeUntil(IntStream input, int tokenType) {
//System.out.println("consumeUntil "+tokenType);
int ttype = input.LA(1);
while (ttype != Token.EOF && ttype != tokenType) {
input.consume();
ttype = input.LA(1);
}
}
/** Consume tokens until one matches the given token set */
public void consumeUntil(IntStream input, BitSet set) {
//System.out.println("consumeUntil("+set.toString(getTokenNames())+")");
int ttype = input.LA(1);
while (ttype != Token.EOF && !set.member(ttype) ) {
//System.out.println("consume during recover LA(1)="+getTokenNames()[input.LA(1)]);
input.consume();
ttype = input.LA(1);
}
}
/** Push a rule's follow set using our own hardcoded stack */
protected void pushFollow(BitSet fset) {
if ( (state._fsp +1)>=state.following.length ) {
BitSet[] f = new BitSet[state.following.length*2];
System.arraycopy(state.following, 0, f, 0, state.following.length);
state.following = f;
}
state.following[++state._fsp] = fset;
}
/** Return List of the rules in your parser instance
* leading up to a call to this method. You could override if
* you want more details such as the file/line info of where
* in the parser java code a rule is invoked.
*
* This is very useful for error messages and for context-sensitive
* error recovery.
*/
public List getRuleInvocationStack() {
String parserClassName = getClass().getName();
return getRuleInvocationStack(new Throwable(), parserClassName);
}
/** A more general version of getRuleInvocationStack where you can
* pass in, for example, a RecognitionException to get it's rule
* stack trace. This routine is shared with all recognizers, hence,
* static.
*
* TODO: move to a utility class or something; weird having lexer call this
*/
public static List getRuleInvocationStack(Throwable e,
String recognizerClassName)
{
List rules = new ArrayList();
StackTraceElement[] stack = e.getStackTrace();
int i = 0;
for (i=stack.length-1; i>=0; i--) {
StackTraceElement t = stack[i];
if ( t.getClassName().startsWith("org.antlr.runtime.") ) {
continue; // skip support code such as this method
}
if ( t.getMethodName().equals(NEXT_TOKEN_RULE_NAME) ) {
continue;
}
if ( !t.getClassName().equals(recognizerClassName) ) {
continue; // must not be part of this parser
}
rules.add(t.getMethodName());
}
return rules;
}
public int getBacktrackingLevel() { return state.backtracking; }
public void setBacktrackingLevel(int n) { state.backtracking = n; }
/** Return whether or not a backtracking attempt failed. */
public boolean failed() { return state.failed; }
/** Used to print out token names like ID during debugging and
* error reporting. The generated parsers implement a method
* that overrides this to point to their String[] tokenNames.
*/
public String[] getTokenNames() {
return null;
}
/** For debugging and other purposes, might want the grammar name.
* Have ANTLR generate an implementation for this method.
*/
public String getGrammarFileName() {
return null;
}
public abstract String getSourceName();
/** A convenience method for use most often with template rewrites.
* Convert a List to List
*/
public List toStrings(List tokens) {
if ( tokens==null ) return null;
List strings = new ArrayList(tokens.size());
for (int i=0; i= state.ruleMemo.length ) {
System.err.println("!!!!!!!!! memo size is "+state.ruleMemo.length+", but rule index is "+ruleIndex);
}
if ( state.ruleMemo[ruleIndex]!=null ) {
state.ruleMemo[ruleIndex].put(
new Integer(ruleStartIndex), new Integer(stopTokenIndex)
);
}
}
/** return how many rule/input-index pairs there are in total.
* TODO: this includes synpreds. :(
*/
public int getRuleMemoizationCacheSize() {
int n = 0;
for (int i = 0; state.ruleMemo!=null && i < state.ruleMemo.length; i++) {
Map ruleMap = state.ruleMemo[i];
if ( ruleMap!=null ) {
n += ruleMap.size(); // how many input indexes are recorded?
}
}
return n;
}
public void traceIn(String ruleName, int ruleIndex, Object inputSymbol) {
System.out.print("enter "+ruleName+" "+inputSymbol);
if ( state.backtracking>0 ) {
System.out.print(" backtracking="+state.backtracking);
}
System.out.println();
}
public void traceOut(String ruleName,
int ruleIndex,
Object inputSymbol)
{
System.out.print("exit "+ruleName+" "+inputSymbol);
if ( state.backtracking>0 ) {
System.out.print(" backtracking="+state.backtracking);
if ( state.failed ) System.out.print(" failed");
else System.out.print(" succeeded");
}
System.out.println();
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/EarlyExitException.java 0000644 0001750 0001750 00000003460 11256465214 027467 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** The recognizer did not match anything for a (..)+ loop. */
public class EarlyExitException extends RecognitionException {
public int decisionNumber;
/** Used for remote debugger deserialization */
public EarlyExitException() {;}
public EarlyExitException(int decisionNumber, IntStream input) {
super(input);
this.decisionNumber = decisionNumber;
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/MismatchedTokenException.java 0000644 0001750 0001750 00000003644 11256465214 030644 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A mismatched char or Token or tree node */
public class MismatchedTokenException extends RecognitionException {
public int expecting = Token.INVALID_TOKEN_TYPE;
/** Used for remote debugger deserialization */
public MismatchedTokenException() {;}
public MismatchedTokenException(int expecting, IntStream input) {
super(input);
this.expecting = expecting;
}
public String toString() {
return "MismatchedTokenException("+getUnexpectedType()+"!="+expecting+")";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/Parser.java 0000644 0001750 0001750 00000006416 11256465214 025142 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** A parser for TokenStreams. "parser grammars" result in a subclass
* of this.
*/
public class Parser extends BaseRecognizer {
public TokenStream input;
public Parser(TokenStream input) {
super(); // highlight that we go to super to set state object
setTokenStream(input);
}
public Parser(TokenStream input, RecognizerSharedState state) {
super(state); // share the state object with another parser
setTokenStream(input);
}
public void reset() {
super.reset(); // reset all recognizer state variables
if ( input!=null ) {
input.seek(0); // rewind the input
}
}
protected Object getCurrentInputSymbol(IntStream input) {
return ((TokenStream)input).LT(1);
}
protected Object getMissingSymbol(IntStream input,
RecognitionException e,
int expectedTokenType,
BitSet follow)
{
String tokenText = null;
if ( expectedTokenType==Token.EOF ) tokenText = "";
else tokenText = "";
CommonToken t = new CommonToken(expectedTokenType, tokenText);
Token current = ((TokenStream)input).LT(1);
if ( current.getType() == Token.EOF ) {
current = ((TokenStream)input).LT(-1);
}
t.line = current.getLine();
t.charPositionInLine = current.getCharPositionInLine();
t.channel = DEFAULT_TOKEN_CHANNEL;
return t;
}
/** Set the token stream and reset the parser */
public void setTokenStream(TokenStream input) {
this.input = null;
reset();
this.input = input;
}
public TokenStream getTokenStream() {
return input;
}
public String getSourceName() {
return input.getSourceName();
}
public void traceIn(String ruleName, int ruleIndex) {
super.traceIn(ruleName, ruleIndex, input.LT(1));
}
public void traceOut(String ruleName, int ruleIndex) {
super.traceOut(ruleName, ruleIndex, input.LT(1));
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/UnwantedTokenException.java 0000644 0001750 0001750 00000004110 11256465214 030340 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
/** An extra token while parsing a TokenStream */
public class UnwantedTokenException extends MismatchedTokenException {
/** Used for remote debugger deserialization */
public UnwantedTokenException() {;}
public UnwantedTokenException(int expecting, IntStream input) {
super(expecting, input);
}
public Token getUnexpectedToken() {
return token;
}
public String toString() {
String exp = ", expected "+expecting;
if ( expecting==Token.INVALID_TOKEN_TYPE ) {
exp = "";
}
if ( token==null ) {
return "UnwantedTokenException(found="+null+exp+")";
}
return "UnwantedTokenException(found="+token.getText()+exp+")";
}
}
antlr-3.2/runtime/Java/src/main/java/org/antlr/runtime/ANTLRInputStream.java 0000644 0001750 0001750 00000004733 11256465214 026762 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.runtime;
import java.io.*;
/** A kind of ReaderStream that pulls from an InputStream.
* Useful for reading from stdin and specifying file encodings etc...
*/
public class ANTLRInputStream extends ANTLRReaderStream {
public ANTLRInputStream() {
}
public ANTLRInputStream(InputStream input) throws IOException {
this(input, null);
}
public ANTLRInputStream(InputStream input, int size) throws IOException {
this(input, size, null);
}
public ANTLRInputStream(InputStream input, String encoding) throws IOException {
this(input, INITIAL_BUFFER_SIZE, encoding);
}
public ANTLRInputStream(InputStream input, int size, String encoding) throws IOException {
this(input, size, READ_BUFFER_SIZE, encoding);
}
public ANTLRInputStream(InputStream input,
int size,
int readBufferSize,
String encoding)
throws IOException
{
InputStreamReader isr;
if ( encoding!=null ) {
isr = new InputStreamReader(input, encoding);
}
else {
isr = new InputStreamReader(input);
}
load(isr, size, readBufferSize);
}
}
antlr-3.2/runtime/Java/doxyfile 0000644 0001750 0001750 00000023235 11256465214 016561 0 ustar twerner twerner # Doxyfile 1.5.2
#---------------------------------------------------------------------------
# Project related configuration options
#---------------------------------------------------------------------------
DOXYFILE_ENCODING = UTF-8
PROJECT_NAME = "ANTLR API"
PROJECT_NUMBER = 3.1.2
OUTPUT_DIRECTORY = api
CREATE_SUBDIRS = NO
OUTPUT_LANGUAGE = English
BRIEF_MEMBER_DESC = YES
REPEAT_BRIEF = YES
ABBREVIATE_BRIEF = "The $name class" \
"The $name widget" \
"The $name file" \
is \
provides \
specifies \
contains \
represents \
a \
an \
the
ALWAYS_DETAILED_SEC = YES
INLINE_INHERITED_MEMB = NO
FULL_PATH_NAMES = YES
STRIP_FROM_PATH = /Applications/
STRIP_FROM_INC_PATH =
SHORT_NAMES = NO
JAVADOC_AUTOBRIEF = NO
MULTILINE_CPP_IS_BRIEF = NO
DETAILS_AT_TOP = NO
INHERIT_DOCS = YES
SEPARATE_MEMBER_PAGES = NO
TAB_SIZE = 8
ALIASES =
OPTIMIZE_OUTPUT_FOR_C = NO
OPTIMIZE_OUTPUT_JAVA = YES
BUILTIN_STL_SUPPORT = NO
CPP_CLI_SUPPORT = NO
DISTRIBUTE_GROUP_DOC = NO
SUBGROUPING = YES
#---------------------------------------------------------------------------
# Build related configuration options
#---------------------------------------------------------------------------
EXTRACT_ALL = YES
EXTRACT_PRIVATE = YES
EXTRACT_STATIC = YES
EXTRACT_LOCAL_CLASSES = YES
EXTRACT_LOCAL_METHODS = NO
HIDE_UNDOC_MEMBERS = NO
HIDE_UNDOC_CLASSES = NO
HIDE_FRIEND_COMPOUNDS = NO
HIDE_IN_BODY_DOCS = NO
INTERNAL_DOCS = NO
CASE_SENSE_NAMES = NO
HIDE_SCOPE_NAMES = NO
SHOW_INCLUDE_FILES = YES
INLINE_INFO = YES
SORT_MEMBER_DOCS = YES
SORT_BRIEF_DOCS = NO
SORT_BY_SCOPE_NAME = NO
GENERATE_TODOLIST = YES
GENERATE_TESTLIST = NO
GENERATE_BUGLIST = NO
GENERATE_DEPRECATEDLIST= NO
ENABLED_SECTIONS =
MAX_INITIALIZER_LINES = 30
SHOW_USED_FILES = YES
SHOW_DIRECTORIES = NO
FILE_VERSION_FILTER =
#---------------------------------------------------------------------------
# configuration options related to warning and progress messages
#---------------------------------------------------------------------------
QUIET = NO
WARNINGS = YES
WARN_IF_UNDOCUMENTED = YES
WARN_IF_DOC_ERROR = YES
WARN_NO_PARAMDOC = NO
WARN_FORMAT = "$file:$line: $text"
WARN_LOGFILE =
#---------------------------------------------------------------------------
# configuration options related to the input files
#---------------------------------------------------------------------------
INPUT = /Users/parrt/antlr/code/antlr/main/runtime/Java/src
INPUT_ENCODING = UTF-8
FILE_PATTERNS = *.c \
*.cc \
*.cxx \
*.cpp \
*.c++ \
*.d \
*.java \
*.ii \
*.ixx \
*.ipp \
*.i++ \
*.inl \
*.h \
*.hh \
*.hxx \
*.hpp \
*.h++ \
*.idl \
*.odl \
*.cs \
*.php \
*.php3 \
*.inc \
*.m \
*.mm \
*.dox \
*.py
RECURSIVE = YES
EXCLUDE =
EXCLUDE_SYMLINKS = NO
EXCLUDE_PATTERNS =
EXCLUDE_SYMBOLS = java::util \
java::io
EXAMPLE_PATH =
EXAMPLE_PATTERNS = *
EXAMPLE_RECURSIVE = NO
IMAGE_PATH =
INPUT_FILTER =
FILTER_PATTERNS =
FILTER_SOURCE_FILES = NO
#---------------------------------------------------------------------------
# configuration options related to source browsing
#---------------------------------------------------------------------------
SOURCE_BROWSER = YES
INLINE_SOURCES = NO
STRIP_CODE_COMMENTS = YES
REFERENCED_BY_RELATION = NO
REFERENCES_RELATION = NO
REFERENCES_LINK_SOURCE = YES
USE_HTAGS = NO
VERBATIM_HEADERS = YES
#---------------------------------------------------------------------------
# configuration options related to the alphabetical class index
#---------------------------------------------------------------------------
ALPHABETICAL_INDEX = NO
COLS_IN_ALPHA_INDEX = 5
IGNORE_PREFIX =
#---------------------------------------------------------------------------
# configuration options related to the HTML output
#---------------------------------------------------------------------------
GENERATE_HTML = YES
HTML_OUTPUT = .
HTML_FILE_EXTENSION = .html
HTML_HEADER =
HTML_FOOTER =
HTML_STYLESHEET =
HTML_ALIGN_MEMBERS = YES
GENERATE_HTMLHELP = NO
CHM_FILE =
HHC_LOCATION =
GENERATE_CHI = NO
BINARY_TOC = NO
TOC_EXPAND = NO
DISABLE_INDEX = NO
ENUM_VALUES_PER_LINE = 4
GENERATE_TREEVIEW = NO
TREEVIEW_WIDTH = 250
#---------------------------------------------------------------------------
# configuration options related to the LaTeX output
#---------------------------------------------------------------------------
GENERATE_LATEX = NO
LATEX_OUTPUT = latex
LATEX_CMD_NAME = latex
MAKEINDEX_CMD_NAME = makeindex
COMPACT_LATEX = NO
PAPER_TYPE = a4wide
EXTRA_PACKAGES =
LATEX_HEADER =
PDF_HYPERLINKS = NO
USE_PDFLATEX = YES
LATEX_BATCHMODE = NO
LATEX_HIDE_INDICES = NO
#---------------------------------------------------------------------------
# configuration options related to the RTF output
#---------------------------------------------------------------------------
GENERATE_RTF = NO
RTF_OUTPUT = rtf
COMPACT_RTF = NO
RTF_HYPERLINKS = NO
RTF_STYLESHEET_FILE =
RTF_EXTENSIONS_FILE =
#---------------------------------------------------------------------------
# configuration options related to the man page output
#---------------------------------------------------------------------------
GENERATE_MAN = NO
MAN_OUTPUT = man
MAN_EXTENSION = .3
MAN_LINKS = NO
#---------------------------------------------------------------------------
# configuration options related to the XML output
#---------------------------------------------------------------------------
GENERATE_XML = NO
XML_OUTPUT = xml
XML_SCHEMA =
XML_DTD =
XML_PROGRAMLISTING = YES
#---------------------------------------------------------------------------
# configuration options for the AutoGen Definitions output
#---------------------------------------------------------------------------
GENERATE_AUTOGEN_DEF = NO
#---------------------------------------------------------------------------
# configuration options related to the Perl module output
#---------------------------------------------------------------------------
GENERATE_PERLMOD = NO
PERLMOD_LATEX = NO
PERLMOD_PRETTY = YES
PERLMOD_MAKEVAR_PREFIX =
#---------------------------------------------------------------------------
# Configuration options related to the preprocessor
#---------------------------------------------------------------------------
ENABLE_PREPROCESSING = YES
MACRO_EXPANSION = NO
EXPAND_ONLY_PREDEF = NO
SEARCH_INCLUDES = YES
INCLUDE_PATH =
INCLUDE_FILE_PATTERNS =
PREDEFINED =
EXPAND_AS_DEFINED =
SKIP_FUNCTION_MACROS = YES
#---------------------------------------------------------------------------
# Configuration::additions related to external references
#---------------------------------------------------------------------------
TAGFILES =
GENERATE_TAGFILE =
ALLEXTERNALS = NO
EXTERNAL_GROUPS = YES
PERL_PATH = /usr/bin/perl
#---------------------------------------------------------------------------
# Configuration options related to the dot tool
#---------------------------------------------------------------------------
CLASS_DIAGRAMS = NO
MSCGEN_PATH = /Applications/Doxygen.app/Contents/Resources/
HIDE_UNDOC_RELATIONS = YES
HAVE_DOT = YES
CLASS_GRAPH = YES
COLLABORATION_GRAPH = YES
GROUP_GRAPHS = YES
UML_LOOK = NO
TEMPLATE_RELATIONS = NO
INCLUDE_GRAPH = YES
INCLUDED_BY_GRAPH = YES
CALL_GRAPH = NO
CALLER_GRAPH = NO
GRAPHICAL_HIERARCHY = YES
DIRECTORY_GRAPH = YES
DOT_IMAGE_FORMAT = png
DOT_PATH = /Applications/Doxygen.app/Contents/Resources/
DOTFILE_DIRS =
DOT_GRAPH_MAX_NODES = 50
DOT_TRANSPARENT = NO
DOT_MULTI_TARGETS = NO
GENERATE_LEGEND = YES
DOT_CLEANUP = YES
#---------------------------------------------------------------------------
# Configuration::additions related to the search engine
#---------------------------------------------------------------------------
SEARCHENGINE = NO
antlr-3.2/runtime/Java/pom.xml 0000644 0001750 0001750 00000007721 11256466764 016345 0 ustar twerner twerner
4.0.0
org.antlr
antlr-runtime
jar
org.antlr
antlr-master
3.2
Antlr 3 Runtime
A framework for constructing recognizers, compilers, and translators from grammatical descriptions containing Java, C#, C++, or Python actions.
http://www.antlr.org
Terence Parr
USFCA
http://www.cs.usfca.edu
parrt@antlr.org
Project Leader
Developer - Java Target
PST
Jim Idle
Temporal Wave LLC
http://www.temporal-wave.com
jimi@temporal-wave.com
Developer - Maven stuff
Developer - C Target
PST
http://fisheye2.cenqua.com/browse/antlr
http://fisheye2.cenqua.com/browse/antlr
antlr-repo
ANTLR Testing repository
scpexe://antlr.org/home/mavensync/antlr-repo
antlr-snapshot
ANTLR Testing Snapshot Repository
scpexe://antlr.org/home/mavensync/antlr-snapshot
org.antlr
stringtemplate
3.2
compile
install
org.apache.maven.wagon
wagon-ssh-external
1.0-beta-2
maven-compiler-plugin
2.0.2
1.5
jsr14
org.codehaus.mojo
findbugs-maven-plugin
true
true
true
antlr-3.2/gunit/ 0000755 0001750 0001750 00000000000 11410174107 013556 5 ustar twerner twerner antlr-3.2/gunit/LICENSE.txt 0000644 0001750 0001750 00000002625 11256465172 015423 0 ustar twerner twerner [The "BSD licence"]
Copyright (c) 2007-2008 Leon, Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
antlr-3.2/gunit/README.txt 0000644 0001750 0001750 00000003551 11256465172 015275 0 ustar twerner twerner gUnit 1.0.5
Feb 21, 2009
Leon, Jen-Yuan Su
leonsu at mac com
INTRODUCTION
Welcome to gUnit! I've been working on gUnit from 2007 summer and
this is a project in USF CS, sponsored by professor Terence.
You should use the latest ANTLR v3.1 with gUnit:
http://www.antlr.org/download.html
See the wiki document:
http://www.antlr.org/wiki/display/ANTLR3/gUnit+-+Grammar+Unit+Testing
Per the license in LICENSE.txt, this software is not guaranteed to
work and might even destroy all life on this planet:
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
----------------------------------------------------------------------
EXAMPLES
See the wiki tutorial of gUnit:
http://www.antlr.org/wiki/display/ANTLR3/gUnit+-+Grammar+Unit+Testing
----------------------------------------------------------------------
What is gUnit?
gUnit is an unit testing framework for ANTLR grammars. It provides a
simple way to write and run automated tests for grammars in a manner
similar to what jUnit does for unit testing.
----------------------------------------------------------------------
How do I install gUnit?
Just add gunit-1.0.5.jar to your CLASSPATH, and also make sure that
both ANTLR and StringTemplate jars lie in CLASSPATH.
antlr-3.2/gunit/antlr.config 0000644 0001750 0001750 00000000000 11256465172 016070 0 ustar twerner twerner antlr-3.2/gunit/src/ 0000755 0001750 0001750 00000000000 11410174107 014345 5 ustar twerner twerner antlr-3.2/gunit/src/test/ 0000755 0001750 0001750 00000000000 11410174107 015324 5 ustar twerner twerner antlr-3.2/gunit/src/test/java/ 0000755 0001750 0001750 00000000000 11410174107 016245 5 ustar twerner twerner antlr-3.2/gunit/src/test/java/org/ 0000755 0001750 0001750 00000000000 11410174107 017034 5 ustar twerner twerner antlr-3.2/gunit/src/test/java/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 020154 5 ustar twerner twerner antlr-3.2/gunit/src/test/java/org/antlr/gunit/ 0000755 0001750 0001750 00000000000 11410174107 021302 5 ustar twerner twerner antlr-3.2/gunit/src/test/java/org/antlr/gunit/GunitTest.java 0000644 0001750 0001750 00000001223 11256465172 024106 0 ustar twerner twerner package org.antlr.gunit;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
/**
* Unit test for gUnit itself....
*/
public class GunitTest
extends TestCase
{
/**
* Create the test case
*
* @param testName name of the test case
*/
public GunitTest( String testName )
{
super( testName );
}
/**
* @return the suite of tests being tested
*/
public static Test suite()
{
return new TestSuite( GunitTest.class );
}
/**
* Rigourous Test :-)
*/
public void testGunitTest()
{
assertTrue( true );
}
}
antlr-3.2/gunit/src/main/ 0000755 0001750 0001750 00000000000 11410174107 015271 5 ustar twerner twerner antlr-3.2/gunit/src/main/antlr3/ 0000755 0001750 0001750 00000000000 11410174107 016474 5 ustar twerner twerner antlr-3.2/gunit/src/main/antlr3/org/ 0000755 0001750 0001750 00000000000 11410174107 017263 5 ustar twerner twerner antlr-3.2/gunit/src/main/antlr3/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 020403 5 ustar twerner twerner antlr-3.2/gunit/src/main/antlr3/org/antlr/gunit/ 0000755 0001750 0001750 00000000000 11410174107 021531 5 ustar twerner twerner antlr-3.2/gunit/src/main/antlr3/org/antlr/gunit/swingui/ 0000755 0001750 0001750 00000000000 11410174107 023216 5 ustar twerner twerner antlr-3.2/gunit/src/main/antlr3/org/antlr/gunit/swingui/parsers/ 0000755 0001750 0001750 00000000000 11410174107 024675 5 ustar twerner twerner antlr-3.2/gunit/src/main/antlr3/org/antlr/gunit/swingui/parsers/StGUnit.g 0000644 0001750 0001750 00000010056 11256465172 026421 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
grammar StGUnit;
options {language=Java;}
tokens {
OK = 'OK';
FAIL = 'FAIL';
DOC_COMMENT;
}
@header {
package org.antlr.gunit.swingui.parsers;
import org.antlr.gunit.swingui.model.*;
import org.antlr.gunit.swingui.runner.*;
}
@lexer::header {package org.antlr.gunit.swingui.parsers;}
@members {
public TestSuiteAdapter adapter ;;
}
gUnitDef
: 'gunit' name=id {adapter.setGrammarName($name.text);}
('walks' id)? ';'
header? suite*
;
header
: '@header' ACTION
;
suite
: ( parserRule=RULE_REF ('walks' RULE_REF)?
{adapter.startRule($parserRule.text);}
| lexerRule=TOKEN_REF
{adapter.startRule($lexerRule.text);}
)
':'
test+
{adapter.endRule();}
;
test
: input expect
{adapter.addTestCase($input.in, $expect.out);}
;
expect returns [ITestCaseOutput out]
: OK {$out = adapter.createBoolOutput(true);}
| FAIL {$out = adapter.createBoolOutput(false);}
| 'returns' RETVAL {$out = adapter.createReturnOutput($RETVAL.text);}
| '->' output {$out = adapter.createStdOutput($output.text);}
| '->' AST {$out = adapter.createAstOutput($AST.text);}
;
input returns [ITestCaseInput in]
: STRING {$in = adapter.createStringInput($STRING.text);}
| ML_STRING {$in = adapter.createMultiInput($ML_STRING.text);}
| fileInput {$in = adapter.createFileInput($fileInput.path);}
;
output
: STRING
| ML_STRING
| ACTION
;
fileInput returns [String path]
: id {$path = $id.text;} (EXT {$path += $EXT.text;})?
;
id : TOKEN_REF
| RULE_REF
;
// L E X I C A L R U L E S
SL_COMMENT
: '//' ~('\r'|'\n')* '\r'? '\n' {$channel=HIDDEN;}
;
ML_COMMENT
: '/*' {$channel=HIDDEN;} .* '*/'
;
STRING
: '"' ( ESC | ~('\\'|'"') )* '"'
;
ML_STRING
: '<<' .* '>>'
;
TOKEN_REF
: 'A'..'Z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
RULE_REF
: 'a'..'z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
EXT : '.'('a'..'z'|'A'..'Z'|'0'..'9')+;
RETVAL : NESTED_RETVAL
;
fragment
NESTED_RETVAL :
'['
( options {greedy=false;}
: NESTED_RETVAL
| .
)*
']'
;
AST : NESTED_AST (' '? NESTED_AST)*;
fragment
NESTED_AST :
'('
( options {greedy=false;}
: NESTED_AST
| .
)*
')'
;
ACTION
: NESTED_ACTION
;
fragment
NESTED_ACTION :
'{'
( options {greedy=false; k=3;}
: NESTED_ACTION
| STRING_LITERAL
| CHAR_LITERAL
| .
)*
'}'
;
fragment
CHAR_LITERAL
: '\'' ( ESC | ~('\''|'\\') ) '\''
;
fragment
STRING_LITERAL
: '"' ( ESC | ~('\\'|'"') )* '"'
;
fragment
ESC : '\\'
( 'n'
| 'r'
| 't'
| 'b'
| 'f'
| '"'
| '\''
| '\\'
| '>'
| 'u' XDIGIT XDIGIT XDIGIT XDIGIT
| . // unknown, leave as it is
)
;
fragment
XDIGIT :
'0' .. '9'
| 'a' .. 'f'
| 'A' .. 'F'
;
WS : ( ' '
| '\t'
| '\r'? '\n'
)+
{$channel=HIDDEN;}
;
antlr-3.2/gunit/src/main/antlr3/org/antlr/gunit/swingui/parsers/ANTLRv3.g 0000644 0001750 0001750 00000033032 11256465172 026214 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2007 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** ANTLR v3 grammar written in ANTLR v3 with AST construction */
grammar ANTLRv3;
options {
output=AST;
ASTLabelType=CommonTree;
}
tokens {
DOC_COMMENT;
PARSER;
LEXER;
RULE;
BLOCK;
OPTIONAL;
CLOSURE;
POSITIVE_CLOSURE;
SYNPRED;
RANGE;
CHAR_RANGE;
EPSILON;
ALT;
EOR;
EOB;
EOA; // end of alt
ID;
ARG;
ARGLIST;
RET;
LEXER_GRAMMAR;
PARSER_GRAMMAR;
TREE_GRAMMAR;
COMBINED_GRAMMAR;
INITACTION;
LABEL; // $x used in rewrite rules
TEMPLATE;
SCOPE='scope';
SEMPRED;
GATED_SEMPRED; // {p}? =>
SYN_SEMPRED; // (...) => it's a manually-specified synpred converted to sempred
BACKTRACK_SEMPRED; // auto backtracking mode syn pred converted to sempred
FRAGMENT='fragment';
TREE_BEGIN='^(';
ROOT='^';
BANG='!';
RANGE='..';
REWRITE='->';
}
@members {
int gtype;
public List rules;
}
@header {
package org.antlr.gunit.swingui.parsers;
import java.util.List;
}
@lexer::header {
package org.antlr.gunit.swingui.parsers;
}
grammarDef
: DOC_COMMENT?
( 'lexer' {gtype=LEXER_GRAMMAR;} // pure lexer
| 'parser' {gtype=PARSER_GRAMMAR;} // pure parser
| 'tree' {gtype=TREE_GRAMMAR;} // a tree parser
| {gtype=COMBINED_GRAMMAR;} // merged parser/lexer
)
g='grammar' id ';' optionsSpec? tokensSpec? attrScope* action*
rule+
EOF
-> ^( {adaptor.create(gtype,$g)}
id DOC_COMMENT? optionsSpec? tokensSpec? attrScope* action* rule+
)
;
tokensSpec
: TOKENS tokenSpec+ '}' -> ^(TOKENS tokenSpec+)
;
tokenSpec
: TOKEN_REF
( '=' (lit=STRING_LITERAL|lit=CHAR_LITERAL) -> ^('=' TOKEN_REF $lit)
| -> TOKEN_REF
)
';'
;
attrScope
: 'scope' id ACTION -> ^('scope' id ACTION)
;
/** Match stuff like @parser::members {int i;} */
action
: '@' (actionScopeName '::')? id ACTION -> ^('@' actionScopeName? id ACTION)
;
/** Sometimes the scope names will collide with keywords; allow them as
* ids for action scopes.
*/
actionScopeName
: id
| l='lexer' -> ID[$l]
| p='parser' -> ID[$p]
;
optionsSpec
: OPTIONS (option ';')+ '}' -> ^(OPTIONS option+)
;
option
: id '=' optionValue -> ^('=' id optionValue)
;
optionValue
: id
| STRING_LITERAL
| CHAR_LITERAL
| INT
| s='*' -> STRING_LITERAL[$s] // used for k=*
;
rule
scope {
String name;
}
@after{
this.rules.add($rule::name);
}
: DOC_COMMENT?
( modifier=('protected'|'public'|'private'|'fragment') )?
id {$rule::name = $id.text;}
'!'?
( arg=ARG_ACTION )?
( 'returns' rt=ARG_ACTION )?
throwsSpec? optionsSpec? ruleScopeSpec? ruleAction*
':' altList ';'
exceptionGroup?
-> ^( RULE id {modifier!=null?adaptor.create(modifier):null} ^(ARG $arg)? ^(RET $rt)?
optionsSpec? ruleScopeSpec? ruleAction*
altList
exceptionGroup?
EOR["EOR"]
)
;
/** Match stuff like @init {int i;} */
ruleAction
: '@' id ACTION -> ^('@' id ACTION)
;
throwsSpec
: 'throws' id ( ',' id )* -> ^('throws' id+)
;
ruleScopeSpec
: 'scope' ACTION -> ^('scope' ACTION)
| 'scope' id (',' id)* ';' -> ^('scope' id+)
| 'scope' ACTION
'scope' id (',' id)* ';'
-> ^('scope' ACTION id+ )
;
block
: lp='('
( (opts=optionsSpec)? ':' )?
a1=alternative rewrite ( '|' a2=alternative rewrite )*
rp=')'
-> ^( BLOCK[$lp,"BLOCK"] optionsSpec? (alternative rewrite?)+ EOB[$rp,"EOB"] )
;
altList
@init {
// must create root manually as it's used by invoked rules in real antlr tool.
// leave here to demonstrate use of {...} in rewrite rule
// it's really BLOCK[firstToken,"BLOCK"]; set line/col to previous ( or : token.
CommonTree blkRoot = (CommonTree)adaptor.create(BLOCK,input.LT(-1),"BLOCK");
}
: a1=alternative rewrite ( '|' a2=alternative rewrite )*
-> ^( {blkRoot} (alternative rewrite?)+ EOB["EOB"] )
;
alternative
@init {
Token firstToken = input.LT(1);
Token prevToken = input.LT(-1); // either : or | I think
}
: element+ -> ^(ALT[firstToken,"ALT"] element+ EOA["EOA"])
| -> ^(ALT[prevToken,"ALT"] EPSILON[prevToken,"EPSILON"] EOA["EOA"])
;
exceptionGroup
: ( exceptionHandler )+ ( finallyClause )?
| finallyClause
;
exceptionHandler
: 'catch' ARG_ACTION ACTION -> ^('catch' ARG_ACTION ACTION)
;
finallyClause
: 'finally' ACTION -> ^('finally' ACTION)
;
element
: elementNoOptionSpec
;
elementNoOptionSpec
: id (labelOp='='|labelOp='+=') atom
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] ^($labelOp id atom) EOA["EOA"]) EOB["EOB"]))
| -> ^($labelOp id atom)
)
| id (labelOp='='|labelOp='+=') block
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] ^($labelOp id block) EOA["EOA"]) EOB["EOB"]))
| -> ^($labelOp id block)
)
| atom
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] atom EOA["EOA"]) EOB["EOB"]) )
| -> atom
)
| ebnf
| ACTION
| SEMPRED ( '=>' -> GATED_SEMPRED | -> SEMPRED )
| treeSpec
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] treeSpec EOA["EOA"]) EOB["EOB"]) )
| -> treeSpec
)
;
atom: range ( (op='^'|op='!') -> ^($op range) | -> range )
| terminal
| notSet ( (op='^'|op='!') -> ^($op notSet) | -> notSet )
| RULE_REF ( arg=ARG_ACTION )? ( (op='^'|op='!') )?
-> {$arg!=null&&op!=null}? ^($op RULE_REF $arg)
-> {$arg!=null}? ^(RULE_REF $arg)
-> {$op!=null}? ^($op RULE_REF)
-> RULE_REF
;
notSet
: '~'
( notTerminal -> ^('~' notTerminal)
| block -> ^('~' block)
)
;
treeSpec
: '^(' element ( element )+ ')' -> ^(TREE_BEGIN element+)
;
/** Matches ENBF blocks (and token sets via block rule) */
ebnf
@init {
Token firstToken = input.LT(1);
}
@after {
$ebnf.tree.getToken().setLine(firstToken.getLine());
$ebnf.tree.getToken().setCharPositionInLine(firstToken.getCharPositionInLine());
}
: block
( op='?' -> ^(OPTIONAL[op] block)
| op='*' -> ^(CLOSURE[op] block)
| op='+' -> ^(POSITIVE_CLOSURE[op] block)
| '=>' // syntactic predicate
-> {gtype==COMBINED_GRAMMAR &&
Character.isUpperCase($rule::name.charAt(0))}?
// if lexer rule in combined, leave as pred for lexer
^(SYNPRED["=>"] block)
// in real antlr tool, text for SYN_SEMPRED is predname
-> SYN_SEMPRED
| -> block
)
;
range!
: c1=CHAR_LITERAL RANGE c2=CHAR_LITERAL -> ^(CHAR_RANGE[$c1,".."] $c1 $c2)
;
terminal
: ( CHAR_LITERAL -> CHAR_LITERAL
// Args are only valid for lexer rules
| TOKEN_REF
( ARG_ACTION -> ^(TOKEN_REF ARG_ACTION)
| -> TOKEN_REF
)
| STRING_LITERAL -> STRING_LITERAL
| '.' -> '.'
)
( '^' -> ^('^' $terminal)
| '!' -> ^('!' $terminal)
)?
;
notTerminal
: CHAR_LITERAL
| TOKEN_REF
| STRING_LITERAL
;
ebnfSuffix
@init {
Token op = input.LT(1);
}
: '?' -> OPTIONAL[op]
| '*' -> CLOSURE[op]
| '+' -> POSITIVE_CLOSURE[op]
;
// R E W R I T E S Y N T A X
rewrite
@init {
Token firstToken = input.LT(1);
}
: (rew+='->' preds+=SEMPRED predicated+=rewrite_alternative)*
rew2='->' last=rewrite_alternative
-> ^($rew $preds $predicated)* ^($rew2 $last)
|
;
rewrite_alternative
options {backtrack=true;}
: rewrite_template
| rewrite_tree_alternative
| /* empty rewrite */ -> ^(ALT["ALT"] EPSILON["EPSILON"] EOA["EOA"])
;
rewrite_tree_block
: lp='(' rewrite_tree_alternative ')'
-> ^(BLOCK[$lp,"BLOCK"] rewrite_tree_alternative EOB[$lp,"EOB"])
;
rewrite_tree_alternative
: rewrite_tree_element+ -> ^(ALT["ALT"] rewrite_tree_element+ EOA["EOA"])
;
rewrite_tree_element
: rewrite_tree_atom
| rewrite_tree_atom ebnfSuffix
-> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] rewrite_tree_atom EOA["EOA"]) EOB["EOB"]))
| rewrite_tree
( ebnfSuffix
-> ^(ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] rewrite_tree EOA["EOA"]) EOB["EOB"]))
| -> rewrite_tree
)
| rewrite_tree_ebnf
;
rewrite_tree_atom
: CHAR_LITERAL
| TOKEN_REF ARG_ACTION? -> ^(TOKEN_REF ARG_ACTION?) // for imaginary nodes
| RULE_REF
| STRING_LITERAL
| d='$' id -> LABEL[$d,$id.text] // reference to a label in a rewrite rule
| ACTION
;
rewrite_tree_ebnf
@init {
Token firstToken = input.LT(1);
}
@after {
$rewrite_tree_ebnf.tree.getToken().setLine(firstToken.getLine());
$rewrite_tree_ebnf.tree.getToken().setCharPositionInLine(firstToken.getCharPositionInLine());
}
: rewrite_tree_block ebnfSuffix -> ^(ebnfSuffix rewrite_tree_block)
;
rewrite_tree
: '^(' rewrite_tree_atom rewrite_tree_element* ')'
-> ^(TREE_BEGIN rewrite_tree_atom rewrite_tree_element* )
;
/** Build a tree for a template rewrite:
^(TEMPLATE (ID|ACTION) ^(ARGLIST ^(ARG ID ACTION) ...) )
where ARGLIST is always there even if no args exist.
ID can be "template" keyword. If first child is ACTION then it's
an indirect template ref
-> foo(a={...}, b={...})
-> ({string-e})(a={...}, b={...}) // e evaluates to template name
-> {%{$ID.text}} // create literal template from string (done in ActionTranslator)
-> {st-expr} // st-expr evaluates to ST
*/
rewrite_template
: // -> template(a={...},...) "..." inline template
id lp='(' rewrite_template_args ')'
( str=DOUBLE_QUOTE_STRING_LITERAL | str=DOUBLE_ANGLE_STRING_LITERAL )
-> ^(TEMPLATE[$lp,"TEMPLATE"] id rewrite_template_args $str)
| // -> foo(a={...}, ...)
rewrite_template_ref
| // -> ({expr})(a={...}, ...)
rewrite_indirect_template_head
| // -> {...}
ACTION
;
/** -> foo(a={...}, ...) */
rewrite_template_ref
: id lp='(' rewrite_template_args ')'
-> ^(TEMPLATE[$lp,"TEMPLATE"] id rewrite_template_args)
;
/** -> ({expr})(a={...}, ...) */
rewrite_indirect_template_head
: lp='(' ACTION ')' '(' rewrite_template_args ')'
-> ^(TEMPLATE[$lp,"TEMPLATE"] ACTION rewrite_template_args)
;
rewrite_template_args
: rewrite_template_arg (',' rewrite_template_arg)*
-> ^(ARGLIST rewrite_template_arg+)
| -> ARGLIST
;
rewrite_template_arg
: id '=' ACTION -> ^(ARG[$id.start] id ACTION)
;
id : TOKEN_REF -> ID[$TOKEN_REF]
| RULE_REF -> ID[$RULE_REF]
;
// L E X I C A L R U L E S
SL_COMMENT
: '//'
( ' $ANTLR ' SRC // src directive
| ~('\r'|'\n')*
)
'\r'? '\n'
{$channel=HIDDEN;}
;
ML_COMMENT
: '/*' {if (input.LA(1)=='*') $type=DOC_COMMENT; else $channel=HIDDEN;} .* '*/'
;
CHAR_LITERAL
: '\'' LITERAL_CHAR '\''
;
STRING_LITERAL
: '\'' LITERAL_CHAR LITERAL_CHAR* '\''
;
fragment
LITERAL_CHAR
: ESC
| ~('\''|'\\')
;
DOUBLE_QUOTE_STRING_LITERAL
: '"' (ESC | ~('\\'|'"'))* '"'
;
DOUBLE_ANGLE_STRING_LITERAL
: '<<' .* '>>'
;
fragment
ESC : '\\'
( 'n'
| 'r'
| 't'
| 'b'
| 'f'
| '"'
| '\''
| '\\'
| '>'
| 'u' XDIGIT XDIGIT XDIGIT XDIGIT
| . // unknown, leave as it is
)
;
fragment
XDIGIT :
'0' .. '9'
| 'a' .. 'f'
| 'A' .. 'F'
;
INT : '0'..'9'+
;
ARG_ACTION
: NESTED_ARG_ACTION
;
fragment
NESTED_ARG_ACTION :
'['
( options {greedy=false; k=1;}
: NESTED_ARG_ACTION
| ACTION_STRING_LITERAL
| ACTION_CHAR_LITERAL
| .
)*
']'
{setText(getText().substring(1, getText().length()-1));}
;
ACTION
: NESTED_ACTION ( '?' {$type = SEMPRED;} )?
;
fragment
NESTED_ACTION :
'{'
( options {greedy=false; k=2;}
: NESTED_ACTION
| SL_COMMENT
| ML_COMMENT
| ACTION_STRING_LITERAL
| ACTION_CHAR_LITERAL
| .
)*
'}'
;
fragment
ACTION_CHAR_LITERAL
: '\'' (ACTION_ESC|~('\\'|'\'')) '\''
;
fragment
ACTION_STRING_LITERAL
: '"' (ACTION_ESC|~('\\'|'"'))* '"'
;
fragment
ACTION_ESC
: '\\\''
| '\\' '"' // ANTLR doesn't like: '\\"'
| '\\' ~('\''|'"')
;
TOKEN_REF
: 'A'..'Z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
RULE_REF
: 'a'..'z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
/** Match the start of an options section. Don't allow normal
* action processing on the {...} as it's not a action.
*/
OPTIONS
: 'options' WS_LOOP '{'
;
TOKENS
: 'tokens' WS_LOOP '{'
;
/** Reset the file and line information; useful when the grammar
* has been generated so that errors are shown relative to the
* original file like the old C preprocessor used to do.
*/
fragment
SRC : 'src' ' ' file=ACTION_STRING_LITERAL ' ' line=INT
;
WS : ( ' '
| '\t'
| '\r'? '\n'
)+
{$channel=HIDDEN;}
;
fragment
WS_LOOP
: ( WS
| SL_COMMENT
| ML_COMMENT
)*
;
antlr-3.2/gunit/src/main/antlr3/org/antlr/gunit/gUnit.g 0000644 0001750 0001750 00000021076 11256465172 023012 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
grammar gUnit;
options {language=Java;}
tokens {
OK = 'OK';
FAIL = 'FAIL';
DOC_COMMENT;
}
@header {package org.antlr.gunit;}
@lexer::header {
package org.antlr.gunit;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.StringReader;
}
@members {
public GrammarInfo grammarInfo;
public gUnitParser(TokenStream input, GrammarInfo grammarInfo) {
super(input);
this.grammarInfo = grammarInfo;
}
}
gUnitDef: 'gunit' g1=id ('walks' g2=id)? ';'
{
if ( $g2.text!=null ) {
grammarInfo.setGrammarName($g2.text);
grammarInfo.setTreeGrammarName($g1.text);
}
else {
grammarInfo.setGrammarName($g1.text);
}
}
optionsSpec? header? testsuite*
;
optionsSpec
: OPTIONS (option ';')+ '}'
;
// Note: currently, this is the only valid option for setting customized tree adaptor
option : id '=' treeAdaptor
{
if ( $id.text.equals("TreeAdaptor") ) {
grammarInfo.setAdaptor($treeAdaptor.text);
}
// TODO: need a better error logging strategy
else System.err.println("Invalid option detected: "+$text);
}
;
treeAdaptor
: id EXT*
;
header : '@header' ACTION
{
int pos1, pos2;
if ( (pos1=$ACTION.text.indexOf("package"))!=-1 && (pos2=$ACTION.text.indexOf(';'))!=-1 ) {
grammarInfo.setHeader($ACTION.text.substring(pos1+8, pos2).trim()); // substring the package path
}
else {
System.err.println("error(line "+$ACTION.getLine()+"): invalid header");
}
}
;
testsuite // gUnit test suite based on individual rule
scope {
boolean isLexicalRule;
}
@init {
gUnitTestSuite ts = null;
$testsuite::isLexicalRule = false;
}
: ( r1=RULE_REF ('walks' r2=RULE_REF)?
{
if ( $r2==null ) ts = new gUnitTestSuite($r1.text);
else ts = new gUnitTestSuite($r1.text, $r2.text);
}
| t=TOKEN_REF
{
ts = new gUnitTestSuite();
ts.setLexicalRuleName($t.text);
$testsuite::isLexicalRule = true;
}
)
':'
testcase[ts]+ {grammarInfo.addRuleTestSuite(ts);}
;
// TODO : currently gUnit just ignores illegal test for lexer rule, but should also emit a reminding message
testcase[gUnitTestSuite ts] // individual test within a (rule)testsuite
: input expect {$ts.addTestCase($input.in, $expect.out);}
;
input returns [gUnitTestInput in]
@init {
String testInput = null;
boolean inputIsFile = false;
int line = -1;
}
@after {
in = new gUnitTestInput(testInput, inputIsFile, line);
}
: STRING
{
testInput = $STRING.text.replace("\\n", "\n").replace("\\r", "\r").replace("\\t", "\t")
.replace("\\b", "\b").replace("\\f", "\f").replace("\\\"", "\"").replace("\\'", "\'").replace("\\\\", "\\");
line = $STRING.line;
}
| ML_STRING
{
testInput = $ML_STRING.text;
line = $ML_STRING.line;
}
| file
{
testInput = $file.text;
inputIsFile = true;
line = $file.line;
}
;
expect returns [AbstractTest out]
: OK {$out = new BooleanTest(true);}
| FAIL {$out = new BooleanTest(false);}
| 'returns' RETVAL {if ( !$testsuite::isLexicalRule ) $out = new ReturnTest($RETVAL);}
| '->' output {if ( !$testsuite::isLexicalRule ) $out = new OutputTest($output.token);}
;
output returns [Token token]
: STRING
{
$STRING.setText($STRING.text.replace("\\n", "\n").replace("\\r", "\r").replace("\\t", "\t")
.replace("\\b", "\b").replace("\\f", "\f").replace("\\\"", "\"").replace("\\'", "\'").replace("\\\\", "\\"));
$token = $STRING;
}
| ML_STRING {$token = $ML_STRING;}
| AST {$token = $AST;}
| ACTION {$token = $ACTION;}
;
file returns [int line]
: id EXT? {$line = $id.line;}
;
id returns [int line]
: TOKEN_REF {$line = $TOKEN_REF.line;}
| RULE_REF {$line = $RULE_REF.line;}
;
// L E X I C A L R U L E S
SL_COMMENT
: '//' ~('\r'|'\n')* '\r'? '\n' {$channel=HIDDEN;}
;
ML_COMMENT
: '/*' {$channel=HIDDEN;} .* '*/'
;
STRING : '"' ( ESC | ~('\\'|'"') )* '"' {setText(getText().substring(1, getText().length()-1));}
;
ML_STRING
: {// we need to determine the number of spaces or tabs (indentation) for multi-line input
StringBuffer buf = new StringBuffer();
int i = -1;
int c = input.LA(-1);
while ( c==' ' || c=='\t' ) {
buf.append((char)c);
c = input.LA(--i);
}
String indentation = buf.reverse().toString();
}
'<<' .* '>>'
{// also determine the appropriate newline separator and get info of the first and last 2 characters (exclude '<<' and '>>')
String newline = System.getProperty("line.separator");
String front, end;
int oldFrontIndex = 2;
int oldEndIndex = getText().length()-2;
int newFrontIndex, newEndIndex;
if ( newline.length()==1 ) {
front = getText().substring(2, 3);
end = getText().substring(getText().length()-3, getText().length()-2);
newFrontIndex = 3;
newEndIndex = getText().length()-3;
}
else {// must be 2, e.g. Windows System which uses \r\n as a line separator
front = getText().substring(2, 4);
end = getText().substring(getText().length()-4, getText().length()-2);
newFrontIndex = 4;
newEndIndex = getText().length()-4;
}
// strip unwanted characters, e.g. '<<' (including a newline after it) or '>>' (including a newline before it)
String temp = null;
if ( front.equals(newline) && end.equals(newline) ) {
// need to handle the special case: <<\n>> or <<\r\n>>
if ( newline.length()==1 && getText().length()==5 ) temp = "";
else if ( newline.length()==2 && getText().length()==6 ) temp = "";
else temp = getText().substring(newFrontIndex, newEndIndex);
}
else if ( front.equals(newline) ) {
temp = getText().substring(newFrontIndex, oldEndIndex);
}
else if ( end.equals(newline) ) {
temp = getText().substring(oldFrontIndex, newEndIndex);
}
else {
temp = getText().substring(oldFrontIndex, oldEndIndex);
}
// finally we need to prpcess the indentation line by line
BufferedReader bufReader = new BufferedReader(new StringReader(temp));
buf = new StringBuffer();
String line = null;
int count = 0;
try {
while((line = bufReader.readLine()) != null) {
if ( line.startsWith(indentation) ) line = line.substring(indentation.length());
if ( count>0 ) buf.append(newline);
buf.append(line);
count++;
}
setText(buf.toString());
}
catch (IOException ioe) {
setText(temp);
}
}
;
TOKEN_REF
: 'A'..'Z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
RULE_REF
: 'a'..'z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
EXT : '.'('a'..'z'|'A'..'Z'|'0'..'9')+;
RETVAL : NESTED_RETVAL {setText(getText().substring(1, getText().length()-1));}
;
fragment
NESTED_RETVAL :
'['
( options {greedy=false;}
: NESTED_RETVAL
| .
)*
']'
;
AST : NESTED_AST (' '? NESTED_AST)*;
fragment
NESTED_AST :
'('
( options {greedy=false;}
: NESTED_AST
| .
)*
')'
;
OPTIONS : 'options' WS* '{'
;
ACTION
: NESTED_ACTION {setText(getText().substring(1, getText().length()-1));}
;
fragment
NESTED_ACTION :
'{'
( options {greedy=false; k=3;}
: NESTED_ACTION
| STRING_LITERAL
| CHAR_LITERAL
| .
)*
'}'
;
fragment
CHAR_LITERAL
: '\'' ( ESC | ~('\''|'\\') ) '\''
;
fragment
STRING_LITERAL
: '"' ( ESC | ~('\\'|'"') )* '"'
;
fragment
ESC : '\\'
( 'n'
| 'r'
| 't'
| 'b'
| 'f'
| '"'
| '\''
| '\\'
| '>'
| 'u' XDIGIT XDIGIT XDIGIT XDIGIT
| . // unknown, leave as it is
)
;
fragment
XDIGIT :
'0' .. '9'
| 'a' .. 'f'
| 'A' .. 'F'
;
WS : ( ' '
| '\t'
| '\r'? '\n'
)+
{$channel=HIDDEN;}
;
antlr-3.2/gunit/src/main/resources/ 0000755 0001750 0001750 00000000000 11410174107 017303 5 ustar twerner twerner antlr-3.2/gunit/src/main/resources/org/ 0000755 0001750 0001750 00000000000 11410174107 020072 5 ustar twerner twerner antlr-3.2/gunit/src/main/resources/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 021212 5 ustar twerner twerner antlr-3.2/gunit/src/main/resources/org/antlr/gunit/ 0000755 0001750 0001750 00000000000 11410174107 022340 5 ustar twerner twerner antlr-3.2/gunit/src/main/resources/org/antlr/gunit/junit.stg 0000644 0001750 0001750 00000006741 11256465172 024235 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon, Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group junit;
classHeader(header,junitFileName,hasPackage,packagePath,lexerPath,parserPath,treeParserPath,isTreeGrammar) ::= <<
import org.antlr.gunit.gUnitBaseTest;
public class extends gUnitBaseTest {
public void setUp() {
<\t><\t>this.packagePath = "";
this.lexerPath = "";
this.parserPath = "";
<\t><\t>this.treeParserPath = "";
}<\n><\n>
>>
testTreeRuleMethod(methodName,testTreeRuleName,testRuleName,testInput,isFile,tokenType,expecting) ::= <<
public void () throws Exception {
// test input:
Object retval = execTreeParser(, , , );
Object actual = examineExecResult(, retval);
Object expecting = ;
assertEquals("testing rule "+, expecting, actual);
}<\n><\n>
>>
testTreeRuleMethod2(methodName,testTreeRuleName,testRuleName,testInput,isFile,returnType,expecting) ::= <<
public void () throws Exception {
// test input:
retval = ()execTreeParser(, , , );
assertTrue("testing rule "+, );
}<\n><\n>
>>
testRuleMethod(isLexicalRule,methodName,testRuleName,testInput,isFile,tokenType,expecting) ::= <<
public void () throws Exception {
// test input:
Object retval = execLexerexecParser(, , );
Object actual = examineExecResult(, retval);
Object expecting = ;
assertEquals("testing rule "+, expecting, actual);
}<\n><\n>
>>
testRuleMethod2(methodName,testRuleName,testInput,isFile,returnType,expecting) ::= <<
public void () throws Exception {
// test input:
retval = ()execParser(, , );
assertTrue("testing rule "+, );
}<\n><\n>
>>
antlr-3.2/gunit/src/main/resources/org/antlr/gunit/gUnitTestResult.stg 0000644 0001750 0001750 00000003772 11256465172 026232 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon, Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group gUnitTestResult;
testResult(title, num_of_test, num_of_failure, failure, has_invalid, num_of_invalid, invalid) ::= <<
-----------------------------------------------------------------------
with <num_of_test> tests
-----------------------------------------------------------------------
<num_of_failure> failures found:
<failure:{<it.header>
expected: <it.expectedResult>
actual: <it.actualResult>
}>
<if(has_invalid)>
<num_of_invalid> invalid inputs found:
<invalid:{<it.header>
invalid input: <it.actual>
}>
<endif>
Tests run: <num_of_test>, Failures: <num_of_failure>
>>
antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/ 0000755 0001750 0001750 00000000000 11410174107 024025 5 ustar twerner twerner antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/ 0000755 0001750 0001750 00000000000 11410174107 025272 5 ustar twerner twerner antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/help24.png 0000644 0001750 0001750 00000002677 11256465172 027127 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/4/04y^£Õ IDATxœµ•mleÇ÷s?ÏsžÓóÒöØÁÖÚͬ
qNÝb ,k6 h¢Æ±è‚šaDgnš˜ Á 01§`"‰Ùp›dYº,Ì
1ÙÚ6¶¶{iO»szNÏóvß÷åBI+~ëóuýùÿ¯;÷¥D„³¼U ð?¨á÷·©k» |¹XäkxÜä<ú±ã¥ŽcÄ{¿÷¼Œ/7¯–‹èÐO•Ö¯ñÝR™‡zV®-ø…B:Ÿ§]¿HýâWŽ+9ĨßjøŃwõÿ °XõõVØ»rýºMýw|‡Rm%4Î# ØùK. mFóÂÞzë,gšPOÕ)…úÆŽöÄ<ó%U]³Šç‡†ï¿eåð.8{ óúÓèÖ$®“‘g;H¥› |
¤ “ç§x½.\N9§ðnÛyÈ\XpäAUdŽç>qç=›¶î& ÙAp陃Sg!Ö!Z‰ì=™
DÄ(fëmŽÏ -ÃAåøÊ#‡] ï[rcŠû®[·zóÀ-ß&;°
¿~’z/ׯ£ø1jŸÂ|Š:‡É—Y8° {u ë‡BŸ¡bÎk-6;Øü
ÞóL´Q•Ëv®¹ñ^ìø_1'ÉãWùäûÙ°q5×=Êà‘-T_}‚Òú¯²jûŸ IãŒN.ô† 0ȶ‡oVþ"k«ÜÞÝÿ‘Ït—#Z¯ìGyÇ|ü>úzCNÿj+´pâñwxgì,Ÿß}œp`=ó§Fq&Ð µ‚e&“»*Z®¦ÞuPîb{Ï5kÈ&FÉM¬10УȽ*åo Å_{:‡ù\îøx€x!@ø: ª5ºD±aQDÊãs‘¯É[³˜¬,¬ºðG¢+/2XM±õ Nœ‡7znf㎿`¦NКG‚ž_À
hO
žE‰Ç Ÿ61Êa,H
ZƒjÎÒùݽ´sx¥¡ðoßÃÝßz wr o?óâæ ªRÃ"´1ˆÊp¢ù±w$žo¨¢q`s…3‚Ràûp®©(myŠ[ïÞÆôo†™þçA2 Ú‹_è¢ð: ‰ "(¡µ(¢T8Ýî$¸Ô` qÖ'ÏY
i >½i;W_x’©C‰QH¹J•)–Ê„Aˆ8˜Í ‰u¼±ÈA,jt¦•ÝÐ ù¨°Šu'Öd8kéÑŽóþ>Áä(¦'Ä+„Å
ÅJ7…0„NÌl’3“<‘ÉcrÀâ?;ÈöfæQ©”1ZCn0yŠgrÞ¾jYQ bÅPa»‰.–ˆÊÝ„aˆJ3’…˜ñù˜47ìûùK’/D¬™»#çÚöŽu] ~TFŠ>žsˆ5ÔüœÏ~ý1˜>ÃÅ™“øÊ¢ƒ"¤)¦Õfl®ÍäB‚8wÚ9ö-ù=}O×¾§þqC_w4´¢†W© ´‡§<è_v)éÅ¡”ëÄÄs
Þœžcl¶I'NgÝÖ=Gåð’ €_ovù~ð³ë{+¬í«QªV¢‚ IÛˆ§B’v›éÆ<ã3
&šm²4Cœ}øÇ#òË÷ê-yž .ìñµ~¼Vêâ£Õ2µR‘0 Ešç4ã„Ë—Ú :clËY÷Ðî£ò‡÷k-{ÑžØlA© *¿PC"O£€Ø:“c² g¬µâžá©=#òêR:Ë vÞ¨*µªþ¢òÔ 6ˆ0 âŒX7&"#Ö²÷'ÇdtYü?êߣ}
à# IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/add.png 0000644 0001750 0001750 00000002746 11256465172 026556 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü tEXtCreation Time 9/4/04y^£Õ %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï 7IDATxœµ•kŒœU€Ÿó]ç›ËÎvº×¶»”UŠJl Š`bbZ¢b”hHD
!âhŒ(1úGcŒ$F•š4Qk) 6Q.ÁÒVšÚ]Ú]énw—ÙËÌ7³ßÌ|×sŽ?Œ„%»ây¿çyrÞóž÷ZkÞÍ0ÞU:`½SÂîŸZFÉø¨íxŸ–RÜ 3Q•èÈ|1 Qš ž9迶Ùy±Y‰îýÝs6÷üWó…ÂýÛK»Æócäí
¡è2ç_âìâÞXªAÓnF¡<”Zé÷ l¯þO‚ü¶·Z©ö ¾nè¦Û÷
‘áÂ8M]§ž,á§
R%1-—þÌñsGiÏvI[ú|¬²»êt_ý¯‚=¿ôJ#;úíßyï‡ ÝÇE^áDû×,„søa‹Nœ S#Í1QÚM » Ÿüþ…ÒW¯GVzsðídnCÁ]OÝè5í¹'>¹ëž}ŸÙö5žÕ2™½D jâD‘’Ñ ;´Ú!ívŒßñv"œ¾|†èB‚êȧãL „ð¶.š]›Úuu÷¾mûG¢ïp6þ+"s1°i·#^xúþ‹Ü•*¶i㘠“þ9.·.“÷ŠÈ~…ÒìnÿóMÁøA·`õXܶý³ü=>Á?:'щ‹`{3WøÖû åñ½OòØž'ØšDŠ ?lDF…Î ÈôçÄ~aÁ[ÚT §·l/_{]¥´•ßׂÂ"Ö)§ÏMSoù,½–rëWnÁs\bq~zš…°†“Ë#01˜&dE…jÈÛ¨0 ,¼)°sÎÝ£½ïe*<Ã|ç2å|§Nÿ“t÷²wâã¸ïqÑVF¤‘Šù|ÿ}¸†Ç3£üþ2¶‘Ã0YN‚Nó\¿N æ Ç£ ®°ä\—ÕÙ„ïü&½[ ð•Ò
aî¹áËxä0¦5¯4NÑcHɈt ŠÞu%BW´ H"‹$–4Úk\;º‹ ¯‰¦©D*4¶a“’à‘C9#OÙ)ÓÕ º Z'P©¡kk5‘3\ÒTvô/óã¥o°ºÐæŽÑO114A¢R.¬Mq®ñ*†²8vé)*îJ¹
2ö‰Ô
Á:ŒÅd#ð¯)ôÊ!K
êIÀ±Úo˜?ßdOùfƇÇÁ€ãsOòøÔ/pùA¶¯Ä&~Ø$keh#2ήÿ‘uryy™nQ0ûp(`Ê Yh±µ8D¹X&!«aƒ¢ÕÇŽÒ;+T½ê¿AHäG(Ôsd̯»!Í#A½sw©SôciƒP„ØôÐéëðÇæ3Ì„IÈRw…kªïc¸°¢]$HZQ ÿuŸ8ŠAsTÿJ§ë Ùå/i ÿÔZ n”ªTÜ*}®C¿L ½.‹Ý7XìÔ° ‡ñ¾«);elÃ!H–£j35Zµ™Ê&Q Ýp}wèÃŽi?;xÕHntlƒù!z¬–aBƒ&B@"SQÙæ 3çg˜¿8O«ÝŠ¤”ŸÐ‡ôs
®|xà ã8ßአ£ ”)»½¸¦ @*S‚´C=Xe¾6ÏÜô KóK´Ã6Y–}] Ò?|+oÃ}0ðPß#Žm?ÜÛ·…êp?½•2ž“aÒj®Q_©³Z[¥4‰“8JÞ¯ Ó?;kÓV~ p‡â!Çpoôr –k£$NˆÃ˜n’f‰ÌTvD+ý#}HŸÚˆ³© @Ü)Šî s“ù%!ÄõZ3"µÌ¤Ì¦¤”'P Ö?Ó'7¼“àÿÿ\§'Î óú IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/favb16.png 0000644 0001750 0001750 00000001512 11256465172 027101 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/7/04kë ; ›IDATxœ¥“ÛKqÇ¿¿™Ùu/Îîª{q]ÛõB›fZn$‰>DAQ[A>E”F’oE"ý!=D!H‘/áï&™)j+ÞwgÙ™mwfö×K âÖ‹çñ\>‡ó=çJ)cÜ¿ýOHFÈ{U'EJ2sþÆ3ú#[ “Íù¨0œÅp;x®ñ¬¿6䧴\«&Y›eT]À™ÂŠÚûew‘ï÷Âjg.‡üþw„Î*b©hB95Áì=QpÏwê’S–LLÁYâ-.ØØ wÝ"ï"sˆw`…þPJÑ×AŽ™mÖ6W ,l²Û¹Î&øN†°³ô bLƒ®ðØYաȬ&Åw¿Eľè¦øêñÛÌ< ªÊvkÂ¥õ7¡k 0YSH'ž‚Å"rX4VDqÀ•-çŒ9¦ê¹‘¡ê‘¨Ø ‘€_iæ»"ɈEú! 30–Àó 0„€ÑEHQ‚°Þ<‡JB¦cvOÄhL}1ûùËÀTÿ¤D
-íA*aAR Pâ@*í€ÕUu ÂÌÄFïØ mÝÓ ž_'Gry´{J\ÍuOea²]w@£nL¯`yQêù:‹¶ÞyÝ·ÆÖ×42?‰n_ÄšluØÙÊc
’bTM‚B’Ðó·x €Â ¯÷5jjX%)Áj÷B ÖWd–‡Ù]ŠÒÊ ¼ÅÆ«áJb: h®€Õ`V¯ØóŒ@ò'ÄuŒL`|L@LPÁ[9¸] 4ìð ÿÀ!ùŽÂ°½%?|2–£â® ·k0ã£ÛBl˜Ïd4Ȳ–Ãó0 ¸¼$Q}8=ºv‡›„à¥Ç Ì Qô险h aQeàÐå°auoôþóo[’ FÈrC IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/redo24.png 0000644 0001750 0001750 00000002431 11256465172 027114 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/10/04¶úŠ0 iIDATxœ½•[lTEÇ3gÎÙ=»g{ƒ¥\Ä·HIAEBDÐ@ j"–ª„7"<`¢ñeÑ' Œ!”#!! <hB|0F• X
Q
mÓ¡ôÆv÷ì9gÆ*TˆxÃÿÛd¾ùÿò]fFc¸›’wÕýÿ ¨?Ûln~óþ0Ô˜-„Œ,K wœøÁ]»6^ø« q» ¬^ývm±è¿bŒ~)™ôâž—BJɵkËþ ã8û]7öîŽ ëÚþ6 ¥å*ß/–JUÌ™7w>Ó3¤RI”†‡ø¹«‡cÇÛ¸zu ϶M;w®Û>r¶±q£\´hÖ{¶Ú¶n}þƒÛ–/ßô¡ç¥Ö¬XÙÂ}õ÷0Ð{¨HLYxI*ÒI|]âСVŽ ùõÖiÓ¦¼ÚÑÑ=Øêyî¬r9Ú¶}ûšõð‡ 47ç‚yaéÒ§7&Mëá£ÄµEe2EdYe»ñÆ%xfÙ&MJ³ßçÎëš®5 —,yÄŽÅ< øªfÄSÞ0ßb•J¥µõuuïäz~ü®7’x®KGÏE¾<qŒoÚZ¹ÒßGñr@×É^æ͞ΪUOÊr9\¼lÙ,»¥e.cÆT %馦Õî¨ „“ÊåòsM3 ·§'‚DÂåL×9NvœÅ,iÑÙs‘¦L#6ÎäìÉ>2Sxãõ©¬ª¢³Ó‹YHiML&I ø;À÷‹‹=Ïsãñ8~ÿ•ñ$ýCƒœîjÇ|”:!ýÃ~qÑ·§¿§ûÊ% ; cÕP]ëÒvj€X\aÛ!ÍxÛ¶ÜQ%Š¢è©šš4åkT¨IØqºû~¥àÁ˜u,Ú½ûµ'„ æDçÅN}˜BïÝíý…A©H€D×tl@J«ÉhMX,‘°l ¥¸6H…£?Ù³gó € ý”ýÔتj šÚH•tHjpÊA¡„_
QJP‘JÍÝ)ŒÖØ\ËÁ‘
ÀpcŒ³Ù\µÑòÓêÊÊÌãsæÓ8®[C€&2’ XF+‰ë:DQ0ñ–1E€’qeãHEÊM ¤Ä£¯GìBÌÑ:¢íì:/tr=ÜX K)°-â¥$:2XJ4ÜB` %1aQ¬À’ŠP‡õÙl®„G†'ôö]õ„%Jª¸²$–´Rb„AHÊKÝ(EQ£K ©¨ ‹£ LÍïÍ]ž ^±rscîÕF7”C]a 0¨•B¤…éQ )d»ïû3 å!m45ÉɘKÁ/eVf7'öæ· ÄïÛ»å<p øâæ*d³¹4ìÑ )–üÒÌ_.÷PW5•T„Â`ÛÀÃ@pŠ;(ŸÏ].¬oºÉâ#Ú/t”2ã'Ó?<Hï` Rˆ’@ ßÖñõšf³¹ïè(š`+Ei „XŸÏç¶ýkÀuÈdcÌËƘµB_±9ŸÏ½ÿOÌo ø¯u×?ýß ƒJÐSbÏ%“ IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/file16.png 0000644 0001750 0001750 00000001477 11256465172 027114 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/8/043ˆ m IDATxœ¥“=ˆ\U†ŸïÜsî½3³;“Íêj ëF1("h! ,ƒ…v‚}ˆ°RXD´0$V–ZL!
¢‚ÅŸ°0Yãß&!3³sÎœ?«èN¯{‹ïáå}y%¥ÄÝœÞ)¾|CVê–#Y/?†‘ÃÁK×Öêiú}ÚÌ®‡˜NŸ8Ÿ¾Ýù#w \~Gž+ú½3‹<y¨»ò¦w™ÎO»}ï?¿ÀæÆ?Câ•S曆s€OOËû׊+ž}»¿´öð¡ÝÀnob§[Hœrå‹KüðõIac’cãqº|æBJÀ^è?x¸¿´v’?ÂÕ?a§c\5Æ×SRô(•Ñë÷ðqVTS6Fž*•¬—Kß1«~$† !G¤@t•/pðé§8þâQ yì~œK“ ÿ‡¨9˜—»ñÍUìd (‚m¶!E蜢èP˜HUy¼çÒ[Sû À R¹ÁN6±“-Ct–è,ˆ)Cð;™pãïmbàÚ\ÎAŒ5® 3›ÜeˆÁAðHfS’™‚èg4µ#&ó ®¹…Ï<¾P¦ !ˆÊP™FtNfròÒ°«w
iøgzû2•@娢‡îì"_\&_¸Ó]Æt”ƒûØ»º„ ™´ŽŸo^»ŠÎ
ÚhŒV˜,«Fµh©É°Ý.ûö¯°ØWÏ¿z\ ÝÙÂgÃ_‡ë{öýB¬o2ÍZš* ™b¡ße°<`°«„Ùˆn)ƒÞïG5€R|XUñäWŸ|³*$Zwiè ÅD?%öǤdŠÂ@UŒayn çNÈË1ðZL¼Ÿ" ‡À–µÜE'&öxÇÞ$¬‡´f(›¯†r·sþc 9O$Gð# IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/favb24.png 0000644 0001750 0001750 00000002611 11256465172 027101 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/7/04kë ; ÚIDATxœµ•YhœU€¿óÿÿ,ÿÌ䟙ÌL–&¶.©¶Ñ*]@QKq)¨PA (
¢ˆâ‚¾¸BmPÜEA_DÁ¥¢U±‚¨¸‹æ¡©mmM«™I'™d’Yÿí^ ÄÒ´h êy>çû¸÷œ{hù?Ã:ÞÄÏŸ‘XµÆã¡bµÖܾþQ=|<uÆñ$m¹_¬–˃)Ǽ'éDW¼ñæ}²ê?`pE<e>´ì²+9{ý$Òñž˯Þ#ο¼u‹ä#67õ/_mv/}–Xf€L¯A$&k•bÅ¿ØiÖfú:.Y´ò.µQ~ yŽŽ|†Ž\¼# 8ëXõG5ù¥3Åȯ¡ÇŠ’öt§z¸µ÷ô‹‰&Ï¥¼ã6Ï#]Äé¥ü[kÍÆKåË d¦|™—·éÙ#yrø˜¾ÿ€œ+ÂEq› ¢6KÀèµóy–_ù:ÑD‚±kh·NÅîè§9y€ßîgvFUƒ@ÕëÁ¶¹
Ãs
¾ÛUgëG ¶ Éí¦ÉãÙ¾N§pÊJ’™¥Xv
;{>©ÜJZÕ»© Ø„Û:ÃÌ6J^7VrV}–j¹Èäï“Œï¯LÎÎ5_\žä]];tEJñâ¢åK\ûV¼0œB¤Œ0ß Âo½M,. fÜ&:¨àd $òønšþ‘ˆÁÞ ¾)|ùÑÖ¡r]í 6 ø„éž%¦`ßÈc¸õ/ˆÙ“Ø)—¸
(KæP¡‰-Š¿ì >¼ev‘ÈD÷¹\„hÄ@iåÌk²ïóÁ®ï¾ºÊ÷®Ã×
”êÁsóˆQAÔ–á¢<…òBü¶¢]·ð‚<V¦‹¸ÓãÄñ¦ðãð ª5õ ðñ<Aêå±Êšv£’_tÖb2=iÄ2P®Âo»(<”~Kð¼8í"ݹ€d"B2ÁÏ#{Ù½s&l´ôO~¨Çš¢×î•Ë—í”Ù“?!K¶'I6¯i¢ÜÊ÷Q¡Æk˜´Ü<fb!aaªXblO‘ò„ë¹ ÷>ö¾~ñ/漇vãsúC®ž›_ÿuÛT³^‰‘;a í†ÅÌ”P¯gðZ BßD‡!†nS›*³cä ‹îöPqíáð£ 7<«¿žgH1kÑŠuØùs¨Œ7iû Qª@èG]¯Ñ$™€hÌB|¶ñ ýÞ‘¼£ËĈX¬,,èŠö-^Æþ‘i{‰ìRl§€X ¡M˜x…ÎÒÛŸFàÂ;.”žc
z;IØ)óúþÁ3ÀÒ Ü·‹Lß™8ÙÕR…ßǦ1ÌfÜA¢i¬Ž.ôa'cË¢1Î8¦ jÓéä“«.a¦´h2I6—`¶¸—íßïdûÖ JÅ:±X„dÚA¢ÙB–®î8ñën>Oì¿<rHÌa•ïkGys¨Æqf)îü‰ï?û‰É‰æD»&¶ÖÙ»g
ñ§±¥Š5T ë3æíˆy¿iÒFŸ\µÚà‹Í[èêŽSúm–âx ×
C±xØÐìs]ýæèèl÷äänN˜ 6×¢Tja™ä")ÌÙräÒâЊWBÅjD´«áSà…hœOÌ6a-`…ÖlКË1ÑáažÞ°I{+ ºVÒ¢8[iòJ1lX”6lÒõ#r~Ài(EH&?‹ðÆMºý'ø¯ãÐI4ASxW IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/addfile24.png 0000644 0001750 0001750 00000003165 11256465172 027560 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/8/043ˆ m ÆIDATxœµ•{hg Ç?Ïû¼ïyÏ=—“K“4Í’%m³´ŠsÒ²áì¨ ueûkÚ¡¨lÿ 7¥V˜J±8Q&tNÂìRÔ]š
Û-«+M“vmBÚ4ksiNÎɹå}ÏyŸ‹¨¥!ÝŸ~ÿü=ðûðýÝ aåÿ)÷ã Æ~#ÄêR&èªä4ìPšmé*R8ZSt<&1¼þè/méVyÄ |pDd¢ûê!÷i#÷ȸÛs…ðˆ"V¥-õPS\©£´ UŠï*Å™o YŸpƒƒ±çĈ/¸¾¿«¹©Ža’#ø™ \?ÀÚ5¬Q]š`ìÕ—¹z¥°8<üíc Œ>+ o¤š’÷÷ß÷$-ý_:U¬YA‡×P*XB7Vq]Éøè_?q (Åbð÷Àoíù[:°ŠOZ—½ ; ¥ÿi D7ŽÓƉj‹èp
U¯¢¦QG AµŒ1`,hÃ&Ïÿüâ ßÑV6 ŒáK1ß‘¹Z¿‚6—ÂÁñ2X-F€u1"ܘ”.‚ٻ±v?ðü€¶ìpSí$š†ÁL Í<R¶‚ £ˆ ˆ¬,‰p$Ý·Of0* ´¼Ì¹ñe,´n(Ñ_žN ºã™n Ðx‡jµDhÖ6"®#’Ââ»’Õp$8.¹ž :{»PµS§´½®å䀖´‹Ž”—åJu‚w/ c"?É|aBC+ v·¶±+—£Ó±f Æ‘X+h4 äÚGe´aN)f6 úVð—“$FÕ,—g~M½ Çkþ ±¦2åj‹øÓÙ1Fd–ƒ;GØÝÕE $V8X ˜Åb€5Ìüèv~ £îÇŽ
Ùl±wÏw=HSkk²DÅæY*•¸“÷/ó÷ScÌþ³Ê¯îÞÅžþÊÊ‚0µ0ˆ ò7÷õà{wÆï‰
lmylç<¿À¿ì뜽~†‰éY„tØ”¢¥%Ãà Ýœ?u…ƒ'?àX®Îl5õ Ž1k¨l ÜûbwZe½' ø2UoŽwê/Q®—Xš2|Ó;L=]äÈò3*!YÙNË–gN¯ðÆìU ¿³•µcî` ™›@¾–ß»%5ô©t&ÍÛ«fv¦Ä‰4ÛÃ{y|×·x s?;Õgi&G9¨à%cvx{n‘R¤q¤G"åã8Ʋéɇ„³ ”Ø×æ÷2Ó8ǹ… äòÃütÇïyj÷OÄ›Zºxö3Gøζ PS!q7Ÿ‰3¾§¬^<C"ÛL®=…;²>›×PÞör½ÊLé2×+2-i²]’Z¬BÁ)QÀMIVT¿“L¬¡ž#qý$ÒOâ&sôö·`èp×:€nÈkW+ áš$mÎ&£Ó|õÍ™.^¤ËéæÈôs<tü‹¼µø&Ûs#ø§žo0Øž#ëÇ0ÖâHŸ¶ö&’IàÑu 1zuá=r¾Ô0¦á“׫̪Ëü£ôV/¢°ôdzIúij 5ìšâžîi½Œ –Ð2‰„¤»ËCkîÿñÃâ.øï¹î8˜Ì¥²Éw· ö
Ý6<DÖmBi…#$XpÄÝe]áËçyïýIÒ•2¿ØÜÆímbYŸ¦&—¸¨òÞÉy&·Ä<^;tÔî»ñ lúaó×ñØ ú‡‡ ØFGº¤ÀƒRš|y…é™).LN±–_adØ¡¡×“Ä|WZ*•ˆ04‡ŽZqcÑŠaíe¥tÿôøÔá•ù2]=]¤R)”2
EYZ\¤ÔŠŸVæ•Oh’5c»•fkØP½TÕjÖmrý™(Š?!f«ÁUª>µ8wm·ãÅ0ÚP
kõ°`y Ãïº- ¶XDÅÁ±Š úpè¶%`8u£7+þ!ú$—b³–Œ`‰[k—‰8‡Ëœ}Á*n¡ÃÇZÀ¡£Vÿ/þo‚7Ê1d…$ü IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/textfile16.png 0000644 0001750 0001750 00000001533 11256465172 030012 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/8/043ˆ m ¬IDATxœ¥“Ï‹”u Ç_ß Ï<ó{fwÖÕñÇŠF(D (,HA`té èd?º <(zQòR§ŽtðP’H +j!Q²Y㮳;Û°³Îø<3ÏÌóýÑA̳ïÛçð~ñáýæ-¼÷¼ˆôöãç³bvpTÃãhñº³"?6~÷ ò«ÑpüÈ:êä%ÿëvxöÁÒçâD¶R8Sª 9\Úu•¯¡T ¤ û-î^ýŽ•5ëùø“/ýå À•SbßôîðîÁ·/–s;Þ¥ßþ‰¸û'fÔFú¥jÈ K7ùíæ_xÉÈyq¼×óKg¾÷^¤–÷*7ÊÕÆÜ»ö>kË?¢ƒA¨@7—G‡ŠÕ"©…qd.8Ç›@¬T–Åpê5ð¿P™I) 9>ÅŽcÆQ ›f÷î ¾¿Áß¿ßãÖÕ¾sØç!j^ 5¶Z·xpç6Î9”)€ÐqD}$ñÃõó—}ò?ÀXöJq£6Ó³U2…i”ñ4`ä:ÄŒ#6Ö ã,'jLSð.ÆŽ#z Oî7ÑFi‰ÖHM®<ÅT-K’8œ¥? Á8ÙÄ%]ryEu¦1h¤Î«…*ÒtÉd êÏ `8dãÉf‹ÊL „ »ñ˜ÎZ‡öJ›öJ‹õæ
›G
¼T¯a G' IÊòæê}¤Dýˆ\1dnÏ …y :ÈüÂ<ù|ðìÙ7C¹¬Þùô-ñêö®6—ÿYÜUßÉìÎ"½¾aía‡áÀ"t›b)Oyª„ôCÜh‹|VT
E¾Ži )ù6ŠÝ‡7®Ü® )H;JSß4©_·Ž²÷bÁ#+(IF{âØÔ&¶ðÅIñ‘³|æ<_{ÇÖòï(aK(rÎ1g /{Á"pXkšBpîô7¾)^tÎÿ4.ð–E| IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/filesearch24.png 0000644 0001750 0001750 00000003165 11256465172 030275 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/8/043ˆ m ÆIDATxœµ•ml•gÇ÷ó<眧眞úrúBAÚ“¶8ÝSS£ÆeÓ š Éf6]Æ}`Ü"~ Ã\âLœs˦a!ºP5ËÌ\$°‚Š¥²–¶@[´Ð Êé9íy{Îórß· P Rã'¯äJ®äN®_þù_¹ÿBkÍÿ³¬ÿöpâ§ÂðÊØn@½ïÓê´)èòŒ / -˜Â丒 >ý’–Ëíÿ©à쯄ø¬õ=>ïºâÓ´ 0"fÒU 5ž$ð„…l™b!(kø¹ ØóíWuéN¿" ×ð£P<±&ÑÜE¬~UmØñZD(*@+駙¹p”“ï ¦PR(Í©èùÎkK!K ý/‹ÕBr¦¾µ®µ{fõ#@È 3àÌP.ñŠ3<ürŽÃû÷1£ˆH~¡|vì|CËz | ŽE¨kÿøf¬¯355ÁÌ BÞ0í‰4Í•Œr §à"•Â)”ð< ¥@jPŠgâäÞÇÄ]ojµðaDÝåúv^8ÿ~?|†ÙôeòŽÆÔ ÔY |¢!Å·6_£³6KÉÕøÒÀ´, S!´&@!?Ž³KQb“…äý/^û.ÓU<´Êàñ{ ÅFÒ.GFgØ78ÊÁÑ
^ûÂ8¬ËáÊ8·|˜r±„òò\¹F*å˜wî½5ô¹]ÑçÇÖ¦VnçÈ—t&-ŽLJ†ReaÉC õL®Ip¼_Ðó»ê™bCb›: (ãdÿÎÅ‘Y”dp×›zú.Àó;?Ke§ýîçâtÔ[¼~Úa`:GºP$ë{¤K Õ
+ÙØÝÁ…·gØ{$Oï“ržFú™tBÑEiÎÜé«qËà ü½5VÒU敇¡”‡ 2ˆÙ!"¦EÈ
‘÷]V4Õ[¿†¾ñ(ãsì(rÙ+50¾, iol‰kRš¡T©=r®GÁ p|IÉWD*ÂÔ®´©jjÀóa:c6A É/:¸žV@ñNÀm“½2E7àò‚$]p@k¼ ïzdË:bQ›Œ±CHÏC A(´V%)}|- Ìe^zl8µÀ| YD’q]æ ŸO¢£-ëWв¦Š\A‘œ"
ÑR#ð•
Ziñ–UÐf^|çÒ•æ{†ÚS¬[ßÌôBÛHX‚DMŒº¦Ž‚Kg‡ðÆ&imÍб*D®l¡„I¼2‚eaø>UË*xnÅókƒ“¼×?ŒÎÎcÖÄYÑ–$Ùž$Z›`60ðçsL¼÷i¶WîgüoDâ5vuÕ„B¦’û–UÐ ÎL|³ò×<£Šý¿™#¾¡‰ÕmD++ÈÏ/ríò4ùscà§y±ç
OuÝÏé? ÅN¶P÷6jêˆ'lòùâgö|YÄöüV— θ¾ZÌɽío˜o«'è¼ÊÈP5ÈhÔ"ÉÏ=jòµZõë².õ¿C¼¦‡p´’U-q®Ï–:@oáæwq0ë“›Œ6xS¿Ü6Ìñ®*ŽžwH»ÕÈbsn’Û äSŸìdaºu©—äºíŒö câÔ1Ú7µÑÐ`cšÂÒJ?u
‹âìÜ"3çN°uÓj¶ð²yHÍä¹p£Ä⨋»ùƒ`oeþâϨn !QÛÈÔ…qš!í µÁø±ew¯>u ð—µ~õѧá«ï¥˜¾2’בø>Dlxÿì(5-‡¹¯ûAlû£È\²ãpõªË ÞCú·RÓZSKó@rPØ’úžës Z šóZsÙ3!‹Þ†¿ êÛêÒ47ÛdæJ
¦(” Wr€qó.ÿ«ïŽÌ—ž›5< Ì¢9¨4 d™4–„ï[¡ð³ÑJ;\Êñ<¹Ï
qA“€:`xkw¯^X ð“Ž„ v¼~;úþ]?Ø&LßåCÀ½†É
Çb1$¡5л{o¦À?RìôO‘î IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/undo24.png 0000644 0001750 0001750 00000002453 11256465172 027134 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/10/04¶úŠ0 {IDATxœµ–MlUE†Ÿ™sν·åÞ@±‹&-Š‚DÀ~
²ÐÁ
RRX‚Œ° Á ML\`p1@\YDƒ 1À‚?‰Œ4¥´
-´÷ÞþÞ¿sÎœ™q#¤E‚A»ï}ŸÌ7óMFXky‘’/4 pŸÕ°cÇOÉ’¯«ŒQ‹ÖãبK:æÁÁƒ …Oª[´cÇÉrubÿþ
ÃO*ln>UEºÉ÷ýõZ«5B´ŽPJ…`N{žûáC[~{
àçYR:‡A Ý·ïýÖ‰E»vý¾N©ðk¬YX]=ÚÙ3I¥Êц‡ÇèèèâÞ½ûHG´ 9üÉž'¶o?Qç8nÒø¾9P]]Ù¼sçj¿¹ùô¶HEßÎœ9Õkh¨§úå
´‚Âx€¶–D² ¥
çÎ^àJÛ5¤#Ö =ºíä¿ [· “JMí^´h6éô(·o§/¤R‰3J™/¿9G®][¾åA÷(ª #MÁ/‘Š¤ª§óJ] ¿þrŠÎΛÝñ¸³ôÈ‘Ïr0áårãq! Ö²eËJêë+Wv¯X>_®{ï
ŠŠûíãø…€¾L–;ƒƒX‰$Ù[ýÜë¹MCÃ* GÎU*Úü0÷Àuåk
B ÄÙ´i%ë×/gÍê¹ôw ¸7<À¹«m\èhçrgo\g´§29¡Þ~<7F]ÝlJ¥bSSÓW‰I ¥O‹ë
ÚÛóôÞÑ,]RKº/$ÎóÇÍ뜿Úƃá AXBé€ûÃþºÓƒ’NŒñt–šÚZ‚Ð_ ¢&ÌRAʘ!ô‹ŒeùL°péJ½÷‘Žèð½±tXk*ŽvŽdjFr£T%+rEeex®ãÿ.Ðù`m4_
KPŠˆŠž²Ä´eF£uˆÖöÒ±cŸï}èÙ¼yïùthl”šŠ™DQ€ö#¦UTN.›Ô¢ ( ×±„E…)„$4”ÁŒøÞyk fÏ«ßß´ù› z¢(ÌE*$Šq×ÃC‹¸Ž‹òµI „™áÅ<üB€‰4~0\çnöwú(KÄñ<׋Tø]ccËÛ ÚDÂbˆÇbÄ„GÂñ0a„Õ!„žt‰D|^:=ÀÙóçK>ÆZ´Ö„*$T!A¬I"8ÜظgÖ–»R2µ,IÌqˆ E„aˆ”²j@
Y644D&›åŸáÓF… '#Ñ)…èBü t#È[xÕ•.•ÉiÄpp c
Æ€ü$ Ð+¤ìr` ¸t[¸…µw oå1}°a÷t`N²|
e) !0”ŽÐFƒ ó8àãÖÖ=ý=E5^iêt Ç¡d"ŠZ1’ÏQòK8Ò¹0 ðŒáH!gLl,Ÿ£/Ÿ%îxôdú¸ÙÛ!Å™Çwð¬jˆ®Á‘̼3×.â¹.éá!”‰Rþˆ¥
&¼¦ÿG7¶,²Öž0ÆÔ[@
qCq 8ÜÚºG=7 `Cã—UX,„BpùxkKvâúsþK/üWñ7|‚T,û®‰ù IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/next24.png 0000644 0001750 0001750 00000002743 11256465172 027147 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/4/04y^£Õ 4IDATxœµÕkŒœU Çñï9Ïež¹íάÃîÌnkÛÅV(F¤Bª¨¨IIlI£ÆD£D|aûmˆÆSo‰hc$X’â*—„èƒh´Šuw lê^º½ÌmŸ™y.ç9Ïñ•„%»â 9¯Ïù}rÎÿŸóÆÞÊ%ßÒtÀ~³
{~dÈùÇÍ~\kq“NDE‡`Bë¹$¡R§fŽ¶.lv^löD÷üê5ë=û…\¡pïökÆ«¹qrN™@ô™ký‹—Ïqy©m§ ú¤²Õƒ‹_í®þOÀ-¿,UÊ•Ò©ëj7ßz`ô³Œ®¤m4â%Zª‰J5–áÏ—þÈãçÏÐí£:f2J“»Çúÿ¯ÀdÆv ÿîàÎ{Þ¨öE^á&ºq)˜£tèE1ZYHåñÎ=øºÏãS¿¥õrÝJ/†¶ú ÿõxnCàîßï˶¹_ÜyÍç|bË—yÊü€©ä/Ä!QB¬5m? …tû}š=Ÿ1o':…³óç_ŽI{ú‰(1‡‚oǼ¡‹fצîªì9pÛ–Ïp:x€¨g±Æ"‰
^¬ë!AJWzLµÎ3ß™'—-¢‡SRÃàÖÿd¾\y4S°óö}û·’¿EL/ º—WšD&&ÁÙAö cñòÆJHÁ¡4ñC ™·1ž€Ä|J 6¼®MŨúÐöÁw]W ¸‚ß4~ˆÌ¸üó¯uäj™¦œg¶y‘]ÞM|éÚ£ô&b ~¯æ‘¦ŒD"°,HŠ)iSï§Ìpé5ÀñÜÃÛJW3 œc¡7OYæYkŠ߃„QÈ|s'+ˆEÌ‘[¾Æðs[81ý ýR€.ŽtR“xŒÊ!Ù» ë×ÍÒVX } ÇAJ‹±j•j¶JÊ
DDøø8ÂáÓï=L¥Xá[“ÇXtÉÛy ‘ÀDRZ_äTníâÓ }âHÓ‹CÔšM¢
}ºô‰ˆ‰éÒe•n÷ Üyý]8"à ;HÎÉãà@
Òu5H•4õµºðd¥=_‘»Jððòqò"%»ÂláÚêõx"ƒ…Å÷gNðØ…ŸSñ†È”ÑQ K´IAà¯t$¦š~kw!_@¤.q‰óäâ¯I!ŒbvE{ؽÿ=`ÃCç¿ÇÏ& ¡–ßJ0Š MÒI0Æ„$¼´ ´Ÿ_^^Þ-‡l
Ö*MÐqTB¢a˜b{.¯rjú§<3û »ÊW1ZØJÞÉÓ [„~@Ø
IIŸ&aa µuÚoôt"*åal# D€‹Oßô±Ý<uQç;g³Ò[eïÈŒäk"¾òé„ Z[Da†3æQ£Öß ÃŸ”¯ÿÐYô?\ ¨PÎTʸ(T€JÒ4E
›¥«)ºE éâÇ>Ëá
õ™:z‡$M¦H9³á_4þÚ\ËyªúŽ1oÛøª¹y»€-m@"Œ@
bh†
fÛsÌLΰðÊn'ÔZÔœ4Oo ¼ýþ‘£®ë~³6>ÆÖ Û¬2˜)‘±2 (ðU†¿ÊB}¹WçXZX¢tI’ä+æ¤9ñú¼
çÁÈ‘¡c®ãÜ_z•ÑaJåA²na@§½Fc¥Áj}•¶ß&Š#_§ú^ócóð³6hƒ÷îB qef_ÖËbg G1QÑTë$MN›Ô|ל4/n”³) îÅLÕ½Yb}^±×ƴщÖÉ´Öz‚”Sæ'æùMÞ ø¬mþ±rXþî• IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/refresh24.png 0000644 0001750 0001750 00000003012 11256465172 027615 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/10/04¶úŠ0 ZIDATxœµ–{lÕÇ?ç÷ºÞÛ˽@ ´P÷ÐBØÖ:4‹3º€ÀÜĸ©L¶º°M ºì™XýË .™N7^Ë€?4Û²dÐ Ý :®+](0V ôÚÒû¾÷wïïqüc²ì‘ÙÄd'9œää|òýþqÎUå³ ëÃ…Ü/MÜkY֢܂Á)ß÷pPwhùÿÈ5ò$ùmª+õÀÊžUÌ̧Tœ<ÆÅñ‰CÕ º1^Ó}J¿8ÜÈrb„ºIßž ýbQä¹›znz°ÿ{[Áñ°$DMŸ$iß aç›»‚éÉ\ W8Ørk붲YÚTõ«ô‡ºen‹ò|-¶ öàÖõñ—K/q&7„ú!Mv„›»–²rù*~Õ9`þüÏÛŸÏE*¾2Ðõ¦{=ÿx©6—E–lƒ
Vß»šS•a†¦ÿ…•ré<¾ú\é*“½õk—}ƒ»Bör¡k}ïÝœ >
:çe¾$ä[™iF‹o‘ š=1I£ìåìÅ¡Üâéñêò©+{ãí÷±¨»'g Ç2bȹUnlꌷžsG‰Îù€FÁƒïè ‘Gd…›« ÎO„ö`îßP 0¼8V%é¦lSŸÑàÓ –ŽÉL0IÌ110 Du·ŽÈÃÒNÀ™®D²»·˜•À š¸ZË“n^ˆ8)jØ׆É-ù¨¯èi—™x’l<*$C’‡0XW/ø ½qÑh+bPÃ¥ãº/axF†pe³¬á6ÖóE²²Aé~}ý}€‰!˜¡Ð‰’Y˜ÀIXÒ¨ú+€£ËçÝ–ò·›6,Z±h9pâ-5jI 2²MVØÍæ‹ó:›¡¯äÆÊJŸlÑݺÏB¹Hª®G`dZ“ÄR¶4füÛõY ø¸§•8ߤ«\/-¢ƒ4Ê Â“ÑŒøò×oÀ²MŽ >º:RzJ¾-¯Z˜ iƒÙéSÅÌ`ó†eàÕ0é‘ £À]øiå .¨«àÔó}èc2¤ëb8´§Óœ^’eö\)£MôZºGó²Iú+ãîÓƒ…ó)Ó¼bPÆç)\º°ØK„²“±§6¦ìßYWÔ /‹ã~#˜Æg“Öj
š)J&Ä4u “%€>§û¤Oƃ©`M`°‹ú‚þ]~-7¶áaÉçÚÄi´¥¢q,ääV9?1Maº
Âøü!¨‡Z(U¥Rw DU
k2u·¾®/èOºKÿ@‰¬¥ÖXOÛ2Zæ%XÜ’âºE pÄÆ-{ ¼ŒÁNÝ®³„ˆWÈUʨg0`d]èóZý ¨ Œf»™šÇr,j
—lv†ú¤W"Ï/ô½,ýr31Ät f+%$tˆ¡T.~ð±Õ
à¾Ï—´tP5fðýÝÄL;zîôÌNÙ,ãÝ —4Iš« ×ÛËx/ûÚqÞýŸ ý¥ÖåÇ2k†&w8÷óȾ-ô.íæG_}œ?-Øk o9Ý›½PèÓ$³8F®‘gUÇ:N Âs½Wèæê\
À=6ò6ß?ÞÇÑ“GÞ;2üÆõµZ
«·Ò;ÿ-ŽM c¶–C}‹ÛZ×ræÌ'‡OB‚ýú3
e®›,[丅ÙáûÁÓTx–wELgWOÏrsã ëYÜÞ‰1¨»‡†sàåýTKÕ=¤Ù¤¿QnÀÃr
Ƙ嬾¢ž<*UVâñX<[›êLКjçb2^ØØK”ú{-Á‡nò§ ѽŸl’>I²Ÿ;Q–! ãðW„Wu—ú ô}ÖoË Ÿct…x^MÑ IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/testsuite.png 0000644 0001750 0001750 00000001650 11256465172 030050 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/5/04ÁâÄ° ùIDATxœ¥“KlTU †¿sï¹÷Î rgZ“Š¨ j6Z AT0¢)1² hbŒ+MØ–F 1ÑnLPH1£ø
¦Q E‚AH5Z[ú´3mæqŸsîÜã‚hØóïþÝ—?ÿ'´ÖÜHäõeÇð+YÛ%e™ ¥¼9çØëm)»ZQ8©Ã`Nªö)Ã4~vâæÌño>Ðâ?‚/½ûPlØ…[Üš+¸®“͵m0$Õ¨…×ôH£…XW*³2ô¶‚ƈÐZ³ÿÀ;îLÎ=án ,QQ‚±J@µæ4<âÀ#T&{qväæ8Tï#¾\½œ[.o‘ Úv6ÖSãžFÒæÜÙq–H R¤`jÈØ¢ÿfxê0 9/ã›ù^×rz%À?µøÉûÖw‰Mù3k3g¥Lù óaL9R„¶EÈy»¾£Þó˜f7=Êa’”®(Ìû_xz+¬ëaô϶
ÝŠZTÊK,úS+ AÐ@6p˽ҟwøâøÆggWK€Lƺ^óyÿó“|úåOí~„8Œh)Íðžíô\I0ˆ¬Ä€u$êR½‡¤-RE|wìâV̺¾~V’zSbYÓµ@šm º:=¿Dyq™ƒ›¸ó®«ó $a“ïGŽ2û×ùUÐaCw òMèy ÄU $§ SpëíkyîÀnö¾¸‡µw”˜™˜`ââEÂz•®N(v@— (d¥RV¦´7 R• IEäG4k‚84kU„6Ù¹ï }¢B¬éÎU06z¿éŸÂ˜” "m+_,.îúä½1MƒÚrS
æ'§ááͤm¸ôÛ<çOÿÊ…± þòõ[õÿ¯üسo®IÛí7ÚJí׈¬e[¤Zã‹t\®.-á5½0[? C8™Ì¡ ?{5×Û¸mßëŽNÓ!Äã± èI”R-¥¦Mà Nù
ïá‰#¯i q£:ÿ €:e© ~ût IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/testgroup.png 0000644 0001750 0001750 00000001450 11256465172 030051 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/4/04y^£Õ yIDATxœ¥“½kÔp ‡Ÿär¹$wµ–³/¾P_PÁ¢qQ'qrqtwÿAtA A»¨ bÕÁŠ¢"Z_k«zæ¸Ëå’K.Éýòû9 ‚¸ô Ÿñû >¦”b9§/ë0 ž^Ô¶(ŠGÌ•ÎVèìH»É6ßkŸ™¸&Ob«‚¾fpXŽëEj^Àƒ“WUþ )¥˜ºbN® Û»¿ºa;Ž
Yäòâöä×¹Þõ›•
;ûœ¨<?/MâPµêÌ—Jšî6Ï šY¬ŽŽQYñ¦m³e×Æõ›ÆÇVõ tÃ]¢IƒÄ9[]©„¨œ&ê¨øé«NíãžJµ]=÷6Qrˆ¤Ñ®YdY™Ü,6º-#áS˜§j·Ðl¨o‘ï‚l’'u|·KO®¥ÕHÈr›4nã×ßbé V±I˜Á× ¨fÀ‡E€'?‡tˆ-üx m¯‹;7M¼$’’ EJ6ø)´#^§Ï
€0e!A°`dQ?Jô˜}v‰<» ¦–eJ&|Ÿ…N̽˪¥$‚Nt¼´Y§' ܹûtËÛ†¾ÊbºüjB–óhiQªÚa ~^4g!"ž¡² ¬ %´xÿ
‚˜w"çé éÖ[Z£kˆ¬†–' UAià‡P÷t!I©Kʼn 7”·Š^©æÏZ>8…z ®íÄ n.™’;<9w]}ùk‰ §Žk*6·Ê6ý
• ÞHÉ]‘3•öxa´Î_WÑ?]XìǘÚÎx©ÈŒL¥=ü 7•øŸLÚruþ
ÝWå€/•¦ IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/saveas24.png 0000644 0001750 0001750 00000003126 11256465172 027447 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/4/04y^£Õ §IDATxœµ–[l”ÇÇó]v×ÞÏÞ]¯½¾_jÊæB
Á
.¢4Ž¢44ÍS©Q©"xhµA¢ê{+UUâÒ$m®R+òà¢B@
(¦”8˜`—¶ã]›õeíØìÅ»Þý®Ó‡R«Äí#çufþ?ó?sf„”’{Ê=U´ÿ·ðÔS‡Õ@ ¨ƒ)Ë
>Ÿ¯¨PUJŽcÞÔ4±hÛÖr"1e¦RGÝÿ¥#þ»D/¿üUJÒumëÒÒò6Æbu‡BÕ 55‚A Çq(m²ÙùüÊçKKÙK¦i]…ª>u<ïS×ñrï¼ý#o àðá÷ëm[~³R©|èÇ7ÐÝÝDss”ÖÖѨ‚ªþûmÃü¼ÅÜ\Ž¥¥ Å•,ߌs;›ÿDÕį5EùË›oþ w<8tèh”¼_[kü6 oïÕ4èêêdçÎ8aÜå3,ÏH æÏýI£Ë
ÆÂßÙ³«‘]?@¤¶–íí½5Fð÷À/¼ð«ú5€ëºŠ¢ý‰'¶qðàãlݺ'Î21QaáÒßøì•LŸÄ”–8e× urÔk˜ ù+¯ýî4_Ž4rè;ßfÏö^U}ÎqÝ}k&;Ž}<¨êkn6D©Ñè¤LpîÜ{Ÿü*bÿÂñ ‚?ÄUØzŒÎ‡Ú˜(GIO|Ì#;ºQmÉý
M ÇqœÙµ ÆÇS¯¬¬ä?¼xq‚[·\lÛG<¾D"£
6?³¥V£X\ÄòŠ8Îm:7ÕSûp |t;»¨«av1Ç™+c˜åòE)ܳw™ü⋯?FÏ>ýô®@©™Ì2çÏŸ£µ5DWW;Éä¶m#¥ÄçÓéîŽ1•^dzr½; ã¶VŠ9Þ>u
Û¶¼ûÎK¿YË P°F
…ÂP"1A8ìCQ6oÞÄò²G2YÀ0ꇨ©‰ „Ÿ±ÑÒ©%¶='l ø}ŒÜø ÇqòŠ"ά»hž·êºnàx:=×ßÓs ~¿ ®§¯¯]÷áy¶mcÛ–mÑÒÒ€îyÔù«0~
v™éÌžç¨BÞúîZG ’¦i•J¦³²R¤®®šb±„”’LfÑÑq†‡GK0ysš\.‹*ÅU“p(ÈR)Ïj¹ pMÓuw àN»–,ËvLÓ&1Pɵ«ÿdhè årŽhTÁ0$ùüm†‡G¹>™ÂAb„ª0 ˶ñ<™¨kš_˜EB“RªB|~ 7&HϦxöÙìÞ½P)ÊXV‰ œ:=‚c¹|ãk ë:B@àŠu€ýß?"…>¿¦Å˜O/°˜™cß¾~¾²¥‡¡¡«tvvÐÑÁï³wo+ê '?¸ÀÔ|†®ö&ªÌŠÕî8ÞÚ,Z+Q®˜W5MÝ‹FDSs-3“³´´µÐÓ³…ÁÁ!Þãø‰³‹¤„l¶?ÚK}}Œ›ÉÚ¾¡9dŸí¸Më AU{À¯õoÚt?^œ“p8¦§§Èçs,}ž!öhiÀ`PÁ0j)dWQèë}]×Û<Ï}n @ÓÔíªª¶Vª)Ý6ñIX-›$'PTèêìÀ±=þÄ‘WO0ôÑ%ÒéBxXIqÉ¢º*ˆ¦«Á÷žþwy l™6WFÿA¥«‹é™4ÇŽ ö-bQ¤”$S9)¥Ð´IªÎ_Æs<ºï«ãòXŠ‘ë×±L!DP}w ñ±å:§o$“ýs™yŠ•
žôfÍJ夢(ãf¥”+Bâ8nh9—ÝQíóëêÔDç•ä
²+…;RâçÀ |áEÛ¿ÿÕ˜ëy?–R>üѧk¿DŠ ϶mùÖ[?\Û|ðàëb9—U=áúW B¼¤ªª¥ñSo¼ûÞOÌu€{÷üWñ/u
–ÚSø IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/runpass.png 0000644 0001750 0001750 00000001610 11256465172 027506 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü tEXtCreation Time 9/4/04y^£Õ %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï ÙIDATxœ•’]ˆT†ŸsfÎü¸óë:³®«±Žšk]ÔEQ&’Ð`îE*áA(f´t]FEÝÅÆân‰NS¨íZíºã쬻;sæÌ9s~gæë¢viË‹z?¾›¾ç{/^D„ÿ²Œ ÜÉWD„¿kïèý9 ªºßït vÜ Kw¶L¬¸oUSf èÊ🇫 /œß6´yý=ß<3x0×ÝH£]§â–¹e—ù¼t–¹©Ó2½} ùÖy–ö*Àž¯·=x÷Cã/ny-v¡ý×"†íÐhÙ8¶×
üPþžùŸ–°mowsÀ —a‘ðò÷LfÝèó›Ä¾
N(Š¢°<Ó7æ˜TÊ„Qd=0©œ
@[xüÓÂ+OlÙ—ÿÂxº£c: ^Ð&è´ùñb™—ú1úä–ßB‰‡!®nÒ®‡ö ¨ ]UÙ ŽÄ¨´*4m‡¦íÐr=fgêìŽ?Å‘Gr©1N¤–BKF®ì ÒÝö»5‰é8LOù ø[iÝ{ƒ›%“s‡NÒô ÎL ¢Ä鉦°4—N`<°’ÀsÛ‰ŠUÁ²=¢>Þ{–üo;9þÈ›dâ>™þˆ[V•ll ÉHµ B„镞ݩÜnÕüŽJ56ƒá48ñìiñ¹´4Á»¿œ¢Í] ¦ TÍyÜš T ß’oëzƒlh=ѵ + ¥lÏ2ëÞäää;Ü—ÝÎPß.Z’¥Å¦nðåJßÎí\›ë½²}×6$6áw}ÔNU
‘Ô’4=‹Ë3—)^(¢ëúùP^]I0óúbqi¾öÁÔÕ_©Y:—\O ¹ž>R‘4Usž‰k”.–0F áåþ¬ªrþxöýl&{¸oc?©TЛ
ª³s,,,â8Θˆ<'#bÜ ~Y},±&q(Ž>ÝévÓ¶k®çŽ§eD¾ã úàÿê{˜Àk¿Øb IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/windowb16.png 0000644 0001750 0001750 00000001664 11256465172 027644 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/5/04ÁâÄ° IDATxœ¥“Mle †Ÿï›ofvvÛín[è¡®t«T-F-1‘DcÓÀÓGšxQÓ³Q¯˜õæŘ&`“úÓ&IQÒP¹ @–n¡lÙvËÎîÎÌîìÌçÁh¸óÞÞÛsx ¡µæQ¦ >‡?p,SŒ„RŽyFb¯¯Ô@Ò¶†-¥zZSûÞš
£ Òçífmu~á -þ#8òö§/7¥5-2Ý“ÙLÆv8–R±´¨×êÄA“0ð1+Eå7¦[ž;+´Ö {÷Df5™ù%óìØ /î ¡
.nxl=ððª5š~ ?4xÓþ“‰ä3Õ§h¶
Éré€Ð–ý´ËçÝvÄÅ߯Pq}H9a†'Ë}ƒÃ·Nò•ýžÑ™ËšvNÜÝnNîË÷Š¼UÇN+Öˆ¥âm6x¦IÃ4¨Š€µ=ÌùÇQª—þ°.ü°=ò//MMŒ39ž'J-˜ÿá7\·ÊÀ“C¬Öb*•2Qß(¹C¯±Û€Ù ÏrXìR ÙL:—J%XÙöùnñ
û‡HîÂ-U}k¥L÷{Ø(ÞâïåËT_™ÓFÇíPÄ åzDe³F2±~Ä¥Ëëœüìs "Ö1gOÿÊTþ9¤)LøÁ}×mt8©^zvîDÙë%<¿Í`¾‹CGß`0×ÇÀ`/û÷£U'K?Ÿ!·%@uskùN©Š
ÏkÑl4 ´! ǦoGŠRa…3ßÏSX)Шn‡J™I€ ^=µôÇuên C™˜ ‹®l™lº±ÅÂ7_s¯p ½xAÄ¥Åó´ÙTæM ÚáÂʵ?š™ƒ8¢»'CÜ
HX’‘ÑAŽ¿÷ý» ÇoI”e³yw(lynî“êÿ*ï;2ý˜i[ ¦RαdG*i(“ÎtšÎî.jj”×ïxž Gñ9 dzB0³8ûQ ®q|ò}ÛNØc^R¾ª£¨¿ E¡!ÛÒ‹q _ÞÕL6íŸùöc
5ç êP\‡H¥W IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/runfail.png 0000644 0001750 0001750 00000001545 11256465172 027462 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/5/04ÁâÄ° ¶IDATxœ¥“KhTg Åß}ͽ3“8&‘4VïDQ ‚Eh]X¤nléÚîZK!è·5¨+Q´à¦ Í¢êF„R.D0
¢‹h&“Äd&3sçqß÷ûÜè kÿ»³8?‡ÿJ)¾äŒOÅC!z\ؤÃ
†%"1ˆ’:Œ×áŸi¸C©@|Lð¿»\È}³x8;4ˆÙ¿#k“†i«EX«RŸ] 8]e¶ Ýñ4ö
–PJ1&D÷×&OV Ü»®÷àa˜ /Ç æÊPs‰ê
â¦&a˜òïó)^ÕÚ¿ ¼Û€}ëz÷í‡Ç˜º÷ ‘é¢Ë[ Tɘ¿è.ëía` ‡¢ í nk yø1;4“¯™¼;ÆܦÝ,:‹©Åëá§Cô_ºNºa%Ê4qrŽ¡o锨Á6£°ˆøÅs”S`èÀ>òkÙpõº&Á2±ÿáÝÈï(vÆBÓµþ^P°ÌrL¼Ò ywž‰cGõ
zï!ßÍP¾réºÇÆÔ5””«:€Öø¨jR9Dߺ ’’GqŠ4,tà ”¥Òì B˜óæ+5LËfÅŽïÀÒP&i«žuX9:J×;IŒFÄR;€6¼©Ï”QJ#—&T._$zöŒÙS'™<}‰@'°ˆ¶ë¶ðƒ¸Ø)±wK³µ]}ìåK‘n™ÊÙÈ Ä²L^Úí•rƒb¹N'ÿu þ.5ƒsb~Õš•_‘/ä±2&ZOaè$/Çñ”†[kñôõóÕ†§dró³W¾¤›ß
S urÎ÷}Ý9²yòH•Â"j
Ÿj³ßŠ2N~=¡Ò±Ï G…ÈçàgÍ032Öfaè¤RÇI#¢Çù—‚G甚þè_:ç÷<ÙQ“~Ñ8 IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/testgroupx.png 0000644 0001750 0001750 00000001411 11256465172 030236 0 ustar twerner twerner ‰PNG
IHDR óÿa sRGB ®Î é bKGD ÿ ÿ ÿ ½§“ pHYs ÒÝ~ü tIMEÙ /(®2ý2 ‰IDAT8ËÅÓËo ÇñïÌÎl÷¥ÕNµ R%HÚTC Jp
.âÀ NÎ$HÄ…ˆˆˆ. D$^A‰×¢ÑV]»³³óÚyƒGâ¿?àsúþàO x|‚¥1òöäìÌ2AÎô¹ug¹¦Ö_» jï" 'ÄùsÚ£~Q¦ êÜ=pð/àÑiùÞÜÞµë•E=dÒàYEžß¸791¦Þ_°$Õ˱bVÆÊNMEßÞ3¨(™©†A,V Uˆ„¤¬tõ’ëè{šd:ÍÒUÝ ÷–w·6ˆ’b„1Gw2G•Ùb‡aZY׉wI ºæ¾4 c9¥ê*b¨ÒÜž$ŽÚpªµB
ÏË&‰%]ÅM)ÉaܱÝxµP)ÇŸ-Q…Ð)¡ëøѪe/LãÚ5´Ò[R¢JJ®`x09C\ÑyÿP£Fµ õi¬rÍ^GM£½zÉÀ±<¯7‚·ä t_±ÃëÈ» ÏD ÃeÚÖõ }ÏJ>Ÿžžâ{=Ïó °â&´šÐ™‡Æ7pg ‡[g®S• œ ÓÐMÕ”Úü@¡8q›„-`¶Â—
Ðw d >í…I ¼a Àrãš¡3ºja›ý‘\#dÓ 4C}3$+`̓‡è6£AÈ“?€ãc”ªBÙ´»€:´)?À–“0±ÄP!{_A ‚ß°âÊׂOó,Û…’Ez €ôàÒf®Ê2>4¾#”Žü*Q8wÿà Î>ÉûƒÙ4MeØ xÓ?ÆèÀ ¶½žÇ)1²%C ¸ÃJ,áwÓûw’ˆCÖ"ý
2 ƒ€Ç®vü*Á?}ãY¶DöÆ#D IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/textfile24.png 0000644 0001750 0001750 00000002725 11256465172 030015 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/8/043ˆ m &IDATxœµÕ]ˆœWÀñÿyÏygÞùÚÙ™5;i2ÝMv“Òb—j‹Dƒ¨^X)X¥^´P‹ªWmª^X¥7J
E)ZõÂКÚÄ61Mkè6É~u3»;;ŸïÌ{> /‚mF¯ô¹=çÇ9çyΣD„ÿg˜ÿ´pòIÙÉØ1k ÖqÄÃíΡCY¢xÍ_‚çÌ?7i õþüýJ»ŒE›qw–©ObÌG¢Ø4¢\/à3uïöˆ^Ï~æ ßyèç2ø¯À«O+…å>à{qyºYiÞAeÿ‡)TIÊ3D¹â Rü¸ÅÚ›äå^¢?á9 ¸ïá§ö"{®ÈŽ™Çó£úÂÑúüÇ¿M±öy@ãí*ÞÖÀ§ˆm–™æ¹$AzCPÜ#Bûɯª‡¾ñŒØ‰€·|VǺ~ó±G(Ö¾L÷êIÞ=ý4¿bÇ;ˆD‘¡P*R›ý q¡@–9B /÷ êÔ÷¿¤~õ_‹ßüþ*2ŠOL lRÚ÷9»§yåÙ»i-÷)Tªë46sôw·ÙÝêÒ¼õ(I±È /DÎz|à à%`m cªÙ»Ê[PÔ°éó,Þõ1ŽÝ³D©:…¸ 7 àÇìhHÚë£tÌRaša»uxëì
C+Âõ‡½xG%æò•ÃÀ•šOî¤uå
+çÎàÒÁ[¼Í@…©
•™ÏÃá›H[ïò×Wž× ý¬ß Œ†ÜœËS0I ]Îÿéo¿ò7ÒnF¤Áˆ h£…ÒŠ(2TûYøÐí · c Âk«( ð¸ô[—.RÝw‡–¦™n4ˆcC¤¹|ŒaÇÎÖ6ƒnܘn{€Í‚ ' ÖBpd— ?N¹ZÃ9O–¦x;Âec¼ËPÊPªÕ™]8ŠÉçl.óö™ dc x<€2"£Hw·Y>ýÛ뛤ƒ<¨kùùbÂôlƒÙËìk”HS³ˆŠˆ&.€‰À¶°i› /Ÿbû½!…’¦T©Kò˜XçóH!Aôz¬_¼@19B¤”ìd@Ðʃ ·ñ¶Ccn?‡–jLÕë˜\¥"“Kˆ %La1EÒN‡0lQ®¬ÇDãŒÆD ÁG0îï µcÿ¡&Dyv®îÐow±Y†Ï2‚@œO0I‰™æ<õƒ‡(Ì qà2qnE¥CûQà™ þé:ï]&WLØÙÜaùü2ýÝ!(
ÆD$å
0Äû«o½IR¬qà¶[hÎÍ2]+ÐíØO?þEUüYÙÙd-¥m'.ëãƒÂäò,Ü1GiªJ¥>…1 çQÚ L‚ šÁnÈ€Ò¹¹2«Wz Ÿž{ lGš×[W‡Ç6οÊÁ#G¨NÅäËElvÛ„P*BiC¾X¤Z¯1Õ,á¬#ënÓ˜ÑZEäÁïÞ«žì„ØëÀ?•ÁS_Wg¬ãØÙ“§¸uiŽÝÍM®¶,½^Æ(µ8/ˆ@E¤5&—£V/QŸI¨M ½v— ‚R >¼¸ç»ž¾¶ry—µ !΂ð$\+i àÃÕË ¢Hi…÷òï61€Þ[E€óüY)~+Âœõ\Bñ"p6ë@*
”f€Û¼—;½“% Ž¥à à0a&ÿø~µxTÁ²¿C³œC¾ùËkÉ?üŠRA¡Äe–rQæ&p8÷؉kçà ÿ¤r 3 Ð IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/folder24.png 0000644 0001750 0001750 00000002501 11256465172 027434 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/4/04y^£Õ ’IDATxœ½–Kh]U…¿}Þ÷‘››D“´IÓJ›Ö×ÄêDE Hq,ŠŽ‹h¬‚ ³‚"mqäÈZPAThë£uЦøèÀÖƦi 7÷qν÷Üó>g;8·)Ò8¬?ìéZ{ÿßëßBJɵ,嚢ÿ Àé…H&=›;¢P™V b›VT¦QµÛƒ0zñË»gå[ ¯?*”£„D\_´¨ ,¼¸_¶7"RJN¼'îDªŸ”®ÝR™D/X”ªUJêÎó뉹_Ö–äžb?ŽØ ¦â–BAì æÖÐÏF_Ú]žÞó¾¬m¨Àïò@ujlËöÝÏb”F6„->ã,ž]ØѵϪCZ±RÑ Ò¨Va¬ ðùê×êÜóÜCâ뙊©TŒÑza_âj ½®øs¤PÄ(X×!lCâƒÐ…·Mk7͸ÚØx M Aõ uÀÈT „áªùø
êc##Æ]ª*&–jÞ‡÷ïTŸ× ‚Xžs›
‚µóXC#ú CH|Ô,a|Ë8Béw:62ð‰½2¾_Ä*›ìº»ðÈ`1Ã4%dõ†÷ÌÔ¸|7'ˆh^àFn³l
¸Å Y·Mà{ÄaHäY¸î4a4€b’éE†—QÒe"¿K»¾ÆêjÂêÜ8ÉV
@Ñ Ýn²¸™Š È ˆñœöZU›"Lwà…¤0QQºBûÒ ‰ ‘4i†®ƒU„X@/¢&àŒÖoµçöXõœÆ ÉHÀs˜?Ó¦´éI¤ZÀ÷ Ò$ÆY:†³tœ$E]ƒRJ0uH¨µ Mù¢vy
`ñ"½MÙ%×qÀoAⵚè•{™Üö ? ~‰•sGÉÐL0 @SÁ4Á²@UAh÷ nƒ|³ÿ˜”€T!IeÝm»wˆ£ zÀðÔÍœ=þ&ó§ŽR Õ úÑ%¨è:h(ýL¸Xƒ8¥•ÁÉõw A&™\r;>„iàâC‘Íò G¨\º2¡€*r{ Œ>Ðóa 2㔀¥+*2ÎXëº1ÒoØzq½æÏtZmJå XÑ@UòÛª}ÿ/ƒ›œ¯C/ GLúa7û‘”IFÛé¤Ä=¿'šŠ½<‡¡ƒe€i€eæç²ïåŒ\QÓ…$ðéìA™®+ HSZ~(ÃVKšeDTõW Ãì+¹%†I
í.´º`·óÉqPöpîoY'4µ˜æJ]Ý<XÄñ2Y")æÍTr…h¶¡ÙÉ}ÏúW¨:o¿ràÊ['ðCZªFÇÙ<¦ûØ
b1¿yχ–5; à €8ÉJ~‚CŠÂ·ªàôÞÒ½*M¤ÄcæŸO·k¢G½•Òós]ˆ¢ 0“tœP¾ð¹¦±*íÙƒ2Ü` äûàr½ñ”Ø%%‡4 $„ѺôU)9,? sŠÂ¢¢¾zPÆþ#ÀkOˆû¤à ¡"8 |'?*
6îýàßý®"ø¯ëš/ý¿ 8A¤aÒ IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/floppy24.png 0000644 0001750 0001750 00000003062 11256465172 027475 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/4/04y^£Õ ƒIDATxœµ–Ûo\G Ç?sn»Þ]gw½±ã»ni¨ !i¢6BÁ¥j¤ªŠÒ>¡J•TáJþú@L!ˆ¦—JD¢ÒD¢Ž¢ÔÛÚ
N·!Ž×vÖ—µ½›½_ΙsÎðà`‘ ó“~O3óýèw™ßŒPJq?M»¯ê€ñÿžzê¸ †MÐâ•J½Õ²¬ @שº®}Ë0Äš”N.™œ·çæN{ÿKGüwŠ^}õºR"jšÆžl6·Ä#mm-_FC½ÍÍAÂa×u©T$ù|•b±¼žÍæ?¶mçJ4Úô‰ëûŸx®_xëÍŸú[ Çÿi»”êûFã`ÿààNÚéèHÐÕ'‘ÐÐõCRÂÊŠÃòrl¶@¥œgü£iîä‹ é†ø¡iï½ñÆOŠ›€cÇNišx-
? ‹5‘J¥ æàÁ>j5д
|Ã
èéÌz‰“'. ù°^ÎS¯Õÿ¬”9uêå¬àyÞ¨¦™?ñÄ^FF¾Ãž=;9wî"33
ÊeÈå P€rŠÅ
__߈DJßýá_ŽïàØŸåÉ}ûÑtý9×óŽÀÝ.r]yVTGG„j‰(åpéÒ–¦ „B`Y ëàyÐ× ŽÏžY%²-Œ.¶¶£Àu]wi0==÷ër¹ø÷«Wg¸}ÛCJ‹ÁÁ$“Il»L_Ÿ‹çå¨TÖh4ÖqÝ;tw{Ôê5Þ;?ÁC}ý´lkfiÀצ°ëõ«Jxï)òK/|4‘H\|úéƒÁj2™ —/_¢«+J©Ô*RJ”RX–ÉÀ@óé5fW9üØ£|µ»‹ÕJ7ÏŸGJùã·ßzå·› ”JÎD©TK&gˆÅ,4Mãá‡w‘Ëù¤R%"‘b±Vš›ã`jr‘ô\–½
‹„ ,&nþ ×u‹š&>ØrÑ|¿æy^ðl:½|hhèA,¶v`š¾ï"¥DJG:tv¶bú>-&"Á %Yg!³ŠïûºP·ÿ£»ÁéÓÇ”m;SÕªí–ËZZBT*U”Rd2«LNN3>>ÉÔT’Ù[
yt!¨ÔlbÑ0Ùj‘Z½ð™ašÞÀÝv:Žtm[GÐ4Åg×?glìõzDB#Q‹w ŸäÆì .ŠH´ Ûµq¤Ä÷U^ oj~a C)¥ !°37gH/ÍñÌ3ßbxx7 S*Õqœ*W®$9a×ñøî·¿iš! Ol ýÑëBÓøŠ0Œ¶ÖVVÒ«¬e–9räßÜ=ÄØØuúúzéíÄ8|¸‹`(Ìû»ÂüJ†þžvš‚Aì†Óãºþæ,ÚLQ¡RÔ
C²-í ÛXœ]¢³»“¡¡Ý¼ûïpöÜE*•JA>û ÙÏöímÜJ-Òý¥8 mq@ ®×¾4õÆ¡]» įƒ[¶‰Å¢ `aažb±@v=C<æÓÙ ‘„ÑÈ6Jùšö
Ó4»}ß{n À0ô}º®w…‚!ªwl,µºMjvM‡þ¾^\é3:úW^?qŽ±|L:]B §¡¨d BMa SG^|þù_í¸§êŽ-¹6ùOýýTqI/¦9sæ R:´¶%PJ‘šË¨Y¥„aÌÒtùS|×gà>šcâÆ
ÛFÑ „îí"!>t<÷ÂÍTêÐrf…J£¯ü%»Ñx_Ó´iKJ©²ÂE ¹®Íò…¬À®ÏÏô]KÝ$_.Ý•¿á /ÚÑ£'Ú<ßÿ™Rê{À_,Óx
%
|)¥:uêåÍÍ##'E®×}áYš'Ž !^ÑuÝÑ„ø…€ß¿ýÎÏí-€ûa÷ýWñoIl¼É’îŠ IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/edit16.png 0000644 0001750 0001750 00000001407 11256465172 027113 0 ustar twerner twerner ‰PNG
IHDR óÿa sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 2/7/04,üø XIDATxœ¥“MHÓaÀïÿ?Q[s 3\*µ@kBVâÔN¦„‚¤x)%!,èСCI‡‚¾¿”.éÁN&bB!DÒ-£ÐP3*]ÊüLÙ›m:·¹ÿÛ¡4
WBÏõy~?ž÷RJþ't)ª;%b= NJ•¾@4Ï« Hm9§ü ¾rHè£õº™ó¯&%íl^JlÙBl¨ƒÚ|aH´êêR³KJ3Òqê_Eyf5åíÓ€Çí ¡XS¹oÍ9Pº7¯ 9¬7o#wËe›jn8_¢„´& 5ÁuÓ’•U¾¿p7:S>Ë"3Ó$Žg‘,0DÑ V /¢Ú¼'½*؆Jš½ _ÔeæÒcÓ†˜ ÙBú×ÝAs•¨´dî»´«ÀŠª¶²{Ѧ ๆKï¦;~€aGG¿¢q@¬~+DŽÅ–Ôf+Í3l2v ?;9aƳÓðú Nc(½÷Tö®¡¶PÄ·›êSò3 zS;rÐÁ’œ.p¹¡ó# 2$Ý”/Ãk›¨4&ZRÌÉGññOüÝ.èê…ÁaÆC'n½«GV î” ³yÚv$A A]!®%3®è껃ñ@€ã·ŸÉŽ?w¦P{4SÒ¤'ž˜@7Ò7Êè”BϘtòÉ秢¦]¾]ïb¿® æLÌ™9{î%E™ýÄG|ã}ßwéÓ4*kÚe÷zðŠ`1$¿†FHð0Úã Méèõi*-ןÈ/áàׯÝÕ¹íæ2Úëçü~ƒ‘Ù‹~ÿºpñŒ)ò¿6×a IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/accept.png 0000644 0001750 0001750 00000002754 11256465172 027264 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü tEXtCreation Time 9/4/04y^£Õ %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï =IDATxœµ•{ˆœÕ‡Ÿï>·Ëîì=Éf7Z+…D¬š¢6mj$¥Å’H«¥J¥T¡©¥èÒ*¢ ŠV¼PC¡´¡¢F"Ú.hmµk‚&»‰kÜÙÛ$ÙÙ™o¿™ù.ç;ßé RÉJVýCß¿Ïy Î{Î：RŠ/³ô/•˜Ÿµ`Ë Í^=¯o·ìô¤Ô®”±V–¨Àøwìëãû¦ÇêSkí×ÖjÑ ßm̤þõ‹L.wçÆüèH_f„ŒUÂ×ÚTê rta‚ÅÓUhXÀ—{…) Z¸»¹ô¹Ûž-–Kåâ¾ÍýW_¿sà6r›h¨µè4u±ŒH$†éðúü?9ôîAš3m„«Ž…I|síÞö‘O|ý/éüàpÏó».¼ãšÝý¿ä$o1Þ<À¼_¡î»´Â) t‘â¢ü<ÙæÐñ稟p‘õäT`Šoz¿*çÜüÂÖtêüí£?ÛyÓº_qX=Îñø?D´A;´¢ˆ¦ Ð |A“~{2†ÿÎNœˆHZòÅ0V»ý‡# >ñŠfV&w}¥¼eçw×ý”ýÁ}
_C‹ Tb ”
HÀ4
|?¢v2àÈÌæVfɤ;= ‰b'pýÿ™ 6993kÞµcãy' ç½ö[¨Ð!c"Ë‘H4C£²°ˆ>Yæ¹ëÆyeÇÔÚËx‡ž5Q)
bõm—f®hâ[ —n.å»yµþ2Òi‡í($ˆ"B#•¢î`œ*ó×0:8ʲ¨¡è† :‰d%zᜠX)ûÖ¡âW™ô'˜kÍ’µÒ= ”º²B29±È“Wìg¨{ OxŒH
Òz ]—Ä) Jdй ˜?'hÆ嶦æŸe%ð0u‹…I—þì íÎ%4ScjºÂöÜ.¾sév ^Z|ž—æ_$edɘY„Š M*„„âêKNôõM<ÜÀCˆ„Åå%.Š¶òàÅûØfüÙÚ"vµ›±m¿ 7ybê1ü8¢Ã,P° d¬,$€"YÕ¢D誺RÕRºƒˆ %ZŒ®»„¾®nn/þ†©ùiz6¬g}×: Ì<Ãøâ8©!ŠN‰¬G†u´HGª4¼UjÇ—½ú%¹l¤MÊÒyÍ}…[üŸ“I§»ê>,Ë@qÒ›âÑcÑi÷Еî¦;Û‹®Ôý±£”
ˆ9ººEùæ™3ghG9£“¬êâ}5Å£S÷Ó.Z*Aè³Ñ,{&~KK„ ç7±±0L9]þáùõ€„äebæV@—Æ~¯Öº5K=˜‰Ž“Îr¸~çD†.¼‘ Ÿ§ß{Šš§øZ÷fz³ýtX xÂÃ
\ê§ê„AŠƒê%V pxUxò î‚÷íR¾LÉ)Ó™ê£'5È‘ö;¬|ÐÄÔ-–¢e¾1x
y;¥Ûx‘Ç™à,Õé*nÕ%Nâã$ ü8_çþE#÷÷_kÖá¾ SC#ÃôeúÉš ˜ºñQŠ8f
€H
–ƒ3
ÓǦ™;9‡Ût)å÷Õ^õòy öôŽÙ¶ý@ÿÈ ë‡‡è-ôQpŠ8†€O´¨yKÌU稼_áôÜiš~“8ŽöªGÎåw ôÞÓy¯mY{Š]”z(–
¤í hà>nc…ÚÙKÕ%^ƒ0
=™È;ÕÓêÏŸd9Ñ
w徧iÚ=¶îlM§Ò˜Ž…¢0"ôCÚ‘ˆ#'ñ~•¨?¨½êíóqÖ h7j NŸ}µŽq»¦i—)Å T2–2ž”RŽ“°OýI½¹&à³_Dý÷´²qÑöÄ IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/images/delete24.png 0000644 0001750 0001750 00000002674 11256465172 027436 0 ustar twerner twerner ‰PNG
IHDR àw=ø sBIT|dˆ pHYs ÒÝ~ü %tEXtSoftware Macromedia Fireworks MX 2004‡v¬Ï tEXtCreation Time 9/4/04y^£Õ
IDATxœµ•kˆ]WÇû¼îûNf2éL2Í£3©ÄZ¤„Z¡ñi ‚ÔEÔЪ-5RÓ >bA‘B"-H5Ú¢©m!-)-Ò‚`´jÒ&3“6M2™¹sonÎäÜ{Ϲg?– JK§f¬ ì‚ým¯ÿo¯µÿ¥D„÷3¼÷U ÞëÂ~¥Fá3¸]ࣆ3 í |[df¹|µ\‹žQÊ/ÂÝõ€ µÍ /OŽÖk sŽK3§Y˜k3ßwq†üÒÃ{ðn±ÿ ðG¥†‡áÀèÖ›·®¾ë.Êãë±óç1s0Í$í¡Œ¦óÊ Î þ§š Üžpx_Þ)öè<®T} MìÞ±eä¾]È«‡ÑÏ D_˜Å&]tOcB‹†^ÊÂÑiŽÍÆ4´}Ý)ﻜ9{EÀsJ•|øýµ÷ܹmì»ßCž~5}gYG£3ÒÓËé%i'Å•jXc‰ßhðÏvËNþ¤<¾´ËÚÞ5E ܶjó‡¶~ãëè_?€:~˜ógcNý« ÎúŠsg[œ»ƒ5(Y³M7NJ!ë
>À6ql}KómÀn¥ªu_í ûÊí¸—_DN¾Âé©ÓzŒ¡Ÿ ¿éV¦^šá+Y÷èSÈ羈Ír¬1äYNÚ×ÔÀªï¡_Ý¥ü Þ1¦“ð©ú6|¤62Hþç?`º}fk¹ñ‘Ǩ Á^³Ž(>yËg©oº–ë×2W,‘üj?6×|"_ <.öÝçK¸`öí
ª°½:9Ž9y =ßÀõ-“›6P[9 ý >Âu÷ßG}rš(kîØN0º
…QH5ðqž* ê†%-R°9*Ð[ènŠ¨€ò±¿0ý{Èã6`¡Ý‚n|…]\dvï>’3o ¥~1Â+ð ”B`Å’ ¬
ó;lîðµãµÓóÌοÄÚŘ(£ÁˆBÒ©ã¼þÛÇÉñð*E¢rPåàû
·¤’¶Ú¸x±Ðk'´×³eÿ/¨\5üæË1ÐI(_=c;ï…j T X)ãG!¹§àÍÑO– ú0Õ\šbñqÚ0þÁ êcWAç28 JÀP‚§5×|óNÆïø¥z™B± @Û
ˆdŽ-ôàHûb Ú"Å"R @þþ7Î=ü"PŽ³ûöñÚ£¿Áù
Øf’KTVÔEèfšV_ã Ï[8¿äŒ<¹Ðɶ_îö©`”‡çû$Ï ¢9P£ßnsáàAråS™œ`ÕM72÷ÔïèŸ<A1 É»13I~®yæ'"z‰U<2P/Ym úôu× 5\`rMÞKÑÆ"Aˆò}üJ‰ÊêQl»…—;\£ÅÑ3ó¼ÚhcÒ|ÊŠÝò#‘ÖxÑÏëµûž÷즵«ŠFPC+P¥ñ|œRðÖA~Žt2Ò¹S³Mf—èuÓÌYû…=âž¿¢Ù<TìÃðÇ׬dâêU”VVʈRˆ±˜^J?î°Ðˆ™i\d¶}™<ÍãîýؽïÔ»â>Ø[*íñ¢ðûCõ
W0X«•"”@?ω») ‹=æ.%t;)6׉svÇÅ=öne7ÚO‹…[QÞý* >V((„>
ÈŒ%Í56ËqÚXçäI'ò³=b_¾’β €JÕ†ÂèfÏó¾%ŠÆÄ9#ÆN‹“ îÀ"G–x/Àÿ#þ
K{ZvÊp— IEND®B`‚ antlr-3.2/gunit/src/main/resources/org/antlr/gunit/swingui/gunit.stg 0000644 0001750 0001750 00000000457 11256465172 025715 0 ustar twerner twerner group gunit;
gUnitFile(testSuite) ::= <<gunit <testSuite.grammarName>;
<testSuite.rulesForStringTemplate:testGroup()>
>>
testGroup() ::= <<
<if(it.notEmpty)>
//------------------- <it.name>
<it.name>:
<it.testCases: testCase(); separator="\n\n">
<endif>
>>
testCase() ::= "<it.input> <it.output>" antlr-3.2/gunit/src/main/java/ 0000755 0001750 0001750 00000000000 11410174107 016212 5 ustar twerner twerner antlr-3.2/gunit/src/main/java/org/ 0000755 0001750 0001750 00000000000 11410174107 017001 5 ustar twerner twerner antlr-3.2/gunit/src/main/java/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 020121 5 ustar twerner twerner antlr-3.2/gunit/src/main/java/org/antlr/gunit/ 0000755 0001750 0001750 00000000000 11410174107 021247 5 ustar twerner twerner antlr-3.2/gunit/src/main/java/org/antlr/gunit/InvalidInputException.java 0000644 0001750 0001750 00000003052 11256465172 026414 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2007 Kenny MacDermid
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
public class InvalidInputException extends Exception {
private static final long serialVersionUID = 1L;
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/gUnitExecutor.java 0000644 0001750 0001750 00000065041 11256465172 024742 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
import java.io.*;
import java.util.List;
import java.util.ArrayList;
import java.lang.reflect.*;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.CommonGroupLoader;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.StringTemplateGroupLoader;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
public class gUnitExecutor implements ITestSuite {
public GrammarInfo grammarInfo;
private final ClassLoader grammarClassLoader;
private final String testsuiteDir;
public int numOfTest;
public int numOfSuccess;
public int numOfFailure;
private String title;
public int numOfInvalidInput;
private String parserName;
private String lexerName;
public List<AbstractTest> failures;
public List<AbstractTest> invalids;
private PrintStream console = System.out;
private PrintStream consoleErr = System.err;
public gUnitExecutor(GrammarInfo grammarInfo, String testsuiteDir) {
this( grammarInfo, determineClassLoader(), testsuiteDir);
}
private static ClassLoader determineClassLoader() {
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
if ( classLoader == null ) {
classLoader = gUnitExecutor.class.getClassLoader();
}
return classLoader;
}
public gUnitExecutor(GrammarInfo grammarInfo, ClassLoader grammarClassLoader, String testsuiteDir) {
this.grammarInfo = grammarInfo;
this.grammarClassLoader = grammarClassLoader;
this.testsuiteDir = testsuiteDir;
numOfTest = 0;
numOfSuccess = 0;
numOfFailure = 0;
numOfInvalidInput = 0;
failures = new ArrayList<AbstractTest>();
invalids = new ArrayList<AbstractTest>();
}
protected ClassLoader getGrammarClassLoader() {
return grammarClassLoader;
}
protected final Class classForName(String name) throws ClassNotFoundException {
return getGrammarClassLoader().loadClass( name );
}
public String execTest() throws IOException{
// Set up string template for testing result
StringTemplate testResultST = getTemplateGroup().getInstanceOf("testResult");
try {
/** Set up appropriate path for parser/lexer if using package */
if (grammarInfo.getHeader()!=null ) {
parserName = grammarInfo.getHeader()+"."+grammarInfo.getGrammarName()+"Parser";
lexerName = grammarInfo.getHeader()+"."+grammarInfo.getGrammarName()+"Lexer";
}
else {
parserName = grammarInfo.getGrammarName()+"Parser";
lexerName = grammarInfo.getGrammarName()+"Lexer";
}
/*** Start Unit/Functional Testing ***/
// Execute unit test of for parser, lexer and tree grammar
if ( grammarInfo.getTreeGrammarName()!=null ) {
title = "executing testsuite for tree grammar:"+grammarInfo.getTreeGrammarName()+" walks "+parserName;
}
else {
title = "executing testsuite for grammar:"+grammarInfo.getGrammarName();
}
executeTests();
// End of exection of unit testing
// Fill in the template holes with the test results
testResultST.setAttribute("title", title);
testResultST.setAttribute("num_of_test", numOfTest);
testResultST.setAttribute("num_of_failure", numOfFailure);
if ( numOfFailure>0 ) {
testResultST.setAttribute("failure", failures);
}
if ( numOfInvalidInput>0 ) {
testResultST.setAttribute("has_invalid", true);
testResultST.setAttribute("num_of_invalid", numOfInvalidInput);
testResultST.setAttribute("invalid", invalids);
}
}
catch (Exception e) {
e.printStackTrace();
System.exit(1);
}
return testResultST.toString();
}
private StringTemplateGroup getTemplateGroup() {
StringTemplateGroupLoader loader = new CommonGroupLoader("org/antlr/gunit", null);
StringTemplateGroup.registerGroupLoader(loader);
StringTemplateGroup.registerDefaultLexer(AngleBracketTemplateLexer.class);
StringTemplateGroup group = StringTemplateGroup.loadGroup("gUnitTestResult");
return group;
}
// TODO: throw more specific exceptions
private gUnitTestResult runCorrectParser(String parserName, String lexerName, String rule, String lexicalRule, String treeRule, gUnitTestInput input) throws Exception
{
if ( lexicalRule!=null ) return runLexer(lexerName, lexicalRule, input);
else if ( treeRule!=null ) return runTreeParser(parserName, lexerName, rule, treeRule, input);
else return runParser(parserName, lexerName, rule, input);
}
private void executeTests() throws Exception {
for ( gUnitTestSuite ts: grammarInfo.getRuleTestSuites() ) {
String rule = ts.getRuleName();
String lexicalRule = ts.getLexicalRuleName();
String treeRule = ts.getTreeRuleName();
for ( gUnitTestInput input: ts.testSuites.keySet() ) { // each rule may contain multiple tests
numOfTest++;
// Run parser, and get the return value or stdout or stderr if there is
gUnitTestResult result = null;
AbstractTest test = ts.testSuites.get(input);
try {
// TODO: create a -debug option to turn on logging, which shows progress of running tests
//System.out.print(numOfTest + ". Running rule: " + rule + "; input: '" + input.testInput + "'");
result = runCorrectParser(parserName, lexerName, rule, lexicalRule, treeRule, input);
// TODO: create a -debug option to turn on logging, which shows progress of running tests
//System.out.println("; Expecting " + test.getExpected() + "; Success?: " + test.getExpected().equals(test.getResult(result)));
} catch ( InvalidInputException e) {
numOfInvalidInput++;
test.setHeader(rule, lexicalRule, treeRule, numOfTest, input.getLine());
test.setActual(input.testInput);
invalids.add(test);
continue;
} // TODO: ensure there's no other exceptions required to be handled here...
String expected = test.getExpected();
String actual = test.getResult(result);
test.setActual(actual);
if (actual == null) {
numOfFailure++;
test.setHeader(rule, lexicalRule, treeRule, numOfTest, input.getLine());
test.setActual("null");
failures.add(test);
onFail(test);
}
// the 2nd condition is used for the assertFAIL test of lexer rule because BooleanTest return err msg instead of 'FAIL' if isLexerTest
else if ( expected.equals(actual) || (expected.equals("FAIL")&&!actual.equals("OK") ) ) {
numOfSuccess++;
onPass(test);
}
// TODO: something with ACTIONS - at least create action test type and throw exception.
else if ( ts.testSuites.get(input).getType()==gUnitParser.ACTION ) { // expected Token: ACTION
numOfFailure++;
test.setHeader(rule, lexicalRule, treeRule, numOfTest, input.getLine());
test.setActual("\t"+"{ACTION} is not supported in the grammarInfo yet...");
failures.add(test);
onFail(test);
}
else {
numOfFailure++;
test.setHeader(rule, lexicalRule, treeRule, numOfTest, input.getLine());
failures.add(test);
onFail(test);
}
} // end of 2nd for-loop: tests for individual rule
} // end of 1st for-loop: testsuites for grammar
}
// TODO: throw proper exceptions
protected gUnitTestResult runLexer(String lexerName, String testRuleName, gUnitTestInput testInput) throws Exception {
CharStream input;
Class lexer = null;
PrintStream ps = null; // for redirecting stdout later
PrintStream ps2 = null; // for redirecting stderr later
try {
/** Set up ANTLR input stream based on input source, file or String */
input = getANTLRInputStream(testInput);
/** Use Reflection to create instances of lexer and parser */
lexer = classForName(lexerName);
Class[] lexArgTypes = new Class[]{CharStream.class}; // assign type to lexer's args
Constructor lexConstructor = lexer.getConstructor(lexArgTypes);
Object[] lexArgs = new Object[]{input}; // assign value to lexer's args
Object lexObj = lexConstructor.newInstance(lexArgs); // makes new instance of lexer
Method ruleName = lexer.getMethod("m"+testRuleName, new Class[0]);
/** Start of I/O Redirecting */
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayOutputStream err = new ByteArrayOutputStream();
ps = new PrintStream(out);
ps2 = new PrintStream(err);
System.setOut(ps);
System.setErr(ps2);
/** End of redirecting */
/** Invoke lexer rule, and get the current index in CharStream */
ruleName.invoke(lexObj, new Object[0]);
Method ruleName2 = lexer.getMethod("getCharIndex", new Class[0]);
int currentIndex = (Integer) ruleName2.invoke(lexObj, new Object[0]);
if ( currentIndex!=input.size() ) {
ps2.print("extra text found, '"+input.substring(currentIndex, input.size()-1)+"'");
}
if ( err.toString().length()>0 ) {
gUnitTestResult testResult = new gUnitTestResult(false, err.toString(), true);
testResult.setError(err.toString());
return testResult;
}
String stdout = null;
if ( out.toString().length()>0 ) {
stdout = out.toString();
}
return new gUnitTestResult(true, stdout, true);
} catch (IOException e) {
return getTestExceptionResult(e);
} catch (ClassNotFoundException e) {
e.printStackTrace(); System.exit(1);
} catch (SecurityException e) {
e.printStackTrace(); System.exit(1);
} catch (NoSuchMethodException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalArgumentException e) {
e.printStackTrace(); System.exit(1);
} catch (InstantiationException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalAccessException e) {
e.printStackTrace(); System.exit(1);
} catch (InvocationTargetException e) { // This exception could be caused from ANTLR Runtime Exception, e.g. MismatchedTokenException
return getTestExceptionResult(e);
} finally {
try {
if ( ps!=null ) ps.close();
if ( ps2!=null ) ps2.close();
System.setOut(console); // Reset standard output
System.setErr(consoleErr); // Reset standard err out
} catch (Exception e) {
e.printStackTrace();
}
}
// TODO: verify this:
throw new Exception("This should be unreachable?");
}
// TODO: throw proper exceptions
protected gUnitTestResult runParser(String parserName, String lexerName, String testRuleName, gUnitTestInput testInput) throws Exception {
CharStream input;
Class lexer = null;
Class parser = null;
PrintStream ps = null; // for redirecting stdout later
PrintStream ps2 = null; // for redirecting stderr later
try {
/** Set up ANTLR input stream based on input source, file or String */
input = getANTLRInputStream(testInput);
/** Use Reflection to create instances of lexer and parser */
lexer = classForName(lexerName);
Class[] lexArgTypes = new Class[]{CharStream.class}; // assign type to lexer's args
Constructor lexConstructor = lexer.getConstructor(lexArgTypes);
Object[] lexArgs = new Object[]{input}; // assign value to lexer's args
Object lexObj = lexConstructor.newInstance(lexArgs); // makes new instance of lexer
CommonTokenStream tokens = new CommonTokenStream((Lexer) lexObj);
parser = classForName(parserName);
Class[] parArgTypes = new Class[]{TokenStream.class}; // assign type to parser's args
Constructor parConstructor = parser.getConstructor(parArgTypes);
Object[] parArgs = new Object[]{tokens}; // assign value to parser's args
Object parObj = parConstructor.newInstance(parArgs); // makes new instance of parser
// set up customized tree adaptor if necessary
if ( grammarInfo.getAdaptor()!=null ) {
parArgTypes = new Class[]{TreeAdaptor.class};
Method _setTreeAdaptor = parser.getMethod("setTreeAdaptor", parArgTypes);
classForName("llvm.CC");
Class _treeAdaptor = classForName(grammarInfo.getAdaptor());
_setTreeAdaptor.invoke(parObj, _treeAdaptor.newInstance());
}
Method ruleName = parser.getMethod(testRuleName);
/** Start of I/O Redirecting */
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayOutputStream err = new ByteArrayOutputStream();
ps = new PrintStream(out);
ps2 = new PrintStream(err);
System.setOut(ps);
System.setErr(ps2);
/** End of redirecting */
/** Invoke grammar rule, and store if there is a return value */
Object ruleReturn = ruleName.invoke(parObj);
String astString = null;
String stString = null;
/** If rule has return value, determine if it contains an AST or a ST */
if ( ruleReturn!=null ) {
if ( ruleReturn.getClass().toString().indexOf(testRuleName+"_return")>0 ) {
try { // NullPointerException may happen here...
Class _return = classForName(parserName+"$"+testRuleName+"_return");
Method[] methods = _return.getDeclaredMethods();
for(Method method : methods) {
if ( method.getName().equals("getTree") ) {
Method returnName = _return.getMethod("getTree");
CommonTree tree = (CommonTree) returnName.invoke(ruleReturn);
astString = tree.toStringTree();
}
else if ( method.getName().equals("getTemplate") ) {
Method returnName = _return.getMethod("getTemplate");
StringTemplate st = (StringTemplate) returnName.invoke(ruleReturn);
stString = st.toString();
}
}
}
catch(Exception e) {
System.err.println(e); // Note: If any exception occurs, the test is viewed as failed.
}
}
}
/** Invalid input */
if ( tokens.index()!=tokens.size() ) {
//throw new InvalidInputException();
ps2.print("Invalid input");
}
if ( err.toString().length()>0 ) {
gUnitTestResult testResult = new gUnitTestResult(false, err.toString());
testResult.setError(err.toString());
return testResult;
}
String stdout = null;
// TODO: need to deal with the case which has both ST return value and stdout
if ( out.toString().length()>0 ) {
stdout = out.toString();
}
if ( astString!=null ) { // Return toStringTree of AST
return new gUnitTestResult(true, stdout, astString);
}
else if ( stString!=null ) {// Return toString of ST
return new gUnitTestResult(true, stdout, stString);
}
if ( ruleReturn!=null ) {
// TODO: currently only works for a single return with int or String value
return new gUnitTestResult(true, stdout, String.valueOf(ruleReturn));
}
return new gUnitTestResult(true, stdout, stdout);
} catch (IOException e) {
return getTestExceptionResult(e);
} catch (ClassNotFoundException e) {
e.printStackTrace(); System.exit(1);
} catch (SecurityException e) {
e.printStackTrace(); System.exit(1);
} catch (NoSuchMethodException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalArgumentException e) {
e.printStackTrace(); System.exit(1);
} catch (InstantiationException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalAccessException e) {
e.printStackTrace(); System.exit(1);
} catch (InvocationTargetException e) { // This exception could be caused from ANTLR Runtime Exception, e.g. MismatchedTokenException
return getTestExceptionResult(e);
} finally {
try {
if ( ps!=null ) ps.close();
if ( ps2!=null ) ps2.close();
System.setOut(console); // Reset standard output
System.setErr(consoleErr); // Reset standard err out
} catch (Exception e) {
e.printStackTrace();
}
}
// TODO: verify this:
throw new Exception("This should be unreachable?");
}
protected gUnitTestResult runTreeParser(String parserName, String lexerName, String testRuleName, String testTreeRuleName, gUnitTestInput testInput) throws Exception {
CharStream input;
String treeParserPath;
Class lexer = null;
Class parser = null;
Class treeParser = null;
PrintStream ps = null; // for redirecting stdout later
PrintStream ps2 = null; // for redirecting stderr later
try {
/** Set up ANTLR input stream based on input source, file or String */
input = getANTLRInputStream(testInput);
/** Set up appropriate path for tree parser if using package */
if ( grammarInfo.getHeader()!=null ) {
treeParserPath = grammarInfo.getHeader()+"."+grammarInfo.getTreeGrammarName();
}
else {
treeParserPath = grammarInfo.getTreeGrammarName();
}
/** Use Reflection to create instances of lexer and parser */
lexer = classForName(lexerName);
Class[] lexArgTypes = new Class[]{CharStream.class}; // assign type to lexer's args
Constructor lexConstructor = lexer.getConstructor(lexArgTypes);
Object[] lexArgs = new Object[]{input}; // assign value to lexer's args
Object lexObj = lexConstructor.newInstance(lexArgs); // makes new instance of lexer
CommonTokenStream tokens = new CommonTokenStream((Lexer) lexObj);
parser = classForName(parserName);
Class[] parArgTypes = new Class[]{TokenStream.class}; // assign type to parser's args
Constructor parConstructor = parser.getConstructor(parArgTypes);
Object[] parArgs = new Object[]{tokens}; // assign value to parser's args
Object parObj = parConstructor.newInstance(parArgs); // makes new instance of parser
Method ruleName = parser.getMethod(testRuleName);
/** Start of I/O Redirecting */
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayOutputStream err = new ByteArrayOutputStream();
ps = new PrintStream(out);
ps2 = new PrintStream(err);
System.setOut(ps);
System.setErr(ps2);
/** End of redirecting */
/** Invoke grammar rule, and get the return value */
Object ruleReturn = ruleName.invoke(parObj);
Class _return = classForName(parserName+"$"+testRuleName+"_return");
Method returnName = _return.getMethod("getTree");
CommonTree tree = (CommonTree) returnName.invoke(ruleReturn);
// Walk resulting tree; create tree nodes stream first
CommonTreeNodeStream nodes = new CommonTreeNodeStream(tree);
// AST nodes have payload that point into token stream
nodes.setTokenStream(tokens);
// Create a tree walker attached to the nodes stream
treeParser = classForName(treeParserPath);
Class[] treeParArgTypes = new Class[]{TreeNodeStream.class}; // assign type to tree parser's args
Constructor treeParConstructor = treeParser.getConstructor(treeParArgTypes);
Object[] treeParArgs = new Object[]{nodes}; // assign value to tree parser's args
Object treeParObj = treeParConstructor.newInstance(treeParArgs); // makes new instance of tree parser
// Invoke the tree rule, and store the return value if there is
Method treeRuleName = treeParser.getMethod(testTreeRuleName);
Object treeRuleReturn = treeRuleName.invoke(treeParObj);
String astString = null;
String stString = null;
/** If tree rule has return value, determine if it contains an AST or a ST */
if ( treeRuleReturn!=null ) {
if ( treeRuleReturn.getClass().toString().indexOf(testTreeRuleName+"_return")>0 ) {
try { // NullPointerException may happen here...
Class _treeReturn = classForName(treeParserPath+"$"+testTreeRuleName+"_return");
Method[] methods = _treeReturn.getDeclaredMethods();
for(Method method : methods) {
if ( method.getName().equals("getTree") ) {
Method treeReturnName = _treeReturn.getMethod("getTree");
CommonTree returnTree = (CommonTree) treeReturnName.invoke(treeRuleReturn);
astString = returnTree.toStringTree();
}
else if ( method.getName().equals("getTemplate") ) {
Method treeReturnName = _return.getMethod("getTemplate");
StringTemplate st = (StringTemplate) treeReturnName.invoke(treeRuleReturn);
stString = st.toString();
}
}
}
catch(Exception e) {
System.err.println(e); // Note: If any exception occurs, the test is viewed as failed.
}
}
}
/** Invalid input */
if ( tokens.index()!=tokens.size() ) {
//throw new InvalidInputException();
ps2.print("Invalid input");
}
if ( err.toString().length()>0 ) {
gUnitTestResult testResult = new gUnitTestResult(false, err.toString());
testResult.setError(err.toString());
return testResult;
}
String stdout = null;
// TODO: need to deal with the case which has both ST return value and stdout
if ( out.toString().length()>0 ) {
stdout = out.toString();
}
if ( astString!=null ) { // Return toStringTree of AST
return new gUnitTestResult(true, stdout, astString);
}
else if ( stString!=null ) {// Return toString of ST
return new gUnitTestResult(true, stdout, stString);
}
if ( treeRuleReturn!=null ) {
// TODO: again, currently only works for a single return with int or String value
return new gUnitTestResult(true, stdout, String.valueOf(treeRuleReturn));
}
return new gUnitTestResult(true, stdout, stdout);
} catch (IOException e) {
return getTestExceptionResult(e);
} catch (ClassNotFoundException e) {
e.printStackTrace(); System.exit(1);
} catch (SecurityException e) {
e.printStackTrace(); System.exit(1);
} catch (NoSuchMethodException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalArgumentException e) {
e.printStackTrace(); System.exit(1);
} catch (InstantiationException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalAccessException e) {
e.printStackTrace(); System.exit(1);
} catch (InvocationTargetException e) { // note: This exception could be caused from ANTLR Runtime Exception...
return getTestExceptionResult(e);
} finally {
try {
if ( ps!=null ) ps.close();
if ( ps2!=null ) ps2.close();
System.setOut(console); // Reset standard output
System.setErr(consoleErr); // Reset standard err out
} catch (Exception e) {
e.printStackTrace();
}
}
// TODO: verify this:
throw new Exception("Should not be reachable?");
}
// Create ANTLR input stream based on input source, file or String
private CharStream getANTLRInputStream(gUnitTestInput testInput) throws IOException {
CharStream input;
if ( testInput.inputIsFile ) {
String filePath = testInput.testInput;
File testInputFile = new File(filePath);
// if input test file is not found under the current dir, try to look for it from dir where the testsuite file locates
if ( !testInputFile.exists() ) {
testInputFile = new File(this.testsuiteDir, filePath);
if ( testInputFile.exists() ) filePath = testInputFile.getCanonicalPath();
// if still not found, also try to look for it under the package dir
else if ( grammarInfo.getHeader()!=null ) {
testInputFile = new File("."+File.separator+grammarInfo.getHeader().replace(".", File.separator), filePath);
if ( testInputFile.exists() ) filePath = testInputFile.getCanonicalPath();
}
}
input = new ANTLRFileStream(filePath);
}
else {
input = new ANTLRStringStream(testInput.testInput);
}
return input;
}
// set up the cause of exception or the exception name into a gUnitTestResult instance
private gUnitTestResult getTestExceptionResult(Exception e) {
gUnitTestResult testResult;
if ( e.getCause()!=null ) {
testResult = new gUnitTestResult(false, e.getCause().toString(), true);
testResult.setError(e.getCause().toString());
}
else {
testResult = new gUnitTestResult(false, e.toString(), true);
testResult.setError(e.toString());
}
return testResult;
}
public void onPass(ITestCase passTest) {
}
public void onFail(ITestCase failTest) {
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/gUnitTestInput.java 0000644 0001750 0001750 00000003720 11256465172 025077 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon, Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
/** A class which contains input information of an individual testuite */
public class gUnitTestInput {
protected String testInput; // a test input string for a testsuite
protected boolean inputIsFile; // if true, the testInput represents a filename
protected int line; // number of line in the script
public gUnitTestInput(String testInput, boolean inputIsFile, int line) {
this.testInput = testInput;
this.inputIsFile = inputIsFile;
this.line = line;
}
public int getLine() { return this.line; }
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/OutputTest.java 0000644 0001750 0001750 00000004404 11256465172 024271 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2007 Kenny MacDermid
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
import org.antlr.runtime.Token;
/** OutputTest represents a test for not only standard output string,
* but also AST output which is actually a return value from a parser.
*/
public class OutputTest extends AbstractTest {
private final Token token;
public OutputTest(Token token) {
this.token = token;
}
@Override
public String getText() {
return token.getText();
}
@Override
public int getType() {
return token.getType();
}
@Override
// return ANTLR error msg if test failed
public String getResult(gUnitTestResult testResult) {
// Note: we treat the standard output string as a return value also
if ( testResult.isSuccess() ) return testResult.getReturned();
else {
hasErrorMsg = true;
return testResult.getError();
}
}
@Override
public String getExpected() {
return token.getText();
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/AbstractTest.java 0000644 0001750 0001750 00000006610 11256465172 024535 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2007 Kenny MacDermid
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
public abstract class AbstractTest implements ITestCase {
// store essential individual test result for string template
protected String header;
protected String actual;
protected boolean hasErrorMsg;
private String testedRuleName;
private int testCaseIndex;
// TODO: remove these. They're only used as part of a refactor to keep the
// code cleaner. It is a mock-instanceOf() replacement.
public abstract int getType();
public abstract String getText();
public abstract String getExpected();
// return an escaped string of the expected result
public String getExpectedResult() {
String expected = getExpected();
if ( expected!=null ) expected = JUnitCodeGen.escapeForJava(expected);
return expected;
}
public abstract String getResult(gUnitTestResult testResult);
public String getHeader() { return this.header; }
public String getActual() { return this.actual; }
// return an escaped string of the actual result
public String getActualResult() {
String actual = getActual();
// there is no need to escape the error message from ANTLR
if ( actual!=null && !hasErrorMsg ) actual = JUnitCodeGen.escapeForJava(actual);
return actual;
}
public String getTestedRuleName() { return this.testedRuleName; }
public int getTestCaseIndex() { return this.testCaseIndex; }
public void setHeader(String rule, String lexicalRule, String treeRule, int numOfTest, int line) {
StringBuffer buf = new StringBuffer();
buf.append("test" + numOfTest + " (");
if ( treeRule!=null ) {
buf.append(treeRule+" walks ");
}
if ( lexicalRule!=null ) {
buf.append(lexicalRule + ", line"+line+")" + " - ");
}
else buf.append(rule + ", line"+line+")" + " - ");
this.header = buf.toString();
}
public void setActual(String actual) { this.actual = actual; }
public void setTestedRuleName(String testedRuleName) { this.testedRuleName = testedRuleName; }
public void setTestCaseIndex(int testCaseIndex) { this.testCaseIndex = testCaseIndex; }
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/GrammarInfo.java 0000644 0001750 0001750 00000006363 11256465172 024341 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon, Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class GrammarInfo {
private String grammarName; // targeted grammar for unit test
private String treeGrammarName = null; // optional, required for testing tree grammar
private String header = null; // optional, required if using java package
private String adaptor = null; // optional, required if using customized tree adaptor
private List<gUnitTestSuite> ruleTestSuites = new ArrayList<gUnitTestSuite>(); // testsuites for each testing rule
private StringBuffer unitTestResult = new StringBuffer();
public String getGrammarName() {
return grammarName;
}
public void setGrammarName(String grammarName) {
this.grammarName = grammarName;
}
public String getTreeGrammarName() {
return treeGrammarName;
}
public void setTreeGrammarName(String treeGrammarName) {
this.treeGrammarName = treeGrammarName;
}
public String getHeader() {
return header;
}
public void setHeader(String header) {
this.header = header;
}
public String getAdaptor() {
return adaptor;
}
public void setAdaptor(String adaptor) {
this.adaptor = adaptor;
}
public List<gUnitTestSuite> getRuleTestSuites() {
// Make this list unmodifiable so that we can refactor knowing it's not changed.
return Collections.unmodifiableList(ruleTestSuites);
}
public void addRuleTestSuite(gUnitTestSuite testSuite) {
this.ruleTestSuites.add(testSuite);
}
public void appendUnitTestResult(String result) {
this.unitTestResult.append(result);
}
// We don't want people messing with the string buffer here, so don't return it.
public String getUnitTestResult() {
return unitTestResult.toString();
}
public void setUnitTestResult(StringBuffer unitTestResult) {
this.unitTestResult = unitTestResult;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/gUnitTestResult.java 0000644 0001750 0001750 00000004562 11256465172 025263 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2007 Kenny MacDermid
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
public class gUnitTestResult {
private boolean success;
private String output; // stdout
private String error; // stderr
private String returned; // AST (toStringTree) or ST (toString)
private boolean isLexerTest;
public gUnitTestResult(boolean success, String output) {
this.success = success;
this.output = output;
}
public gUnitTestResult(boolean success, String output, boolean isLexerTest) {
this(success, output);
this.isLexerTest = isLexerTest;
}
public gUnitTestResult(boolean success, String output, String returned) {
this(success, output);
this.returned = returned;
}
public boolean isSuccess() {
return success;
}
public String getOutput() {
return output;
}
public String getError() {
return error;
}
public String getReturned() {
return returned;
}
public boolean isLexerTest() {
return isLexerTest;
}
public void setError(String error) {
this.error = error;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/gUnitTestSuite.java 0000644 0001750 0001750 00000006440 11256465172 025073 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
/** A class which wraps all testsuites for an individual rule */
import java.util.Map;
import java.util.LinkedHashMap;
public class gUnitTestSuite {
protected String rule = null; // paeser rule name for unit testing
protected String lexicalRule = null; // lexical rule name
protected String treeRule = null; // optional, required for testing tree grammar rule
protected boolean isLexicalRule = false;
/** A map which stores input/output pairs (individual testsuites).
* In other words, it maps input data for unit test (gUnitTestInput object)
* to an expected output (Token object).
*/
protected Map<gUnitTestInput, AbstractTest> testSuites = new LinkedHashMap<gUnitTestInput, AbstractTest>();
public gUnitTestSuite() {
}
public gUnitTestSuite(String rule) {
this.rule = rule;
}
public gUnitTestSuite(String treeRule, String rule) {
this.rule = rule;
this.treeRule = treeRule;
}
public void setRuleName(String ruleName) { this.rule = ruleName; }
public void setLexicalRuleName(String lexicalRule) { this.lexicalRule = lexicalRule; this.isLexicalRule = true; }
public void setTreeRuleName(String treeRuleName) { this.treeRule = treeRuleName; }
public String getRuleName() { return this.rule; }
public String getLexicalRuleName() { return this.lexicalRule; }
public String getTreeRuleName() { return this.treeRule; }
public boolean isLexicalRule() { return this.isLexicalRule; }
public void addTestCase(gUnitTestInput input, AbstractTest expect) {
if ( input!=null && expect!=null ) {
/*
* modified by shaoting
* if rule is null, use lexRule name
*/
//expect.setTestedRuleName(this.rule);
expect.setTestedRuleName(this.rule ==null ? this.lexicalRule : this.rule);
expect.setTestCaseIndex(this.testSuites.size());
this.testSuites.put(input, expect);
}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/gUnitBaseTest.java 0000644 0001750 0001750 00000045325 11256465172 024661 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon, Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.PrintStream;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import org.antlr.runtime.ANTLRFileStream;
import org.antlr.runtime.ANTLRStringStream;
import org.antlr.runtime.CharStream;
import org.antlr.runtime.CommonTokenStream;
import org.antlr.runtime.Lexer;
import org.antlr.runtime.TokenStream;
import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.tree.CommonTreeNodeStream;
import org.antlr.runtime.tree.TreeNodeStream;
import org.antlr.stringtemplate.StringTemplate;
import junit.framework.TestCase;
/** All gUnit-generated JUnit class should extend this class
* which implements the essential methods for triggering
* ANTLR parser/tree walker
*/
public abstract class gUnitBaseTest extends TestCase {
public String packagePath;
public String lexerPath;
public String parserPath;
public String treeParserPath;
protected String stdout;
protected String stderr;
private PrintStream console = System.out;
private PrintStream consoleErr = System.err;
// Invoke target lexer.rule
public String execLexer(String testRuleName, String testInput, boolean isFile) throws Exception {
CharStream input;
/** Set up ANTLR input stream based on input source, file or String */
if ( isFile ) {
String filePath = testInput;
File testInputFile = new File(filePath);
// if input test file is not found under the current dir, also try to look for it under the package dir
if ( !testInputFile.exists() && packagePath!=null ) {
testInputFile = new File(packagePath, filePath);
if ( testInputFile.exists() ) filePath = testInputFile.getCanonicalPath();
}
input = new ANTLRFileStream(filePath);
}
else {
input = new ANTLRStringStream(testInput);
}
Class lexer = null;
PrintStream ps = null; // for redirecting stdout later
PrintStream ps2 = null; // for redirecting stderr later
try {
/** Use Reflection to create instances of lexer and parser */
lexer = Class.forName(lexerPath);
Class[] lexArgTypes = new Class[]{CharStream.class}; // assign type to lexer's args
Constructor lexConstructor = lexer.getConstructor(lexArgTypes);
Object[] lexArgs = new Object[]{input}; // assign value to lexer's args
Object lexObj = lexConstructor.newInstance(lexArgs); // makes new instance of lexer
Method ruleName = lexer.getMethod("m"+testRuleName, new Class[0]);
/** Start of I/O Redirecting */
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayOutputStream err = new ByteArrayOutputStream();
ps = new PrintStream(out);
ps2 = new PrintStream(err);
System.setOut(ps);
System.setErr(ps2);
/** End of redirecting */
/** Invoke lexer rule, and get the current index in CharStream */
ruleName.invoke(lexObj, new Object[0]);
Method ruleName2 = lexer.getMethod("getCharIndex", new Class[0]);
int currentIndex = (Integer) ruleName2.invoke(lexObj, new Object[0]);
if ( currentIndex!=input.size() ) {
ps2.println("extra text found, '"+input.substring(currentIndex, input.size()-1)+"'");
}
this.stdout = null;
this.stderr = null;
if ( err.toString().length()>0 ) {
this.stderr = err.toString();
return this.stderr;
}
if ( out.toString().length()>0 ) {
this.stdout = out.toString();
}
if ( err.toString().length()==0 && out.toString().length()==0 ) {
return null;
}
} catch (ClassNotFoundException e) {
e.printStackTrace(); System.exit(1);
} catch (SecurityException e) {
e.printStackTrace(); System.exit(1);
} catch (NoSuchMethodException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalArgumentException e) {
e.printStackTrace(); System.exit(1);
} catch (InstantiationException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalAccessException e) {
e.printStackTrace(); System.exit(1);
} catch (InvocationTargetException e) { // This exception could be caused from ANTLR Runtime Exception, e.g. MismatchedTokenException
if ( e.getCause()!=null ) this.stderr = e.getCause().toString();
else this.stderr = e.toString();
return this.stderr;
} finally {
try {
if ( ps!=null ) ps.close();
if ( ps2!=null ) ps2.close();
System.setOut(console); // Reset standard output
System.setErr(consoleErr); // Reset standard err out
} catch (Exception e) {
e.printStackTrace();
}
}
return this.stdout;
}
// Invoke target parser.rule
public Object execParser(String testRuleName, String testInput, boolean isFile) throws Exception {
CharStream input;
/** Set up ANTLR input stream based on input source, file or String */
if ( isFile ) {
String filePath = testInput;
File testInputFile = new File(filePath);
// if input test file is not found under the current dir, also try to look for it under the package dir
if ( !testInputFile.exists() && packagePath!=null ) {
testInputFile = new File(packagePath, filePath);
if ( testInputFile.exists() ) filePath = testInputFile.getCanonicalPath();
}
input = new ANTLRFileStream(filePath);
}
else {
input = new ANTLRStringStream(testInput);
}
Class lexer = null;
Class parser = null;
PrintStream ps = null; // for redirecting stdout later
PrintStream ps2 = null; // for redirecting stderr later
try {
/** Use Reflection to create instances of lexer and parser */
lexer = Class.forName(lexerPath);
Class[] lexArgTypes = new Class[]{CharStream.class}; // assign type to lexer's args
Constructor lexConstructor = lexer.getConstructor(lexArgTypes);
Object[] lexArgs = new Object[]{input}; // assign value to lexer's args
Object lexObj = lexConstructor.newInstance(lexArgs); // makes new instance of lexer
CommonTokenStream tokens = new CommonTokenStream((Lexer) lexObj);
parser = Class.forName(parserPath);
Class[] parArgTypes = new Class[]{TokenStream.class}; // assign type to parser's args
Constructor parConstructor = parser.getConstructor(parArgTypes);
Object[] parArgs = new Object[]{tokens}; // assign value to parser's args
Object parObj = parConstructor.newInstance(parArgs); // makes new instance of parser
Method ruleName = parser.getMethod(testRuleName);
/** Start of I/O Redirecting */
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayOutputStream err = new ByteArrayOutputStream();
ps = new PrintStream(out);
ps2 = new PrintStream(err);
System.setOut(ps);
System.setErr(ps2);
/** End of redirecting */
/** Invoke grammar rule, and store if there is a return value */
Object ruleReturn = ruleName.invoke(parObj);
String astString = null;
String stString = null;
/** If rule has return value, determine if it contains an AST or a ST */
if ( ruleReturn!=null ) {
if ( ruleReturn.getClass().toString().indexOf(testRuleName+"_return")>0 ) {
try { // NullPointerException may happen here...
Class _return = Class.forName(parserPath+"$"+testRuleName+"_return");
Method[] methods = _return.getDeclaredMethods();
for(Method method : methods) {
if ( method.getName().equals("getTree") ) {
Method returnName = _return.getMethod("getTree");
CommonTree tree = (CommonTree) returnName.invoke(ruleReturn);
astString = tree.toStringTree();
}
else if ( method.getName().equals("getTemplate") ) {
Method returnName = _return.getMethod("getTemplate");
StringTemplate st = (StringTemplate) returnName.invoke(ruleReturn);
stString = st.toString();
}
}
}
catch(Exception e) {
System.err.println(e); // Note: If any exception occurs, the test is viewed as failed.
}
}
}
this.stdout = null;
this.stderr = null;
/** Invalid input */
if ( tokens.index()!=tokens.size() ) {
//throw new InvalidInputException();
this.stderr = "Invalid input";
}
// retVal could be actual return object from rule, stderr or stdout
if ( err.toString().length()>0 ) {
this.stderr = err.toString();
return this.stderr;
}
if ( out.toString().length()>0 ) {
this.stdout = out.toString();
}
if ( astString!=null ) { // Return toStringTree of AST
return astString;
}
else if ( stString!=null ) {// Return toString of ST
return stString;
}
if ( ruleReturn!=null ) {
return ruleReturn;
}
if ( err.toString().length()==0 && out.toString().length()==0 ) {
return null;
}
} catch (ClassNotFoundException e) {
e.printStackTrace(); System.exit(1);
} catch (SecurityException e) {
e.printStackTrace(); System.exit(1);
} catch (NoSuchMethodException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalAccessException e) {
e.printStackTrace(); System.exit(1);
} catch (InvocationTargetException e) {
if ( e.getCause()!=null ) this.stderr = e.getCause().toString();
else this.stderr = e.toString();
return this.stderr;
} finally {
try {
if ( ps!=null ) ps.close();
if ( ps2!=null ) ps2.close();
System.setOut(console); // Reset standard output
System.setErr(consoleErr); // Reset standard err out
} catch (Exception e) {
e.printStackTrace();
}
}
return this.stdout;
}
// Invoke target parser.rule
public Object execTreeParser(String testTreeRuleName, String testRuleName, String testInput, boolean isFile) throws Exception {
CharStream input;
if ( isFile ) {
String filePath = testInput;
File testInputFile = new File(filePath);
// if input test file is not found under the current dir, also try to look for it under the package dir
if ( !testInputFile.exists() && packagePath!=null ) {
testInputFile = new File(packagePath, filePath);
if ( testInputFile.exists() ) filePath = testInputFile.getCanonicalPath();
}
input = new ANTLRFileStream(filePath);
}
else {
input = new ANTLRStringStream(testInput);
}
Class lexer = null;
Class parser = null;
Class treeParser = null;
PrintStream ps = null; // for redirecting stdout later
PrintStream ps2 = null; // for redirecting stderr later
try {
/** Use Reflection to create instances of lexer and parser */
lexer = Class.forName(lexerPath);
Class[] lexArgTypes = new Class[]{CharStream.class}; // assign type to lexer's args
Constructor lexConstructor = lexer.getConstructor(lexArgTypes);
Object[] lexArgs = new Object[]{input}; // assign value to lexer's args
Object lexObj = lexConstructor.newInstance(lexArgs); // makes new instance of lexer
CommonTokenStream tokens = new CommonTokenStream((Lexer) lexObj);
parser = Class.forName(parserPath);
Class[] parArgTypes = new Class[]{TokenStream.class}; // assign type to parser's args
Constructor parConstructor = parser.getConstructor(parArgTypes);
Object[] parArgs = new Object[]{tokens}; // assign value to parser's args
Object parObj = parConstructor.newInstance(parArgs); // makes new instance of parser
Method ruleName = parser.getMethod(testRuleName);
/** Start of I/O Redirecting */
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayOutputStream err = new ByteArrayOutputStream();
ps = new PrintStream(out);
ps2 = new PrintStream(err);
System.setOut(ps);
System.setErr(ps2);
/** End of redirecting */
/** Invoke grammar rule, and get the return value */
Object ruleReturn = ruleName.invoke(parObj);
Class _return = Class.forName(parserPath+"$"+testRuleName+"_return");
Method returnName = _return.getMethod("getTree");
CommonTree tree = (CommonTree) returnName.invoke(ruleReturn);
// Walk resulting tree; create tree nodes stream first
CommonTreeNodeStream nodes = new CommonTreeNodeStream(tree);
// AST nodes have payload that point into token stream
nodes.setTokenStream(tokens);
// Create a tree walker attached to the nodes stream
treeParser = Class.forName(treeParserPath);
Class[] treeParArgTypes = new Class[]{TreeNodeStream.class}; // assign type to tree parser's args
Constructor treeParConstructor = treeParser.getConstructor(treeParArgTypes);
Object[] treeParArgs = new Object[]{nodes}; // assign value to tree parser's args
Object treeParObj = treeParConstructor.newInstance(treeParArgs); // makes new instance of tree parser
// Invoke the tree rule, and store the return value if there is
Method treeRuleName = treeParser.getMethod(testTreeRuleName);
Object treeRuleReturn = treeRuleName.invoke(treeParObj);
String astString = null;
String stString = null;
/** If tree rule has return value, determine if it contains an AST or a ST */
if ( treeRuleReturn!=null ) {
if ( treeRuleReturn.getClass().toString().indexOf(testTreeRuleName+"_return")>0 ) {
try { // NullPointerException may happen here...
Class _treeReturn = Class.forName(treeParserPath+"$"+testTreeRuleName+"_return");
Method[] methods = _treeReturn.getDeclaredMethods();
for(Method method : methods) {
if ( method.getName().equals("getTree") ) {
Method treeReturnName = _treeReturn.getMethod("getTree");
CommonTree returnTree = (CommonTree) treeReturnName.invoke(treeRuleReturn);
astString = returnTree.toStringTree();
}
else if ( method.getName().equals("getTemplate") ) {
Method treeReturnName = _return.getMethod("getTemplate");
StringTemplate st = (StringTemplate) treeReturnName.invoke(treeRuleReturn);
stString = st.toString();
}
}
}
catch(Exception e) {
System.err.println(e); // Note: If any exception occurs, the test is viewed as failed.
}
}
}
this.stdout = null;
this.stderr = null;
/** Invalid input */
if ( tokens.index()!=tokens.size() ) {
throw new InvalidInputException();
}
// retVal could be actual return object from rule, stderr or stdout
if ( err.toString().length()>0 ) {
this.stderr = err.toString();
return this.stderr;
}
if ( out.toString().length()>0 ) {
this.stdout = out.toString();
}
if ( astString!=null ) { // Return toStringTree of AST
return astString;
}
else if ( stString!=null ) {// Return toString of ST
return stString;
}
if ( treeRuleReturn!=null ) {
return treeRuleReturn;
}
if ( err.toString().length()==0 && out.toString().length()==0 ) {
return null;
}
} catch (ClassNotFoundException e) {
e.printStackTrace(); System.exit(1);
} catch (SecurityException e) {
e.printStackTrace(); System.exit(1);
} catch (NoSuchMethodException e) {
e.printStackTrace(); System.exit(1);
} catch (IllegalAccessException e) {
e.printStackTrace(); System.exit(1);
} catch (InvocationTargetException e) {
if ( e.getCause()!=null ) this.stderr = e.getCause().toString();
else this.stderr = e.toString();
return this.stderr;
} finally {
try {
if ( ps!=null ) ps.close();
if ( ps2!=null ) ps2.close();
System.setOut(console); // Reset standard output
System.setErr(consoleErr); // Reset standard err out
} catch (Exception e) {
e.printStackTrace();
}
}
return stdout;
}
// Modify the return value if the expected token type is OK or FAIL
public Object examineExecResult(int tokenType, Object retVal) {
if ( tokenType==gUnitParser.OK ) { // expected Token: OK
if ( this.stderr==null ) {
return "OK";
}
else {
return "FAIL, "+this.stderr;
}
}
else if ( tokenType==gUnitParser.FAIL ) { // expected Token: FAIL
if ( this.stderr!=null ) {
return "FAIL";
}
else {
return "OK";
}
}
else { // return the same object for the other token types
return retVal;
}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/ReturnTest.java 0000644 0001750 0001750 00000004324 11256465172 024251 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2007 Kenny MacDermid
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
import org.antlr.runtime.Token;
public class ReturnTest extends AbstractTest {
private final Token retval;
public ReturnTest(Token retval) {
this.retval = retval;
}
@Override
public String getText() {
return retval.getText();
}
@Override
public int getType() {
return retval.getType();
}
@Override
// return ANTLR error msg if test failed
public String getResult(gUnitTestResult testResult) {
if ( testResult.isSuccess() ) return testResult.getReturned();
else {
hasErrorMsg = true;
return testResult.getError();
}
}
@Override
public String getExpected() {
String expect = retval.getText();
if ( expect.charAt(0)=='"' && expect.charAt(expect.length()-1)=='"' ) {
expect = expect.substring(1, expect.length()-1);
}
return expect;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/ 0000755 0001750 0001750 00000000000 11410174107 022734 5 ustar twerner twerner antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/RunnerController.java 0000644 0001750 0001750 00000016337 11256465172 027143 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Component;
import java.awt.Dimension;
import javax.swing.BorderFactory;
import javax.swing.BoxLayout;
import javax.swing.JLabel;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.JTree;
import javax.swing.event.TreeModelListener;
import javax.swing.tree.DefaultMutableTreeNode;
import javax.swing.tree.DefaultTreeModel;
import javax.swing.tree.TreeCellRenderer;
import javax.swing.tree.TreeModel;
import javax.swing.tree.TreePath;
import org.antlr.gunit.swingui.ImageFactory;
import org.antlr.gunit.swingui.model.*;
/**
*
* @author scai
*/
public class RunnerController implements IController {
/* MODEL */
//private TestSuite testSuite;
/* VIEW */
private RunnerView view = new RunnerView();
public class RunnerView extends JPanel {
private JTextArea textArea = new JTextArea();
private JTree tree = new JTree();
private JScrollPane scroll = new JScrollPane(tree,
JScrollPane.VERTICAL_SCROLLBAR_ALWAYS,
JScrollPane.HORIZONTAL_SCROLLBAR_AS_NEEDED);
public void initComponents() {
//textArea.setOpaque(false);
tree.setOpaque(false);
scroll.setBorder(BorderFactory.createLineBorder(Color.LIGHT_GRAY));
scroll.setOpaque(false);
this.setLayout(new BoxLayout(this, BoxLayout.Y_AXIS));
this.add(scroll);
this.setBorder(BorderFactory.createEmptyBorder());
this.setOpaque(false);
}
};
public RunnerController() {
}
public Object getModel() {
return null;
}
public Component getView() {
return view;
}
public void update() {
view.initComponents();
}
public void OnShowSuiteResult(TestSuite suite) {
update();
view.tree.setModel(new RunnerTreeModel(suite));
view.tree.setCellRenderer(new RunnerTreeRenderer());
}
public void OnShowRuleResult(Rule rule) {
update();
/*
StringBuffer result = new StringBuffer();
result.append("Testing result for rule: " + rule.getName());
result.append("\n--------------------\n\n");
for(TestCase testCase: rule.getTestCases()){
result.append(testCase.isPass() ? "PASS" : "FAIL");
result.append("\n");
}
result.append("\n--------------------\n");
view.textArea.setText(result.toString());
*/
}
private class TestSuiteTreeNode extends DefaultMutableTreeNode {
private TestSuite data ;
public TestSuiteTreeNode(TestSuite suite) {
super(suite.getGrammarName());
for(int i=0; i<suite.getRuleCount(); ++i) {
final Rule rule = suite.getRule(i);
if(rule.getNotEmpty()) this.add(new TestGroupTreeNode(rule));
}
data = suite;
}
@Override
public String toString() {
return String.format("%s (%d test groups)",
data.getGrammarName(),
this.getChildCount());
}
} ;
private class TestGroupTreeNode extends DefaultMutableTreeNode {
private Rule data;
private boolean hasFail = false;
private TestGroupTreeNode(Rule rule) {
super(rule.getName());
for(TestCase tc: rule.getTestCases()) {
this.add(new TestCaseTreeNode(tc));
}
data = rule;
}
@Override
public String toString() {
int iPass = 0;
int iFail = 0;
for(TestCase tc: data.getTestCases()) {
if(tc.isPass())
++iPass;
else
++iFail;
}
hasFail = iFail > 0;
return String.format("%s (pass %d, fail %d)",
data.getName(), iPass, iFail);
}
} ;
private class TestCaseTreeNode extends DefaultMutableTreeNode {
private TestCase data;
private TestCaseTreeNode(TestCase tc) {
super(tc.toString());
data = tc;
}
} ;
private class RunnerTreeModel extends DefaultTreeModel {
public RunnerTreeModel(TestSuite testSuite) {
super(new TestSuiteTreeNode(testSuite));
}
}
private class RunnerTreeRenderer implements TreeCellRenderer {
public Component getTreeCellRendererComponent(JTree tree, Object value,
boolean selected, boolean expanded, boolean leaf, int row,
boolean hasFocus) {
JLabel label = new JLabel();
if(value instanceof TestSuiteTreeNode) {
label.setText(value.toString());
label.setIcon(ImageFactory.getSingleton().TESTSUITE);
} else if(value instanceof TestGroupTreeNode) {
TestGroupTreeNode node = (TestGroupTreeNode) value;
label.setText(value.toString());
label.setIcon( node.hasFail ?
ImageFactory.getSingleton().TESTGROUPX :
ImageFactory.getSingleton().TESTGROUP);
} else if(value instanceof TestCaseTreeNode) {
TestCaseTreeNode node = (TestCaseTreeNode) value;
label.setIcon( (node.data.isPass())?
ImageFactory.getSingleton().RUN_PASS :
ImageFactory.getSingleton().RUN_FAIL);
label.setText(value.toString());
} else {
throw new IllegalArgumentException(
"Invalide tree node type + " + value.getClass().getName());
}
return label;
}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/runner/ 0000755 0001750 0001750 00000000000 11410174107 024245 5 ustar twerner twerner antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/runner/NotifiedTestExecuter.java 0000644 0001750 0001750 00000006227 11256465172 031242 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.runner;
import org.antlr.gunit.*;
import org.antlr.gunit.swingui.model.*;
/**
* The gUnit test executer that will respond to the fail/pass event during the
* execution. The executer is passed into gUnit Interp for execution.
* @author scai
*/
public class NotifiedTestExecuter extends gUnitExecutor {
private TestSuite testSuite ;
public NotifiedTestExecuter(GrammarInfo grammarInfo, ClassLoader loader, String testsuiteDir, TestSuite suite) {
super(grammarInfo, loader, testsuiteDir);
testSuite = suite;
}
@Override
public void onFail(ITestCase failTest) {
if(failTest == null) throw new IllegalArgumentException("Null fail test");
final String ruleName = failTest.getTestedRuleName();
if(ruleName == null) throw new NullPointerException("Null rule name");
final Rule rule = testSuite.getRule(ruleName);
final TestCase failCase = (TestCase) rule.getElementAt(failTest.getTestCaseIndex());
failCase.setPass(false);
//System.out.println(String.format("[FAIL] %s (%d) ", failTest.getTestedRuleName(), failTest.getTestCaseIndex()));
}
@Override
public void onPass(ITestCase passTest) {
if(passTest == null) throw new IllegalArgumentException("Null pass test");
final String ruleName = passTest.getTestedRuleName();
if(ruleName == null) throw new NullPointerException("Null rule name");
final Rule rule = testSuite.getRule(ruleName);
final TestCase passCase = (TestCase) rule.getElementAt(passTest.getTestCaseIndex());
passCase.setPass(true);
//System.out.println(String.format("[PASS] %s (%d) ", passTest.getTestedRuleName(), passTest.getTestCaseIndex()));
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/runner/TestSuiteAdapter.java 0000644 0001750 0001750 00000007371 11256465172 030367 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.runner;
import org.antlr.gunit.swingui.model.*;
/**
* Adapter class for gunit parser to save information into testsuite object.
* @author Shaoting
*/
public class TestSuiteAdapter {
private TestSuite model ;
private Rule currentRule;
public TestSuiteAdapter(TestSuite testSuite) {
model = testSuite;
}
public void setGrammarName(String name) {
model.setGrammarName(name);
}
public void startRule(String name) {
currentRule = new Rule(name);
}
public void endRule() {
model.addRule(currentRule);
currentRule = null;
}
public void addTestCase(ITestCaseInput in, ITestCaseOutput out) {
TestCase testCase = new TestCase(in, out);
currentRule.addTestCase(testCase);
}
private static String trimChars(String text, int numOfChars) {
return text.substring(numOfChars, text.length() - numOfChars);
}
public static ITestCaseInput createFileInput(String fileName) {
if(fileName == null) throw new IllegalArgumentException("null");
return new TestCaseInputFile(fileName);
}
public static ITestCaseInput createStringInput(String line) {
if(line == null) throw new IllegalArgumentException("null");
// trim double quotes
return new TestCaseInputString(trimChars(line, 1));
}
public static ITestCaseInput createMultiInput(String text) {
if(text == null) throw new IllegalArgumentException("null");
// trim << and >>
return new TestCaseInputMultiString(trimChars(text, 2));
}
public static ITestCaseOutput createBoolOutput(boolean bool) {
return new TestCaseOutputResult(bool);
}
public static ITestCaseOutput createAstOutput(String ast) {
if(ast == null) throw new IllegalArgumentException("null");
return new TestCaseOutputAST(ast);
}
public static ITestCaseOutput createStdOutput(String text) {
if(text == null) throw new IllegalArgumentException("null");
// trim double quotes
return new TestCaseOutputStdOut(trimChars(text, 1));
}
public static ITestCaseOutput createReturnOutput(String text) {
if(text == null) throw new IllegalArgumentException("null");
// trim square brackets
return new TestCaseOutputReturn(trimChars(text, 1));
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/runner/gUnitAdapter.java 0000644 0001750 0001750 00000006115 11256465172 027517 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.runner;
import java.io.File;
import java.io.IOException;
import org.antlr.runtime.*;
import org.antlr.runtime.CharStream;
import org.antlr.gunit.*;
import org.antlr.gunit.swingui.model.TestSuite;
/**
* Adapter between gUnitEditor Swing GUI and gUnit command-line tool.
* @author scai
*/
public class gUnitAdapter {
private ParserLoader loader ;
private TestSuite testSuite;
public gUnitAdapter(TestSuite suite) throws IOException, ClassNotFoundException {
int i = 3;
loader = new ParserLoader(suite.getGrammarName(),
suite.getTestSuiteFile().getParent());
testSuite = suite;
}
public void run() {
if (testSuite == null)
throw new IllegalArgumentException("Null testsuite.");
try {
// Parse gUnit test suite file
final CharStream input = new ANTLRFileStream(testSuite.getTestSuiteFile().getCanonicalPath());
final gUnitLexer lexer = new gUnitLexer(input);
final CommonTokenStream tokens = new CommonTokenStream(lexer);
final GrammarInfo grammarInfo = new GrammarInfo();
final gUnitParser parser = new gUnitParser(tokens, grammarInfo);
parser.gUnitDef(); // parse gunit script and save elements to grammarInfo
// Execute test suite
final gUnitExecutor executer = new NotifiedTestExecuter(
grammarInfo, loader,
testSuite.getTestSuiteFile().getParent(), testSuite);
executer.execTest();
} catch (Exception e) {
e.printStackTrace();
}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/runner/ParserLoader.java 0000644 0001750 0001750 00000010632 11256465172 027512 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.runner;
import java.io.*;
import java.util.HashMap;
/**
* Class loader for parser & lexer generated by antlr.
* @author Shaoting
*/
public class ParserLoader extends ClassLoader {
private HashMap<String, Class> classList;
private String grammar;
/**
* Create a class loader for antlr parser/lexer.
* @param grammarName
* @param classDir
*/
public ParserLoader(String grammarName, String classDir) throws IOException, ClassNotFoundException {
final String lexerName = grammarName + "Lexer";
// load all the class files in the "classDir" related to the grammarName
File dir = new File(classDir);
if(dir.isDirectory()) {
classList = new HashMap<String, Class>();
grammar = grammarName;
File[] files = dir.listFiles(new ClassFilenameFilter(grammarName));
for(File f : files) {
// load class data
final InputStream in = new BufferedInputStream(new FileInputStream(f));
final byte[] classData = new byte[in.available()];
in.read(classData);
in.close();
// define class
final Class newClass = defineClass(null, classData, 0, classData.length);
assert(newClass != null);
resolveClass(newClass);
// save to hashtable
final String fileName = f.getName();
final String className = fileName.substring(0, fileName.lastIndexOf("."));
classList.put(className, newClass);
//System.out.println("adding: " + className);
}
} else {
throw new IOException(classDir + " is not a directory.");
}
if(classList.isEmpty() || !classList.containsKey(lexerName)) {
throw new ClassNotFoundException(lexerName + " not found.");
}
}
@Override
public synchronized Class loadClass(String name, boolean resolve) throws ClassNotFoundException {
//System.out.print("loading: " + name);
if(name.startsWith(grammar)) {
if(classList.containsKey(name)) {
//System.out.println(" .... found");
return classList.get(name);
} else {
//System.out.println(" .... not found");
throw new ClassNotFoundException(name);
}
} else {
final Class c = findSystemClass(name);
//System.out.println(" .... system found " + c.getName());
return c;
}
}
/**
* Accepts grammarname...($...)?.class
*/
protected static class ClassFilenameFilter implements FilenameFilter {
private String grammarName;
protected ClassFilenameFilter(String name) {
grammarName = name;
}
public boolean accept(File dir, String name) {
return name.startsWith(grammarName) && name.endsWith(".class");
}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/RuleListController.java 0000644 0001750 0001750 00000012175 11256465172 027431 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import javax.swing.event.ListDataListener;
import org.antlr.gunit.swingui.model.Rule;
import org.antlr.gunit.swingui.ImageFactory;
import java.awt.Color;
import java.awt.Component;
import java.awt.Dimension;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import java.util.List;
import javax.swing.BorderFactory;
import javax.swing.DefaultListModel;
import javax.swing.JLabel;
import javax.swing.JList;
import javax.swing.JScrollPane;
import javax.swing.ListCellRenderer;
import javax.swing.ListModel;
import javax.swing.ListSelectionModel;
import javax.swing.event.ListSelectionListener;
import org.antlr.gunit.swingui.model.TestSuite;
public class RuleListController implements IController {
/* Sub-controls */
private final JList list = new JList();
private final JScrollPane scroll = new JScrollPane( list,
JScrollPane.VERTICAL_SCROLLBAR_ALWAYS,
JScrollPane.HORIZONTAL_SCROLLBAR_AS_NEEDED);
/* Model */
private ListModel model = null;
private TestSuite testSuite = null;
public RuleListController() {
this.initComponents();
}
public JScrollPane getView() {
return scroll;
}
private void setTestSuite(TestSuite newTestSuite) {
testSuite = newTestSuite;
model = new RuleListModel();
list.setModel(model);
}
public void initialize(TestSuite ts) {
setTestSuite(ts);
if(model.getSize() > 0) list.setSelectedIndex(0);
list.updateUI();
}
/**
* Initialize view.
*/
private void initComponents() {
scroll.setViewportBorder(BorderFactory.createEtchedBorder());
scroll.setBorder(BorderFactory.createTitledBorder(
BorderFactory.createEmptyBorder(), "Rules"));
scroll.setOpaque(false);
list.setOpaque(false);
list.setSelectionMode(ListSelectionModel.SINGLE_INTERVAL_SELECTION);
list.setLayoutOrientation(JList.VERTICAL);
list.setCellRenderer(new RuleListItemRenderer());
}
public void setListSelectionListener(ListSelectionListener l) {
this.list.addListSelectionListener(l);
}
public Object getModel() {
return model;
}
/* ITEM RENDERER */
private class RuleListItemRenderer extends JLabel implements ListCellRenderer{
public RuleListItemRenderer() {
this.setPreferredSize(new Dimension(50, 18));
}
public Component getListCellRendererComponent(
JList list, Object value, int index,
boolean isSelected, boolean hasFocus) {
if(value instanceof Rule) {
final Rule item = (Rule) value;
setText(item.toString());
setForeground(list.getForeground());
setIcon(item.getNotEmpty() ? ImageFactory.getSingleton().FAV16 : null);
if(list.getSelectedValue() == item ) {
setBackground(Color.LIGHT_GRAY);
setOpaque(true);
} else {
setOpaque(false);
}
} else {
this.setText("Error!");
}
return this;
}
}
private class RuleListModel implements ListModel {
public RuleListModel() {
if(testSuite == null)
throw new NullPointerException("Null test suite");
}
public int getSize() {
return testSuite.getRuleCount();
}
public Object getElementAt(int index) {
return testSuite.getRule(index);
}
public void addListDataListener(ListDataListener l) {}
public void removeListDataListener(ListDataListener l) {}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/StatusBarController.java 0000644 0001750 0001750 00000006173 11256465172 027577 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import java.awt.Dimension;
import java.awt.FlowLayout;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JProgressBar;
public class StatusBarController implements IController {
private final JPanel panel = new JPanel();
private final JLabel labelText = new JLabel("Ready");
private final JLabel labelRuleName = new JLabel("");
private final JProgressBar progress = new JProgressBar();
public StatusBarController() {
initComponents();
}
private void initComponents() {
labelText.setPreferredSize(new Dimension(300, 20));
labelText.setHorizontalTextPosition(JLabel.LEFT);
progress.setPreferredSize(new Dimension(100, 15));
final JLabel labRuleHint = new JLabel("Rule: ");
FlowLayout layout = new FlowLayout();
layout.setAlignment(FlowLayout.LEFT);
panel.setLayout(layout);
panel.add(labelText);
panel.add(progress);
panel.add(labRuleHint);
panel.add(labelRuleName);
panel.setOpaque(false);
panel.setBorder(javax.swing.BorderFactory.createEmptyBorder());
}
public void setText(String text) {
labelText.setText(text);
}
public void setRule(String name) {
this.labelRuleName.setText(name);
}
public Object getModel() {
throw new UnsupportedOperationException("Not supported yet.");
}
public JPanel getView() {
return panel;
}
public void setProgressIndetermined(boolean value) {
this.progress.setIndeterminate(value);
}
public void setProgress(int value) {
this.progress.setIndeterminate(false);
this.progress.setValue(value);
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/IController.java 0000644 0001750 0001750 00000003077 11256465172 026057 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import java.awt.Component;
public interface IController {
public Object getModel() ;
public Component getView() ;
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/TestCaseEditController.java 0000644 0001750 0001750 00000052233 11256465172 030206 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import org.antlr.gunit.swingui.model.*;
import org.antlr.gunit.swingui.ImageFactory;
import java.awt.*;
import java.awt.event.*;
import java.util.HashMap;
import javax.swing.*;
import javax.swing.event.*;
/**
*
* @author scai
*/
public class TestCaseEditController implements IController {
private JPanel view = new JPanel();
private JScrollPane scroll;
private JPanel paneDetail;
private AbstractEditorPane paneDetailInput, paneDetailOutput;
private JToolBar toolbar;
private JList listCases;
private ListModel listModel ;
public ActionListener onTestCaseNumberChange;
/* EDITORS */
private InputFileEditor editInputFile;
private InputStringEditor editInputString;
private InputMultiEditor editInputMulti;
private OutputResultEditor editOutputResult;
private OutputAstEditor editOutputAST;
private OutputStdEditor editOutputStd;
private OutputReturnEditor editOutputReturn;
private JComboBox comboInputType, comboOutputType;
/* TYPE NAME */
private static final String IN_TYPE_STRING = "Single-line Text";
private static final String IN_TYPE_MULTI = "Multi-line Text";
private static final String IN_TYPE_FILE = "Disk File";
private static final String OUT_TYPE_BOOL = "OK or Fail";
private static final String OUT_TYPE_AST = "AST";
private static final String OUT_TYPE_STD = "Standard Output";
private static final String OUT_TYPE_RET = "Return Value";
private static final String DEFAULT_IN_SCRIPT = "";
private static final String DEFAULT_OUT_SCRIPT = "";
private static final Object[] INPUT_TYPE = {
IN_TYPE_STRING, IN_TYPE_MULTI, IN_TYPE_FILE
};
private static final Object[] OUTPUT_TYPE = {
OUT_TYPE_BOOL, OUT_TYPE_AST, OUT_TYPE_STD, OUT_TYPE_RET
};
/* SIZE */
private static final int TEST_CASE_DETAIL_WIDTH = 300;
private static final int TEST_EDITOR_WIDTH = 280;
private static final int TEST_CASE_DETAIL_HEIGHT = 250;
private static final int TEST_EDITOR_HEIGHT = 120;
/* MODEL */
private Rule currentRule = null;
private TestCase currentTestCase = null;
/* END OF MODEL*/
private static final HashMap<Class, String> TypeNameTable;
static {
TypeNameTable = new HashMap<Class, String> ();
TypeNameTable.put(TestCaseInputString.class, IN_TYPE_STRING);
TypeNameTable.put(TestCaseInputMultiString.class, IN_TYPE_MULTI);
TypeNameTable.put(TestCaseInputFile.class, IN_TYPE_FILE);
TypeNameTable.put(TestCaseOutputResult.class, OUT_TYPE_BOOL);
TypeNameTable.put(TestCaseOutputAST.class, OUT_TYPE_AST);
TypeNameTable.put(TestCaseOutputStdOut.class, OUT_TYPE_STD);
TypeNameTable.put(TestCaseOutputReturn.class, OUT_TYPE_RET);
}
//private WorkSpaceView owner;
public TestCaseEditController(WorkSpaceView workspace) {
//this.owner = workspace;
initComponents();
}
public TestCaseEditController() {
initComponents();
}
public void OnLoadRule(Rule rule) {
if(rule == null) throw new IllegalArgumentException("Null");
this.currentRule = rule;
this.currentTestCase = null;
this.listModel = rule;
this.listCases.setModel(this.listModel);
}
public void setCurrentTestCase(TestCase testCase) {
if(testCase == null) throw new IllegalArgumentException("Null");
this.listCases.setSelectedValue(testCase, true);
this.currentTestCase = testCase;
}
public Rule getCurrentRule() {
return this.currentRule;
}
private void initComponents() {
/* CASE LIST */
listCases = new JList();
listCases.addListSelectionListener(new TestCaseListSelectionListener());
listCases.setCellRenderer(listRenderer);
listCases.setOpaque(false);
scroll = new JScrollPane(listCases);
scroll.setBorder(BorderFactory.createTitledBorder(
BorderFactory.createEmptyBorder(), "Test Cases"));
scroll.setOpaque(false);
scroll.setViewportBorder(BorderFactory.createEtchedBorder());
/* CASE DETAIL */
editInputString = new InputStringEditor();
editInputMulti = new InputMultiEditor();
editInputFile = new InputFileEditor();
editOutputResult = new OutputResultEditor();
editOutputAST = new OutputAstEditor();
editOutputStd = new OutputStdEditor();
editOutputReturn = new OutputReturnEditor();
paneDetail = new JPanel();
paneDetail.setBorder(BorderFactory.createEmptyBorder());
paneDetail.setOpaque(false);
comboInputType = new JComboBox(INPUT_TYPE);
comboInputType.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
OnInputTestCaseTypeChanged(comboInputType.getSelectedItem());
}
});
comboOutputType = new JComboBox(OUTPUT_TYPE);
comboOutputType.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
OnOutputTestCaseTypeChanged(comboOutputType.getSelectedItem());
}
});
paneDetailInput = new InputEditorPane(comboInputType);
paneDetailOutput = new OutputEditorPane(comboOutputType);
BoxLayout layout = new BoxLayout(paneDetail, BoxLayout.PAGE_AXIS);
paneDetail.setLayout(layout);
paneDetail.add(this.paneDetailInput);
paneDetail.add(this.paneDetailOutput);
/* TOOLBAR */
toolbar = new JToolBar("Edit TestCases", JToolBar.VERTICAL);
toolbar.setFloatable(false);
toolbar.add(new AddTestCaseAction());
toolbar.add(new RemoveTestCaseAction());
/* COMPOSITE */
view.setLayout(new BorderLayout());
view.setBorder(BorderFactory.createEmptyBorder());
view.setOpaque(false);
view.add(toolbar, BorderLayout.WEST);
view.add(scroll, BorderLayout.CENTER);
view.add(paneDetail, BorderLayout.EAST);
}
private void updateInputEditor() {
JComponent editor = null;
if(currentTestCase != null ) {
ITestCaseInput input = this.currentTestCase.getInput();
if(input instanceof TestCaseInputString) {
this.editInputString.setText(input.getScript());
editor = this.editInputString;
comboInputType.setSelectedItem(IN_TYPE_STRING);
} else if(input instanceof TestCaseInputMultiString) {
this.editInputMulti.setText(input.getScript());
editor = this.editInputMulti.getView();
comboInputType.setSelectedItem(IN_TYPE_MULTI);
} else if(input instanceof TestCaseInputFile) {
this.editInputFile.setText(input.getScript());
editor = this.editInputFile;
comboInputType.setSelectedItem(IN_TYPE_FILE);
} else {
throw new Error("Wrong type");
}
}
paneDetailInput.setEditor(editor);
}
private void updateOutputEditor() {
JComponent editor = null;
if(currentTestCase != null) {
ITestCaseOutput output = this.currentTestCase.getOutput();
if(output instanceof TestCaseOutputAST) {
this.editOutputAST.setText(output.getScript());
editor = this.editOutputAST.getView();
comboOutputType.setSelectedItem(OUT_TYPE_AST);
} else if(output instanceof TestCaseOutputResult) {
this.editOutputResult.setValue(output.getScript());
editor = this.editOutputResult;
comboOutputType.setSelectedItem(OUT_TYPE_BOOL);
} else if(output instanceof TestCaseOutputStdOut) {
this.editOutputStd.setText(output.getScript());
editor = this.editOutputStd.getView();
comboOutputType.setSelectedItem(OUT_TYPE_STD);
} else if(output instanceof TestCaseOutputReturn) {
this.editOutputReturn.setText(output.getScript());
editor = this.editOutputReturn.getView();
comboOutputType.setSelectedItem(OUT_TYPE_RET);
} else {
throw new Error("Wrong type");
}
}
this.paneDetailOutput.setEditor(editor);
}
private void OnInputTestCaseTypeChanged(Object inputTypeStr) {
if(this.currentTestCase != null) {
ITestCaseInput input ;
if(inputTypeStr == IN_TYPE_STRING) {
input = new TestCaseInputString(DEFAULT_IN_SCRIPT);
} else if(inputTypeStr == IN_TYPE_MULTI) {
input = new TestCaseInputMultiString(DEFAULT_IN_SCRIPT);
} else if(inputTypeStr == IN_TYPE_FILE) {
input = new TestCaseInputFile(DEFAULT_IN_SCRIPT);
} else {
throw new Error("Wrong Type");
}
if(input.getClass().equals(this.currentTestCase.getInput().getClass()))
return ;
this.currentTestCase.setInput(input);
}
this.updateInputEditor();
}
private void OnOutputTestCaseTypeChanged(Object outputTypeStr) {
if(this.currentTestCase != null) {
ITestCaseOutput output ;
if(outputTypeStr == OUT_TYPE_AST) {
output = new TestCaseOutputAST(DEFAULT_OUT_SCRIPT);
} else if(outputTypeStr == OUT_TYPE_BOOL) {
output = new TestCaseOutputResult(false);
} else if(outputTypeStr == OUT_TYPE_STD) {
output = new TestCaseOutputStdOut(DEFAULT_OUT_SCRIPT);
} else if(outputTypeStr == OUT_TYPE_RET) {
output = new TestCaseOutputReturn(DEFAULT_OUT_SCRIPT);
} else {
throw new Error("Wrong Type");
}
if(output.getClass().equals(this.currentTestCase.getOutput().getClass()))
return ;
this.currentTestCase.setOutput(output);
}
this.updateOutputEditor();
}
private void OnTestCaseSelected(TestCase testCase) {
//if(testCase == null) throw new RuntimeException("Null TestCase");
this.currentTestCase = testCase;
updateInputEditor();
updateOutputEditor();
}
private void OnAddTestCase() {
if(currentRule == null) return;
final TestCase newCase = new TestCase(
new TestCaseInputString(""),
new TestCaseOutputResult(true));
this.currentRule.addTestCase(newCase);
setCurrentTestCase(newCase);
this.listCases.setSelectedValue(newCase, true);
this.listCases.updateUI();
this.OnTestCaseSelected(newCase);
this.onTestCaseNumberChange.actionPerformed(null);
}
private void OnRemoveTestCase() {
if(currentTestCase == null) return;
currentRule.removeElement(currentTestCase);
listCases.updateUI();
final TestCase nextActiveCase = listCases.isSelectionEmpty() ?
null : (TestCase) listCases.getSelectedValue() ;
OnTestCaseSelected(nextActiveCase);
this.onTestCaseNumberChange.actionPerformed(null);
}
public Object getModel() {
return currentRule;
}
public Component getView() {
return view;
}
/* EDITOR CONTAINER */
abstract public class AbstractEditorPane extends JPanel {
private JComboBox combo;
private JComponent editor;
private String title;
private JLabel placeHolder = new JLabel();
public AbstractEditorPane(JComboBox comboBox, String title) {
this.combo = comboBox;
this.editor = placeHolder;
this.title = title;
this.initComponents();
}
private void initComponents() {
placeHolder.setPreferredSize(new Dimension(
TEST_CASE_DETAIL_WIDTH, TEST_CASE_DETAIL_HEIGHT));
this.setLayout(new BoxLayout(this, BoxLayout.Y_AXIS));
this.add(combo, BorderLayout.NORTH);
this.add(editor, BorderLayout.CENTER);
this.setOpaque(false);
this.setBorder(BorderFactory.createTitledBorder(title));
this.setPreferredSize(new Dimension(
TEST_CASE_DETAIL_WIDTH, TEST_CASE_DETAIL_HEIGHT));
}
public void setEditor(JComponent newEditor) {
if(newEditor == null) newEditor = placeHolder;
this.remove(editor);
this.add(newEditor);
this.editor = newEditor;
this.updateUI();
}
}
public class InputEditorPane extends AbstractEditorPane {
public InputEditorPane(JComboBox comboBox) {
super(comboBox, "Input");
}
}
public class OutputEditorPane extends AbstractEditorPane {
public OutputEditorPane(JComboBox comboBox) {
super(comboBox, "Output");
}
}
/* INPUT EDITORS */
public class InputStringEditor extends JTextField implements CaretListener {
public InputStringEditor() {
super();
this.setBorder(BorderFactory.createLineBorder(Color.LIGHT_GRAY));
this.addCaretListener(this);
}
public void caretUpdate(CaretEvent arg0) {
currentTestCase.getInput().setScript(getText());
listCases.updateUI();
}
}
public class InputMultiEditor implements CaretListener {
private JTextArea textArea = new JTextArea(20, 30);
private JScrollPane scroll = new JScrollPane(textArea,
JScrollPane.VERTICAL_SCROLLBAR_ALWAYS,
JScrollPane.HORIZONTAL_SCROLLBAR_ALWAYS);
public InputMultiEditor() {
super();
scroll.setBorder(BorderFactory.createLineBorder(Color.LIGHT_GRAY));
textArea.addCaretListener(this);
}
public void caretUpdate(CaretEvent arg0) {
currentTestCase.getInput().setScript(getText());
listCases.updateUI();
}
public String getText() {
return textArea.getText();
}
public void setText(String text) {
textArea.setText(text);
}
public JComponent getView() {
return scroll;
}
}
public class InputFileEditor extends InputStringEditor {};
public class OutputResultEditor extends JPanel implements ActionListener {
private JToggleButton tbFail, tbOk;
public OutputResultEditor() {
super();
tbFail = new JToggleButton("Fail");
tbOk = new JToggleButton("OK");
ButtonGroup group = new ButtonGroup();
group.add(tbFail);
group.add(tbOk);
this.add(tbFail);
this.add(tbOk);
this.tbFail.addActionListener(this);
this.tbOk.addActionListener(this);
this.setPreferredSize(
new Dimension(TEST_EDITOR_WIDTH, 100));
}
public void actionPerformed(ActionEvent e) {
TestCaseOutputResult output =
(TestCaseOutputResult) currentTestCase.getOutput();
if(e.getSource() == tbFail) {
output.setScript(false);
} else {
output.setScript(true);
}
listCases.updateUI();
}
public void setValue(String value) {
if(TestCaseOutputResult.OK.equals(value)) {
this.tbOk.setSelected(true);
} else {
this.tbFail.setSelected(true);
}
}
}
public class OutputAstEditor implements CaretListener {
private JTextArea textArea = new JTextArea(20, 30);
private JScrollPane scroll = new JScrollPane(textArea,
JScrollPane.VERTICAL_SCROLLBAR_ALWAYS,
JScrollPane.HORIZONTAL_SCROLLBAR_ALWAYS);
public OutputAstEditor() {
super();
scroll.setBorder(BorderFactory.createLineBorder(Color.LIGHT_GRAY));
textArea.addCaretListener(this);
}
public void caretUpdate(CaretEvent arg0) {
currentTestCase.getOutput().setScript(getText());
listCases.updateUI();
}
public void setText(String text) {
this.textArea.setText(text);
}
public String getText() {
return this.textArea.getText();
}
public JScrollPane getView() {
return this.scroll;
}
}
public class OutputStdEditor extends OutputAstEditor {}
public class OutputReturnEditor extends OutputAstEditor {}
/* EVENT HANDLERS */
private class TestCaseListSelectionListener implements ListSelectionListener {
public void valueChanged(ListSelectionEvent e) {
if(e.getValueIsAdjusting()) return;
final JList list = (JList) e.getSource();
final TestCase value = (TestCase) list.getSelectedValue();
if(value != null) OnTestCaseSelected(value);
}
}
/* ACTIONS */
private class AddTestCaseAction extends AbstractAction {
public AddTestCaseAction() {
super("Add", ImageFactory.getSingleton().ADD);
putValue(SHORT_DESCRIPTION, "Add a gUnit test case.");
}
public void actionPerformed(ActionEvent e) {
OnAddTestCase();
}
}
private class RemoveTestCaseAction extends AbstractAction {
public RemoveTestCaseAction() {
super("Remove", ImageFactory.getSingleton().DELETE);
putValue(SHORT_DESCRIPTION, "Remove a gUnit test case.");
}
public void actionPerformed(ActionEvent e) {
OnRemoveTestCase();
}
}
/* CELL RENDERERS */
private static TestCaseListRenderer listRenderer
= new TestCaseListRenderer();
private static class TestCaseListRenderer implements ListCellRenderer {
private static Font IN_FONT = new Font("mono", Font.PLAIN, 12);
private static Font OUT_FONT = new Font("default", Font.BOLD, 12);
public static String clamp(String text, int len) {
if(text.length() > len) {
return text.substring(0, len - 3).concat("...");
} else {
return text;
}
}
public static String clampAtNewLine(String text) {
int pos = text.indexOf('\n');
if(pos >= 0) {
return text.substring(0, pos).concat("...");
} else {
return text;
}
}
public Component getListCellRendererComponent(
JList list, Object value, int index,
boolean isSelected, boolean hasFocus) {
final JPanel pane = new JPanel();
if (value instanceof TestCase) {
final TestCase item = (TestCase) value;
// create components
final JLabel labIn = new JLabel(
clamp(clampAtNewLine(item.getInput().getScript()), 18));
final JLabel labOut = new JLabel(
clamp(clampAtNewLine(item.getOutput().getScript()), 18));
labOut.setFont(OUT_FONT);
labIn.setFont(IN_FONT);
labIn.setIcon(item.getInput() instanceof TestCaseInputFile ?
ImageFactory.getSingleton().FILE16 :
ImageFactory.getSingleton().EDIT16);
pane.setBorder(BorderFactory.createEtchedBorder());
pane.setLayout(new BoxLayout(pane, BoxLayout.Y_AXIS));
pane.add(labIn);
pane.add(labOut);
pane.setBackground(isSelected ? Color.LIGHT_GRAY : Color.WHITE);
}
return pane;
}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/ 0000755 0001750 0001750 00000000000 11410174107 024034 5 ustar twerner twerner antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/ITestCaseOutput.java 0000644 0001750 0001750 00000003267 11256465172 027771 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package org.antlr.gunit.swingui.model;
/**
*
* @author scai
*/
public interface ITestCaseOutput {
public void setScript(String script);
public String getScript() ;
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCase.java 0000644 0001750 0001750 00000004620 11256465172 026431 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
public class TestCase {
private ITestCaseInput input;
private ITestCaseOutput output;
private boolean pass;
public boolean isPass() {
return pass;
}
public void setPass(boolean value) {
pass = value;
}
public ITestCaseInput getInput() {
return this.input;
}
public ITestCaseOutput getOutput() {
return this.output;
}
public TestCase(ITestCaseInput input, ITestCaseOutput output) {
this.input = input;
this.output = output;
}
@Override
public String toString() {
return String.format("[%s]->[%s]", input.getScript(), output.getScript());
}
public void setInput(ITestCaseInput in) {
this.input = in;
}
public void setOutput(ITestCaseOutput out) {
this.output = out;
}
public static String convertPreservedChars(String input) {
//return input.replace("\"", "\\\"");
return input;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCaseOutputResult.java 0000644 0001750 0001750 00000004067 11256465172 031056 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
/**
*
* @author scai
*/
public class TestCaseOutputResult implements ITestCaseOutput {
public static String OK = "OK";
public static String FAIL = "FAIL";
private boolean success ;
public TestCaseOutputResult(boolean result) {
this.success = result;
}
@Override
public String toString() {
return getScript();
}
public String getScript() {
return success ? OK : FAIL;
}
public void setScript(boolean value) {
this.success = value;
}
public void setScript(String script) {
success = Boolean.parseBoolean(script);
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCaseInputString.java 0000644 0001750 0001750 00000003642 11256465172 030643 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
/**
*
* @author scai
*/
public class TestCaseInputString implements ITestCaseInput {
private String script;
public TestCaseInputString(String text) {
this.script = text;
}
@Override
public String toString() {
return '"' + TestCase.convertPreservedChars(script) + '"';
}
public void setScript(String script) {
this.script = script;
}
public String getScript() {
return this.script;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestSuite.java 0000644 0001750 0001750 00000006260 11256465172 026651 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
import java.io.*;
import java.util.*;
import org.antlr.runtime.*;
public class TestSuite {
protected List<Rule> rules ;
protected String grammarName ;
protected CommonTokenStream tokens;
protected File testSuiteFile;
protected TestSuite(String gname, File testFile) {
grammarName = gname;
testSuiteFile = testFile;
rules = new ArrayList<Rule>();
}
/* Get the gUnit test suite file name. */
public File getTestSuiteFile() {
return testSuiteFile;
}
public void addRule(Rule currentRule) {
if(currentRule == null) throw new IllegalArgumentException("Null rule");
rules.add(currentRule);
}
// test rule name
public boolean hasRule(Rule rule) {
for(Rule r: rules) {
if(r.getName().equals(rule.getName())) {
return true;
}
}
return false;
}
public int getRuleCount() {
return rules.size();
}
public void setRules(List<Rule> newRules) {
rules.clear();
rules.addAll(newRules);
}
/* GETTERS AND SETTERS */
public void setGrammarName(String name) { grammarName = name;}
public String getGrammarName() { return grammarName; }
public Rule getRule(int index) { return rules.get(index); }
public CommonTokenStream getTokens() { return tokens; }
public void setTokens(CommonTokenStream ts) { tokens = ts; }
public Rule getRule(String name) {
for(Rule rule: rules) {
if(rule.getName().equals(name)) {
return rule;
}
}
return null;
}
// only for stringtemplate use
public List getRulesForStringTemplate() {return rules;}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCaseInputMultiString.java 0000644 0001750 0001750 00000003650 11256465172 031655 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
/**
*
* @author scai
*/
public class TestCaseInputMultiString implements ITestCaseInput {
private String script;
public TestCaseInputMultiString(String text) {
this.script = text;
}
@Override
public String toString() {
return "<<" + TestCase.convertPreservedChars(script) + ">>";
}
public void setScript(String script) {
this.script = script;
}
public String getScript() {
return this.script;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/Rule.java 0000644 0001750 0001750 00000004421 11256465172 025624 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
import java.util.ArrayList;
import java.util.List;
import javax.swing.DefaultListModel;
/**
* ANTLR v3 Rule Information.
* @author scai
*/
public class Rule extends DefaultListModel {
private String name;
public Rule(String name) {
this.name = name;
}
public String getName() { return name; }
public boolean getNotEmpty() {
return !this.isEmpty();
}
@Override
public String toString() {
return this.name;
}
public void addTestCase(TestCase newItem) {
this.addElement(newItem);
}
// for string template
public List<TestCase> getTestCases() {
List<TestCase> result = new ArrayList<TestCase>();
for(int i=0; i<this.size(); i++) {
result.add((TestCase)this.getElementAt(i));
}
return result;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCaseInputFile.java 0000644 0001750 0001750 00000003767 11256465172 030264 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
import javax.swing.JComponent;
import javax.swing.JLabel;
/**
*
* @author scai
*/
public class TestCaseInputFile implements ITestCaseInput {
private String fileName;
public TestCaseInputFile(String file) {
this.fileName = file;
}
public String getLabel() {
return "FILE:" + fileName;
}
public void setScript(String script) {
this.fileName = script;
}
@Override
public String toString() {
return fileName;
}
public String getScript() {
return this.fileName;
}
} antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCaseOutputStdOut.java 0000644 0001750 0001750 00000003614 11256465172 031017 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
/**
*
* @author scai
*/
public class TestCaseOutputStdOut implements ITestCaseOutput {
private String script;
public TestCaseOutputStdOut(String text) {
this.script = text;
}
@Override
public String toString() {
return String.format(" -> \"%s\"", script);
}
public void setScript(String script) {
this.script = script;
}
public String getScript() {
return this.script;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCaseOutputAST.java 0000644 0001750 0001750 00000003636 11256465172 030230 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
/**
*
* @author scai
*/
public class TestCaseOutputAST implements ITestCaseOutput {
private String treeString;
public TestCaseOutputAST(String script) {
this.treeString = script;
}
public void setScript(String script) {
this.treeString = script;
}
public String getScript() {
return this.treeString;
}
@Override
public String toString() {
return String.format(" -> %s", treeString);
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestCaseOutputReturn.java 0000644 0001750 0001750 00000003563 11256465172 031057 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
public class TestCaseOutputReturn implements ITestCaseOutput {
private String script;
public TestCaseOutputReturn(String text) {
this.script = text;
}
@Override
public String toString() {
return String.format(" returns [%s]", script);
}
public void setScript(String script) {
this.script = script;
}
public String getScript() {
return this.script;
}
} antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/ITestCaseInput.java 0000644 0001750 0001750 00000003066 11256465172 027565 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
public interface ITestCaseInput {
public void setScript(String script);
public String getScript();
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/model/TestSuiteFactory.java 0000644 0001750 0001750 00000016432 11256465172 030203 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui.model;
import org.antlr.gunit.swingui.runner.TestSuiteAdapter;
import java.io.*;
import java.util.*;
import org.antlr.gunit.swingui.Tool;
import org.antlr.gunit.swingui.parsers.*;
import org.antlr.runtime.*;
import org.antlr.stringtemplate.*;
public class TestSuiteFactory {
private static String TEMPLATE_FILE = "org/antlr/gunit/swingui/gunit.stg";
private static StringTemplateGroup templates;
public static final String TEST_SUITE_EXT = ".gunit";
public static final String GRAMMAR_EXT = ".g";
static {
ClassLoader loader = TestSuiteFactory.class.getClassLoader();
InputStream in = loader.getResourceAsStream(TEMPLATE_FILE);
Reader rd = new InputStreamReader(in);
templates = new StringTemplateGroup(rd);
}
/**
* Factory method: create a testsuite from ANTLR grammar. Save the test
* suite file in the same directory of the grammar file.
* @param grammarFile ANTLRv3 grammar file.
* @return test suite object
*/
public static TestSuite createTestSuite(File grammarFile) {
if(grammarFile != null && grammarFile.exists() && grammarFile.isFile()) {
final String fileName = grammarFile.getName();
final String grammarName = fileName.substring(0, fileName.lastIndexOf('.'));
final String grammarDir = grammarFile.getParent();
final File testFile = new File(grammarDir + File.separator + grammarName + TEST_SUITE_EXT);
final TestSuite result = new TestSuite(grammarName, testFile);
result.rules = loadRulesFromGrammar(grammarFile);
if(saveTestSuite(result)) {
return result;
} else {
throw new RuntimeException("Can't save test suite file.");
}
} else {
throw new RuntimeException("Invalid grammar file.");
}
}
/* Load rules from an ANTLR grammar file. */
private static List<Rule> loadRulesFromGrammar(File grammarFile) {
// get all the rule names
final List<String> ruleNames = new ArrayList<String>();
try {
final Reader reader = new BufferedReader(new FileReader(grammarFile));
final ANTLRv3Lexer lexer = new ANTLRv3Lexer(new ANTLRReaderStream(reader));
final CommonTokenStream tokens = new CommonTokenStream(lexer);
final ANTLRv3Parser parser = new ANTLRv3Parser(tokens);
parser.rules = ruleNames;
parser.grammarDef();
reader.close();
} catch (Exception e) {
e.printStackTrace();
}
// convert to rule object
final List<Rule> ruleList = new ArrayList<Rule>();
for(String str: ruleNames) {
ruleList.add(new Rule(str));
}
return ruleList;
}
/* Save testsuite to *.gunit file. */
public static boolean saveTestSuite(TestSuite testSuite) {
final String data = getScript(testSuite);
try {
FileWriter fw = new FileWriter(testSuite.getTestSuiteFile());
fw.write(data);
fw.flush();
fw.close();
} catch (IOException e) {
e.printStackTrace();
return false;
}
return true;
}
/**
* Get the text script from the testSuite.
* @param testSuite
* @return test script
*/
public static String getScript(TestSuite testSuite) {
if(testSuite == null) return null;
StringTemplate gUnitScript = templates.getInstanceOf("gUnitFile");
gUnitScript.setAttribute("testSuite", testSuite);
return gUnitScript.toString();
}
/**
* From textual script to program model.
* @param file testsuite file (.gunit)
* @return test suite object
*/
public static TestSuite loadTestSuite(File file) {
// check grammar file
final File grammarFile = getGrammarFile(file);
if(grammarFile == null)
throw new RuntimeException("Can't find grammar file.");
TestSuite result = new TestSuite("", file);
// read in test suite
try {
final Reader reader = new BufferedReader(new FileReader(file));
final StGUnitLexer lexer = new StGUnitLexer(new ANTLRReaderStream(reader));
final CommonTokenStream tokens = new CommonTokenStream(lexer);
final StGUnitParser parser = new StGUnitParser(tokens);
final TestSuiteAdapter adapter = new TestSuiteAdapter(result);
parser.adapter = adapter;
parser.gUnitDef();
result.setTokens(tokens);
reader.close();
} catch (Exception ex) {
throw new RuntimeException("Error reading test suite file.\n" + ex.getMessage());
}
// load un-tested rules from grammar
final List<Rule> completeRuleList = loadRulesFromGrammar(grammarFile);
for(Rule rule: completeRuleList) {
if(!result.hasRule(rule)) {
result.addRule(rule);
//System.out.println("Add rule:" + rule);
}
}
return result;
}
/**
* Get the grammar file of the testsuite file in the same directory.
* @param testsuiteFile
* @return grammar file or null
*/
private static File getGrammarFile(File testsuiteFile) {
final String sTestFile;
try {
sTestFile = testsuiteFile.getCanonicalPath();
} catch (IOException e) {
return null;
}
final String sGrammarFile = sTestFile.substring(0, sTestFile.lastIndexOf('.')) + GRAMMAR_EXT;
final File fileGrammar = new File(sGrammarFile);
if(fileGrammar.exists() && fileGrammar.isFile())
return fileGrammar;
else
return null;
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/ImageFactory.java 0000644 0001750 0001750 00000004774 11256465172 026202 0 ustar twerner twerner package org.antlr.gunit.swingui;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.ImageIcon;
public class ImageFactory {
private static ImageFactory singleton ;
public static ImageFactory getSingleton() {
if(singleton == null) singleton = new ImageFactory();
return singleton;
}
private ImageFactory() {
ACCEPT = getImage("accept.png");
ADD = getImage("add.png");
DELETE = getImage("delete24.png");
TEXTFILE = getImage("textfile24.png");
TEXTFILE16 = getImage("textfile16.png");
ADDFILE = getImage("addfile24.png");
WINDOW16 = getImage("windowb16.png");
FAV16 = getImage("favb16.png");
SAVE = getImage("floppy24.png");
OPEN = getImage("folder24.png");
EDIT16 = getImage("edit16.png");
FILE16 = getImage("file16.png");
RUN_PASS = getImage("runpass.png");
RUN_FAIL = getImage("runfail.png");
TESTSUITE = getImage("testsuite.png");
TESTGROUP = getImage("testgroup.png");
TESTGROUPX = getImage("testgroupx.png");
NEXT = getImage("next24.png");
}
private ImageIcon getImage(String name) {
name = IMG_DIR + name;
try {
final ClassLoader loader = ImageFactory.class.getClassLoader();
final InputStream in = loader.getResourceAsStream(name);
final byte[] data = new byte[in.available()];
in.read(data);
in.close();
return new ImageIcon(data);
} catch (IOException ex) {
System.err.println("Can't load image file: " + name);
System.exit(1);
} catch(RuntimeException e) {
System.err.println("Can't load image file: " + name);
System.exit(1);
}
return null;
}
private static final String IMG_DIR = "org/antlr/gunit/swingui/images/";
public ImageIcon ACCEPT;
public ImageIcon ADD;
public ImageIcon DELETE;
public ImageIcon TEXTFILE ;
public ImageIcon ADDFILE;
public ImageIcon TEXTFILE16 ;
public ImageIcon WINDOW16;
public ImageIcon FAV16;
public ImageIcon SAVE ;
public ImageIcon OPEN ;
public ImageIcon EDIT16;
public ImageIcon FILE16;
public ImageIcon NEXT;
public ImageIcon RUN_PASS;
public ImageIcon RUN_FAIL;
public ImageIcon TESTSUITE;
public ImageIcon TESTGROUP ;
public ImageIcon TESTGROUPX;
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/WorkSpaceController.java 0000644 0001750 0001750 00000023402 11256465172 027557 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.antlr.gunit.swingui.runner.gUnitAdapter;
import java.awt.*;
import java.io.IOException;
import org.antlr.gunit.swingui.model.*;
import org.antlr.gunit.swingui.ImageFactory;
import java.awt.event.*;
import java.io.File;
import javax.swing.*;
import javax.swing.event.*;
import javax.swing.filechooser.FileFilter;
/**
*
* @author scai
*/
public class WorkSpaceController implements IController{
/* MODEL */
private TestSuite currentTestSuite;
private String testSuiteFileName = null; // path + file
/* VIEW */
private final WorkSpaceView view = new WorkSpaceView();
/* SUB-CONTROL */
private final RunnerController runner = new RunnerController();
public WorkSpaceController() {
view.resultPane = (JPanel) runner.getView();
view.initComponents();
this.initEventHandlers();
this.initToolbar();
}
public void show() {
this.view.setTitle("gUnitEditor");
this.view.setVisible(true);
this.view.pack();
}
public Component getEmbeddedView() {
return view.paneEditor.getView();
}
private void initEventHandlers() {
this.view.tabEditors.addChangeListener(new TabChangeListener());
this.view.listRules.setListSelectionListener(new RuleListSelectionListener());
this.view.paneEditor.onTestCaseNumberChange = new ActionListener() {
public void actionPerformed(ActionEvent e) {
view.listRules.getView().updateUI();
}
};
}
private void OnCreateTest() {
JFileChooser jfc = new JFileChooser();
jfc.setDialogTitle("Create test suite from grammar");
jfc.setDialogType(JFileChooser.OPEN_DIALOG);
jfc.setFileFilter(new FileFilter() {
@Override
public boolean accept(File f) {
return f.isDirectory() || f.getName().toLowerCase().endsWith(TestSuiteFactory.GRAMMAR_EXT);
}
@Override
public String getDescription() {
return "ANTLR grammar file (*.g)";
}
});
if( jfc.showOpenDialog(view) != JFileChooser.APPROVE_OPTION ) return;
view.paneStatus.setProgressIndetermined(true);
final File grammarFile = jfc.getSelectedFile();
currentTestSuite = TestSuiteFactory.createTestSuite(grammarFile);
view.listRules.initialize(currentTestSuite);
view.tabEditors.setSelectedIndex(0);
view.paneStatus.setText("Grammar: " + currentTestSuite.getGrammarName());
view.paneStatus.setProgressIndetermined(false);
testSuiteFileName = null;
}
private void OnSaveTest() {
TestSuiteFactory.saveTestSuite(currentTestSuite);
JOptionPane.showMessageDialog(view, "Testsuite saved to:\n" + currentTestSuite.getTestSuiteFile().getAbsolutePath());
}
private void OnOpenTest() {
JFileChooser jfc = new JFileChooser();
jfc.setDialogTitle("Open existing gUnit test suite");
jfc.setDialogType(JFileChooser.OPEN_DIALOG);
jfc.setFileFilter(new FileFilter() {
@Override
public boolean accept(File f) {
return f.isDirectory() || f.getName().toLowerCase().endsWith(TestSuiteFactory.TEST_SUITE_EXT);
}
@Override
public String getDescription() {
return "ANTLR unit test file (*.gunit)";
}
});
if( jfc.showOpenDialog(view) != JFileChooser.APPROVE_OPTION ) return;
final File testSuiteFile = jfc.getSelectedFile();
try {
testSuiteFileName = testSuiteFile.getCanonicalPath();
} catch (IOException e) {
throw new RuntimeException(e);
}
view.paneStatus.setProgressIndetermined(true);
currentTestSuite = TestSuiteFactory.loadTestSuite(testSuiteFile);
view.listRules.initialize(currentTestSuite);
view.paneStatus.setText(currentTestSuite.getGrammarName());
view.tabEditors.setSelectedIndex(0);
view.paneStatus.setProgressIndetermined(false);
}
private void OnSelectRule(Rule rule) {
if(rule == null) throw new IllegalArgumentException("Null");
this.view.paneEditor.OnLoadRule(rule);
this.view.paneStatus.setRule(rule.getName());
// run result
this.runner.OnShowRuleResult(rule);
}
private void OnSelectTextPane() {
Thread worker = new Thread () {
@Override
public void run() {
view.paneStatus.setProgressIndetermined(true);
view.txtEditor.setText(
TestSuiteFactory.getScript(currentTestSuite));
view.paneStatus.setProgressIndetermined(false);
}
};
worker.start();
}
private void OnRunTest() {
// save before run
TestSuiteFactory.saveTestSuite(currentTestSuite);
// run
try {
final gUnitAdapter adapter = new gUnitAdapter(currentTestSuite);
if(currentTestSuite == null) return;
adapter.run();
runner.OnShowSuiteResult(currentTestSuite);
view.tabEditors.addTab("Test Result", ImageFactory.getSingleton().FILE16, runner.getView());
view.tabEditors.setSelectedComponent(runner.getView());
} catch (Exception ex) {
JOptionPane.showMessageDialog(view, "Fail to run test:\n" + ex.getMessage(), "Error", JOptionPane.ERROR_MESSAGE);
}
}
private void initToolbar() {
view.toolbar.add(new JButton(new CreateAction()));
view.toolbar.add(new JButton(new OpenAction()));
view.toolbar.add(new JButton(new SaveAction()));
view.toolbar.add(new JButton(new RunAction()));
}
public Object getModel() {
throw new UnsupportedOperationException("Not supported yet.");
}
public Component getView() {
return view;
}
/** Event handler for rule list selection. */
private class RuleListSelectionListener implements ListSelectionListener {
public void valueChanged(ListSelectionEvent event) {
if(event.getValueIsAdjusting()) return;
final JList list = (JList) event.getSource();
final Rule rule = (Rule) list.getSelectedValue();
if(rule != null) OnSelectRule(rule);
}
}
/** Event handler for switching between editor view and script view. */
public class TabChangeListener implements ChangeListener {
public void stateChanged(ChangeEvent evt) {
if(view.tabEditors.getSelectedIndex() == 1) {
OnSelectTextPane();
}
}
}
/** Create test suite action. */
private class CreateAction extends AbstractAction {
public CreateAction() {
super("Create", ImageFactory.getSingleton().ADDFILE);
putValue(SHORT_DESCRIPTION, "Create a test suite from an ANTLR grammar");
}
public void actionPerformed(ActionEvent e) {
OnCreateTest();
}
}
/** Save test suite action. */
private class SaveAction extends AbstractAction {
public SaveAction() {
super("Save", ImageFactory.getSingleton().SAVE);
putValue(SHORT_DESCRIPTION, "Save the test suite");
}
public void actionPerformed(ActionEvent e) {
OnSaveTest();
}
}
/** Open test suite action. */
private class OpenAction extends AbstractAction {
public OpenAction() {
super("Open", ImageFactory.getSingleton().OPEN);
putValue(SHORT_DESCRIPTION, "Open an existing test suite");
putValue(ACCELERATOR_KEY, KeyStroke.getKeyStroke(
KeyEvent.VK_O, InputEvent.CTRL_MASK));
}
public void actionPerformed(ActionEvent e) {
OnOpenTest();
}
}
/** Run test suite action. */
private class RunAction extends AbstractAction {
public RunAction() {
super("Run", ImageFactory.getSingleton().NEXT);
putValue(SHORT_DESCRIPTION, "Run the current test suite");
putValue(ACCELERATOR_KEY, KeyStroke.getKeyStroke(
KeyEvent.VK_R, InputEvent.CTRL_MASK));
}
public void actionPerformed(ActionEvent e) {
OnRunTest();
}
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/Tool.java 0000644 0001750 0001750 00000004306 11256465172 024534 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import java.io.IOException;
import javax.swing.SwingUtilities;
import javax.swing.UIManager;
public class Tool {
public static void main(String[] args) throws IOException {
if(args.length == 1 && "-version".equals(args[0])) {
System.out.println("gUnitEditor Swing GUI\nby Shaoting Cai\n");
} else {
showUI();
}
}
private static void showUI() {
try {
UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
} catch (Exception e) {
}
SwingUtilities.invokeLater(new Runnable() {
public void run() {
WorkSpaceController control = new WorkSpaceController();
control.show();
}
});
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/WorkSpaceView.java 0000644 0001750 0001750 00000007632 11256465172 026355 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package org.antlr.gunit.swingui;
import org.antlr.gunit.swingui.ImageFactory;
import java.awt.*;
import javax.swing.*;
/**
*
* @author scai
*/
public class WorkSpaceView extends JFrame {
protected JSplitPane splitListClient ;
protected JTabbedPane tabEditors;
protected JPanel paneToolBar;
protected StatusBarController paneStatus;
protected TestCaseEditController paneEditor;
protected JToolBar toolbar;
protected JTextArea txtEditor;
protected RuleListController listRules;
protected JMenuBar menuBar;
protected JScrollPane scrollCode;
protected JPanel resultPane;
protected JButton btnOpenGrammar;
public WorkSpaceView() {
super();
}
protected void initComponents() {
this.paneEditor = new TestCaseEditController(this);
this.paneStatus = new StatusBarController();
this.toolbar = new JToolBar();
this.toolbar.setBorder(BorderFactory.createEmptyBorder());
this.toolbar.setFloatable(false);
this.toolbar.setBorder(BorderFactory.createEmptyBorder());
this.txtEditor = new JTextArea();
this.txtEditor.setLineWrap(false);
this.txtEditor.setFont(new Font("Courier New", Font.PLAIN, 13));
this.scrollCode = new JScrollPane(txtEditor,
JScrollPane.VERTICAL_SCROLLBAR_ALWAYS,
JScrollPane.HORIZONTAL_SCROLLBAR_AS_NEEDED);
this.scrollCode.setBorder(BorderFactory.createLineBorder(Color.LIGHT_GRAY));
this.tabEditors = new JTabbedPane();
this.tabEditors.addTab("Case Editor", ImageFactory.getSingleton().TEXTFILE16, this.paneEditor.getView());
this.tabEditors.addTab("Script Source", ImageFactory.getSingleton().WINDOW16, this.scrollCode);
this.listRules = new RuleListController();
this.splitListClient = new JSplitPane( JSplitPane.HORIZONTAL_SPLIT,
this.listRules.getView(), this.tabEditors);
this.splitListClient.setResizeWeight(0.4);
this.splitListClient.setBorder(BorderFactory.createEmptyBorder());
this.getContentPane().add(this.toolbar, BorderLayout.NORTH);
this.getContentPane().add(this.splitListClient, BorderLayout.CENTER);
this.getContentPane().add(this.paneStatus.getView(), BorderLayout.SOUTH);
// self
this.setPreferredSize(new Dimension(900, 500));
this.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/swingui/AbstractInputEditor.java 0000644 0001750 0001750 00000003624 11256465172 027553 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit.swingui;
import org.antlr.gunit.swingui.model.ITestCaseInput;
import java.awt.event.ActionListener;
import javax.swing.JComponent;
/**
*
* @author scai
*/
public abstract class AbstractInputEditor {
protected ITestCaseInput input;
public void setInput(ITestCaseInput input) {
this.input = input;
}
protected JComponent comp;
public JComponent getControl() { return comp; }
abstract public void addActionListener(ActionListener l) ;
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/ITestCase.java 0000644 0001750 0001750 00000004431 11256465172 023755 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
/**
* ITestCase object locates one test case in a gUnit script by specifying the
* tested rule and the index number of the test case in that group.
*
* For example:
* ----------------------
* ...
* varDef:
* "int i;" OK
* "float 2f;" FAIL
* ...
* ----------------------
* The "testedRuleName" for these two test cases will be "varDef".
* The "index" for the "int"-test will be 0.
* The "index" for the "float"-test will be 1. And so on.
*
* @see ITestSuite
*/
public interface ITestCase {
/**
* Get the name of the rule that is tested by this test case.
* @return name of the tested rule.
*/
public String getTestedRuleName();
/**
* Get the index of the test case in the test group for a rule. Starting
* from 0.
* @return index number of the test case.
*/
public int getTestCaseIndex();
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/ITestSuite.java 0000644 0001750 0001750 00000003543 11256465172 024176 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2009 Shaoting Cai
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
/**
* A gUnit script file is an Antlr "test suite". The interface is defined to
* allow the Swing GUI test runner be notified when gUnit interpreter runner
* runs a passed/failed test case.
*
* CHANGES:
* 2009-03-01: SHAOTING
* - change method return void, parameter test object.
*/
public interface ITestSuite {
public void onPass(ITestCase passTest);
public void onFail(ITestCase failTest);
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/JUnitCodeGen.java 0000644 0001750 0001750 00000033643 11256465172 024416 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.StringTemplateGroupLoader;
import org.antlr.stringtemplate.CommonGroupLoader;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import java.io.*;
import java.lang.reflect.Method;
import java.util.HashMap;
import java.util.Map;
import java.util.logging.ConsoleHandler;
import java.util.logging.Handler;
import java.util.logging.Level;
import java.util.logging.Logger;
public class JUnitCodeGen {
public GrammarInfo grammarInfo;
public Map<String, String> ruleWithReturn;
private final String testsuiteDir;
private String outputDirectoryPath = ".";
private final static Handler console = new ConsoleHandler();
private static final Logger logger = Logger.getLogger(JUnitCodeGen.class.getName());
static {
logger.addHandler(console);
}
public JUnitCodeGen(GrammarInfo grammarInfo, String testsuiteDir) throws ClassNotFoundException {
this( grammarInfo, determineClassLoader(), testsuiteDir);
}
private static ClassLoader determineClassLoader() {
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
if ( classLoader == null ) {
classLoader = JUnitCodeGen.class.getClassLoader();
}
return classLoader;
}
public JUnitCodeGen(GrammarInfo grammarInfo, ClassLoader classLoader, String testsuiteDir) throws ClassNotFoundException {
this.grammarInfo = grammarInfo;
this.testsuiteDir = testsuiteDir;
/** Map the name of rules having return value to its return type */
ruleWithReturn = new HashMap<String, String>();
Class parserClass = locateParserClass( grammarInfo, classLoader );
Method[] methods = parserClass.getDeclaredMethods();
for(Method method : methods) {
if ( !method.getReturnType().getName().equals("void") ) {
ruleWithReturn.put(method.getName(), method.getReturnType().getName().replace('$', '.'));
}
}
}
private Class locateParserClass(GrammarInfo grammarInfo, ClassLoader classLoader) throws ClassNotFoundException {
String parserClassName = grammarInfo.getGrammarName() + "Parser";
if ( grammarInfo.getHeader() != null ) {
parserClassName = grammarInfo.getHeader()+ "." + parserClassName;
}
return classLoader.loadClass( parserClassName );
}
public String getOutputDirectoryPath() {
return outputDirectoryPath;
}
public void setOutputDirectoryPath(String outputDirectoryPath) {
this.outputDirectoryPath = outputDirectoryPath;
}
public void compile() throws IOException{
String junitFileName;
if ( grammarInfo.getTreeGrammarName()!=null ) {
junitFileName = "Test"+grammarInfo.getTreeGrammarName();
}
else {
junitFileName = "Test"+grammarInfo.getGrammarName();
}
String lexerName = grammarInfo.getGrammarName()+"Lexer";
String parserName = grammarInfo.getGrammarName()+"Parser";
StringTemplateGroupLoader loader = new CommonGroupLoader("org/antlr/gunit", null);
StringTemplateGroup.registerGroupLoader(loader);
StringTemplateGroup.registerDefaultLexer(AngleBracketTemplateLexer.class);
StringBuffer buf = compileToBuffer(junitFileName, lexerName, parserName);
writeTestFile(".", junitFileName+".java", buf.toString());
}
public StringBuffer compileToBuffer(String className, String lexerName, String parserName) {
StringTemplateGroup group = StringTemplateGroup.loadGroup("junit");
StringBuffer buf = new StringBuffer();
buf.append(genClassHeader(group, className, lexerName, parserName));
buf.append(genTestRuleMethods(group));
buf.append("\n\n}");
return buf;
}
protected String genClassHeader(StringTemplateGroup group, String junitFileName, String lexerName, String parserName) {
StringTemplate classHeaderST = group.getInstanceOf("classHeader");
if ( grammarInfo.getHeader()!=null ) { // Set up class package if there is
classHeaderST.setAttribute("header", "package "+grammarInfo.getHeader()+";");
}
classHeaderST.setAttribute("junitFileName", junitFileName);
String lexerPath = null;
String parserPath = null;
String treeParserPath = null;
String packagePath = null;
boolean isTreeGrammar = false;
boolean hasPackage = false;
/** Set up appropriate class path for parser/tree parser if using package */
if ( grammarInfo.getHeader()!=null ) {
hasPackage = true;
packagePath = "./"+grammarInfo.getHeader().replace('.', '/');
lexerPath = grammarInfo.getHeader()+"."+lexerName;
parserPath = grammarInfo.getHeader()+"."+parserName;
if ( grammarInfo.getTreeGrammarName()!=null ) {
treeParserPath = grammarInfo.getHeader()+"."+grammarInfo.getTreeGrammarName();
isTreeGrammar = true;
}
}
else {
lexerPath = lexerName;
parserPath = parserName;
if ( grammarInfo.getTreeGrammarName()!=null ) {
treeParserPath = grammarInfo.getTreeGrammarName();
isTreeGrammar = true;
}
}
classHeaderST.setAttribute("hasPackage", hasPackage);
classHeaderST.setAttribute("packagePath", packagePath);
classHeaderST.setAttribute("lexerPath", lexerPath);
classHeaderST.setAttribute("parserPath", parserPath);
classHeaderST.setAttribute("treeParserPath", treeParserPath);
classHeaderST.setAttribute("isTreeGrammar", isTreeGrammar);
return classHeaderST.toString();
}
protected String genTestRuleMethods(StringTemplateGroup group) {
StringBuffer buf = new StringBuffer();
if ( grammarInfo.getTreeGrammarName()!=null ) { // Generate junit codes of for tree grammar rule
for ( gUnitTestSuite ts: grammarInfo.getRuleTestSuites() ) {
int i = 0;
for ( gUnitTestInput input: ts.testSuites.keySet() ) { // each rule may contain multiple tests
i++;
StringTemplate testRuleMethodST;
/** If rule has multiple return values or ast*/
if ( ts.testSuites.get(input).getType()==gUnitParser.ACTION && ruleWithReturn.containsKey(ts.getTreeRuleName()) ) {
testRuleMethodST = group.getInstanceOf("testTreeRuleMethod2");
String inputString = escapeForJava(input.testInput);
String outputString = ts.testSuites.get(input).getText();
testRuleMethodST.setAttribute("methodName", "test"+changeFirstCapital(ts.getTreeRuleName())+"_walks_"+
changeFirstCapital(ts.getRuleName())+i);
testRuleMethodST.setAttribute("testTreeRuleName", '"'+ts.getTreeRuleName()+'"');
testRuleMethodST.setAttribute("testRuleName", '"'+ts.getRuleName()+'"');
testRuleMethodST.setAttribute("testInput", '"'+inputString+'"');
testRuleMethodST.setAttribute("returnType", ruleWithReturn.get(ts.getTreeRuleName()));
testRuleMethodST.setAttribute("isFile", input.inputIsFile);
testRuleMethodST.setAttribute("expecting", outputString);
}
else {
testRuleMethodST = group.getInstanceOf("testTreeRuleMethod");
String inputString = escapeForJava(input.testInput);
String outputString = ts.testSuites.get(input).getText();
testRuleMethodST.setAttribute("methodName", "test"+changeFirstCapital(ts.getTreeRuleName())+"_walks_"+
changeFirstCapital(ts.getRuleName())+i);
testRuleMethodST.setAttribute("testTreeRuleName", '"'+ts.getTreeRuleName()+'"');
testRuleMethodST.setAttribute("testRuleName", '"'+ts.getRuleName()+'"');
testRuleMethodST.setAttribute("testInput", '"'+inputString+'"');
testRuleMethodST.setAttribute("isFile", input.inputIsFile);
testRuleMethodST.setAttribute("tokenType", getTypeString(ts.testSuites.get(input).getType()));
if ( ts.testSuites.get(input).getType()==gUnitParser.ACTION ) { // trim ';' at the end of ACTION if there is...
//testRuleMethodST.setAttribute("expecting", outputString.substring(0, outputString.length()-1));
testRuleMethodST.setAttribute("expecting", outputString);
}
else if ( ts.testSuites.get(input).getType()==gUnitParser.RETVAL ) { // Expected: RETVAL
testRuleMethodST.setAttribute("expecting", outputString);
}
else { // Attach "" to expected STRING or AST
testRuleMethodST.setAttribute("expecting", '"'+escapeForJava(outputString)+'"');
}
}
buf.append(testRuleMethodST.toString());
}
}
}
else { // Generate junit codes of for grammar rule
for ( gUnitTestSuite ts: grammarInfo.getRuleTestSuites() ) {
int i = 0;
for ( gUnitTestInput input: ts.testSuites.keySet() ) { // each rule may contain multiple tests
i++;
StringTemplate testRuleMethodST;
/** If rule has multiple return values or ast*/
if ( ts.testSuites.get(input).getType()==gUnitParser.ACTION && ruleWithReturn.containsKey(ts.getRuleName()) ) {
testRuleMethodST = group.getInstanceOf("testRuleMethod2");
String inputString = escapeForJava(input.testInput);
String outputString = ts.testSuites.get(input).getText();
testRuleMethodST.setAttribute("methodName", "test"+changeFirstCapital(ts.getRuleName())+i);
testRuleMethodST.setAttribute("testRuleName", '"'+ts.getRuleName()+'"');
testRuleMethodST.setAttribute("testInput", '"'+inputString+'"');
testRuleMethodST.setAttribute("returnType", ruleWithReturn.get(ts.getRuleName()));
testRuleMethodST.setAttribute("isFile", input.inputIsFile);
testRuleMethodST.setAttribute("expecting", outputString);
}
else {
String testRuleName;
// need to determine whether it's a test for parser rule or lexer rule
if ( ts.isLexicalRule() ) testRuleName = ts.getLexicalRuleName();
else testRuleName = ts.getRuleName();
testRuleMethodST = group.getInstanceOf("testRuleMethod");
String inputString = escapeForJava(input.testInput);
String outputString = ts.testSuites.get(input).getText();
testRuleMethodST.setAttribute("isLexicalRule", ts.isLexicalRule());
testRuleMethodST.setAttribute("methodName", "test"+changeFirstCapital(testRuleName)+i);
testRuleMethodST.setAttribute("testRuleName", '"'+testRuleName+'"');
testRuleMethodST.setAttribute("testInput", '"'+inputString+'"');
testRuleMethodST.setAttribute("isFile", input.inputIsFile);
testRuleMethodST.setAttribute("tokenType", getTypeString(ts.testSuites.get(input).getType()));
if ( ts.testSuites.get(input).getType()==gUnitParser.ACTION ) { // trim ';' at the end of ACTION if there is...
//testRuleMethodST.setAttribute("expecting", outputString.substring(0, outputString.length()-1));
testRuleMethodST.setAttribute("expecting", outputString);
}
else if ( ts.testSuites.get(input).getType()==gUnitParser.RETVAL ) { // Expected: RETVAL
testRuleMethodST.setAttribute("expecting", outputString);
}
else { // Attach "" to expected STRING or AST
testRuleMethodST.setAttribute("expecting", '"'+escapeForJava(outputString)+'"');
}
}
buf.append(testRuleMethodST.toString());
}
}
}
return buf.toString();
}
// return a meaningful gUnit token type name instead of using the magic number
public String getTypeString(int type) {
String typeText;
switch (type) {
case gUnitParser.OK :
typeText = "org.antlr.gunit.gUnitParser.OK";
break;
case gUnitParser.FAIL :
typeText = "org.antlr.gunit.gUnitParser.FAIL";
break;
case gUnitParser.STRING :
typeText = "org.antlr.gunit.gUnitParser.STRING";
break;
case gUnitParser.ML_STRING :
typeText = "org.antlr.gunit.gUnitParser.ML_STRING";
break;
case gUnitParser.RETVAL :
typeText = "org.antlr.gunit.gUnitParser.RETVAL";
break;
case gUnitParser.AST :
typeText = "org.antlr.gunit.gUnitParser.AST";
break;
default :
typeText = "org.antlr.gunit.gUnitParser.EOF";
break;
}
return typeText;
}
protected void writeTestFile(String dir, String fileName, String content) {
try {
File f = new File(dir, fileName);
FileWriter w = new FileWriter(f);
BufferedWriter bw = new BufferedWriter(w);
bw.write(content);
bw.close();
w.close();
}
catch (IOException ioe) {
logger.log(Level.SEVERE, "can't write file", ioe);
}
}
public static String escapeForJava(String inputString) {
// Gotta escape literal backslash before putting in specials that use escape.
inputString = inputString.replace("\\", "\\\\");
// Then double quotes need escaping (singles are OK of course).
inputString = inputString.replace("\"", "\\\"");
// note: replace newline to String ".\n", replace tab to String ".\t"
inputString = inputString.replace("\n", "\\n").replace("\t", "\\t").replace("\r", "\\r").replace("\b", "\\b").replace("\f", "\\f");
return inputString;
}
protected String changeFirstCapital(String ruleName) {
String firstChar = String.valueOf(ruleName.charAt(0));
return firstChar.toUpperCase()+ruleName.substring(1);
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/Interp.java 0000644 0001750 0001750 00000006623 11256465172 023377 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Leon Jen-Yuan Su
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
import java.io.File;
import java.io.IOException;
import org.antlr.runtime.*;
/** The main gUnit interpreter entry point.
* Read a gUnit script, run unit tests or generate a junit file.
*/
public class Interp {
public static void main(String[] args) throws IOException, ClassNotFoundException, RecognitionException {
/** Pull char from where? */
CharStream input = null;
/** If the input source is a testsuite file, where is it? */
String testsuiteDir = System.getProperty("user.dir");
/** Generate junit codes */
if ( args.length>0 && args[0].equals("-o") ) {
if ( args.length==2 ) {
input = new ANTLRFileStream(args[1]);
File f = new File(args[1]);
testsuiteDir = getTestsuiteDir(f.getCanonicalPath(), f.getName());
}
else
input = new ANTLRInputStream(System.in);
JUnitCodeGen generater = new JUnitCodeGen(parse(input), testsuiteDir);
generater.compile();
return;
}
/** Run gunit tests */
if ( args.length==1 ) {
input = new ANTLRFileStream(args[0]);
File f = new File(args[0]);
testsuiteDir = getTestsuiteDir(f.getCanonicalPath(), f.getName());
}
else
input = new ANTLRInputStream(System.in);
gUnitExecutor executer = new gUnitExecutor(parse(input), testsuiteDir);
System.out.print(executer.execTest()); // unit test result
//return an error code of the number of failures
System.exit(executer.failures.size() + executer.invalids.size());
}
public static GrammarInfo parse(CharStream input) throws RecognitionException {
gUnitLexer lexer = new gUnitLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lexer);
GrammarInfo grammarInfo = new GrammarInfo();
gUnitParser parser = new gUnitParser(tokens, grammarInfo);
parser.gUnitDef(); // parse gunit script and save elements to grammarInfo
return grammarInfo;
}
public static String getTestsuiteDir(String fullPath, String fileName) {
return fullPath.substring(0, fullPath.length()-fileName.length());
}
}
antlr-3.2/gunit/src/main/java/org/antlr/gunit/BooleanTest.java 0000644 0001750 0001750 00000004162 11256465172 024351 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2007 Kenny MacDermid
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.gunit;
public class BooleanTest extends AbstractTest {
private boolean ok;
public BooleanTest(boolean ok) {
this.ok = ok;
}
@Override
public String getText() {
return (ok)? "OK" : "FAIL";
}
@Override
public int getType() {
return (ok)? gUnitParser.OK : gUnitParser.FAIL;
}
@Override
public String getResult(gUnitTestResult testResult) {
if ( testResult.isLexerTest() ) {
if ( testResult.isSuccess() ) return "OK";
else {
hasErrorMsg = true; // return error message for boolean test of lexer
return testResult.getError();
}
}
return (testResult.isSuccess())? "OK" : "FAIL";
}
@Override
public String getExpected() {
return (ok)? "OK" : "FAIL";
}
}
antlr-3.2/gunit/CHANGES.txt 0000644 0001750 0001750 00000004070 11256465172 015405 0 ustar twerner twerner gUnit 1.0.5
Nov 25, 2008
Leon, Jen-Yuan Su
leonsu at mac com
CHANGES
March 21, 2009
* gUnitTestSuite.java: use lexer rule name if parser rule name is null (by Shaoting)
* add gunit/swingui package for gUnitEditor GUI
Feb 17, 2009
* added new interfaces for GUI editor
* recognizes invalid input as a FAIL case instead of throwing an exception
Steve Ebersole provided a patch for the following two fixes.
* allows setting an output directory (for JUnitCodeGen)
* allows providing a classloader (for both JUnitCodeGen and gUnitExecutor)
Nov 25, 2008
* fixed external test file path issue. if an input test file is not found under the current dir, then try to look for it under the package dir also.
* fixed multiple-line input indentation issue.
* fixed bug: FileNotFoundException terminated gUnit tests due to any non-existent input test file.
* display escaped text for newline characters in the test result for comparing expected and actual string.
Nov 20, 2008
* added new functionality of testing lexical rules
* fixed bug of using PipedInput/Output Stream and changed to ByteArrayOutputStream. Jared Bunting provided a patch on this issue.
* improved jUnit translation mode and moved supporting codes into gUnitBaseTest.
Oct 31, 2008
* fixed bug of testing a tree grammar's template output
July 9, 2008
* fixed bug: program exited upon InvocationTargetException
Sumanto Biswas pointed out the issue and provided suggestions.
* Better handle on test rule's StringTemplate output
May 10, 2008
* added exit code functionality
* fixed string escaping bug for junit generator
1.0.2 - Apr 01, 2008
* fixed grammar bug: multiple-line input, AST output
* adjusted the output of test result
Mar 20, 2008
* moved test result to string template (gUnitTestResult.stg)
* added the display of line of test in the test result
Feb 19, 2008
* fixed bug of displaying test sequence and error message from ANTLR
Feb 8, 2008
* made compatible with ANTLR 3.1b1
1.0.1 - Jan 11, 2008
* Kenny MacDermid helps with code refactoring
1.0 - Aug 20, 2007
Initial early access release
antlr-3.2/gunit/pom.xml 0000644 0001750 0001750 00000012234 11256466764 015122 0 ustar twerner twerner <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.antlr</groupId>
<artifactId>gunit</artifactId>
<packaging>jar</packaging>
<name>ANTLR gUnit</name>
<!--
Inherit from the ANTLR master pom, which tells us what
version we are and allows us to inherit dependencies
and so on.
-->
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr-master</artifactId>
<version>3.2</version>
</parent>
<url>http://www.antlr.org/wiki/display/ANTLR3/gUnit+-+Grammar+Unit+Testing</url>
<!--
Define where the ANTLR gUnit jar is deployed both for
the main ANTLR repository, which syncs with the maven main
repository, and the snapshot repository, which can be
used by developers that need the latest development version of
something, but is used here to ensure that snapshot builds of the
ANTLR tool pick up the latest snapshot of the runtime and string tempalte
and whatever else it might need in the future.
-->
<distributionManagement>
<repository>
<id>antlr-repo</id>
<name>ANTLR Testing repository</name>
<url>scpexe://antlr.org/home/mavensync/antlr-repo</url>
</repository>
<snapshotRepository>
<id>antlr-snapshot</id>
<name>ANTLR Testing Snapshot Repository</name>
<url>scpexe://antlr.org/home/mavensync/antlr-snapshot</url>
</snapshotRepository>
</distributionManagement>
<!--
Inform Maven of the ANTLR snapshot repository, which it will
need to consult to get the latest snapshot build of the runtime and so on, should
this project need the latest builds from the antlr snapshots and not the pre-built
released jars from the repository.
-->
<repositories>
<!--
This is the ANTLR repository.
-->
<repository>
<id>antlr-snapshot</id>
<name>ANTLR Testing Snapshot Repository</name>
<url>http://antlr.org/antlr-snapshot</url>
<snapshots>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
<!--
Tell Maven which other artifacts we need in order to
build, run and test the ANTLR Tool. The ANTLR Tool uses earlier versions
of ANTLR at runtime (for the moment), uses the current
released version of ANTLR String template, but obviously is
reliant on the latest snapshot of the runtime, which will either be
taken from the antlr-snapshot repository, or your local .m2
repository if you built and installed that locally.
-->
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.5</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr</artifactId>
<version>${project.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>stringtemplate</artifactId>
<version>3.2.1</version>
<scope>compile</scope>
</dependency>
</dependencies>
<build>
<defaultGoal>install</defaultGoal>
<plugins>
<plugin>
<groupId>org.antlr</groupId>
<artifactId>antlr3-maven-plugin</artifactId>
<version>${project.version}</version>
<configuration></configuration>
<executions>
<execution>
<goals>
<goal>antlr</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>jsr14</target>
<sourceDirectory>src</sourceDirectory>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<configuration>
<findbugsXmlOutput>true</findbugsXmlOutput>
<findbugsXmlWithMessages>true</findbugsXmlWithMessages>
<xmlOutput>true</xmlOutput>
</configuration>
</plugin>
</plugins>
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ssh-external</artifactId>
<version>1.0-beta-2</version>
</extension>
</extensions>
</build>
</project>
antlr-3.2/antlrjar.xml 0000644 0001750 0001750 00000005401 11256465172 015004 0 ustar twerner twerner <?xml version="1.0" encoding="UTF-8"?>
<!--
This file defines what goes in to ANTLR Uber jar, which includes
all of the classes we need to run an executable jar in standalone
mode, other than junit, which has a non Free BSD license and so
we are reluctanct to include it in the jar.
-->
<assembly>
<!--
This is the suffix that will be used to name the uber jar
once it is jared up.
-->
<id>completejar</id>
<!--
Exclude the antlr-master pom from the jar - we don't need it
and it causes silly things to happen.
-->
<useProjectArtifact>false</useProjectArtifact>
<!--
The only output format we need is the executable jar file
-->
<formats>
<format>jar</format>
</formats>
<!--
Make all jars unpack at the same level and don't include
any extranous directories.
-->
<includeBaseDirectory>false</includeBaseDirectory>
<!--
Which of the modules that the master pom builds do we
wish to include in the uber jar. We are including
dependencies, so we only need to name the Tool module
and the gunit module.
-->
<moduleSets>
<moduleSet>
<includes>
<include>org.antlr:antlr</include>
</includes>
<!--
Of the binaries, such as the depencies that the
above modules need, which do we want and which do we not.
Currently we want all the dependencies in the Tool jar.
-->
<binaries>
<!--
Unpack the binary dependencies so we have a nice
uber jar that can run with java -jar and need not have
CLASSPATH configured and so on.
-->
<unpack>true</unpack>
</binaries>
</moduleSet>
<moduleSet>
<includes>
<include>org.antlr:gunit</include>
</includes>
<!--
We do not want any of the dependencies that gunit has as they
are inlcuded with the tool. In particular we want to exclude
the junit dependency.
-->
<binaries>
<includeDependencies>false</includeDependencies>
</binaries>
</moduleSet>
</moduleSets>
<!--
What do we want to include in the jar from each project
-->
<fileSets>
<fileSet>
<!--
We need the output classes and resources etc.
-->
<directory>${project.build.outputDirectory}</directory>
</fileSet>
</fileSets>
</assembly>
antlr-3.2/tool/ 0000755 0001750 0001750 00000000000 11410174110 013377 5 ustar twerner twerner antlr-3.2/tool/LICENSE.txt 0000644 0001750 0001750 00000002620 11256465226 015245 0 ustar twerner twerner [The "BSD licence"]
Copyright (c) 2003-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
antlr-3.2/tool/README.txt 0000644 0001750 0001750 00000010012 11256465226 015112 0 ustar twerner twerner ANTLR v3.1.3
March 17, 2009
Terence Parr, parrt at cs usfca edu
ANTLR project lead and supreme dictator for life
University of San Francisco
INTRODUCTION
Welcome to ANTLR v3! ANTLR (ANother Tool for Language Recognition) is
a language tool that provides a framework for constructing
recognizers, interpreters, compilers, and translators from grammatical
descriptions containing actions in a variety of target
languages. ANTLR provides excellent support for tree construction,
tree walking, translation, error recovery, and error reporting. I've
been working on parser generators for 20 years and on this particular
version of ANTLR for 5 years.
You should use v3 in conjunction with ANTLRWorks:
http://www.antlr.org/works/index.html
and gUnit (grammar unit testing tool included in distribution):
http://www.antlr.org/wiki/display/ANTLR3/gUnit+-+Grammar+Unit+Testing
The book will also help you a great deal (printed May 15, 2007); you
can also buy the PDF:
http://www.pragmaticprogrammer.com/titles/tpantlr/index.html
(New book coming out in beta Summer 2009: "Language Design Patterns")
See the getting started document:
http://www.antlr.org/wiki/display/ANTLR3/FAQ+-+Getting+Started
You also have the examples plus the source to guide you.
See the wiki FAQ:
http://www.antlr.org/wiki/display/ANTLR3/ANTLR+v3+FAQ
and general doc root:
http://www.antlr.org/wiki/display/ANTLR3/ANTLR+3+Wiki+Home
Please help add/update FAQ entries.
If all else fails, you can buy support or ask the antlr-interest list:
http://www.antlr.org/support.html
Per the license in LICENSE.txt, this software is not guaranteed to
work and might even destroy all life on this planet:
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
----------------------------------------------------------------------
EXAMPLES
ANTLR v3 sample grammars:
http://www.antlr.org/download/examples-v3.tar.gz
Also check out Mantra Programming Language for a prototype (work in
progress) using v3:
http://www.linguamantra.org/
----------------------------------------------------------------------
What is ANTLR?
ANTLR stands for (AN)other (T)ool for (L)anguage (R)ecognition and was
originally known as PCCTS. ANTLR is a language tool that provides a
framework for constructing recognizers, compilers, and translators
from grammatical descriptions containing actions. Target language list:
http://www.antlr.org/wiki/display/ANTLR3/Code+Generation+Targets
----------------------------------------------------------------------
How is ANTLR v3 different than ANTLR v2?
See "What is the difference between ANTLR v2 and v3?"
http://www.antlr.org/wiki/pages/viewpage.action?pageId=719
See migration guide:
http://www.antlr.org/wiki/display/ANTLR3/Migrating+from+ANTLR+2+to+ANTLR+3
----------------------------------------------------------------------
How do I install this damn thing?
Just untar and you'll get:
antlr-3.1.3/README.txt (this file)
antlr-3.1.3/LICENSE.txt
antlr-3.1.3/src/main/java/org/antlr/...
antlr-3.1.3/lib/stringtemplate-3.2.jar
antlr-3.1.3/lib/antlr-2.7.7.jar (ANTLR v3 currently written in v2)
antlr-3.1.3/lib/antlr-3.1.3.jar (all jars combined, runtime and tools)
antlr-3.1.3/lib/antlr-runtime-3.1.3.jar (only what is needed to use ANTLR parsers)
Then you need to add all the jars in lib to your CLASSPATH.
Please see the FAQ
http://www.antlr.org/wiki/display/ANTLR3/ANTLR+v3+FAQ
antlr-3.2/tool/antlr.config 0000644 0001750 0001750 00000000000 11256465226 015717 0 ustar twerner twerner antlr-3.2/tool/src/ 0000755 0001750 0001750 00000000000 11410174110 014166 5 ustar twerner twerner antlr-3.2/tool/src/test/ 0000755 0001750 0001750 00000000000 11410174107 015153 5 ustar twerner twerner antlr-3.2/tool/src/test/java/ 0000755 0001750 0001750 00000000000 11410174107 016074 5 ustar twerner twerner antlr-3.2/tool/src/test/java/org/ 0000755 0001750 0001750 00000000000 11410174107 016663 5 ustar twerner twerner antlr-3.2/tool/src/test/java/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 020003 5 ustar twerner twerner antlr-3.2/tool/src/test/java/org/antlr/test/ 0000755 0001750 0001750 00000000000 11410174107 020762 5 ustar twerner twerner antlr-3.2/tool/src/test/java/org/antlr/test/TestTreeIterator.java 0000644 0001750 0001750 00000012215 11256465236 025115 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Test;
import static org.junit.Assert.*;
import org.antlr.runtime.tree.*;
public class TestTreeIterator {
static final String[] tokens = new String[] {
"<invalid>", "<EOR>", "<DOWN>", "<UP>", "A", "B", "C", "D", "E", "F", "G"
};
@Test public void testNode() {
TreeAdaptor adaptor = new CommonTreeAdaptor();
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("A");
TreeIterator it = new TreeIterator(t);
StringBuffer buf = toString(it);
String expecting = "A EOF";
String found = buf.toString();
assertEquals(expecting, found);
}
@Test public void testFlatAB() {
TreeAdaptor adaptor = new CommonTreeAdaptor();
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(nil A B)");
TreeIterator it = new TreeIterator(t);
StringBuffer buf = toString(it);
String expecting = "nil DOWN A B UP EOF";
String found = buf.toString();
assertEquals(expecting, found);
}
@Test public void testAB() {
TreeAdaptor adaptor = new CommonTreeAdaptor();
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B)");
TreeIterator it = new TreeIterator(t);
StringBuffer buf = toString(it);
String expecting = "A DOWN B UP EOF";
String found = buf.toString();
assertEquals(expecting, found);
}
@Test public void testABC() {
TreeAdaptor adaptor = new CommonTreeAdaptor();
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C)");
TreeIterator it = new TreeIterator(t);
StringBuffer buf = toString(it);
String expecting = "A DOWN B C UP EOF";
String found = buf.toString();
assertEquals(expecting, found);
}
@Test public void testVerticalList() {
TreeAdaptor adaptor = new CommonTreeAdaptor();
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A (B C))");
TreeIterator it = new TreeIterator(t);
StringBuffer buf = toString(it);
String expecting = "A DOWN B DOWN C UP UP EOF";
String found = buf.toString();
assertEquals(expecting, found);
}
@Test public void testComplex() {
TreeAdaptor adaptor = new CommonTreeAdaptor();
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A (B (C D E) F) G)");
TreeIterator it = new TreeIterator(t);
StringBuffer buf = toString(it);
String expecting = "A DOWN B DOWN C DOWN D E UP F UP G UP EOF";
String found = buf.toString();
assertEquals(expecting, found);
}
@Test public void testReset() {
TreeAdaptor adaptor = new CommonTreeAdaptor();
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A (B (C D E) F) G)");
TreeIterator it = new TreeIterator(t);
StringBuffer buf = toString(it);
String expecting = "A DOWN B DOWN C DOWN D E UP F UP G UP EOF";
String found = buf.toString();
assertEquals(expecting, found);
it.reset();
buf = toString(it);
expecting = "A DOWN B DOWN C DOWN D E UP F UP G UP EOF";
found = buf.toString();
assertEquals(expecting, found);
}
protected static StringBuffer toString(TreeIterator it) {
StringBuffer buf = new StringBuffer();
while ( it.hasNext() ) {
CommonTree n = (CommonTree)it.next();
buf.append(n);
if ( it.hasNext() ) buf.append(" ");
}
return buf;
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/DebugTestAutoAST.java 0000644 0001750 0001750 00000002772 11256465234 024740 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
public class DebugTestAutoAST extends TestAutoAST {
public DebugTestAutoAST() {debug=true;}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTreeGrammarRewriteAST.java 0000644 0001750 0001750 00000110437 11256465236 026631 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.tool.ErrorManager;
import org.antlr.tool.Grammar;
import org.antlr.tool.GrammarSemanticsMessage;
import org.antlr.tool.GrammarSyntaxMessage;
import org.antlr.Tool;
import org.antlr.codegen.CodeGenerator;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
/** Tree rewrites in tree parsers are basically identical to rewrites
* in a normal grammar except that the atomic element is a node not
* a Token. Tests here ensure duplication of nodes occurs properly
* and basic functionality.
*/
public class TestTreeGrammarRewriteAST extends BaseTest {
protected boolean debug = false;
@Test public void testFlatList() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID INT -> INT ID\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("34 abc\n", found);
}
@Test public void testSimpleTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(ID INT) -> ^(INT ID)\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("(34 abc)\n", found);
}
@Test public void testNonImaginaryWithCtor() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : INT -> INT[\"99\"]\n" + // make new INT node
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "34");
assertEquals("99\n", found);
}
@Test public void testCombinedRewriteAndAuto() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT) | INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(ID INT) -> ^(INT ID) | INT\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("(34 abc)\n", found);
found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "34");
assertEquals("34\n", found);
}
@Test public void testAvoidDup() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID -> ^(ID ID)\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("(abc abc)\n", found);
}
@Test public void testLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID+ INT+ -> (^(ID INT))+ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : (^(ID INT))+ -> INT+ ID+\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a b c 3 4 5");
assertEquals("3 4 5 a b c\n", found);
}
@Test public void testAutoDup() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID \n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("abc\n", found);
}
@Test public void testAutoDupRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : b c ;\n" +
"b : ID ;\n" +
"c : INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 1");
assertEquals("a 1\n", found);
}
@Test public void testAutoWildcard() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID . \n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("abc 34\n", found);
}
@Test public void testNoWildcardAsRootError() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST;}\n" +
"a : ^(. INT) \n" +
" ;\n";
Grammar g = new Grammar(treeGrammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
assertEquals("unexpected errors: "+equeue, 1, equeue.errors.size());
int expectedMsgID = ErrorManager.MSG_WILDCARD_AS_ROOT;
Object expectedArg = null;
antlr.RecognitionException expectedExc = null;
GrammarSyntaxMessage expectedMessage =
new GrammarSyntaxMessage(expectedMsgID, g, null, expectedArg, expectedExc);
checkError(equeue, expectedMessage);
}
@Test public void testAutoWildcard2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(ID .) \n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("(abc 34)\n", found);
}
@Test public void testAutoWildcardWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID c=. \n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("abc 34\n", found);
}
@Test public void testAutoWildcardWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID c+=. \n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("abc 34\n", found);
}
@Test public void testAutoDupMultiple() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ID INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID ID INT\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a b 3");
assertEquals("a b 3\n", found);
}
@Test public void testAutoDupTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(ID INT)\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 3");
assertEquals("(a 3)\n", found);
}
@Test public void testAutoDupTree2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT INT -> ^(ID INT INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(ID b b)\n" +
" ;\n" +
"b : INT ;";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 3 4");
assertEquals("(a 3 4)\n", found);
}
@Test public void testAutoDupTreeWithLabels() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(x=ID y=INT)\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 3");
assertEquals("(a 3)\n", found);
}
@Test public void testAutoDupTreeWithListLabels() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(x+=ID y+=INT)\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 3");
assertEquals("(a 3)\n", found);
}
@Test public void testAutoDupTreeWithRuleRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(b INT) ;\n" +
"b : ID ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 3");
assertEquals("(a 3)\n", found);
}
@Test public void testAutoDupTreeWithRuleRootAndLabels() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(x=b INT) ;\n" +
"b : ID ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 3");
assertEquals("(a 3)\n", found);
}
@Test public void testAutoDupTreeWithRuleRootAndListLabels() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(x+=b y+=c) ;\n" +
"b : ID ;\n" +
"c : INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 3");
assertEquals("(a 3)\n", found);
}
@Test public void testAutoDupNestedTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=ID y=ID INT -> ^($x ^($y INT));\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(ID ^(ID INT))\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a b 3");
assertEquals("(a (b 3))\n", found);
}
@Test public void testAutoDupTreeWithSubruleInside() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {OP;}\n" +
"a : (x=ID|x=INT) -> ^(OP $x) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(OP (b|c)) ;\n" +
"b : ID ;\n" +
"c : INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a");
assertEquals("(OP a)\n", found);
}
@Test public void testDelete() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID -> \n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("", found);
}
@Test public void testSetMatchNoRewrite() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : b INT\n" +
" ;\n" +
"b : ID | INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("abc 34\n", found);
}
@Test public void testSetOptionalMatchNoRewrite() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : (ID|INT)? INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("abc 34\n", found);
}
@Test public void testSetMatchNoRewriteLevel2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=ID INT -> ^($x INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^(ID (ID | INT) ) ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("(abc 34)\n", found);
}
@Test public void testSetMatchNoRewriteLevel2Root() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=ID INT -> ^($x INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ^((ID | INT) INT) ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("(abc 34)\n", found);
}
// REWRITE MODE
@Test public void testRewriteModeCombinedRewriteAndAuto() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT) | INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"a : ^(ID INT) -> ^(ID[\"ick\"] INT)\n" +
" | INT\n" + // leaves it alone, returning $a.start
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("(ick 34)\n", found);
found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "34");
assertEquals("34\n", found);
}
@Test public void testRewriteModeFlatTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ID INT | INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ID a ;\n" +
"a : INT -> INT[\"1\"]\n"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("abc 1\n", found);
}
@Test public void testRewriteModeChainRuleFlatTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ID INT | INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : a ;\n" +
"a : b ;\n" +
"b : ID INT -> INT ID\n"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("34 abc\n", found);
}
@Test public void testRewriteModeChainRuleTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : a ;\n" +
"a : b ;\n" + // a.tree must become b.tree
"b : ^(ID INT) -> INT\n"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("34\n", found);
}
@Test public void testRewriteModeChainRuleTree2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"tokens { X; }\n" +
"s : a* b ;\n" + // only b contributes to tree, but it's after a*; s.tree = b.tree
"a : X ;\n" +
"b : ^(ID INT) -> INT\n"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("34\n", found);
}
@Test public void testRewriteModeChainRuleTree3() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'boo' ID INT -> 'boo' ^(ID INT) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"tokens { X; }\n" +
"s : 'boo' a* b ;\n" + // don't reset s.tree to b.tree due to 'boo'
"a : X ;\n" +
"b : ^(ID INT) -> INT\n"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "boo abc 34");
assertEquals("boo 34\n", found);
}
@Test public void testRewriteModeChainRuleTree4() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'boo' ID INT -> ^('boo' ^(ID INT)) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"tokens { X; }\n" +
"s : ^('boo' a* b) ;\n" + // don't reset s.tree to b.tree due to 'boo'
"a : X ;\n" +
"b : ^(ID INT) -> INT\n"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "boo abc 34");
assertEquals("(boo 34)\n", found);
}
@Test public void testRewriteModeChainRuleTree5() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'boo' ID INT -> ^('boo' ^(ID INT)) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"tokens { X; }\n" +
"s : ^(a b) ;\n" + // s.tree is a.tree
"a : 'boo' ;\n" +
"b : ^(ID INT) -> INT\n"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "boo abc 34");
assertEquals("(boo 34)\n", found);
}
@Test public void testRewriteOfRuleRef() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ID INT | INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : a -> a ;\n" +
"a : ID INT -> ID INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("abc 34\n", found);
}
@Test public void testRewriteOfRuleRefRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT INT -> ^(INT ^(ID INT));\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(a ^(ID INT)) -> a ;\n" +
"a : INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 12 34");
// emits whole tree when you ref the root since I can't know whether
// you want the children or not. You might be returning a whole new
// tree. Hmm...still seems weird. oh well.
assertEquals("(12 (abc 34))\n", found);
}
@Test public void testRewriteOfRuleRefRootLabeled() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT INT -> ^(INT ^(ID INT));\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(label=a ^(ID INT)) -> a ;\n" +
"a : INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 12 34");
// emits whole tree when you ref the root since I can't know whether
// you want the children or not. You might be returning a whole new
// tree. Hmm...still seems weird. oh well.
assertEquals("(12 (abc 34))\n", found);
}
@Ignore
@Test public void testRewriteOfRuleRefRootListLabeled() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT INT -> ^(INT ^(ID INT));\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(label+=a ^(ID INT)) -> a ;\n" +
"a : INT ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 12 34");
// emits whole tree when you ref the root since I can't know whether
// you want the children or not. You might be returning a whole new
// tree. Hmm...still seems weird. oh well.
assertEquals("(12 (abc 34))\n", found);
}
@Test public void testRewriteOfRuleRefChild() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID ^(INT INT));\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(ID a) -> a ;\n" +
"a : ^(INT INT) ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("(34 34)\n", found);
}
@Test public void testRewriteOfRuleRefLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID ^(INT INT));\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(ID label=a) -> a ;\n" +
"a : ^(INT INT) ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("(34 34)\n", found);
}
@Test public void testRewriteOfRuleRefListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID ^(INT INT));\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(ID label+=a) -> a ;\n" +
"a : ^(INT INT) ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("(34 34)\n", found);
}
@Test public void testRewriteModeWithPredicatedRewrites() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID[\"root\"] ^(ID INT)) | INT -> ^(ID[\"root\"] INT) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(ID a) {System.out.println(\"altered tree=\"+$s.start.toStringTree());};\n" +
"a : ^(ID INT) -> {true}? ^(ID[\"ick\"] INT)\n" +
" -> INT\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("altered tree=(root (ick 34))\n" +
"(root (ick 34))\n", found);
}
@Test public void testWildcardSingleNode() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID[\"root\"] INT);\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"s : ^(ID c=.) -> $c\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("34\n", found);
}
@Test public void testWildcardUnlabeledSingleNode() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"s : ^(ID .) -> ID\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("abc\n", found);
}
@Test public void testWildcardGrabsSubtree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID x=INT y=INT z=INT -> ^(ID[\"root\"] ^($x $y $z));\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"s : ^(ID c=.) -> $c\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 1 2 3");
assertEquals("(1 2 3)\n", found);
}
@Test public void testWildcardGrabsSubtree2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID x=INT y=INT z=INT -> ID ^($x $y $z);\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"s : ID c=. -> $c\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 1 2 3");
assertEquals("(1 2 3)\n", found);
}
@Test public void testWildcardListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT INT INT ;\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"s : (c+=.)+ -> $c+\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "1 2 3");
assertEquals("1 2 3\n", found);
}
@Test public void testWildcardListLabel2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST; ASTLabelType=CommonTree;}\n" +
"a : x=INT y=INT z=INT -> ^($x ^($y $z) ^($y $z));\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T; rewrite=true;}\n" +
"s : ^(INT (c+=.)+) -> $c+\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "1 2 3");
assertEquals("(2 3) (2 3)\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/BaseTest.java 0000644 0001750 0001750 00000074447 11256465234 023373 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.Tool;
import org.antlr.analysis.Label;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.tool.ErrorManager;
import org.antlr.tool.Message;
import org.antlr.tool.GrammarSemanticsMessage;
import org.antlr.tool.ANTLRErrorListener;
import org.junit.Before;
import org.junit.After;
import org.junit.Assert;
import static org.junit.Assert.*;
import java.io.*;
import java.util.*;
public abstract class BaseTest {
public static final String jikes = null;//"/usr/bin/jikes";
public static final String pathSep = System.getProperty("path.separator");
/**
* When runnning from Maven, the junit tests are run via the surefire plugin. It sets the
* classpath for the test environment into the following property. We need to pick this up
* for the junit tests that are going to generate and try to run code.
*/
public static final String SUREFIRE_CLASSPATH = System.getProperty("surefire.test.class.path", "");
/**
* Build up the full classpath we need, including the surefire path (if present)
*/
public static final String CLASSPATH = System.getProperty("java.class.path") + (SUREFIRE_CLASSPATH.equals("") ? "" : pathSep + SUREFIRE_CLASSPATH);
public String tmpdir = null;
/** reset during setUp and set to true if we find a problem */
protected boolean lastTestFailed = false;
/** If error during parser execution, store stderr here; can't return
* stdout and stderr. This doesn't trap errors from running antlr.
*/
protected String stderrDuringParse;
@Before
public void setUp() throws Exception {
lastTestFailed = false; // hope for the best, but set to true in asserts that fail
// new output dir for each test
tmpdir = new File(System.getProperty("java.io.tmpdir"), "antlr-"+getClass().getName()+"-"+System.currentTimeMillis()).getAbsolutePath();
ErrorManager.resetErrorState();
// force reset of static caches
new StringTemplateGroup("") {
{
StringTemplateGroup.nameToGroupMap = Collections.synchronizedMap(new HashMap());
StringTemplateGroup.nameToInterfaceMap = Collections.synchronizedMap(new HashMap());
}
};
StringTemplate.resetTemplateCounter();
StringTemplate.defaultGroup = new StringTemplateGroup("defaultGroup", ".");
}
@After
public void tearDown() throws Exception {
// remove tmpdir if no error.
if ( !lastTestFailed ) eraseTempDir();
}
protected Tool newTool(String[] args) {
Tool tool = new Tool(args);
tool.setOutputDirectory(tmpdir);
return tool;
}
protected Tool newTool() {
Tool tool = new Tool();
tool.setOutputDirectory(tmpdir);
return tool;
}
protected boolean compile(String fileName) {
String compiler = "javac";
String classpathOption = "-classpath";
if (jikes!=null) {
compiler = jikes;
classpathOption = "-bootclasspath";
}
String[] args = new String[] {
compiler, "-d", tmpdir,
classpathOption, tmpdir+pathSep+CLASSPATH,
tmpdir+"/"+fileName
};
String cmdLine = compiler+" -d "+tmpdir+" "+classpathOption+" "+tmpdir+pathSep+CLASSPATH+" "+fileName;
//System.out.println("compile: "+cmdLine);
File outputDir = new File(tmpdir);
try {
Process process =
Runtime.getRuntime().exec(args, null, outputDir);
StreamVacuum stdout = new StreamVacuum(process.getInputStream());
StreamVacuum stderr = new StreamVacuum(process.getErrorStream());
stdout.start();
stderr.start();
process.waitFor();
stdout.join();
stderr.join();
if ( stdout.toString().length()>0 ) {
System.err.println("compile stdout from: "+cmdLine);
System.err.println(stdout);
}
if ( stderr.toString().length()>0 ) {
System.err.println("compile stderr from: "+cmdLine);
System.err.println(stderr);
}
int ret = process.exitValue();
return ret==0;
}
catch (Exception e) {
System.err.println("can't exec compilation");
e.printStackTrace(System.err);
return false;
}
}
/** Return true if all is ok, no errors */
protected boolean antlr(String fileName, String grammarFileName, String grammarStr, boolean debug) {
boolean allIsWell = true;
mkdir(tmpdir);
writeFile(tmpdir, fileName, grammarStr);
try {
final List options = new ArrayList();
if ( debug ) {
options.add("-debug");
}
options.add("-o");
options.add(tmpdir);
options.add("-lib");
options.add(tmpdir);
options.add(new File(tmpdir,grammarFileName).toString());
final String[] optionsA = new String[options.size()];
options.toArray(optionsA);
/*
final ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
*/
Tool antlr = newTool(optionsA);
antlr.process();
ANTLRErrorListener listener = ErrorManager.getErrorListener();
if ( listener instanceof ErrorQueue ) {
ErrorQueue equeue = (ErrorQueue)listener;
if ( equeue.errors.size()>0 ) {
allIsWell = false;
System.err.println("antlr reports errors from "+options);
for (int i = 0; i < equeue.errors.size(); i++) {
Message msg = (Message) equeue.errors.get(i);
System.err.println(msg);
}
System.out.println("!!!\ngrammar:");
System.out.println(grammarStr);
System.out.println("###");
}
}
}
catch (Exception e) {
allIsWell = false;
System.err.println("problems building grammar: "+e);
e.printStackTrace(System.err);
}
return allIsWell;
}
protected String execLexer(String grammarFileName,
String grammarStr,
String lexerName,
String input,
boolean debug)
{
rawGenerateAndBuildRecognizer(grammarFileName,
grammarStr,
null,
lexerName,
debug);
writeFile(tmpdir, "input", input);
return rawExecRecognizer(null,
null,
lexerName,
null,
null,
false,
false,
false,
debug);
}
protected String execParser(String grammarFileName,
String grammarStr,
String parserName,
String lexerName,
String startRuleName,
String input, boolean debug)
{
rawGenerateAndBuildRecognizer(grammarFileName,
grammarStr,
parserName,
lexerName,
debug);
writeFile(tmpdir, "input", input);
boolean parserBuildsTrees =
grammarStr.indexOf("output=AST")>=0 ||
grammarStr.indexOf("output = AST")>=0;
boolean parserBuildsTemplate =
grammarStr.indexOf("output=template")>=0 ||
grammarStr.indexOf("output = template")>=0;
return rawExecRecognizer(parserName,
null,
lexerName,
startRuleName,
null,
parserBuildsTrees,
parserBuildsTemplate,
false,
debug);
}
protected String execTreeParser(String parserGrammarFileName,
String parserGrammarStr,
String parserName,
String treeParserGrammarFileName,
String treeParserGrammarStr,
String treeParserName,
String lexerName,
String parserStartRuleName,
String treeParserStartRuleName,
String input)
{
return execTreeParser(parserGrammarFileName,
parserGrammarStr,
parserName,
treeParserGrammarFileName,
treeParserGrammarStr,
treeParserName,
lexerName,
parserStartRuleName,
treeParserStartRuleName,
input,
false);
}
protected String execTreeParser(String parserGrammarFileName,
String parserGrammarStr,
String parserName,
String treeParserGrammarFileName,
String treeParserGrammarStr,
String treeParserName,
String lexerName,
String parserStartRuleName,
String treeParserStartRuleName,
String input,
boolean debug)
{
// build the parser
rawGenerateAndBuildRecognizer(parserGrammarFileName,
parserGrammarStr,
parserName,
lexerName,
debug);
// build the tree parser
rawGenerateAndBuildRecognizer(treeParserGrammarFileName,
treeParserGrammarStr,
treeParserName,
lexerName,
debug);
writeFile(tmpdir, "input", input);
boolean parserBuildsTrees =
parserGrammarStr.indexOf("output=AST")>=0 ||
parserGrammarStr.indexOf("output = AST")>=0;
boolean treeParserBuildsTrees =
treeParserGrammarStr.indexOf("output=AST")>=0 ||
treeParserGrammarStr.indexOf("output = AST")>=0;
boolean parserBuildsTemplate =
parserGrammarStr.indexOf("output=template")>=0 ||
parserGrammarStr.indexOf("output = template")>=0;
return rawExecRecognizer(parserName,
treeParserName,
lexerName,
parserStartRuleName,
treeParserStartRuleName,
parserBuildsTrees,
parserBuildsTemplate,
treeParserBuildsTrees,
debug);
}
/** Return true if all is well */
protected boolean rawGenerateAndBuildRecognizer(String grammarFileName,
String grammarStr,
String parserName,
String lexerName,
boolean debug)
{
boolean allIsWell =
antlr(grammarFileName, grammarFileName, grammarStr, debug);
if ( lexerName!=null ) {
boolean ok;
if ( parserName!=null ) {
ok = compile(parserName+".java");
if ( !ok ) { allIsWell = false; }
}
ok = compile(lexerName+".java");
if ( !ok ) { allIsWell = false; }
}
else {
boolean ok = compile(parserName+".java");
if ( !ok ) { allIsWell = false; }
}
return allIsWell;
}
protected String rawExecRecognizer(String parserName,
String treeParserName,
String lexerName,
String parserStartRuleName,
String treeParserStartRuleName,
boolean parserBuildsTrees,
boolean parserBuildsTemplate,
boolean treeParserBuildsTrees,
boolean debug)
{
this.stderrDuringParse = null;
if ( treeParserBuildsTrees && parserBuildsTrees ) {
writeTreeAndTreeTestFile(parserName,
treeParserName,
lexerName,
parserStartRuleName,
treeParserStartRuleName,
debug);
}
else if ( parserBuildsTrees ) {
writeTreeTestFile(parserName,
treeParserName,
lexerName,
parserStartRuleName,
treeParserStartRuleName,
debug);
}
else if ( parserBuildsTemplate ) {
writeTemplateTestFile(parserName,
lexerName,
parserStartRuleName,
debug);
}
else if ( parserName==null ) {
writeLexerTestFile(lexerName, debug);
}
else {
writeTestFile(parserName,
lexerName,
parserStartRuleName,
debug);
}
compile("Test.java");
try {
String[] args = new String[] {
"java", "-classpath", tmpdir+pathSep+CLASSPATH,
"Test", new File(tmpdir, "input").getAbsolutePath()
};
//String cmdLine = "java -classpath "+CLASSPATH+pathSep+tmpdir+" Test " + new File(tmpdir, "input").getAbsolutePath();
//System.out.println("execParser: "+cmdLine);
Process process =
Runtime.getRuntime().exec(args, null, new File(tmpdir));
StreamVacuum stdoutVacuum = new StreamVacuum(process.getInputStream());
StreamVacuum stderrVacuum = new StreamVacuum(process.getErrorStream());
stdoutVacuum.start();
stderrVacuum.start();
process.waitFor();
stdoutVacuum.join();
stderrVacuum.join();
String output = null;
output = stdoutVacuum.toString();
if ( stderrVacuum.toString().length()>0 ) {
this.stderrDuringParse = stderrVacuum.toString();
//System.err.println("exec stderrVacuum: "+ stderrVacuum);
}
return output;
}
catch (Exception e) {
System.err.println("can't exec recognizer");
e.printStackTrace(System.err);
}
return null;
}
protected void checkGrammarSemanticsError(ErrorQueue equeue,
GrammarSemanticsMessage expectedMessage)
throws Exception
{
/*
System.out.println(equeue.infos);
System.out.println(equeue.warnings);
System.out.println(equeue.errors);
assertTrue("number of errors mismatch", n, equeue.errors.size());
*/
Message foundMsg = null;
for (int i = 0; i < equeue.errors.size(); i++) {
Message m = (Message)equeue.errors.get(i);
if (m.msgID==expectedMessage.msgID ) {
foundMsg = m;
}
}
assertNotNull("no error; "+expectedMessage.msgID+" expected", foundMsg);
assertTrue("error is not a GrammarSemanticsMessage",
foundMsg instanceof GrammarSemanticsMessage);
assertEquals(expectedMessage.arg, foundMsg.arg);
if ( equeue.size()!=1 ) {
System.err.println(equeue);
}
}
protected void checkGrammarSemanticsWarning(ErrorQueue equeue,
GrammarSemanticsMessage expectedMessage)
throws Exception
{
Message foundMsg = null;
for (int i = 0; i < equeue.warnings.size(); i++) {
Message m = (Message)equeue.warnings.get(i);
if (m.msgID==expectedMessage.msgID ) {
foundMsg = m;
}
}
assertNotNull("no error; "+expectedMessage.msgID+" expected", foundMsg);
assertTrue("error is not a GrammarSemanticsMessage",
foundMsg instanceof GrammarSemanticsMessage);
assertEquals(expectedMessage.arg, foundMsg.arg);
}
protected void checkError(ErrorQueue equeue,
Message expectedMessage)
throws Exception
{
//System.out.println("errors="+equeue);
Message foundMsg = null;
for (int i = 0; i < equeue.errors.size(); i++) {
Message m = (Message)equeue.errors.get(i);
if (m.msgID==expectedMessage.msgID ) {
foundMsg = m;
}
}
assertTrue("no error; "+expectedMessage.msgID+" expected", equeue.errors.size()>0);
assertTrue("too many errors; "+equeue.errors, equeue.errors.size()<=1);
assertNotNull("couldn't find expected error: "+expectedMessage.msgID, foundMsg);
/*
assertTrue("error is not a GrammarSemanticsMessage",
foundMsg instanceof GrammarSemanticsMessage);
*/
assertEquals(expectedMessage.arg, foundMsg.arg);
assertEquals(expectedMessage.arg2, foundMsg.arg2);
ErrorManager.resetErrorState(); // wack errors for next test
}
public static class StreamVacuum implements Runnable {
StringBuffer buf = new StringBuffer();
BufferedReader in;
Thread sucker;
public StreamVacuum(InputStream in) {
this.in = new BufferedReader( new InputStreamReader(in) );
}
public void start() {
sucker = new Thread(this);
sucker.start();
}
public void run() {
try {
String line = in.readLine();
while (line!=null) {
buf.append(line);
buf.append('\n');
line = in.readLine();
}
}
catch (IOException ioe) {
System.err.println("can't read output from process");
}
}
/** wait for the thread to finish */
public void join() throws InterruptedException {
sucker.join();
}
public String toString() {
return buf.toString();
}
}
protected void writeFile(String dir, String fileName, String content) {
try {
File f = new File(dir, fileName);
FileWriter w = new FileWriter(f);
BufferedWriter bw = new BufferedWriter(w);
bw.write(content);
bw.close();
w.close();
}
catch (IOException ioe) {
System.err.println("can't write file");
ioe.printStackTrace(System.err);
}
}
protected void mkdir(String dir) {
File f = new File(dir);
f.mkdirs();
}
protected void writeTestFile(String parserName,
String lexerName,
String parserStartRuleName,
boolean debug)
{
StringTemplate outputFileST = new StringTemplate(
"import org.antlr.runtime.*;\n" +
"import org.antlr.runtime.tree.*;\n" +
"import org.antlr.runtime.debug.*;\n" +
"\n" +
"class Profiler2 extends Profiler {\n" +
" public void terminate() { ; }\n" +
"}\n"+
"public class Test {\n" +
" public static void main(String[] args) throws Exception {\n" +
" CharStream input = new ANTLRFileStream(args[0]);\n" +
" $lexerName$ lex = new $lexerName$(input);\n" +
" CommonTokenStream tokens = new CommonTokenStream(lex);\n" +
" $createParser$\n"+
" parser.$parserStartRuleName$();\n" +
" }\n" +
"}"
);
StringTemplate createParserST =
new StringTemplate(
" Profiler2 profiler = new Profiler2();\n"+
" $parserName$ parser = new $parserName$(tokens,profiler);\n" +
" profiler.setParser(parser);\n");
if ( !debug ) {
createParserST =
new StringTemplate(
" $parserName$ parser = new $parserName$(tokens);\n");
}
outputFileST.setAttribute("createParser", createParserST);
outputFileST.setAttribute("parserName", parserName);
outputFileST.setAttribute("lexerName", lexerName);
outputFileST.setAttribute("parserStartRuleName", parserStartRuleName);
writeFile(tmpdir, "Test.java", outputFileST.toString());
}
protected void writeLexerTestFile(String lexerName, boolean debug) {
StringTemplate outputFileST = new StringTemplate(
"import org.antlr.runtime.*;\n" +
"import org.antlr.runtime.tree.*;\n" +
"import org.antlr.runtime.debug.*;\n" +
"\n" +
"class Profiler2 extends Profiler {\n" +
" public void terminate() { ; }\n" +
"}\n"+
"public class Test {\n" +
" public static void main(String[] args) throws Exception {\n" +
" CharStream input = new ANTLRFileStream(args[0]);\n" +
" $lexerName$ lex = new $lexerName$(input);\n" +
" CommonTokenStream tokens = new CommonTokenStream(lex);\n" +
" System.out.println(tokens);\n" +
" }\n" +
"}"
);
outputFileST.setAttribute("lexerName", lexerName);
writeFile(tmpdir, "Test.java", outputFileST.toString());
}
protected void writeTreeTestFile(String parserName,
String treeParserName,
String lexerName,
String parserStartRuleName,
String treeParserStartRuleName,
boolean debug)
{
StringTemplate outputFileST = new StringTemplate(
"import org.antlr.runtime.*;\n" +
"import org.antlr.runtime.tree.*;\n" +
"import org.antlr.runtime.debug.*;\n" +
"\n" +
"class Profiler2 extends Profiler {\n" +
" public void terminate() { ; }\n" +
"}\n"+
"public class Test {\n" +
" public static void main(String[] args) throws Exception {\n" +
" CharStream input = new ANTLRFileStream(args[0]);\n" +
" $lexerName$ lex = new $lexerName$(input);\n" +
" TokenRewriteStream tokens = new TokenRewriteStream(lex);\n" +
" $createParser$\n"+
" $parserName$.$parserStartRuleName$_return r = parser.$parserStartRuleName$();\n" +
" $if(!treeParserStartRuleName)$\n" +
" if ( r.tree!=null ) {\n" +
" System.out.println(((Tree)r.tree).toStringTree());\n" +
" ((CommonTree)r.tree).sanityCheckParentAndChildIndexes();\n" +
" }\n" +
" $else$\n" +
" CommonTreeNodeStream nodes = new CommonTreeNodeStream((Tree)r.tree);\n" +
" nodes.setTokenStream(tokens);\n" +
" $treeParserName$ walker = new $treeParserName$(nodes);\n" +
" walker.$treeParserStartRuleName$();\n" +
" $endif$\n" +
" }\n" +
"}"
);
StringTemplate createParserST =
new StringTemplate(
" Profiler2 profiler = new Profiler2();\n"+
" $parserName$ parser = new $parserName$(tokens,profiler);\n" +
" profiler.setParser(parser);\n");
if ( !debug ) {
createParserST =
new StringTemplate(
" $parserName$ parser = new $parserName$(tokens);\n");
}
outputFileST.setAttribute("createParser", createParserST);
outputFileST.setAttribute("parserName", parserName);
outputFileST.setAttribute("treeParserName", treeParserName);
outputFileST.setAttribute("lexerName", lexerName);
outputFileST.setAttribute("parserStartRuleName", parserStartRuleName);
outputFileST.setAttribute("treeParserStartRuleName", treeParserStartRuleName);
writeFile(tmpdir, "Test.java", outputFileST.toString());
}
/** Parser creates trees and so does the tree parser */
protected void writeTreeAndTreeTestFile(String parserName,
String treeParserName,
String lexerName,
String parserStartRuleName,
String treeParserStartRuleName,
boolean debug)
{
StringTemplate outputFileST = new StringTemplate(
"import org.antlr.runtime.*;\n" +
"import org.antlr.runtime.tree.*;\n" +
"import org.antlr.runtime.debug.*;\n" +
"\n" +
"class Profiler2 extends Profiler {\n" +
" public void terminate() { ; }\n" +
"}\n"+
"public class Test {\n" +
" public static void main(String[] args) throws Exception {\n" +
" CharStream input = new ANTLRFileStream(args[0]);\n" +
" $lexerName$ lex = new $lexerName$(input);\n" +
" TokenRewriteStream tokens = new TokenRewriteStream(lex);\n" +
" $createParser$\n"+
" $parserName$.$parserStartRuleName$_return r = parser.$parserStartRuleName$();\n" +
" ((CommonTree)r.tree).sanityCheckParentAndChildIndexes();\n" +
" CommonTreeNodeStream nodes = new CommonTreeNodeStream((Tree)r.tree);\n" +
" nodes.setTokenStream(tokens);\n" +
" $treeParserName$ walker = new $treeParserName$(nodes);\n" +
" $treeParserName$.$treeParserStartRuleName$_return r2 = walker.$treeParserStartRuleName$();\n" +
" CommonTree rt = ((CommonTree)r2.tree);\n" +
" if ( rt!=null ) System.out.println(((CommonTree)r2.tree).toStringTree());\n" +
" }\n" +
"}"
);
StringTemplate createParserST =
new StringTemplate(
" Profiler2 profiler = new Profiler2();\n"+
" $parserName$ parser = new $parserName$(tokens,profiler);\n" +
" profiler.setParser(parser);\n");
if ( !debug ) {
createParserST =
new StringTemplate(
" $parserName$ parser = new $parserName$(tokens);\n");
}
outputFileST.setAttribute("createParser", createParserST);
outputFileST.setAttribute("parserName", parserName);
outputFileST.setAttribute("treeParserName", treeParserName);
outputFileST.setAttribute("lexerName", lexerName);
outputFileST.setAttribute("parserStartRuleName", parserStartRuleName);
outputFileST.setAttribute("treeParserStartRuleName", treeParserStartRuleName);
writeFile(tmpdir, "Test.java", outputFileST.toString());
}
protected void writeTemplateTestFile(String parserName,
String lexerName,
String parserStartRuleName,
boolean debug)
{
StringTemplate outputFileST = new StringTemplate(
"import org.antlr.runtime.*;\n" +
"import org.antlr.stringtemplate.*;\n" +
"import org.antlr.stringtemplate.language.*;\n" +
"import org.antlr.runtime.debug.*;\n" +
"import java.io.*;\n" +
"\n" +
"class Profiler2 extends Profiler {\n" +
" public void terminate() { ; }\n" +
"}\n"+
"public class Test {\n" +
" static String templates =\n" +
" \"group test;\"+" +
" \"foo(x,y) ::= \\\"<x> <y>\\\"\";\n"+
" static StringTemplateGroup group ="+
" new StringTemplateGroup(new StringReader(templates)," +
" AngleBracketTemplateLexer.class);"+
" public static void main(String[] args) throws Exception {\n" +
" CharStream input = new ANTLRFileStream(args[0]);\n" +
" $lexerName$ lex = new $lexerName$(input);\n" +
" CommonTokenStream tokens = new CommonTokenStream(lex);\n" +
" $createParser$\n"+
" parser.setTemplateLib(group);\n"+
" $parserName$.$parserStartRuleName$_return r = parser.$parserStartRuleName$();\n" +
" if ( r.st!=null )\n" +
" System.out.print(r.st.toString());\n" +
" else\n" +
" System.out.print(\"\");\n" +
" }\n" +
"}"
);
StringTemplate createParserST =
new StringTemplate(
" Profiler2 profiler = new Profiler2();\n"+
" $parserName$ parser = new $parserName$(tokens,profiler);\n" +
" profiler.setParser(parser);\n");
if ( !debug ) {
createParserST =
new StringTemplate(
" $parserName$ parser = new $parserName$(tokens);\n");
}
outputFileST.setAttribute("createParser", createParserST);
outputFileST.setAttribute("parserName", parserName);
outputFileST.setAttribute("lexerName", lexerName);
outputFileST.setAttribute("parserStartRuleName", parserStartRuleName);
writeFile(tmpdir, "Test.java", outputFileST.toString());
}
protected void eraseFiles(final String filesEndingWith) {
File tmpdirF = new File(tmpdir);
String[] files = tmpdirF.list();
for(int i = 0; files!=null && i < files.length; i++) {
if ( files[i].endsWith(filesEndingWith) ) {
new File(tmpdir+"/"+files[i]).delete();
}
}
}
protected void eraseFiles() {
File tmpdirF = new File(tmpdir);
String[] files = tmpdirF.list();
for(int i = 0; files!=null && i < files.length; i++) {
new File(tmpdir+"/"+files[i]).delete();
}
}
protected void eraseTempDir() {
File tmpdirF = new File(tmpdir);
if ( tmpdirF.exists() ) {
eraseFiles();
tmpdirF.delete();
}
}
public String getFirstLineOfException() {
if ( this.stderrDuringParse ==null ) {
return null;
}
String[] lines = this.stderrDuringParse.split("\n");
String prefix="Exception in thread \"main\" ";
return lines[0].substring(prefix.length(),lines[0].length());
}
public List realElements(List elements) {
List n = new ArrayList();
for (int i = Label.NUM_FAUX_LABELS+Label.MIN_TOKEN_TYPE - 1; i < elements.size(); i++) {
Object o = (Object) elements.get(i);
if ( o!=null ) {
n.add(o);
}
}
return n;
}
public List<String> realElements(Map<String, Integer> elements) {
List n = new ArrayList();
Iterator iterator = elements.keySet().iterator();
while (iterator.hasNext()) {
String tokenID = (String) iterator.next();
if ( elements.get(tokenID) >= Label.MIN_TOKEN_TYPE ) {
n.add(tokenID+"="+elements.get(tokenID));
}
}
Collections.sort(n);
return n;
}
public String sortLinesInString(String s) {
String lines[] = s.split("\n");
Arrays.sort(lines);
List<String> linesL = Arrays.asList(lines);
StringBuffer buf = new StringBuffer();
for (String l : linesL) {
buf.append(l);
buf.append('\n');
}
return buf.toString();
}
/**
* When looking at a result set that consists of a Map/HashTable
* we cannot rely on the output order, as the hashing algorithm or other aspects
* of the implementation may be different on differnt JDKs or platforms. Hence
* we take the Map, convert the keys to a List, sort them and Stringify the Map, which is a
* bit of a hack, but guarantees that we get the same order on all systems. We assume that
* the keys are strings.
*
* @param m The Map that contains keys we wish to return in sorted order
* @return A string that represents all the keys in sorted order.
*/
public String sortMapToString(Map m) {
System.out.println("Map toString looks like: " + m.toString());
// Pass in crap, and get nothing back
//
if (m == null) {
return null;
}
// Sort the keys in the Map
//
TreeMap nset = new TreeMap(m);
System.out.println("Tree map looks like: " + nset.toString());
return nset.toString();
}
// override to track errors
public void assertEquals(String msg, Object a, Object b) { try {Assert.assertEquals(msg,a,b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertEquals(Object a, Object b) { try {Assert.assertEquals(a,b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertEquals(String msg, long a, long b) { try {Assert.assertEquals(msg,a,b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertEquals(long a, long b) { try {Assert.assertEquals(a,b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertTrue(String msg, boolean b) { try {Assert.assertTrue(msg,b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertTrue(boolean b) { try {Assert.assertTrue(b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertFalse(String msg, boolean b) { try {Assert.assertFalse(msg,b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertFalse(boolean b) { try {Assert.assertFalse(b);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertNotNull(String msg, Object p) { try {Assert.assertNotNull(msg, p);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertNotNull(Object p) { try {Assert.assertNotNull(p);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertNull(String msg, Object p) { try {Assert.assertNull(msg, p);} catch (Error e) {lastTestFailed=true; throw e;} }
public void assertNull(Object p) { try {Assert.assertNull(p);} catch (Error e) {lastTestFailed=true; throw e;} }
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTemplates.java 0000644 0001750 0001750 00000027715 11256465234 024453 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.tool.*;
import org.antlr.Tool;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import org.antlr.codegen.CodeGenerator;
import org.antlr.grammar.v2.ANTLRParser;
import org.antlr.grammar.v3.ActionTranslator;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
/** Test templates in actions; %... shorthands */
public class TestTemplates extends BaseTest {
private static final String LINE_SEP = System.getProperty("line.separator");
@Test
public void testTemplateConstructor() throws Exception {
String action = "x = %foo(name={$ID.text});";
String expecting = "x = templateLib.getInstanceOf(\"foo\"," +
LINE_SEP + " new STAttrMap().put(\"name\", (ID1!=null?ID1.getText():null)));";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertNoErrors(equeue);
assertEquals(expecting, found);
}
@Test
public void testTemplateConstructorNoArgs() throws Exception {
String action = "x = %foo();";
String expecting = "x = templateLib.getInstanceOf(\"foo\");";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertNoErrors(equeue);
assertEquals(expecting, found);
}
@Test
public void testIndirectTemplateConstructor() throws Exception {
String action = "x = %({\"foo\"})(name={$ID.text});";
String expecting = "x = templateLib.getInstanceOf(\"foo\"," +
LINE_SEP + " new STAttrMap().put(\"name\", (ID1!=null?ID1.getText():null)));";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertNoErrors(equeue);
assertEquals(expecting, found);
}
@Test public void testStringConstructor() throws Exception {
String action = "x = %{$ID.text};";
String expecting = "x = new StringTemplate(templateLib,(ID1!=null?ID1.getText():null));";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertNoErrors(equeue);
assertEquals(expecting, found);
}
@Test public void testSetAttr() throws Exception {
String action = "%x.y = z;";
String expecting = "(x).setAttribute(\"y\", z);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertNoErrors(equeue);
assertEquals(expecting, found);
}
@Test public void testSetAttrOfExpr() throws Exception {
String action = "%{foo($ID.text).getST()}.y = z;";
String expecting = "(foo((ID1!=null?ID1.getText():null)).getST()).setAttribute(\"y\", z);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertNoErrors(equeue);
assertEquals(expecting, found);
}
@Test public void testSetAttrOfExprInMembers() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"@members {\n" +
"%code.instr = o;" + // must not get null ptr!
"}\n" +
"a : ID\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
assertNoErrors(equeue);
}
@Test public void testCannotHaveSpaceBeforeDot() throws Exception {
String action = "%x .y = z;";
String expecting = null;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
int expectedMsgID = ErrorManager.MSG_INVALID_TEMPLATE_ACTION;
Object expectedArg = "%x";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testCannotHaveSpaceAfterDot() throws Exception {
String action = "%x. y = z;";
String expecting = null;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {\n" +
" output=template;\n" +
"}\n" +
"\n" +
"a : ID {"+action+"}\n" +
" ;\n" +
"\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
int expectedMsgID = ErrorManager.MSG_INVALID_TEMPLATE_ACTION;
Object expectedArg = "%x.";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
protected void checkError(ErrorQueue equeue,
GrammarSemanticsMessage expectedMessage)
throws Exception
{
/*
System.out.println(equeue.infos);
System.out.println(equeue.warnings);
System.out.println(equeue.errors);
*/
Message foundMsg = null;
for (int i = 0; i < equeue.errors.size(); i++) {
Message m = (Message)equeue.errors.get(i);
if (m.msgID==expectedMessage.msgID ) {
foundMsg = m;
}
}
assertTrue("no error; "+expectedMessage.msgID+" expected", equeue.errors.size()>0);
assertTrue("too many errors; "+equeue.errors, equeue.errors.size()<=1);
assertTrue("couldn't find expected error: "+expectedMessage.msgID, foundMsg!=null);
assertTrue("error is not a GrammarSemanticsMessage",
foundMsg instanceof GrammarSemanticsMessage);
assertEquals(expectedMessage.arg, foundMsg.arg);
assertEquals(expectedMessage.arg2, foundMsg.arg2);
}
// S U P P O R T
private void assertNoErrors(ErrorQueue equeue) {
assertTrue("unexpected errors: "+equeue, equeue.errors.size()==0);
}
} antlr-3.2/tool/src/test/java/org/antlr/test/TestMessages.java 0000644 0001750 0001750 00000005756 11256465234 024265 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.Tool;
import org.antlr.codegen.CodeGenerator;
import org.antlr.grammar.v2.ANTLRParser;
import org.antlr.grammar.v3.ActionTranslator;
import org.antlr.tool.*;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestMessages extends BaseTest {
/** Public default constructor used by TestRig */
public TestMessages() {
}
@Test public void testMessageStringificationIsConsistent() throws Exception {
String action = "$other.tree = null;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"options { output = AST;}" +
"otherrule\n" +
" : 'y' ;" +
"rule\n" +
" : other=otherrule {" + action +"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
int expectedMsgID = ErrorManager.MSG_WRITE_TO_READONLY_ATTR;
Object expectedArg = "other";
Object expectedArg2 = "tree";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
String expectedMessageString = expectedMessage.toString();
assertEquals(expectedMessageString, expectedMessage.toString());
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/ErrorQueue.java 0000644 0001750 0001750 00000004420 11256465234 023737 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.tool.ANTLRErrorListener;
import org.antlr.tool.Message;
import org.antlr.tool.ToolMessage;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import java.util.List;
import java.util.LinkedList;
public class ErrorQueue implements ANTLRErrorListener {
List infos = new LinkedList();
List errors = new LinkedList();
List warnings = new LinkedList();
public void info(String msg) {
infos.add(msg);
}
public void error(Message msg) {
errors.add(msg);
}
public void warning(Message msg) {
warnings.add(msg);
}
public void error(ToolMessage msg) {
errors.add(msg);
}
public int size() {
return infos.size() + errors.size() + warnings.size();
}
public String toString() {
return "infos: "+infos+
"errors: "+errors+
"warnings: "+warnings;
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTopologicalSort.java 0000644 0001750 0001750 00000007707 11256465236 025642 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Test;
import org.antlr.misc.Graph;
import java.util.List;
/** Test topo sort in GraphNode. */
public class TestTopologicalSort extends BaseTest {
@Test
public void testFairlyLargeGraph() throws Exception {
Graph g = new Graph();
g.addEdge("C", "F");
g.addEdge("C", "G");
g.addEdge("C", "A");
g.addEdge("C", "B");
g.addEdge("A", "D");
g.addEdge("A", "E");
g.addEdge("B", "E");
g.addEdge("D", "E");
g.addEdge("D", "F");
g.addEdge("F", "H");
g.addEdge("E", "F");
String expecting = "[H, F, E, D, A, G, B, C]";
List nodes = g.sort();
String result = nodes.toString();
assertEquals(expecting, result);
}
@Test
public void testCyclicGraph() throws Exception {
Graph g = new Graph();
g.addEdge("A", "B");
g.addEdge("B", "C");
g.addEdge("C", "A");
g.addEdge("C", "D");
String expecting = "[D, C, B, A]";
List nodes = g.sort();
String result = nodes.toString();
assertEquals(expecting, result);
}
@Test
public void testRepeatedEdges() throws Exception {
Graph g = new Graph();
g.addEdge("A", "B");
g.addEdge("B", "C");
g.addEdge("A", "B"); // dup
g.addEdge("C", "D");
String expecting = "[D, C, B, A]";
List nodes = g.sort();
String result = nodes.toString();
assertEquals(expecting, result);
}
@Test
public void testSimpleTokenDependence() throws Exception {
Graph g = new Graph();
g.addEdge("Java.g", "MyJava.tokens"); // Java feeds off manual token file
g.addEdge("Java.tokens", "Java.g");
g.addEdge("Def.g", "Java.tokens"); // walkers feed off generated tokens
g.addEdge("Ref.g", "Java.tokens");
String expecting = "[MyJava.tokens, Java.g, Java.tokens, Def.g, Ref.g]";
List nodes = g.sort();
String result = nodes.toString();
assertEquals(expecting, result);
}
@Test
public void testParserLexerCombo() throws Exception {
Graph g = new Graph();
g.addEdge("JavaLexer.tokens", "JavaLexer.g");
g.addEdge("JavaParser.g", "JavaLexer.tokens");
g.addEdge("Def.g", "JavaLexer.tokens");
g.addEdge("Ref.g", "JavaLexer.tokens");
String expecting = "[JavaLexer.g, JavaLexer.tokens, JavaParser.g, Def.g, Ref.g]";
List nodes = g.sort();
String result = nodes.toString();
assertEquals(expecting, result);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestHeteroAST.java 0000644 0001750 0001750 00000043367 11256465234 024314 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2007 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
/** Test hetero trees in parsers and tree parsers */
public class TestHeteroAST extends BaseTest {
protected boolean debug = false;
// PARSERS -- AUTO AST
@Test public void testToken() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID<V> ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a<V>\n", found);
}
@Test public void testTokenWithQualifiedType() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID<TParser.V> ;\n"+ // TParser.V is qualified name
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a<V>\n", found);
}
@Test public void testTokenWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : x=ID<V> ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a<V>\n", found);
}
@Test public void testTokenWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : x+=ID<V> ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a<V>\n", found);
}
@Test public void testTokenRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID<V>^ ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a<V>\n", found);
}
@Test public void testTokenRootWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : x+=ID<V>^ ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a<V>\n", found);
}
@Test public void testString() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : 'begin'<V> ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "begin", debug);
assertEquals("begin<V>\n", found);
}
@Test public void testStringRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : 'begin'<V>^ ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "begin", debug);
assertEquals("begin<V>\n", found);
}
// PARSERS -- REWRITE AST
@Test public void testRewriteToken() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID -> ID<V> ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a<V>\n", found);
}
@Test public void testRewriteTokenWithArgs() throws Exception {
// arg to ID<V>[42,19,30] means you're constructing node not associated with ID
// so must pass in token manually
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {\n" +
"static class V extends CommonTree {\n" +
" public int x,y,z;\n"+
" public V(int ttype, int x, int y, int z) { this.x=x; this.y=y; this.z=z; token=new CommonToken(ttype,\"\"); }\n" +
" public V(int ttype, Token t, int x) { token=t; this.x=x;}\n" +
" public String toString() { return (token!=null?token.getText():\"\")+\"<V>;\"+x+y+z;}\n" +
"}\n" +
"}\n"+
"a : ID -> ID<V>[42,19,30] ID<V>[$ID,99] ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("<V>;421930 a<V>;9900\n", found);
}
@Test public void testRewriteTokenRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID INT -> ^(ID<V> INT) ;\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 2", debug);
assertEquals("(a<V> 2)\n", found);
}
@Test public void testRewriteString() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : 'begin' -> 'begin'<V> ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "begin", debug);
assertEquals("begin<V>\n", found);
}
@Test public void testRewriteStringRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : 'begin' INT -> ^('begin'<V> INT) ;\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "begin 2", debug);
assertEquals("(begin<V> 2)\n", found);
}
@Test public void testRewriteRuleResults() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {LIST;}\n" +
"@members {\n" +
"static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"static class W extends CommonTree {\n" +
" public W(int tokenType, String txt) { super(new CommonToken(tokenType,txt)); }\n" +
" public W(Token t) { token=t;}\n" +
" public String toString() { return token.getText()+\"<W>\";}\n" +
"}\n" +
"}\n"+
"a : id (',' id)* -> ^(LIST<W>[\"LIST\"] id+);\n" +
"id : ID -> ID<V>;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a,b,c", debug);
assertEquals("(LIST<W> a<V> b<V> c<V>)\n", found);
}
@Test public void testCopySemanticsWithHetero() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"@members {\n" +
"static class V extends CommonTree {\n" +
" public V(Token t) { token=t;}\n" + // for 'int'<V>
" public V(V node) { super(node); }\n\n" + // for dupNode
" public Tree dupNode() { return new V(this); }\n" + // for dup'ing type
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"}\n" +
"a : type ID (',' ID)* ';' -> ^(type ID)+;\n" +
"type : 'int'<V> ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a, b, c;", debug);
assertEquals("(int<V> a) (int<V> b) (int<V> c)\n", found);
}
// TREE PARSERS -- REWRITE AST
@Test public void testTreeParserRewriteFlatList() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"@members {\n" +
"static class V extends CommonTree {\n" +
" public V(Object t) { super((CommonTree)t); }\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"static class W extends CommonTree {\n" +
" public W(Object t) { super((CommonTree)t); }\n" +
" public String toString() { return token.getText()+\"<W>\";}\n" +
"}\n" +
"}\n"+
"a : ID INT -> INT<V> ID<W>\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("34<V> abc<W>\n", found);
}
@Test public void testTreeParserRewriteTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"@members {\n" +
"static class V extends CommonTree {\n" +
" public V(Object t) { super((CommonTree)t); }\n" +
" public String toString() { return token.getText()+\"<V>\";}\n" +
"}\n" +
"static class W extends CommonTree {\n" +
" public W(Object t) { super((CommonTree)t); }\n" +
" public String toString() { return token.getText()+\"<W>\";}\n" +
"}\n" +
"}\n"+
"a : ID INT -> ^(INT<V> ID<W>)\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("(34<V> abc<W>)\n", found);
}
@Test public void testTreeParserRewriteImaginary() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"tokens { ROOT; }\n" +
"@members {\n" +
"class V extends CommonTree {\n" +
" public V(int tokenType) { super(new CommonToken(tokenType)); }\n" +
" public String toString() { return tokenNames[token.getType()]+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID -> ROOT<V> ID\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("ROOT<V> abc\n", found);
}
@Test public void testTreeParserRewriteImaginaryWithArgs() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"tokens { ROOT; }\n" +
"@members {\n" +
"class V extends CommonTree {\n" +
" public int x;\n" +
" public V(int tokenType, int x) { super(new CommonToken(tokenType)); this.x=x;}\n" +
" public String toString() { return tokenNames[token.getType()]+\"<V>;\"+x;}\n" +
"}\n" +
"}\n"+
"a : ID -> ROOT<V>[42] ID\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("ROOT<V>;42 abc\n", found);
}
@Test public void testTreeParserRewriteImaginaryRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"tokens { ROOT; }\n" +
"@members {\n" +
"class V extends CommonTree {\n" +
" public V(int tokenType) { super(new CommonToken(tokenType)); }\n" +
" public String toString() { return tokenNames[token.getType()]+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID -> ^(ROOT<V> ID)\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("(ROOT<V> abc)\n", found);
}
@Test public void testTreeParserRewriteImaginaryFromReal() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"tokens { ROOT; }\n" +
"@members {\n" +
"class V extends CommonTree {\n" +
" public V(int tokenType) { super(new CommonToken(tokenType)); }\n" +
" public V(int tokenType, Object tree) { super((CommonTree)tree); token.setType(tokenType); }\n" +
" public String toString() { return tokenNames[token.getType()]+\"<V>@\"+token.getLine();}\n" +
"}\n" +
"}\n"+
"a : ID -> ROOT<V>[$ID]\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("ROOT<V>@1\n", found); // at line 1; shows copy of ID's stuff
}
@Test public void testTreeParserAutoHeteroAST() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {output=AST; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"tokens { ROOT; }\n" +
"@members {\n" +
"class V extends CommonTree {\n" +
" public V(CommonTree t) { super(t); }\n" + // NEEDS SPECIAL CTOR
" public String toString() { return super.toString()+\"<V>\";}\n" +
"}\n" +
"}\n"+
"a : ID<V> ';'<V>\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc;");
assertEquals("abc<V> ;<V>\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTreeWizard.java 0000644 0001750 0001750 00000034104 11256465236 024565 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2007 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.runtime.tree.*;
import java.util.Map;
import java.util.List;
import java.util.ArrayList;
import java.util.HashMap;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestTreeWizard extends BaseTest {
protected static final String[] tokens =
new String[] {"", "", "", "", "", "A", "B", "C", "D", "E", "ID", "VAR"};
protected static final TreeAdaptor adaptor = new CommonTreeAdaptor();
@Test public void testSingleNode() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("ID");
String found = t.toStringTree();
String expecting = "ID";
assertEquals(expecting, found);
}
@Test public void testSingleNodeWithArg() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("ID[foo]");
String found = t.toStringTree();
String expecting = "foo";
assertEquals(expecting, found);
}
@Test public void testSingleNodeTree() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A)");
String found = t.toStringTree();
String expecting = "A";
assertEquals(expecting, found);
}
@Test public void testSingleLevelTree() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C D)");
String found = t.toStringTree();
String expecting = "(A B C D)";
assertEquals(expecting, found);
}
@Test public void testListTree() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(nil A B C)");
String found = t.toStringTree();
String expecting = "A B C";
assertEquals(expecting, found);
}
@Test public void testInvalidListTree() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("A B C");
assertTrue(t==null);
}
@Test public void testDoubleLevelTree() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A (B C) (B D) E)");
String found = t.toStringTree();
String expecting = "(A (B C) (B D) E)";
assertEquals(expecting, found);
}
@Test public void testSingleNodeIndex() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("ID");
Map m = wiz.index(t);
String found = m.toString();
String expecting = "{10=[ID]}";
assertEquals(expecting, found);
}
@Test public void testNoRepeatsIndex() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C D)");
Map m = wiz.index(t);
String found = sortMapToString(m);
String expecting = "{5=[A], 6=[B], 7=[C], 8=[D]}";
assertEquals(expecting, found);
}
@Test public void testRepeatsIndex() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B (A C B) B D D)");
Map m = wiz.index(t);
String found = sortMapToString(m);
String expecting = "{5=[A, A], 6=[B, B, B], 7=[C], 8=[D, D]}";
assertEquals(expecting, found);
}
@Test public void testNoRepeatsVisit() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C D)");
final List elements = new ArrayList();
wiz.visit(t, wiz.getTokenType("B"), new TreeWizard.Visitor() {
public void visit(Object t) {
elements.add(t);
}
});
String found = elements.toString();
String expecting = "[B]";
assertEquals(expecting, found);
}
@Test public void testNoRepeatsVisit2() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B (A C B) B D D)");
final List elements = new ArrayList();
wiz.visit(t, wiz.getTokenType("C"),
new TreeWizard.Visitor() {
public void visit(Object t) {
elements.add(t);
}
});
String found = elements.toString();
String expecting = "[C]";
assertEquals(expecting, found);
}
@Test public void testRepeatsVisit() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B (A C B) B D D)");
final List elements = new ArrayList();
wiz.visit(t, wiz.getTokenType("B"),
new TreeWizard.Visitor() {
public void visit(Object t) {
elements.add(t);
}
});
String found = elements.toString();
String expecting = "[B, B, B]";
assertEquals(expecting, found);
}
@Test public void testRepeatsVisit2() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B (A C B) B D D)");
final List elements = new ArrayList();
wiz.visit(t, wiz.getTokenType("A"),
new TreeWizard.Visitor() {
public void visit(Object t) {
elements.add(t);
}
});
String found = elements.toString();
String expecting = "[A, A]";
assertEquals(expecting, found);
}
@Test public void testRepeatsVisitWithContext() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B (A C B) B D D)");
final List elements = new ArrayList();
wiz.visit(t, wiz.getTokenType("B"),
new TreeWizard.ContextVisitor() {
public void visit(Object t, Object parent, int childIndex, Map labels) {
elements.add(adaptor.getText(t)+"@"+
(parent!=null?adaptor.getText(parent):"nil")+
"["+childIndex+"]");
}
});
String found = elements.toString();
String expecting = "[B@A[0], B@A[1], B@A[2]]";
assertEquals(expecting, found);
}
@Test public void testRepeatsVisitWithNullParentAndContext() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B (A C B) B D D)");
final List elements = new ArrayList();
wiz.visit(t, wiz.getTokenType("A"),
new TreeWizard.ContextVisitor() {
public void visit(Object t, Object parent, int childIndex, Map labels) {
elements.add(adaptor.getText(t)+"@"+
(parent!=null?adaptor.getText(parent):"nil")+
"["+childIndex+"]");
}
});
String found = elements.toString();
String expecting = "[A@nil[0], A@A[1]]";
assertEquals(expecting, found);
}
@Test public void testVisitPattern() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C (A B) D)");
final List elements = new ArrayList();
wiz.visit(t, "(A B)",
new TreeWizard.Visitor() {
public void visit(Object t) {
elements.add(t);
}
});
String found = elements.toString();
String expecting = "[A]"; // shouldn't match overall root, just (A B)
assertEquals(expecting, found);
}
@Test public void testVisitPatternMultiple() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C (A B) (D (A B)))");
final List elements = new ArrayList();
wiz.visit(t, "(A B)",
new TreeWizard.ContextVisitor() {
public void visit(Object t, Object parent, int childIndex, Map labels) {
elements.add(adaptor.getText(t)+"@"+
(parent!=null?adaptor.getText(parent):"nil")+
"["+childIndex+"]");
}
});
String found = elements.toString();
String expecting = "[A@A[2], A@D[0]]"; // shouldn't match overall root, just (A B)
assertEquals(expecting, found);
}
@Test public void testVisitPatternMultipleWithLabels() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C (A[foo] B[bar]) (D (A[big] B[dog])))");
final List elements = new ArrayList();
wiz.visit(t, "(%a:A %b:B)",
new TreeWizard.ContextVisitor() {
public void visit(Object t, Object parent, int childIndex, Map labels) {
elements.add(adaptor.getText(t)+"@"+
(parent!=null?adaptor.getText(parent):"nil")+
"["+childIndex+"]"+labels.get("a")+"&"+labels.get("b"));
}
});
String found = elements.toString();
String expecting = "[foo@A[2]foo&bar, big@D[0]big&dog]";
assertEquals(expecting, found);
}
@Test public void testParse() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C)");
boolean valid = wiz.parse(t, "(A B C)");
assertTrue(valid);
}
@Test public void testParseSingleNode() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("A");
boolean valid = wiz.parse(t, "A");
assertTrue(valid);
}
@Test public void testParseFlatTree() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(nil A B C)");
boolean valid = wiz.parse(t, "(nil A B C)");
assertTrue(valid);
}
@Test public void testWildcard() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C)");
boolean valid = wiz.parse(t, "(A . .)");
assertTrue(valid);
}
@Test public void testParseWithText() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B[foo] C[bar])");
// C pattern has no text arg so despite [bar] in t, no need
// to match text--check structure only.
boolean valid = wiz.parse(t, "(A B[foo] C)");
assertTrue(valid);
}
@Test public void testParseWithTextFails() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C)");
boolean valid = wiz.parse(t, "(A[foo] B C)");
assertTrue(!valid); // fails
}
@Test public void testParseLabels() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C)");
Map labels = new HashMap();
boolean valid = wiz.parse(t, "(%a:A %b:B %c:C)", labels);
assertTrue(valid);
assertEquals("A", labels.get("a").toString());
assertEquals("B", labels.get("b").toString());
assertEquals("C", labels.get("c").toString());
}
@Test public void testParseWithWildcardLabels() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C)");
Map labels = new HashMap();
boolean valid = wiz.parse(t, "(A %b:. %c:.)", labels);
assertTrue(valid);
assertEquals("B", labels.get("b").toString());
assertEquals("C", labels.get("c").toString());
}
@Test public void testParseLabelsAndTestText() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B[foo] C)");
Map labels = new HashMap();
boolean valid = wiz.parse(t, "(%a:A %b:B[foo] %c:C)", labels);
assertTrue(valid);
assertEquals("A", labels.get("a").toString());
assertEquals("foo", labels.get("b").toString());
assertEquals("C", labels.get("c").toString());
}
@Test public void testParseLabelsInNestedTree() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A (B C) (D E))");
Map labels = new HashMap();
boolean valid = wiz.parse(t, "(%a:A (%b:B %c:C) (%d:D %e:E) )", labels);
assertTrue(valid);
assertEquals("A", labels.get("a").toString());
assertEquals("B", labels.get("b").toString());
assertEquals("C", labels.get("c").toString());
assertEquals("D", labels.get("d").toString());
assertEquals("E", labels.get("e").toString());
}
@Test public void testEquals() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t1 = (CommonTree)wiz.create("(A B C)");
CommonTree t2 = (CommonTree)wiz.create("(A B C)");
boolean same = TreeWizard.equals(t1, t2, adaptor);
assertTrue(same);
}
@Test public void testEqualsWithText() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t1 = (CommonTree)wiz.create("(A B[foo] C)");
CommonTree t2 = (CommonTree)wiz.create("(A B[foo] C)");
boolean same = TreeWizard.equals(t1, t2, adaptor);
assertTrue(same);
}
@Test public void testEqualsWithMismatchedText() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t1 = (CommonTree)wiz.create("(A B[foo] C)");
CommonTree t2 = (CommonTree)wiz.create("(A B C)");
boolean same = TreeWizard.equals(t1, t2, adaptor);
assertTrue(!same);
}
@Test public void testFindPattern() throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokens);
CommonTree t = (CommonTree)wiz.create("(A B C (A[foo] B[bar]) (D (A[big] B[dog])))");
final List subtrees = wiz.find(t, "(A B)");
List elements = subtrees;
String found = elements.toString();
String expecting = "[foo, big]";
assertEquals(expecting, found);
}
} antlr-3.2/tool/src/test/java/org/antlr/test/TestSemanticPredicates.java 0000644 0001750 0001750 00000073243 11256465234 026261 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.analysis.DFA;
import org.antlr.analysis.DecisionProbe;
import org.antlr.codegen.CodeGenerator;
import org.antlr.misc.BitSet;
import org.antlr.tool.*;
import java.util.*;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import antlr.Token;
public class TestSemanticPredicates extends BaseTest {
/** Public default constructor used by TestRig */
public TestSemanticPredicates() {
}
@Test public void testPredsButSyntaxResolves() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? A | {p2}? B ;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testLL_1_Pred() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? A | {p2}? A ;");
String expecting =
".s0-A->.s1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testLL_1_Pred_forced_k_1() throws Exception {
// should stop just like before w/o k set.
Grammar g = new Grammar(
"parser grammar P;\n"+
"a options {k=1;} : {p1}? A | {p2}? A ;");
String expecting =
".s0-A->.s1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testLL_2_Pred() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? A B | {p2}? A B ;");
String expecting =
".s0-A->.s1\n" +
".s1-B->.s2\n" +
".s2-{p1}?->:s3=>1\n" +
".s2-{p2}?->:s4=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testPredicatedLoop() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : ( {p1}? A | {p2}? A )+;");
String expecting = // loop back
".s0-A->.s2\n" +
".s0-EOF->:s1=>3\n" +
".s2-{p1}?->:s3=>1\n" +
".s2-{p2}?->:s4=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testPredicatedToStayInLoop() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : ( {p1}? A )+ (A)+;");
String expecting =
".s0-A->.s1\n" +
".s1-{!(p1)}?->:s2=>1\n" +
".s1-{p1}?->:s3=>2\n"; // loop back
}
@Test public void testAndPredicates() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? {p1a}? A | {p2}? A ;");
String expecting =
".s0-A->.s1\n" +
".s1-{(p1&&p1a)}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test
public void testOrPredicates() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b | {p2}? A ;\n" +
"b : {p1}? A | {p1a}? A ;");
String expecting =
".s0-A->.s1\n" +
".s1-{(p1a||p1)}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testIgnoresHoistingDepthGreaterThanZero() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A {p1}? | A {p2}?;");
String expecting =
".s0-A->:s1=>1\n";
checkDecision(g, 1, expecting, new int[] {2},
new int[] {1,2}, "A", null, null, 2, false);
}
@Test public void testIgnoresPredsHiddenByActions() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {a1} {p1}? A | {a2} {p2}? A ;");
String expecting =
".s0-A->:s1=>1\n";
checkDecision(g, 1, expecting, new int[] {2},
new int[] {1,2}, "A", null, null, 2, true);
}
@Test public void testIgnoresPredsHiddenByActionsOneAlt() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? A | {a2} {p2}? A ;"); // ok since 1 pred visible
String expecting =
".s0-A->.s1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n";
checkDecision(g, 1, expecting, null,
null, null, null, null, 0, true);
}
/*
@Test public void testIncompleteSemanticHoistedContextk2() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : b | A B;\n" +
"b : {p1}? A B | A B ;");
String expecting =
".s0-A->.s1\n" +
".s1-B->:s2=>1\n";
checkDecision(g, 1, expecting, new int[] {2},
new int[] {1,2}, "A B", new int[] {1}, null, 3);
}
*/
@Test public void testHoist2() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b | c ;\n" +
"b : {p1}? A ;\n" +
"c : {p2}? A ;\n");
String expecting =
".s0-A->.s1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testHoistCorrectContext() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b | {p2}? ID ;\n" +
"b : {p1}? ID | INT ;\n");
String expecting = // only tests after ID, not INT :)
".s0-ID->.s1\n" +
".s0-INT->:s2=>1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testDefaultPredNakedAltIsLast() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b | ID ;\n" +
"b : {p1}? ID | INT ;\n");
String expecting =
".s0-ID->.s1\n" +
".s0-INT->:s2=>1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testDefaultPredNakedAltNotLast() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : ID | b ;\n" +
"b : {p1}? ID | INT ;\n");
String expecting =
".s0-ID->.s1\n" +
".s0-INT->:s3=>2\n" +
".s1-{!(p1)}?->:s2=>1\n" +
".s1-{p1}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testLeftRecursivePred() throws Exception {
// No analysis possible. but probably good to fail. Not sure we really want
// left-recursion even if guarded with pred.
Grammar g = new Grammar(
"parser grammar P;\n"+
"s : a ;\n" +
"a : {p1}? a | ID ;\n");
String expecting =
".s0-ID->.s1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n";
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
CodeGenerator generator = new CodeGenerator(newTool(), g, "Java");
g.setCodeGenerator(generator);
if ( g.getNumberOfDecisions()==0 ) {
g.buildNFA();
g.createLookaheadDFAs(false);
}
DFA dfa = g.getLookaheadDFA(1);
assertEquals(null, dfa); // can't analyze.
/*
String result = serializer.serialize(dfa.startState);
assertEquals(expecting, result);
*/
assertEquals("unexpected number of expected problems", 1, equeue.size());
Message msg = (Message)equeue.warnings.get(0);
assertTrue("warning must be a left recursion msg",
msg instanceof LeftRecursionCyclesMessage);
}
@Test public void testIgnorePredFromLL2AltLastAltIsDefaultTrue() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? A B | A C | {p2}? A | {p3}? A | A ;\n");
// two situations of note:
// 1. A B syntax is enough to predict that alt, so p1 is not used
// to distinguish it from alts 2..5
// 2. Alts 3, 4, 5 are nondeterministic with upon A. p2, p3 and the
// complement of p2||p3 is sufficient to resolve the conflict. Do
// not include alt 1's p1 pred in the "complement of other alts"
// because it is not considered nondeterministic with alts 3..5
String expecting =
".s0-A->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-C->:s3=>2\n" +
".s1-{p2}?->:s4=>3\n" +
".s1-{p3}?->:s5=>4\n" +
".s1-{true}?->:s6=>5\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testIgnorePredFromLL2AltPredUnionNeeded() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? A B | A C | {p2}? A | A | {p3}? A ;\n");
// two situations of note:
// 1. A B syntax is enough to predict that alt, so p1 is not used
// to distinguish it from alts 2..5
// 2. Alts 3, 4, 5 are nondeterministic with upon A. p2, p3 and the
// complement of p2||p3 is sufficient to resolve the conflict. Do
// not include alt 1's p1 pred in the "complement of other alts"
// because it is not considered nondeterministic with alts 3..5
String expecting =
".s0-A->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-C->:s3=>2\n" +
".s1-{!((p3||p2))}?->:s5=>4\n" +
".s1-{p2}?->:s4=>3\n" +
".s1-{p3}?->:s6=>5\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testPredGets2SymbolSyntacticContext() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b | A B | C ;\n" +
"b : {p1}? A B ;\n");
String expecting =
".s0-A->.s1\n" +
".s0-C->:s5=>3\n" +
".s1-B->.s2\n" +
".s2-{p1}?->:s3=>1\n" +
".s2-{true}?->:s4=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testMatchesLongestThenTestPred() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b | c ;\n" +
"b : {p}? A ;\n" +
"c : {q}? (A|B)+ ;");
String expecting =
".s0-A->.s1\n" +
".s0-B->:s3=>2\n" +
".s1-{p}?->:s2=>1\n" +
".s1-{q}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testPredsUsedAfterRecursionOverflow() throws Exception {
// analysis must bail out due to non-LL(*) nature (ovf)
// retries with k=1 (but with LL(*) algorithm not optimized version
// as it has preds)
Grammar g = new Grammar(
"parser grammar P;\n"+
"s : {p1}? e '.' | {p2}? e ':' ;\n" +
"e : '(' e ')' | INT ;\n");
String expecting =
".s0-'('->.s1\n" +
".s0-INT->.s4\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n" +
".s4-{p1}?->:s2=>1\n" +
".s4-{p2}?->:s3=>2\n";
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
CodeGenerator generator = new CodeGenerator(newTool(), g, "Java");
g.setCodeGenerator(generator);
if ( g.getNumberOfDecisions()==0 ) {
g.buildNFA();
g.createLookaheadDFAs(false);
}
assertEquals("unexpected number of expected problems", 0, equeue.size());
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testPredsUsedAfterK2FailsNoRecursionOverflow() throws Exception {
// analysis must bail out due to non-LL(*) nature (ovf)
// retries with k=1 (but with LL(*) algorithm not optimized version
// as it has preds)
Grammar g = new Grammar(
"grammar P;\n" +
"options {k=2;}\n"+
"s : {p1}? e '.' | {p2}? e ':' ;\n" +
"e : '(' e ')' | INT ;\n");
String expecting =
".s0-'('->.s1\n" +
".s0-INT->.s6\n" +
".s1-'('->.s2\n" +
".s1-INT->.s5\n" +
".s2-{p1}?->:s3=>1\n" +
".s2-{p2}?->:s4=>2\n" +
".s5-{p1}?->:s3=>1\n" +
".s5-{p2}?->:s4=>2\n" +
".s6-'.'->:s3=>1\n" +
".s6-':'->:s4=>2\n";
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
CodeGenerator generator = new CodeGenerator(newTool(), g, "Java");
g.setCodeGenerator(generator);
if ( g.getNumberOfDecisions()==0 ) {
g.buildNFA();
g.createLookaheadDFAs(false);
}
assertEquals("unexpected number of expected problems", 0, equeue.size());
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testLexerMatchesLongestThenTestPred() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"B : {p}? 'a' ;\n" +
"C : {q}? ('a'|'b')+ ;");
String expecting =
".s0-'a'->.s1\n" +
".s0-'b'->:s4=>2\n" +
".s1-'a'..'b'->:s4=>2\n" +
".s1-<EOT>->.s2\n" +
".s2-{p}?->:s3=>1\n" +
".s2-{q}?->:s4=>2\n";
checkDecision(g, 2, expecting, null, null, null, null, null, 0, false);
}
@Test public void testLexerMatchesLongestMinusPred() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"B : 'a' ;\n" +
"C : ('a'|'b')+ ;");
String expecting =
".s0-'a'->.s1\n" +
".s0-'b'->:s3=>2\n" +
".s1-'a'..'b'->:s3=>2\n" +
".s1-<EOT>->:s2=>1\n";
checkDecision(g, 2, expecting, null, null, null, null, null, 0, false);
}
@Test
public void testGatedPred() throws Exception {
// gated preds are present on all arcs in predictor
Grammar g = new Grammar(
"lexer grammar P;\n"+
"B : {p}? => 'a' ;\n" +
"C : {q}? => ('a'|'b')+ ;");
String expecting =
".s0-'a'&&{(q||p)}?->.s1\n" +
".s0-'b'&&{q}?->:s4=>2\n" +
".s1-'a'..'b'&&{q}?->:s4=>2\n" +
".s1-<EOT>&&{(q||p)}?->.s2\n" +
".s2-{p}?->:s3=>1\n" +
".s2-{q}?->:s4=>2\n";
checkDecision(g, 2, expecting, null, null, null, null, null, 0, false);
}
@Test public void testGatedPredHoistsAndCanBeInStopState() throws Exception {
// I found a bug where merging stop states made us throw away
// a stop state with a gated pred!
Grammar g = new Grammar(
"grammar u;\n" +
"a : b+ ;\n" +
"b : 'x' | {p}?=> 'y' ;");
String expecting =
".s0-'x'->:s2=>1\n" +
".s0-'y'&&{p}?->:s3=>1\n" +
".s0-EOF->:s1=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test
public void testGatedPredInCyclicDFA() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : {p}?=> ('a')+ 'x' ;\n" +
"B : {q}?=> ('a'|'b')+ 'x' ;");
String expecting =
".s0-'a'&&{(q||p)}?->.s1\n" +
".s0-'b'&&{q}?->:s5=>2\n" +
".s1-'a'&&{(q||p)}?->.s1\n" +
".s1-'b'&&{q}?->:s5=>2\n" +
".s1-'x'&&{(q||p)}?->.s2\n" +
".s2-<EOT>&&{(q||p)}?->.s3\n" +
".s3-{p}?->:s4=>1\n" +
".s3-{q}?->:s5=>2\n";
checkDecision(g, 3, expecting, null, null, null, null, null, 0, false);
}
@Test public void testGatedPredNotActuallyUsedOnEdges() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : ('a' | {p}?=> 'a')\n" +
" | 'a' 'b'\n" +
" ;");
String expecting1 =
".s0-'a'->.s1\n" +
".s1-{!(p)}?->:s2=>1\n" + // Used to disambig subrule
".s1-{p}?->:s3=>2\n";
// rule A decision can't test p from s0->1 because 'a' is valid
// for alt1 *and* alt2 w/o p. Can't test p from s1 to s3 because
// we might have passed the first alt of subrule. The same state
// is listed in s2 in 2 different configurations: one with and one
// w/o p. Can't test therefore. p||true == true.
String expecting2 =
".s0-'a'->.s1\n" +
".s1-'b'->:s2=>2\n" +
".s1-<EOT>->:s3=>1\n";
checkDecision(g, 1, expecting1, null, null, null, null, null, 0, false);
checkDecision(g, 2, expecting2, null, null, null, null, null, 0, false);
}
@Test public void testGatedPredDoesNotForceAllToBeGated() throws Exception {
Grammar g = new Grammar(
"grammar w;\n" +
"a : b | c ;\n" +
"b : {p}? B ;\n" +
"c : {q}?=> d ;\n" +
"d : {r}? C ;\n");
String expecting =
".s0-B->:s1=>1\n" +
".s0-C&&{q}?->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testGatedPredDoesNotForceAllToBeGated2() throws Exception {
Grammar g = new Grammar(
"grammar w;\n" +
"a : b | c ;\n" +
"b : {p}? B ;\n" +
"c : {q}?=> d ;\n" +
"d : {r}?=> C\n" +
" | B\n" +
" ;\n");
String expecting =
".s0-B->.s1\n" +
".s0-C&&{(q&&r)}?->:s3=>2\n" +
".s1-{p}?->:s2=>1\n" +
".s1-{q}?->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
@Test public void testORGatedPred() throws Exception {
Grammar g = new Grammar(
"grammar w;\n" +
"a : b | c ;\n" +
"b : {p}? B ;\n" +
"c : {q}?=> d ;\n" +
"d : {r}?=> C\n" +
" | {s}?=> B\n" +
" ;\n");
String expecting =
".s0-B->.s1\n" +
".s0-C&&{(q&&r)}?->:s3=>2\n" +
".s1-{(q&&s)}?->:s3=>2\n" +
".s1-{p}?->:s2=>1\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
/** The following grammar should yield an error that rule 'a' has
* insufficient semantic info pulled from 'b'.
*/
@Test public void testIncompleteSemanticHoistedContext() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : b | B;\n" +
"b : {p1}? B | B ;");
String expecting =
".s0-B->:s1=>1\n";
checkDecision(g, 1, expecting, new int[] {2},
new int[] {1,2}, "B", new int[] {1}, null, 3, false);
}
@Test public void testIncompleteSemanticHoistedContextk2() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : b | A B;\n" +
"b : {p1}? A B | A B ;");
String expecting =
".s0-A->.s1\n" +
".s1-B->:s2=>1\n";
checkDecision(g, 1, expecting, new int[] {2},
new int[] {1,2}, "A B", new int[] {1}, null, 3, false);
}
@Test public void testIncompleteSemanticHoistedContextInFOLLOW() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"options {k=1;}\n" + // limit to k=1 because it's LL(2); force pred hoist
"a : A? ;\n" + // need FOLLOW
"b : X a {p1}? A | Y a A ;"); // only one A is covered
String expecting =
".s0-A->:s1=>1\n"; // s0-EOF->s2 branch pruned during optimization
checkDecision(g, 1, expecting, new int[] {2},
new int[] {1,2}, "A", new int[] {2}, null, 3, false);
}
@Test public void testIncompleteSemanticHoistedContextInFOLLOWk2() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A B)? ;\n" + // need FOLLOW
"b : X a {p1}? A B | Y a A B | Z a ;"); // only first alt is covered
String expecting =
".s0-A->.s1\n" +
".s0-EOF->:s3=>2\n" +
".s1-B->:s2=>1\n";
checkDecision(g, 1, expecting, null,
new int[] {1,2}, "A B", new int[] {2}, null, 2, false);
}
@Test public void testIncompleteSemanticHoistedContextInFOLLOWDueToHiddenPred() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A B)? ;\n" + // need FOLLOW
"b : X a {p1}? A B | Y a {a1} {p2}? A B | Z a ;"); // only first alt is covered
String expecting =
".s0-A->.s1\n" +
".s0-EOF->:s3=>2\n" +
".s1-B->:s2=>1\n";
checkDecision(g, 1, expecting, null,
new int[] {1,2}, "A B", new int[] {2}, null, 2, true);
}
/** The following grammar should yield an error that rule 'a' has
* insufficient semantic info pulled from 'b'. This is the same
* as the previous case except that the D prevents the B path from
* "pinching" together into a single NFA state.
*
* This test also demonstrates that just because B D could predict
* alt 1 in rule 'a', it is unnecessary to continue NFA->DFA
* conversion to include an edge for D. Alt 1 is the only possible
* prediction because we resolve the ambiguity by choosing alt 1.
*/
@Test public void testIncompleteSemanticHoistedContext2() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : b | B;\n" +
"b : {p1}? B | B D ;");
String expecting =
".s0-B->:s1=>1\n";
checkDecision(g, 1, expecting, new int[] {2},
new int[] {1,2}, "B", new int[] {1},
null, 3, false);
}
@Test public void testTooFewSemanticPredicates() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : {p1}? A | A | A ;");
String expecting =
".s0-A->:s1=>1\n";
checkDecision(g, 1, expecting, new int[] {2,3},
new int[] {1,2,3}, "A",
null, null, 2, false);
}
@Test public void testPredWithK1() throws Exception {
Grammar g = new Grammar(
"\tlexer grammar TLexer;\n" +
"A\n" +
"options {\n" +
" k=1;\n" +
"}\n" +
" : {p1}? ('x')+ '.'\n" +
" | {p2}? ('x')+ '.'\n" +
" ;\n");
String expecting =
".s0-'x'->.s1\n" +
".s1-{p1}?->:s2=>1\n" +
".s1-{p2}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] insufficientPredAlts = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 3, expecting, unreachableAlts,
nonDetAlts, ambigInput, insufficientPredAlts,
danglingAlts, numWarnings, false);
}
@Test public void testPredWithArbitraryLookahead() throws Exception {
Grammar g = new Grammar(
"\tlexer grammar TLexer;\n" +
"A : {p1}? ('x')+ '.'\n" +
" | {p2}? ('x')+ '.'\n" +
" ;\n");
String expecting =
".s0-'x'->.s1\n" +
".s1-'.'->.s2\n" +
".s1-'x'->.s1\n" +
".s2-{p1}?->:s3=>1\n" +
".s2-{p2}?->:s4=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] insufficientPredAlts = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 3, expecting, unreachableAlts,
nonDetAlts, ambigInput, insufficientPredAlts,
danglingAlts, numWarnings, false);
}
@Test
/** For a DFA state with lots of configurations that have the same
* predicate, don't just OR them all together as it's a waste to
* test a||a||b||a||a etc... ANTLR makes a unique set and THEN
* OR's them together.
*/
public void testUniquePredicateOR() throws Exception {
Grammar g = new Grammar(
"parser grammar v;\n" +
"\n" +
"a : {a}? b\n" +
" | {b}? b\n" +
" ;\n" +
"\n" +
"b : {c}? (X)+ ;\n" +
"\n" +
"c : a\n" +
" | b\n" +
" ;\n");
String expecting =
".s0-X->.s1\n" +
".s1-{((a&&c)||(b&&c))}?->:s2=>1\n" +
".s1-{c}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] insufficientPredAlts = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 3, expecting, unreachableAlts,
nonDetAlts, ambigInput, insufficientPredAlts,
danglingAlts, numWarnings, false);
}
@Test
public void testSemanticContextPreventsEarlyTerminationOfClosure() throws Exception {
Grammar g = new Grammar(
"parser grammar T;\n" +
"a : loop SEMI | ID SEMI\n" +
" ;\n" +
"loop\n" +
" : {while}? ID\n" +
" | {do}? ID\n" +
" | {for}? ID\n" +
" ;");
String expecting =
".s0-ID->.s1\n" +
".s1-SEMI->.s2\n" +
".s2-{(for||do||while)}?->:s3=>1\n" +
".s2-{true}?->:s4=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, null, 0, false);
}
// S U P P O R T
public void _template() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A | B;");
String expecting =
"\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "L ID R";
int[] insufficientPredAlts = new int[] {1};
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, insufficientPredAlts,
danglingAlts, numWarnings, false);
}
protected void checkDecision(Grammar g,
int decision,
String expecting,
int[] expectingUnreachableAlts,
int[] expectingNonDetAlts,
String expectingAmbigInput,
int[] expectingInsufficientPredAlts,
int[] expectingDanglingAlts,
int expectingNumWarnings,
boolean hasPredHiddenByAction)
throws Exception
{
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
CodeGenerator generator = new CodeGenerator(newTool(), g, "Java");
g.setCodeGenerator(generator);
// mimic actions of org.antlr.Tool first time for grammar g
if ( g.getNumberOfDecisions()==0 ) {
g.buildNFA();
g.createLookaheadDFAs(false);
}
if ( equeue.size()!=expectingNumWarnings ) {
System.err.println("Warnings issued: "+equeue);
}
assertEquals("unexpected number of expected problems",
expectingNumWarnings, equeue.size());
DFA dfa = g.getLookaheadDFA(decision);
FASerializer serializer = new FASerializer(g);
String result = serializer.serialize(dfa.startState);
//System.out.print(result);
List unreachableAlts = dfa.getUnreachableAlts();
// make sure unreachable alts are as expected
if ( expectingUnreachableAlts!=null ) {
BitSet s = new BitSet();
s.addAll(expectingUnreachableAlts);
BitSet s2 = new BitSet();
s2.addAll(unreachableAlts);
assertEquals("unreachable alts mismatch", s, s2);
}
else {
assertEquals("unreachable alts mismatch", 0,
unreachableAlts!=null?unreachableAlts.size():0);
}
// check conflicting input
if ( expectingAmbigInput!=null ) {
// first, find nondet message
Message msg = getNonDeterminismMessage(equeue.warnings);
assertNotNull("no nondeterminism warning?", msg);
assertTrue("expecting nondeterminism; found "+msg.getClass().getName(),
msg instanceof GrammarNonDeterminismMessage);
GrammarNonDeterminismMessage nondetMsg =
getNonDeterminismMessage(equeue.warnings);
List labels =
nondetMsg.probe.getSampleNonDeterministicInputSequence(nondetMsg.problemState);
String input = nondetMsg.probe.getInputSequenceDisplay(labels);
assertEquals(expectingAmbigInput, input);
}
// check nondet alts
if ( expectingNonDetAlts!=null ) {
GrammarNonDeterminismMessage nondetMsg =
getNonDeterminismMessage(equeue.warnings);
assertNotNull("found no nondet alts; expecting: "+
str(expectingNonDetAlts), nondetMsg);
List nonDetAlts =
nondetMsg.probe.getNonDeterministicAltsForState(nondetMsg.problemState);
// compare nonDetAlts with expectingNonDetAlts
BitSet s = new BitSet();
s.addAll(expectingNonDetAlts);
BitSet s2 = new BitSet();
s2.addAll(nonDetAlts);
assertEquals("nondet alts mismatch", s, s2);
assertEquals("mismatch between expected hasPredHiddenByAction", hasPredHiddenByAction,
nondetMsg.problemState.dfa.hasPredicateBlockedByAction);
}
else {
// not expecting any nondet alts, make sure there are none
GrammarNonDeterminismMessage nondetMsg =
getNonDeterminismMessage(equeue.warnings);
assertNull("found nondet alts, but expecting none", nondetMsg);
}
if ( expectingInsufficientPredAlts!=null ) {
GrammarInsufficientPredicatesMessage insuffPredMsg =
getGrammarInsufficientPredicatesMessage(equeue.warnings);
assertNotNull("found no GrammarInsufficientPredicatesMessage alts; expecting: "+
str(expectingNonDetAlts), insuffPredMsg);
Map<Integer, Set<Token>> locations = insuffPredMsg.altToLocations;
Set actualAlts = locations.keySet();
BitSet s = new BitSet();
s.addAll(expectingInsufficientPredAlts);
BitSet s2 = new BitSet();
s2.addAll(actualAlts);
assertEquals("mismatch between insufficiently covered alts", s, s2);
assertEquals("mismatch between expected hasPredHiddenByAction", hasPredHiddenByAction,
insuffPredMsg.problemState.dfa.hasPredicateBlockedByAction);
}
else {
// not expecting any nondet alts, make sure there are none
GrammarInsufficientPredicatesMessage nondetMsg =
getGrammarInsufficientPredicatesMessage(equeue.warnings);
if ( nondetMsg!=null ) {
System.out.println(equeue.warnings);
}
assertNull("found insufficiently covered alts, but expecting none", nondetMsg);
}
assertEquals(expecting, result);
}
protected GrammarNonDeterminismMessage getNonDeterminismMessage(List warnings) {
for (int i = 0; i < warnings.size(); i++) {
Message m = (Message) warnings.get(i);
if ( m instanceof GrammarNonDeterminismMessage ) {
return (GrammarNonDeterminismMessage)m;
}
}
return null;
}
protected GrammarInsufficientPredicatesMessage getGrammarInsufficientPredicatesMessage(List warnings) {
for (int i = 0; i < warnings.size(); i++) {
Message m = (Message) warnings.get(i);
if ( m instanceof GrammarInsufficientPredicatesMessage ) {
return (GrammarInsufficientPredicatesMessage)m;
}
}
return null;
}
protected String str(int[] elements) {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < elements.length; i++) {
if ( i>0 ) {
buf.append(", ");
}
int element = elements[i];
buf.append(element);
}
return buf.toString();
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTrees.java 0000644 0001750 0001750 00000035345 11256465236 023577 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.runtime.tree.*;
import org.antlr.runtime.CommonToken;
import org.antlr.runtime.Token;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestTrees extends BaseTest {
TreeAdaptor adaptor = new CommonTreeAdaptor();
protected boolean debug = false;
static class V extends CommonTree {
public int x;
public V(Token t) { this.token = t;}
public V(int ttype, int x) { this.x=x; token=new CommonToken(ttype); }
public V(int ttype, Token t, int x) { token=t; this.x=x;}
public String toString() { return (token!=null?token.getText():"")+"<V>";}
}
@Test public void testSingleNode() throws Exception {
CommonTree t = new CommonTree(new CommonToken(101));
assertNull(t.parent);
assertEquals(-1, t.childIndex);
}
@Test public void testTwoChildrenOfNilRoot() throws Exception {
CommonTree root_0 = (CommonTree)adaptor.nil();
CommonTree t = new V(101, 2);
CommonTree u = new V(new CommonToken(102,"102"));
adaptor.addChild(root_0, t);
adaptor.addChild(root_0, u);
assertNull(root_0.parent);
assertEquals(-1, root_0.childIndex);
assertEquals(0, t.childIndex);
assertEquals(1, u.childIndex);
}
@Test public void test4Nodes() throws Exception {
// ^(101 ^(102 103) 104)
CommonTree r0 = new CommonTree(new CommonToken(101));
r0.addChild(new CommonTree(new CommonToken(102)));
r0.getChild(0).addChild(new CommonTree(new CommonToken(103)));
r0.addChild(new CommonTree(new CommonToken(104)));
assertNull(r0.parent);
assertEquals(-1, r0.childIndex);
}
@Test public void testList() throws Exception {
// ^(nil 101 102 103)
CommonTree r0 = new CommonTree((Token)null);
CommonTree c0, c1, c2;
r0.addChild(c0=new CommonTree(new CommonToken(101)));
r0.addChild(c1=new CommonTree(new CommonToken(102)));
r0.addChild(c2=new CommonTree(new CommonToken(103)));
assertNull(r0.parent);
assertEquals(-1, r0.childIndex);
assertEquals(r0, c0.parent);
assertEquals(0, c0.childIndex);
assertEquals(r0, c1.parent);
assertEquals(1, c1.childIndex);
assertEquals(r0, c2.parent);
assertEquals(2, c2.childIndex);
}
@Test public void testList2() throws Exception {
// Add child ^(nil 101 102 103) to root 5
// should pull 101 102 103 directly to become 5's child list
CommonTree root = new CommonTree(new CommonToken(5));
// child tree
CommonTree r0 = new CommonTree((Token)null);
CommonTree c0, c1, c2;
r0.addChild(c0=new CommonTree(new CommonToken(101)));
r0.addChild(c1=new CommonTree(new CommonToken(102)));
r0.addChild(c2=new CommonTree(new CommonToken(103)));
root.addChild(r0);
assertNull(root.parent);
assertEquals(-1, root.childIndex);
// check children of root all point at root
assertEquals(root, c0.parent);
assertEquals(0, c0.childIndex);
assertEquals(root, c0.parent);
assertEquals(1, c1.childIndex);
assertEquals(root, c0.parent);
assertEquals(2, c2.childIndex);
}
@Test public void testAddListToExistChildren() throws Exception {
// Add child ^(nil 101 102 103) to root ^(5 6)
// should add 101 102 103 to end of 5's child list
CommonTree root = new CommonTree(new CommonToken(5));
root.addChild(new CommonTree(new CommonToken(6)));
// child tree
CommonTree r0 = new CommonTree((Token)null);
CommonTree c0, c1, c2;
r0.addChild(c0=new CommonTree(new CommonToken(101)));
r0.addChild(c1=new CommonTree(new CommonToken(102)));
r0.addChild(c2=new CommonTree(new CommonToken(103)));
root.addChild(r0);
assertNull(root.parent);
assertEquals(-1, root.childIndex);
// check children of root all point at root
assertEquals(root, c0.parent);
assertEquals(1, c0.childIndex);
assertEquals(root, c0.parent);
assertEquals(2, c1.childIndex);
assertEquals(root, c0.parent);
assertEquals(3, c2.childIndex);
}
@Test public void testDupTree() throws Exception {
// ^(101 ^(102 103 ^(106 107) ) 104 105)
CommonTree r0 = new CommonTree(new CommonToken(101));
CommonTree r1 = new CommonTree(new CommonToken(102));
r0.addChild(r1);
r1.addChild(new CommonTree(new CommonToken(103)));
Tree r2 = new CommonTree(new CommonToken(106));
r2.addChild(new CommonTree(new CommonToken(107)));
r1.addChild(r2);
r0.addChild(new CommonTree(new CommonToken(104)));
r0.addChild(new CommonTree(new CommonToken(105)));
CommonTree dup = (CommonTree)(new CommonTreeAdaptor()).dupTree(r0);
assertNull(dup.parent);
assertEquals(-1, dup.childIndex);
dup.sanityCheckParentAndChildIndexes();
}
@Test public void testBecomeRoot() throws Exception {
// 5 becomes new root of ^(nil 101 102 103)
CommonTree newRoot = new CommonTree(new CommonToken(5));
CommonTree oldRoot = new CommonTree((Token)null);
oldRoot.addChild(new CommonTree(new CommonToken(101)));
oldRoot.addChild(new CommonTree(new CommonToken(102)));
oldRoot.addChild(new CommonTree(new CommonToken(103)));
TreeAdaptor adaptor = new CommonTreeAdaptor();
adaptor.becomeRoot(newRoot, oldRoot);
newRoot.sanityCheckParentAndChildIndexes();
}
@Test public void testBecomeRoot2() throws Exception {
// 5 becomes new root of ^(101 102 103)
CommonTree newRoot = new CommonTree(new CommonToken(5));
CommonTree oldRoot = new CommonTree(new CommonToken(101));
oldRoot.addChild(new CommonTree(new CommonToken(102)));
oldRoot.addChild(new CommonTree(new CommonToken(103)));
TreeAdaptor adaptor = new CommonTreeAdaptor();
adaptor.becomeRoot(newRoot, oldRoot);
newRoot.sanityCheckParentAndChildIndexes();
}
@Test public void testBecomeRoot3() throws Exception {
// ^(nil 5) becomes new root of ^(nil 101 102 103)
CommonTree newRoot = new CommonTree((Token)null);
newRoot.addChild(new CommonTree(new CommonToken(5)));
CommonTree oldRoot = new CommonTree((Token)null);
oldRoot.addChild(new CommonTree(new CommonToken(101)));
oldRoot.addChild(new CommonTree(new CommonToken(102)));
oldRoot.addChild(new CommonTree(new CommonToken(103)));
TreeAdaptor adaptor = new CommonTreeAdaptor();
adaptor.becomeRoot(newRoot, oldRoot);
newRoot.sanityCheckParentAndChildIndexes();
}
@Test public void testBecomeRoot5() throws Exception {
// ^(nil 5) becomes new root of ^(101 102 103)
CommonTree newRoot = new CommonTree((Token)null);
newRoot.addChild(new CommonTree(new CommonToken(5)));
CommonTree oldRoot = new CommonTree(new CommonToken(101));
oldRoot.addChild(new CommonTree(new CommonToken(102)));
oldRoot.addChild(new CommonTree(new CommonToken(103)));
TreeAdaptor adaptor = new CommonTreeAdaptor();
adaptor.becomeRoot(newRoot, oldRoot);
newRoot.sanityCheckParentAndChildIndexes();
}
@Test public void testBecomeRoot6() throws Exception {
// emulates construction of ^(5 6)
CommonTree root_0 = (CommonTree)adaptor.nil();
CommonTree root_1 = (CommonTree)adaptor.nil();
root_1 = (CommonTree)adaptor.becomeRoot(new CommonTree(new CommonToken(5)), root_1);
adaptor.addChild(root_1, new CommonTree(new CommonToken(6)));
adaptor.addChild(root_0, root_1);
root_0.sanityCheckParentAndChildIndexes();
}
// Test replaceChildren
@Test public void testReplaceWithNoChildren() throws Exception {
CommonTree t = new CommonTree(new CommonToken(101));
CommonTree newChild = new CommonTree(new CommonToken(5));
boolean error = false;
try {
t.replaceChildren(0, 0, newChild);
}
catch (IllegalArgumentException iae) {
error = true;
}
assertTrue(error);
}
@Test public void testReplaceWithOneChildren() throws Exception {
// assume token type 99 and use text
CommonTree t = new CommonTree(new CommonToken(99,"a"));
CommonTree c0 = new CommonTree(new CommonToken(99, "b"));
t.addChild(c0);
CommonTree newChild = new CommonTree(new CommonToken(99, "c"));
t.replaceChildren(0, 0, newChild);
String expecting = "(a c)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceInMiddle() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c"))); // index 1
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChild = new CommonTree(new CommonToken(99,"x"));
t.replaceChildren(1, 1, newChild);
String expecting = "(a b x d)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceAtLeft() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b"))); // index 0
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChild = new CommonTree(new CommonToken(99,"x"));
t.replaceChildren(0, 0, newChild);
String expecting = "(a x c d)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceAtRight() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d"))); // index 2
CommonTree newChild = new CommonTree(new CommonToken(99,"x"));
t.replaceChildren(2, 2, newChild);
String expecting = "(a b c x)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceOneWithTwoAtLeft() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChildren = (CommonTree)adaptor.nil();
newChildren.addChild(new CommonTree(new CommonToken(99,"x")));
newChildren.addChild(new CommonTree(new CommonToken(99,"y")));
t.replaceChildren(0, 0, newChildren);
String expecting = "(a x y c d)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceOneWithTwoAtRight() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChildren = (CommonTree)adaptor.nil();
newChildren.addChild(new CommonTree(new CommonToken(99,"x")));
newChildren.addChild(new CommonTree(new CommonToken(99,"y")));
t.replaceChildren(2, 2, newChildren);
String expecting = "(a b c x y)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceOneWithTwoInMiddle() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChildren = (CommonTree)adaptor.nil();
newChildren.addChild(new CommonTree(new CommonToken(99,"x")));
newChildren.addChild(new CommonTree(new CommonToken(99,"y")));
t.replaceChildren(1, 1, newChildren);
String expecting = "(a b x y d)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceTwoWithOneAtLeft() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChild = new CommonTree(new CommonToken(99,"x"));
t.replaceChildren(0, 1, newChild);
String expecting = "(a x d)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceTwoWithOneAtRight() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChild = new CommonTree(new CommonToken(99,"x"));
t.replaceChildren(1, 2, newChild);
String expecting = "(a b x)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceAllWithOne() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChild = new CommonTree(new CommonToken(99,"x"));
t.replaceChildren(0, 2, newChild);
String expecting = "(a x)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
@Test public void testReplaceAllWithTwo() throws Exception {
CommonTree t = new CommonTree(new CommonToken(99, "a"));
t.addChild(new CommonTree(new CommonToken(99, "b")));
t.addChild(new CommonTree(new CommonToken(99, "c")));
t.addChild(new CommonTree(new CommonToken(99, "d")));
CommonTree newChildren = (CommonTree)adaptor.nil();
newChildren.addChild(new CommonTree(new CommonToken(99,"x")));
newChildren.addChild(new CommonTree(new CommonToken(99,"y")));
t.replaceChildren(0, 2, newChildren);
String expecting = "(a x y)";
assertEquals(expecting, t.toStringTree());
t.sanityCheckParentAndChildIndexes();
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestFastQueue.java 0000644 0001750 0001750 00000010667 11256465234 024415 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Test;
import static org.junit.Assert.*;
import org.antlr.runtime.misc.FastQueue;
import java.util.NoSuchElementException;
public class TestFastQueue {
@Test public void testQueueNoRemove() throws Exception {
FastQueue<String> q = new FastQueue<String>();
q.add("a");
q.add("b");
q.add("c");
q.add("d");
q.add("e");
String expecting = "a b c d e";
String found = q.toString();
assertEquals(expecting, found);
}
@Test public void testQueueThenRemoveAll() throws Exception {
FastQueue<String> q = new FastQueue<String>();
q.add("a");
q.add("b");
q.add("c");
q.add("d");
q.add("e");
StringBuffer buf = new StringBuffer();
while ( q.size()>0 ) {
String o = q.remove();
buf.append(o);
if ( q.size()>0 ) buf.append(" ");
}
assertEquals("queue should be empty", 0, q.size());
String expecting = "a b c d e";
String found = buf.toString();
assertEquals(expecting, found);
}
@Test public void testQueueThenRemoveOneByOne() throws Exception {
StringBuffer buf = new StringBuffer();
FastQueue<String> q = new FastQueue<String>();
q.add("a");
buf.append(q.remove());
q.add("b");
buf.append(q.remove());
q.add("c");
buf.append(q.remove());
q.add("d");
buf.append(q.remove());
q.add("e");
buf.append(q.remove());
assertEquals("queue should be empty", 0, q.size());
String expecting = "abcde";
String found = buf.toString();
assertEquals(expecting, found);
}
// E r r o r s
@Test public void testGetFromEmptyQueue() throws Exception {
FastQueue<String> q = new FastQueue<String>();
String msg = null;
try { q.remove(); }
catch (NoSuchElementException nsee) {
msg = nsee.getMessage();
}
String expecting = "queue index 0 > size 0";
String found = msg;
assertEquals(expecting, found);
}
@Test public void testGetFromEmptyQueueAfterSomeAdds() throws Exception {
FastQueue<String> q = new FastQueue<String>();
q.add("a");
q.add("b");
q.remove();
q.remove();
String msg = null;
try { q.remove(); }
catch (NoSuchElementException nsee) {
msg = nsee.getMessage();
}
String expecting = "queue index 0 > size 0";
String found = msg;
assertEquals(expecting, found);
}
@Test public void testGetFromEmptyQueueAfterClear() throws Exception {
FastQueue<String> q = new FastQueue<String>();
q.add("a");
q.add("b");
q.clear();
String msg = null;
try { q.remove(); }
catch (NoSuchElementException nsee) {
msg = nsee.getMessage();
}
String expecting = "queue index 0 > size 0";
String found = msg;
assertEquals(expecting, found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestRewriteTemplates.java 0000644 0001750 0001750 00000025666 11256465234 026020 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.Tool;
import org.antlr.codegen.CodeGenerator;
import org.antlr.tool.ErrorManager;
import org.antlr.tool.Grammar;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestRewriteTemplates extends BaseTest {
protected boolean debug = false;
@Test public void testDelete() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("", found);
}
@Test public void testAction() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> {new StringTemplate($ID.text)} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("abc\n", found);
}
@Test public void testEmbeddedLiteralConstructor() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> {%{$ID.text}} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("abc\n", found);
}
@Test public void testInlineTemplate() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> template(x={$ID},y={$INT}) <<x:<x.text>, y:<y.text>;>> ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("x:abc, y:34;\n", found);
}
@Test public void testNamedTemplate() throws Exception {
// the support code adds template group in it's output Test.java
// that defines template foo.
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> foo(x={$ID.text},y={$INT.text}) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("abc 34\n", found);
}
@Test public void testIndirectTemplate() throws Exception {
// the support code adds template group in it's output Test.java
// that defines template foo.
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> ({\"foo\"})(x={$ID.text},y={$INT.text}) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("abc 34\n", found);
}
@Test public void testInlineTemplateInvokingLib() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> template(x={$ID.text},y={$INT.text}) \"<foo(...)>\" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("abc 34\n", found);
}
@Test public void testPredicatedAlts() throws Exception {
// the support code adds template group in it's output Test.java
// that defines template foo.
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : ID INT -> {false}? foo(x={$ID.text},y={$INT.text})\n" +
" -> foo(x={\"hi\"}, y={$ID.text})\n" +
" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("hi abc\n", found);
}
@Test public void testTemplateReturn() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : b {System.out.println($b.st);} ;\n" +
"b : ID INT -> foo(x={$ID.text},y={$INT.text}) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("abc 34\n", found);
}
@Test public void testReturnValueWithTemplate() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a : b {System.out.println($b.i);} ;\n" +
"b returns [int i] : ID INT {$i=8;} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("8\n", found);
}
@Test public void testTemplateRefToDynamicAttributes() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a scope {String id;} : ID {$a::id=$ID.text;} b\n" +
" {System.out.println($b.st.toString());}\n" +
" ;\n" +
"b : INT -> foo(x={$a::id}) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("abc \n", found);
}
// tests for rewriting templates in tree parsers
@Test public void testSingleNode() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {ASTLabelType=CommonTree; output=template;}\n" +
"s : a {System.out.println($a.st);} ;\n" +
"a : ID -> template(x={$ID.text}) <<|<x>|>> ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc");
assertEquals("|abc|\n", found);
}
@Test public void testSingleNodeRewriteMode() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n"+
"options {ASTLabelType=CommonTree; output=template; rewrite=true;}\n" +
"s : a {System.out.println(input.getTokenStream().toString(0,0));} ;\n" +
"a : ID -> template(x={$ID.text}) <<|<x>|>> ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc");
assertEquals("|abc|\n", found);
}
@Test public void testRewriteRuleAndRewriteModeOnSimpleElements() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"tree grammar TP;\n"+
"options {ASTLabelType=CommonTree; output=template; rewrite=true;}\n" +
"a: ^(A B) -> {ick}\n" +
" | y+=INT -> {ick}\n" +
" | x=ID -> {ick}\n" +
" | BLORT -> {ick}\n" +
" ;\n"
);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
assertEquals("unexpected errors: "+equeue, 0, equeue.warnings.size());
}
@Test public void testRewriteRuleAndRewriteModeIgnoreActionsPredicates() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"tree grammar TP;\n"+
"options {ASTLabelType=CommonTree; output=template; rewrite=true;}\n" +
"a: {action} {action2} x=A -> {ick}\n" +
" | {pred1}? y+=B -> {ick}\n" +
" | C {action} -> {ick}\n" +
" | {pred2}?=> z+=D -> {ick}\n" +
" | (E)=> ^(F G) -> {ick}\n" +
" ;\n"
);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
assertEquals("unexpected errors: "+equeue, 0, equeue.warnings.size());
}
@Test public void testRewriteRuleAndRewriteModeNotSimple() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"tree grammar TP;\n"+
"options {ASTLabelType=CommonTree; output=template; rewrite=true;}\n" +
"a : ID+ -> {ick}\n" +
" | INT INT -> {ick}\n" +
" ;\n"
);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
assertEquals("unexpected errors: "+equeue, 2, equeue.warnings.size());
}
@Test public void testRewriteRuleAndRewriteModeRefRule() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"tree grammar TP;\n"+
"options {ASTLabelType=CommonTree; output=template; rewrite=true;}\n" +
"a : b+ -> {ick}\n" +
" | b b A -> {ick}\n" +
" ;\n" +
"b : B ;\n"
);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
assertEquals("unexpected errors: "+equeue, 2, equeue.warnings.size());
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestSets.java 0000644 0001750 0001750 00000022351 11256465234 023422 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
/** Test the set stuff in lexer and parser */
public class TestSets extends BaseTest {
protected boolean debug = false;
/** Public default constructor used by TestRig */
public TestSets() {
}
@Test public void testSeqDoesNotBecomeSet() throws Exception {
// this must return A not I to the parser; calling a nonfragment rule
// from a nonfragment rule does not set the overall token.
String grammar =
"grammar P;\n" +
"a : C {System.out.println(input);} ;\n" +
"fragment A : '1' | '2';\n" +
"fragment B : '3' '4';\n" +
"C : A | B;\n";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "34", debug);
assertEquals("34\n", found);
}
@Test public void testParserSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : t=('x'|'y') {System.out.println($t.text);} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
@Test public void testParserNotSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : t=~('x'|'y') 'z' {System.out.println($t.text);} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "zz", debug);
assertEquals("z\n", found);
}
@Test public void testParserNotToken() throws Exception {
String grammar =
"grammar T;\n" +
"a : ~'x' 'z' {System.out.println(input);} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "zz", debug);
assertEquals("zz\n", found);
}
@Test public void testParserNotTokenWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"a : t=~'x' 'z' {System.out.println($t.text);} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "zz", debug);
assertEquals("z\n", found);
}
@Test public void testRuleAsSet() throws Exception {
String grammar =
"grammar T;\n" +
"a @after {System.out.println(input);} : 'a' | 'b' |'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "b", debug);
assertEquals("b\n", found);
}
@Test public void testRuleAsSetAST() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'a' | 'b' |'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "b", debug);
assertEquals("b\n", found);
}
@Test public void testNotChar() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println($A.text);} ;\n" +
"A : ~'b' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
@Test public void testOptionalSingleElement() throws Exception {
String grammar =
"grammar T;\n" +
"a : A? 'c' {System.out.println(input);} ;\n" +
"A : 'b' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "bc", debug);
assertEquals("bc\n", found);
}
@Test public void testOptionalLexerSingleElement() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println(input);} ;\n" +
"A : 'b'? 'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "bc", debug);
assertEquals("bc\n", found);
}
@Test public void testStarLexerSingleElement() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println(input);} ;\n" +
"A : 'b'* 'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "bbbbc", debug);
assertEquals("bbbbc\n", found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "c", debug);
assertEquals("c\n", found);
}
@Test public void testPlusLexerSingleElement() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println(input);} ;\n" +
"A : 'b'+ 'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "bbbbc", debug);
assertEquals("bbbbc\n", found);
}
@Test public void testOptionalSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : ('a'|'b')? 'c' {System.out.println(input);} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "ac", debug);
assertEquals("ac\n", found);
}
@Test public void testStarSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : ('a'|'b')* 'c' {System.out.println(input);} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abaac", debug);
assertEquals("abaac\n", found);
}
@Test public void testPlusSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : ('a'|'b')+ 'c' {System.out.println(input);} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abaac", debug);
assertEquals("abaac\n", found);
}
@Test public void testLexerOptionalSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println(input);} ;\n" +
"A : ('a'|'b')? 'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "ac", debug);
assertEquals("ac\n", found);
}
@Test public void testLexerStarSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println(input);} ;\n" +
"A : ('a'|'b')* 'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abaac", debug);
assertEquals("abaac\n", found);
}
@Test public void testLexerPlusSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println(input);} ;\n" +
"A : ('a'|'b')+ 'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abaac", debug);
assertEquals("abaac\n", found);
}
@Test public void testNotCharSet() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println($A.text);} ;\n" +
"A : ~('b'|'c') ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
@Test public void testNotCharSetWithLabel() throws Exception {
// This doesn't work in lexer yet.
// Generates: h=input.LA(1); but h is defined as a Token
String grammar =
"grammar T;\n" +
"a : A {System.out.println($A.text);} ;\n" +
"A : h=~('b'|'c') ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
@Test public void testNotCharSetWithRuleRef() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println($A.text);} ;\n" +
"A : ~('a'|B) ;\n" +
"B : 'b' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
@Test public void testNotCharSetWithRuleRef2() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println($A.text);} ;\n" +
"A : ~('a'|B) ;\n" +
"B : 'b'|'c' ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
@Test public void testNotCharSetWithRuleRef3() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println($A.text);} ;\n" +
"A : ('a'|B) ;\n" +
"fragment\n" +
"B : ~('a'|'c') ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
@Test public void testNotCharSetWithRuleRef4() throws Exception {
String grammar =
"grammar T;\n" +
"a : A {System.out.println($A.text);} ;\n" +
"A : ('a'|B) ;\n" +
"fragment\n" +
"B : ~('a'|C) ;\n" +
"fragment\n" +
"C : 'c'|'d' ;\n ";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", debug);
assertEquals("x\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestIntervalSet.java 0000644 0001750 0001750 00000032707 11256465234 024752 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.analysis.Label;
import org.antlr.misc.IntervalSet;
import java.util.ArrayList;
import java.util.List;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestIntervalSet extends BaseTest {
/** Public default constructor used by TestRig */
public TestIntervalSet() {
}
@Test public void testSingleElement() throws Exception {
IntervalSet s = IntervalSet.of(99);
String expecting = "99";
assertEquals(s.toString(), expecting);
}
@Test public void testIsolatedElements() throws Exception {
IntervalSet s = new IntervalSet();
s.add(1);
s.add('z');
s.add('\uFFF0');
String expecting = "{1, 122, 65520}";
assertEquals(s.toString(), expecting);
}
@Test public void testMixedRangesAndElements() throws Exception {
IntervalSet s = new IntervalSet();
s.add(1);
s.add('a','z');
s.add('0','9');
String expecting = "{1, 48..57, 97..122}";
assertEquals(s.toString(), expecting);
}
@Test public void testSimpleAnd() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(13,15);
String expecting = "13..15";
String result = (s.and(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testRangeAndIsolatedElement() throws Exception {
IntervalSet s = IntervalSet.of('a','z');
IntervalSet s2 = IntervalSet.of('d');
String expecting = "100";
String result = (s.and(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testEmptyIntersection() throws Exception {
IntervalSet s = IntervalSet.of('a','z');
IntervalSet s2 = IntervalSet.of('0','9');
String expecting = "{}";
String result = (s.and(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testEmptyIntersectionSingleElements() throws Exception {
IntervalSet s = IntervalSet.of('a');
IntervalSet s2 = IntervalSet.of('d');
String expecting = "{}";
String result = (s.and(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testNotSingleElement() throws Exception {
IntervalSet vocabulary = IntervalSet.of(1,1000);
vocabulary.add(2000,3000);
IntervalSet s = IntervalSet.of(50,50);
String expecting = "{1..49, 51..1000, 2000..3000}";
String result = (s.complement(vocabulary)).toString();
assertEquals(result, expecting);
}
@Test public void testNotSet() throws Exception {
IntervalSet vocabulary = IntervalSet.of(1,1000);
IntervalSet s = IntervalSet.of(50,60);
s.add(5);
s.add(250,300);
String expecting = "{1..4, 6..49, 61..249, 301..1000}";
String result = (s.complement(vocabulary)).toString();
assertEquals(result, expecting);
}
@Test public void testNotEqualSet() throws Exception {
IntervalSet vocabulary = IntervalSet.of(1,1000);
IntervalSet s = IntervalSet.of(1,1000);
String expecting = "{}";
String result = (s.complement(vocabulary)).toString();
assertEquals(result, expecting);
}
@Test public void testNotSetEdgeElement() throws Exception {
IntervalSet vocabulary = IntervalSet.of(1,2);
IntervalSet s = IntervalSet.of(1);
String expecting = "2";
String result = (s.complement(vocabulary)).toString();
assertEquals(result, expecting);
}
@Test public void testNotSetFragmentedVocabulary() throws Exception {
IntervalSet vocabulary = IntervalSet.of(1,255);
vocabulary.add(1000,2000);
vocabulary.add(9999);
IntervalSet s = IntervalSet.of(50,60);
s.add(3);
s.add(250,300);
s.add(10000); // this is outside range of vocab and should be ignored
String expecting = "{1..2, 4..49, 61..249, 1000..2000, 9999}";
String result = (s.complement(vocabulary)).toString();
assertEquals(result, expecting);
}
@Test public void testSubtractOfCompletelyContainedRange() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(12,15);
String expecting = "{10..11, 16..20}";
String result = (s.subtract(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testSubtractOfOverlappingRangeFromLeft() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(5,11);
String expecting = "12..20";
String result = (s.subtract(s2)).toString();
assertEquals(result, expecting);
IntervalSet s3 = IntervalSet.of(5,10);
expecting = "11..20";
result = (s.subtract(s3)).toString();
assertEquals(result, expecting);
}
@Test public void testSubtractOfOverlappingRangeFromRight() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(15,25);
String expecting = "10..14";
String result = (s.subtract(s2)).toString();
assertEquals(result, expecting);
IntervalSet s3 = IntervalSet.of(20,25);
expecting = "10..19";
result = (s.subtract(s3)).toString();
assertEquals(result, expecting);
}
@Test public void testSubtractOfCompletelyCoveredRange() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(1,25);
String expecting = "{}";
String result = (s.subtract(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testSubtractOfRangeSpanningMultipleRanges() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
s.add(30,40);
s.add(50,60); // s has 3 ranges now: 10..20, 30..40, 50..60
IntervalSet s2 = IntervalSet.of(5,55); // covers one and touches 2nd range
String expecting = "56..60";
String result = (s.subtract(s2)).toString();
assertEquals(result, expecting);
IntervalSet s3 = IntervalSet.of(15,55); // touches both
expecting = "{10..14, 56..60}";
result = (s.subtract(s3)).toString();
assertEquals(result, expecting);
}
/** The following was broken:
{0..113, 115..65534}-{0..115, 117..65534}=116..65534
*/
@Test public void testSubtractOfWackyRange() throws Exception {
IntervalSet s = IntervalSet.of(0,113);
s.add(115,200);
IntervalSet s2 = IntervalSet.of(0,115);
s2.add(117,200);
String expecting = "116";
String result = (s.subtract(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testSimpleEquals() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(10,20);
Boolean expecting = new Boolean(true);
Boolean result = new Boolean(s.equals(s2));
assertEquals(result, expecting);
IntervalSet s3 = IntervalSet.of(15,55);
expecting = new Boolean(false);
result = new Boolean(s.equals(s3));
assertEquals(result, expecting);
}
@Test public void testEquals() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
s.add(2);
s.add(499,501);
IntervalSet s2 = IntervalSet.of(10,20);
s2.add(2);
s2.add(499,501);
Boolean expecting = new Boolean(true);
Boolean result = new Boolean(s.equals(s2));
assertEquals(result, expecting);
IntervalSet s3 = IntervalSet.of(10,20);
s3.add(2);
expecting = new Boolean(false);
result = new Boolean(s.equals(s3));
assertEquals(result, expecting);
}
@Test public void testSingleElementMinusDisjointSet() throws Exception {
IntervalSet s = IntervalSet.of(15,15);
IntervalSet s2 = IntervalSet.of(1,5);
s2.add(10,20);
String expecting = "{}"; // 15 - {1..5, 10..20} = {}
String result = s.subtract(s2).toString();
assertEquals(result, expecting);
}
@Test public void testMembership() throws Exception {
IntervalSet s = IntervalSet.of(15,15);
s.add(50,60);
assertTrue(!s.member(0));
assertTrue(!s.member(20));
assertTrue(!s.member(100));
assertTrue(s.member(15));
assertTrue(s.member(55));
assertTrue(s.member(50));
assertTrue(s.member(60));
}
// {2,15,18} & 10..20
@Test public void testIntersectionWithTwoContainedElements() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(2,2);
s2.add(15);
s2.add(18);
String expecting = "{15, 18}";
String result = (s.and(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testIntersectionWithTwoContainedElementsReversed() throws Exception {
IntervalSet s = IntervalSet.of(10,20);
IntervalSet s2 = IntervalSet.of(2,2);
s2.add(15);
s2.add(18);
String expecting = "{15, 18}";
String result = (s2.and(s)).toString();
assertEquals(result, expecting);
}
@Test public void testComplement() throws Exception {
IntervalSet s = IntervalSet.of(100,100);
s.add(101,101);
IntervalSet s2 = IntervalSet.of(100,102);
String expecting = "102";
String result = (s.complement(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testComplement2() throws Exception {
IntervalSet s = IntervalSet.of(100,101);
IntervalSet s2 = IntervalSet.of(100,102);
String expecting = "102";
String result = (s.complement(s2)).toString();
assertEquals(result, expecting);
}
@Test public void testComplement3() throws Exception {
IntervalSet s = IntervalSet.of(1,96);
s.add(99,Label.MAX_CHAR_VALUE);
String expecting = "97..98";
String result = (s.complement(1,Label.MAX_CHAR_VALUE)).toString();
assertEquals(result, expecting);
}
@Test public void testMergeOfRangesAndSingleValues() throws Exception {
// {0..41, 42, 43..65534}
IntervalSet s = IntervalSet.of(0,41);
s.add(42);
s.add(43,65534);
String expecting = "0..65534";
String result = s.toString();
assertEquals(result, expecting);
}
@Test public void testMergeOfRangesAndSingleValuesReverse() throws Exception {
IntervalSet s = IntervalSet.of(43,65534);
s.add(42);
s.add(0,41);
String expecting = "0..65534";
String result = s.toString();
assertEquals(result, expecting);
}
@Test public void testMergeWhereAdditionMergesTwoExistingIntervals() throws Exception {
// 42, 10, {0..9, 11..41, 43..65534}
IntervalSet s = IntervalSet.of(42);
s.add(10);
s.add(0,9);
s.add(43,65534);
s.add(11,41);
String expecting = "0..65534";
String result = s.toString();
assertEquals(result, expecting);
}
@Test public void testMergeWithDoubleOverlap() throws Exception {
IntervalSet s = IntervalSet.of(1,10);
s.add(20,30);
s.add(5,25); // overlaps two!
String expecting = "1..30";
String result = s.toString();
assertEquals(result, expecting);
}
@Test public void testSize() throws Exception {
IntervalSet s = IntervalSet.of(20,30);
s.add(50,55);
s.add(5,19);
String expecting = "32";
String result = String.valueOf(s.size());
assertEquals(result, expecting);
}
@Test public void testToList() throws Exception {
IntervalSet s = IntervalSet.of(20,25);
s.add(50,55);
s.add(5,5);
String expecting = "[5, 20, 21, 22, 23, 24, 25, 50, 51, 52, 53, 54, 55]";
List foo = new ArrayList();
String result = String.valueOf(s.toList());
assertEquals(result, expecting);
}
/** The following was broken:
{'\u0000'..'s', 'u'..'\uFFFE'} & {'\u0000'..'q', 's'..'\uFFFE'}=
{'\u0000'..'q', 's'}!!!! broken...
'q' is 113 ascii
'u' is 117
*/
@Test public void testNotRIntersectionNotT() throws Exception {
IntervalSet s = IntervalSet.of(0,'s');
s.add('u',200);
IntervalSet s2 = IntervalSet.of(0,'q');
s2.add('s',200);
String expecting = "{0..113, 115, 117..200}";
String result = (s.and(s2)).toString();
assertEquals(result, expecting);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestAutoAST.java 0000644 0001750 0001750 00000065116 11256465234 023772 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestAutoAST extends BaseTest {
protected boolean debug = false;
@Test public void testTokenList() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc 34", debug);
assertEquals("abc 34\n", found);
}
@Test public void testTokenListInSingleAltBlock() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : (ID INT) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc 34", debug);
assertEquals("abc 34\n", found);
}
@Test public void testSimpleRootAtOuterLevel() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : ID^ INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc 34", debug);
assertEquals("(abc 34)\n", found);
}
@Test public void testSimpleRootAtOuterLevelReverse() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT ID^ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "34 abc", debug);
assertEquals("(abc 34)\n", found);
}
@Test public void testBang() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT! ID! INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34 dag 4532", debug);
assertEquals("abc 4532\n", found);
}
@Test public void testOptionalThenRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ( ID INT )? ID^ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 1 b", debug);
assertEquals("(b a 1)\n", found);
}
@Test public void testLabeledStringRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : v='void'^ ID ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void foo;", debug);
assertEquals("(void foo ;)\n", found);
}
@Test public void testWildcard() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : v='void'^ . ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void foo;", debug);
assertEquals("(void foo ;)\n", found);
}
@Test public void testWildcardRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : v='void' .^ ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void foo;", debug);
assertEquals("(foo void ;)\n", found);
}
@Test public void testWildcardRootWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : v='void' x=.^ ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void foo;", debug);
assertEquals("(foo void ;)\n", found);
}
@Test public void testWildcardRootWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : v='void' x=.^ ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void foo;", debug);
assertEquals("(foo void ;)\n", found);
}
@Test public void testWildcardBangWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : v='void' x=.! ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void foo;", debug);
assertEquals("void ;\n", found);
}
@Test public void testRootRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID^ INT^ ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 34 c", debug);
assertEquals("(34 a c)\n", found);
}
@Test public void testRootRoot2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT^ ID^ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 34 c", debug);
assertEquals("(c (34 a))\n", found);
}
@Test public void testRootThenRootInLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID^ (INT '*'^ ID)+ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 34 * b 9 * c", debug);
assertEquals("(* (* (a 34) b 9) c)\n", found);
}
@Test public void testNestedSubrule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'void' (({;}ID|INT) ID | 'null' ) ';' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void a b;", debug);
assertEquals("void a b ;\n", found);
}
@Test public void testInvokeRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : type ID ;\n" +
"type : {;}'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a", debug);
assertEquals("int a\n", found);
}
@Test public void testInvokeRuleAsRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : type^ ID ;\n" +
"type : {;}'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a", debug);
assertEquals("(int a)\n", found);
}
@Test public void testInvokeRuleAsRootWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=type^ ID ;\n" +
"type : {;}'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a", debug);
assertEquals("(int a)\n", found);
}
@Test public void testInvokeRuleAsRootWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x+=type^ ID ;\n" +
"type : {;}'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a", debug);
assertEquals("(int a)\n", found);
}
@Test public void testRuleRootInLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ('+'^ ID)* ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a+b+c+d", debug);
assertEquals("(+ (+ (+ a b) c) d)\n", found);
}
@Test public void testRuleInvocationRuleRootInLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID (op^ ID)* ;\n" +
"op : {;}'+' | '-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a+b+c-d", debug);
assertEquals("(- (+ (+ a b) c) d)\n", found);
}
@Test public void testTailRecursion() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"s : a ;\n" +
"a : atom ('exp'^ a)? ;\n" +
"atom : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "3 exp 4 exp 5", debug);
assertEquals("(exp 3 (exp 4 5))\n", found);
}
@Test public void testSet() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID|INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("abc\n", found);
}
@Test public void testSetRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ('+' | '-')^ ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "+abc", debug);
assertEquals("(+ abc)\n", found);
}
@Ignore
// TODO: FAILS until I rebuild the antlr.g in v3
//
public void testSetRootWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=('+' | '-')^ ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "+abc", debug);
assertEquals("(+ abc)\n", found);
}
@Test public void testSetAsRuleRootInLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID (('+'|'-')^ ID)* ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a+b-c", debug);
assertEquals("(- (+ a b) c)\n", found);
}
@Test public void testNotSet() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ~ID '+' INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "34+2", debug);
assertEquals("34 + 2\n", found);
}
@Test public void testNotSetWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=~ID '+' INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "34+2", debug);
assertEquals("34 + 2\n", found);
}
@Test public void testNotSetWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=~ID '+' INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "34+2", debug);
assertEquals("34 + 2\n", found);
}
@Test public void testNotSetRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ~'+'^ INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "34 55", debug);
assertEquals("(34 55)\n", found);
}
@Test public void testNotSetRootWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ~'+'^ INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "34 55", debug);
assertEquals("(34 55)\n", found);
}
@Test public void testNotSetRootWithListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ~'+'^ INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "34 55", debug);
assertEquals("(34 55)\n", found);
}
@Test public void testNotSetRuleRootInLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT (~INT^ INT)* ;\n" +
"blort : '+' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "3+4+5", debug);
assertEquals("(+ (+ 3 4) 5)\n", found);
}
@Test public void testTokenLabelReuse() throws Exception {
// check for compilation problem due to multiple defines
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : id=ID id=ID {System.out.print(\"2nd id=\"+$id.text+';');} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("2nd id=b;a b\n", found);
}
@Test public void testTokenLabelReuse2() throws Exception {
// check for compilation problem due to multiple defines
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : id=ID id=ID^ {System.out.print(\"2nd id=\"+$id.text+';');} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("2nd id=b;(b a)\n", found);
}
@Test public void testTokenListLabelReuse() throws Exception {
// check for compilation problem due to multiple defines
// make sure ids has both ID tokens
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ids+=ID ids+=ID {System.out.print(\"id list=\"+$ids+';');} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
String expecting = "id list=[[@0,0:0='a',<4>,1:0], [@2,2:2='b',<4>,1:2]];a b\n";
assertEquals(expecting, found);
}
@Test public void testTokenListLabelReuse2() throws Exception {
// check for compilation problem due to multiple defines
// make sure ids has both ID tokens
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ids+=ID^ ids+=ID {System.out.print(\"id list=\"+$ids+';');} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
String expecting = "id list=[[@0,0:0='a',<4>,1:0], [@2,2:2='b',<4>,1:2]];(a b)\n";
assertEquals(expecting, found);
}
@Test public void testTokenListLabelRuleRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : id+=ID^ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a\n", found);
}
@Test public void testTokenListLabelBang() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : id+=ID! ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("", found);
}
@Test public void testRuleListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x+=b x+=b {" +
"Tree t=(Tree)$x.get(1);" +
"System.out.print(\"2nd x=\"+t.toStringTree()+';');} ;\n" +
"b : ID;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("2nd x=b;a b\n", found);
}
@Test public void testRuleListLabelRuleRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ( x+=b^ )+ {" +
"System.out.print(\"x=\"+((CommonTree)$x.get(1)).toStringTree()+';');} ;\n" +
"b : ID;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("x=(b a);(b a)\n", found);
}
@Test public void testRuleListLabelBang() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x+=b! x+=b {" +
"System.out.print(\"1st x=\"+((CommonTree)$x.get(0)).toStringTree()+';');} ;\n" +
"b : ID;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("1st x=a;b\n", found);
}
@Test public void testComplicatedMelange() throws Exception {
// check for compilation problem
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : A b=B b=B c+=C c+=C D {String s = $D.text;} ;\n" +
"A : 'a' ;\n" +
"B : 'b' ;\n" +
"C : 'c' ;\n" +
"D : 'd' ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b b c c d", debug);
assertEquals("a b b c c d\n", found);
}
@Test public void testReturnValueWithAST() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : ID b {System.out.println($b.i);} ;\n" +
"b returns [int i] : INT {$i=Integer.parseInt($INT.text);} ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc 34", debug);
assertEquals("34\nabc 34\n", found);
}
@Test public void testSetLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options { output=AST; }\n" +
"r : (INT|ID)+ ; \n" +
"ID : 'a'..'z' + ;\n" +
"INT : '0'..'9' +;\n" +
"WS: (' ' | '\\n' | '\\t')+ {$channel = HIDDEN;};\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"r", "abc 34 d", debug);
assertEquals("abc 34 d\n", found);
}
@Test public void testExtraTokenInSimpleDecl() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"decl : type^ ID '='! INT ';'! ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"decl", "int 34 x=1;", debug);
assertEquals("line 1:4 extraneous input '34' expecting ID\n", this.stderrDuringParse);
assertEquals("(int x 1)\n", found); // tree gets correct x and 1 tokens
}
@Test public void testMissingIDInSimpleDecl() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"tokens {EXPR;}\n" +
"decl : type^ ID '='! INT ';'! ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"decl", "int =1;", debug);
assertEquals("line 1:4 missing ID at '='\n", this.stderrDuringParse);
assertEquals("(int <missing ID> 1)\n", found); // tree gets invented ID token
}
@Test public void testMissingSetInSimpleDecl() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"tokens {EXPR;}\n" +
"decl : type^ ID '='! INT ';'! ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"decl", "x=1;", debug);
assertEquals("line 1:0 mismatched input 'x' expecting set null\n", this.stderrDuringParse);
assertEquals("(<error: x> x 1)\n", found); // tree gets invented ID token
}
@Test public void testMissingTokenGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" + // follow is EOF
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc", debug);
assertEquals("line 0:-1 missing INT at '<EOF>'\n", this.stderrDuringParse);
assertEquals("abc <missing INT>\n", found);
}
@Test public void testMissingTokenGivesErrorNodeInInvokedRule() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : b ;\n" +
"b : ID INT ;\n" + // follow should see EOF
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc", debug);
assertEquals("line 0:-1 mismatched input '<EOF>' expecting INT\n", this.stderrDuringParse);
assertEquals("<mismatched token: [@-1,0:0='<no text>',<-1>,0:-1], resync=abc>\n", found);
}
@Test public void testExtraTokenGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : b c ;\n" +
"b : ID ;\n" +
"c : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc ick 34", debug);
assertEquals("line 1:4 extraneous input 'ick' expecting INT\n", this.stderrDuringParse);
assertEquals("abc 34\n", found);
}
@Test public void testMissingFirstTokenGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "34", debug);
assertEquals("line 1:0 missing ID at '34'\n", this.stderrDuringParse);
assertEquals("<missing ID> 34\n", found);
}
@Test public void testMissingFirstTokenGivesErrorNode2() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : b c ;\n" +
"b : ID ;\n" +
"c : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "34", debug);
// finds an error at the first token, 34, and re-syncs.
// re-synchronizing does not consume a token because 34 follows
// ref to rule b (start of c). It then matches 34 in c.
assertEquals("line 1:0 missing ID at '34'\n", this.stderrDuringParse);
assertEquals("<missing ID> 34\n", found);
}
@Test public void testNoViableAltGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : b | c ;\n" +
"b : ID ;\n" +
"c : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"S : '*' ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "*", debug);
assertEquals("line 1:0 no viable alternative at input '*'\n", this.stderrDuringParse);
assertEquals("<unexpected: [@0,0:0='*',<6>,1:0], resync=*>\n", found);
}
// S U P P O R T
public void _test() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer", "a", "abc 34", debug);
assertEquals("\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTokenRewriteStream.java 0000644 0001750 0001750 00000063131 11256465236 026305 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.runtime.ANTLRStringStream;
import org.antlr.runtime.CharStream;
import org.antlr.runtime.TokenRewriteStream;
import org.antlr.tool.Grammar;
import org.antlr.tool.Interpreter;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestTokenRewriteStream extends BaseTest {
/** Public default constructor used by TestRig */
public TestTokenRewriteStream() {
}
@Test public void testInsertBeforeIndex0() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(0, "0");
String result = tokens.toString();
String expecting = "0abc";
assertEquals(expecting, result);
}
@Test public void testInsertAfterLastIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertAfter(2, "x");
String result = tokens.toString();
String expecting = "abcx";
assertEquals(expecting, result);
}
@Test public void test2InsertBeforeAfterMiddleIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(1, "x");
tokens.insertAfter(1, "x");
String result = tokens.toString();
String expecting = "axbxc";
assertEquals(expecting, result);
}
@Test public void testReplaceIndex0() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(0, "x");
String result = tokens.toString();
String expecting = "xbc";
assertEquals(expecting, result);
}
@Test public void testReplaceLastIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, "x");
String result = tokens.toString();
String expecting = "abx";
assertEquals(expecting, result);
}
@Test public void testReplaceMiddleIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(1, "x");
String result = tokens.toString();
String expecting = "axc";
assertEquals(expecting, result);
}
@Test public void testToStringStartStop() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"ID : 'a'..'z'+;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';';\n" +
"MUL : '*';\n" +
"ASSIGN : '=';\n" +
"WS : ' '+;\n");
// Tokens: 0123456789
// Input: x = 3 * 0;
CharStream input = new ANTLRStringStream("x = 3 * 0;");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(4, 8, "0"); // replace 3 * 0 with 0
String result = tokens.toOriginalString();
String expecting = "x = 3 * 0;";
assertEquals(expecting, result);
result = tokens.toString();
expecting = "x = 0;";
assertEquals(expecting, result);
result = tokens.toString(0,9);
expecting = "x = 0;";
assertEquals(expecting, result);
result = tokens.toString(4,8);
expecting = "0";
assertEquals(expecting, result);
}
@Test public void testToStringStartStop2() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"ID : 'a'..'z'+;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';';\n" +
"ASSIGN : '=';\n" +
"PLUS : '+';\n" +
"MULT : '*';\n" +
"WS : ' '+;\n");
// Tokens: 012345678901234567
// Input: x = 3 * 0 + 2 * 0;
CharStream input = new ANTLRStringStream("x = 3 * 0 + 2 * 0;");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
String result = tokens.toOriginalString();
String expecting = "x = 3 * 0 + 2 * 0;";
assertEquals(expecting, result);
tokens.replace(4, 8, "0"); // replace 3 * 0 with 0
result = tokens.toString();
expecting = "x = 0 + 2 * 0;";
assertEquals(expecting, result);
result = tokens.toString(0,17);
expecting = "x = 0 + 2 * 0;";
assertEquals(expecting, result);
result = tokens.toString(4,8);
expecting = "0";
assertEquals(expecting, result);
result = tokens.toString(0,8);
expecting = "x = 0";
assertEquals(expecting, result);
result = tokens.toString(12,16);
expecting = "2 * 0";
assertEquals(expecting, result);
tokens.insertAfter(17, "// comment");
result = tokens.toString(12,17);
expecting = "2 * 0;// comment";
assertEquals(expecting, result);
result = tokens.toString(0,8); // try again after insert at end
expecting = "x = 0";
assertEquals(expecting, result);
}
@Test public void test2ReplaceMiddleIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(1, "x");
tokens.replace(1, "y");
String result = tokens.toString();
String expecting = "ayc";
assertEquals(expecting, result);
}
@Test public void test2ReplaceMiddleIndex1InsertBefore() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(0, "_");
tokens.replace(1, "x");
tokens.replace(1, "y");
String result = tokens.toString();
String expecting = "_ayc";
assertEquals(expecting, result);
}
@Test public void testReplaceThenDeleteMiddleIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(1, "x");
tokens.delete(1);
String result = tokens.toString();
String expecting = "ac";
assertEquals(expecting, result);
}
@Test public void testInsertInPriorReplace() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(0, 2, "x");
tokens.insertBefore(1, "0");
Exception exc = null;
try {
tokens.toString();
}
catch (IllegalArgumentException iae) {
exc = iae;
}
String expecting = "insert op <InsertBeforeOp@1:\"0\"> within boundaries of previous <ReplaceOp@0..2:\"x\">";
assertNotNull(exc);
assertEquals(expecting, exc.getMessage());
}
@Test public void testInsertThenReplaceSameIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(0, "0");
tokens.replace(0, "x"); // supercedes insert at 0
String result = tokens.toString();
String expecting = "xbc";
assertEquals(expecting, result);
}
@Test public void test2InsertMiddleIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(1, "x");
tokens.insertBefore(1, "y");
String result = tokens.toString();
String expecting = "ayxbc";
assertEquals(expecting, result);
}
@Test public void test2InsertThenReplaceIndex0() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(0, "x");
tokens.insertBefore(0, "y");
tokens.replace(0, "z");
String result = tokens.toString();
String expecting = "zbc";
assertEquals(expecting, result);
}
@Test public void testReplaceThenInsertBeforeLastIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, "x");
tokens.insertBefore(2, "y");
String result = tokens.toString();
String expecting = "abyx";
assertEquals(expecting, result);
}
@Test public void testInsertThenReplaceLastIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(2, "y");
tokens.replace(2, "x");
String result = tokens.toString();
String expecting = "abx";
assertEquals(expecting, result);
}
@Test public void testReplaceThenInsertAfterLastIndex() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, "x");
tokens.insertAfter(2, "y");
String result = tokens.toString();
String expecting = "abxy";
assertEquals(expecting, result);
}
@Test public void testReplaceRangeThenInsertAtLeftEdge() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcccba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 4, "x");
tokens.insertBefore(2, "y");
String result = tokens.toString();
String expecting = "abyxba";
assertEquals(expecting, result);
}
@Test public void testReplaceRangeThenInsertAtRightEdge() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcccba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 4, "x");
tokens.insertBefore(4, "y"); // no effect; within range of a replace
Exception exc = null;
try {
tokens.toString();
}
catch (IllegalArgumentException iae) {
exc = iae;
}
String expecting = "insert op <InsertBeforeOp@4:\"y\"> within boundaries of previous <ReplaceOp@2..4:\"x\">";
assertNotNull(exc);
assertEquals(expecting, exc.getMessage());
}
@Test public void testReplaceRangeThenInsertAfterRightEdge() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcccba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 4, "x");
tokens.insertAfter(4, "y");
String result = tokens.toString();
String expecting = "abxyba";
assertEquals(expecting, result);
}
@Test public void testReplaceAll() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcccba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(0, 6, "x");
String result = tokens.toString();
String expecting = "x";
assertEquals(expecting, result);
}
@Test public void testReplaceSubsetThenFetch() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcccba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 4, "xyz");
String result = tokens.toString(0,6);
String expecting = "abxyzba";
assertEquals(expecting, result);
}
@Test public void testReplaceThenReplaceSuperset() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcccba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 4, "xyz");
tokens.replace(3, 5, "foo"); // overlaps, error
Exception exc = null;
try {
tokens.toString();
}
catch (IllegalArgumentException iae) {
exc = iae;
}
String expecting = "replace op boundaries of <ReplaceOp@3..5:\"foo\"> overlap with previous <ReplaceOp@2..4:\"xyz\">";
assertNotNull(exc);
assertEquals(expecting, exc.getMessage());
}
@Test public void testReplaceThenReplaceLowerIndexedSuperset() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcccba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 4, "xyz");
tokens.replace(1, 3, "foo"); // overlap, error
Exception exc = null;
try {
tokens.toString();
}
catch (IllegalArgumentException iae) {
exc = iae;
}
String expecting = "replace op boundaries of <ReplaceOp@1..3:\"foo\"> overlap with previous <ReplaceOp@2..4:\"xyz\">";
assertNotNull(exc);
assertEquals(expecting, exc.getMessage());
}
@Test public void testReplaceSingleMiddleThenOverlappingSuperset() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcba");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 2, "xyz");
tokens.replace(0, 3, "foo");
String result = tokens.toString();
String expecting = "fooa";
assertEquals(expecting, result);
}
// June 2, 2008 I rewrote core of rewrite engine; just adding lots more tests here
@Test public void testCombineInserts() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(0, "x");
tokens.insertBefore(0, "y");
String result = tokens.toString();
String expecting = "yxabc";
assertEquals(expecting, result);
}
@Test public void testCombine3Inserts() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(1, "x");
tokens.insertBefore(0, "y");
tokens.insertBefore(1, "z");
String result = tokens.toString();
String expecting = "yazxbc";
assertEquals(expecting, result);
}
@Test public void testCombineInsertOnLeftWithReplace() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(0, 2, "foo");
tokens.insertBefore(0, "z"); // combine with left edge of rewrite
String result = tokens.toString();
String expecting = "zfoo";
assertEquals(expecting, result);
}
@Test public void testCombineInsertOnLeftWithDelete() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.delete(0, 2);
tokens.insertBefore(0, "z"); // combine with left edge of rewrite
String result = tokens.toString();
String expecting = "z"; // make sure combo is not znull
assertEquals(expecting, result);
}
@Test public void testDisjointInserts() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(1, "x");
tokens.insertBefore(2, "y");
tokens.insertBefore(0, "z");
String result = tokens.toString();
String expecting = "zaxbyc";
assertEquals(expecting, result);
}
@Test public void testOverlappingReplace() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(1, 2, "foo");
tokens.replace(0, 3, "bar"); // wipes prior nested replace
String result = tokens.toString();
String expecting = "bar";
assertEquals(expecting, result);
}
@Test public void testOverlappingReplace2() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(0, 3, "bar");
tokens.replace(1, 2, "foo"); // cannot split earlier replace
Exception exc = null;
try {
tokens.toString();
}
catch (IllegalArgumentException iae) {
exc = iae;
}
String expecting = "replace op boundaries of <ReplaceOp@1..2:\"foo\"> overlap with previous <ReplaceOp@0..3:\"bar\">";
assertNotNull(exc);
assertEquals(expecting, exc.getMessage());
}
@Test public void testOverlappingReplace3() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(1, 2, "foo");
tokens.replace(0, 2, "bar"); // wipes prior nested replace
String result = tokens.toString();
String expecting = "barc";
assertEquals(expecting, result);
}
@Test public void testOverlappingReplace4() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(1, 2, "foo");
tokens.replace(1, 3, "bar"); // wipes prior nested replace
String result = tokens.toString();
String expecting = "abar";
assertEquals(expecting, result);
}
@Test public void testDropIdenticalReplace() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(1, 2, "foo");
tokens.replace(1, 2, "foo"); // drop previous, identical
String result = tokens.toString();
String expecting = "afooc";
assertEquals(expecting, result);
}
@Test public void testDropPrevCoveredInsert() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(1, "foo");
tokens.replace(1, 2, "foo"); // kill prev insert
String result = tokens.toString();
String expecting = "afooc";
assertEquals(expecting, result);
}
@Test public void testLeaveAloneDisjointInsert() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.insertBefore(1, "x");
tokens.replace(2, 3, "foo");
String result = tokens.toString();
String expecting = "axbfoo";
assertEquals(expecting, result);
}
@Test public void testLeaveAloneDisjointInsert2() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';\n" +
"B : 'b';\n" +
"C : 'c';\n");
CharStream input = new ANTLRStringStream("abcc");
Interpreter lexEngine = new Interpreter(g, input);
TokenRewriteStream tokens = new TokenRewriteStream(lexEngine);
tokens.LT(1); // fill buffer
tokens.replace(2, 3, "foo");
tokens.insertBefore(1, "x");
String result = tokens.toString();
String expecting = "axbfoo";
assertEquals(expecting, result);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTreeParsing.java 0000644 0001750 0001750 00000026476 11256465236 024745 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestTreeParsing extends BaseTest {
@Test public void testFlatList() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : ID INT\n" +
" {System.out.println($ID+\", \"+$INT);}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("abc, 34\n", found);
}
@Test public void testSimpleTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(ID INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : ^(ID INT)\n" +
" {System.out.println($ID+\", \"+$INT);}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc 34");
assertEquals("abc, 34\n", found);
}
@Test public void testFlatVsTreeDecision() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : b c ;\n" +
"b : ID INT -> ^(ID INT);\n" +
"c : ID INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : b b ;\n" +
"b : ID INT {System.out.print($ID+\" \"+$INT);}\n" +
" | ^(ID INT) {System.out.print(\"^(\"+$ID+\" \"+$INT+')');}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 1 b 2");
assertEquals("^(a 1)b 2\n", found);
}
@Test public void testFlatVsTreeDecision2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : b c ;\n" +
"b : ID INT+ -> ^(ID INT+);\n" +
"c : ID INT+;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : b b ;\n" +
"b : ID INT+ {System.out.print($ID+\" \"+$INT);}\n" +
" | ^(x=ID (y=INT)+) {System.out.print(\"^(\"+$x+' '+$y+')');}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a",
"a 1 2 3 b 4 5");
assertEquals("^(a 3)b 5\n", found);
}
@Test public void testCyclicDFALookahead() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT+ PERIOD;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"PERIOD : '.' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : ID INT+ PERIOD {System.out.print(\"alt 1\");}"+
" | ID INT+ SEMI {System.out.print(\"alt 2\");}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 1 2 3.");
assertEquals("alt 1\n", found);
}
@Test public void testTemplateOutput() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n" +
"options {output=template; ASTLabelType=CommonTree;}\n" +
"s : a {System.out.println($a.st);};\n" +
"a : ID INT -> {new StringTemplate($INT.text)}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "s", "abc 34");
assertEquals("34\n", found);
}
@Test public void testNullableChildList() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT? -> ^(ID INT?);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : ^(ID INT?)\n" +
" {System.out.println($ID);}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc");
assertEquals("abc\n", found);
}
@Test public void testNullableChildList2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT? SEMI -> ^(ID INT?) SEMI ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : ^(ID INT?) SEMI\n" +
" {System.out.println($ID);}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc;");
assertEquals("abc\n", found);
}
@Test public void testNullableChildList3() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=ID INT? (y=ID)? SEMI -> ^($x INT? $y?) SEMI ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a : ^(ID INT? b) SEMI\n" +
" {System.out.println($ID+\", \"+$b.text);}\n" +
" ;\n"+
"b : ID? ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc def;");
assertEquals("abc, def\n", found);
}
@Test public void testActionsAfterRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x=ID INT? SEMI -> ^($x INT?) ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {ASTLabelType=CommonTree;}\n" +
"a @init {int x=0;} : ^(ID {x=1;} {x=2;} INT?)\n" +
" {System.out.println($ID+\", \"+x);}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "abc;");
assertEquals("abc, 2\n", found);
}
@Test public void testWildcardLookahead() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID '+'^ INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"PERIOD : '.' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {tokenVocab=T; ASTLabelType=CommonTree;}\n" +
"a : ^('+' . INT) {System.out.print(\"alt 1\");}"+
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a + 2");
assertEquals("alt 1\n", found);
}
@Test public void testWildcardLookahead2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID '+'^ INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"PERIOD : '.' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {tokenVocab=T; ASTLabelType=CommonTree;}\n" +
"a : ^('+' . INT) {System.out.print(\"alt 1\");}"+
" | ^('+' . .) {System.out.print(\"alt 2\");}\n" +
" ;\n";
// AMBIG upon '+' DOWN INT UP etc.. but so what.
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a + 2");
assertEquals("alt 1\n", found);
}
@Test public void testWildcardLookahead3() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID '+'^ INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"PERIOD : '.' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {tokenVocab=T; ASTLabelType=CommonTree;}\n" +
"a : ^('+' ID INT) {System.out.print(\"alt 1\");}"+
" | ^('+' . .) {System.out.print(\"alt 2\");}\n" +
" ;\n";
// AMBIG upon '+' DOWN INT UP etc.. but so what.
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a + 2");
assertEquals("alt 1\n", found);
}
@Test public void testWildcardPlusLookahead() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID '+'^ INT;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"PERIOD : '.' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP; options {tokenVocab=T; ASTLabelType=CommonTree;}\n" +
"a : ^('+' INT INT ) {System.out.print(\"alt 1\");}"+
" | ^('+' .+) {System.out.print(\"alt 2\");}\n" +
" ;\n";
// AMBIG upon '+' DOWN INT UP etc.. but so what.
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a + 2");
assertEquals("alt 2\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestDFAConversion.java 0000644 0001750 0001750 00000143260 11256465234 025147 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.analysis.DFA;
import org.antlr.analysis.DecisionProbe;
import org.antlr.misc.BitSet;
import org.antlr.tool.*;
import org.antlr.Tool;
import org.antlr.codegen.CodeGenerator;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import java.util.*;
public class TestDFAConversion extends BaseTest {
@Test public void testA() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A C | B;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testAB_or_AC() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A B | A C;");
String expecting =
".s0-A->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-C->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testAB_or_AC_k2() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n" +
"options {k=2;}\n"+
"a : A B | A C;");
String expecting =
".s0-A->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-C->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testAB_or_AC_k1() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n" +
"options {k=1;}\n"+
"a : A B | A C;");
String expecting =
".s0-A->:s1=>1\n";
int[] unreachableAlts = new int[] {2};
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "A" ;
int[] danglingAlts = new int[] {2};
int numWarnings = 2; // ambig upon A
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testselfRecurseNonDet() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : A a X | A a Y;");
List altsWithRecursion = Arrays.asList(new Object[] {1,2});
assertNonLLStar(g, altsWithRecursion);
}
@Test public void testRecursionOverflow() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a Y | A A A A A X ;\n" + // force recursion past m=4
"a : A a | Q;");
List expectedTargetRules = Arrays.asList(new Object[] {"a"});
int expectedAlt = 1;
assertRecursionOverflow(g, expectedTargetRules, expectedAlt);
}
@Test public void testRecursionOverflow2() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a Y | A+ X ;\n" + // force recursion past m=4
"a : A a | Q;");
List expectedTargetRules = Arrays.asList(new Object[] {"a"});
int expectedAlt = 1;
assertRecursionOverflow(g, expectedTargetRules, expectedAlt);
}
@Test public void testRecursionOverflowWithPredOk() throws Exception {
// overflows with k=*, but resolves with pred
// no warnings/errors
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : (a Y)=> a Y | A A A A A X ;\n" + // force recursion past m=4
"a : A a | Q;");
String expecting =
".s0-A->.s1\n" +
".s0-Q&&{synpred1_t}?->:s11=>1\n" +
".s1-A->.s2\n" +
".s1-Q&&{synpred1_t}?->:s10=>1\n" +
".s2-A->.s3\n" +
".s2-Q&&{synpred1_t}?->:s9=>1\n" +
".s3-A->.s4\n" +
".s3-Q&&{synpred1_t}?->:s8=>1\n" +
".s4-A->.s5\n" +
".s4-Q&&{synpred1_t}?->:s6=>1\n" +
".s5-{synpred1_t}?->:s6=>1\n" +
".s5-{true}?->:s7=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testRecursionOverflowWithPredOk2() throws Exception {
// must predict Z w/o predicate
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : (a Y)=> a Y | A A A A A X | Z;\n" + // force recursion past m=4
"a : A a | Q;");
String expecting =
".s0-A->.s1\n" +
".s0-Q&&{synpred1_t}?->:s11=>1\n" +
".s0-Z->:s12=>3\n" +
".s1-A->.s2\n" +
".s1-Q&&{synpred1_t}?->:s10=>1\n" +
".s2-A->.s3\n" +
".s2-Q&&{synpred1_t}?->:s9=>1\n" +
".s3-A->.s4\n" +
".s3-Q&&{synpred1_t}?->:s8=>1\n" +
".s4-A->.s5\n" +
".s4-Q&&{synpred1_t}?->:s6=>1\n" +
".s5-{synpred1_t}?->:s6=>1\n" +
".s5-{true}?->:s7=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testCannotSeePastRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"x : y X\n" +
" | y Y\n" +
" ;\n" +
"y : L y R\n" +
" | B\n" +
" ;");
List altsWithRecursion = Arrays.asList(new Object[] {1,2});
assertNonLLStar(g, altsWithRecursion);
}
@Test public void testSynPredResolvesRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"x : (y X)=> y X\n" +
" | y Y\n" +
" ;\n" +
"y : L y R\n" +
" | B\n" +
" ;");
String expecting =
".s0-B->.s4\n" +
".s0-L->.s1\n" +
".s1-{synpred1_t}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n" +
".s4-{synpred1_t}?->:s2=>1\n" +
".s4-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testSemPredResolvesRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"x : {p}? y X\n" +
" | y Y\n" +
" ;\n" +
"y : L y R\n" +
" | B\n" +
" ;");
String expecting =
".s0-B->.s4\n" +
".s0-L->.s1\n" +
".s1-{p}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n" +
".s4-{p}?->:s2=>1\n" +
".s4-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testSemPredResolvesRecursion2() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"x\n" +
"options {k=1;}\n" +
" : {p}? y X\n" +
" | y Y\n" +
" ;\n" +
"y : L y R\n" +
" | B\n" +
" ;");
String expecting =
".s0-B->.s4\n" +
".s0-L->.s1\n" +
".s1-{p}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n" +
".s4-{p}?->:s2=>1\n" +
".s4-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testSemPredResolvesRecursion3() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"x\n" +
"options {k=2;}\n" + // just makes bigger DFA
" : {p}? y X\n" +
" | y Y\n" +
" ;\n" +
"y : L y R\n" +
" | B\n" +
" ;");
String expecting =
".s0-B->.s6\n" +
".s0-L->.s1\n" +
".s1-B->.s5\n" +
".s1-L->.s2\n" +
".s2-{p}?->:s3=>1\n" +
".s2-{true}?->:s4=>2\n" +
".s5-{p}?->:s3=>1\n" +
".s5-{true}?->:s4=>2\n" +
".s6-X->:s3=>1\n" +
".s6-Y->:s4=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testSynPredResolvesRecursion2() throws Exception {
// k=* fails and it retries/succeeds with k=1 silently
// because of predicate
Grammar g = new Grammar(
"parser grammar t;\n"+
"statement\n" +
" : (reference ASSIGN)=> reference ASSIGN expr\n" +
" | expr\n" +
" ;\n" +
"expr: reference\n" +
" | INT\n" +
" | FLOAT\n" +
" ;\n" +
"reference\n" +
" : ID L argument_list R\n" +
" ;\n" +
"argument_list\n" +
" : expr COMMA expr\n" +
" ;");
String expecting =
".s0-ID->.s1\n" +
".s0-INT..FLOAT->:s3=>2\n" +
".s1-{synpred1_t}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testSynPredResolvesRecursion3() throws Exception {
// No errors with k=1; don't try k=* first
Grammar g = new Grammar(
"parser grammar t;\n"+
"statement\n" +
"options {k=1;}\n" +
" : (reference ASSIGN)=> reference ASSIGN expr\n" +
" | expr\n" +
" ;\n" +
"expr: reference\n" +
" | INT\n" +
" | FLOAT\n" +
" ;\n" +
"reference\n" +
" : ID L argument_list R\n" +
" ;\n" +
"argument_list\n" +
" : expr COMMA expr\n" +
" ;");
String expecting =
".s0-ID->.s1\n" +
".s0-INT..FLOAT->:s3=>2\n" +
".s1-{synpred1_t}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testSynPredResolvesRecursion4() throws Exception {
// No errors with k=2; don't try k=* first
// Should be ok like k=1 'except bigger DFA
Grammar g = new Grammar(
"parser grammar t;\n"+
"statement\n" +
"options {k=2;}\n" +
" : (reference ASSIGN)=> reference ASSIGN expr\n" +
" | expr\n" +
" ;\n" +
"expr: reference\n" +
" | INT\n" +
" | FLOAT\n" +
" ;\n" +
"reference\n" +
" : ID L argument_list R\n" +
" ;\n" +
"argument_list\n" +
" : expr COMMA expr\n" +
" ;");
String expecting =
".s0-ID->.s1\n" +
".s0-INT..FLOAT->:s4=>2\n" +
".s1-L->.s2\n" +
".s2-{synpred1_t}?->:s3=>1\n" +
".s2-{true}?->:s4=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testSynPredResolvesRecursionInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : (B ';')=> B ';'\n" +
" | B '.'\n" +
" ;\n" +
"fragment\n" +
"B : '(' B ')'\n" +
" | 'x'\n" +
" ;\n");
String expecting =
".s0-'('->.s1\n" +
".s0-'x'->.s4\n" +
".s1-{synpred1_t}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n" +
".s4-{synpred1_t}?->:s2=>1\n" +
".s4-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testAutoBacktrackResolvesRecursionInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"options {backtrack=true;}\n"+
"A : B ';'\n" +
" | B '.'\n" +
" ;\n" +
"fragment\n" +
"B : '(' B ')'\n" +
" | 'x'\n" +
" ;\n");
String expecting =
".s0-'('->.s1\n" +
".s0-'x'->.s4\n" +
".s1-{synpred1_t}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n" +
".s4-{synpred1_t}?->:s2=>1\n" +
".s4-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testAutoBacktrackResolvesRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n" +
"options {backtrack=true;}\n"+
"x : y X\n" +
" | y Y\n" +
" ;\n" +
"y : L y R\n" +
" | B\n" +
" ;");
String expecting =
".s0-B->.s4\n" +
".s0-L->.s1\n" +
".s1-{synpred1_t}?->:s2=>1\n" +
".s1-{true}?->:s3=>2\n" +
".s4-{synpred1_t}?->:s2=>1\n" +
".s4-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testselfRecurseNonDet2() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : P a P | P;");
// nondeterministic from left edge
String expecting =
".s0-P->.s1\n" +
".s1-EOF->:s2=>2\n"+
".s1-P->:s3=>1\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "P P";
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testIndirectRecursionLoop() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : b X ;\n"+
"b : a B ;\n");
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Set<Rule> leftRecursive = g.getLeftRecursiveRules();
Set expectedRules =
new HashSet() {{add("a"); add("b");}};
assertEquals(expectedRules, ruleNames(leftRecursive));
g.createLookaheadDFAs(false);
Message msg = (Message)equeue.warnings.get(0);
assertTrue("expecting left recursion cycles; found "+msg.getClass().getName(),
msg instanceof LeftRecursionCyclesMessage);
LeftRecursionCyclesMessage cyclesMsg = (LeftRecursionCyclesMessage)msg;
// cycle of [a, b]
Collection result = cyclesMsg.cycles;
Set expecting = new HashSet() {{add("a"); add("b");}};
assertEquals(expecting, ruleNames2(result));
}
@Test public void testIndirectRecursionLoop2() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : i b X ;\n"+ // should see through i
"b : a B ;\n" +
"i : ;\n");
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Set leftRecursive = g.getLeftRecursiveRules();
Set expectedRules =
new HashSet() {{add("a"); add("b");}};
assertEquals(expectedRules, ruleNames(leftRecursive));
g.createLookaheadDFAs(false);
Message msg = (Message)equeue.warnings.get(0);
assertTrue("expecting left recursion cycles; found "+msg.getClass().getName(),
msg instanceof LeftRecursionCyclesMessage);
LeftRecursionCyclesMessage cyclesMsg = (LeftRecursionCyclesMessage)msg;
// cycle of [a, b]
Collection result = cyclesMsg.cycles;
Set expecting = new HashSet() {{add("a"); add("b");}};
assertEquals(expecting, ruleNames2(result));
}
@Test public void testIndirectRecursionLoop3() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : i b X ;\n"+ // should see through i
"b : a B ;\n" +
"i : ;\n" +
"d : e ;\n" +
"e : d ;\n");
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Set leftRecursive = g.getLeftRecursiveRules();
Set expectedRules =
new HashSet() {{add("a"); add("b"); add("e"); add("d");}};
assertEquals(expectedRules, ruleNames(leftRecursive));
Message msg = (Message)equeue.warnings.get(0);
assertTrue("expecting left recursion cycles; found "+msg.getClass().getName(),
msg instanceof LeftRecursionCyclesMessage);
LeftRecursionCyclesMessage cyclesMsg = (LeftRecursionCyclesMessage)msg;
// cycle of [a, b]
Collection result = cyclesMsg.cycles;
Set expecting = new HashSet() {{add("a"); add("b"); add("d"); add("e");}};
assertEquals(expecting, ruleNames2(result));
}
@Test public void testifThenElse() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : IF s (E s)? | B;\n" +
"slist: s SEMI ;");
String expecting =
".s0-E->:s1=>1\n" +
".s0-SEMI->:s2=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "E";
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
expecting =
".s0-B->:s2=>2\n" +
".s0-IF->:s1=>1\n";
checkDecision(g, 2, expecting, null, null, null, null, 0);
}
@Test public void testifThenElseChecksStackSuffixConflict() throws Exception {
// if you don't check stack soon enough, this finds E B not just E
// as ambig input
Grammar g = new Grammar(
"parser grammar t;\n"+
"slist: s SEMI ;\n"+
"s : IF s el | B;\n" +
"el: (E s)? ;\n");
String expecting =
".s0-E->:s1=>1\n" +
".s0-SEMI->:s2=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "E";
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 2, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
expecting =
".s0-B->:s2=>2\n" +
".s0-IF->:s1=>1\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test
public void testInvokeRule() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : b A\n" +
" | b B\n" +
" | C\n" +
" ;\n" +
"b : X\n" +
" ;\n");
String expecting =
".s0-C->:s4=>3\n" +
".s0-X->.s1\n" +
".s1-A->:s2=>1\n" +
".s1-B->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test
public void testDoubleInvokeRuleLeftEdge() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : b X\n" +
" | b Y\n" +
" ;\n" +
"b : c B\n" +
" | c\n" +
" ;\n" +
"c : C ;\n");
String expecting =
".s0-C->.s1\n" +
".s1-B->.s2\n" +
".s1-X->:s4=>1\n" +
".s1-Y->:s3=>2\n" +
".s2-X->:s4=>1\n" +
".s2-Y->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
expecting =
".s0-C->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-X..Y->:s3=>2\n";
checkDecision(g, 2, expecting, null, null, null, null, 0);
}
@Test public void testimmediateTailRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : A a | A B;");
String expecting =
".s0-A->.s1\n" +
".s1-A->:s3=>1\n" +
".s1-B->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test
public void testAStar_immediateTailRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : A a | ;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-EOF->:s2=>2\n";
int[] unreachableAlts = null; // without
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testNoStartRule() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A a | X;"); // single rule 'a' refers to itself; no start rule
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
Message msg = (Message)equeue.warnings.get(0);
assertTrue("expecting no start rules; found "+msg.getClass().getName(),
msg instanceof GrammarSemanticsMessage);
}
@Test
public void testAStar_immediateTailRecursion2() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : A a | A ;");
String expecting =
".s0-A->.s1\n" +
".s1-A->:s2=>1\n" +
".s1-EOF->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testimmediateLeftRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : a A | B;");
Set leftRecursive = g.getLeftRecursiveRules();
Set expectedRules = new HashSet() {{add("a");}};
assertEquals(expectedRules, ruleNames(leftRecursive));
}
@Test public void testIndirectLeftRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : b | A ;\n" +
"b : c ;\n" +
"c : a | C ;\n");
Set leftRecursive = g.getLeftRecursiveRules();
Set expectedRules = new HashSet() {{add("a"); add("b"); add("c");}};
assertEquals(expectedRules, ruleNames(leftRecursive));
}
@Test public void testLeftRecursionInMultipleCycles() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a x ;\n" +
"a : b | A ;\n" +
"b : c ;\n" +
"c : a | C ;\n" +
"x : y | X ;\n" +
"y : x ;\n");
Set leftRecursive = g.getLeftRecursiveRules();
Set expectedRules =
new HashSet() {{add("a"); add("b"); add("c"); add("x"); add("y");}};
assertEquals(expectedRules, ruleNames(leftRecursive));
}
@Test public void testCycleInsideRuleDoesNotForceInfiniteRecursion() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : (A|)+ B;\n");
// before I added a visitedStates thing, it was possible to loop
// forever inside of a rule if there was an epsilon loop.
Set leftRecursive = g.getLeftRecursiveRules();
Set expectedRules = new HashSet();
assertEquals(expectedRules, leftRecursive);
}
// L O O P S
@Test public void testAStar() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ( A )* ;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-EOF->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testAorBorCStar() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ( A | B | C )* ;");
String expecting =
".s0-A..C->:s1=>1\n" +
".s0-EOF->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testAPlus() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ( A )+ ;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-EOF->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0); // loopback decision
}
@Test public void testAPlusNonGreedyWhenDeterministic() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (options {greedy=false;}:A)+ ;\n");
// should look the same as A+ since no ambiguity
String expecting =
".s0-A->:s1=>1\n" +
".s0-EOF->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testAPlusNonGreedyWhenNonDeterministic() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (options {greedy=false;}:A)+ A+ ;\n");
// should look the same as A+ since no ambiguity
String expecting =
".s0-A->:s1=>2\n"; // always chooses to exit
int[] unreachableAlts = new int[] {1};
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "A";
int[] danglingAlts = null;
int numWarnings = 2;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testAPlusGreedyWhenNonDeterministic() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (options {greedy=true;}:A)+ A+ ;\n");
// should look the same as A+ since no ambiguity
String expecting =
".s0-A->:s1=>1\n"; // always chooses to enter loop upon A
int[] unreachableAlts = new int[] {2};
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "A";
int[] danglingAlts = null;
int numWarnings = 2;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testAorBorCPlus() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ( A | B | C )+ ;");
String expecting =
".s0-A..C->:s1=>1\n" +
".s0-EOF->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testAOptional() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ( A )? B ;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0); // loopback decision
}
@Test public void testAorBorCOptional() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ( A | B | C )? Z ;");
String expecting =
".s0-A..C->:s1=>1\n" +
".s0-Z->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0); // loopback decision
}
// A R B I T R A R Y L O O K A H E A D
@Test
public void testAStarBOrAStarC() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A)* B | (A)* C;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0); // loopback
expecting =
".s0-A->:s1=>1\n" +
".s0-C->:s2=>2\n";
checkDecision(g, 2, expecting, null, null, null, null, 0); // loopback
expecting =
".s0-A->.s1\n" +
".s0-B->:s2=>1\n" +
".s0-C->:s3=>2\n" +
".s1-A->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-C->:s3=>2\n";
checkDecision(g, 3, expecting, null, null, null, null, 0); // rule block
}
@Test
public void testAStarBOrAPlusC() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A)* B | (A)+ C;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0); // loopback
expecting =
".s0-A->:s1=>1\n" +
".s0-C->:s2=>2\n";
checkDecision(g, 2, expecting, null, null, null, null, 0); // loopback
expecting =
".s0-A->.s1\n" +
".s0-B->:s2=>1\n" +
".s1-A->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-C->:s3=>2\n";
checkDecision(g, 3, expecting, null, null, null, null, 0); // rule block
}
@Test
public void testAOrBPlusOrAPlus() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A|B)* X | (A)+ Y;");
String expecting =
".s0-A..B->:s1=>1\n" +
".s0-X->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0); // loopback (A|B)*
expecting =
".s0-A->:s1=>1\n" +
".s0-Y->:s2=>2\n";
checkDecision(g, 2, expecting, null, null, null, null, 0); // loopback (A)+
expecting =
".s0-A->.s1\n" +
".s0-B..X->:s2=>1\n" +
".s1-A->.s1\n" +
".s1-B..X->:s2=>1\n" +
".s1-Y->:s3=>2\n";
checkDecision(g, 3, expecting, null, null, null, null, 0); // rule
}
@Test public void testLoopbackAndExit() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A|B)+ B;");
String expecting =
".s0-A->:s2=>1\n" +
".s0-B->.s1\n" +
".s1-A..B->:s2=>1\n" +
".s1-EOF->:s3=>2\n"; // sees A|B as a set
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testOptionalAltAndBypass() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A|B)? B;");
String expecting =
".s0-A->:s2=>1\n" +
".s0-B->.s1\n" +
".s1-B->:s2=>1\n" +
".s1-EOF->:s3=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
// R E S O L V E S Y N C O N F L I C T S
@Test public void testResolveLL1ByChoosingFirst() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A C | A C;");
String expecting =
".s0-A->.s1\n" +
".s1-C->:s2=>1\n";
int[] unreachableAlts = new int[] {2};
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "A C";
int[] danglingAlts = null;
int numWarnings = 2;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testResolveLL2ByChoosingFirst() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A B | A B;");
String expecting =
".s0-A->.s1\n" +
".s1-B->:s2=>1\n";
int[] unreachableAlts = new int[] {2};
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "A B";
int[] danglingAlts = null;
int numWarnings = 2;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testResolveLL2MixAlt() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A B | A C | A B | Z;");
String expecting =
".s0-A->.s1\n" +
".s0-Z->:s4=>4\n" +
".s1-B->:s2=>1\n" +
".s1-C->:s3=>2\n";
int[] unreachableAlts = new int[] {3};
int[] nonDetAlts = new int[] {1,3};
String ambigInput = "A B";
int[] danglingAlts = null;
int numWarnings = 2;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testIndirectIFThenElseStyleAmbig() throws Exception {
// the (c)+ loopback is ambig because it could match "CASE"
// by entering the loop or by falling out and ignoring (s)*
// back falling back into (cg)* loop which stats over and
// calls cg again. Either choice allows it to get back to
// the same node. The software catches it as:
// "avoid infinite closure computation emanating from alt 1
// of ():27|2|[8 $]" where state 27 is the first alt of (c)+
// and 8 is the first alt of the (cg)* loop.
Grammar g = new Grammar(
"parser grammar t;\n" +
"s : stat ;\n" +
"stat : LCURLY ( cg )* RCURLY | E SEMI ;\n" +
"cg : (c)+ (stat)* ;\n" +
"c : CASE E ;\n");
String expecting =
".s0-CASE->:s2=>1\n" +
".s0-LCURLY..E->:s1=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "CASE";
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 3, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
// S E T S
@Test public void testComplement() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ~(A | B | C) | C {;} ;\n" +
"b : X Y Z ;");
String expecting =
".s0-C->:s2=>2\n" +
".s0-X..Z->:s1=>1\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testComplementToken() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ~C | C {;} ;\n" +
"b : X Y Z ;");
String expecting =
".s0-C->:s2=>2\n" +
".s0-X..Z->:s1=>1\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testComplementChar() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : ~'x' | 'x' {;} ;\n");
String expecting =
".s0-'x'->:s2=>2\n" +
".s0-{'\\u0000'..'w', 'y'..'\\uFFFF'}->:s1=>1\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testComplementCharSet() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : ~(' '|'\t'|'x'|'y') | 'x';\n" + // collapse into single set
"B : 'y' ;");
String expecting =
".s0-'y'->:s2=>2\n" +
".s0-{'\\u0000'..'\\b', '\\n'..'\\u001F', '!'..'x', 'z'..'\\uFFFF'}->:s1=>1\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testNoSetCollapseWithActions() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : (A | B {foo}) | C;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testRuleAltsSetCollapse() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A | B | C ;"
);
String expecting = // still looks like block
" ( grammar t ( rule a ARG RET scope ( BLOCK ( ALT A <end-of-alt> ) ( ALT B <end-of-alt> ) ( ALT C <end-of-alt> ) <end-of-block> ) <end-of-rule> ) )";
assertEquals(expecting, g.getGrammarTree().toStringTree());
}
@Test public void testTokensRuleAltsDoNotCollapse() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a';" +
"B : 'b';\n"
);
String expecting =
".s0-'a'->:s1=>1\n" +
".s0-'b'->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testMultipleSequenceCollision() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n" +
"a : (A{;}|B)\n" +
" | (A{;}|B)\n" +
" | A\n" +
" ;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>1\n"; // not optimized because states are nondet
int[] unreachableAlts = new int[] {2,3};
int[] nonDetAlts = new int[] {1,2,3};
String ambigInput = "A";
int[] danglingAlts = null;
int numWarnings = 3;
checkDecision(g, 3, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
/* There are 2 nondet errors, but the checkDecision only checks first one :(
The "B" conflicting input is not checked except by virtue of the
result DFA.
<string>:2:5: Decision can match input such as "A" using multiple alternatives:
alt 1 via NFA path 7,2,3
alt 2 via NFA path 14,9,10
alt 3 via NFA path 16,17
As a result, alternative(s) 2,3 were disabled for that input,
<string>:2:5: Decision can match input such as "B" using multiple alternatives:
alt 1 via NFA path 7,8,4,5
alt 2 via NFA path 14,15,11,12
As a result, alternative(s) 2 were disabled for that input
<string>:2:5: The following alternatives are unreachable: 2,3
*/
}
@Test public void testMultipleAltsSameSequenceCollision() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n" +
"a : type ID \n" +
" | type ID\n" +
" | type ID\n" +
" | type ID\n" +
" ;\n" +
"\n" +
"type : I | F;");
// nondeterministic from left edge; no stop state
String expecting =
".s0-I..F->.s1\n" +
".s1-ID->:s2=>1\n";
int[] unreachableAlts = new int[] {2,3,4};
int[] nonDetAlts = new int[] {1,2,3,4};
String ambigInput = "I..F ID";
int[] danglingAlts = null;
int numWarnings = 2;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testFollowReturnsToLoopReenteringSameRule() throws Exception {
// D07 can be matched in the (...)? or fall out of esc back into (..)*
// loop in sl. Note that D07 is matched by ~(R|SLASH). No good
// way to write that grammar I guess
Grammar g = new Grammar(
"parser grammar t;\n"+
"sl : L ( esc | ~(R|SLASH) )* R ;\n" +
"\n" +
"esc : SLASH ( N | D03 (D07)? ) ;");
String expecting =
".s0-R->:s3=>3\n" +
".s0-SLASH->:s1=>1\n" +
".s0-{L, N..D07}->:s2=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "D07";
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testTokenCallsAnotherOnLeftEdge() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"F : I '.'\n" +
" ;\n" +
"I : '0'\n" +
" ;\n"
);
String expecting =
".s0-'0'->.s1\n" +
".s1-'.'->:s3=>1\n" +
".s1-<EOT>->:s2=>2\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
@Test public void testSelfRecursionAmbigAlts() throws Exception {
// ambiguous grammar for "L ID R" (alts 1,2 of a)
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a;\n" +
"a : L ID R\n" +
" | L a R\n" + // disabled for L ID R
" | b\n" +
" ;\n" +
"\n" +
"b : ID\n" +
" ;\n");
String expecting =
".s0-ID->:s5=>3\n" +
".s0-L->.s1\n" +
".s1-ID->.s2\n" +
".s1-L->:s4=>2\n" +
".s2-R->:s3=>1\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "L ID R";
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testIndirectRecursionAmbigAlts() throws Exception {
// ambiguous grammar for "L ID R" (alts 1,2 of a)
// This was derived from the java grammar 12/4/2004 when it
// was not handling a unaryExpression properly. I traced it
// to incorrect closure-busy condition. It thought that the trace
// of a->b->a->b again for "L ID" was an infinite loop, but actually
// the repeat call to b only happens *after* an L has been matched.
// I added a check to see what the initial stack looks like and it
// seems to work now.
Grammar g = new Grammar(
"parser grammar t;\n"+
"s : a ;\n" +
"a : L ID R\n" +
" | b\n" +
" ;\n" +
"\n" +
"b : ID\n" +
" | L a R\n" +
" ;");
String expecting =
".s0-ID->:s4=>2\n" +
".s0-L->.s1\n" +
".s1-ID->.s2\n" +
".s1-L->:s4=>2\n" +
".s2-R->:s3=>1\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = "L ID R";
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testTailRecursionInvokedFromArbitraryLookaheadDecision() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : b X\n" +
" | b Y\n" +
" ;\n" +
"\n" +
"b : A\n" +
" | A b\n" +
" ;\n");
List altsWithRecursion = Arrays.asList(new Object[] {1,2});
assertNonLLStar(g, altsWithRecursion);
}
@Test public void testWildcardStarK1AndNonGreedyByDefaultInParser() throws Exception {
// no error because .* assumes it should finish when it sees R
Grammar g = new Grammar(
"parser grammar t;\n" +
"s : A block EOF ;\n" +
"block : L .* R ;");
String expecting =
".s0-A..L->:s2=>1\n" +
".s0-R->:s1=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testWildcardPlusK1AndNonGreedyByDefaultInParser() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n" +
"s : A block EOF ;\n" +
"block : L .+ R ;");
String expecting =
".s0-A..L->:s2=>1\n" +
".s0-R->:s1=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test public void testGatedSynPred() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"x : (X)=> X\n" +
" | Y\n" +
" ;\n");
String expecting =
".s0-X&&{synpred1_t}?->:s1=>1\n" + // does not hoist; it gates edges
".s0-Y->:s2=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
Set<String> preds = g.synPredNamesUsedInDFA;
Set<String> expectedPreds = new HashSet<String>() {{add("synpred1_t");}};
assertEquals("predicate names not recorded properly in grammar", expectedPreds, preds);
}
@Test public void testHoistedGatedSynPred() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"x : (X)=> X\n" +
" | X\n" +
" ;\n");
String expecting =
".s0-X->.s1\n" +
".s1-{synpred1_t}?->:s2=>1\n" + // hoists into decision
".s1-{true}?->:s3=>2\n";
int[] unreachableAlts = null;
int[] nonDetAlts = null;
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 0;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
Set<String> preds = g.synPredNamesUsedInDFA;
Set<String> expectedPreds = new HashSet<String>() {{add("synpred1_t");}};
assertEquals("predicate names not recorded properly in grammar", expectedPreds, preds);
}
// Check state table creation
@Test public void testCyclicTableCreation() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A+ X | A+ Y ;");
String expecting =
".s0-A->:s1=>1\n" +
".s0-B->:s2=>2\n";
}
// S U P P O R T
public void _template() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A | B;");
String expecting =
"\n";
checkDecision(g, 1, expecting, null, null, null, null, 0);
}
protected void assertNonLLStar(Grammar g, List expectedBadAlts) {
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// mimic actions of org.antlr.Tool first time for grammar g
if ( g.getNumberOfDecisions()==0 ) {
g.buildNFA();
g.createLookaheadDFAs(false);
}
NonRegularDecisionMessage msg = getNonRegularDecisionMessage(equeue.errors);
assertTrue("expected fatal non-LL(*) msg", msg!=null);
List<Integer> alts = new ArrayList();
alts.addAll(msg.altsWithRecursion);
Collections.sort(alts);
assertEquals(expectedBadAlts,alts);
}
protected void assertRecursionOverflow(Grammar g,
List expectedTargetRules,
int expectedAlt) {
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// mimic actions of org.antlr.Tool first time for grammar g
if ( g.getNumberOfDecisions()==0 ) {
g.buildNFA();
g.createLookaheadDFAs(false);
}
RecursionOverflowMessage msg = getRecursionOverflowMessage(equeue.errors);
assertTrue("missing expected recursion overflow msg"+msg, msg!=null);
assertEquals("target rules mismatch",
expectedTargetRules.toString(), msg.targetRules.toString());
assertEquals("mismatched alt", expectedAlt, msg.alt);
}
@Test
public void testWildcardInTreeGrammar() throws Exception {
Grammar g = new Grammar(
"tree grammar t;\n" +
"a : A B | A . ;\n");
String expecting =
".s0-A->.s1\n" +
".s1-A->:s3=>2\n" +
".s1-B->:s2=>1\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
@Test
public void testWildcardInTreeGrammar2() throws Exception {
Grammar g = new Grammar(
"tree grammar t;\n" +
"a : ^(A X Y) | ^(A . .) ;\n");
String expecting =
".s0-A->.s1\n" +
".s1-DOWN->.s2\n" +
".s2-X->.s3\n" +
".s2-{A, Y}->:s6=>2\n" +
".s3-Y->.s4\n" +
".s3-{DOWN, A..X}->:s6=>2\n" +
".s4-DOWN->:s6=>2\n" +
".s4-UP->:s5=>1\n";
int[] unreachableAlts = null;
int[] nonDetAlts = new int[] {1,2};
String ambigInput = null;
int[] danglingAlts = null;
int numWarnings = 1;
checkDecision(g, 1, expecting, unreachableAlts,
nonDetAlts, ambigInput, danglingAlts, numWarnings);
}
protected void checkDecision(Grammar g,
int decision,
String expecting,
int[] expectingUnreachableAlts,
int[] expectingNonDetAlts,
String expectingAmbigInput,
int[] expectingDanglingAlts,
int expectingNumWarnings)
throws Exception
{
DecisionProbe.verbose=true; // make sure we get all error info
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// mimic actions of org.antlr.Tool first time for grammar g
if ( g.getNumberOfDecisions()==0 ) {
g.buildNFA();
g.createLookaheadDFAs(false);
}
CodeGenerator generator = new CodeGenerator(newTool(), g, "Java");
g.setCodeGenerator(generator);
if ( equeue.size()!=expectingNumWarnings ) {
System.err.println("Warnings issued: "+equeue);
}
assertEquals("unexpected number of expected problems",
expectingNumWarnings, equeue.size());
DFA dfa = g.getLookaheadDFA(decision);
assertNotNull("no DFA for decision "+decision, dfa);
FASerializer serializer = new FASerializer(g);
String result = serializer.serialize(dfa.startState);
List unreachableAlts = dfa.getUnreachableAlts();
// make sure unreachable alts are as expected
if ( expectingUnreachableAlts!=null ) {
BitSet s = new BitSet();
s.addAll(expectingUnreachableAlts);
BitSet s2 = new BitSet();
s2.addAll(unreachableAlts);
assertEquals("unreachable alts mismatch", s, s2);
}
else {
assertEquals("number of unreachable alts", 0,
unreachableAlts!=null?unreachableAlts.size():0);
}
// check conflicting input
if ( expectingAmbigInput!=null ) {
// first, find nondet message
Message msg = (Message)equeue.warnings.get(0);
assertTrue("expecting nondeterminism; found "+msg.getClass().getName(),
msg instanceof GrammarNonDeterminismMessage);
GrammarNonDeterminismMessage nondetMsg =
getNonDeterminismMessage(equeue.warnings);
List labels =
nondetMsg.probe.getSampleNonDeterministicInputSequence(nondetMsg.problemState);
String input = nondetMsg.probe.getInputSequenceDisplay(labels);
assertEquals(expectingAmbigInput, input);
}
// check nondet alts
if ( expectingNonDetAlts!=null ) {
RecursionOverflowMessage recMsg = null;
GrammarNonDeterminismMessage nondetMsg =
getNonDeterminismMessage(equeue.warnings);
List nonDetAlts = null;
if ( nondetMsg!=null ) {
nonDetAlts =
nondetMsg.probe.getNonDeterministicAltsForState(nondetMsg.problemState);
}
else {
recMsg = getRecursionOverflowMessage(equeue.warnings);
if ( recMsg!=null ) {
//nonDetAlts = new ArrayList(recMsg.alts);
}
}
// compare nonDetAlts with expectingNonDetAlts
BitSet s = new BitSet();
s.addAll(expectingNonDetAlts);
BitSet s2 = new BitSet();
s2.addAll(nonDetAlts);
assertEquals("nondet alts mismatch", s, s2);
assertTrue("found no nondet alts; expecting: "+
str(expectingNonDetAlts),
nondetMsg!=null||recMsg!=null);
}
else {
// not expecting any nondet alts, make sure there are none
GrammarNonDeterminismMessage nondetMsg =
getNonDeterminismMessage(equeue.warnings);
assertNull("found nondet alts, but expecting none", nondetMsg);
}
assertEquals(expecting, result);
}
protected GrammarNonDeterminismMessage getNonDeterminismMessage(List warnings) {
for (int i = 0; i < warnings.size(); i++) {
Message m = (Message) warnings.get(i);
if ( m instanceof GrammarNonDeterminismMessage ) {
return (GrammarNonDeterminismMessage)m;
}
}
return null;
}
protected NonRegularDecisionMessage getNonRegularDecisionMessage(List errors) {
for (int i = 0; i < errors.size(); i++) {
Message m = (Message) errors.get(i);
if ( m instanceof NonRegularDecisionMessage ) {
return (NonRegularDecisionMessage)m;
}
}
return null;
}
protected RecursionOverflowMessage getRecursionOverflowMessage(List warnings) {
for (int i = 0; i < warnings.size(); i++) {
Message m = (Message) warnings.get(i);
if ( m instanceof RecursionOverflowMessage ) {
return (RecursionOverflowMessage)m;
}
}
return null;
}
protected LeftRecursionCyclesMessage getLeftRecursionCyclesMessage(List warnings) {
for (int i = 0; i < warnings.size(); i++) {
Message m = (Message) warnings.get(i);
if ( m instanceof LeftRecursionCyclesMessage ) {
return (LeftRecursionCyclesMessage)m;
}
}
return null;
}
protected GrammarDanglingStateMessage getDanglingStateMessage(List warnings) {
for (int i = 0; i < warnings.size(); i++) {
Message m = (Message) warnings.get(i);
if ( m instanceof GrammarDanglingStateMessage ) {
return (GrammarDanglingStateMessage)m;
}
}
return null;
}
protected String str(int[] elements) {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < elements.length; i++) {
if ( i>0 ) {
buf.append(", ");
}
int element = elements[i];
buf.append(element);
}
return buf.toString();
}
protected Set<String> ruleNames(Set<Rule> rules) {
Set<String> x = new HashSet<String>();
for (Rule r : rules) {
x.add(r.name);
}
return x;
}
protected Set<String> ruleNames2(Collection<HashSet> rules) {
Set<String> x = new HashSet<String>();
for (HashSet s : rules) {
x.addAll(ruleNames(s));
}
return x;
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestRewriteAST.java 0000644 0001750 0001750 00000133571 11256465234 024504 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.tool.ErrorManager;
import org.antlr.tool.GrammarSemanticsMessage;
import org.antlr.tool.Grammar;
import org.antlr.Tool;
import org.antlr.codegen.CodeGenerator;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestRewriteAST extends BaseTest {
protected boolean debug = false;
@Test public void testDelete() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("", found);
}
@Test public void testSingleToken() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID -> ID;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("abc\n", found);
}
@Test public void testSingleTokenToNewNode() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID -> ID[\"x\"];\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("x\n", found);
}
@Test public void testSingleTokenToNewNodeRoot() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID -> ^(ID[\"x\"] INT);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("(x INT)\n", found);
}
@Test public void testSingleTokenToNewNode2() throws Exception {
// Allow creation of new nodes w/o args.
String grammar =
"grammar TT;\n" +
"options {output=AST;}\n" +
"a : ID -> ID[ ];\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("TT.g", grammar, "TTParser", "TTLexer",
"a", "abc", debug);
assertEquals("ID\n", found);
}
@Test public void testSingleCharLiteral() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'c' -> 'c';\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "c", debug);
assertEquals("c\n", found);
}
@Test public void testSingleStringLiteral() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'ick' -> 'ick';\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "ick", debug);
assertEquals("ick\n", found);
}
@Test public void testSingleRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : b -> b;\n" +
"b : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("abc\n", found);
}
@Test public void testReorderTokens() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> INT ID;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("34 abc\n", found);
}
@Test public void testReorderTokenAndRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : b INT -> INT b;\n" +
"b : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("34 abc\n", found);
}
@Test public void testTokenTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ^(INT ID);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("(34 abc)\n", found);
}
@Test public void testTokenTreeAfterOtherStuff() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'void' ID INT -> 'void' ^(INT ID);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "void abc 34", debug);
assertEquals("void (34 abc)\n", found);
}
@Test public void testNestedTokenTreeWithOuterLoop() throws Exception {
// verify that ID and INT both iterate over outer index variable
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {DUH;}\n" +
"a : ID INT ID INT -> ^( DUH ID ^( DUH INT) )+ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 1 b 2", debug);
assertEquals("(DUH a (DUH 1)) (DUH b (DUH 2))\n", found);
}
@Test public void testOptionalSingleToken() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID -> ID? ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("abc\n", found);
}
@Test public void testClosureSingleToken() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ID -> ID* ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testPositiveClosureSingleToken() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID ID -> ID+ ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testOptionalSingleRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : b -> b?;\n" +
"b : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("abc\n", found);
}
@Test public void testClosureSingleRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : b b -> b*;\n" +
"b : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testClosureOfLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : x+=b x+=b -> $x*;\n" +
"b : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testOptionalLabelNoListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : (x=ID)? -> $x?;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a\n", found);
}
@Test public void testPositiveClosureSingleRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : b b -> b+;\n" +
"b : ID ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testSinglePredicateT() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID -> {true}? ID -> ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("abc\n", found);
}
@Test public void testSinglePredicateF() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID -> {false}? ID -> ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc", debug);
assertEquals("", found);
}
@Test public void testMultiplePredicate() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> {false}? ID\n" +
" -> {true}? INT\n" +
" -> \n" +
" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 2", debug);
assertEquals("2\n", found);
}
@Test public void testMultiplePredicateTrees() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT -> {false}? ^(ID INT)\n" +
" -> {true}? ^(INT ID)\n" +
" -> ID\n" +
" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 2", debug);
assertEquals("(2 a)\n", found);
}
@Test public void testSimpleTree() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : op INT -> ^(op INT);\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "-34", debug);
assertEquals("(- 34)\n", found);
}
@Test public void testSimpleTree2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : op INT -> ^(INT op);\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "+ 34", debug);
assertEquals("(34 +)\n", found);
}
@Test public void testNestedTrees() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'var' (ID ':' type ';')+ -> ^('var' ^(':' ID type)+) ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "var a:int; b:float;", debug);
assertEquals("(var (: a int) (: b float))\n", found);
}
@Test public void testImaginaryTokenCopy() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {VAR;}\n" +
"a : ID (',' ID)*-> ^(VAR ID)+ ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a,b,c", debug);
assertEquals("(VAR a) (VAR b) (VAR c)\n", found);
}
@Test public void testTokenUnreferencedOnLeftButDefined() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {VAR;}\n" +
"a : b -> ID ;\n" +
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("ID\n", found);
}
@Test public void testImaginaryTokenCopySetText() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {VAR;}\n" +
"a : ID (',' ID)*-> ^(VAR[\"var\"] ID)+ ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a,b,c", debug);
assertEquals("(var a) (var b) (var c)\n", found);
}
@Test public void testImaginaryTokenNoCopyFromToken() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : lc='{' ID+ '}' -> ^(BLOCK[$lc] ID+) ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "{a b c}", debug);
assertEquals("({ a b c)\n", found);
}
@Test public void testImaginaryTokenNoCopyFromTokenSetText() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : lc='{' ID+ '}' -> ^(BLOCK[$lc,\"block\"] ID+) ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "{a b c}", debug);
assertEquals("(block a b c)\n", found);
}
@Test public void testMixedRewriteAndAutoAST() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : b b^ ;\n" + // 2nd b matches only an INT; can make it root
"b : ID INT -> INT ID\n" +
" | INT\n" +
" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 1 2", debug);
assertEquals("(2 1 a)\n", found);
}
@Test public void testSubruleWithRewrite() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : b b ;\n" +
"b : (ID INT -> INT ID | INT INT -> INT+ )\n" +
" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a 1 2 3", debug);
assertEquals("1 a 2 3\n", found);
}
@Test public void testSubruleWithRewrite2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {TYPE;}\n" +
"a : b b ;\n" +
"b : 'int'\n" +
" ( ID -> ^(TYPE 'int' ID)\n" +
" | ID '=' INT -> ^(TYPE 'int' ID INT)\n" +
" )\n" +
" ';'\n" +
" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a; int b=3;", debug);
assertEquals("(TYPE int a) (TYPE int b 3)\n", found);
}
@Test public void testNestedRewriteShutsOffAutoAST() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : b b ;\n" +
"b : ID ( ID (last=ID -> $last)+ ) ';'\n" + // get last ID
" | INT\n" + // should still get auto AST construction
" ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b c d; 42", debug);
assertEquals("d 42\n", found);
}
@Test public void testRewriteActions() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : atom -> ^({adaptor.create(INT,\"9\")} atom) ;\n" +
"atom : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "3", debug);
assertEquals("(9 3)\n", found);
}
@Test public void testRewriteActions2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : atom -> {adaptor.create(INT,\"9\")} atom ;\n" +
"atom : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "3", debug);
assertEquals("9 3\n", found);
}
@Test public void testRefToOldValue() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : (atom -> atom) (op='+' r=atom -> ^($op $a $r) )* ;\n" +
"atom : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "3+4+5", debug);
assertEquals("(+ (+ 3 4) 5)\n", found);
}
@Test public void testCopySemanticsForRules() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : atom -> ^(atom atom) ;\n" + // NOT CYCLE! (dup atom)
"atom : INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "3", debug);
assertEquals("(3 3)\n", found);
}
@Test public void testCopySemanticsForRules2() throws Exception {
// copy type as a root for each invocation of (...)+ in rewrite
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : type ID (',' ID)* ';' -> ^(type ID)+ ;\n" +
"type : 'int' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a,b,c;", debug);
assertEquals("(int a) (int b) (int c)\n", found);
}
@Test public void testCopySemanticsForRules3() throws Exception {
// copy type *and* modifier even though it's optional
// for each invocation of (...)+ in rewrite
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : modifier? type ID (',' ID)* ';' -> ^(type modifier? ID)+ ;\n" +
"type : 'int' ;\n" +
"modifier : 'public' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "public int a,b,c;", debug);
assertEquals("(int public a) (int public b) (int public c)\n", found);
}
@Test public void testCopySemanticsForRules3Double() throws Exception {
// copy type *and* modifier even though it's optional
// for each invocation of (...)+ in rewrite
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : modifier? type ID (',' ID)* ';' -> ^(type modifier? ID)+ ^(type modifier? ID)+ ;\n" +
"type : 'int' ;\n" +
"modifier : 'public' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "public int a,b,c;", debug);
assertEquals("(int public a) (int public b) (int public c) (int public a) (int public b) (int public c)\n", found);
}
@Test public void testCopySemanticsForRules4() throws Exception {
// copy type *and* modifier even though it's optional
// for each invocation of (...)+ in rewrite
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {MOD;}\n" +
"a : modifier? type ID (',' ID)* ';' -> ^(type ^(MOD modifier)? ID)+ ;\n" +
"type : 'int' ;\n" +
"modifier : 'public' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "public int a,b,c;", debug);
assertEquals("(int (MOD public) a) (int (MOD public) b) (int (MOD public) c)\n", found);
}
@Test public void testCopySemanticsLists() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {MOD;}\n" +
"a : ID (',' ID)* ';' -> ID+ ID+ ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a,b,c;", debug);
assertEquals("a b c a b c\n", found);
}
@Test public void testCopyRuleLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x=b -> $x $x;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a a\n", found);
}
@Test public void testCopyRuleLabel2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x=b -> ^($x $x);\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("(a a)\n", found);
}
@Test public void testQueueingOfTokens() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'int' ID (',' ID)* ';' -> ^('int' ID+) ;\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a,b,c;", debug);
assertEquals("(int a b c)\n", found);
}
@Test public void testCopyOfTokens() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'int' ID ';' -> 'int' ID 'int' ID ;\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a;", debug);
assertEquals("int a int a\n", found);
}
@Test public void testTokenCopyInLoop() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'int' ID (',' ID)* ';' -> ^('int' ID)+ ;\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a,b,c;", debug);
assertEquals("(int a) (int b) (int c)\n", found);
}
@Test public void testTokenCopyInLoopAgainstTwoOthers() throws Exception {
// must smear 'int' copies across as root of multiple trees
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : 'int' ID ':' INT (',' ID ':' INT)* ';' -> ^('int' ID INT)+ ;\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "int a:1,b:2,c:3;", debug);
assertEquals("(int a 1) (int b 2) (int c 3)\n", found);
}
@Test public void testListRefdOneAtATime() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID+ -> ID ID ID ;\n" + // works if 3 input IDs
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b c", debug);
assertEquals("a b c\n", found);
}
@Test public void testSplitListWithLabels() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {VAR;}\n"+
"a : first=ID others+=ID* -> $first VAR $others+ ;\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b c", debug);
assertEquals("a VAR b c\n", found);
}
@Test public void testComplicatedMelange() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : A A b=B B b=B c+=C C c+=C D {String s=$D.text;} -> A+ B+ C+ D ;\n" +
"type : 'int' | 'float' ;\n" +
"A : 'a' ;\n" +
"B : 'b' ;\n" +
"C : 'c' ;\n" +
"D : 'd' ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a a b b b c c c d", debug);
assertEquals("a a b b b c c c d\n", found);
}
@Test public void testRuleLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x=b -> $x;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a\n", found);
}
@Test public void testAmbiguousRule() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID a -> a | INT ;\n"+
"ID : 'a'..'z'+ ;\n" +
"INT: '0'..'9'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("34\n", found);
}
@Test public void testWeirdRuleRef() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID a -> $a | INT ;\n"+
"ID : 'a'..'z'+ ;\n" +
"INT: '0'..'9'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
Grammar g = new Grammar(grammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
// $a is ambig; is it previous root or ref to a ref in alt?
assertEquals("unexpected errors: "+equeue, 1, equeue.errors.size());
}
@Test public void testRuleListLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x+=b x+=b -> $x+;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testRuleListLabel2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x+=b x+=b -> $x $x*;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testOptional() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x=b (y=b)? -> $x $y?;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a\n", found);
}
@Test public void testOptional2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x=ID (y=b)? -> $x $y?;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testOptional3() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x=ID (y=b)? -> ($x $y)?;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testOptional4() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x+=ID (y=b)? -> ($x $y)?;\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("a b\n", found);
}
@Test public void testOptional5() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : ID -> ID? ;\n"+ // match an ID to optional ID
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a\n", found);
}
@Test public void testArbitraryExprType() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : x+=b x+=b -> {new CommonTree()};\n"+
"b : ID ;\n"+
"ID : 'a'..'z'+ ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
assertEquals("", found);
}
@Test public void testSet() throws Exception {
String grammar =
"grammar T;\n" +
"options { output = AST; } \n" +
"a: (INT|ID)+ -> INT+ ID+ ;\n" +
"INT: '0'..'9'+;\n" +
"ID : 'a'..'z'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "2 a 34 de", debug);
assertEquals("2 34 a de\n", found);
}
@Test public void testSet2() throws Exception {
String grammar =
"grammar T;\n" +
"options { output = AST; } \n" +
"a: (INT|ID) -> INT? ID? ;\n" +
"INT: '0'..'9'+;\n" +
"ID : 'a'..'z'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "2", debug);
assertEquals("2\n", found);
}
@Ignore
// TODO: FAILS. The should probably generate a warning from antlr
// See http://www.antlr.org:8888/browse/ANTLR-162
//
public void testSetWithLabel() throws Exception {
String grammar =
"grammar T;\n" +
"options { output = AST; } \n" +
"a : x=(INT|ID) -> $x ;\n" +
"INT: '0'..'9'+;\n" +
"ID : 'a'..'z'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "2", debug);
assertEquals("2\n", found);
}
@Test public void testRewriteAction() throws Exception {
String grammar =
"grammar T; \n" +
"options { output = AST; }\n" +
"tokens { FLOAT; }\n" +
"r\n" +
" : INT -> {new CommonTree(new CommonToken(FLOAT,$INT.text+\".0\"))} \n" +
" ; \n" +
"INT : '0'..'9'+; \n" +
"WS: (' ' | '\\n' | '\\t')+ {$channel = HIDDEN;}; \n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"r", "25", debug);
assertEquals("25.0\n", found);
}
@Test public void testOptionalSubruleWithoutRealElements() throws Exception {
// copy type *and* modifier even though it's optional
// for each invocation of (...)+ in rewrite
String grammar =
"grammar T;\n" +
"options {output=AST;} \n" +
"tokens {PARMS;} \n" +
"\n" +
"modulo \n" +
" : 'modulo' ID ('(' parms+ ')')? -> ^('modulo' ID ^(PARMS parms+)?) \n" +
" ; \n" +
"parms : '#'|ID; \n" +
"ID : ('a'..'z' | 'A'..'Z')+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"modulo", "modulo abc (x y #)", debug);
assertEquals("(modulo abc (PARMS x y #))\n", found);
}
// C A R D I N A L I T Y I S S U E S
@Test public void testCardinality() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"tokens {BLOCK;}\n" +
"a : ID ID INT INT INT -> (ID INT)+;\n"+
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+; \n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b 3 4 5", debug);
String expecting =
"org.antlr.runtime.tree.RewriteCardinalityException: token ID";
String found = getFirstLineOfException();
assertEquals(expecting, found);
}
@Test public void testCardinality2() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID+ -> ID ID ID ;\n" + // only 2 input IDs
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
execParser("T.g", grammar, "TParser", "TLexer",
"a", "a b", debug);
String expecting =
"org.antlr.runtime.tree.RewriteCardinalityException: token ID";
String found = getFirstLineOfException();
assertEquals(expecting, found);
}
@Test public void testCardinality3() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID? INT -> ID INT ;\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
execParser("T.g", grammar, "TParser", "TLexer",
"a", "3", debug);
String expecting =
"org.antlr.runtime.tree.RewriteEmptyStreamException: token ID";
String found = getFirstLineOfException();
assertEquals(expecting, found);
}
@Test public void testLoopCardinality() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID? INT -> ID+ INT ;\n" +
"op : '+'|'-' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
execParser("T.g", grammar, "TParser", "TLexer",
"a", "3", debug);
String expecting =
"org.antlr.runtime.tree.RewriteEarlyExitException";
String found = getFirstLineOfException();
assertEquals(expecting, found);
}
@Test public void testWildcard() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID c=. -> $c;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "abc 34", debug);
assertEquals("34\n", found);
}
// E R R O R S
@Test public void testUnknownRule() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT -> ugh ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
Grammar g = new Grammar(grammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_UNDEFINED_RULE_REF;
Object expectedArg = "ugh";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testKnownRuleButNotInLHS() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT -> b ;\n" +
"b : 'b' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
Grammar g = new Grammar(grammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_REWRITE_ELEMENT_NOT_PRESENT_ON_LHS;
Object expectedArg = "b";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testUnknownToken() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT -> ICK ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
Grammar g = new Grammar(grammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_UNDEFINED_TOKEN_REF_IN_REWRITE;
Object expectedArg = "ICK";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testUnknownLabel() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT -> $foo ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
Grammar g = new Grammar(grammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_UNDEFINED_LABEL_REF_IN_REWRITE;
Object expectedArg = "foo";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testUnknownCharLiteralToken() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT -> 'a' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
Grammar g = new Grammar(grammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_UNDEFINED_TOKEN_REF_IN_REWRITE;
Object expectedArg = "'a'";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testUnknownStringLiteralToken() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : INT -> 'foo' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
Grammar g = new Grammar(grammar);
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_UNDEFINED_TOKEN_REF_IN_REWRITE;
Object expectedArg = "'foo'";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testExtraTokenInSimpleDecl() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"tokens {EXPR;}\n" +
"decl : type ID '=' INT ';' -> ^(EXPR type ID INT) ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"decl", "int 34 x=1;", debug);
assertEquals("line 1:4 extraneous input '34' expecting ID\n", this.stderrDuringParse);
assertEquals("(EXPR int x 1)\n", found); // tree gets correct x and 1 tokens
}
@Test public void testMissingIDInSimpleDecl() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"tokens {EXPR;}\n" +
"decl : type ID '=' INT ';' -> ^(EXPR type ID INT) ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"decl", "int =1;", debug);
assertEquals("line 1:4 missing ID at '='\n", this.stderrDuringParse);
assertEquals("(EXPR int <missing ID> 1)\n", found); // tree gets invented ID token
}
@Test public void testMissingSetInSimpleDecl() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"tokens {EXPR;}\n" +
"decl : type ID '=' INT ';' -> ^(EXPR type ID INT) ;\n" +
"type : 'int' | 'float' ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"decl", "x=1;", debug);
assertEquals("line 1:0 mismatched input 'x' expecting set null\n", this.stderrDuringParse);
assertEquals("(EXPR <error: x> x 1)\n", found); // tree gets invented ID token
}
@Test public void testMissingTokenGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc", debug);
assertEquals("line 0:-1 missing INT at '<EOF>'\n", this.stderrDuringParse);
// doesn't do in-line recovery for sets (yet?)
assertEquals("abc <missing INT>\n", found);
}
@Test public void testExtraTokenGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : b c -> b c;\n" +
"b : ID -> ID ;\n" +
"c : INT -> INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "abc ick 34", debug);
assertEquals("line 1:4 extraneous input 'ick' expecting INT\n", this.stderrDuringParse);
assertEquals("abc 34\n", found);
}
@Test public void testMissingFirstTokenGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : ID INT -> ID INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "34", debug);
assertEquals("line 1:0 missing ID at '34'\n", this.stderrDuringParse);
assertEquals("<missing ID> 34\n", found);
}
@Test public void testMissingFirstTokenGivesErrorNode2() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : b c -> b c;\n" +
"b : ID -> ID ;\n" +
"c : INT -> INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "34", debug);
// finds an error at the first token, 34, and re-syncs.
// re-synchronizing does not consume a token because 34 follows
// ref to rule b (start of c). It then matches 34 in c.
assertEquals("line 1:0 missing ID at '34'\n", this.stderrDuringParse);
assertEquals("<missing ID> 34\n", found);
}
@Test public void testNoViableAltGivesErrorNode() throws Exception {
String grammar =
"grammar foo;\n" +
"options {output=AST;}\n" +
"a : b -> b | c -> c;\n" +
"b : ID -> ID ;\n" +
"c : INT -> INT ;\n" +
"ID : 'a'..'z'+ ;\n" +
"S : '*' ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "*", debug);
// finds an error at the first token, 34, and re-syncs.
// re-synchronizing does not consume a token because 34 follows
// ref to rule b (start of c). It then matches 34 in c.
assertEquals("line 1:0 no viable alternative at input '*'\n", this.stderrDuringParse);
assertEquals("<unexpected: [@0,0:0='*',<6>,1:0], resync=*>\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestAttributes.java 0000644 0001750 0001750 00000370244 11256465234 024641 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.Tool;
import org.antlr.grammar.v3.ActionTranslator;
import org.antlr.codegen.CodeGenerator;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import org.antlr.tool.*;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import org.antlr.grammar.v2.ANTLRParser;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
/** Check the $x, $x.y attributes. For checking the actual
* translation, assume the Java target. This is still a great test
* for the semantics of the $x.y stuff regardless of the target.
*/
public class TestAttributes extends BaseTest {
/** Public default constructor used by TestRig */
public TestAttributes() {
}
@Test public void testEscapedLessThanInAction() throws Exception {
Grammar g = new Grammar();
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
String action = "i<3; '<xmltag>'";
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),0);
String expecting = action;
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, "<action>");
actionST.setAttribute("action", rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
}
@Test public void testEscaped$InAction() throws Exception {
String action = "int \\$n; \"\\$in string\\$\"";
String expecting = "int $n; \"$in string$\"";
Grammar g = new Grammar(
"parser grammar t;\n"+
"@members {"+action+"}\n"+
"a[User u, int i]\n" +
" : {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),0);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
}
@Test public void testArguments() throws Exception {
String action = "$i; $i.x; $u; $u.x";
String expecting = "i; i.x; u; u.x";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[User u, int i]\n" +
" : {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testComplicatedArgParsing() throws Exception {
String action = "x, (*a).foo(21,33), 3.2+1, '\\n', "+
"\"a,oo\\nick\", {bl, \"fdkj\"eck}";
String expecting = "x, (*a).foo(21,33), 3.2+1, '\\n', \"a,oo\\nick\", {bl, \"fdkj\"eck}";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// now check in actual grammar.
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[User u, int i]\n" +
" : A a["+action+"] B\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation = translator.translate();
assertEquals(expecting, rawTranslation);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testBracketArgParsing() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// now check in actual grammar.
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[String[\\] ick, int i]\n" +
" : A \n"+
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
Rule r = g.getRule("a");
AttributeScope parameters = r.parameterScope;
List<Attribute> attrs = parameters.getAttributes();
assertEquals("attribute mismatch","String[] ick",attrs.get(0).decl.toString());
assertEquals("parameter name mismatch","ick",attrs.get(0).name);
assertEquals("declarator mismatch", "String[]", attrs.get(0).type);
assertEquals("attribute mismatch","int i",attrs.get(1).decl.toString());
assertEquals("parameter name mismatch","i",attrs.get(1).name);
assertEquals("declarator mismatch", "int", attrs.get(1).type);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testStringArgParsing() throws Exception {
String action = "34, '{', \"it's<\", '\"', \"\\\"\", 19";
String expecting = "34, '{', \"it's<\", '\"', \"\\\"\", 19";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// now check in actual grammar.
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[User u, int i]\n" +
" : A a["+action+"] B\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation = translator.translate();
assertEquals(expecting, rawTranslation);
List<String> expectArgs = new ArrayList<String>() {
{add("34");}
{add("'{'");}
{add("\"it's<\"");}
{add("'\"'");}
{add("\"\\\"\"");} // that's "\""
{add("19");}
};
List<String> actualArgs = CodeGenerator.getListOfArgumentsFromAction(action, ',');
assertEquals("args mismatch", expectArgs, actualArgs);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testComplicatedSingleArgParsing() throws Exception {
String action = "(*a).foo(21,33,\",\")";
String expecting = "(*a).foo(21,33,\",\")";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// now check in actual grammar.
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[User u, int i]\n" +
" : A a["+action+"] B\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation = translator.translate();
assertEquals(expecting, rawTranslation);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testArgWithLT() throws Exception {
String action = "34<50";
String expecting = "34<50";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// now check in actual grammar.
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[boolean b]\n" +
" : A a["+action+"] B\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
assertEquals(expecting, rawTranslation);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testGenericsAsArgumentDefinition() throws Exception {
String action = "$foo.get(\"ick\");";
String expecting = "foo.get(\"ick\");";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"parser grammar T;\n"+
"a[HashMap<String,String> foo]\n" +
" : {"+action+"}\n" +
" ;";
Grammar g = new Grammar(grammar);
Rule ra = g.getRule("a");
List<Attribute> attrs = ra.parameterScope.getAttributes();
assertEquals("attribute mismatch","HashMap<String,String> foo",attrs.get(0).decl.toString());
assertEquals("parameter name mismatch","foo",attrs.get(0).name);
assertEquals("declarator mismatch", "HashMap<String,String>", attrs.get(0).type);
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testGenericsAsArgumentDefinition2() throws Exception {
String action = "$foo.get(\"ick\"); x=3;";
String expecting = "foo.get(\"ick\"); x=3;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"parser grammar T;\n"+
"a[HashMap<String,String> foo, int x, List<String> duh]\n" +
" : {"+action+"}\n" +
" ;";
Grammar g = new Grammar(grammar);
Rule ra = g.getRule("a");
List<Attribute> attrs = ra.parameterScope.getAttributes();
assertEquals("attribute mismatch","HashMap<String,String> foo",attrs.get(0).decl.toString().trim());
assertEquals("parameter name mismatch","foo",attrs.get(0).name);
assertEquals("declarator mismatch", "HashMap<String,String>", attrs.get(0).type);
assertEquals("attribute mismatch","int x",attrs.get(1).decl.toString().trim());
assertEquals("parameter name mismatch","x",attrs.get(1).name);
assertEquals("declarator mismatch", "int", attrs.get(1).type);
assertEquals("attribute mismatch","List<String> duh",attrs.get(2).decl.toString().trim());
assertEquals("parameter name mismatch","duh",attrs.get(2).name);
assertEquals("declarator mismatch", "List<String>", attrs.get(2).type);
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testGenericsAsReturnValue() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"parser grammar T;\n"+
"a returns [HashMap<String,String> foo] : ;\n";
Grammar g = new Grammar(grammar);
Rule ra = g.getRule("a");
List<Attribute> attrs = ra.returnScope.getAttributes();
assertEquals("attribute mismatch","HashMap<String,String> foo",attrs.get(0).decl.toString());
assertEquals("parameter name mismatch","foo",attrs.get(0).name);
assertEquals("declarator mismatch", "HashMap<String,String>", attrs.get(0).type);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testComplicatedArgParsingWithTranslation() throws Exception {
String action = "x, $A.text+\"3242\", (*$A).foo(21,33), 3.2+1, '\\n', "+
"\"a,oo\\nick\", {bl, \"fdkj\"eck}";
String expecting = "x, (A1!=null?A1.getText():null)+\"3242\", (*A1).foo(21,33), 3.2+1, '\\n', \"a,oo\\nick\", {bl, \"fdkj\"eck}";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
// now check in actual grammar.
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[User u, int i]\n" +
" : A a["+action+"] B\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
/** $x.start refs are checked during translation not before so ANTLR misses
the fact that rule r has refs to predefined attributes if the ref is after
the def of the method or self-referential. Actually would be ok if I didn't
convert actions to strings; keep as templates.
June 9, 2006: made action translation leave templates not strings
*/
@Test public void testRefToReturnValueBeforeRefToPredefinedAttr() throws Exception {
String action = "$x.foo";
String expecting = "(x!=null?x.foo:0)";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : x=b {"+action+"} ;\n" +
"b returns [int foo] : B {$b.start} ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRuleLabelBeforeRefToPredefinedAttr() throws Exception {
// As of Mar 2007, I'm removing unused labels. Unfortunately,
// the action is not seen until code gen. Can't see $x.text
// before stripping unused labels. We really need to translate
// actions first so code gen logic can use info.
String action = "$x.text";
String expecting = "(x!=null?input.toString(x.start,x.stop):null)";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : x=b {"+action+"} ;\n" +
"b : B ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testInvalidArguments() throws Exception {
String action = "$x";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[User u, int i]\n" +
" : {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_UNKNOWN_SIMPLE_ATTRIBUTE;
Object expectedArg = "x";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testReturnValue() throws Exception {
String action = "$x.i";
String expecting = "x";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a returns [int i]\n" +
" : 'a'\n" +
" ;\n" +
"b : x=a {"+action+"} ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testReturnValueWithNumber() throws Exception {
String action = "$x.i1";
String expecting = "x";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a returns [int i1]\n" +
" : 'a'\n" +
" ;\n" +
"b : x=a {"+action+"} ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testReturnValues() throws Exception {
String action = "$i; $i.x; $u; $u.x";
String expecting = "retval.i; retval.i.x; retval.u; retval.u.x";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a returns [User u, int i]\n" +
" : {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
/* regression test for ANTLR-46 */
@Test public void testReturnWithMultipleRuleRefs() throws Exception {
String action1 = "$obj = $rule2.obj;";
String action2 = "$obj = $rule3.obj;";
String expecting1 = "obj = rule21;";
String expecting2 = "obj = rule32;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"rule1 returns [ Object obj ]\n" +
": rule2 { "+action1+" }\n" +
"| rule3 { "+action2+" }\n" +
";\n"+
"rule2 returns [ Object obj ]\n"+
": foo='foo' { $obj = $foo.text; }\n"+
";\n"+
"rule3 returns [ Object obj ]\n"+
": bar='bar' { $obj = $bar.text; }\n"+
";");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
int i = 0;
String action = action1;
String expecting = expecting1;
do {
ActionTranslator translator = new ActionTranslator(generator,"rule1",
new antlr.CommonToken(ANTLRParser.ACTION,action),i+1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
action = action2;
expecting = expecting2;
} while (i++ < 1);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testInvalidReturnValues() throws Exception {
String action = "$x";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a returns [User u, int i]\n" +
" : {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_UNKNOWN_SIMPLE_ATTRIBUTE;
Object expectedArg = "x";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testTokenLabels() throws Exception {
String action = "$id; $f; $id.text; $id.getText(); $id.dork " +
"$id.type; $id.line; $id.pos; " +
"$id.channel; $id.index;";
String expecting = "id; f; (id!=null?id.getText():null); id.getText(); id.dork (id!=null?id.getType():0); (id!=null?id.getLine():0); (id!=null?id.getCharPositionInLine():0); (id!=null?id.getChannel():0); (id!=null?id.getTokenIndex():0);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : id=ID f=FLOAT {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRuleLabels() throws Exception {
String action = "$r.x; $r.start;\n $r.stop;\n $r.tree; $a.x; $a.stop;";
String expecting = "(r!=null?r.x:0); (r!=null?((Token)r.start):null);\n" +
" (r!=null?((Token)r.stop):null);\n" +
" (r!=null?((Object)r.tree):null); (r!=null?r.x:0); (r!=null?((Token)r.stop):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a returns [int x]\n" +
" :\n" +
" ;\n"+
"b : r=a {###"+action+"!!!}\n" +
" ;");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // codegen phase sets some vars we need
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testAmbiguRuleRef() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A a {$a.text} | B ;");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
// error(132): <string>:2:9: reference $a is ambiguous; rule a is enclosing rule and referenced in the production
assertEquals("unexpected errors: "+equeue, 1, equeue.errors.size());
}
@Test public void testRuleLabelsWithSpecialToken() throws Exception {
String action = "$r.x; $r.start; $r.stop; $r.tree; $a.x; $a.stop;";
String expecting = "(r!=null?r.x:0); (r!=null?((MYTOKEN)r.start):null); (r!=null?((MYTOKEN)r.stop):null); (r!=null?((Object)r.tree):null); (r!=null?r.x:0); (r!=null?((MYTOKEN)r.stop):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"options {TokenLabelType=MYTOKEN;}\n"+
"a returns [int x]\n" +
" :\n" +
" ;\n"+
"b : r=a {###"+action+"!!!}\n" +
" ;");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // codegen phase sets some vars we need
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testForwardRefRuleLabels() throws Exception {
String action = "$r.x; $r.start; $r.stop; $r.tree; $a.x; $a.tree;";
String expecting = "(r!=null?r.x:0); (r!=null?((Token)r.start):null); (r!=null?((Token)r.stop):null); (r!=null?((Object)r.tree):null); (r!=null?r.x:0); (r!=null?((Object)r.tree):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"b : r=a {###"+action+"!!!}\n" +
" ;\n" +
"a returns [int x]\n" +
" : ;\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // codegen phase sets some vars we need
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testInvalidRuleLabelAccessesParameter() throws Exception {
String action = "$r.z";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[int z] returns [int x]\n" +
" :\n" +
" ;\n"+
"b : r=a[3] {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_INVALID_RULE_PARAMETER_REF;
Object expectedArg = "a";
Object expectedArg2 = "z";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testInvalidRuleLabelAccessesScopeAttribute() throws Exception {
String action = "$r.n";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a\n" +
"scope { int n; }\n" +
" :\n" +
" ;\n"+
"b : r=a[3] {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_INVALID_RULE_SCOPE_ATTRIBUTE_REF;
Object expectedArg = "a";
Object expectedArg2 = "n";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testInvalidRuleAttribute() throws Exception {
String action = "$r.blort";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[int z] returns [int x]\n" +
" :\n" +
" ;\n"+
"b : r=a[3] {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_UNKNOWN_RULE_ATTRIBUTE;
Object expectedArg = "a";
Object expectedArg2 = "blort";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testMissingRuleAttribute() throws Exception {
String action = "$r";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[int z] returns [int x]\n" +
" :\n" +
" ;\n"+
"b : r=a[3] {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
int expectedMsgID = ErrorManager.MSG_ISOLATED_RULE_SCOPE;
Object expectedArg = "r";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testMissingUnlabeledRuleAttribute() throws Exception {
String action = "$a";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a returns [int x]:\n" +
" ;\n"+
"b : a {"+action+"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
int expectedMsgID = ErrorManager.MSG_ISOLATED_RULE_SCOPE;
Object expectedArg = "a";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testNonDynamicAttributeOutsideRule() throws Exception {
String action = "public void foo() { $x; }";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"@members {'+action+'}\n" +
"a : ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator,
null,
new antlr.CommonToken(ANTLRParser.ACTION,action),0);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_ATTRIBUTE_REF_NOT_IN_RULE;
Object expectedArg = "x";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testNonDynamicAttributeOutsideRule2() throws Exception {
String action = "public void foo() { $x.y; }";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"@members {'+action+'}\n" +
"a : ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator = new ActionTranslator(generator,
null,
new antlr.CommonToken(ANTLRParser.ACTION,action),0);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_ATTRIBUTE_REF_NOT_IN_RULE;
Object expectedArg = "x";
Object expectedArg2 = "y";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
// D Y N A M I C A L L Y S C O P E D A T T R I B U T E S
@Test public void testBasicGlobalScope() throws Exception {
String action = "$Symbols::names.add($id.text);";
String expecting = "((Symbols_scope)Symbols_stack.peek()).names.add((id!=null?id.getText():null));";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" int n;\n" +
" List names;\n" +
"}\n" +
"a scope Symbols; : (id=ID ';' {"+action+"} )+\n" +
" ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testUnknownGlobalScope() throws Exception {
String action = "$Symbols::names.add($id.text);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a scope Symbols; : (id=ID ';' {"+action+"} )+\n" +
" ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
assertEquals("unexpected errors: "+equeue, 2, equeue.errors.size());
int expectedMsgID = ErrorManager.MSG_UNKNOWN_DYNAMIC_SCOPE;
Object expectedArg = "Symbols";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testIndexedGlobalScope() throws Exception {
String action = "$Symbols[-1]::names.add($id.text);";
String expecting =
"((Symbols_scope)Symbols_stack.elementAt(Symbols_stack.size()-1-1)).names.add((id!=null?id.getText():null));";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" int n;\n" +
" List names;\n" +
"}\n" +
"a scope Symbols; : (id=ID ';' {"+action+"} )+\n" +
" ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void test0IndexedGlobalScope() throws Exception {
String action = "$Symbols[0]::names.add($id.text);";
String expecting =
"((Symbols_scope)Symbols_stack.elementAt(0)).names.add((id!=null?id.getText():null));";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" int n;\n" +
" List names;\n" +
"}\n" +
"a scope Symbols; : (id=ID ';' {"+action+"} )+\n" +
" ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
assertEquals(expecting, rawTranslation);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testAbsoluteIndexedGlobalScope() throws Exception {
String action = "$Symbols[3]::names.add($id.text);";
String expecting =
"((Symbols_scope)Symbols_stack.elementAt(3)).names.add((id!=null?id.getText():null));";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" int n;\n" +
" List names;\n" +
"}\n" +
"a scope Symbols; : (id=ID ';' {"+action+"} )+\n" +
" ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
assertEquals(expecting, rawTranslation);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testScopeAndAttributeWithUnderscore() throws Exception {
String action = "$foo_bar::a_b;";
String expecting = "((foo_bar_scope)foo_bar_stack.peek()).a_b;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope foo_bar {\n" +
" int a_b;\n" +
"}\n" +
"a scope foo_bar; : (ID {"+action+"} )+\n" +
" ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testSharedGlobalScope() throws Exception {
String action = "$Symbols::x;";
String expecting = "((Symbols_scope)Symbols_stack.peek()).x;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" String x;\n" +
"}\n" +
"a\n"+
"scope { int y; }\n"+
"scope Symbols;\n" +
" : b {"+action+"}\n" +
" ;\n" +
"b : ID {$Symbols::x=$ID.text} ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testGlobalScopeOutsideRule() throws Exception {
String action = "public void foo() {$Symbols::names.add('foo');}";
String expecting = "public void foo() {((Symbols_scope)Symbols_stack.peek()).names.add('foo');}";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" int n;\n" +
" List names;\n" +
"}\n" +
"@members {'+action+'}\n" +
"a : \n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRuleScopeOutsideRule() throws Exception {
String action = "public void foo() {$a::name;}";
String expecting = "public void foo() {((a_scope)a_stack.peek()).name;}";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"@members {"+action+"}\n" +
"a\n" +
"scope { String name; }\n" +
" : {foo();}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
null,
new antlr.CommonToken(ANTLRParser.ACTION,action),0);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testBasicRuleScope() throws Exception {
String action = "$a::n;";
String expecting = "((a_scope)a_stack.peek()).n;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a\n" +
"scope {\n" +
" int n;\n" +
"} : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testUnqualifiedRuleScopeAccessInsideRule() throws Exception {
String action = "$n;";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a\n" +
"scope {\n" +
" int n;\n" +
"} : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
int expectedMsgID = ErrorManager.MSG_ISOLATED_RULE_ATTRIBUTE;
Object expectedArg = "n";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg,
expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testIsolatedDynamicRuleScopeRef() throws Exception {
String action = "$a;"; // refers to stack not top of stack
String expecting = "a_stack;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a\n" +
"scope {\n" +
" int n;\n" +
"} : b ;\n" +
"b : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testDynamicRuleScopeRefInSubrule() throws Exception {
String action = "$a::n;";
String expecting = "((a_scope)a_stack.peek()).n;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a\n" +
"scope {\n" +
" float n;\n" +
"} : b ;\n" +
"b : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testIsolatedGlobalScopeRef() throws Exception {
String action = "$Symbols;";
String expecting = "Symbols_stack;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" String x;\n" +
"}\n" +
"a\n"+
"scope { int y; }\n"+
"scope Symbols;\n" +
" : b {"+action+"}\n" +
" ;\n" +
"b : ID {$Symbols::x=$ID.text} ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRuleScopeFromAnotherRule() throws Exception {
String action = "$a::n;"; // must be qualified
String expecting = "((a_scope)a_stack.peek()).n;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a\n" +
"scope {\n" +
" boolean n;\n" +
"} : b\n" +
" ;\n" +
"b : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testFullyQualifiedRefToCurrentRuleParameter() throws Exception {
String action = "$a.i;";
String expecting = "i;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a[int i]: {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testFullyQualifiedRefToCurrentRuleRetVal() throws Exception {
String action = "$a.i;";
String expecting = "retval.i;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a returns [int i, int j]: {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testSetFullyQualifiedRefToCurrentRuleRetVal() throws Exception {
String action = "$a.i = 1;";
String expecting = "retval.i = 1;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a returns [int i, int j]: {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testIsolatedRefToCurrentRule() throws Exception {
String action = "$a;";
String expecting = "";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : 'a' {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
int expectedMsgID = ErrorManager.MSG_ISOLATED_RULE_SCOPE;
Object expectedArg = "a";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg,
expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testIsolatedRefToRule() throws Exception {
String action = "$x;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : x=b {"+action+"}\n" +
" ;\n" +
"b : 'b' ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
int expectedMsgID = ErrorManager.MSG_ISOLATED_RULE_SCOPE;
Object expectedArg = "x";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
/* I think these have to be errors $a.x makes no sense.
@Test public void testFullyQualifiedRefToLabelInCurrentRule() throws Exception {
String action = "$a.x;";
String expecting = "x;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : x='a' {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testFullyQualifiedRefToListLabelInCurrentRule() throws Exception {
String action = "$a.x;"; // must be qualified
String expecting = "list_x;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : x+='a' {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
*/
@Test public void testFullyQualifiedRefToTemplateAttributeInCurrentRule() throws Exception {
String action = "$a.st;"; // can be qualified
String expecting = "retval.st;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n" +
"options {output=template;}\n"+
"a : (A->{$A.text}) {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRuleRefWhenRuleHasScope() throws Exception {
String action = "$b.start;";
String expecting = "(b1!=null?((Token)b1.start):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"a : b {###"+action+"!!!} ;\n" +
"b\n" +
"scope {\n" +
" int n;\n" +
"} : 'b' \n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testDynamicScopeRefOkEvenThoughRuleRefExists() throws Exception {
String action = "$b::n;";
String expecting = "((b_scope)b_stack.peek()).n;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"s : b ;\n"+
"b\n" +
"scope {\n" +
" int n;\n" +
"} : '(' b ')' {"+action+"}\n" + // refers to current invocation's n
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRefToTemplateAttributeForCurrentRule() throws Exception {
String action = "$st=null;";
String expecting = "retval.st =null;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n" +
"options {output=template;}\n"+
"a : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRefToTextAttributeForCurrentRule() throws Exception {
String action = "$text";
String expecting = "input.toString(retval.start,input.LT(-1))";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n" +
"options {output=template;}\n"+
"a : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRefToStartAttributeForCurrentRule() throws Exception {
String action = "$start;";
String expecting = "((Token)retval.start);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n" +
"a : {###"+action+"!!!}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testTokenLabelFromMultipleAlts() throws Exception {
String action = "$ID.text;"; // must be qualified
String action2 = "$INT.text;"; // must be qualified
String expecting = "(ID1!=null?ID1.getText():null);";
String expecting2 = "(INT2!=null?INT2.getText():null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ID {"+action+"}\n" +
" | INT {"+action2+"}\n" +
" ;\n" +
"ID : 'a';\n" +
"INT : '0';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
translator = new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action2),2);
rawTranslation =
translator.translate();
templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
actionST = new StringTemplate(templates, rawTranslation);
found = actionST.toString();
assertEquals(expecting2, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRuleLabelFromMultipleAlts() throws Exception {
String action = "$b.text;"; // must be qualified
String action2 = "$c.text;"; // must be qualified
String expecting = "(b1!=null?input.toString(b1.start,b1.stop):null);";
String expecting2 = "(c2!=null?input.toString(c2.start,c2.stop):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : b {"+action+"}\n" +
" | c {"+action2+"}\n" +
" ;\n" +
"b : 'a';\n" +
"c : '0';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
translator = new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action2),2);
rawTranslation =
translator.translate();
templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
actionST = new StringTemplate(templates, rawTranslation);
found = actionST.toString();
assertEquals(expecting2, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testUnknownDynamicAttribute() throws Exception {
String action = "$a::x";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a\n" +
"scope {\n" +
" int n;\n" +
"} : {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_UNKNOWN_DYNAMIC_SCOPE_ATTRIBUTE;
Object expectedArg = "a";
Object expectedArg2 = "x";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testUnknownGlobalDynamicAttribute() throws Exception {
String action = "$Symbols::x";
String expecting = action;
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"scope Symbols {\n" +
" int n;\n" +
"}\n" +
"a : {'+action+'}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_UNKNOWN_DYNAMIC_SCOPE_ATTRIBUTE;
Object expectedArg = "Symbols";
Object expectedArg2 = "x";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testUnqualifiedRuleScopeAttribute() throws Exception {
String action = "$n;"; // must be qualified
String expecting = "$n;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a\n" +
"scope {\n" +
" int n;\n" +
"} : b\n" +
" ;\n" +
"b : {'+action+'}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator =
new ActionTranslator(generator,
"b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_UNKNOWN_SIMPLE_ATTRIBUTE;
Object expectedArg = "n";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testRuleAndTokenLabelTypeMismatch() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : id='foo' id=b\n" +
" ;\n" +
"b : ;\n");
int expectedMsgID = ErrorManager.MSG_LABEL_TYPE_CONFLICT;
Object expectedArg = "id";
Object expectedArg2 = "rule!=token";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testListAndTokenLabelTypeMismatch() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ids+='a' ids='b'\n" +
" ;\n" +
"b : ;\n");
int expectedMsgID = ErrorManager.MSG_LABEL_TYPE_CONFLICT;
Object expectedArg = "ids";
Object expectedArg2 = "token!=token-list";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testListAndRuleLabelTypeMismatch() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"options {output=AST;}\n"+
"a : bs+=b bs=b\n" +
" ;\n" +
"b : 'b';\n");
int expectedMsgID = ErrorManager.MSG_LABEL_TYPE_CONFLICT;
Object expectedArg = "bs";
Object expectedArg2 = "rule!=rule-list";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testArgReturnValueMismatch() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a[int i] returns [int x, int i]\n" +
" : \n" +
" ;\n" +
"b : ;\n");
int expectedMsgID = ErrorManager.MSG_ARG_RETVAL_CONFLICT;
Object expectedArg = "i";
Object expectedArg2 = "a";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testSimplePlusEqualLabel() throws Exception {
String action = "$ids.size();"; // must be qualified
String expecting = "list_ids.size();";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ids+=ID ( COMMA ids+=ID {"+action+"})* ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testPlusEqualStringLabel() throws Exception {
String action = "$ids.size();"; // must be qualified
String expecting = "list_ids.size();";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ids+='if' ( ',' ids+=ID {"+action+"})* ;" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testPlusEqualSetLabel() throws Exception {
String action = "$ids.size();"; // must be qualified
String expecting = "list_ids.size();";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ids+=('a'|'b') ( ',' ids+=ID {"+action+"})* ;" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testPlusEqualWildcardLabel() throws Exception {
String action = "$ids.size();"; // must be qualified
String expecting = "list_ids.size();";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ids+=. ( ',' ids+=ID {"+action+"})* ;" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testImplicitTokenLabel() throws Exception {
String action = "$ID; $ID.text; $ID.getText()";
String expecting = "ID1; (ID1!=null?ID1.getText():null); ID1.getText()";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ID {"+action+"} ;" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testImplicitRuleLabel() throws Exception {
String action = "$r.start;";
String expecting = "(r1!=null?((Token)r1.start):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : r {###"+action+"!!!} ;" +
"r : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testReuseExistingLabelWithImplicitRuleLabel() throws Exception {
String action = "$r.start;";
String expecting = "(x!=null?((Token)x.start):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : x=r {###"+action+"!!!} ;" +
"r : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testReuseExistingListLabelWithImplicitRuleLabel() throws Exception {
String action = "$r.start;";
String expecting = "(x!=null?((Token)x.start):null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"options {output=AST;}\n" +
"a : x+=r {###"+action+"!!!} ;" +
"r : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testReuseExistingLabelWithImplicitTokenLabel() throws Exception {
String action = "$ID.text;";
String expecting = "(x!=null?x.getText():null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : x=ID {"+action+"} ;" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testReuseExistingListLabelWithImplicitTokenLabel() throws Exception {
String action = "$ID.text;";
String expecting = "(x!=null?x.getText():null);";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : x+=ID {"+action+"} ;" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRuleLabelWithoutOutputOption() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar T;\n"+
"s : x+=a ;" +
"a : 'a';\n"+
"b : 'b';\n"+
"WS : ' '|'\n';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_LIST_LABEL_INVALID_UNLESS_RETVAL_STRUCT;
Object expectedArg = "x";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testRuleLabelOnTwoDifferentRulesAST() throws Exception {
String grammar =
"grammar T;\n"+
"options {output=AST;}\n"+
"s : x+=a x+=b {System.out.println($x);} ;" +
"a : 'a';\n"+
"b : 'b';\n"+
"WS : (' '|'\n') {skip();};\n";
String expecting = "[a, b]\na b\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "a b", false);
assertEquals(expecting, found);
}
@Test public void testRuleLabelOnTwoDifferentRulesTemplate() throws Exception {
String grammar =
"grammar T;\n"+
"options {output=template;}\n"+
"s : x+=a x+=b {System.out.println($x);} ;" +
"a : 'a' -> {%{\"hi\"}} ;\n"+
"b : 'b' -> {%{\"mom\"}} ;\n"+
"WS : (' '|'\n') {skip();};\n";
String expecting = "[hi, mom]\n";
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "a b", false);
assertEquals(expecting, found);
}
@Test public void testMissingArgs() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : r ;" +
"r[int i] : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_MISSING_RULE_ARGS;
Object expectedArg = "r";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testArgsWhenNoneDefined() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : r[32,34] ;" +
"r : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_RULE_HAS_NO_ARGS;
Object expectedArg = "r";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testReturnInitValue() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : r ;\n" +
"r returns [int x=0] : 'a' {$x = 4;} ;\n");
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
Rule r = g.getRule("r");
AttributeScope retScope = r.returnScope;
List parameters = retScope.getAttributes();
assertNotNull("missing return action", parameters);
assertEquals(1, parameters.size());
String found = parameters.get(0).toString();
String expecting = "int x=0";
assertEquals(expecting, found);
}
@Test public void testMultipleReturnInitValue() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : r ;\n" +
"r returns [int x=0, int y, String s=new String(\"foo\")] : 'a' {$x = 4;} ;\n");
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
Rule r = g.getRule("r");
AttributeScope retScope = r.returnScope;
List parameters = retScope.getAttributes();
assertNotNull("missing return action", parameters);
assertEquals(3, parameters.size());
assertEquals("int x=0", parameters.get(0).toString());
assertEquals("int y", parameters.get(1).toString());
assertEquals("String s=new String(\"foo\")", parameters.get(2).toString());
}
@Test public void testCStyleReturnInitValue() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : r ;\n" +
"r returns [int (*x)()=NULL] : 'a' ;\n");
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
Rule r = g.getRule("r");
AttributeScope retScope = r.returnScope;
List parameters = retScope.getAttributes();
assertNotNull("missing return action", parameters);
assertEquals(1, parameters.size());
String found = parameters.get(0).toString();
String expecting = "int (*)() x=NULL";
assertEquals(expecting, found);
}
@Test public void testArgsWithInitValues() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : r[32,34] ;" +
"r[int x, int y=3] : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_ARG_INIT_VALUES_ILLEGAL;
Object expectedArg = "y";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testArgsOnToken() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ID[32,34] ;" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_ARGS_ON_TOKEN_REF;
Object expectedArg = "ID";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testArgsOnTokenInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : 'z' ID[32,34] ;" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_RULE_HAS_NO_ARGS;
Object expectedArg = "ID";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testLabelOnRuleRefInLexer() throws Exception {
String action = "$i.text";
String expecting = "(i!=null?i.getText():null)";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : 'z' i=ID {"+action+"};" +
"fragment ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"R",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRefToRuleRefInLexer() throws Exception {
String action = "$ID.text";
String expecting = "(ID1!=null?ID1.getText():null)";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : 'z' ID {"+action+"};" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"R",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testRefToRuleRefInLexerNoAttribute() throws Exception {
String action = "$ID";
String expecting = "ID1";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : 'z' ID {"+action+"};" +
"ID : 'a';\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"R",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testCharLabelInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : x='z' ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testCharListLabelInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : x+='z' ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testWildcardCharLabelInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : x=. ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testWildcardCharListLabelInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : x+=. ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testMissingArgsInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : R ;" +
"R[int i] : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_MISSING_RULE_ARGS;
Object expectedArg = "R";
Object expectedArg2 = null;
// getting a second error @1:12, probably from nextToken
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testLexerRulePropertyRefs() throws Exception {
String action = "$text $type $line $pos $channel $index $start $stop";
String expecting = "getText() _type state.tokenStartLine state.tokenStartCharPositionInLine _channel -1 state.tokenStartCharIndex (getCharIndex()-1)";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : 'r' {"+action+"};\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"R",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testLexerLabelRefs() throws Exception {
String action = "$a $b.text $c $d.text";
String expecting = "a (b!=null?b.getText():null) c (d!=null?d.getText():null)";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : a='c' b='hi' c=. d=DUH {"+action+"};\n" +
"DUH : 'd' ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"R",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testSettingLexerRulePropertyRefs() throws Exception {
String action = "$text $type=1 $line=1 $pos=1 $channel=1 $index";
String expecting = "getText() _type=1 state.tokenStartLine=1 state.tokenStartCharPositionInLine=1 _channel=1 -1";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar t;\n"+
"R : 'r' {"+action+"};\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator =
new ActionTranslator(generator,
"R",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testArgsOnTokenInLexerRuleOfCombined() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : R;\n" +
"R : 'z' ID[32] ;\n" +
"ID : 'a';\n");
String lexerGrammarStr = g.getLexerGrammar();
StringReader sr = new StringReader(lexerGrammarStr);
Grammar lexerGrammar = new Grammar();
lexerGrammar.setFileName("<internally-generated-lexer>");
lexerGrammar.importTokenVocabulary(g);
lexerGrammar.parseAndBuildAST(sr);
lexerGrammar.defineGrammarSymbols();
lexerGrammar.checkNameSpaceAndActions();
sr.close();
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, lexerGrammar, "Java");
lexerGrammar.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_RULE_HAS_NO_ARGS;
Object expectedArg = "ID";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, lexerGrammar, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testMissingArgsOnTokenInLexerRuleOfCombined() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : R;\n" +
"R : 'z' ID ;\n" +
"ID[int i] : 'a';\n");
String lexerGrammarStr = g.getLexerGrammar();
StringReader sr = new StringReader(lexerGrammarStr);
Grammar lexerGrammar = new Grammar();
lexerGrammar.setFileName("<internally-generated-lexer>");
lexerGrammar.importTokenVocabulary(g);
lexerGrammar.parseAndBuildAST(sr);
lexerGrammar.defineGrammarSymbols();
lexerGrammar.checkNameSpaceAndActions();
sr.close();
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, lexerGrammar, "Java");
lexerGrammar.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_MISSING_RULE_ARGS;
Object expectedArg = "ID";
Object expectedArg2 = null;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, lexerGrammar, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
// T R E E S
@Test public void testTokenLabelTreeProperty() throws Exception {
String action = "$id.tree;";
String expecting = "id_tree;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : id=ID {"+action+"} ;\n" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
ActionTranslator translator =
new ActionTranslator(generator,
"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testTokenRefTreeProperty() throws Exception {
String action = "$ID.tree;";
String expecting = "ID1_tree;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ID {"+action+"} ;" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
ActionTranslator translator = new ActionTranslator(generator,"a",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
}
@Test public void testAmbiguousTokenRef() throws Exception {
String action = "$ID;";
String expecting = "";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ID ID {"+action+"};" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_NONUNIQUE_REF;
Object expectedArg = "ID";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testAmbiguousTokenRefWithProp() throws Exception {
String action = "$ID.text;";
String expecting = "";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n"+
"a : ID ID {"+action+"};" +
"ID : 'a';\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
int expectedMsgID = ErrorManager.MSG_NONUNIQUE_REF;
Object expectedArg = "ID";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkError(equeue, expectedMessage);
}
@Test public void testRuleRefWithDynamicScope() throws Exception {
String action = "$field::x = $field.st;";
String expecting = "((field_scope)field_stack.peek()).x = retval.st;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"field\n" +
"scope { StringTemplate x; }\n" +
" : 'y' {"+action+"}\n" +
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"field",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testAssignToOwnRulenameAttr() throws Exception {
String action = "$rule.tree = null;";
String expecting = "retval.tree = null;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"rule\n" +
" : 'y' {" + action +"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testAssignToOwnParamAttr() throws Exception {
String action = "$rule.i = 42; $i = 23;";
String expecting = "i = 42; i = 23;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"rule[int i]\n" +
" : 'y' {" + action +"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testIllegalAssignToOwnRulenameAttr() throws Exception {
String action = "$rule.stop = 0;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"rule\n" +
" : 'y' {" + action +"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
int expectedMsgID = ErrorManager.MSG_WRITE_TO_READONLY_ATTR;
Object expectedArg = "rule";
Object expectedArg2 = "stop";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testIllegalAssignToLocalAttr() throws Exception {
String action = "$tree = null; $st = null; $start = 0; $stop = 0; $text = 0;";
String expecting = "retval.tree = null; retval.st = null; ";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"rule\n" +
" : 'y' {" + action +"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
int expectedMsgID = ErrorManager.MSG_WRITE_TO_READONLY_ATTR;
ArrayList expectedErrors = new ArrayList(3);
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, "start", "");
expectedErrors.add(expectedMessage);
GrammarSemanticsMessage expectedMessage2 =
new GrammarSemanticsMessage(expectedMsgID, g, null, "stop", "");
expectedErrors.add(expectedMessage2);
GrammarSemanticsMessage expectedMessage3 =
new GrammarSemanticsMessage(expectedMsgID, g, null, "text", "");
expectedErrors.add(expectedMessage3);
checkErrors(equeue, expectedErrors);
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
}
@Test public void testIllegalAssignRuleRefAttr() throws Exception {
String action = "$other.tree = null;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"options { output = AST;}" +
"otherrule\n" +
" : 'y' ;" +
"rule\n" +
" : other=otherrule {" + action +"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
int expectedMsgID = ErrorManager.MSG_WRITE_TO_READONLY_ATTR;
Object expectedArg = "other";
Object expectedArg2 = "tree";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testIllegalAssignTokenRefAttr() throws Exception {
String action = "$ID.text = \"test\";";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"ID\n" +
" : 'y' ;" +
"rule\n" +
" : ID {" + action +"}\n" +
" ;");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
int expectedMsgID = ErrorManager.MSG_WRITE_TO_READONLY_ATTR;
Object expectedArg = "ID";
Object expectedArg2 = "text";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkError(equeue, expectedMessage);
}
@Test public void testAssignToTreeNodeAttribute() throws Exception {
String action = "$tree.scope = localScope;";
String expecting = "(()retval.tree).scope = localScope;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"options { output=AST; }" +
"rule\n" +
"@init {\n" +
" Scope localScope=null;\n" +
"}\n" +
"@after {\n" +
" $tree.scope = localScope;\n" +
"}\n" +
" : 'a' -> ^('a')\n" +
";");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator,
"rule",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
assertEquals(expecting, found);
}
@Test public void testDoNotTranslateAttributeCompare() throws Exception {
String action = "$a.line == $b.line";
String expecting = "(a!=null?a.getLine():0) == (b!=null?b.getLine():0)";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"lexer grammar a;\n" +
"RULE:\n" +
" a=ID b=ID {" + action + "}" +
" ;\n" +
"ID : 'id';"
);
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
ActionTranslator translator = new ActionTranslator(generator,
"RULE",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
assertEquals(expecting, found);
}
@Test public void testDoNotTranslateScopeAttributeCompare() throws Exception {
String action = "if ($rule::foo == \"foo\" || 1) { System.out.println(\"ouch\"); }";
String expecting = "if (((rule_scope)rule_stack.peek()).foo == \"foo\" || 1) { System.out.println(\"ouch\"); }";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar a;\n" +
"rule\n" +
"scope {\n" +
" String foo;" +
"} :\n" +
" twoIDs" +
" ;\n" +
"twoIDs:\n" +
" ID ID {" + action + "}\n" +
" ;\n" +
"ID : 'id';"
);
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer();
ActionTranslator translator = new ActionTranslator(generator,
"twoIDs",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
// check that we didn't use scopeSetAttributeRef int translation!
boolean foundScopeSetAttributeRef = false;
for (int i = 0; i < translator.chunks.size(); i++) {
Object chunk = translator.chunks.get(i);
if (chunk instanceof StringTemplate) {
if (((StringTemplate)chunk).getName().equals("scopeSetAttributeRef")) {
foundScopeSetAttributeRef = true;
}
}
}
assertFalse("action translator used scopeSetAttributeRef template in comparison!", foundScopeSetAttributeRef);
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
assertEquals(expecting, found);
}
@Test public void testTreeRuleStopAttributeIsInvalid() throws Exception {
String action = "$r.x; $r.start; $r.stop";
String expecting = "(r!=null?r.x:0); (r!=null?((CommonTree)r.start):null); $r.stop";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"tree grammar t;\n" +
"options {ASTLabelType=CommonTree;}\n"+
"a returns [int x]\n" +
" :\n" +
" ;\n"+
"b : r=a {###"+action+"!!!}\n" +
" ;");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // codegen phase sets some vars we need
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
int expectedMsgID = ErrorManager.MSG_UNKNOWN_RULE_ATTRIBUTE;
Object expectedArg = "a";
Object expectedArg2 = "stop";
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
System.out.println("equeue:"+equeue);
checkError(equeue, expectedMessage);
}
@Test public void testRefToTextAttributeForCurrentTreeRule() throws Exception {
String action = "$text";
String expecting = "input.getTokenStream().toString(\n" +
" input.getTreeAdaptor().getTokenStartIndex(retval.start),\n" +
" input.getTreeAdaptor().getTokenStopIndex(retval.start))";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"tree grammar t;\n" +
"options {ASTLabelType=CommonTree;}\n" +
"a : {###"+action+"!!!}\n" +
" ;\n");
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // codegen phase sets some vars we need
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
String found = code.substring(code.indexOf("###")+3,code.indexOf("!!!"));
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testTypeOfGuardedAttributeRefIsCorrect() throws Exception {
String action = "int x = $b::n;";
String expecting = "int x = ((b_scope)b_stack.peek()).n;";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar t;\n" +
"s : b ;\n"+
"b\n" +
"scope {\n" +
" int n;\n" +
"} : '(' b ')' {"+action+"}\n" + // refers to current invocation's n
" ;\n");
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // forces load of templates
ActionTranslator translator = new ActionTranslator(generator, "b",
new antlr.CommonToken(ANTLRParser.ACTION,action),1);
String rawTranslation =
translator.translate();
StringTemplateGroup templates =
new StringTemplateGroup(".", AngleBracketTemplateLexer.class);
StringTemplate actionST = new StringTemplate(templates, rawTranslation);
String found = actionST.toString();
assertEquals(expecting, found);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
// S U P P O R T
protected void checkError(ErrorQueue equeue,
GrammarSemanticsMessage expectedMessage)
throws Exception
{
/*
System.out.println(equeue.infos);
System.out.println(equeue.warnings);
System.out.println(equeue.errors);
*/
Message foundMsg = null;
for (int i = 0; i < equeue.errors.size(); i++) {
Message m = (Message)equeue.errors.get(i);
if (m.msgID==expectedMessage.msgID ) {
foundMsg = m;
}
}
assertTrue("no error; "+expectedMessage.msgID+" expected", equeue.errors.size() > 0);
assertNotNull("couldn't find expected error: "+expectedMessage.msgID+" in "+equeue, foundMsg);
assertTrue("error is not a GrammarSemanticsMessage",
foundMsg instanceof GrammarSemanticsMessage);
assertEquals(expectedMessage.arg, foundMsg.arg);
assertEquals(expectedMessage.arg2, foundMsg.arg2);
}
/** Allow checking for multiple errors in one test */
protected void checkErrors(ErrorQueue equeue,
ArrayList expectedMessages)
throws Exception
{
ArrayList messageExpected = new ArrayList(equeue.errors.size());
for (int i = 0; i < equeue.errors.size(); i++) {
Message m = (Message)equeue.errors.get(i);
boolean foundMsg = false;
for (int j = 0; j < expectedMessages.size(); j++) {
Message em = (Message)expectedMessages.get(j);
if (m.msgID==em.msgID && m.arg.equals(em.arg) && m.arg2.equals(em.arg2)) {
foundMsg = true;
}
}
if (foundMsg) {
messageExpected.add(i, Boolean.TRUE);
} else
messageExpected.add(i, Boolean.FALSE);
}
for (int i = 0; i < equeue.errors.size(); i++) {
assertTrue("unexpected error:" + equeue.errors.get(i), ((Boolean)messageExpected.get(i)).booleanValue());
}
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestASTConstruction.java 0000644 0001750 0001750 00000033573 11256465234 025556 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.tool.Grammar;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestASTConstruction extends BaseTest {
/** Public default constructor used by TestRig */
public TestASTConstruction() {
}
@Test public void testA() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT A <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testNakeRulePlusInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : B+;\n" +
"B : 'a';");
String expecting =
" ( rule A ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT B <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("A").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testRulePlus() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (b)+;\n" +
"b : B;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT b <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testNakedRulePlus() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b+;\n" +
"b : B;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT b <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testRuleOptional() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (b)?;\n" +
"b : B;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( ? ( BLOCK ( ALT b <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testNakedRuleOptional() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b?;\n" +
"b : B;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( ? ( BLOCK ( ALT b <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testRuleStar() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (b)*;\n" +
"b : B;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT b <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testNakedRuleStar() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b*;\n" +
"b : B;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT b <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testCharStar() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : 'a'*;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT 'a' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testCharStarInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"B : 'b'*;");
String expecting =
" ( rule B ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT 'b' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("B").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testStringStar() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : 'while'*;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT 'while' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testStringStarInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"B : 'while'*;");
String expecting =
" ( rule B ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT 'while' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("B").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testCharPlus() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : 'a'+;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT 'a' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testCharPlusInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"B : 'b'+;");
String expecting =
" ( rule B ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT 'b' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("B").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testCharOptional() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : 'a'?;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( ? ( BLOCK ( ALT 'a' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testCharOptionalInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"B : 'b'?;");
String expecting =
" ( rule B ARG RET scope ( BLOCK ( ALT ( ? ( BLOCK ( ALT 'b' <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("B").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testCharRangePlus() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"ID : 'a'..'z'+;");
String expecting =
" ( rule ID ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT ( .. 'a' 'z' ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("ID").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testLabel() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x=ID;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( = x ID ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testLabelOfOptional() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x=ID?;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( ? ( BLOCK ( ALT ( = x ID ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testLabelOfClosure() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x=ID*;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT ( = x ID ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testRuleLabel() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x=b;\n" +
"b : ID;\n");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( = x b ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testSetLabel() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x=(A|B);\n");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( = x ( BLOCK ( ALT A <end-of-alt> ) ( ALT B <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testNotSetLabel() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x=~(A|B);\n");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( = x ( ~ ( BLOCK ( ALT A <end-of-alt> ) ( ALT B <end-of-alt> ) <end-of-block> ) ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testNotSetListLabel() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x+=~(A|B);\n");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( += x ( ~ ( BLOCK ( ALT A <end-of-alt> ) ( ALT B <end-of-alt> ) <end-of-block> ) ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testNotSetListLabelInLoop() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x+=~(A|B)+;\n");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT ( += x ( ~ ( BLOCK ( ALT A <end-of-alt> ) ( ALT B <end-of-alt> ) <end-of-block> ) ) ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testRuleLabelOfPositiveClosure() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x=b+;\n" +
"b : ID;\n");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT ( = x b ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testListLabelOfClosure() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x+=ID*;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT ( += x ID ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testListLabelOfClosure2() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : x+='int'*;");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( * ( BLOCK ( ALT ( += x 'int' ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testRuleListLabelOfPositiveClosure() throws Exception {
Grammar g = new Grammar(
"grammar P;\n" +
"options {output=AST;}\n"+
"a : x+=b+;\n" +
"b : ID;\n");
String expecting =
" ( rule a ARG RET scope ( BLOCK ( ALT ( + ( BLOCK ( ALT ( += x b ) <end-of-alt> ) <end-of-block> ) ) <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("a").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testRootTokenInStarLoop() throws Exception {
Grammar g = new Grammar(
"grammar Expr;\n" +
"options { backtrack=true; }\n" +
"a : ('*'^)* ;\n"); // bug: the synpred had nothing in it
String expecting =
" ( rule synpred1_Expr ARG RET scope ( BLOCK ( ALT '*' <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("synpred1_Expr").tree.toStringTree();
assertEquals(expecting, found);
}
@Test public void testActionInStarLoop() throws Exception {
Grammar g = new Grammar(
"grammar Expr;\n" +
"options { backtrack=true; }\n" +
"a : ({blort} 'x')* ;\n"); // bug: the synpred had nothing in it
String expecting =
" ( rule synpred1_Expr ARG RET scope ( BLOCK ( ALT blort 'x' <end-of-alt> ) <end-of-block> ) <end-of-rule> )";
String found = g.getRule("synpred1_Expr").tree.toStringTree();
assertEquals(expecting, found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/DebugTestRewriteAST.java 0000644 0001750 0001750 00000003047 11256465234 025445 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
public class DebugTestRewriteAST extends TestRewriteAST {
public DebugTestRewriteAST() {debug=true;}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestCharDFAConversion.java 0000644 0001750 0001750 00000041552 11256465234 025746 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.analysis.DFA;
import org.antlr.analysis.DFAOptimizer;
import org.antlr.codegen.CodeGenerator;
import org.antlr.tool.*;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import java.util.List;
public class TestCharDFAConversion extends BaseTest {
/** Public default constructor used by TestRig */
public TestCharDFAConversion() {
}
// R A N G E S & S E T S
@Test public void testSimpleRangeVersusChar() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a'..'z' '@' | 'k' '$' ;");
g.createLookaheadDFAs();
String expecting =
".s0-'k'->.s1\n" +
".s0-{'a'..'j', 'l'..'z'}->:s2=>1\n" +
".s1-'$'->:s3=>2\n" +
".s1-'@'->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testRangeWithDisjointSet() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a'..'z' '@'\n" +
" | ('k'|'9'|'p') '$'\n" +
" ;\n");
g.createLookaheadDFAs();
// must break up a..z into {'a'..'j', 'l'..'o', 'q'..'z'}
String expecting =
".s0-'9'->:s3=>2\n" +
".s0-{'a'..'j', 'l'..'o', 'q'..'z'}->:s2=>1\n" +
".s0-{'k', 'p'}->.s1\n" +
".s1-'$'->:s3=>2\n" +
".s1-'@'->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testDisjointSetCollidingWithTwoRanges() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : ('a'..'z'|'0'..'9') '@'\n" +
" | ('k'|'9'|'p') '$'\n" +
" ;\n");
g.createLookaheadDFAs(false);
// must break up a..z into {'a'..'j', 'l'..'o', 'q'..'z'} and 0..9
// into 0..8
String expecting =
".s0-{'0'..'8', 'a'..'j', 'l'..'o', 'q'..'z'}->:s2=>1\n" +
".s0-{'9', 'k', 'p'}->.s1\n" +
".s1-'$'->:s3=>2\n" +
".s1-'@'->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testDisjointSetCollidingWithTwoRangesCharsFirst() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : ('k'|'9'|'p') '$'\n" +
" | ('a'..'z'|'0'..'9') '@'\n" +
" ;\n");
// must break up a..z into {'a'..'j', 'l'..'o', 'q'..'z'} and 0..9
// into 0..8
String expecting =
".s0-{'0'..'8', 'a'..'j', 'l'..'o', 'q'..'z'}->:s3=>2\n" +
".s0-{'9', 'k', 'p'}->.s1\n" +
".s1-'$'->:s2=>1\n" +
".s1-'@'->:s3=>2\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testDisjointSetCollidingWithTwoRangesAsSeparateAlts() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a'..'z' '@'\n" +
" | 'k' '$'\n" +
" | '9' '$'\n" +
" | 'p' '$'\n" +
" | '0'..'9' '@'\n" +
" ;\n");
// must break up a..z into {'a'..'j', 'l'..'o', 'q'..'z'} and 0..9
// into 0..8
String expecting =
".s0-'0'..'8'->:s8=>5\n" +
".s0-'9'->.s6\n" +
".s0-'k'->.s1\n" +
".s0-'p'->.s4\n" +
".s0-{'a'..'j', 'l'..'o', 'q'..'z'}->:s2=>1\n" +
".s1-'$'->:s3=>2\n" +
".s1-'@'->:s2=>1\n" +
".s4-'$'->:s5=>4\n" +
".s4-'@'->:s2=>1\n" +
".s6-'$'->:s7=>3\n" +
".s6-'@'->:s8=>5\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testKeywordVersusID() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"IF : 'if' ;\n" + // choose this over ID
"ID : ('a'..'z')+ ;\n");
String expecting =
".s0-'a'..'z'->:s2=>1\n" +
".s0-<EOT>->:s1=>2\n";
checkDecision(g, 1, expecting, null);
expecting =
".s0-'i'->.s1\n" +
".s0-{'a'..'h', 'j'..'z'}->:s4=>2\n" +
".s1-'f'->.s2\n" +
".s1-<EOT>->:s4=>2\n" +
".s2-'a'..'z'->:s4=>2\n" +
".s2-<EOT>->:s3=>1\n";
checkDecision(g, 2, expecting, null);
}
@Test public void testIdenticalRules() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a' ;\n" +
"B : 'a' ;\n"); // can't reach this
String expecting =
".s0-'a'->.s1\n" +
".s1-<EOT>->:s2=>1\n";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
checkDecision(g, 1, expecting, new int[] {2});
assertEquals("unexpected number of expected problems",
1, equeue.size());
Message msg = (Message)equeue.errors.get(0);
assertTrue("warning must be an unreachable alt",
msg instanceof GrammarUnreachableAltsMessage);
GrammarUnreachableAltsMessage u = (GrammarUnreachableAltsMessage)msg;
assertEquals("[2]", u.alts.toString());
}
@Test public void testAdjacentNotCharLoops() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : (~'r')+ ;\n" +
"B : (~'s')+ ;\n");
String expecting =
".s0-'r'->:s3=>2\n" +
".s0-'s'->:s2=>1\n" +
".s0-{'\\u0000'..'q', 't'..'\\uFFFF'}->.s1\n" +
".s1-'r'->:s3=>2\n" +
".s1-<EOT>->:s2=>1\n" +
".s1-{'\\u0000'..'q', 't'..'\\uFFFF'}->.s1\n";
checkDecision(g, 3, expecting, null);
}
@Test public void testNonAdjacentNotCharLoops() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : (~'r')+ ;\n" +
"B : (~'t')+ ;\n");
String expecting =
".s0-'r'->:s3=>2\n" +
".s0-'t'->:s2=>1\n" +
".s0-{'\\u0000'..'q', 's', 'u'..'\\uFFFF'}->.s1\n" +
".s1-'r'->:s3=>2\n" +
".s1-<EOT>->:s2=>1\n" +
".s1-{'\\u0000'..'q', 's', 'u'..'\\uFFFF'}->.s1\n";
checkDecision(g, 3, expecting, null);
}
@Test public void testLoopsWithOptimizedOutExitBranches() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'x'* ~'x'+ ;\n");
String expecting =
".s0-'x'->:s1=>1\n" +
".s0-{'\\u0000'..'w', 'y'..'\\uFFFF'}->:s2=>2\n";
checkDecision(g, 1, expecting, null);
// The optimizer yanks out all exit branches from EBNF blocks
// This is ok because we've already verified there are no problems
// with the enter/exit decision
DFAOptimizer optimizer = new DFAOptimizer(g);
optimizer.optimize();
FASerializer serializer = new FASerializer(g);
DFA dfa = g.getLookaheadDFA(1);
String result = serializer.serialize(dfa.startState);
expecting = ".s0-'x'->:s1=>1\n";
assertEquals(expecting, result);
}
// N O N G R E E D Y
@Test public void testNonGreedy() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"CMT : '/*' ( options {greedy=false;} : . )* '*/' ;");
String expecting =
".s0-'*'->.s1\n" +
".s0-{'\\u0000'..')', '+'..'\\uFFFF'}->:s3=>1\n" +
".s1-'/'->:s2=>2\n" +
".s1-{'\\u0000'..'.', '0'..'\\uFFFF'}->:s3=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonGreedyWildcardStar() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"SLCMT : '//' ( options {greedy=false;} : . )* '\n' ;");
String expecting =
".s0-'\\n'->:s1=>2\n" +
".s0-{'\\u0000'..'\\t', '\\u000B'..'\\uFFFF'}->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonGreedyByDefaultWildcardStar() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"SLCMT : '//' .* '\n' ;");
String expecting =
".s0-'\\n'->:s1=>2\n" +
".s0-{'\\u0000'..'\\t', '\\u000B'..'\\uFFFF'}->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonGreedyWildcardPlus() throws Exception {
// same DFA as nongreedy .* but code gen checks number of
// iterations at runtime
Grammar g = new Grammar(
"lexer grammar t;\n"+
"SLCMT : '//' ( options {greedy=false;} : . )+ '\n' ;");
String expecting =
".s0-'\\n'->:s1=>2\n" +
".s0-{'\\u0000'..'\\t', '\\u000B'..'\\uFFFF'}->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonGreedyByDefaultWildcardPlus() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"SLCMT : '//' .+ '\n' ;");
String expecting =
".s0-'\\n'->:s1=>2\n" +
".s0-{'\\u0000'..'\\t', '\\u000B'..'\\uFFFF'}->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonGreedyByDefaultWildcardPlusWithParens() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"SLCMT : '//' (.)+ '\n' ;");
String expecting =
".s0-'\\n'->:s1=>2\n" +
".s0-{'\\u0000'..'\\t', '\\u000B'..'\\uFFFF'}->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonWildcardNonGreedy() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"DUH : (options {greedy=false;}:'x'|'y')* 'xy' ;");
String expecting =
".s0-'x'->.s1\n" +
".s0-'y'->:s4=>2\n" +
".s1-'x'->:s3=>1\n" +
".s1-'y'->:s2=>3\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonWildcardEOTMakesItWorkWithoutNonGreedyOption() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"DUH : ('x'|'y')* 'xy' ;");
String expecting =
".s0-'x'->.s1\n" +
".s0-'y'->:s3=>1\n" +
".s1-'x'->:s3=>1\n" +
".s1-'y'->.s2\n" +
".s2-'x'..'y'->:s3=>1\n" +
".s2-<EOT>->:s4=>2\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testAltConflictsWithLoopThenExit() throws Exception {
// \" predicts alt 1, but wildcard then " can predict exit also
Grammar g = new Grammar(
"lexer grammar t;\n"+
"STRING : '\"' (options {greedy=false;}: '\\\\\"' | .)* '\"' ;\n"
);
String expecting =
".s0-'\"'->:s1=>3\n" +
".s0-'\\\\'->.s2\n" +
".s0-{'\\u0000'..'!', '#'..'[', ']'..'\\uFFFF'}->:s4=>2\n" +
".s2-'\"'->:s3=>1\n" +
".s2-{'\\u0000'..'!', '#'..'\\uFFFF'}->:s4=>2\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNonGreedyLoopThatNeverLoops() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"DUH : (options {greedy=false;}:'x')+ ;"); // loop never matched
String expecting =
":s0=>2\n";
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
checkDecision(g, 1, expecting, new int[] {1});
assertEquals("unexpected number of expected problems",
1, equeue.size());
Message msg = (Message)equeue.errors.get(0);
assertTrue("warning must be an unreachable alt",
msg instanceof GrammarUnreachableAltsMessage);
GrammarUnreachableAltsMessage u = (GrammarUnreachableAltsMessage)msg;
assertEquals("[1]", u.alts.toString());
}
@Test public void testRecursive() throws Exception {
// this is cool because the 3rd alt includes !(all other possibilities)
Grammar g = new Grammar(
"lexer grammar duh;\n" +
"SUBTEMPLATE\n" +
" : '{'\n" +
" ( SUBTEMPLATE\n" +
" | ESC\n" +
" | ~('}'|'\\\\'|'{')\n" +
" )*\n" +
" '}'\n" +
" ;\n" +
"fragment\n" +
"ESC : '\\\\' . ;");
g.createLookaheadDFAs();
String expecting =
".s0-'\\\\'->:s2=>2\n" +
".s0-'{'->:s1=>1\n" +
".s0-'}'->:s4=>4\n" +
".s0-{'\\u0000'..'[', ']'..'z', '|', '~'..'\\uFFFF'}->:s3=>3\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testRecursive2() throws Exception {
// this is also cool because it resolves \\ to be ESC alt; it's just
// less efficient of a DFA
Grammar g = new Grammar(
"lexer grammar duh;\n" +
"SUBTEMPLATE\n" +
" : '{'\n" +
" ( SUBTEMPLATE\n" +
" | ESC\n" +
" | ~('}'|'{')\n" +
" )*\n" +
" '}'\n" +
" ;\n" +
"fragment\n" +
"ESC : '\\\\' . ;");
g.createLookaheadDFAs();
String expecting =
".s0-'\\\\'->.s3\n" +
".s0-'{'->:s2=>1\n" +
".s0-'}'->:s1=>4\n" +
".s0-{'\\u0000'..'[', ']'..'z', '|', '~'..'\\uFFFF'}->:s5=>3\n" +
".s3-'\\\\'->:s8=>2\n" +
".s3-'{'->:s7=>2\n" +
".s3-'}'->.s4\n" +
".s3-{'\\u0000'..'[', ']'..'z', '|', '~'..'\\uFFFF'}->:s6=>2\n" +
".s4-'\\u0000'..'\\uFFFF'->:s6=>2\n" +
".s4-<EOT>->:s5=>3\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNotFragmentInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar T;\n"+
"A : 'a' | ~B {;} ;\n" +
"fragment B : 'a' ;\n");
g.createLookaheadDFAs();
String expecting =
".s0-'a'->:s1=>1\n" +
".s0-{'\\u0000'..'`', 'b'..'\\uFFFF'}->:s2=>2\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNotSetFragmentInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar T;\n"+
"A : B | ~B {;} ;\n" +
"fragment B : 'a'|'b' ;\n");
g.createLookaheadDFAs();
String expecting =
".s0-'a'..'b'->:s1=>1\n" +
".s0-{'\\u0000'..'`', 'c'..'\\uFFFF'}->:s2=>2\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNotTokenInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar T;\n"+
"A : 'x' ('a' | ~B {;}) ;\n" +
"B : 'a' ;\n");
g.createLookaheadDFAs();
String expecting =
".s0-'a'->:s1=>1\n" +
".s0-{'\\u0000'..'`', 'b'..'\\uFFFF'}->:s2=>2\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNotComplicatedSetRuleInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar T;\n"+
"A : B | ~B {;} ;\n" +
"fragment B : 'a'|'b'|'c'..'e'|C ;\n" +
"fragment C : 'f' ;\n"); // has to seen from B to C
String expecting =
".s0-'a'..'f'->:s1=>1\n" +
".s0-{'\\u0000'..'`', 'g'..'\\uFFFF'}->:s2=>2\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testNotSetWithRuleInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar T;\n"+
"T : ~('a' | B) | 'a';\n" +
"fragment\n" +
"B : 'b' ;\n" +
"C : ~'x'{;} ;"); // force Tokens to not collapse T|C
String expecting =
".s0-'b'->:s3=>2\n" +
".s0-'x'->:s2=>1\n" +
".s0-{'\\u0000'..'a', 'c'..'w', 'y'..'\\uFFFF'}->.s1\n" +
".s1-<EOT>->:s2=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testSetCallsRuleWithNot() throws Exception {
Grammar g = new Grammar(
"lexer grammar A;\n" +
"T : ~'x' ;\n" +
"S : 'x' (T | 'x') ;\n");
String expecting =
".s0-'x'->:s2=>2\n" +
".s0-{'\\u0000'..'w', 'y'..'\\uFFFF'}->:s1=>1\n";
checkDecision(g, 1, expecting, null);
}
@Test public void testSynPredInLexer() throws Exception {
Grammar g = new Grammar(
"lexer grammar T;\n"+
"LT: '<' ' '*\n" +
" | ('<' IDENT) => '<' IDENT '>'\n" + // this was causing syntax error
" ;\n" +
"IDENT: 'a'+;\n");
// basically, Tokens rule should not do set compression test
String expecting =
".s0-'<'->:s1=>1\n" +
".s0-'a'->:s2=>2\n";
checkDecision(g, 4, expecting, null); // 4 is Tokens rule
}
// S U P P O R T
public void _template() throws Exception {
Grammar g = new Grammar(
"grammar T;\n"+
"a : A | B;");
String expecting =
"\n";
checkDecision(g, 1, expecting, null);
}
protected void checkDecision(Grammar g,
int decision,
String expecting,
int[] expectingUnreachableAlts)
throws Exception
{
// mimic actions of org.antlr.Tool first time for grammar g
if ( g.getCodeGenerator()==null ) {
CodeGenerator generator = new CodeGenerator(null, g, "Java");
g.setCodeGenerator(generator);
g.buildNFA();
g.createLookaheadDFAs(false);
}
DFA dfa = g.getLookaheadDFA(decision);
assertNotNull("unknown decision #"+decision, dfa);
FASerializer serializer = new FASerializer(g);
String result = serializer.serialize(dfa.startState);
//System.out.print(result);
List nonDetAlts = dfa.getUnreachableAlts();
//System.out.println("alts w/o predict state="+nonDetAlts);
// first make sure nondeterministic alts are as expected
if ( expectingUnreachableAlts==null ) {
if ( nonDetAlts!=null && nonDetAlts.size()!=0 ) {
System.err.println("nondeterministic alts (should be empty): "+nonDetAlts);
}
assertEquals("unreachable alts mismatch", 0, nonDetAlts!=null?nonDetAlts.size():0);
}
else {
for (int i=0; i<expectingUnreachableAlts.length; i++) {
assertTrue("unreachable alts mismatch",
nonDetAlts!=null?nonDetAlts.contains(new Integer(expectingUnreachableAlts[i])):false);
}
}
assertEquals(expecting, result);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestBufferedTreeNodeStream.java 0000644 0001750 0001750 00000005770 11256465234 027036 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.runtime.tree.*;
import org.antlr.runtime.CommonToken;
import org.junit.Test;
public class TestBufferedTreeNodeStream extends TestTreeNodeStream {
// inherits tests; these methods make it use a new buffer
public TreeNodeStream newStream(Object t) {
return new BufferedTreeNodeStream(t);
}
public String toTokenTypeString(TreeNodeStream stream) {
return ((BufferedTreeNodeStream)stream).toTokenTypeString();
}
@Test public void testSeek() throws Exception {
// ^(101 ^(102 103 ^(106 107) ) 104 105)
// stream has 7 real + 6 nav nodes
// Sequence of types: 101 DN 102 DN 103 106 DN 107 UP UP 104 105 UP EOF
Tree r0 = new CommonTree(new CommonToken(101));
Tree r1 = new CommonTree(new CommonToken(102));
r0.addChild(r1);
r1.addChild(new CommonTree(new CommonToken(103)));
Tree r2 = new CommonTree(new CommonToken(106));
r2.addChild(new CommonTree(new CommonToken(107)));
r1.addChild(r2);
r0.addChild(new CommonTree(new CommonToken(104)));
r0.addChild(new CommonTree(new CommonToken(105)));
TreeNodeStream stream = newStream(r0);
stream.consume(); // consume 101
stream.consume(); // consume DN
stream.consume(); // consume 102
stream.seek(7); // seek to 107
assertEquals(107, ((Tree)stream.LT(1)).getType());
stream.consume(); // consume 107
stream.consume(); // consume UP
stream.consume(); // consume UP
assertEquals(104, ((Tree)stream.LT(1)).getType());
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestInterpretedParsing.java 0000644 0001750 0001750 00000015425 11256465234 026321 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.tool.Grammar;
import org.antlr.tool.Interpreter;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.ParseTree;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestInterpretedParsing extends BaseTest {
/** Public default constructor used by TestRig */
public TestInterpretedParsing() {
}
@Test public void testSimpleParse() throws Exception {
Grammar pg = new Grammar(
"parser grammar p;\n"+
"prog : WHILE ID LCURLY (assign)* RCURLY EOF;\n" +
"assign : ID ASSIGN expr SEMI ;\n" +
"expr : INT | FLOAT | ID ;\n");
Grammar g = new Grammar();
g.importTokenVocabulary(pg);
g.setFileName(Grammar.IGNORE_STRING_IN_GRAMMAR_FILE_NAME +"string");
g.setGrammarContent(
"lexer grammar t;\n"+
"WHILE : 'while';\n"+
"LCURLY : '{';\n"+
"RCURLY : '}';\n"+
"ASSIGN : '=';\n"+
"SEMI : ';';\n"+
"ID : ('a'..'z')+ ;\n"+
"INT : (DIGIT)+ ;\n"+
"FLOAT : (DIGIT)+ '.' (DIGIT)* ;\n"+
"fragment DIGIT : '0'..'9';\n" +
"WS : (' ')+ ;\n");
CharStream input = new ANTLRStringStream("while x { i=1; y=3.42; z=y; }");
Interpreter lexEngine = new Interpreter(g, input);
CommonTokenStream tokens = new CommonTokenStream(lexEngine);
tokens.setTokenTypeChannel(g.getTokenType("WS"), 99);
//System.out.println("tokens="+tokens.toString());
Interpreter parseEngine = new Interpreter(pg, tokens);
ParseTree t = parseEngine.parse("prog");
String result = t.toStringTree();
String expecting =
"(<grammar p> (prog while x { (assign i = (expr 1) ;) (assign y = (expr 3.42) ;) (assign z = (expr y) ;) } <EOF>))";
assertEquals(expecting, result);
}
@Test public void testMismatchedTokenError() throws Exception {
Grammar pg = new Grammar(
"parser grammar p;\n"+
"prog : WHILE ID LCURLY (assign)* RCURLY;\n" +
"assign : ID ASSIGN expr SEMI ;\n" +
"expr : INT | FLOAT | ID ;\n");
Grammar g = new Grammar();
g.setFileName(Grammar.IGNORE_STRING_IN_GRAMMAR_FILE_NAME +"string");
g.importTokenVocabulary(pg);
g.setGrammarContent(
"lexer grammar t;\n"+
"WHILE : 'while';\n"+
"LCURLY : '{';\n"+
"RCURLY : '}';\n"+
"ASSIGN : '=';\n"+
"SEMI : ';';\n"+
"ID : ('a'..'z')+ ;\n"+
"INT : (DIGIT)+ ;\n"+
"FLOAT : (DIGIT)+ '.' (DIGIT)* ;\n"+
"fragment DIGIT : '0'..'9';\n" +
"WS : (' ')+ ;\n");
CharStream input = new ANTLRStringStream("while x { i=1 y=3.42; z=y; }");
Interpreter lexEngine = new Interpreter(g, input);
CommonTokenStream tokens = new CommonTokenStream(lexEngine);
tokens.setTokenTypeChannel(g.getTokenType("WS"), 99);
//System.out.println("tokens="+tokens.toString());
Interpreter parseEngine = new Interpreter(pg, tokens);
ParseTree t = parseEngine.parse("prog");
String result = t.toStringTree();
String expecting =
"(<grammar p> (prog while x { (assign i = (expr 1) MismatchedTokenException(5!=9))))";
assertEquals(expecting, result);
}
@Test public void testMismatchedSetError() throws Exception {
Grammar pg = new Grammar(
"parser grammar p;\n"+
"prog : WHILE ID LCURLY (assign)* RCURLY;\n" +
"assign : ID ASSIGN expr SEMI ;\n" +
"expr : INT | FLOAT | ID ;\n");
Grammar g = new Grammar();
g.importTokenVocabulary(pg);
g.setFileName("<string>");
g.setGrammarContent(
"lexer grammar t;\n"+
"WHILE : 'while';\n"+
"LCURLY : '{';\n"+
"RCURLY : '}';\n"+
"ASSIGN : '=';\n"+
"SEMI : ';';\n"+
"ID : ('a'..'z')+ ;\n"+
"INT : (DIGIT)+ ;\n"+
"FLOAT : (DIGIT)+ '.' (DIGIT)* ;\n"+
"fragment DIGIT : '0'..'9';\n" +
"WS : (' ')+ ;\n");
CharStream input = new ANTLRStringStream("while x { i=; y=3.42; z=y; }");
Interpreter lexEngine = new Interpreter(g, input);
CommonTokenStream tokens = new CommonTokenStream(lexEngine);
tokens.setTokenTypeChannel(g.getTokenType("WS"), 99);
//System.out.println("tokens="+tokens.toString());
Interpreter parseEngine = new Interpreter(pg, tokens);
ParseTree t = parseEngine.parse("prog");
String result = t.toStringTree();
String expecting =
"(<grammar p> (prog while x { (assign i = (expr MismatchedSetException(9!={5,10,11})))))";
assertEquals(expecting, result);
}
@Test public void testNoViableAltError() throws Exception {
Grammar pg = new Grammar(
"parser grammar p;\n"+
"prog : WHILE ID LCURLY (assign)* RCURLY;\n" +
"assign : ID ASSIGN expr SEMI ;\n" +
"expr : {;}INT | FLOAT | ID ;\n");
Grammar g = new Grammar();
g.importTokenVocabulary(pg);
g.setFileName("<string>");
g.setGrammarContent(
"lexer grammar t;\n"+
"WHILE : 'while';\n"+
"LCURLY : '{';\n"+
"RCURLY : '}';\n"+
"ASSIGN : '=';\n"+
"SEMI : ';';\n"+
"ID : ('a'..'z')+ ;\n"+
"INT : (DIGIT)+ ;\n"+
"FLOAT : (DIGIT)+ '.' (DIGIT)* ;\n"+
"fragment DIGIT : '0'..'9';\n" +
"WS : (' ')+ ;\n");
CharStream input = new ANTLRStringStream("while x { i=; y=3.42; z=y; }");
Interpreter lexEngine = new Interpreter(g, input);
CommonTokenStream tokens = new CommonTokenStream(lexEngine);
tokens.setTokenTypeChannel(g.getTokenType("WS"), 99);
//System.out.println("tokens="+tokens.toString());
Interpreter parseEngine = new Interpreter(pg, tokens);
ParseTree t = parseEngine.parse("prog");
String result = t.toStringTree();
String expecting =
"(<grammar p> (prog while x { (assign i = (expr NoViableAltException(9@[4:1: expr : ( INT | FLOAT | ID );])))))";
assertEquals(expecting, result);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestSyntacticPredicateEvaluation.java 0000644 0001750 0001750 00000030160 11256465234 030313 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestSyntacticPredicateEvaluation extends BaseTest {
@Test public void testTwoPredsWithNakedAlt() throws Exception {
String grammar =
"grammar T;\n" +
"s : (a ';')+ ;\n" +
"a\n" +
"options {\n" +
" k=1;\n" +
"}\n" +
" : (b '.')=> b '.' {System.out.println(\"alt 1\");}\n" +
" | (b)=> b {System.out.println(\"alt 2\");}\n" +
" | c {System.out.println(\"alt 3\");}\n" +
" ;\n" +
"b\n" +
"@init {System.out.println(\"enter b\");}\n" +
" : '(' 'x' ')' ;\n" +
"c\n" +
"@init {System.out.println(\"enter c\");}\n" +
" : '(' c ')' | 'x' ;\n" +
"WS : (' '|'\\n')+ {$channel=HIDDEN;}\n" +
" ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "(x) ;", false);
String expecting =
"enter b\n" +
"enter b\n" +
"enter b\n" +
"alt 2\n";
assertEquals(expecting, found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "(x). ;", false);
expecting =
"enter b\n" +
"enter b\n" +
"alt 1\n";
assertEquals(expecting, found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "((x)) ;", false);
expecting =
"enter b\n" +
"enter b\n" +
"enter c\n" +
"enter c\n" +
"enter c\n" +
"alt 3\n";
assertEquals(expecting, found);
}
@Test public void testTwoPredsWithNakedAltNotLast() throws Exception {
String grammar =
"grammar T;\n" +
"s : (a ';')+ ;\n" +
"a\n" +
"options {\n" +
" k=1;\n" +
"}\n" +
" : (b '.')=> b '.' {System.out.println(\"alt 1\");}\n" +
" | c {System.out.println(\"alt 2\");}\n" +
" | (b)=> b {System.out.println(\"alt 3\");}\n" +
" ;\n" +
"b\n" +
"@init {System.out.println(\"enter b\");}\n" +
" : '(' 'x' ')' ;\n" +
"c\n" +
"@init {System.out.println(\"enter c\");}\n" +
" : '(' c ')' | 'x' ;\n" +
"WS : (' '|'\\n')+ {$channel=HIDDEN;}\n" +
" ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "(x) ;", false);
String expecting =
"enter b\n" +
"enter c\n" +
"enter c\n" +
"alt 2\n";
assertEquals(expecting, found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "(x). ;", false);
expecting =
"enter b\n" +
"enter b\n" +
"alt 1\n";
assertEquals(expecting, found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "((x)) ;", false);
expecting =
"enter b\n" +
"enter c\n" +
"enter c\n" +
"enter c\n" +
"alt 2\n";
assertEquals(expecting, found);
}
@Test public void testLexerPred() throws Exception {
String grammar =
"grammar T;\n" +
"s : A ;\n" +
"A options {k=1;}\n" + // force backtracking
" : (B '.')=>B '.' {System.out.println(\"alt1\");}\n" +
" | B {System.out.println(\"alt2\");}" +
" ;\n" +
"fragment\n" +
"B : 'x'+ ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "xxx", false);
assertEquals("alt2\n", found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "xxx.", false);
assertEquals("alt1\n", found);
}
@Test public void testLexerWithPredLongerThanAlt() throws Exception {
String grammar =
"grammar T;\n" +
"s : A ;\n" +
"A options {k=1;}\n" + // force backtracking
" : (B '.')=>B {System.out.println(\"alt1\");}\n" +
" | B {System.out.println(\"alt2\");}" +
" ;\n" +
"D : '.' {System.out.println(\"D\");} ;\n" +
"fragment\n" +
"B : 'x'+ ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "xxx", false);
assertEquals("alt2\n", found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "xxx.", false);
assertEquals("alt1\nD\n", found);
}
@Test public void testLexerPredCyclicPrediction() throws Exception {
String grammar =
"grammar T;\n" +
"s : A ;\n" +
"A : (B)=>(B|'y'+) {System.out.println(\"alt1\");}\n" +
" | B {System.out.println(\"alt2\");}\n" +
" | 'y'+ ';'" +
" ;\n" +
"fragment\n" +
"B : 'x'+ ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "xxx", false);
assertEquals("alt1\n", found);
}
@Test public void testLexerPredCyclicPrediction2() throws Exception {
String grammar =
"grammar T;\n" +
"s : A ;\n" +
"A : (B '.')=>(B|'y'+) {System.out.println(\"alt1\");}\n" +
" | B {System.out.println(\"alt2\");}\n" +
" | 'y'+ ';'" +
" ;\n" +
"fragment\n" +
"B : 'x'+ ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "xxx", false);
assertEquals("alt2\n", found);
}
@Test public void testSimpleNestedPred() throws Exception {
String grammar =
"grammar T;\n" +
"s : (expr ';')+ ;\n" +
"expr\n" +
"options {\n" +
" k=1;\n" +
"}\n" +
"@init {System.out.println(\"enter expr \"+input.LT(1).getText());}\n" +
" : (atom 'x') => atom 'x'\n" +
" | atom\n" +
";\n" +
"atom\n" +
"@init {System.out.println(\"enter atom \"+input.LT(1).getText());}\n" +
" : '(' expr ')'\n" +
" | INT\n" +
" ;\n" +
"INT: '0'..'9'+ ;\n" +
"WS : (' '|'\\n')+ {$channel=HIDDEN;}\n" +
" ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "(34)x;", false);
String expecting =
"enter expr (\n" +
"enter atom (\n" +
"enter expr 34\n" +
"enter atom 34\n" +
"enter atom 34\n" +
"enter atom (\n" +
"enter expr 34\n" +
"enter atom 34\n" +
"enter atom 34\n";
assertEquals(expecting, found);
}
@Test public void testTripleNestedPredInLexer() throws Exception {
String grammar =
"grammar T;\n" +
"s : (.)+ {System.out.println(\"done\");} ;\n" +
"EXPR\n" +
"options {\n" +
" k=1;\n" +
"}\n" +
"@init {System.out.println(\"enter expr \"+(char)input.LT(1));}\n" +
" : (ATOM 'x') => ATOM 'x' {System.out.println(\"ATOM x\");}\n" +
" | ATOM {System.out.println(\"ATOM \"+$ATOM.text);}\n" +
";\n" +
"fragment ATOM\n" +
"@init {System.out.println(\"enter atom \"+(char)input.LT(1));}\n" +
" : '(' EXPR ')'\n" +
" | INT\n" +
" ;\n" +
"fragment INT: '0'..'9'+ ;\n" +
"fragment WS : (' '|'\\n')+ \n" +
" ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"s", "((34)x)x", false);
String expecting = // has no memoization
"enter expr (\n" +
"enter atom (\n" +
"enter expr (\n" +
"enter atom (\n" +
"enter expr 3\n" +
"enter atom 3\n" +
"enter atom 3\n" +
"enter atom (\n" +
"enter expr 3\n" +
"enter atom 3\n" +
"enter atom 3\n" +
"enter atom (\n" +
"enter expr (\n" +
"enter atom (\n" +
"enter expr 3\n" +
"enter atom 3\n" +
"enter atom 3\n" +
"enter atom (\n" +
"enter expr 3\n" +
"enter atom 3\n" +
"enter atom 3\n" +
"ATOM 34\n" +
"ATOM x\n" +
"ATOM x\n" +
"done\n";
assertEquals(expecting, found);
}
@Test public void testTreeParserWithSynPred() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT+ (PERIOD|SEMI);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"PERIOD : '.' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
String treeGrammar =
"tree grammar TP;\n" +
"options {k=1; backtrack=true; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID INT+ PERIOD {System.out.print(\"alt 1\");}"+
" | ID INT+ SEMI {System.out.print(\"alt 2\");}\n" +
" ;\n";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 1 2 3;");
assertEquals("alt 2\n", found);
}
@Test public void testTreeParserWithNestedSynPred() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ID INT+ (PERIOD|SEMI);\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"SEMI : ';' ;\n"+
"PERIOD : '.' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;\n";
// backtracks in a and b due to k=1
String treeGrammar =
"tree grammar TP;\n" +
"options {k=1; backtrack=true; ASTLabelType=CommonTree; tokenVocab=T;}\n" +
"a : ID b {System.out.print(\" a:alt 1\");}"+
" | ID INT+ SEMI {System.out.print(\" a:alt 2\");}\n" +
" ;\n" +
"b : INT PERIOD {System.out.print(\"b:alt 1\");}" + // choose this alt for just one INT
" | INT+ PERIOD {System.out.print(\"b:alt 2\");}" +
" ;";
String found = execTreeParser("T.g", grammar, "TParser", "TP.g",
treeGrammar, "TP", "TLexer", "a", "a", "a 1 2 3.");
assertEquals("b:alt 2 a:alt 1\n", found);
}
@Test public void testSynPredWithOutputTemplate() throws Exception {
// really just seeing if it will compile
String grammar =
"grammar T;\n" +
"options {output=template;}\n" +
"a\n" +
"options {\n" +
" k=1;\n" +
"}\n" +
" : ('x'+ 'y')=> 'x'+ 'y' -> template(a={$text}) <<1:<a>;>>\n" +
" | 'x'+ 'z' -> template(a={$text}) <<2:<a>;>>\n"+
" ;\n" +
"WS : (' '|'\\n')+ {$channel=HIDDEN;}\n" +
" ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "xxxy", false);
assertEquals("1:xxxy;\n", found);
}
@Test public void testSynPredWithOutputAST() throws Exception {
// really just seeing if it will compile
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a\n" +
"options {\n" +
" k=1;\n" +
"}\n" +
" : ('x'+ 'y')=> 'x'+ 'y'\n" +
" | 'x'+ 'z'\n"+
" ;\n" +
"WS : (' '|'\\n')+ {$channel=HIDDEN;}\n" +
" ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "xxxy", false);
assertEquals("x x x y\n", found);
}
@Test public void testOptionalBlockWithSynPred() throws Exception {
String grammar =
"grammar T;\n" +
"\n" +
"a : ( (b)=> b {System.out.println(\"b\");})? b ;\n" +
"b : 'x' ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "xx", false);
assertEquals("b\n", found);
found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", false);
assertEquals("", found);
}
@Test public void testSynPredK2() throws Exception {
// all manually specified syn predicates are gated (i.e., forced
// to execute).
String grammar =
"grammar T;\n" +
"\n" +
"a : (b)=> b {System.out.println(\"alt1\");} | 'a' 'c' ;\n" +
"b : 'a' 'b' ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "ab", false);
assertEquals("alt1\n", found);
}
@Test public void testSynPredKStar() throws Exception {
String grammar =
"grammar T;\n" +
"\n" +
"a : (b)=> b {System.out.println(\"alt1\");} | 'a'+ 'c' ;\n" +
"b : 'a'+ 'b' ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "aaab", false);
assertEquals("alt1\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestSemanticPredicateEvaluation.java 0000644 0001750 0001750 00000020600 11256465234 030113 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestSemanticPredicateEvaluation extends BaseTest {
@Test public void testSimpleCyclicDFAWithPredicate() throws Exception {
String grammar =
"grammar foo;\n" +
"a : {false}? 'x'* 'y' {System.out.println(\"alt1\");}\n" +
" | {true}? 'x'* 'y' {System.out.println(\"alt2\");}\n" +
" ;\n" ;
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "xxxy", false);
assertEquals("alt2\n", found);
}
@Test public void testSimpleCyclicDFAWithInstanceVarPredicate() throws Exception {
String grammar =
"grammar foo;\n" +
"@members {boolean v=true;}\n" +
"a : {false}? 'x'* 'y' {System.out.println(\"alt1\");}\n" +
" | {v}? 'x'* 'y' {System.out.println(\"alt2\");}\n" +
" ;\n" ;
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "xxxy", false);
assertEquals("alt2\n", found);
}
@Test public void testPredicateValidation() throws Exception {
String grammar =
"grammar foo;\n" +
"@members {\n" +
"public void reportError(RecognitionException e) {\n" +
" System.out.println(\"error: \"+e.toString());\n" +
"}\n" +
"}\n" +
"\n" +
"a : {false}? 'x'\n" +
" ;\n" ;
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "x", false);
assertEquals("error: FailedPredicateException(a,{false}?)\n", found);
}
@Test public void testLexerPreds() throws Exception {
String grammar =
"grammar foo;" +
"@lexer::members {boolean p=false;}\n" +
"a : (A|B)+ ;\n" +
"A : {p}? 'a' {System.out.println(\"token 1\");} ;\n" +
"B : {!p}? 'a' {System.out.println(\"token 2\");} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "a", false);
// "a" is ambig; can match both A, B. Pred says match 2
assertEquals("token 2\n", found);
}
@Test public void testLexerPreds2() throws Exception {
String grammar =
"grammar foo;" +
"@lexer::members {boolean p=true;}\n" +
"a : (A|B)+ ;\n" +
"A : {p}? 'a' {System.out.println(\"token 1\");} ;\n" +
"B : ('a'|'b')+ {System.out.println(\"token 2\");} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "a", false);
// "a" is ambig; can match both A, B. Pred says match 1
assertEquals("token 1\n", found);
}
@Test public void testLexerPredInExitBranch() throws Exception {
// p says it's ok to exit; it has precendence over the !p loopback branch
String grammar =
"grammar foo;" +
"@lexer::members {boolean p=true;}\n" +
"a : (A|B)+ ;\n" +
"A : ('a' {System.out.print(\"1\");})*\n" +
" {p}?\n" +
" ('a' {System.out.print(\"2\");})* ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aaa", false);
assertEquals("222\n", found);
}
@Test public void testLexerPredInExitBranch2() throws Exception {
String grammar =
"grammar foo;" +
"@lexer::members {boolean p=true;}\n" +
"a : (A|B)+ ;\n" +
"A : ({p}? 'a' {System.out.print(\"1\");})*\n" +
" ('a' {System.out.print(\"2\");})* ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aaa", false);
assertEquals("111\n", found);
}
@Test public void testLexerPredInExitBranch3() throws Exception {
String grammar =
"grammar foo;" +
"@lexer::members {boolean p=true;}\n" +
"a : (A|B)+ ;\n" +
"A : ({p}? 'a' {System.out.print(\"1\");} | )\n" +
" ('a' {System.out.print(\"2\");})* ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aaa", false);
assertEquals("122\n", found);
}
@Test public void testLexerPredInExitBranch4() throws Exception {
String grammar =
"grammar foo;" +
"a : (A|B)+ ;\n" +
"A @init {int n=0;} : ({n<2}? 'a' {System.out.print(n++);})+\n" +
" ('a' {System.out.print(\"x\");})* ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aaaaa", false);
assertEquals("01xxx\n", found);
}
@Test public void testLexerPredsInCyclicDFA() throws Exception {
String grammar =
"grammar foo;" +
"@lexer::members {boolean p=false;}\n" +
"a : (A|B)+ ;\n" +
"A : {p}? ('a')+ 'x' {System.out.println(\"token 1\");} ;\n" +
"B : ('a')+ 'x' {System.out.println(\"token 2\");} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aax", false);
assertEquals("token 2\n", found);
}
@Test public void testLexerPredsInCyclicDFA2() throws Exception {
String grammar =
"grammar foo;" +
"@lexer::members {boolean p=false;}\n" +
"a : (A|B)+ ;\n" +
"A : {p}? ('a')+ 'x' ('y')? {System.out.println(\"token 1\");} ;\n" +
"B : ('a')+ 'x' {System.out.println(\"token 2\");} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aax", false);
assertEquals("token 2\n", found);
}
@Test public void testGatedPred() throws Exception {
String grammar =
"grammar foo;" +
"a : (A|B)+ ;\n" +
"A : {true}?=> 'a' {System.out.println(\"token 1\");} ;\n" +
"B : {false}?=>('a'|'b')+ {System.out.println(\"token 2\");} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aa", false);
// "a" is ambig; can match both A, B. Pred says match A twice
assertEquals("token 1\ntoken 1\n", found);
}
@Test public void testGatedPred2() throws Exception {
String grammar =
"grammar foo;\n" +
"@lexer::members {boolean sig=false;}\n"+
"a : (A|B)+ ;\n" +
"A : 'a' {System.out.print(\"A\"); sig=true;} ;\n" +
"B : 'b' ;\n" +
"C : {sig}?=> ('a'|'b') {System.out.print(\"C\");} ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aa", false);
assertEquals("AC\n", found);
}
@Test public void testPredWithActionTranslation() throws Exception {
String grammar =
"grammar foo;\n" +
"a : b[2] ;\n" +
"b[int i]\n" +
" : {$i==1}? 'a' {System.out.println(\"alt 1\");}\n" +
" | {$b.i==2}? 'a' {System.out.println(\"alt 2\");}\n" +
" ;\n";
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "aa", false);
assertEquals("alt 2\n", found);
}
@Test public void testPredicatesOnEOTTarget() throws Exception {
String grammar =
"grammar foo; \n" +
"@lexer::members {boolean p=true, q=false;}" +
"a : B ;\n" +
"A: '</'; \n" +
"B: {p}? '<!' {System.out.println(\"B\");};\n" +
"C: {q}? '<' {System.out.println(\"C\");}; \n" +
"D: '<';\n" ;
String found = execParser("foo.g", grammar, "fooParser", "fooLexer",
"a", "<!", false);
assertEquals("B\n", found);
}
// S U P P O R T
public void _test() throws Exception {
String grammar =
"grammar T;\n" +
"options {output=AST;}\n" +
"a : ;\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+;\n" +
"WS : (' '|'\\n') {channel=99;} ;\n";
String found = execParser("t.g", grammar, "T", "TLexer",
"a", "abc 34", false);
assertEquals("\n", found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/DebugTestCompositeGrammars.java 0000644 0001750 0001750 00000003073 11256465234 027107 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
public class DebugTestCompositeGrammars extends TestCompositeGrammars {
public DebugTestCompositeGrammars() {debug=true;}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestDFAMatching.java 0000644 0001750 0001750 00000007531 11256465234 024554 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.analysis.DFA;
import org.antlr.analysis.NFA;
import org.antlr.runtime.ANTLRStringStream;
import org.antlr.tool.Grammar;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestDFAMatching extends BaseTest {
/** Public default constructor used by TestRig */
public TestDFAMatching() {
}
@Test public void testSimpleAltCharTest() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : {;}'a' | 'b' | 'c';");
g.buildNFA();
g.createLookaheadDFAs(false);
DFA dfa = g.getLookaheadDFA(1);
checkPrediction(dfa,"a",1);
checkPrediction(dfa,"b",2);
checkPrediction(dfa,"c",3);
checkPrediction(dfa,"d", NFA.INVALID_ALT_NUMBER);
}
@Test public void testSets() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : {;}'a'..'z' | ';' | '0'..'9' ;");
g.buildNFA();
g.createLookaheadDFAs(false);
DFA dfa = g.getLookaheadDFA(1);
checkPrediction(dfa,"a",1);
checkPrediction(dfa,"q",1);
checkPrediction(dfa,"z",1);
checkPrediction(dfa,";",2);
checkPrediction(dfa,"9",3);
}
@Test public void testFiniteCommonLeftPrefixes() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a' 'b' | 'a' 'c' | 'd' 'e' ;");
g.buildNFA();
g.createLookaheadDFAs(false);
DFA dfa = g.getLookaheadDFA(1);
checkPrediction(dfa,"ab",1);
checkPrediction(dfa,"ac",2);
checkPrediction(dfa,"de",3);
checkPrediction(dfa,"q", NFA.INVALID_ALT_NUMBER);
}
@Test public void testSimpleLoops() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : (DIGIT)+ '.' DIGIT | (DIGIT)+ ;\n" +
"fragment DIGIT : '0'..'9' ;\n");
g.buildNFA();
g.createLookaheadDFAs(false);
DFA dfa = g.getLookaheadDFA(3);
checkPrediction(dfa,"32",2);
checkPrediction(dfa,"999.2",1);
checkPrediction(dfa,".2", NFA.INVALID_ALT_NUMBER);
}
protected void checkPrediction(DFA dfa, String input, int expected)
throws Exception
{
ANTLRStringStream stream = new ANTLRStringStream(input);
assertEquals(dfa.predict(stream), expected);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestTreeNodeStream.java 0000644 0001750 0001750 00000030402 11256465236 025363 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.runtime.CommonToken;
import org.antlr.runtime.Token;
import org.antlr.runtime.tree.*;
import org.junit.Test;
/** Test the tree node stream. */
public class TestTreeNodeStream extends BaseTest {
/** Build new stream; let's us override to test other streams. */
public TreeNodeStream newStream(Object t) {
return new CommonTreeNodeStream(t);
}
public String toTokenTypeString(TreeNodeStream stream) {
return ((CommonTreeNodeStream)stream).toTokenTypeString();
}
@Test public void testSingleNode() throws Exception {
Tree t = new CommonTree(new CommonToken(101));
TreeNodeStream stream = newStream(t);
String expecting = " 101";
String found = toNodesOnlyString(stream);
assertEquals(expecting, found);
expecting = " 101";
found = toTokenTypeString(stream);
assertEquals(expecting, found);
}
@Test public void test4Nodes() throws Exception {
// ^(101 ^(102 103) 104)
Tree t = new CommonTree(new CommonToken(101));
t.addChild(new CommonTree(new CommonToken(102)));
t.getChild(0).addChild(new CommonTree(new CommonToken(103)));
t.addChild(new CommonTree(new CommonToken(104)));
TreeNodeStream stream = newStream(t);
String expecting = " 101 102 103 104";
String found = toNodesOnlyString(stream);
assertEquals(expecting, found);
expecting = " 101 2 102 2 103 3 104 3";
found = toTokenTypeString(stream);
assertEquals(expecting, found);
}
@Test public void testList() throws Exception {
Tree root = new CommonTree((Token)null);
Tree t = new CommonTree(new CommonToken(101));
t.addChild(new CommonTree(new CommonToken(102)));
t.getChild(0).addChild(new CommonTree(new CommonToken(103)));
t.addChild(new CommonTree(new CommonToken(104)));
Tree u = new CommonTree(new CommonToken(105));
root.addChild(t);
root.addChild(u);
TreeNodeStream stream = newStream(root);
String expecting = " 101 102 103 104 105";
String found = toNodesOnlyString(stream);
assertEquals(expecting, found);
expecting = " 101 2 102 2 103 3 104 3 105";
found = toTokenTypeString(stream);
assertEquals(expecting, found);
}
@Test public void testFlatList() throws Exception {
Tree root = new CommonTree((Token)null);
root.addChild(new CommonTree(new CommonToken(101)));
root.addChild(new CommonTree(new CommonToken(102)));
root.addChild(new CommonTree(new CommonToken(103)));
TreeNodeStream stream = newStream(root);
String expecting = " 101 102 103";
String found = toNodesOnlyString(stream);
assertEquals(expecting, found);
expecting = " 101 102 103";
found = toTokenTypeString(stream);
assertEquals(expecting, found);
}
@Test public void testListWithOneNode() throws Exception {
Tree root = new CommonTree((Token)null);
root.addChild(new CommonTree(new CommonToken(101)));
TreeNodeStream stream = newStream(root);
String expecting = " 101";
String found = toNodesOnlyString(stream);
assertEquals(expecting, found);
expecting = " 101";
found = toTokenTypeString(stream);
assertEquals(expecting, found);
}
@Test public void testAoverB() throws Exception {
Tree t = new CommonTree(new CommonToken(101));
t.addChild(new CommonTree(new CommonToken(102)));
TreeNodeStream stream = newStream(t);
String expecting = " 101 102";
String found = toNodesOnlyString(stream);
assertEquals(expecting, found);
expecting = " 101 2 102 3";
found = toTokenTypeString(stream);
assertEquals(expecting, found);
}
@Test public void testLT() throws Exception {
// ^(101 ^(102 103) 104)
Tree t = new CommonTree(new CommonToken(101));
t.addChild(new CommonTree(new CommonToken(102)));
t.getChild(0).addChild(new CommonTree(new CommonToken(103)));
t.addChild(new CommonTree(new CommonToken(104)));
TreeNodeStream stream = newStream(t);
assertEquals(101, ((Tree)stream.LT(1)).getType());
assertEquals(Token.DOWN, ((Tree)stream.LT(2)).getType());
assertEquals(102, ((Tree)stream.LT(3)).getType());
assertEquals(Token.DOWN, ((Tree)stream.LT(4)).getType());
assertEquals(103, ((Tree)stream.LT(5)).getType());
assertEquals(Token.UP, ((Tree)stream.LT(6)).getType());
assertEquals(104, ((Tree)stream.LT(7)).getType());
assertEquals(Token.UP, ((Tree)stream.LT(8)).getType());
assertEquals(Token.EOF, ((Tree)stream.LT(9)).getType());
// check way ahead
assertEquals(Token.EOF, ((Tree)stream.LT(100)).getType());
}
@Test public void testMarkRewindEntire() throws Exception {
// ^(101 ^(102 103 ^(106 107) ) 104 105)
// stream has 7 real + 6 nav nodes
// Sequence of types: 101 DN 102 DN 103 106 DN 107 UP UP 104 105 UP EOF
Tree r0 = new CommonTree(new CommonToken(101));
Tree r1 = new CommonTree(new CommonToken(102));
r0.addChild(r1);
r1.addChild(new CommonTree(new CommonToken(103)));
Tree r2 = new CommonTree(new CommonToken(106));
r2.addChild(new CommonTree(new CommonToken(107)));
r1.addChild(r2);
r0.addChild(new CommonTree(new CommonToken(104)));
r0.addChild(new CommonTree(new CommonToken(105)));
TreeNodeStream stream = newStream(r0);
int m = stream.mark(); // MARK
for (int k=1; k<=13; k++) { // consume til end
stream.LT(1);
stream.consume();
}
assertEquals(Token.EOF, ((Tree)stream.LT(1)).getType());
assertEquals(Token.UP, ((Tree)stream.LT(-1)).getType());
stream.rewind(m); // REWIND
// consume til end again :)
for (int k=1; k<=13; k++) { // consume til end
stream.LT(1);
stream.consume();
}
assertEquals(Token.EOF, ((Tree)stream.LT(1)).getType());
assertEquals(Token.UP, ((Tree)stream.LT(-1)).getType());
}
@Test public void testMarkRewindInMiddle() throws Exception {
// ^(101 ^(102 103 ^(106 107) ) 104 105)
// stream has 7 real + 6 nav nodes
// Sequence of types: 101 DN 102 DN 103 106 DN 107 UP UP 104 105 UP EOF
Tree r0 = new CommonTree(new CommonToken(101));
Tree r1 = new CommonTree(new CommonToken(102));
r0.addChild(r1);
r1.addChild(new CommonTree(new CommonToken(103)));
Tree r2 = new CommonTree(new CommonToken(106));
r2.addChild(new CommonTree(new CommonToken(107)));
r1.addChild(r2);
r0.addChild(new CommonTree(new CommonToken(104)));
r0.addChild(new CommonTree(new CommonToken(105)));
TreeNodeStream stream = newStream(r0);
for (int k=1; k<=7; k++) { // consume til middle
//System.out.println(((Tree)stream.LT(1)).getType());
stream.consume();
}
assertEquals(107, ((Tree)stream.LT(1)).getType());
stream.mark(); // MARK
stream.consume(); // consume 107
stream.consume(); // consume UP
stream.consume(); // consume UP
stream.consume(); // consume 104
stream.rewind(); // REWIND
stream.mark(); // keep saving nodes though
assertEquals(107, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(Token.UP, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(Token.UP, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(104, ((Tree)stream.LT(1)).getType());
stream.consume();
// now we're past rewind position
assertEquals(105, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(Token.UP, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(Token.EOF, ((Tree)stream.LT(1)).getType());
assertEquals(Token.UP, ((Tree)stream.LT(-1)).getType());
}
@Test public void testMarkRewindNested() throws Exception {
// ^(101 ^(102 103 ^(106 107) ) 104 105)
// stream has 7 real + 6 nav nodes
// Sequence of types: 101 DN 102 DN 103 106 DN 107 UP UP 104 105 UP EOF
Tree r0 = new CommonTree(new CommonToken(101));
Tree r1 = new CommonTree(new CommonToken(102));
r0.addChild(r1);
r1.addChild(new CommonTree(new CommonToken(103)));
Tree r2 = new CommonTree(new CommonToken(106));
r2.addChild(new CommonTree(new CommonToken(107)));
r1.addChild(r2);
r0.addChild(new CommonTree(new CommonToken(104)));
r0.addChild(new CommonTree(new CommonToken(105)));
TreeNodeStream stream = newStream(r0);
int m = stream.mark(); // MARK at start
stream.consume(); // consume 101
stream.consume(); // consume DN
int m2 = stream.mark(); // MARK on 102
stream.consume(); // consume 102
stream.consume(); // consume DN
stream.consume(); // consume 103
stream.consume(); // consume 106
stream.rewind(m2); // REWIND to 102
assertEquals(102, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(Token.DOWN, ((Tree)stream.LT(1)).getType());
stream.consume();
// stop at 103 and rewind to start
stream.rewind(m); // REWIND to 101
assertEquals(101, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(Token.DOWN, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(102, ((Tree)stream.LT(1)).getType());
stream.consume();
assertEquals(Token.DOWN, ((Tree)stream.LT(1)).getType());
}
@Test public void testSeekFromStart() throws Exception {
// ^(101 ^(102 103 ^(106 107) ) 104 105)
// stream has 7 real + 6 nav nodes
// Sequence of types: 101 DN 102 DN 103 106 DN 107 UP UP 104 105 UP EOF
Tree r0 = new CommonTree(new CommonToken(101));
Tree r1 = new CommonTree(new CommonToken(102));
r0.addChild(r1);
r1.addChild(new CommonTree(new CommonToken(103)));
Tree r2 = new CommonTree(new CommonToken(106));
r2.addChild(new CommonTree(new CommonToken(107)));
r1.addChild(r2);
r0.addChild(new CommonTree(new CommonToken(104)));
r0.addChild(new CommonTree(new CommonToken(105)));
TreeNodeStream stream = newStream(r0);
stream.seek(7); // seek to 107
assertEquals(107, ((Tree)stream.LT(1)).getType());
stream.consume(); // consume 107
stream.consume(); // consume UP
stream.consume(); // consume UP
assertEquals(104, ((Tree)stream.LT(1)).getType());
}
@Test public void testReset() throws Exception {
// ^(101 ^(102 103 ^(106 107) ) 104 105)
// stream has 7 real + 6 nav nodes
// Sequence of types: 101 DN 102 DN 103 106 DN 107 UP UP 104 105 UP EOF
Tree r0 = new CommonTree(new CommonToken(101));
Tree r1 = new CommonTree(new CommonToken(102));
r0.addChild(r1);
r1.addChild(new CommonTree(new CommonToken(103)));
Tree r2 = new CommonTree(new CommonToken(106));
r2.addChild(new CommonTree(new CommonToken(107)));
r1.addChild(r2);
r0.addChild(new CommonTree(new CommonToken(104)));
r0.addChild(new CommonTree(new CommonToken(105)));
TreeNodeStream stream = newStream(r0);
String v = toNodesOnlyString(stream); // scan all
stream.reset();
String v2 = toNodesOnlyString(stream); // scan all
assertEquals(v,v2);
}
public String toNodesOnlyString(TreeNodeStream nodes) {
TreeAdaptor adaptor = nodes.getTreeAdaptor();
StringBuffer buf = new StringBuffer();
Object o = nodes.LT(1);
int type = adaptor.getType(o);
while ( o!=null && type!=Token.EOF ) {
if ( !(type==Token.DOWN||type==Token.UP) ) {
buf.append(" ");
buf.append(type);
}
nodes.consume();
o = nodes.LT(1);
type = adaptor.getType(o);
}
return buf.toString();
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestLexer.java 0000644 0001750 0001750 00000021236 11256465234 023564 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Test;
import static org.junit.Assert.assertEquals;
import org.antlr.Tool;
import org.antlr.tool.Grammar;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.codegen.CodeGenerator;
public class TestLexer extends BaseTest {
protected boolean debug = false;
/** Public default constructor used by TestRig */
public TestLexer() {
}
@Test public void testSetText() throws Exception {
// this must return A not I to the parser; calling a nonfragment rule
// from a nonfragment rule does not set the overall token.
String grammar =
"grammar P;\n"+
"a : A {System.out.println(input);} ;\n"+
"A : '\\\\' 't' {setText(\"\t\");} ;\n" +
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "\\t", debug);
assertEquals("\t\n", found);
}
@Test public void testRefToRuleDoesNotSetTokenNorEmitAnother() throws Exception {
// this must return A not I to the parser; calling a nonfragment rule
// from a nonfragment rule does not set the overall token.
String grammar =
"grammar P;\n"+
"a : A EOF {System.out.println(input);} ;\n"+
"A : '-' I ;\n" +
"I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "-34", debug);
assertEquals("-34\n", found);
}
@Test public void testRefToRuleDoesNotSetChannel() throws Exception {
// this must set channel of A to HIDDEN. $channel is local to rule
// like $type.
String grammar =
"grammar P;\n"+
"a : A EOF {System.out.println($A.text+\", channel=\"+$A.channel);} ;\n"+
"A : '-' WS I ;\n" +
"I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "- 34", debug);
assertEquals("- 34, channel=0\n", found);
}
@Test public void testWeCanSetType() throws Exception {
String grammar =
"grammar P;\n"+
"tokens {X;}\n" +
"a : X EOF {System.out.println(input);} ;\n"+
"A : '-' I {$type = X;} ;\n" +
"I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "-34", debug);
assertEquals("-34\n", found);
}
@Test public void testRefToFragment() throws Exception {
// this must return A not I to the parser; calling a nonfragment rule
// from a nonfragment rule does not set the overall token.
String grammar =
"grammar P;\n"+
"a : A {System.out.println(input);} ;\n"+
"A : '-' I ;\n" +
"fragment I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "-34", debug);
assertEquals("-34\n", found);
}
@Test public void testMultipleRefToFragment() throws Exception {
// this must return A not I to the parser; calling a nonfragment rule
// from a nonfragment rule does not set the overall token.
String grammar =
"grammar P;\n"+
"a : A EOF {System.out.println(input);} ;\n"+
"A : I '.' I ;\n" +
"fragment I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "3.14159", debug);
assertEquals("3.14159\n", found);
}
@Test public void testLabelInSubrule() throws Exception {
// can we see v outside?
String grammar =
"grammar P;\n"+
"a : A EOF ;\n"+
"A : 'hi' WS (v=I)? {$channel=0; System.out.println($v.text);} ;\n" +
"fragment I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "hi 342", debug);
assertEquals("342\n", found);
}
@Test public void testRefToTokenInLexer() throws Exception {
String grammar =
"grammar P;\n"+
"a : A EOF ;\n"+
"A : I {System.out.println($I.text);} ;\n" +
"fragment I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "342", debug);
assertEquals("342\n", found);
}
@Test public void testListLabelInLexer() throws Exception {
String grammar =
"grammar P;\n"+
"a : A ;\n"+
"A : i+=I+ {for (Object t : $i) System.out.print(\" \"+((Token)t).getText());} ;\n" +
"fragment I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "33 297", debug);
assertEquals(" 33 297\n", found);
}
@Test public void testDupListRefInLexer() throws Exception {
String grammar =
"grammar P;\n"+
"a : A ;\n"+
"A : i+=I WS i+=I {$channel=0; for (Object t : $i) System.out.print(\" \"+((Token)t).getText());} ;\n" +
"fragment I : '0'..'9'+ ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "33 297", debug);
assertEquals(" 33 297\n", found);
}
@Test public void testCharLabelInLexer() {
String grammar =
"grammar T;\n" +
"a : B ;\n" +
"B : x='a' {System.out.println((char)$x);} ;\n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "a", debug);
assertEquals("a\n", found);
}
@Test public void testRepeatedLabelInLexer() {
String grammar =
"lexer grammar T;\n" +
"B : x='a' x='b' ;\n" ;
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, null, "T", false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
@Test public void testRepeatedRuleLabelInLexer() {
String grammar =
"lexer grammar T;\n" +
"B : x=A x=A ;\n" +
"fragment A : 'a' ;\n" ;
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, null, "T", false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
@Test public void testIsolatedEOTEdge() {
String grammar =
"lexer grammar T;\n" +
"QUOTED_CONTENT \n" +
" : 'q' (~'q')* (('x' 'q') )* 'q' ; \n";
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, null, "T", false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
@Test public void testEscapedLiterals() {
/* Grammar:
A : '\"' ; should match a single double-quote: "
B : '\\\"' ; should match input \"
*/
String grammar =
"lexer grammar T;\n" +
"A : '\\\"' ;\n" +
"B : '\\\\\\\"' ;\n" ; // '\\\"'
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, null, "T", false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
@Test public void testNewlineLiterals() throws Exception {
Grammar g = new Grammar(
"lexer grammar T;\n" +
"A : '\\n\\n' ;\n" // ANTLR sees '\n\n'
);
String expecting = "match(\"\\n\\n\")";
Tool antlr = newTool();
antlr.setOutputDirectory(null); // write to /dev/null
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
generator.genRecognizer(); // codegen phase sets some vars we need
StringTemplate codeST = generator.getRecognizerST();
String code = codeST.toString();
int m = code.indexOf("match(\"");
String found = code.substring(m,m+expecting.length());
assertEquals(expecting, found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestCompositeGrammars.java 0000644 0001750 0001750 00000074010 11256465234 026137 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2007 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.Tool;
import org.antlr.tool.*;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestCompositeGrammars extends BaseTest {
protected boolean debug = false;
@Test public void testWildcardStillWorks() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammar =
"parser grammar S;\n" +
"a : B . C ;\n"; // not qualified ID
Grammar g = new Grammar(grammar);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testDelegatorInvokesDelegateRule() throws Exception {
String slave =
"parser grammar S;\n" +
"a : B {System.out.println(\"S.a\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"s : a ;\n" +
"B : 'b' ;" + // defines B from inherited token space
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"s", "b", debug);
assertEquals("S.a\n", found);
}
@Test public void testDelegatorInvokesDelegateRuleWithArgs() throws Exception {
// must generate something like:
// public int a(int x) throws RecognitionException { return gS.a(x); }
// in M.
String slave =
"parser grammar S;\n" +
"a[int x] returns [int y] : B {System.out.print(\"S.a\"); $y=1000;} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"s : label=a[3] {System.out.println($label.y);} ;\n" +
"B : 'b' ;" + // defines B from inherited token space
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"s", "b", debug);
assertEquals("S.a1000\n", found);
}
@Test public void testDelegatorInvokesDelegateRuleWithReturnStruct() throws Exception {
// must generate something like:
// public int a(int x) throws RecognitionException { return gS.a(x); }
// in M.
String slave =
"parser grammar S;\n" +
"a : B {System.out.print(\"S.a\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"s : a {System.out.println($a.text);} ;\n" +
"B : 'b' ;" + // defines B from inherited token space
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"s", "b", debug);
assertEquals("S.ab\n", found);
}
@Test public void testDelegatorAccessesDelegateMembers() throws Exception {
String slave =
"parser grammar S;\n" +
"@members {\n" +
" public void foo() {System.out.println(\"foo\");}\n" +
"}\n" +
"a : B ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" + // uses no rules from the import
"import S;\n" +
"s : 'b' {gS.foo();} ;\n" + // gS is import pointer
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"s", "b", debug);
assertEquals("foo\n", found);
}
@Test public void testDelegatorInvokesFirstVersionOfDelegateRule() throws Exception {
String slave =
"parser grammar S;\n" +
"a : b {System.out.println(\"S.a\");} ;\n" +
"b : B ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String slave2 =
"parser grammar T;\n" +
"a : B {System.out.println(\"T.a\");} ;\n"; // hidden by S.a
writeFile(tmpdir, "T.g", slave2);
String master =
"grammar M;\n" +
"import S,T;\n" +
"s : a ;\n" +
"B : 'b' ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"s", "b", debug);
assertEquals("S.a\n", found);
}
@Test public void testDelegatesSeeSameTokenType() throws Exception {
String slave =
"parser grammar S;\n" + // A, B, C token type order
"tokens { A; B; C; }\n" +
"x : A {System.out.println(\"S.x\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String slave2 =
"parser grammar T;\n" +
"tokens { C; B; A; }\n" + // reverse order
"y : A {System.out.println(\"T.y\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "T.g", slave2);
// The lexer will create rules to match letters a, b, c.
// The associated token types A, B, C must have the same value
// and all import'd parsers. Since ANTLR regenerates all imports
// for use with the delegator M, it can generate the same token type
// mapping in each parser:
// public static final int C=6;
// public static final int EOF=-1;
// public static final int B=5;
// public static final int WS=7;
// public static final int A=4;
String master =
"grammar M;\n" +
"import S,T;\n" +
"s : x y ;\n" + // matches AA, which should be "aa"
"B : 'b' ;\n" + // another order: B, A, C
"A : 'a' ;\n" +
"C : 'c' ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"s", "aa", debug);
assertEquals("S.x\n" +
"T.y\n", found);
}
@Test public void testDelegatesSeeSameTokenType2() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar S;\n" + // A, B, C token type order
"tokens { A; B; C; }\n" +
"x : A {System.out.println(\"S.x\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String slave2 =
"parser grammar T;\n" +
"tokens { C; B; A; }\n" + // reverse order
"y : A {System.out.println(\"T.y\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "T.g", slave2);
String master =
"grammar M;\n" +
"import S,T;\n" +
"s : x y ;\n" + // matches AA, which should be "aa"
"B : 'b' ;\n" + // another order: B, A, C
"A : 'a' ;\n" +
"C : 'c' ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
String expectedTokenIDToTypeMap = "[A=4, B=5, C=6, WS=7]";
String expectedStringLiteralToTypeMap = "{}";
String expectedTypeToTokenList = "[A, B, C, WS]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testCombinedImportsCombined() throws Exception {
// for now, we don't allow combined to import combined
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"grammar S;\n" + // A, B, C token type order
"tokens { A; B; C; }\n" +
"x : 'x' INT {System.out.println(\"S.x\");} ;\n" +
"INT : '0'..'9'+ ;\n" +
"WS : (' '|'\\n') {skip();} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"s : x INT ;\n";
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
assertEquals("unexpected errors: "+equeue, 1, equeue.errors.size());
String expectedError = "error(161): "+tmpdir.toString().replaceFirst("\\-[0-9]+","")+"/M.g:2:8: combined grammar M cannot import combined grammar S";
assertEquals("unexpected errors: "+equeue, expectedError, equeue.errors.get(0).toString().replaceFirst("\\-[0-9]+",""));
}
@Test public void testSameStringTwoNames() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar S;\n" +
"tokens { A='a'; }\n" +
"x : A {System.out.println(\"S.x\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String slave2 =
"parser grammar T;\n" +
"tokens { X='a'; }\n" +
"y : X {System.out.println(\"T.y\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "T.g", slave2);
String master =
"grammar M;\n" +
"import S,T;\n" +
"s : x y ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
String expectedTokenIDToTypeMap = "[A=4, WS=6, X=5]";
String expectedStringLiteralToTypeMap = "{'a'=4}";
String expectedTypeToTokenList = "[A, X, WS]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
Object expectedArg = "X='a'";
Object expectedArg2 = "A";
int expectedMsgID = ErrorManager.MSG_TOKEN_ALIAS_CONFLICT;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkGrammarSemanticsError(equeue, expectedMessage);
assertEquals("unexpected errors: "+equeue, 1, equeue.errors.size());
String expectedError =
"error(158): T.g:2:10: cannot alias X='a'; string already assigned to A";
assertEquals(expectedError, equeue.errors.get(0).toString());
}
@Test public void testSameNameTwoStrings() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar S;\n" +
"tokens { A='a'; }\n" +
"x : A {System.out.println(\"S.x\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String slave2 =
"parser grammar T;\n" +
"tokens { A='x'; }\n" +
"y : A {System.out.println(\"T.y\");} ;\n";
writeFile(tmpdir, "T.g", slave2);
String master =
"grammar M;\n" +
"import S,T;\n" +
"s : x y ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
String expectedTokenIDToTypeMap = "[A=4, T__6=6, WS=5]";
String expectedStringLiteralToTypeMap = "{'a'=4, 'x'=6}";
String expectedTypeToTokenList = "[A, WS, T__6]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, sortMapToString(g.composite.stringLiteralToTypeMap));
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
Object expectedArg = "A='x'";
Object expectedArg2 = "'a'";
int expectedMsgID = ErrorManager.MSG_TOKEN_ALIAS_REASSIGNMENT;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkGrammarSemanticsError(equeue, expectedMessage);
assertEquals("unexpected errors: "+equeue, 1, equeue.errors.size());
String expectedError =
"error(159): T.g:2:10: cannot alias A='x'; token name already assigned to 'a'";
assertEquals(expectedError, equeue.errors.get(0).toString());
}
@Test public void testImportedTokenVocabIgnoredWithWarning() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar S;\n" +
"options {tokenVocab=whatever;}\n" +
"tokens { A='a'; }\n" +
"x : A {System.out.println(\"S.x\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"s : x ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
Object expectedArg = "S";
int expectedMsgID = ErrorManager.MSG_TOKEN_VOCAB_IN_DELEGATE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsWarning(equeue, expectedMessage);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
assertEquals("unexpected errors: "+equeue, 1, equeue.warnings.size());
String expectedError =
"warning(160): S.g:2:10: tokenVocab option ignored in imported grammar S";
assertEquals(expectedError, equeue.warnings.get(0).toString());
}
@Test public void testImportedTokenVocabWorksInRoot() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar S;\n" +
"tokens { A='a'; }\n" +
"x : A {System.out.println(\"S.x\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String tokens =
"A=99\n";
writeFile(tmpdir, "Test.tokens", tokens);
String master =
"grammar M;\n" +
"options {tokenVocab=Test;}\n" +
"import S;\n" +
"s : x ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
String expectedTokenIDToTypeMap = "[A=99, WS=101]";
String expectedStringLiteralToTypeMap = "{'a'=100}";
String expectedTypeToTokenList = "[A, 'a', WS]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testSyntaxErrorsInImportsNotThrownOut() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar S;\n" +
"options {toke\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"s : x ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
// whole bunch of errors from bad S.g file
assertEquals("unexpected errors: "+equeue, 5, equeue.errors.size());
}
@Test public void testSyntaxErrorsInImportsNotThrownOut2() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar S;\n" +
": A {System.out.println(\"S.x\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"s : x ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
// whole bunch of errors from bad S.g file
assertEquals("unexpected errors: "+equeue, 3, equeue.errors.size());
}
@Test public void testDelegatorRuleOverridesDelegate() throws Exception {
String slave =
"parser grammar S;\n" +
"a : b {System.out.println(\"S.a\");} ;\n" +
"b : B ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"b : 'b'|'c' ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"a", "c", debug);
assertEquals("S.a\n", found);
}
@Test public void testDelegatorRuleOverridesLookaheadInDelegate() throws Exception {
String slave =
"parser grammar JavaDecl;\n" +
"type : 'int' ;\n" +
"decl : type ID ';'\n" +
" | type ID init ';' {System.out.println(\"JavaDecl: \"+$decl.text);}\n" +
" ;\n" +
"init : '=' INT ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "JavaDecl.g", slave);
String master =
"grammar Java;\n" +
"import JavaDecl;\n" +
"prog : decl ;\n" +
"type : 'int' | 'float' ;\n" +
"\n" +
"ID : 'a'..'z'+ ;\n" +
"INT : '0'..'9'+ ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
// for float to work in decl, type must be overridden
String found = execParser("Java.g", master, "JavaParser", "JavaLexer",
"prog", "float x = 3;", debug);
assertEquals("JavaDecl: floatx=3;\n", found);
}
@Test public void testDelegatorRuleOverridesDelegates() throws Exception {
String slave =
"parser grammar S;\n" +
"a : b {System.out.println(\"S.a\");} ;\n" +
"b : B ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String slave2 =
"parser grammar T;\n" +
"tokens { A='x'; }\n" +
"b : B {System.out.println(\"T.b\");} ;\n";
writeFile(tmpdir, "T.g", slave2);
String master =
"grammar M;\n" +
"import S, T;\n" +
"b : 'b'|'c' {System.out.println(\"M.b\");}|B|A ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"a", "c", debug);
assertEquals("M.b\n" +
"S.a\n", found);
}
// LEXER INHERITANCE
@Test public void testLexerDelegatorInvokesDelegateRule() throws Exception {
String slave =
"lexer grammar S;\n" +
"A : 'a' {System.out.println(\"S.A\");} ;\n" +
"C : 'c' ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"lexer grammar M;\n" +
"import S;\n" +
"B : 'b' ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execLexer("M.g", master, "M", "abc", debug);
assertEquals("S.A\nabc\n", found);
}
@Test public void testLexerDelegatorRuleOverridesDelegate() throws Exception {
String slave =
"lexer grammar S;\n" +
"A : 'a' {System.out.println(\"S.A\");} ;\n" +
"B : 'b' {System.out.println(\"S.B\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"lexer grammar M;\n" +
"import S;\n" +
"A : 'a' B {System.out.println(\"M.A\");} ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execLexer("M.g", master, "M", "ab", debug);
assertEquals("S.B\n" +
"M.A\n" +
"ab\n", found);
}
@Test public void testLexerDelegatorRuleOverridesDelegateLeavingNoRules() throws Exception {
// M.Tokens has nothing to predict tokens from S. Should
// not include S.Tokens alt in this case?
String slave =
"lexer grammar S;\n" +
"A : 'a' {System.out.println(\"S.A\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"lexer grammar M;\n" +
"import S;\n" +
"A : 'a' {System.out.println(\"M.A\");} ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
writeFile(tmpdir, "/M.g", master);
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
composite.assignTokenTypes();
composite.defineGrammarSymbols();
composite.createNFAs();
g.createLookaheadDFAs(false);
// predict only alts from M not S
String expectingDFA =
".s0-'a'->.s1\n" +
".s0-{'\\n', ' '}->:s3=>2\n" +
".s1-<EOT>->:s2=>1\n";
org.antlr.analysis.DFA dfa = g.getLookaheadDFA(1);
FASerializer serializer = new FASerializer(g);
String result = serializer.serialize(dfa.startState);
assertEquals(expectingDFA, result);
// must not be a "unreachable alt: Tokens" error
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testInvalidImportMechanism() throws Exception {
// M.Tokens has nothing to predict tokens from S. Should
// not include S.Tokens alt in this case?
String slave =
"lexer grammar S;\n" +
"A : 'a' {System.out.println(\"S.A\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"tree grammar M;\n" +
"import S;\n" +
"a : A ;";
writeFile(tmpdir, "/M.g", master);
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
assertEquals("unexpected errors: "+equeue, 1, equeue.errors.size());
assertEquals("unexpected errors: "+equeue, 0, equeue.warnings.size());
String expectedError =
"error(161): "+tmpdir.toString().replaceFirst("\\-[0-9]+","")+"/M.g:2:8: tree grammar M cannot import lexer grammar S";
assertEquals(expectedError, equeue.errors.get(0).toString().replaceFirst("\\-[0-9]+",""));
}
@Test public void testSyntacticPredicateRulesAreNotInherited() throws Exception {
// if this compiles, it means that synpred1_S is defined in S.java
// but not MParser.java. MParser has its own synpred1_M which must
// be separate to compile.
String slave =
"parser grammar S;\n" +
"a : 'a' {System.out.println(\"S.a1\");}\n" +
" | 'a' {System.out.println(\"S.a2\");}\n" +
" ;\n" +
"b : 'x' | 'y' {;} ;\n"; // preds generated but not need in DFA here
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"options {backtrack=true;}\n" +
"import S;\n" +
"start : a b ;\n" +
"nonsense : 'q' | 'q' {;} ;" + // forces def of preds here in M
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"start", "ax", debug);
assertEquals("S.a1\n", found);
}
@Test public void testKeywordVSIDGivesNoWarning() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"lexer grammar S;\n" +
"A : 'abc' {System.out.println(\"S.A\");} ;\n" +
"ID : 'a'..'z'+ ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"a : A {System.out.println(\"M.a\");} ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
String found = execParser("M.g", master, "MParser", "MLexer",
"a", "abc", debug);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
assertEquals("unexpected warnings: "+equeue, 0, equeue.warnings.size());
assertEquals("S.A\nM.a\n", found);
}
@Test public void testWarningForUndefinedToken() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"lexer grammar S;\n" +
"A : 'abc' {System.out.println(\"S.A\");} ;\n";
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
String master =
"grammar M;\n" +
"import S;\n" +
"a : ABC A {System.out.println(\"M.a\");} ;\n" +
"WS : (' '|'\\n') {skip();} ;\n" ;
// A is defined in S but M should still see it and not give warning.
// only problem is ABC.
rawGenerateAndBuildRecognizer("M.g", master, "MParser", "MLexer", debug);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
assertEquals("unexpected warnings: "+equeue, 1, equeue.warnings.size());
String expectedError =
"warning(105): "+tmpdir.toString().replaceFirst("\\-[0-9]+","")+"/M.g:3:5: no lexer rule corresponding to token: ABC";
assertEquals(expectedError, equeue.warnings.get(0).toString().replaceFirst("\\-[0-9]+",""));
}
/** Make sure that M can import S that imports T. */
@Test public void test3LevelImport() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar T;\n" +
"a : T ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "T.g", slave);
String slave2 =
"parser grammar S;\n" + // A, B, C token type order
"import T;\n" +
"a : S ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave2);
String master =
"grammar M;\n" +
"import S;\n" +
"a : M ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
g.composite.defineGrammarSymbols();
String expectedTokenIDToTypeMap = "[M=6, S=5, T=4]";
String expectedStringLiteralToTypeMap = "{}";
String expectedTypeToTokenList = "[T, S, M]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
boolean ok =
rawGenerateAndBuildRecognizer("M.g", master, "MParser", null, false);
boolean expecting = true; // should be ok
assertEquals(expecting, ok);
}
@Test public void testBigTreeOfImports() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar T;\n" +
"x : T ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "T.g", slave);
slave =
"parser grammar S;\n" +
"import T;\n" +
"y : S ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave);
slave =
"parser grammar C;\n" +
"i : C ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "C.g", slave);
slave =
"parser grammar B;\n" +
"j : B ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "B.g", slave);
slave =
"parser grammar A;\n" +
"import B,C;\n" +
"k : A ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "A.g", slave);
String master =
"grammar M;\n" +
"import S,A;\n" +
"a : M ;\n" ;
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
g.composite.defineGrammarSymbols();
String expectedTokenIDToTypeMap = "[A=8, B=6, C=7, M=9, S=5, T=4]";
String expectedStringLiteralToTypeMap = "{}";
String expectedTypeToTokenList = "[T, S, B, C, A, M]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
boolean ok =
rawGenerateAndBuildRecognizer("M.g", master, "MParser", null, false);
boolean expecting = true; // should be ok
assertEquals(expecting, ok);
}
@Test public void testRulesVisibleThroughMultilevelImport() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String slave =
"parser grammar T;\n" +
"x : T ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "T.g", slave);
String slave2 =
"parser grammar S;\n" + // A, B, C token type order
"import T;\n" +
"a : S ;\n" ;
mkdir(tmpdir);
writeFile(tmpdir, "S.g", slave2);
String master =
"grammar M;\n" +
"import S;\n" +
"a : M x ;\n" ; // x MUST BE VISIBLE TO M
writeFile(tmpdir, "M.g", master);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/M.g",composite);
composite.setDelegationRoot(g);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
g.composite.defineGrammarSymbols();
String expectedTokenIDToTypeMap = "[M=6, S=5, T=4]";
String expectedStringLiteralToTypeMap = "{}";
String expectedTypeToTokenList = "[T, S, M]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
} antlr-3.2/tool/src/test/java/org/antlr/test/TestJavaCodeGeneration.java 0000644 0001750 0001750 00000010753 11256465234 026177 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
/** General code generation testing; compilation and/or execution.
* These tests are more about avoiding duplicate var definitions
* etc... than testing a particular ANTLR feature.
*/
public class TestJavaCodeGeneration extends BaseTest {
@Test public void testDupVarDefForPinchedState() {
// so->s2 and s0->s3->s1 pinches back to s1
// LA3_1, s1 state for DFA 3, was defined twice in similar scope
// just wrapped in curlies and it's cool.
String grammar =
"grammar T;\n" +
"a : (| A | B) X Y\n" +
" | (| A | B) X Z\n" +
" ;\n" ;
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, "TParser", null, false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
@Test public void testLabeledNotSetsInLexer() {
// d must be an int
String grammar =
"lexer grammar T;\n" +
"A : d=~('x'|'y') e='0'..'9'\n" +
" ; \n" ;
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, null, "T", false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
@Test public void testLabeledSetsInLexer() {
// d must be an int
String grammar =
"grammar T;\n" +
"a : A ;\n" +
"A : d=('x'|'y') {System.out.println((char)$d);}\n" +
" ; \n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", false);
assertEquals("x\n", found);
}
@Test public void testLabeledRangeInLexer() {
// d must be an int
String grammar =
"grammar T;\n" +
"a : A;\n" +
"A : d='a'..'z' {System.out.println((char)$d);} \n" +
" ; \n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", false);
assertEquals("x\n", found);
}
@Test public void testLabeledWildcardInLexer() {
// d must be an int
String grammar =
"grammar T;\n" +
"a : A;\n" +
"A : d=. {System.out.println((char)$d);}\n" +
" ; \n" ;
String found = execParser("T.g", grammar, "TParser", "TLexer",
"a", "x", false);
assertEquals("x\n", found);
}
@Test public void testSynpredWithPlusLoop() {
String grammar =
"grammar T; \n" +
"a : (('x'+)=> 'x'+)?;\n";
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, "TParser", "TLexer", false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
@Test public void testDoubleQuoteEscape() {
String grammar =
"lexer grammar T; \n" +
"A : '\\\\\"';\n" + // this is A : '\\"', which should give "\\\"" at Java level;
"B : '\\\"';\n" + // this is B: '\"', which shodl give "\"" at Java level;
"C : '\\'\\'';\n" + // this is C: '\'\'', which shoudl give "''" at Java level
"D : '\\k';\n"; // this is D: '\k', which shoudl give just "k" at Java level;
boolean found =
rawGenerateAndBuildRecognizer(
"T.g", grammar, null, "T", false);
boolean expecting = true; // should be ok
assertEquals(expecting, found);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestSymbolDefinitions.java 0000644 0001750 0001750 00000073542 11256465234 026155 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.Tool;
import org.antlr.analysis.Label;
import org.antlr.codegen.CodeGenerator;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.*;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import java.util.*;
public class TestSymbolDefinitions extends BaseTest {
/** Public default constructor used by TestRig */
public TestSymbolDefinitions() {
}
@Test public void testParserSimpleTokens() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A | B;\n" +
"b : C ;");
String rules = "a, b";
String tokenNames = "A, B, C";
checkSymbols(g, rules, tokenNames);
}
@Test public void testParserTokensSection() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n" +
"tokens {\n" +
" C;\n" +
" D;" +
"}\n"+
"a : A | B;\n" +
"b : C ;");
String rules = "a, b";
String tokenNames = "A, B, C, D";
checkSymbols(g, rules, tokenNames);
}
@Test public void testLexerTokensSection() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n" +
"tokens {\n" +
" C;\n" +
" D;" +
"}\n"+
"A : 'a';\n" +
"C : 'c' ;");
String rules = "A, C, Tokens";
String tokenNames = "A, C, D";
checkSymbols(g, rules, tokenNames);
}
@Test public void testTokensSectionWithAssignmentSection() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"tokens {\n" +
" C='c';\n" +
" D;" +
"}\n"+
"a : A | B;\n" +
"b : C ;");
String rules = "a, b";
String tokenNames = "A, B, C, D, 'c'";
checkSymbols(g, rules, tokenNames);
}
@Test public void testCombinedGrammarLiterals() throws Exception {
Grammar g = new Grammar(
"grammar t;\n"+
"a : 'begin' b 'end';\n" +
"b : C ';' ;\n" +
"ID : 'a' ;\n" +
"FOO : 'foo' ;\n" + // "foo" is not a token name
"C : 'c' ;\n"); // nor is 'c'
String rules = "a, b";
String tokenNames = "C, FOO, ID, 'begin', 'end', ';'";
checkSymbols(g, rules, tokenNames);
}
@Test public void testLiteralInParserAndLexer() throws Exception {
// 'x' is token and char in lexer rule
Grammar g = new Grammar(
"grammar t;\n" +
"a : 'x' E ; \n" +
"E: 'x' '0' ;\n"); // nor is 'c'
String literals = "['x']";
String foundLiterals = g.getStringLiterals().toString();
assertEquals(literals, foundLiterals);
String implicitLexer =
"lexer grammar t;\n" +
"\n" +
"T__5 : 'x' ;\n" +
"\n" +
"// $ANTLR src \"<string>\" 3\n" +
"E: 'x' '0' ;\n";
assertEquals(implicitLexer, g.getLexerGrammar());
}
@Test public void testCombinedGrammarWithRefToLiteralButNoTokenIDRef() throws Exception {
Grammar g = new Grammar(
"grammar t;\n"+
"a : 'a' ;\n" +
"A : 'a' ;\n");
String rules = "a";
String tokenNames = "A, 'a'";
checkSymbols(g, rules, tokenNames);
}
@Test public void testSetDoesNotMissTokenAliases() throws Exception {
Grammar g = new Grammar(
"grammar t;\n"+
"a : 'a'|'b' ;\n" +
"A : 'a' ;\n" +
"B : 'b' ;\n");
String rules = "a";
String tokenNames = "A, 'a', B, 'b'";
checkSymbols(g, rules, tokenNames);
}
@Test public void testSimplePlusEqualLabel() throws Exception {
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ids+=ID ( COMMA ids+=ID )* ;\n");
String rule = "a";
String tokenLabels = "ids";
String ruleLabels = null;
checkPlusEqualsLabels(g, rule, tokenLabels, ruleLabels);
}
@Test public void testMixedPlusEqualLabel() throws Exception {
Grammar g = new Grammar(
"grammar t;\n"+
"options {output=AST;}\n" +
"a : id+=ID ( ',' e+=expr )* ;\n" +
"expr : 'e';\n" +
"ID : 'a';\n");
String rule = "a";
String tokenLabels = "id";
String ruleLabels = "e";
checkPlusEqualsLabels(g, rule, tokenLabels, ruleLabels);
}
// T E S T L I T E R A L E S C A P E S
@Test public void testParserCharLiteralWithEscape() throws Exception {
Grammar g = new Grammar(
"grammar t;\n"+
"a : '\\n';\n");
Set literals = g.getStringLiterals();
// must store literals how they appear in the antlr grammar
assertEquals("'\\n'", literals.toArray()[0]);
}
@Test public void testTokenInTokensSectionAndTokenRuleDef() throws Exception {
// this must return A not I to the parser; calling a nonfragment rule
// from a nonfragment rule does not set the overall token.
String grammar =
"grammar P;\n" +
"tokens { B='}'; }\n"+
"a : A B {System.out.println(input);} ;\n"+
"A : 'a' ;\n" +
"B : '}' ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "a}", false);
assertEquals("a}\n", found);
}
@Test public void testTokenInTokensSectionAndTokenRuleDef2() throws Exception {
// this must return A not I to the parser; calling a nonfragment rule
// from a nonfragment rule does not set the overall token.
String grammar =
"grammar P;\n" +
"tokens { B='}'; }\n"+
"a : A '}' {System.out.println(input);} ;\n"+
"A : 'a' ;\n" +
"B : '}' {/* */} ;\n"+
"WS : (' '|'\\n') {$channel=HIDDEN;} ;";
String found = execParser("P.g", grammar, "PParser", "PLexer",
"a", "a}", false);
assertEquals("a}\n", found);
}
@Test public void testRefToRuleWithNoReturnValue() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String grammarStr =
"grammar P;\n" +
"a : x=b ;\n" +
"b : B ;\n" +
"B : 'b' ;\n";
Grammar g = new Grammar(grammarStr);
Tool antlr = newTool();
CodeGenerator generator = new CodeGenerator(antlr, g, "Java");
g.setCodeGenerator(generator);
StringTemplate recogST = generator.genRecognizer();
String code = recogST.toString();
assertTrue("not expecting label", code.indexOf("x=b();")<0);
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
// T E S T E R R O R S
@Test public void testParserStringLiterals() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : 'begin' b ;\n" +
"b : C ;");
Object expectedArg = "'begin'";
int expectedMsgID = ErrorManager.MSG_LITERAL_NOT_ASSOCIATED_WITH_LEXER_RULE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testParserCharLiterals() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : '(' b ;\n" +
"b : C ;");
Object expectedArg = "'('";
int expectedMsgID = ErrorManager.MSG_LITERAL_NOT_ASSOCIATED_WITH_LEXER_RULE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testEmptyNotChar() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar foo;\n" +
"a : (~'x')+ ;\n");
g.buildNFA();
Object expectedArg = "'x'";
int expectedMsgID = ErrorManager.MSG_EMPTY_COMPLEMENT;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testEmptyNotToken() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar foo;\n" +
"a : (~A)+ ;\n");
g.buildNFA();
Object expectedArg = "A";
int expectedMsgID = ErrorManager.MSG_EMPTY_COMPLEMENT;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testEmptyNotSet() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
Grammar g = new Grammar(
"grammar foo;\n" +
"a : (~(A|B))+ ;\n");
g.buildNFA();
Object expectedArg = null;
int expectedMsgID = ErrorManager.MSG_EMPTY_COMPLEMENT;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testStringLiteralInParserTokensSection() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n" +
"tokens {\n" +
" B='begin';\n" +
"}\n"+
"a : A B;\n" +
"b : C ;");
Object expectedArg = "'begin'";
int expectedMsgID = ErrorManager.MSG_LITERAL_NOT_ASSOCIATED_WITH_LEXER_RULE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testCharLiteralInParserTokensSection() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n" +
"tokens {\n" +
" B='(';\n" +
"}\n"+
"a : A B;\n" +
"b : C ;");
Object expectedArg = "'('";
int expectedMsgID = ErrorManager.MSG_LITERAL_NOT_ASSOCIATED_WITH_LEXER_RULE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testCharLiteralInLexerTokensSection() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"lexer grammar t;\n" +
"tokens {\n" +
" B='(';\n" +
"}\n"+
"ID : 'a';\n");
Object expectedArg = "'('";
int expectedMsgID = ErrorManager.MSG_CANNOT_ALIAS_TOKENS_IN_LEXER;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testRuleRedefinition() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : A | B;\n" +
"a : C ;");
Object expectedArg = "a";
int expectedMsgID = ErrorManager.MSG_RULE_REDEFINITION;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testLexerRuleRedefinition() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"lexer grammar t;\n"+
"ID : 'a' ;\n" +
"ID : 'd' ;");
Object expectedArg = "ID";
int expectedMsgID = ErrorManager.MSG_RULE_REDEFINITION;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testCombinedRuleRedefinition() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"x : ID ;\n" +
"ID : 'a' ;\n" +
"x : ID ID ;");
Object expectedArg = "x";
int expectedMsgID = ErrorManager.MSG_RULE_REDEFINITION;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testUndefinedToken() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"x : ID ;");
Object expectedArg = "ID";
int expectedMsgID = ErrorManager.MSG_NO_TOKEN_DEFINITION;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsWarning(equeue, expectedMessage);
}
@Test public void testUndefinedTokenOkInParser() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"x : ID ;");
assertEquals("should not be an error", 0, equeue.errors.size());
}
@Test public void testUndefinedRule() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"x : r ;");
Object expectedArg = "r";
int expectedMsgID = ErrorManager.MSG_UNDEFINED_RULE_REF;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testLexerRuleInParser() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"X : ;");
Object expectedArg = "X";
int expectedMsgID = ErrorManager.MSG_LEXER_RULES_NOT_ALLOWED;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testParserRuleInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"lexer grammar t;\n"+
"a : ;");
Object expectedArg = "a";
int expectedMsgID = ErrorManager.MSG_PARSER_RULES_NOT_ALLOWED;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testRuleScopeConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"scope a {\n" +
" int n;\n" +
"}\n" +
"a : \n" +
" ;\n");
Object expectedArg = "a";
int expectedMsgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testTokenRuleScopeConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"scope ID {\n" +
" int n;\n" +
"}\n" +
"ID : 'a'\n" +
" ;\n");
Object expectedArg = "ID";
int expectedMsgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testTokenScopeConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"tokens { ID; }\n"+
"scope ID {\n" +
" int n;\n" +
"}\n" +
"a : \n" +
" ;\n");
Object expectedArg = "ID";
int expectedMsgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testTokenRuleScopeConflictInLexerGrammar() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"lexer grammar t;\n"+
"scope ID {\n" +
" int n;\n" +
"}\n" +
"ID : 'a'\n" +
" ;\n");
Object expectedArg = "ID";
int expectedMsgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testTokenLabelScopeConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"scope s {\n" +
" int n;\n" +
"}\n" +
"a : s=ID \n" +
" ;\n");
Object expectedArg = "s";
int expectedMsgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testRuleLabelScopeConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"scope s {\n" +
" int n;\n" +
"}\n" +
"a : s=b \n" +
" ;\n" +
"b : ;\n");
Object expectedArg = "s";
int expectedMsgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testLabelAndRuleNameConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : c=b \n" +
" ;\n" +
"b : ;\n" +
"c : ;\n");
Object expectedArg = "c";
int expectedMsgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_RULE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testLabelAndTokenNameConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a : ID=b \n" +
" ;\n" +
"b : ID ;\n" +
"c : ;\n");
Object expectedArg = "ID";
int expectedMsgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_TOKEN;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testLabelAndArgConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[int i] returns [int x]: i=ID \n" +
" ;\n");
Object expectedArg = "i";
int expectedMsgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_RULE_ARG_RETVAL;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testLabelAndParameterConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[int i] returns [int x]: x=ID \n" +
" ;\n");
Object expectedArg = "x";
int expectedMsgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_RULE_ARG_RETVAL;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testLabelRuleScopeConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a\n" +
"scope {" +
" int n;" +
"}\n" +
" : n=ID\n" +
" ;\n");
Object expectedArg = "n";
Object expectedArg2 = "a";
int expectedMsgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_RULE_SCOPE_ATTRIBUTE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testRuleScopeArgConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a[int n]\n" +
"scope {" +
" int n;" +
"}\n" +
" : \n" +
" ;\n");
Object expectedArg = "n";
Object expectedArg2 = "a";
int expectedMsgID = ErrorManager.MSG_ATTRIBUTE_CONFLICTS_WITH_RULE_ARG_RETVAL;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testRuleScopeReturnValueConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a returns [int n]\n" +
"scope {" +
" int n;" +
"}\n" +
" : \n" +
" ;\n");
Object expectedArg = "n";
Object expectedArg2 = "a";
int expectedMsgID = ErrorManager.MSG_ATTRIBUTE_CONFLICTS_WITH_RULE_ARG_RETVAL;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testRuleScopeRuleNameConflict() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"parser grammar t;\n"+
"a\n" +
"scope {" +
" int a;" +
"}\n" +
" : \n" +
" ;\n");
Object expectedArg = "a";
Object expectedArg2 = null;
int expectedMsgID = ErrorManager.MSG_ATTRIBUTE_CONFLICTS_WITH_RULE;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg, expectedArg2);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testBadGrammarOption() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Tool antlr = newTool();
Grammar g = new Grammar(antlr,
"grammar t;\n"+
"options {foo=3; language=Java;}\n" +
"a : 'a';\n");
Object expectedArg = "foo";
int expectedMsgID = ErrorManager.MSG_ILLEGAL_OPTION;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testBadRuleOption() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"a\n"+
"options {k=3; tokenVocab=blort;}\n" +
" : 'a';\n");
Object expectedArg = "tokenVocab";
int expectedMsgID = ErrorManager.MSG_ILLEGAL_OPTION;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testBadSubRuleOption() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue); // unique listener per thread
Grammar g = new Grammar(
"grammar t;\n"+
"a : ( options {k=3; language=Java;}\n" +
" : 'a'\n" +
" | 'b'\n" +
" )\n" +
" ;\n");
Object expectedArg = "language";
int expectedMsgID = ErrorManager.MSG_ILLEGAL_OPTION;
GrammarSemanticsMessage expectedMessage =
new GrammarSemanticsMessage(expectedMsgID, g, null, expectedArg);
checkGrammarSemanticsError(equeue, expectedMessage);
}
@Test public void testTokenVocabStringUsedInLexer() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String tokens =
"';'=4\n";
mkdir(tmpdir);
writeFile(tmpdir, "T.tokens", tokens);
String importer =
"lexer grammar B; \n" +
"options\t{tokenVocab=T;} \n" +
"SEMI:';' ; \n" ;
writeFile(tmpdir, "B.g", importer);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/B.g",composite);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
String expectedTokenIDToTypeMap = "[SEMI=4]";
String expectedStringLiteralToTypeMap = "{';'=4}";
String expectedTypeToTokenList = "[SEMI]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
@Test public void testTokenVocabStringUsedInCombined() throws Exception {
ErrorQueue equeue = new ErrorQueue();
ErrorManager.setErrorListener(equeue);
String tokens =
"';'=4\n";
mkdir(tmpdir);
writeFile(tmpdir, "T.tokens", tokens);
String importer =
"grammar B; \n" +
"options\t{tokenVocab=T;} \n" +
"SEMI:';' ; \n" ;
writeFile(tmpdir, "B.g", importer);
Tool antlr = newTool(new String[] {"-lib", tmpdir});
CompositeGrammar composite = new CompositeGrammar();
Grammar g = new Grammar(antlr,tmpdir+"/B.g",composite);
g.parseAndBuildAST();
g.composite.assignTokenTypes();
String expectedTokenIDToTypeMap = "[SEMI=4]";
String expectedStringLiteralToTypeMap = "{';'=4}";
String expectedTypeToTokenList = "[SEMI]";
assertEquals(expectedTokenIDToTypeMap,
realElements(g.composite.tokenIDToTypeMap).toString());
assertEquals(expectedStringLiteralToTypeMap, g.composite.stringLiteralToTypeMap.toString());
assertEquals(expectedTypeToTokenList,
realElements(g.composite.typeToTokenList).toString());
assertEquals("unexpected errors: "+equeue, 0, equeue.errors.size());
}
protected void checkPlusEqualsLabels(Grammar g,
String ruleName,
String tokenLabelsStr,
String ruleLabelsStr)
throws Exception
{
// make sure expected += labels are there
Rule r = g.getRule(ruleName);
StringTokenizer st = new StringTokenizer(tokenLabelsStr, ", ");
Set tokenLabels = null;
while ( st.hasMoreTokens() ) {
if ( tokenLabels==null ) {
tokenLabels = new HashSet();
}
String labelName = st.nextToken();
tokenLabels.add(labelName);
}
Set ruleLabels = null;
if ( ruleLabelsStr!=null ) {
st = new StringTokenizer(ruleLabelsStr, ", ");
ruleLabels = new HashSet();
while ( st.hasMoreTokens() ) {
String labelName = st.nextToken();
ruleLabels.add(labelName);
}
}
assertTrue("token += labels mismatch; "+tokenLabels+"!="+r.tokenListLabels,
(tokenLabels!=null && r.tokenListLabels!=null) ||
(tokenLabels==null && r.tokenListLabels==null));
assertTrue("rule += labels mismatch; "+ruleLabels+"!="+r.ruleListLabels,
(ruleLabels!=null && r.ruleListLabels!=null) ||
(ruleLabels==null && r.ruleListLabels==null));
if ( tokenLabels!=null ) {
assertEquals(tokenLabels, r.tokenListLabels.keySet());
}
if ( ruleLabels!=null ) {
assertEquals(ruleLabels, r.ruleListLabels.keySet());
}
}
protected void checkSymbols(Grammar g,
String rulesStr,
String tokensStr)
throws Exception
{
Set tokens = g.getTokenDisplayNames();
// make sure expected tokens are there
StringTokenizer st = new StringTokenizer(tokensStr, ", ");
while ( st.hasMoreTokens() ) {
String tokenName = st.nextToken();
assertTrue("token "+tokenName+" expected",
g.getTokenType(tokenName)!=Label.INVALID);
tokens.remove(tokenName);
}
// make sure there are not any others (other than <EOF> etc...)
for (Iterator iter = tokens.iterator(); iter.hasNext();) {
String tokenName = (String) iter.next();
assertTrue("unexpected token name "+tokenName,
g.getTokenType(tokenName)<Label.MIN_TOKEN_TYPE);
}
// make sure all expected rules are there
st = new StringTokenizer(rulesStr, ", ");
int n = 0;
while ( st.hasMoreTokens() ) {
String ruleName = st.nextToken();
assertNotNull("rule "+ruleName+" expected", g.getRule(ruleName));
n++;
}
Collection rules = g.getRules();
//System.out.println("rules="+rules);
// make sure there are no extra rules
assertEquals("number of rules mismatch; expecting "+n+"; found "+rules.size(), n, rules.size());
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestNFAConstruction.java 0000644 0001750 0001750 00000065647 11256465234 025542 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.analysis.State;
import org.antlr.tool.FASerializer;
import org.antlr.tool.Grammar;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestNFAConstruction extends BaseTest {
/** Public default constructor used by TestRig */
public TestNFAConstruction() {
}
@Test public void testA() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-A->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
}
@Test public void testAB() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A B ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-A->.s3\n" +
".s3-B->.s4\n" +
".s4->:s5\n" +
":s5-EOF->.s6\n";
checkRule(g, "a", expecting);
}
@Test public void testAorB() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A | B {;} ;");
/* expecting (0)--Ep-->(1)--Ep-->(2)--A-->(3)--Ep-->(4)--Ep-->(5,end)
| ^
(6)--Ep-->(7)--B-->(8)--------|
*/
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s1->.s7\n" +
".s10->.s4\n" +
".s2-A->.s3\n" +
".s3->.s4\n" +
".s4->:s5\n" +
".s7->.s8\n" +
".s8-B->.s9\n" +
".s9-{}->.s10\n" +
":s5-EOF->.s6\n";
checkRule(g, "a", expecting);
}
@Test public void testRangeOrRange() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : ('a'..'c' 'h' | 'q' 'j'..'l') ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10-'q'->.s11\n" +
".s11-'j'..'l'->.s12\n" +
".s12->.s6\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3-'a'..'c'->.s4\n" +
".s4-'h'->.s5\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s10\n" +
":s7-<EOT>->.s8\n";
checkRule(g, "A", expecting);
}
@Test public void testRange() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : 'a'..'c' ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-'a'..'c'->.s3\n" +
".s3->:s4\n" +
":s4-<EOT>->.s5\n";
checkRule(g, "A", expecting);
}
@Test public void testCharSetInParser() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : A|'b' ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-A..'b'->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
}
@Test public void testABorCD() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A B | C D;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s1->.s8\n" +
".s10-D->.s11\n" +
".s11->.s5\n" +
".s2-A->.s3\n" +
".s3-B->.s4\n" +
".s4->.s5\n" +
".s5->:s6\n" +
".s8->.s9\n" +
".s9-C->.s10\n" +
":s6-EOF->.s7\n";
checkRule(g, "a", expecting);
}
@Test public void testbA() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b A ;\n"+
"b : B ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4->.s5\n" +
".s5-B->.s6\n" +
".s6->:s7\n" +
".s8-A->.s9\n" +
".s9->:s10\n" +
":s10-EOF->.s11\n" +
":s7->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testbA_bC() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : b A ;\n"+
"b : B ;\n"+
"c : b C;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s12->.s13\n" +
".s13-C->.s14\n" +
".s14->:s15\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4->.s5\n" +
".s5-B->.s6\n" +
".s6->:s7\n" +
".s8-A->.s9\n" +
".s9->:s10\n" +
":s10-EOF->.s11\n" +
":s15-EOF->.s16\n" +
":s7->.s12\n" +
":s7->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testAorEpsilon() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A | ;");
/* expecting (0)--Ep-->(1)--Ep-->(2)--A-->(3)--Ep-->(4)--Ep-->(5,end)
| ^
(6)--Ep-->(7)--Ep-->(8)-------|
*/
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s1->.s7\n" +
".s2-A->.s3\n" +
".s3->.s4\n" +
".s4->:s5\n" +
".s7->.s8\n" +
".s8->.s9\n" +
".s9->.s4\n" +
":s5-EOF->.s6\n";
checkRule(g, "a", expecting);
}
@Test public void testAOptional() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A)?;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s8\n" +
".s3-A->.s4\n" +
".s4->.s5\n" +
".s5->:s6\n" +
".s8->.s5\n" +
":s6-EOF->.s7\n";
checkRule(g, "a", expecting);
}
@Test public void testNakedAoptional() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A?;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s8\n" +
".s3-A->.s4\n" +
".s4->.s5\n" +
".s5->:s6\n" +
".s8->.s5\n" +
":s6-EOF->.s7\n";
checkRule(g, "a", expecting);
}
@Test public void testAorBthenC() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A | B) C;");
/* expecting
(0)--Ep-->(1)--Ep-->(2)--A-->(3)--Ep-->(4)--Ep-->(5)--C-->(6)--Ep-->(7,end)
| ^
(8)--Ep-->(9)--B-->(10)-------|
*/
}
@Test public void testAplus() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A)+;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testNakedAplus() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A+;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testAplusNonGreedy() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : (options {greedy=false;}:'0'..'9')+ ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4-'0'..'9'->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
":s7-<EOT>->.s8\n";
checkRule(g, "A", expecting);
}
@Test public void testAorBplus() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A | B{action})+ ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->.s11\n" +
".s11-B->.s12\n" +
".s12-{}->.s13\n" +
".s13->.s6\n" +
".s2->.s3\n" +
".s3->.s10\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s6\n" +
".s6->.s3\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "a", expecting);
}
@Test public void testAorBorEmptyPlus() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A | B | )+ ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->.s11\n" +
".s10->.s13\n" +
".s11-B->.s12\n" +
".s12->.s6\n" +
".s13->.s14\n" +
".s14->.s15\n" +
".s15->.s6\n" +
".s2->.s3\n" +
".s3->.s10\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s6\n" +
".s6->.s3\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "a", expecting);
}
@Test public void testAStar() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A)*;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s6\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testNestedAstar() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A*)*;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->:s11\n" +
".s13->.s8\n" +
".s14->.s10\n" +
".s2->.s14\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4->.s13\n" +
".s4->.s5\n" +
".s5->.s6\n" +
".s6-A->.s7\n" +
".s7->.s5\n" +
".s7->.s8\n" +
".s8->.s9\n" +
".s9->.s10\n" +
".s9->.s3\n" +
":s11-EOF->.s12\n";
checkRule(g, "a", expecting);
}
@Test public void testPlusNestedInStar() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A+)*;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->:s11\n" +
".s13->.s10\n" +
".s2->.s13\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4->.s5\n" +
".s5->.s6\n" +
".s6-A->.s7\n" +
".s7->.s5\n" +
".s7->.s8\n" +
".s8->.s9\n" +
".s9->.s10\n" +
".s9->.s3\n" +
":s11-EOF->.s12\n";
checkRule(g, "a", expecting);
}
@Test public void testStarNestedInPlus() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A*)+;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->:s11\n" +
".s13->.s8\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4->.s13\n" +
".s4->.s5\n" +
".s5->.s6\n" +
".s6-A->.s7\n" +
".s7->.s5\n" +
".s7->.s8\n" +
".s8->.s9\n" +
".s9->.s10\n" +
".s9->.s3\n" +
":s11-EOF->.s12\n";
checkRule(g, "a", expecting);
}
@Test public void testNakedAstar() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : A*;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s6\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testAorBstar() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : (A | B{action})* ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->.s11\n" +
".s11-B->.s12\n" +
".s12-{}->.s13\n" +
".s13->.s6\n" +
".s14->.s7\n" +
".s2->.s14\n" +
".s2->.s3\n" +
".s3->.s10\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s6\n" +
".s6->.s3\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "a", expecting);
}
@Test public void testAorBOptionalSubrule() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : ( A | B )? ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s8\n" +
".s3-A..B->.s4\n" +
".s4->.s5\n" +
".s5->:s6\n" +
".s8->.s5\n" +
":s6-EOF->.s7\n";
checkRule(g, "a", expecting);
}
@Test public void testPredicatedAorB() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? A | {p2}? B ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s1->.s8\n" +
".s10-B->.s11\n" +
".s11->.s5\n" +
".s2-{p1}?->.s3\n" +
".s3-A->.s4\n" +
".s4->.s5\n" +
".s5->:s6\n" +
".s8->.s9\n" +
".s9-{p2}?->.s10\n" +
":s6-EOF->.s7\n";
checkRule(g, "a", expecting);
}
@Test public void testMultiplePredicates() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : {p1}? {p1a}? A | {p2}? B | {p3} b;\n" +
"b : {p4}? B ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s1->.s9\n" +
".s10-{p2}?->.s11\n" +
".s11-B->.s12\n" +
".s12->.s6\n" +
".s13->.s14\n" +
".s14-{}->.s15\n" +
".s15->.s16\n" +
".s16->.s17\n" +
".s17->.s18\n" +
".s18-{p4}?->.s19\n" +
".s19-B->.s20\n" +
".s2-{p1}?->.s3\n" +
".s20->:s21\n" +
".s22->.s6\n" +
".s3-{p1a}?->.s4\n" +
".s4-A->.s5\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s10\n" +
".s9->.s13\n" +
":s21->.s22\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testSets() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"a : ( A | B )+ ;\n" +
"b : ( A | B{;} )+ ;\n" +
"c : (A|B) (A|B) ;\n" +
"d : ( A | B )* ;\n" +
"e : ( A | B )? ;");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4-A..B->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->.s11\n" +
".s11-B->.s12\n" +
".s12-{}->.s13\n" +
".s13->.s6\n" +
".s2->.s3\n" +
".s3->.s10\n" +
".s3->.s4\n" +
".s4-A->.s5\n" +
".s5->.s6\n" +
".s6->.s3\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "b", expecting);
expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-A..B->.s3\n" +
".s3-A..B->.s4\n" +
".s4->:s5\n" +
":s5-EOF->.s6\n";
checkRule(g, "c", expecting);
expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3->.s4\n" +
".s4-A..B->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s6\n" +
":s7-EOF->.s8\n";
checkRule(g, "d", expecting);
expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s8\n" +
".s3-A..B->.s4\n" +
".s4->.s5\n" +
".s5->:s6\n" +
".s8->.s5\n" +
":s6-EOF->.s7\n";
checkRule(g, "e", expecting);
}
@Test public void testNotSet() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"tokens { A; B; C; }\n"+
"a : ~A ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-B..C->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
String expectingGrammarStr =
"1:8: parser grammar P;\n" +
"a : ~ A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testNotSingletonBlockSet() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"tokens { A; B; C; }\n"+
"a : ~(A) ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-B..C->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
String expectingGrammarStr =
"1:8: parser grammar P;\n" +
"a : ~ ( A ) ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testNotCharSet() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : ~'3' ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-{'\\u0000'..'2', '4'..'\\uFFFF'}->.s3\n" +
".s3->:s4\n" +
":s4-<EOT>->.s5\n";
checkRule(g, "A", expecting);
String expectingGrammarStr =
"1:7: lexer grammar P;\n" +
"A : ~ '3' ;\n"+
"Tokens : A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testNotBlockSet() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : ~('3'|'b') ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-{'\\u0000'..'2', '4'..'a', 'c'..'\\uFFFF'}->.s3\n" +
".s3->:s4\n" +
":s4-<EOT>->.s5\n";
checkRule(g, "A", expecting);
String expectingGrammarStr =
"1:7: lexer grammar P;\n" +
"A : ~ ( '3' | 'b' ) ;\n" +
"Tokens : A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testNotSetLoop() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : ~('3')* ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3->.s4\n" +
".s4-{'\\u0000'..'2', '4'..'\\uFFFF'}->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s6\n" +
":s7-<EOT>->.s8\n";
checkRule(g, "A", expecting);
String expectingGrammarStr =
"1:7: lexer grammar P;\n" +
"A : (~ ( '3' ) )* ;\n" +
"Tokens : A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testNotBlockSetLoop() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : ~('3'|'b')* ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3->.s4\n" +
".s4-{'\\u0000'..'2', '4'..'a', 'c'..'\\uFFFF'}->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s6\n" +
":s7-<EOT>->.s8\n";
checkRule(g, "A", expecting);
String expectingGrammarStr =
"1:7: lexer grammar P;\n" +
"A : (~ ( '3' | 'b' ) )* ;\n" +
"Tokens : A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testSetsInCombinedGrammarSentToLexer() throws Exception {
// not sure this belongs in this test suite, but whatever.
Grammar g = new Grammar(
"grammar t;\n"+
"A : '{' ~('}')* '}';\n");
String result = g.getLexerGrammar();
String expecting =
"lexer grammar t;\n" +
"\n" +
"// $ANTLR src \"<string>\" 2\n"+
"A : '{' ~('}')* '}';\n";
assertEquals(result, expecting);
}
@Test public void testLabeledNotSet() throws Exception {
Grammar g = new Grammar(
"parser grammar P;\n"+
"tokens { A; B; C; }\n"+
"a : t=~A ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-B..C->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
String expectingGrammarStr =
"1:8: parser grammar P;\n" +
"a : t=~ A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testLabeledNotCharSet() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : t=~'3' ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-{'\\u0000'..'2', '4'..'\\uFFFF'}->.s3\n" +
".s3->:s4\n" +
":s4-<EOT>->.s5\n";
checkRule(g, "A", expecting);
String expectingGrammarStr =
"1:7: lexer grammar P;\n" +
"A : t=~ '3' ;\n"+
"Tokens : A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testLabeledNotBlockSet() throws Exception {
Grammar g = new Grammar(
"lexer grammar P;\n"+
"A : t=~('3'|'b') ;\n");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-{'\\u0000'..'2', '4'..'a', 'c'..'\\uFFFF'}->.s3\n" +
".s3->:s4\n" +
":s4-<EOT>->.s5\n";
checkRule(g, "A", expecting);
String expectingGrammarStr =
"1:7: lexer grammar P;\n" +
"A : t=~ ( '3' | 'b' ) ;\n" +
"Tokens : A ;";
assertEquals(expectingGrammarStr, g.toString());
}
@Test public void testEscapedCharLiteral() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : '\\n';");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-'\\n'->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
}
@Test public void testEscapedStringLiteral() throws Exception {
Grammar g = new Grammar(
"grammar P;\n"+
"a : 'a\\nb\\u0030c\\'';");
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-'a\\nb\\u0030c\\''->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
}
// AUTO BACKTRACKING STUFF
@Test public void testAutoBacktracking_RuleBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : 'a'{;}|'b';"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s1->.s9\n" +
".s10-'b'->.s11\n" +
".s11->.s6\n" +
".s2-{synpred1_t}?->.s3\n" +
".s3-'a'->.s4\n" +
".s4-{}->.s5\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s10\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_RuleSetBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : 'a'|'b';"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-'a'..'b'->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_SimpleBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'{;}|'b') ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->.s11\n" +
".s11-'b'->.s12\n" +
".s12->.s7\n" +
".s2->.s10\n" +
".s2->.s3\n" +
".s3-{synpred1_t}?->.s4\n" +
".s4-'a'->.s5\n" +
".s5-{}->.s6\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_SetBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'|'b') ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2-'a'..'b'->.s3\n" +
".s3->:s4\n" +
":s4-EOF->.s5\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_StarBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'{;}|'b')* ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s12->.s13\n" +
".s13-{synpred2_t}?->.s14\n" +
".s14-'b'->.s15\n" +
".s15->.s8\n" +
".s16->.s9\n" +
".s2->.s16\n" +
".s2->.s3\n" +
".s3->.s12\n" +
".s3->.s4\n" +
".s4-{synpred1_t}?->.s5\n" +
".s5-'a'->.s6\n" +
".s6-{}->.s7\n" +
".s7->.s8\n" +
".s8->.s3\n" +
".s8->.s9\n" +
".s9->:s10\n" +
":s10-EOF->.s11\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_StarSetBlock_IgnoresPreds() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'|'b')* ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3->.s4\n" +
".s4-'a'..'b'->.s5\n" +
".s5->.s3\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s6\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_StarSetBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'|'b'{;})* ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s11->.s12\n" +
".s12-{synpred2_t}?->.s13\n" +
".s13-'b'->.s14\n" +
".s14-{}->.s15\n" +
".s15->.s7\n" +
".s16->.s8\n" +
".s2->.s16\n" +
".s2->.s3\n" +
".s3->.s11\n" +
".s3->.s4\n" +
".s4-{synpred1_t}?->.s5\n" +
".s5-'a'->.s6\n" +
".s6->.s7\n" +
".s7->.s3\n" +
".s7->.s8\n" +
".s8->:s9\n" +
":s9-EOF->.s10\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_StarBlock1Alt() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a')* ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->.s7\n" +
".s2->.s10\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4-{synpred1_t}?->.s5\n" +
".s5-'a'->.s6\n" +
".s6->.s3\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_PlusBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'{;}|'b')+ ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s12->.s13\n" +
".s13-{synpred2_t}?->.s14\n" +
".s14-'b'->.s15\n" +
".s15->.s8\n" +
".s2->.s3\n" +
".s3->.s12\n" +
".s3->.s4\n" +
".s4-{synpred1_t}?->.s5\n" +
".s5-'a'->.s6\n" +
".s6-{}->.s7\n" +
".s7->.s8\n" +
".s8->.s3\n" +
".s8->.s9\n" +
".s9->:s10\n" +
":s10-EOF->.s11\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_PlusSetBlock() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'|'b'{;})+ ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s11->.s12\n" +
".s12-{synpred2_t}?->.s13\n" +
".s13-'b'->.s14\n" +
".s14-{}->.s15\n" +
".s15->.s7\n" +
".s2->.s3\n" +
".s3->.s11\n" +
".s3->.s4\n" +
".s4-{synpred1_t}?->.s5\n" +
".s5-'a'->.s6\n" +
".s6->.s7\n" +
".s7->.s3\n" +
".s7->.s8\n" +
".s8->:s9\n" +
":s9-EOF->.s10\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_PlusBlock1Alt() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a')+ ;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s3->.s4\n" +
".s4-{synpred1_t}?->.s5\n" +
".s5-'a'->.s6\n" +
".s6->.s3\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_OptionalBlock2Alts() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a'{;}|'b')?;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s10->.s11\n" +
".s10->.s14\n" +
".s11-{synpred2_t}?->.s12\n" +
".s12-'b'->.s13\n" +
".s13->.s7\n" +
".s14->.s7\n" +
".s2->.s10\n" +
".s2->.s3\n" +
".s3-{synpred1_t}?->.s4\n" +
".s4-'a'->.s5\n" +
".s5-{}->.s6\n" +
".s6->.s7\n" +
".s7->:s8\n" +
":s8-EOF->.s9\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_OptionalBlock1Alt() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a')?;"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s2->.s3\n" +
".s2->.s9\n" +
".s3-{synpred1_t}?->.s4\n" +
".s4-'a'->.s5\n" +
".s5->.s6\n" +
".s6->:s7\n" +
".s9->.s6\n" +
":s7-EOF->.s8\n";
checkRule(g, "a", expecting);
}
@Test public void testAutoBacktracking_ExistingPred() throws Exception {
Grammar g = new Grammar(
"grammar t;\n" +
"options {backtrack=true;}\n"+
"a : ('a')=> 'a' | 'b';"
);
String expecting =
".s0->.s1\n" +
".s1->.s2\n" +
".s1->.s8\n" +
".s10->.s5\n" +
".s2-{synpred1_t}?->.s3\n" +
".s3-'a'->.s4\n" +
".s4->.s5\n" +
".s5->:s6\n" +
".s8->.s9\n" +
".s9-'b'->.s10\n" +
":s6-EOF->.s7\n";
checkRule(g, "a", expecting);
}
private void checkRule(Grammar g, String rule, String expecting)
{
g.buildNFA();
State startState = g.getRuleStartState(rule);
FASerializer serializer = new FASerializer(g);
String result = serializer.serialize(startState);
//System.out.print(result);
assertEquals(expecting, result);
}
}
antlr-3.2/tool/src/test/java/org/antlr/test/TestInterpretedLexing.java 0000644 0001750 0001750 00000014465 11256465234 026147 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.test;
import org.antlr.tool.Grammar;
import org.antlr.tool.Interpreter;
import org.antlr.runtime.*;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
public class TestInterpretedLexing extends BaseTest {
/*
static class Tracer implements ANTLRDebugInterface {
Grammar g;
public DebugActions(Grammar g) {
this.g = g;
}
public void enterRule(String ruleName) {
System.out.println("enterRule("+ruleName+")");
}
public void exitRule(String ruleName) {
System.out.println("exitRule("+ruleName+")");
}
public void matchElement(int type) {
System.out.println("matchElement("+g.getTokenName(type)+")");
}
public void mismatchedElement(MismatchedTokenException e) {
System.out.println(e);
e.printStackTrace(System.out);
}
public void mismatchedSet(MismatchedSetException e) {
System.out.println(e);
e.printStackTrace(System.out);
}
public void noViableAlt(NoViableAltException e) {
System.out.println(e);
e.printStackTrace(System.out);
}
}
*/
/** Public default constructor used by TestRig */
public TestInterpretedLexing() {
}
@Test public void testSimpleAltCharTest() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a' | 'b' | 'c';");
final int Atype = g.getTokenType("A");
Interpreter engine = new Interpreter(g, new ANTLRStringStream("a"));
engine = new Interpreter(g, new ANTLRStringStream("b"));
Token result = engine.scan("A");
assertEquals(result.getType(), Atype);
engine = new Interpreter(g, new ANTLRStringStream("c"));
result = engine.scan("A");
assertEquals(result.getType(), Atype);
}
@Test public void testSingleRuleRef() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : 'a' B 'c' ;\n" +
"B : 'b' ;\n");
final int Atype = g.getTokenType("A");
Interpreter engine = new Interpreter(g, new ANTLRStringStream("abc")); // should ignore the x
Token result = engine.scan("A");
assertEquals(result.getType(), Atype);
}
@Test public void testSimpleLoop() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"INT : (DIGIT)+ ;\n"+
"fragment DIGIT : '0'..'9';\n");
final int INTtype = g.getTokenType("INT");
Interpreter engine = new Interpreter(g, new ANTLRStringStream("12x")); // should ignore the x
Token result = engine.scan("INT");
assertEquals(result.getType(), INTtype);
engine = new Interpreter(g, new ANTLRStringStream("1234"));
result = engine.scan("INT");
assertEquals(result.getType(), INTtype);
}
@Test public void testMultAltLoop() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : ('0'..'9'|'a'|'b')+ ;\n");
final int Atype = g.getTokenType("A");
Interpreter engine = new Interpreter(g, new ANTLRStringStream("a"));
Token result = engine.scan("A");
engine = new Interpreter(g, new ANTLRStringStream("a"));
result = engine.scan("A");
assertEquals(result.getType(), Atype);
engine = new Interpreter(g, new ANTLRStringStream("1234"));
result = engine.scan("A");
assertEquals(result.getType(), Atype);
engine = new Interpreter(g, new ANTLRStringStream("aaa"));
result = engine.scan("A");
assertEquals(result.getType(), Atype);
engine = new Interpreter(g, new ANTLRStringStream("aaaa9"));
result = engine.scan("A");
assertEquals(result.getType(), Atype);
engine = new Interpreter(g, new ANTLRStringStream("b"));
result = engine.scan("A");
assertEquals(result.getType(), Atype);
engine = new Interpreter(g, new ANTLRStringStream("baa"));
result = engine.scan("A");
assertEquals(result.getType(), Atype);
}
@Test public void testSimpleLoops() throws Exception {
Grammar g = new Grammar(
"lexer grammar t;\n"+
"A : ('0'..'9')+ '.' ('0'..'9')* | ('0'..'9')+ ;\n");
final int Atype = g.getTokenType("A");
CharStream input = new ANTLRStringStream("1234.5");
Interpreter engine = new Interpreter(g, input);
Token result = engine.scan("A");
assertEquals(result.getType(), Atype);
}
@Test public void testTokensRules() throws Exception {
Grammar pg = new Grammar(
"grammar p;\n"+
"a : (INT|FLOAT|WS)+;\n");
Grammar g = new Grammar();
g.importTokenVocabulary(pg);
g.setFileName("<string>");
g.setGrammarContent(
"lexer grammar t;\n"+
"INT : (DIGIT)+ ;\n"+
"FLOAT : (DIGIT)+ '.' (DIGIT)* ;\n"+
"fragment DIGIT : '0'..'9';\n" +
"WS : (' ')+ {channel=99;};\n");
CharStream input = new ANTLRStringStream("123 139.52");
Interpreter lexEngine = new Interpreter(g, input);
CommonTokenStream tokens = new CommonTokenStream(lexEngine);
String result = tokens.toString();
//System.out.println(result);
String expecting = "123 139.52";
assertEquals(result,expecting);
}
}
antlr-3.2/tool/src/main/ 0000755 0001750 0001750 00000000000 11410174110 015112 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr3/ 0000755 0001750 0001750 00000000000 11410174107 016323 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr3/org/ 0000755 0001750 0001750 00000000000 11410174107 017112 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr3/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 020232 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr3/org/antlr/grammar/ 0000755 0001750 0001750 00000000000 11410174107 021660 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr3/org/antlr/grammar/v3/ 0000755 0001750 0001750 00000000000 11410174107 022210 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr3/org/antlr/grammar/v3/ActionTranslator.g 0000644 0001750 0001750 00000063271 11256465226 025675 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
lexer grammar ActionTranslator;
options {
filter=true; // try all non-fragment rules in order specified
// output=template; TODO: can we make tokens return templates somehow?
}
@header {
package org.antlr.grammar.v3;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.runtime.*;
import org.antlr.tool.*;
import org.antlr.codegen.*;
import org.antlr.runtime.*;
import java.util.List;
import java.util.ArrayList;
import org.antlr.grammar.v2.ANTLRParser;
}
@members {
public List chunks = new ArrayList();
Rule enclosingRule;
int outerAltNum;
Grammar grammar;
CodeGenerator generator;
antlr.Token actionToken;
public ActionTranslator(CodeGenerator generator,
String ruleName,
GrammarAST actionAST)
{
this(new ANTLRStringStream(actionAST.token.getText()));
this.generator = generator;
this.grammar = generator.grammar;
this.enclosingRule = grammar.getLocallyDefinedRule(ruleName);
this.actionToken = actionAST.token;
this.outerAltNum = actionAST.outerAltNum;
}
public ActionTranslator(CodeGenerator generator,
String ruleName,
antlr.Token actionToken,
int outerAltNum)
{
this(new ANTLRStringStream(actionToken.getText()));
this.generator = generator;
grammar = generator.grammar;
this.enclosingRule = grammar.getRule(ruleName);
this.actionToken = actionToken;
this.outerAltNum = outerAltNum;
}
/** Return a list of strings and StringTemplate objects that
* represent the translated action.
*/
public List translateToChunks() {
// System.out.println("###\naction="+action);
Token t;
do {
t = nextToken();
} while ( t.getType()!= Token.EOF );
return chunks;
}
public String translate() {
List theChunks = translateToChunks();
//System.out.println("chunks="+a.chunks);
StringBuffer buf = new StringBuffer();
for (int i = 0; i < theChunks.size(); i++) {
Object o = (Object) theChunks.get(i);
buf.append(o);
}
//System.out.println("translated: "+buf.toString());
return buf.toString();
}
public List translateAction(String action) {
String rname = null;
if ( enclosingRule!=null ) {
rname = enclosingRule.name;
}
ActionTranslator translator =
new ActionTranslator(generator,
rname,
new antlr.CommonToken(ANTLRParser.ACTION,action),outerAltNum);
return translator.translateToChunks();
}
public boolean isTokenRefInAlt(String id) {
return enclosingRule.getTokenRefsInAlt(id, outerAltNum)!=null;
}
public boolean isRuleRefInAlt(String id) {
return enclosingRule.getRuleRefsInAlt(id, outerAltNum)!=null;
}
public Grammar.LabelElementPair getElementLabel(String id) {
return enclosingRule.getLabel(id);
}
public void checkElementRefUniqueness(String ref, boolean isToken) {
List refs = null;
if ( isToken ) {
refs = enclosingRule.getTokenRefsInAlt(ref, outerAltNum);
}
else {
refs = enclosingRule.getRuleRefsInAlt(ref, outerAltNum);
}
if ( refs!=null && refs.size()>1 ) {
ErrorManager.grammarError(ErrorManager.MSG_NONUNIQUE_REF,
grammar,
actionToken,
ref);
}
}
/** For \$rulelabel.name, return the Attribute found for name. It
* will be a predefined property or a return value.
*/
public Attribute getRuleLabelAttribute(String ruleName, String attrName) {
Rule r = grammar.getRule(ruleName);
AttributeScope scope = r.getLocalAttributeScope(attrName);
if ( scope!=null && !scope.isParameterScope ) {
return scope.getAttribute(attrName);
}
return null;
}
AttributeScope resolveDynamicScope(String scopeName) {
if ( grammar.getGlobalScope(scopeName)!=null ) {
return grammar.getGlobalScope(scopeName);
}
Rule scopeRule = grammar.getRule(scopeName);
if ( scopeRule!=null ) {
return scopeRule.ruleScope;
}
return null; // not a valid dynamic scope
}
protected StringTemplate template(String name) {
StringTemplate st = generator.getTemplates().getInstanceOf(name);
chunks.add(st);
return st;
}
}
/** $x.y x is enclosing rule, y is a return value, parameter, or
* predefined property.
*
* r[int i] returns [int j]
* : {$r.i, $r.j, $r.start, $r.stop, $r.st, $r.tree}
* ;
*/
SET_ENCLOSING_RULE_SCOPE_ATTR
: '$' x=ID '.' y=ID WS? '=' expr=ATTR_VALUE_EXPR ';'
{enclosingRule!=null &&
$x.text.equals(enclosingRule.name) &&
enclosingRule.getLocalAttributeScope($y.text)!=null}?
//{System.out.println("found \$rule.attr");}
{
StringTemplate st = null;
AttributeScope scope = enclosingRule.getLocalAttributeScope($y.text);
if ( scope.isPredefinedRuleScope ) {
if ( $y.text.equals("st") || $y.text.equals("tree") ) {
st = template("ruleSetPropertyRef_"+$y.text);
grammar.referenceRuleLabelPredefinedAttribute($x.text);
st.setAttribute("scope", $x.text);
st.setAttribute("attr", $y.text);
st.setAttribute("expr", translateAction($expr.text));
} else {
ErrorManager.grammarError(ErrorManager.MSG_WRITE_TO_READONLY_ATTR,
grammar,
actionToken,
$x.text,
$y.text);
}
}
else if ( scope.isPredefinedLexerRuleScope ) {
// this is a better message to emit than the previous one...
ErrorManager.grammarError(ErrorManager.MSG_WRITE_TO_READONLY_ATTR,
grammar,
actionToken,
$x.text,
$y.text);
}
else if ( scope.isParameterScope ) {
st = template("parameterSetAttributeRef");
st.setAttribute("attr", scope.getAttribute($y.text));
st.setAttribute("expr", translateAction($expr.text));
}
else { // must be return value
st = template("returnSetAttributeRef");
st.setAttribute("ruleDescriptor", enclosingRule);
st.setAttribute("attr", scope.getAttribute($y.text));
st.setAttribute("expr", translateAction($expr.text));
}
}
;
ENCLOSING_RULE_SCOPE_ATTR
: '$' x=ID '.' y=ID {enclosingRule!=null &&
$x.text.equals(enclosingRule.name) &&
enclosingRule.getLocalAttributeScope($y.text)!=null}?
//{System.out.println("found \$rule.attr");}
{
if ( isRuleRefInAlt($x.text) ) {
ErrorManager.grammarError(ErrorManager.MSG_RULE_REF_AMBIG_WITH_RULE_IN_ALT,
grammar,
actionToken,
$x.text);
}
StringTemplate st = null;
AttributeScope scope = enclosingRule.getLocalAttributeScope($y.text);
if ( scope.isPredefinedRuleScope ) {
st = template("rulePropertyRef_"+$y.text);
grammar.referenceRuleLabelPredefinedAttribute($x.text);
st.setAttribute("scope", $x.text);
st.setAttribute("attr", $y.text);
}
else if ( scope.isPredefinedLexerRuleScope ) {
// perhaps not the most precise error message to use, but...
ErrorManager.grammarError(ErrorManager.MSG_RULE_HAS_NO_ARGS,
grammar,
actionToken,
$x.text);
}
else if ( scope.isParameterScope ) {
st = template("parameterAttributeRef");
st.setAttribute("attr", scope.getAttribute($y.text));
}
else { // must be return value
st = template("returnAttributeRef");
st.setAttribute("ruleDescriptor", enclosingRule);
st.setAttribute("attr", scope.getAttribute($y.text));
}
}
;
/** Setting $tokenlabel.attr or $tokenref.attr where attr is predefined property of a token is an error. */
SET_TOKEN_SCOPE_ATTR
: '$' x=ID '.' y=ID WS? '='
{enclosingRule!=null && input.LA(1)!='=' &&
(enclosingRule.getTokenLabel($x.text)!=null||
isTokenRefInAlt($x.text)) &&
AttributeScope.tokenScope.getAttribute($y.text)!=null}?
//{System.out.println("found \$tokenlabel.attr or \$tokenref.attr");}
{
ErrorManager.grammarError(ErrorManager.MSG_WRITE_TO_READONLY_ATTR,
grammar,
actionToken,
$x.text,
$y.text);
}
;
/** $tokenlabel.attr or $tokenref.attr where attr is predefined property of a token.
* If in lexer grammar, only translate for strings and tokens (rule refs)
*/
TOKEN_SCOPE_ATTR
: '$' x=ID '.' y=ID {enclosingRule!=null &&
(enclosingRule.getTokenLabel($x.text)!=null||
isTokenRefInAlt($x.text)) &&
AttributeScope.tokenScope.getAttribute($y.text)!=null &&
(grammar.type!=Grammar.LEXER ||
getElementLabel($x.text).elementRef.token.getType()==ANTLRParser.TOKEN_REF ||
getElementLabel($x.text).elementRef.token.getType()==ANTLRParser.STRING_LITERAL)}?
// {System.out.println("found \$tokenlabel.attr or \$tokenref.attr");}
{
String label = $x.text;
if ( enclosingRule.getTokenLabel($x.text)==null ) {
// \$tokenref.attr gotta get old label or compute new one
checkElementRefUniqueness($x.text, true);
label = enclosingRule.getElementLabel($x.text, outerAltNum, generator);
if ( label==null ) {
ErrorManager.grammarError(ErrorManager.MSG_FORWARD_ELEMENT_REF,
grammar,
actionToken,
"\$"+$x.text+"."+$y.text);
label = $x.text;
}
}
StringTemplate st = template("tokenLabelPropertyRef_"+$y.text);
st.setAttribute("scope", label);
st.setAttribute("attr", AttributeScope.tokenScope.getAttribute($y.text));
}
;
/** Setting $rulelabel.attr or $ruleref.attr where attr is a predefined property is an error
* This must also fail, if we try to access a local attribute's field, like $tree.scope = localObject
* That must be handled by LOCAL_ATTR below. ANTLR only concerns itself with the top-level scope
* attributes declared in scope {} or parameters, return values and the like.
*/
SET_RULE_SCOPE_ATTR
@init {
Grammar.LabelElementPair pair=null;
String refdRuleName=null;
}
: '$' x=ID '.' y=ID WS? '=' {enclosingRule!=null && input.LA(1)!='='}?
{
pair = enclosingRule.getRuleLabel($x.text);
refdRuleName = $x.text;
if ( pair!=null ) {
refdRuleName = pair.referencedRuleName;
}
}
// supercomplicated because I can't exec the above action.
// This asserts that if it's a label or a ref to a rule proceed but only if the attribute
// is valid for that rule's scope
{(enclosingRule.getRuleLabel($x.text)!=null || isRuleRefInAlt($x.text)) &&
getRuleLabelAttribute(enclosingRule.getRuleLabel($x.text)!=null?enclosingRule.getRuleLabel($x.text).referencedRuleName:$x.text,$y.text)!=null}?
//{System.out.println("found set \$rulelabel.attr or \$ruleref.attr: "+$x.text+"."+$y.text);}
{
ErrorManager.grammarError(ErrorManager.MSG_WRITE_TO_READONLY_ATTR,
grammar,
actionToken,
$x.text,
$y.text);
}
;
/** $rulelabel.attr or $ruleref.attr where attr is a predefined property*/
RULE_SCOPE_ATTR
@init {
Grammar.LabelElementPair pair=null;
String refdRuleName=null;
}
: '$' x=ID '.' y=ID {enclosingRule!=null}?
{
pair = enclosingRule.getRuleLabel($x.text);
refdRuleName = $x.text;
if ( pair!=null ) {
refdRuleName = pair.referencedRuleName;
}
}
// supercomplicated because I can't exec the above action.
// This asserts that if it's a label or a ref to a rule proceed but only if the attribute
// is valid for that rule's scope
{(enclosingRule.getRuleLabel($x.text)!=null || isRuleRefInAlt($x.text)) &&
getRuleLabelAttribute(enclosingRule.getRuleLabel($x.text)!=null?enclosingRule.getRuleLabel($x.text).referencedRuleName:$x.text,$y.text)!=null}?
//{System.out.println("found \$rulelabel.attr or \$ruleref.attr: "+$x.text+"."+$y.text);}
{
String label = $x.text;
if ( pair==null ) {
// \$ruleref.attr gotta get old label or compute new one
checkElementRefUniqueness($x.text, false);
label = enclosingRule.getElementLabel($x.text, outerAltNum, generator);
if ( label==null ) {
ErrorManager.grammarError(ErrorManager.MSG_FORWARD_ELEMENT_REF,
grammar,
actionToken,
"\$"+$x.text+"."+$y.text);
label = $x.text;
}
}
StringTemplate st;
Rule refdRule = grammar.getRule(refdRuleName);
AttributeScope scope = refdRule.getLocalAttributeScope($y.text);
if ( scope.isPredefinedRuleScope ) {
st = template("ruleLabelPropertyRef_"+$y.text);
grammar.referenceRuleLabelPredefinedAttribute(refdRuleName);
st.setAttribute("scope", label);
st.setAttribute("attr", $y.text);
}
else if ( scope.isPredefinedLexerRuleScope ) {
st = template("lexerRuleLabelPropertyRef_"+$y.text);
grammar.referenceRuleLabelPredefinedAttribute(refdRuleName);
st.setAttribute("scope", label);
st.setAttribute("attr", $y.text);
}
else if ( scope.isParameterScope ) {
// TODO: error!
}
else {
st = template("ruleLabelRef");
st.setAttribute("referencedRule", refdRule);
st.setAttribute("scope", label);
st.setAttribute("attr", scope.getAttribute($y.text));
}
}
;
/** $label either a token label or token/rule list label like label+=expr */
LABEL_REF
: '$' ID {enclosingRule!=null &&
getElementLabel($ID.text)!=null &&
enclosingRule.getRuleLabel($ID.text)==null}?
// {System.out.println("found \$label");}
{
StringTemplate st;
Grammar.LabelElementPair pair = getElementLabel($ID.text);
if ( pair.type==Grammar.RULE_LIST_LABEL ||
pair.type==Grammar.TOKEN_LIST_LABEL ||
pair.type==Grammar.WILDCARD_TREE_LIST_LABEL )
{
st = template("listLabelRef");
}
else {
st = template("tokenLabelRef");
}
st.setAttribute("label", $ID.text);
}
;
/** $tokenref in a non-lexer grammar */
ISOLATED_TOKEN_REF
: '$' ID {grammar.type!=Grammar.LEXER && enclosingRule!=null && isTokenRefInAlt($ID.text)}?
//{System.out.println("found \$tokenref");}
{
String label = enclosingRule.getElementLabel($ID.text, outerAltNum, generator);
checkElementRefUniqueness($ID.text, true);
if ( label==null ) {
ErrorManager.grammarError(ErrorManager.MSG_FORWARD_ELEMENT_REF,
grammar,
actionToken,
$ID.text);
}
else {
StringTemplate st = template("tokenLabelRef");
st.setAttribute("label", label);
}
}
;
/** $lexerruleref from within the lexer */
ISOLATED_LEXER_RULE_REF
: '$' ID {grammar.type==Grammar.LEXER &&
enclosingRule!=null &&
isRuleRefInAlt($ID.text)}?
//{System.out.println("found \$lexerruleref");}
{
String label = enclosingRule.getElementLabel($ID.text, outerAltNum, generator);
checkElementRefUniqueness($ID.text, false);
if ( label==null ) {
ErrorManager.grammarError(ErrorManager.MSG_FORWARD_ELEMENT_REF,
grammar,
actionToken,
$ID.text);
}
else {
StringTemplate st = template("lexerRuleLabel");
st.setAttribute("label", label);
}
}
;
/** $y return value, parameter, predefined rule property, or token/rule
* reference within enclosing rule's outermost alt.
* y must be a "local" reference; i.e., it must be referring to
* something defined within the enclosing rule.
*
* r[int i] returns [int j]
* : {$i, $j, $start, $stop, $st, $tree}
* ;
*
* TODO: this might get the dynamic scope's elements too.!!!!!!!!!
*/
SET_LOCAL_ATTR
: '$' ID WS? '=' expr=ATTR_VALUE_EXPR ';' {enclosingRule!=null
&& enclosingRule.getLocalAttributeScope($ID.text)!=null
&& !enclosingRule.getLocalAttributeScope($ID.text).isPredefinedLexerRuleScope}?
//{System.out.println("found set \$localattr");}
{
StringTemplate st;
AttributeScope scope = enclosingRule.getLocalAttributeScope($ID.text);
if ( scope.isPredefinedRuleScope ) {
if ($ID.text.equals("tree") || $ID.text.equals("st")) {
st = template("ruleSetPropertyRef_"+$ID.text);
grammar.referenceRuleLabelPredefinedAttribute(enclosingRule.name);
st.setAttribute("scope", enclosingRule.name);
st.setAttribute("attr", $ID.text);
st.setAttribute("expr", translateAction($expr.text));
} else {
ErrorManager.grammarError(ErrorManager.MSG_WRITE_TO_READONLY_ATTR,
grammar,
actionToken,
$ID.text,
"");
}
}
else if ( scope.isParameterScope ) {
st = template("parameterSetAttributeRef");
st.setAttribute("attr", scope.getAttribute($ID.text));
st.setAttribute("expr", translateAction($expr.text));
}
else {
st = template("returnSetAttributeRef");
st.setAttribute("ruleDescriptor", enclosingRule);
st.setAttribute("attr", scope.getAttribute($ID.text));
st.setAttribute("expr", translateAction($expr.text));
}
}
;
LOCAL_ATTR
: '$' ID {enclosingRule!=null && enclosingRule.getLocalAttributeScope($ID.text)!=null}?
//{System.out.println("found \$localattr");}
{
StringTemplate st;
AttributeScope scope = enclosingRule.getLocalAttributeScope($ID.text);
if ( scope.isPredefinedRuleScope ) {
st = template("rulePropertyRef_"+$ID.text);
grammar.referenceRuleLabelPredefinedAttribute(enclosingRule.name);
st.setAttribute("scope", enclosingRule.name);
st.setAttribute("attr", $ID.text);
}
else if ( scope.isPredefinedLexerRuleScope ) {
st = template("lexerRulePropertyRef_"+$ID.text);
st.setAttribute("scope", enclosingRule.name);
st.setAttribute("attr", $ID.text);
}
else if ( scope.isParameterScope ) {
st = template("parameterAttributeRef");
st.setAttribute("attr", scope.getAttribute($ID.text));
}
else {
st = template("returnAttributeRef");
st.setAttribute("ruleDescriptor", enclosingRule);
st.setAttribute("attr", scope.getAttribute($ID.text));
}
}
;
/** $x::y the only way to access the attributes within a dynamic scope
* regardless of whether or not you are in the defining rule.
*
* scope Symbols { List names; }
* r
* scope {int i;}
* scope Symbols;
* : {$r::i=3;} s {$Symbols::names;}
* ;
* s : {$r::i; $Symbols::names;}
* ;
*/
SET_DYNAMIC_SCOPE_ATTR
: '$' x=ID '::' y=ID WS? '=' expr=ATTR_VALUE_EXPR ';'
{resolveDynamicScope($x.text)!=null &&
resolveDynamicScope($x.text).getAttribute($y.text)!=null}?
//{System.out.println("found set \$scope::attr "+ $x.text + "::" + $y.text + " to " + $expr.text);}
{
AttributeScope scope = resolveDynamicScope($x.text);
if ( scope!=null ) {
StringTemplate st = template("scopeSetAttributeRef");
st.setAttribute("scope", $x.text);
st.setAttribute("attr", scope.getAttribute($y.text));
st.setAttribute("expr", translateAction($expr.text));
}
else {
// error: invalid dynamic attribute
}
}
;
DYNAMIC_SCOPE_ATTR
: '$' x=ID '::' y=ID
{resolveDynamicScope($x.text)!=null &&
resolveDynamicScope($x.text).getAttribute($y.text)!=null}?
//{System.out.println("found \$scope::attr "+ $x.text + "::" + $y.text);}
{
AttributeScope scope = resolveDynamicScope($x.text);
if ( scope!=null ) {
StringTemplate st = template("scopeAttributeRef");
st.setAttribute("scope", $x.text);
st.setAttribute("attr", scope.getAttribute($y.text));
}
else {
// error: invalid dynamic attribute
}
}
;
ERROR_SCOPED_XY
: '$' x=ID '::' y=ID
{
chunks.add(getText());
generator.issueInvalidScopeError($x.text,$y.text,
enclosingRule,actionToken,
outerAltNum);
}
;
/** To access deeper (than top of stack) scopes, use the notation:
*
* $x[-1]::y previous (just under top of stack)
* $x[-i]::y top of stack - i where the '-' MUST BE PRESENT;
* i.e., i cannot simply be negative without the '-' sign!
* $x[i]::y absolute index i (0..size-1)
* $x[0]::y is the absolute 0 indexed element (bottom of the stack)
*/
DYNAMIC_NEGATIVE_INDEXED_SCOPE_ATTR
: '$' x=ID '[' '-' expr=SCOPE_INDEX_EXPR ']' '::' y=ID
// {System.out.println("found \$scope[-...]::attr");}
{
StringTemplate st = template("scopeAttributeRef");
st.setAttribute("scope", $x.text);
st.setAttribute("attr", resolveDynamicScope($x.text).getAttribute($y.text));
st.setAttribute("negIndex", $expr.text);
}
;
DYNAMIC_ABSOLUTE_INDEXED_SCOPE_ATTR
: '$' x=ID '[' expr=SCOPE_INDEX_EXPR ']' '::' y=ID
// {System.out.println("found \$scope[...]::attr");}
{
StringTemplate st = template("scopeAttributeRef");
st.setAttribute("scope", $x.text);
st.setAttribute("attr", resolveDynamicScope($x.text).getAttribute($y.text));
st.setAttribute("index", $expr.text);
}
;
fragment
SCOPE_INDEX_EXPR
: (~']')+
;
/** $r y is a rule's dynamic scope or a global shared scope.
* Isolated $rulename is not allowed unless it has a dynamic scope *and*
* there is no reference to rulename in the enclosing alternative,
* which would be ambiguous. See TestAttributes.testAmbiguousRuleRef()
*/
ISOLATED_DYNAMIC_SCOPE
: '$' ID {resolveDynamicScope($ID.text)!=null}?
// {System.out.println("found isolated \$scope where scope is a dynamic scope");}
{
StringTemplate st = template("isolatedDynamicScopeRef");
st.setAttribute("scope", $ID.text);
}
;
// antlr.g then codegen.g does these first two currently.
// don't want to duplicate that code.
/** %foo(a={},b={},...) ctor */
TEMPLATE_INSTANCE
: '%' ID '(' ( WS? ARG (',' WS? ARG)* WS? )? ')'
// {System.out.println("found \%foo(args)");}
{
String action = getText().substring(1,getText().length());
String ruleName = "<outside-of-rule>";
if ( enclosingRule!=null ) {
ruleName = enclosingRule.name;
}
StringTemplate st =
generator.translateTemplateConstructor(ruleName,
outerAltNum,
actionToken,
action);
if ( st!=null ) {
chunks.add(st);
}
}
;
/** %({name-expr})(a={},...) indirect template ctor reference */
INDIRECT_TEMPLATE_INSTANCE
: '%' '(' ACTION ')' '(' ( WS? ARG (',' WS? ARG)* WS? )? ')'
// {System.out.println("found \%({...})(args)");}
{
String action = getText().substring(1,getText().length());
StringTemplate st =
generator.translateTemplateConstructor(enclosingRule.name,
outerAltNum,
actionToken,
action);
chunks.add(st);
}
;
fragment
ARG : ID '=' ACTION
;
/** %{expr}.y = z; template attribute y of StringTemplate-typed expr to z */
SET_EXPR_ATTRIBUTE
: '%' a=ACTION '.' ID WS? '=' expr=ATTR_VALUE_EXPR ';'
// {System.out.println("found \%{expr}.y = z;");}
{
StringTemplate st = template("actionSetAttribute");
String action = $a.text;
action = action.substring(1,action.length()-1); // stuff inside {...}
st.setAttribute("st", translateAction(action));
st.setAttribute("attrName", $ID.text);
st.setAttribute("expr", translateAction($expr.text));
}
;
/* %x.y = z; set template attribute y of x (always set never get attr)
* to z [languages like python without ';' must still use the
* ';' which the code generator is free to remove during code gen]
*/
SET_ATTRIBUTE
: '%' x=ID '.' y=ID WS? '=' expr=ATTR_VALUE_EXPR ';'
// {System.out.println("found \%x.y = z;");}
{
StringTemplate st = template("actionSetAttribute");
st.setAttribute("st", $x.text);
st.setAttribute("attrName", $y.text);
st.setAttribute("expr", translateAction($expr.text));
}
;
/** Don't allow an = as first char to prevent $x == 3; kind of stuff. */
fragment
ATTR_VALUE_EXPR
: ~'=' (~';')*
;
/** %{string-expr} anonymous template from string expr */
TEMPLATE_EXPR
: '%' a=ACTION
// {System.out.println("found \%{expr}");}
{
StringTemplate st = template("actionStringConstructor");
String action = $a.text;
action = action.substring(1,action.length()-1); // stuff inside {...}
st.setAttribute("stringExpr", translateAction(action));
}
;
fragment
ACTION
: '{' (options {greedy=false;}:.)* '}'
;
ESC : '\\' '$' {chunks.add("\$");}
| '\\' '%' {chunks.add("\%");}
| '\\' ~('$'|'%') {chunks.add(getText());}
;
ERROR_XY
: '$' x=ID '.' y=ID
{
chunks.add(getText());
generator.issueInvalidAttributeError($x.text,$y.text,
enclosingRule,actionToken,
outerAltNum);
}
;
ERROR_X
: '$' x=ID
{
chunks.add(getText());
generator.issueInvalidAttributeError($x.text,
enclosingRule,actionToken,
outerAltNum);
}
;
UNKNOWN_SYNTAX
: '$'
{
chunks.add(getText());
// shouldn't need an error here. Just accept \$ if it doesn't look like anything
}
| '%' (ID|'.'|'('|')'|','|'{'|'}'|'"')*
{
chunks.add(getText());
ErrorManager.grammarError(ErrorManager.MSG_INVALID_TEMPLATE_ACTION,
grammar,
actionToken,
getText());
}
;
TEXT: ~('$'|'%'|'\\')+ {chunks.add(getText());}
;
fragment
ID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
fragment
INT : '0'..'9'+
;
fragment
WS : (' '|'\t'|'\n'|'\r')+
;
antlr-3.2/tool/src/main/antlr3/org/antlr/grammar/v3/ANTLRv3Tree.g 0000644 0001750 0001750 00000012402 11256465226 024345 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2007 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** ANTLR v3 tree grammar to walk trees created by ANTLRv3.g */
tree grammar ANTLRv3Tree;
options {
tokenVocab = ANTLRv3;
ASTLabelType = CommonTree;
}
@header {
package org.antlr.grammar.v3;
}
grammarDef
: ^( grammarType ID DOC_COMMENT? optionsSpec? tokensSpec? attrScope* action* rule+ )
;
grammarType
: LEXER_GRAMMAR
| PARSER_GRAMMAR
| TREE_GRAMMAR
| COMBINED_GRAMMAR
;
tokensSpec
: ^(TOKENS tokenSpec+)
;
tokenSpec
: ^('=' TOKEN_REF STRING_LITERAL)
| ^('=' TOKEN_REF CHAR_LITERAL)
| TOKEN_REF
;
attrScope
: ^('scope' ID ACTION)
;
action
: ^('@' ID ID ACTION)
| ^('@' ID ACTION)
;
optionsSpec
: ^(OPTIONS option+)
;
option
: qid // only allowed in element options
| ^('=' ID optionValue)
;
optionValue
: ID
| STRING_LITERAL
| CHAR_LITERAL
| INT
;
rule
: ^( RULE ID modifier? (^(ARG ARG_ACTION))? (^(RET ARG_ACTION))?
throwsSpec? optionsSpec? ruleScopeSpec? ruleAction*
altList
exceptionGroup? EOR
)
;
modifier
: 'protected'|'public'|'private'|'fragment'
;
/** Match stuff like @init {int i;} */
ruleAction
: ^('@' ID ACTION)
;
throwsSpec
: ^('throws' ID+)
;
ruleScopeSpec
: ^('scope' ACTION)
| ^('scope' ACTION ID+)
| ^('scope' ID+)
;
block
: ^( BLOCK optionsSpec? (alternative rewrite)+ EOB )
;
altList
: ^( BLOCK (alternative rewrite)+ EOB )
;
alternative
: ^(ALT element+ EOA)
| ^(ALT EPSILON EOA)
;
exceptionGroup
: exceptionHandler+ finallyClause?
| finallyClause
;
exceptionHandler
: ^('catch' ARG_ACTION ACTION)
;
finallyClause
: ^('finally' ACTION)
;
element
: ^(('='|'+=') ID block)
| ^(('='|'+=') ID atom)
| atom
| ebnf
| ACTION
| SEMPRED
| GATED_SEMPRED
| ^(TREE_BEGIN element+)
;
atom: ^(('^'|'!') atom)
| ^(CHAR_RANGE CHAR_LITERAL CHAR_LITERAL optionsSpec?)
| ^('~' notTerminal optionsSpec?)
| ^('~' block optionsSpec?)
| ^(RULE_REF ARG_ACTION)
| RULE_REF
| CHAR_LITERAL
| ^(CHAR_LITERAL optionsSpec)
| TOKEN_REF
| ^(TOKEN_REF optionsSpec)
| ^(TOKEN_REF ARG_ACTION optionsSpec)
| ^(TOKEN_REF ARG_ACTION)
| STRING_LITERAL
| ^(STRING_LITERAL optionsSpec)
| '.'
| ^('.' optionsSpec?)
;
/** Matches ENBF blocks (and token sets via block rule) */
ebnf
: ^(SYNPRED block)
| ^(OPTIONAL block)
| ^(CLOSURE block)
| ^(POSITIVE_CLOSURE block)
| SYN_SEMPRED
| block
;
notTerminal
: CHAR_LITERAL
| TOKEN_REF
| STRING_LITERAL
;
// R E W R I T E S Y N T A X
rewrite
: (^('->' SEMPRED rewrite_alternative))* ^('->' rewrite_alternative)
|
;
rewrite_alternative
: rewrite_template
| rewrite_tree_alternative
| ^(ALT EPSILON EOA)
;
rewrite_tree_block
: ^(BLOCK rewrite_tree_alternative EOB)
;
rewrite_tree_alternative
: ^(ALT rewrite_tree_element+ EOA)
;
rewrite_tree_element
: rewrite_tree_atom
| rewrite_tree
| rewrite_tree_block
| rewrite_tree_ebnf
;
rewrite_tree_atom
: CHAR_LITERAL
| TOKEN_REF
| ^(TOKEN_REF ARG_ACTION) // for imaginary nodes
| RULE_REF
| STRING_LITERAL
| LABEL
| ACTION
;
rewrite_tree_ebnf
: ^(OPTIONAL rewrite_tree_block)
| ^(CLOSURE rewrite_tree_block)
| ^(POSITIVE_CLOSURE rewrite_tree_block)
;
rewrite_tree
: ^(TREE_BEGIN rewrite_tree_atom rewrite_tree_element* )
;
rewrite_template
: ^( TEMPLATE ID rewrite_template_args
(DOUBLE_QUOTE_STRING_LITERAL | DOUBLE_ANGLE_STRING_LITERAL)
)
| rewrite_template_ref
| rewrite_indirect_template_head
| ACTION
;
/** foo(a={...}, ...) */
rewrite_template_ref
: ^(TEMPLATE ID rewrite_template_args)
;
/** ({expr})(a={...}, ...) */
rewrite_indirect_template_head
: ^(TEMPLATE ACTION rewrite_template_args)
;
rewrite_template_args
: ^(ARGLIST rewrite_template_arg+)
| ARGLIST
;
rewrite_template_arg
: ^(ARG ID ACTION)
;
qid : ID ('.' ID)* ;
antlr-3.2/tool/src/main/antlr3/org/antlr/grammar/v3/ANTLRv3.g 0000644 0001750 0001750 00000033353 11256465226 023535 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2007 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** ANTLR v3 grammar written in ANTLR v3 with AST construction */
grammar ANTLRv3;
options {
output=AST;
ASTLabelType=CommonTree;
}
tokens {
DOC_COMMENT;
PARSER;
LEXER;
RULE;
BLOCK;
OPTIONAL;
CLOSURE;
POSITIVE_CLOSURE;
SYNPRED;
RANGE;
CHAR_RANGE;
EPSILON;
ALT;
EOR;
EOB;
EOA; // end of alt
ID;
ARG;
ARGLIST;
RET='returns';
LEXER_GRAMMAR;
PARSER_GRAMMAR;
TREE_GRAMMAR;
COMBINED_GRAMMAR;
LABEL; // $x used in rewrite rules
TEMPLATE;
SCOPE='scope';
SEMPRED;
GATED_SEMPRED; // {p}? =>
SYN_SEMPRED; // (...) => it's a manually-specified synpred converted to sempred
BACKTRACK_SEMPRED; // auto backtracking mode syn pred converted to sempred
FRAGMENT='fragment';
TREE_BEGIN='^(';
ROOT='^';
BANG='!';
RANGE='..';
REWRITE='->';
AT='@';
LABEL_ASSIGN='=';
LIST_LABEL_ASSIGN='+=';
}
@parser::header
{
package org.antlr.grammar.v3;
}
@lexer::header
{
package org.antlr.grammar.v3;
}
@members {
int gtype;
}
grammarDef
: DOC_COMMENT?
( 'lexer' {gtype=LEXER_GRAMMAR;} // pure lexer
| 'parser' {gtype=PARSER_GRAMMAR;} // pure parser
| 'tree' {gtype=TREE_GRAMMAR;} // a tree parser
| {gtype=COMBINED_GRAMMAR;} // merged parser/lexer
)
g='grammar' id ';' optionsSpec? tokensSpec? attrScope* action*
rule+
EOF
-> ^( {adaptor.create(gtype,$g)}
id DOC_COMMENT? optionsSpec? tokensSpec? attrScope* action* rule+
)
;
tokensSpec
: TOKENS tokenSpec+ '}' -> ^(TOKENS tokenSpec+)
;
tokenSpec
: TOKEN_REF
( '=' (lit=STRING_LITERAL|lit=CHAR_LITERAL) -> ^('=' TOKEN_REF $lit)
| -> TOKEN_REF
)
';'
;
attrScope
: 'scope' id ACTION -> ^('scope' id ACTION)
;
/** Match stuff like @parser::members {int i;} */
action
: '@' (actionScopeName '::')? id ACTION -> ^('@' actionScopeName? id ACTION)
;
/** Sometimes the scope names will collide with keywords; allow them as
* ids for action scopes.
*/
actionScopeName
: id
| l='lexer' -> ID[$l]
| p='parser' -> ID[$p]
;
optionsSpec
: OPTIONS (option ';')+ '}' -> ^(OPTIONS option+)
;
option
: id '=' optionValue -> ^('=' id optionValue)
;
optionValue
: qid
| STRING_LITERAL
| CHAR_LITERAL
| INT
| s='*' -> STRING_LITERAL[$s] // used for k=*
;
rule
scope {
String name;
}
: DOC_COMMENT?
( modifier=('protected'|'public'|'private'|'fragment') )?
id {$rule::name = $id.text;}
'!'?
( arg=ARG_ACTION )?
( 'returns' rt=ARG_ACTION )?
throwsSpec? optionsSpec? ruleScopeSpec? ruleAction*
':' altList ';'
exceptionGroup?
-> ^( RULE id {modifier!=null?adaptor.create(modifier):null} ^(ARG[$arg] $arg)? ^('returns' $rt)?
throwsSpec? optionsSpec? ruleScopeSpec? ruleAction*
altList
exceptionGroup?
EOR["EOR"]
)
;
/** Match stuff like @init {int i;} */
ruleAction
: '@' id ACTION -> ^('@' id ACTION)
;
throwsSpec
: 'throws' id ( ',' id )* -> ^('throws' id+)
;
ruleScopeSpec
: 'scope' ACTION -> ^('scope' ACTION)
| 'scope' id (',' id)* ';' -> ^('scope' id+)
| 'scope' ACTION
'scope' id (',' id)* ';'
-> ^('scope' ACTION id+ )
;
block
: lp='('
( (opts=optionsSpec)? ':' )?
altpair ( '|' altpair )*
rp=')'
-> ^( BLOCK[$lp,"BLOCK"] optionsSpec? altpair+ EOB[$rp,"EOB"] )
;
altpair : alternative rewrite ;
altList
@init {
// must create root manually as it's used by invoked rules in real antlr tool.
// leave here to demonstrate use of {...} in rewrite rule
// it's really BLOCK[firstToken,"BLOCK"]; set line/col to previous ( or : token.
CommonTree blkRoot = (CommonTree)adaptor.create(BLOCK,input.LT(-1),"BLOCK");
}
: altpair ( '|' altpair )* -> ^( {blkRoot} altpair+ EOB["EOB"] )
;
alternative
@init {
Token firstToken = input.LT(1);
Token prevToken = input.LT(-1); // either : or | I think
}
: element+ -> ^(ALT[firstToken,"ALT"] element+ EOA["EOA"])
| -> ^(ALT[prevToken,"ALT"] EPSILON[prevToken,"EPSILON"] EOA["EOA"])
;
exceptionGroup
: ( exceptionHandler )+ ( finallyClause )?
| finallyClause
;
exceptionHandler
: 'catch' ARG_ACTION ACTION -> ^('catch' ARG_ACTION ACTION)
;
finallyClause
: 'finally' ACTION -> ^('finally' ACTION)
;
element
: id (labelOp='='|labelOp='+=') atom
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] ^($labelOp id atom) EOA["EOA"]) EOB["EOB"]))
| -> ^($labelOp id atom)
)
| id (labelOp='='|labelOp='+=') block
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] ^($labelOp id block) EOA["EOA"]) EOB["EOB"]))
| -> ^($labelOp id block)
)
| atom
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] atom EOA["EOA"]) EOB["EOB"]) )
| -> atom
)
| ebnf
| ACTION
| SEMPRED ( g='=>' -> GATED_SEMPRED[$g] | -> SEMPRED )
| treeSpec
( ebnfSuffix -> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] treeSpec EOA["EOA"]) EOB["EOB"]) )
| -> treeSpec
)
;
atom: terminal
| range
( (op='^'|op='!') -> ^($op range)
| -> range
)
| notSet
( (op='^'|op='!') -> ^($op notSet)
| -> notSet
)
| RULE_REF ARG_ACTION?
( (op='^'|op='!') -> ^($op RULE_REF ARG_ACTION?)
| -> ^(RULE_REF ARG_ACTION?)
)
;
notSet
: '~'
( notTerminal elementOptions? -> ^('~' notTerminal elementOptions?)
| block elementOptions? -> ^('~' block elementOptions?)
)
;
notTerminal
: CHAR_LITERAL
| TOKEN_REF
| STRING_LITERAL
;
elementOptions
: '<' qid '>' -> ^(OPTIONS qid)
| '<' option (';' option)* '>' -> ^(OPTIONS option+)
;
elementOption
: id '=' optionValue -> ^('=' id optionValue)
;
treeSpec
: '^(' element ( element )+ ')' -> ^(TREE_BEGIN element+)
;
range!
: c1=CHAR_LITERAL RANGE c2=CHAR_LITERAL elementOptions?
-> ^(CHAR_RANGE[$c1,".."] $c1 $c2 elementOptions?)
;
terminal
: ( CHAR_LITERAL elementOptions? -> ^(CHAR_LITERAL elementOptions?)
// Args are only valid for lexer rules
| TOKEN_REF ARG_ACTION? elementOptions? -> ^(TOKEN_REF ARG_ACTION? elementOptions?)
| STRING_LITERAL elementOptions? -> ^(STRING_LITERAL elementOptions?)
| '.' elementOptions? -> ^('.' elementOptions?)
)
( '^' -> ^('^' $terminal)
| '!' -> ^('!' $terminal)
)?
;
/** Matches ENBF blocks (and token sets via block rule) */
ebnf
@init {
Token firstToken = input.LT(1);
}
@after {
$ebnf.tree.getToken().setLine(firstToken.getLine());
$ebnf.tree.getToken().setCharPositionInLine(firstToken.getCharPositionInLine());
}
: block
( op='?' -> ^(OPTIONAL[op] block)
| op='*' -> ^(CLOSURE[op] block)
| op='+' -> ^(POSITIVE_CLOSURE[op] block)
| '=>' // syntactic predicate
-> {gtype==COMBINED_GRAMMAR &&
Character.isUpperCase($rule::name.charAt(0))}?
// if lexer rule in combined, leave as pred for lexer
^(SYNPRED["=>"] block)
// in real antlr tool, text for SYN_SEMPRED is predname
-> SYN_SEMPRED
| -> block
)
;
ebnfSuffix
@init {
Token op = input.LT(1);
}
: '?' -> OPTIONAL[op]
| '*' -> CLOSURE[op]
| '+' -> POSITIVE_CLOSURE[op]
;
// R E W R I T E S Y N T A X
rewrite
@init {
Token firstToken = input.LT(1);
}
: (rew+='->' preds+=SEMPRED predicated+=rewrite_alternative)*
rew2='->' last=rewrite_alternative
-> ^($rew $preds $predicated)* ^($rew2 $last)
|
;
rewrite_alternative
options {backtrack=true;}
: rewrite_template
| rewrite_tree_alternative
| /* empty rewrite */ -> ^(ALT["ALT"] EPSILON["EPSILON"] EOA["EOA"])
;
rewrite_tree_block
: lp='(' rewrite_tree_alternative ')'
-> ^(BLOCK[$lp,"BLOCK"] rewrite_tree_alternative EOB[$lp,"EOB"])
;
rewrite_tree_alternative
: rewrite_tree_element+ -> ^(ALT["ALT"] rewrite_tree_element+ EOA["EOA"])
;
rewrite_tree_element
: rewrite_tree_atom
| rewrite_tree_atom ebnfSuffix
-> ^( ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] rewrite_tree_atom EOA["EOA"]) EOB["EOB"]))
| rewrite_tree
( ebnfSuffix
-> ^(ebnfSuffix ^(BLOCK["BLOCK"] ^(ALT["ALT"] rewrite_tree EOA["EOA"]) EOB["EOB"]))
| -> rewrite_tree
)
| rewrite_tree_ebnf
;
rewrite_tree_atom
: CHAR_LITERAL
| TOKEN_REF ARG_ACTION? -> ^(TOKEN_REF ARG_ACTION?) // for imaginary nodes
| RULE_REF
| STRING_LITERAL
| d='$' id -> LABEL[$d,$id.text] // reference to a label in a rewrite rule
| ACTION
;
rewrite_tree_ebnf
@init {
Token firstToken = input.LT(1);
}
@after {
$rewrite_tree_ebnf.tree.getToken().setLine(firstToken.getLine());
$rewrite_tree_ebnf.tree.getToken().setCharPositionInLine(firstToken.getCharPositionInLine());
}
: rewrite_tree_block ebnfSuffix -> ^(ebnfSuffix rewrite_tree_block)
;
rewrite_tree
: '^(' rewrite_tree_atom rewrite_tree_element* ')'
-> ^(TREE_BEGIN rewrite_tree_atom rewrite_tree_element* )
;
/** Build a tree for a template rewrite:
^(TEMPLATE (ID|ACTION) ^(ARGLIST ^(ARG ID ACTION) ...) )
where ARGLIST is always there even if no args exist.
ID can be "template" keyword. If first child is ACTION then it's
an indirect template ref
-> foo(a={...}, b={...})
-> ({string-e})(a={...}, b={...}) // e evaluates to template name
-> {%{$ID.text}} // create literal template from string (done in ActionTranslator)
-> {st-expr} // st-expr evaluates to ST
*/
rewrite_template
: // -> template(a={...},...) "..." inline template
id lp='(' rewrite_template_args ')'
( str=DOUBLE_QUOTE_STRING_LITERAL | str=DOUBLE_ANGLE_STRING_LITERAL )
-> ^(TEMPLATE[$lp,"TEMPLATE"] id rewrite_template_args $str)
| // -> foo(a={...}, ...)
rewrite_template_ref
| // -> ({expr})(a={...}, ...)
rewrite_indirect_template_head
| // -> {...}
ACTION
;
/** -> foo(a={...}, ...) */
rewrite_template_ref
: id lp='(' rewrite_template_args ')'
-> ^(TEMPLATE[$lp,"TEMPLATE"] id rewrite_template_args)
;
/** -> ({expr})(a={...}, ...) */
rewrite_indirect_template_head
: lp='(' ACTION ')' '(' rewrite_template_args ')'
-> ^(TEMPLATE[$lp,"TEMPLATE"] ACTION rewrite_template_args)
;
rewrite_template_args
: rewrite_template_arg (',' rewrite_template_arg)*
-> ^(ARGLIST rewrite_template_arg+)
| -> ARGLIST
;
rewrite_template_arg
: id '=' ACTION -> ^(ARG[$id.start] id ACTION)
;
qid : id ('.' id)* ;
id : TOKEN_REF -> ID[$TOKEN_REF]
| RULE_REF -> ID[$RULE_REF]
;
// L E X I C A L R U L E S
SL_COMMENT
: '//'
( ' $ANTLR ' SRC // src directive
| ~('\r'|'\n')*
)
'\r'? '\n'
{$channel=HIDDEN;}
;
ML_COMMENT
: '/*' {if (input.LA(1)=='*') $type=DOC_COMMENT; else $channel=HIDDEN;} .* '*/'
;
CHAR_LITERAL
: '\'' LITERAL_CHAR '\''
;
STRING_LITERAL
: '\'' LITERAL_CHAR LITERAL_CHAR* '\''
;
fragment
LITERAL_CHAR
: ESC
| ~('\''|'\\')
;
DOUBLE_QUOTE_STRING_LITERAL
: '"' (ESC | ~('\\'|'"'))* '"'
;
DOUBLE_ANGLE_STRING_LITERAL
: '<<' .* '>>'
;
fragment
ESC : '\\'
( 'n'
| 'r'
| 't'
| 'b'
| 'f'
| '"'
| '\''
| '\\'
| '>'
| 'u' XDIGIT XDIGIT XDIGIT XDIGIT
| . // unknown, leave as it is
)
;
fragment
XDIGIT :
'0' .. '9'
| 'a' .. 'f'
| 'A' .. 'F'
;
INT : '0'..'9'+
;
ARG_ACTION
: NESTED_ARG_ACTION
;
fragment
NESTED_ARG_ACTION :
'['
( options {greedy=false; k=1;}
: NESTED_ARG_ACTION
| ACTION_STRING_LITERAL
| ACTION_CHAR_LITERAL
| .
)*
']'
//{setText(getText().substring(1, getText().length()-1));}
;
ACTION
: NESTED_ACTION ( '?' {$type = SEMPRED;} )?
;
fragment
NESTED_ACTION :
'{'
( options {greedy=false; k=2;}
: NESTED_ACTION
| SL_COMMENT
| ML_COMMENT
| ACTION_STRING_LITERAL
| ACTION_CHAR_LITERAL
| .
)*
'}'
;
fragment
ACTION_CHAR_LITERAL
: '\'' (ACTION_ESC|~('\\'|'\'')) '\''
;
fragment
ACTION_STRING_LITERAL
: '"' (ACTION_ESC|~('\\'|'"'))* '"'
;
fragment
ACTION_ESC
: '\\\''
| '\\' '"' // ANTLR doesn't like: '\\"'
| '\\' ~('\''|'"')
;
TOKEN_REF
: 'A'..'Z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
RULE_REF
: 'a'..'z' ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
/** Match the start of an options section. Don't allow normal
* action processing on the {...} as it's not a action.
*/
OPTIONS
: 'options' WS_LOOP '{'
;
TOKENS
: 'tokens' WS_LOOP '{'
;
/** Reset the file and line information; useful when the grammar
* has been generated so that errors are shown relative to the
* original file like the old C preprocessor used to do.
*/
fragment
SRC : 'src' ' ' file=ACTION_STRING_LITERAL ' ' line=INT
;
WS : ( ' '
| '\t'
| '\r'? '\n'
)+
{$channel=HIDDEN;}
;
fragment
WS_LOOP
: ( WS
| SL_COMMENT
| ML_COMMENT
)*
;
antlr-3.2/tool/src/main/antlr3/org/antlr/grammar/v3/ActionAnalysis.g 0000644 0001750 0001750 00000011264 11256465226 025322 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** We need to set Rule.referencedPredefinedRuleAttributes before
* code generation. This filter looks at an action in context of
* its rule and outer alternative number and figures out which
* rules have predefined prefs referenced. I need this so I can
* remove unusued labels. This also tracks, for labeled rules,
* which are referenced by actions.
*/
lexer grammar ActionAnalysis;
options {
filter=true; // try all non-fragment rules in order specified
}
@header {
package org.antlr.grammar.v3;
import org.antlr.runtime.*;
import org.antlr.tool.*;
}
@members {
Rule enclosingRule;
Grammar grammar;
antlr.Token actionToken;
int outerAltNum = 0;
public ActionAnalysis(Grammar grammar, String ruleName, GrammarAST actionAST)
{
this(new ANTLRStringStream(actionAST.token.getText()));
this.grammar = grammar;
this.enclosingRule = grammar.getLocallyDefinedRule(ruleName);
this.actionToken = actionAST.token;
this.outerAltNum = actionAST.outerAltNum;
}
public void analyze() {
// System.out.println("###\naction="+actionToken);
Token t;
do {
t = nextToken();
} while ( t.getType()!= Token.EOF );
}
}
/** $x.y x is enclosing rule or rule ref or rule label
* y is a return value, parameter, or predefined property.
*/
X_Y : '$' x=ID '.' y=ID {enclosingRule!=null}?
{
AttributeScope scope = null;
String refdRuleName = null;
if ( $x.text.equals(enclosingRule.name) ) {
// ref to enclosing rule.
refdRuleName = $x.text;
scope = enclosingRule.getLocalAttributeScope($y.text);
}
else if ( enclosingRule.getRuleLabel($x.text)!=null ) {
// ref to rule label
Grammar.LabelElementPair pair = enclosingRule.getRuleLabel($x.text);
pair.actionReferencesLabel = true;
refdRuleName = pair.referencedRuleName;
Rule refdRule = grammar.getRule(refdRuleName);
if ( refdRule!=null ) {
scope = refdRule.getLocalAttributeScope($y.text);
}
}
else if ( enclosingRule.getRuleRefsInAlt(x.getText(), outerAltNum)!=null ) {
// ref to rule referenced in this alt
refdRuleName = $x.text;
Rule refdRule = grammar.getRule(refdRuleName);
if ( refdRule!=null ) {
scope = refdRule.getLocalAttributeScope($y.text);
}
}
if ( scope!=null &&
(scope.isPredefinedRuleScope||scope.isPredefinedLexerRuleScope) )
{
grammar.referenceRuleLabelPredefinedAttribute(refdRuleName);
//System.out.println("referenceRuleLabelPredefinedAttribute for "+refdRuleName);
}
}
;
/** $x x is an isolated rule label. Just record that the label was referenced */
X : '$' x=ID {enclosingRule!=null && enclosingRule.getRuleLabel($x.text)!=null}?
{
Grammar.LabelElementPair pair = enclosingRule.getRuleLabel($x.text);
pair.actionReferencesLabel = true;
}
;
/** $y y is a return value, parameter, or predefined property of current rule */
Y : '$' ID {enclosingRule!=null && enclosingRule.getLocalAttributeScope($ID.text)!=null}?
{
AttributeScope scope = enclosingRule.getLocalAttributeScope($ID.text);
if ( scope!=null &&
(scope.isPredefinedRuleScope||scope.isPredefinedLexerRuleScope) )
{
grammar.referenceRuleLabelPredefinedAttribute(enclosingRule.name);
//System.out.println("referenceRuleLabelPredefinedAttribute for "+$ID.text);
}
}
;
fragment
ID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;
antlr-3.2/tool/src/main/antlr2/ 0000755 0001750 0001750 00000000000 11410174107 016322 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr2/org/ 0000755 0001750 0001750 00000000000 11410174107 017111 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr2/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 020231 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/ 0000755 0001750 0001750 00000000000 11410174107 021657 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/v2/ 0000755 0001750 0001750 00000000000 11410174107 022206 5 ustar twerner twerner antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/v2/assign.types.g 0000644 0001750 0001750 00000022402 11256465226 025022 0 ustar twerner twerner header {
/*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.grammar.v2;
import java.util.*;
import org.antlr.analysis.*;
import org.antlr.misc.*;
import java.io.*;
import org.antlr.tool.*;
}
/** [Warning: TJP says that this is probably out of date as of 11/19/2005,
* but since it's probably still useful, I'll leave in. Don't have energy
* to update at the moment.]
*
* Compute the token types for all literals and rules etc.. There are
* a few different cases to consider for grammar types and a few situations
* within.
*
* CASE 1 : pure parser grammar
* a) Any reference to a token gets a token type.
* b) The tokens section may alias a token name to a string or char
*
* CASE 2 : pure lexer grammar
* a) Import token vocabulary if available. Set token types for any new tokens
* to values above last imported token type
* b) token rule definitions get token types if not already defined
* c) literals do NOT get token types
*
* CASE 3 : merged parser / lexer grammar
* a) Any char or string literal gets a token type in a parser rule
* b) Any reference to a token gets a token type if not referencing
* a fragment lexer rule
* c) The tokens section may alias a token name to a string or char
* which must add a rule to the lexer
* d) token rule definitions get token types if not already defined
* e) token rule definitions may also alias a token name to a literal.
* E.g., Rule 'FOR : "for";' will alias FOR to "for" in the sense that
* references to either in the parser grammar will yield the token type
*
* What this pass does:
*
* 0. Collects basic info about the grammar like grammar name and type;
* Oh, I have go get the options in case they affect the token types.
* E.g., tokenVocab option.
* Imports any token vocab name/type pairs into a local hashtable.
* 1. Finds a list of all literals and token names.
* 2. Finds a list of all token name rule definitions;
* no token rules implies pure parser.
* 3. Finds a list of all simple token rule defs of form "<NAME> : <literal>;"
* and aliases them.
* 4. Walks token names table and assign types to any unassigned
* 5. Walks aliases and assign types to referenced literals
* 6. Walks literals, assigning types if untyped
* 4. Informs the Grammar object of the type definitions such as:
* g.defineToken(<charliteral>, ttype);
* g.defineToken(<stringliteral>, ttype);
* g.defineToken(<tokenID>, ttype);
* where some of the ttype values will be the same for aliases tokens.
*/
class AssignTokenTypesWalker extends TreeParser;
options {
importVocab = ANTLR;
ASTLabelType = "GrammarAST";
codeGenBitsetTestThreshold=999;
}
{
public void reportError(RecognitionException ex) {
Token token = null;
if ( ex instanceof MismatchedTokenException ) {
token = ((MismatchedTokenException)ex).token;
}
else if ( ex instanceof NoViableAltException ) {
token = ((NoViableAltException)ex).token;
}
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
grammar,
token,
"assign.types: "+ex.toString(),
ex);
}
protected Grammar grammar;
protected String currentRuleName;
protected static GrammarAST stringAlias;
protected static GrammarAST charAlias;
protected static GrammarAST stringAlias2;
protected static GrammarAST charAlias2;
protected void initASTPatterns() {
stringAlias =
#(#[BLOCK], #(#[ALT], #[STRING_LITERAL], #[EOA]), #[EOB]);
charAlias =
#(#[BLOCK], #(#[ALT], #[CHAR_LITERAL], #[EOA]), #[EOB]);
stringAlias2 =
#(#[BLOCK], #(#[ALT], #[STRING_LITERAL], #[ACTION], #[EOA]),#[EOB]);
charAlias2 =
#(#[BLOCK], #(#[ALT], #[CHAR_LITERAL], #[ACTION], #[EOA]), #[EOB]);
}
// Behavior moved to AssignTokenTypesBehavior
protected void trackString(GrammarAST t) {;}
protected void trackToken(GrammarAST t) {;}
protected void trackTokenRule(GrammarAST t, GrammarAST modifier, GrammarAST block) {;}
protected void alias(GrammarAST t, GrammarAST s) {;}
public void defineTokens(Grammar root) {;}
protected void defineStringLiteralsFromDelegates() {;}
protected void assignStringTypes(Grammar root) {;}
protected void aliasTokenIDsAndLiterals(Grammar root) {;}
protected void assignTokenIDTypes(Grammar root) {;}
protected void defineTokenNamesAndLiteralsInGrammar(Grammar root) {;}
protected void init(Grammar root) {;}
}
grammar[Grammar g]
{
init(g);
}
: ( #( LEXER_GRAMMAR grammarSpec )
| #( PARSER_GRAMMAR grammarSpec )
| #( TREE_GRAMMAR grammarSpec )
| #( COMBINED_GRAMMAR grammarSpec )
)
;
grammarSpec
{Map opts=null;}
: id:ID
(cmt:DOC_COMMENT)?
(optionsSpec)?
(delegateGrammars)?
(tokensSpec)?
(attrScope)*
(AMPERSAND)* // skip actions
rules
;
attrScope
: #( "scope" ID ACTION )
;
optionsSpec returns [Map opts=new HashMap()]
: #( OPTIONS (option[opts])+ )
;
option[Map opts]
{
String key=null;
Object value=null;
}
: #( ASSIGN id:ID {key=#id.getText();} value=optionValue )
{
opts.put(key,value);
// check for grammar-level option to import vocabulary
if ( currentRuleName==null && key.equals("tokenVocab") ) {
grammar.importTokenVocabulary(#id,(String)value);
}
}
;
optionValue returns [Object value=null]
: id:ID {value = #id.getText();}
| s:STRING_LITERAL {value = #s.getText();}
| c:CHAR_LITERAL {value = #c.getText();}
| i:INT {value = new Integer(#i.getText());}
// | cs:charSet {value = #cs;} // return set AST in this case
;
charSet
: #( CHARSET charSetElement )
;
charSetElement
: c:CHAR_LITERAL
| #( OR c1:CHAR_LITERAL c2:CHAR_LITERAL )
| #( RANGE c3:CHAR_LITERAL c4:CHAR_LITERAL )
;
delegateGrammars
: #( "import"
( #(ASSIGN ID ID)
| ID
)+
)
;
tokensSpec
: #( TOKENS ( tokenSpec )+ )
;
tokenSpec
: t:TOKEN_REF {trackToken(t);}
| #( ASSIGN
t2:TOKEN_REF {trackToken(t2);}
( s:STRING_LITERAL {trackString(s); alias(t2,s);}
| c:CHAR_LITERAL {trackString(c); alias(t2,c);}
)
)
;
rules
: ( rule )+
;
rule
: #( RULE id:ID {currentRuleName=#id.getText();}
(m:modifier)?
(ARG (ARG_ACTION)?)
(RET (ARG_ACTION)?)
(optionsSpec)?
(ruleScopeSpec)?
(AMPERSAND)*
b:block
(exceptionGroup)?
EOR
{trackTokenRule(#id,#m,#b);}
)
;
modifier
: "protected"
| "public"
| "private"
| "fragment"
;
ruleScopeSpec
: #( "scope" (ACTION)? ( ID )* )
;
block
: #( BLOCK
(optionsSpec)?
( alternative rewrite )+
EOB
)
;
alternative
: #( ALT (element)+ EOA )
;
exceptionGroup
: ( exceptionHandler )+ (finallyClause)?
| finallyClause
;
exceptionHandler
: #("catch" ARG_ACTION ACTION)
;
finallyClause
: #("finally" ACTION)
;
rewrite
: ( #( REWRITE (SEMPRED)? (ALT|TEMPLATE|ACTION|ETC) ) )*
;
element
: #(ROOT element)
| #(BANG element)
| atom
| #(NOT element)
| #(RANGE atom atom)
| #(CHAR_RANGE atom atom)
| #(ASSIGN ID element)
| #(PLUS_ASSIGN ID element)
| ebnf
| tree
| #( SYNPRED block )
| FORCED_ACTION
| ACTION
| SEMPRED
| SYN_SEMPRED
| BACKTRACK_SEMPRED
| GATED_SEMPRED
| EPSILON
;
ebnf: block
| #( OPTIONAL block )
| #( CLOSURE block )
| #( POSITIVE_CLOSURE block )
;
tree: #(TREE_BEGIN element (element)* )
;
atom
: #( rr:RULE_REF (rarg:ARG_ACTION)? )
| #( t:TOKEN_REF (targ:ARG_ACTION )? ) {trackToken(t);}
| c:CHAR_LITERAL {trackString(c);}
| s:STRING_LITERAL {trackString(s);}
| WILDCARD
| #(DOT ID atom) // scope override on rule
;
ast_suffix
: ROOT
| BANG
;
antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/v2/buildnfa.g 0000644 0001750 0001750 00000054026 11256465226 024166 0 ustar twerner twerner header {
/*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.grammar.v2;
import java.util.*;
import org.antlr.analysis.*;
import org.antlr.misc.*;
import org.antlr.tool.*;
}
/** Build an NFA from a tree representing an ANTLR grammar. */
class TreeToNFAConverter extends TreeParser;
options {
importVocab = ANTLR;
ASTLabelType = "GrammarAST";
}
{
/** Factory used to create nodes and submachines */
protected NFAFactory factory = null;
/** Which NFA object are we filling in? */
protected NFA nfa = null;
/** Which grammar are we converting an NFA for? */
protected Grammar grammar = null;
protected String currentRuleName = null;
protected int outerAltNum = 0;
protected int blockLevel = 0;
public TreeToNFAConverter(Grammar g, NFA nfa, NFAFactory factory) {
this();
this.grammar = g;
this.nfa = nfa;
this.factory = factory;
}
/*
protected void init() {
// define all the rule begin/end NFAStates to solve forward reference issues
Collection rules = grammar.getRules();
for (Iterator itr = rules.iterator(); itr.hasNext();) {
Rule r = (Rule) itr.next();
String ruleName = r.name;
NFAState ruleBeginState = factory.newState();
ruleBeginState.setDescription("rule "+ruleName+" start");
ruleBeginState.enclosingRule = r;
r.startState = ruleBeginState;
NFAState ruleEndState = factory.newState();
ruleEndState.setDescription("rule "+ruleName+" end");
ruleEndState.setAcceptState(true);
ruleEndState.enclosingRule = r;
r.stopState = ruleEndState;
}
}
*/
protected void addFollowTransition(String ruleName, NFAState following) {
//System.out.println("adding follow link to rule "+ruleName);
// find last link in FOLLOW chain emanating from rule
Rule r = grammar.getRule(ruleName);
NFAState end = r.stopState;
while ( end.transition(1)!=null ) {
end = (NFAState)end.transition(1).target;
}
if ( end.transition(0)!=null ) {
// already points to a following node
// gotta add another node to keep edges to a max of 2
NFAState n = factory.newState();
Transition e = new Transition(Label.EPSILON, n);
end.addTransition(e);
end = n;
}
Transition followEdge = new Transition(Label.EPSILON, following);
end.addTransition(followEdge);
}
protected void finish() {
List rules = new LinkedList();
rules.addAll(grammar.getRules());
int numEntryPoints = factory.build_EOFStates(rules);
if ( numEntryPoints==0 ) {
ErrorManager.grammarWarning(ErrorManager.MSG_NO_GRAMMAR_START_RULE,
grammar,
null,
grammar.name);
}
}
public void reportError(RecognitionException ex) {
Token token = null;
if ( ex instanceof MismatchedTokenException ) {
token = ((MismatchedTokenException)ex).token;
}
else if ( ex instanceof NoViableAltException ) {
token = ((NoViableAltException)ex).token;
}
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
grammar,
token,
"buildnfa: "+ex.toString(),
ex);
}
}
grammar
: ( #( LEXER_GRAMMAR grammarSpec )
| #( PARSER_GRAMMAR grammarSpec )
| #( TREE_GRAMMAR grammarSpec )
| #( COMBINED_GRAMMAR grammarSpec )
)
{finish();}
;
attrScope
: #( "scope" ID ACTION )
;
grammarSpec
: ID
(cmt:DOC_COMMENT)?
( #(OPTIONS .) )?
( #("import" .) )?
( #(TOKENS .) )?
(attrScope)*
(AMPERSAND)* // skip actions
rules
;
rules
: ( rule )+
;
rule
{
StateCluster g=null;
StateCluster b = null;
String r=null;
}
: #( RULE id:ID {r=#id.getText();}
{
currentRuleName = r;
factory.setCurrentRule(grammar.getLocallyDefinedRule(r));
}
(modifier)?
(ARG (ARG_ACTION)?)
(RET (ARG_ACTION)?)
( OPTIONS )?
( ruleScopeSpec )?
(AMPERSAND)*
{GrammarAST blk = (GrammarAST)_t;}
b=block
(exceptionGroup)?
EOR
{
if ( blk.getSetValue() !=null ) {
// if block comes back as a set not BLOCK, make it
// a single ALT block
b = factory.build_AlternativeBlockFromSet(b);
}
if ( Character.isLowerCase(r.charAt(0)) ||
grammar.type==Grammar.LEXER )
{
// attach start node to block for this rule
Rule thisR = grammar.getLocallyDefinedRule(r);
NFAState start = thisR.startState;
start.associatedASTNode = #id;
start.addTransition(new Transition(Label.EPSILON, b.left));
// track decision if > 1 alts
if ( grammar.getNumberOfAltsForDecisionNFA(b.left)>1 ) {
b.left.setDescription(grammar.grammarTreeToString(#rule,false));
b.left.setDecisionASTNode(blk);
int d = grammar.assignDecisionNumber( b.left );
grammar.setDecisionNFA( d, b.left );
grammar.setDecisionBlockAST(d, blk);
}
// hook to end of rule node
NFAState end = thisR.stopState;
b.right.addTransition(new Transition(Label.EPSILON,end));
}
}
)
;
modifier
: "protected"
| "public"
| "private"
| "fragment"
;
ruleScopeSpec
: #( "scope" (ACTION)? ( ID )* )
;
block returns [StateCluster g = null]
{
StateCluster a = null;
List alts = new LinkedList();
this.blockLevel++;
if ( this.blockLevel==1 ) {this.outerAltNum=1;}
}
: {grammar.isValidSet(this,#block) &&
!currentRuleName.equals(Grammar.ARTIFICIAL_TOKENS_RULENAME)}?
g=set
{this.blockLevel--;}
| #( BLOCK ( OPTIONS )?
( a=alternative rewrite
{
alts.add(a);
if ( this.blockLevel==1 ) {this.outerAltNum++;}
}
)+
EOB
)
{g = factory.build_AlternativeBlock(alts);}
{this.blockLevel--;}
;
alternative returns [StateCluster g=null]
{
StateCluster e = null;
}
: #( ALT (e=element {g = factory.build_AB(g,e);} )+ )
{
if (g==null) { // if alt was a list of actions or whatever
g = factory.build_Epsilon();
}
else {
factory.optimizeAlternative(g);
}
}
;
exceptionGroup
: ( exceptionHandler )+ (finallyClause)?
| finallyClause
;
exceptionHandler
: #("catch" ARG_ACTION ACTION)
;
finallyClause
: #("finally" ACTION)
;
rewrite
: (
{
if ( grammar.getOption("output")==null ) {
ErrorManager.grammarError(ErrorManager.MSG_REWRITE_OR_OP_WITH_NO_OUTPUT_OPTION,
grammar, #rewrite.token, currentRuleName);
}
}
#( REWRITE (SEMPRED)? (ALT|TEMPLATE|ACTION|ETC) )
)*
;
element returns [StateCluster g=null]
: #(ROOT g=element)
| #(BANG g=element)
| #(ASSIGN ID g=element)
| #(PLUS_ASSIGN ID g=element)
| #(RANGE a:atom[null] b:atom[null])
{g = factory.build_Range(grammar.getTokenType(#a.getText()),
grammar.getTokenType(#b.getText()));}
| #(CHAR_RANGE c1:CHAR_LITERAL c2:CHAR_LITERAL)
{
if ( grammar.type==Grammar.LEXER ) {
g = factory.build_CharRange(#c1.getText(), #c2.getText());
}
}
| g=atom_or_notatom
| g=ebnf
| g=tree
| #( SYNPRED block )
| ACTION {g = factory.build_Action(#ACTION);}
| FORCED_ACTION {g = factory.build_Action(#FORCED_ACTION);}
| pred:SEMPRED {g = factory.build_SemanticPredicate(#pred);}
| spred:SYN_SEMPRED {g = factory.build_SemanticPredicate(#spred);}
| bpred:BACKTRACK_SEMPRED {g = factory.build_SemanticPredicate(#bpred);}
| gpred:GATED_SEMPRED {g = factory.build_SemanticPredicate(#gpred);}
| EPSILON {g = factory.build_Epsilon();}
;
ebnf returns [StateCluster g=null]
{
StateCluster b = null;
GrammarAST blk = #ebnf;
if ( blk.getType()!=BLOCK ) {
blk = (GrammarAST)blk.getFirstChild();
}
GrammarAST eob = blk.getLastChild();
}
: {grammar.isValidSet(this,#ebnf)}? g=set
| b=block
{
// track decision if > 1 alts
if ( grammar.getNumberOfAltsForDecisionNFA(b.left)>1 ) {
b.left.setDescription(grammar.grammarTreeToString(blk,false));
b.left.setDecisionASTNode(blk);
int d = grammar.assignDecisionNumber( b.left );
grammar.setDecisionNFA( d, b.left );
grammar.setDecisionBlockAST(d, blk);
}
g = b;
}
| #( OPTIONAL b=block )
{
if ( blk.getSetValue() !=null ) {
// if block comes back SET not BLOCK, make it
// a single ALT block
b = factory.build_AlternativeBlockFromSet(b);
}
g = factory.build_Aoptional(b);
g.left.setDescription(grammar.grammarTreeToString(#ebnf,false));
// there is always at least one alt even if block has just 1 alt
int d = grammar.assignDecisionNumber( g.left );
grammar.setDecisionNFA(d, g.left);
grammar.setDecisionBlockAST(d, blk);
g.left.setDecisionASTNode(#ebnf);
}
| #( CLOSURE b=block )
{
if ( blk.getSetValue() !=null ) {
b = factory.build_AlternativeBlockFromSet(b);
}
g = factory.build_Astar(b);
// track the loop back / exit decision point
b.right.setDescription("()* loopback of "+grammar.grammarTreeToString(#ebnf,false));
int d = grammar.assignDecisionNumber( b.right );
grammar.setDecisionNFA(d, b.right);
grammar.setDecisionBlockAST(d, blk);
b.right.setDecisionASTNode(eob);
// make block entry state also have same decision for interpreting grammar
NFAState altBlockState = (NFAState)g.left.transition(0).target;
altBlockState.setDecisionASTNode(#ebnf);
altBlockState.setDecisionNumber(d);
g.left.setDecisionNumber(d); // this is the bypass decision (2 alts)
g.left.setDecisionASTNode(#ebnf);
}
| #( POSITIVE_CLOSURE b=block )
{
if ( blk.getSetValue() !=null ) {
b = factory.build_AlternativeBlockFromSet(b);
}
g = factory.build_Aplus(b);
// don't make a decision on left edge, can reuse loop end decision
// track the loop back / exit decision point
b.right.setDescription("()+ loopback of "+grammar.grammarTreeToString(#ebnf,false));
int d = grammar.assignDecisionNumber( b.right );
grammar.setDecisionNFA(d, b.right);
grammar.setDecisionBlockAST(d, blk);
b.right.setDecisionASTNode(eob);
// make block entry state also have same decision for interpreting grammar
NFAState altBlockState = (NFAState)g.left.transition(0).target;
altBlockState.setDecisionASTNode(#ebnf);
altBlockState.setDecisionNumber(d);
}
;
tree returns [StateCluster g=null]
{
StateCluster e=null;
GrammarAST el=null;
StateCluster down=null, up=null;
}
: #( TREE_BEGIN
{el=(GrammarAST)_t;}
g=element
{
down = factory.build_Atom(Label.DOWN, el);
// TODO set following states for imaginary nodes?
//el.followingNFAState = down.right;
g = factory.build_AB(g,down);
}
( {el=(GrammarAST)_t;} e=element {g = factory.build_AB(g,e);} )*
{
up = factory.build_Atom(Label.UP, el);
//el.followingNFAState = up.right;
g = factory.build_AB(g,up);
// tree roots point at right edge of DOWN for LOOK computation later
#tree.NFATreeDownState = down.left;
}
)
;
atom_or_notatom returns [StateCluster g=null]
: g=atom[null]
| #( n:NOT
( c:CHAR_LITERAL (ast1:ast_suffix)?
{
int ttype=0;
if ( grammar.type==Grammar.LEXER ) {
ttype = Grammar.getCharValueFromGrammarCharLiteral(#c.getText());
}
else {
ttype = grammar.getTokenType(#c.getText());
}
IntSet notAtom = grammar.complement(ttype);
if ( notAtom.isNil() ) {
ErrorManager.grammarError(ErrorManager.MSG_EMPTY_COMPLEMENT,
grammar,
#c.token,
#c.getText());
}
g=factory.build_Set(notAtom,#n);
}
| t:TOKEN_REF (ast3:ast_suffix)?
{
int ttype=0;
IntSet notAtom = null;
if ( grammar.type==Grammar.LEXER ) {
notAtom = grammar.getSetFromRule(this,#t.getText());
if ( notAtom==null ) {
ErrorManager.grammarError(ErrorManager.MSG_RULE_INVALID_SET,
grammar,
#t.token,
#t.getText());
}
else {
notAtom = grammar.complement(notAtom);
}
}
else {
ttype = grammar.getTokenType(#t.getText());
notAtom = grammar.complement(ttype);
}
if ( notAtom==null || notAtom.isNil() ) {
ErrorManager.grammarError(ErrorManager.MSG_EMPTY_COMPLEMENT,
grammar,
#t.token,
#t.getText());
}
g=factory.build_Set(notAtom,#n);
}
| g=set
{
GrammarAST stNode = (GrammarAST)n.getFirstChild();
//IntSet notSet = grammar.complement(stNode.getSetValue());
// let code generator complement the sets
IntSet s = stNode.getSetValue();
stNode.setSetValue(s);
// let code gen do the complement again; here we compute
// for NFA construction
s = grammar.complement(s);
if ( s.isNil() ) {
ErrorManager.grammarError(ErrorManager.MSG_EMPTY_COMPLEMENT,
grammar,
#n.token);
}
g=factory.build_Set(s,#n);
}
)
{#n.followingNFAState = g.right;}
)
;
atom[String scopeName] returns [StateCluster g=null]
: #( r:RULE_REF (rarg:ARG_ACTION)? (as1:ast_suffix)? )
{
NFAState start = grammar.getRuleStartState(scopeName,r.getText());
if ( start!=null ) {
Rule rr = grammar.getRule(scopeName,r.getText());
g = factory.build_RuleRef(rr, start);
r.followingNFAState = g.right;
r.NFAStartState = g.left;
if ( g.left.transition(0) instanceof RuleClosureTransition
&& grammar.type!=Grammar.LEXER )
{
addFollowTransition(r.getText(), g.right);
}
// else rule ref got inlined to a set
}
}
| #( t:TOKEN_REF (targ:ARG_ACTION)? (as2:ast_suffix)? )
{
if ( grammar.type==Grammar.LEXER ) {
NFAState start = grammar.getRuleStartState(scopeName,t.getText());
if ( start!=null ) {
Rule rr = grammar.getRule(scopeName,t.getText());
g = factory.build_RuleRef(rr, start);
t.NFAStartState = g.left;
// don't add FOLLOW transitions in the lexer;
// only exact context should be used.
}
}
else {
g = factory.build_Atom(t);
t.followingNFAState = g.right;
}
}
| #( c:CHAR_LITERAL (as3:ast_suffix)? )
{
if ( grammar.type==Grammar.LEXER ) {
g = factory.build_CharLiteralAtom(c);
}
else {
g = factory.build_Atom(c);
c.followingNFAState = g.right;
}
}
| #( s:STRING_LITERAL (as4:ast_suffix)? )
{
if ( grammar.type==Grammar.LEXER ) {
g = factory.build_StringLiteralAtom(s);
}
else {
g = factory.build_Atom(s);
s.followingNFAState = g.right;
}
}
| #( w:WILDCARD (as5:ast_suffix)? )
{
if ( nfa.grammar.type==Grammar.TREE_PARSER ) {
g = factory.build_WildcardTree(#w);
}
else {
g = factory.build_Wildcard(#w);
}
}
| #( DOT scope:ID g=atom[#scope.getText()] ) // scope override
;
ast_suffix
: ROOT
| BANG
;
set returns [StateCluster g=null]
{
IntSet elements=new IntervalSet();
#set.setSetValue(elements); // track set for use by code gen
}
: #( b:BLOCK
(#(ALT (BACKTRACK_SEMPRED)? setElement[elements] EOA))+
EOB
)
{
g = factory.build_Set(elements,#b);
#b.followingNFAState = g.right;
#b.setSetValue(elements); // track set value of this block
}
//{System.out.println("set elements="+elements.toString(grammar));}
;
setRule returns [IntSet elements=new IntervalSet()]
{IntSet s=null;}
: #( RULE id:ID (modifier)? ARG RET ( OPTIONS )? ( ruleScopeSpec )?
(AMPERSAND)*
#( BLOCK ( OPTIONS )?
( #(ALT (BACKTRACK_SEMPRED)? setElement[elements] EOA) )+
EOB
)
(exceptionGroup)?
EOR
)
;
exception
catch[RecognitionException re] {throw re;}
setElement[IntSet elements]
{
int ttype;
IntSet ns=null;
StateCluster gset;
}
: c:CHAR_LITERAL
{
if ( grammar.type==Grammar.LEXER ) {
ttype = Grammar.getCharValueFromGrammarCharLiteral(c.getText());
}
else {
ttype = grammar.getTokenType(c.getText());
}
if ( elements.member(ttype) ) {
ErrorManager.grammarError(ErrorManager.MSG_DUPLICATE_SET_ENTRY,
grammar,
#c.token,
#c.getText());
}
elements.add(ttype);
}
| t:TOKEN_REF
{
if ( grammar.type==Grammar.LEXER ) {
// recursively will invoke this rule to match elements in target rule ref
IntSet ruleSet = grammar.getSetFromRule(this,#t.getText());
if ( ruleSet==null ) {
ErrorManager.grammarError(ErrorManager.MSG_RULE_INVALID_SET,
grammar,
#t.token,
#t.getText());
}
else {
elements.addAll(ruleSet);
}
}
else {
ttype = grammar.getTokenType(t.getText());
if ( elements.member(ttype) ) {
ErrorManager.grammarError(ErrorManager.MSG_DUPLICATE_SET_ENTRY,
grammar,
#t.token,
#t.getText());
}
elements.add(ttype);
}
}
| s:STRING_LITERAL
{
ttype = grammar.getTokenType(s.getText());
if ( elements.member(ttype) ) {
ErrorManager.grammarError(ErrorManager.MSG_DUPLICATE_SET_ENTRY,
grammar,
#s.token,
#s.getText());
}
elements.add(ttype);
}
| #(CHAR_RANGE c1:CHAR_LITERAL c2:CHAR_LITERAL)
{
if ( grammar.type==Grammar.LEXER ) {
int a = Grammar.getCharValueFromGrammarCharLiteral(c1.getText());
int b = Grammar.getCharValueFromGrammarCharLiteral(c2.getText());
elements.addAll(IntervalSet.of(a,b));
}
}
| gset=set
{
Transition setTrans = gset.left.transition(0);
elements.addAll(setTrans.label.getSet());
}
| #( NOT {ns=new IntervalSet();}
setElement[ns]
{
IntSet not = grammar.complement(ns);
elements.addAll(not);
}
)
;
/** Check to see if this block can be a set. Can't have actions
* etc... Also can't be in a rule with a rewrite as we need
* to track what's inside set for use in rewrite.
*/
testBlockAsSet
{
int nAlts=0;
Rule r = grammar.getLocallyDefinedRule(currentRuleName);
}
: #( BLOCK
( #(ALT (BACKTRACK_SEMPRED)? testSetElement {nAlts++;} EOA)
{!r.hasRewrite(outerAltNum)}?
)+
EOB
)
{nAlts>1}? // set of 1 element is not good
;
exception
catch[RecognitionException re] {throw re;}
testSetRule
: #( RULE id:ID (modifier)? ARG RET ( OPTIONS )? ( ruleScopeSpec )?
(AMPERSAND)*
#( BLOCK
( #(ALT (BACKTRACK_SEMPRED)? testSetElement EOA) )+
EOB
)
(exceptionGroup)?
EOR
)
;
exception
catch[RecognitionException re] {throw re;}
/** Match just an element; no ast suffix etc.. */
testSetElement
{
AST r = _t;
}
: c:CHAR_LITERAL
| t:TOKEN_REF
{
if ( grammar.type==Grammar.LEXER ) {
Rule rule = grammar.getRule(#t.getText());
if ( rule==null ) {
throw new RecognitionException("invalid rule");
}
// recursively will invoke this rule to match elements in target rule ref
testSetRule(rule.tree);
}
}
| {grammar.type!=Grammar.LEXER}? s:STRING_LITERAL
| #(CHAR_RANGE c1:CHAR_LITERAL c2:CHAR_LITERAL)
| testBlockAsSet
| #( NOT testSetElement )
;
exception
catch[RecognitionException re] {throw re;}
antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/v2/codegen.g 0000644 0001750 0001750 00000136005 11256465226 024004 0 ustar twerner twerner header {
/*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.grammar.v2;
import org.antlr.tool.*;
import org.antlr.analysis.*;
import org.antlr.misc.*;
import java.util.*;
import org.antlr.stringtemplate.*;
import antlr.TokenWithIndex;
import antlr.CommonToken;
import org.antlr.codegen.*;
}
/** Walk a grammar and generate code by gradually building up
* a bigger and bigger StringTemplate.
*
* Terence Parr
* University of San Francisco
* June 15, 2004
*/
class CodeGenTreeWalker extends TreeParser;
options {
// warning! ANTLR cannot see another directory to get vocabs, so I had
// to copy the ANTLRTokenTypes.txt file into this dir from ../tools!
// Yuck! If you modify ../tools/antlr.g, make sure to copy the vocab here.
importVocab = ANTLR;
codeGenBitsetTestThreshold=999;
ASTLabelType=GrammarAST;
}
{
protected static final int RULE_BLOCK_NESTING_LEVEL = 0;
protected static final int OUTER_REWRITE_NESTING_LEVEL = 0;
public String getCurrentRuleName() {
return currentRuleName;
}
public void setCurrentRuleName(String currentRuleName) {
this.currentRuleName = currentRuleName;
}
public int getOuterAltNum() {
return outerAltNum;
}
public void setOuterAltNum(int outerAltNum) {
this.outerAltNum = outerAltNum;
}
protected String currentRuleName = null;
protected int blockNestingLevel = 0;
protected int rewriteBlockNestingLevel = 0;
protected int outerAltNum = 0;
protected StringTemplate currentBlockST = null;
protected boolean currentAltHasASTRewrite = false;
protected int rewriteTreeNestingLevel = 0;
protected Set rewriteRuleRefs = null;
public void reportError(RecognitionException ex) {
Token token = null;
if ( ex instanceof MismatchedTokenException ) {
token = ((MismatchedTokenException)ex).token;
}
else if ( ex instanceof NoViableAltException ) {
token = ((NoViableAltException)ex).token;
}
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
grammar,
token,
"codegen: "+ex.toString(),
ex);
}
public void reportError(String s) {
System.out.println("codegen: error: " + s);
}
protected CodeGenerator generator;
protected Grammar grammar;
protected StringTemplateGroup templates;
/** The overall lexer/parser template; simulate dynamically scoped
* attributes by making this an instance var of the walker.
*/
protected StringTemplate recognizerST;
protected StringTemplate outputFileST;
protected StringTemplate headerFileST;
protected String outputOption = "";
protected StringTemplate getWildcardST(GrammarAST elementAST, GrammarAST ast_suffix, String label) {
String name = "wildcard";
if ( grammar.type==Grammar.LEXER ) {
name = "wildcardChar";
}
return getTokenElementST(name, name, elementAST, ast_suffix, label);
}
protected StringTemplate getRuleElementST(String name,
String ruleTargetName,
GrammarAST elementAST,
GrammarAST ast_suffix,
String label)
{
String suffix = getSTSuffix(elementAST,ast_suffix,label);
name += suffix;
// if we're building trees and there is no label, gen a label
// unless we're in a synpred rule.
Rule r = grammar.getRule(currentRuleName);
if ( (grammar.buildAST()||suffix.length()>0) && label==null &&
(r==null || !r.isSynPred) )
{
// we will need a label to do the AST or tracking, make one
label = generator.createUniqueLabel(ruleTargetName);
CommonToken labelTok = new CommonToken(ANTLRParser.ID, label);
grammar.defineRuleRefLabel(currentRuleName, labelTok, elementAST);
}
StringTemplate elementST = templates.getInstanceOf(name);
if ( label!=null ) {
elementST.setAttribute("label", label);
}
return elementST;
}
protected StringTemplate getTokenElementST(String name,
String elementName,
GrammarAST elementAST,
GrammarAST ast_suffix,
String label)
{
String suffix = getSTSuffix(elementAST,ast_suffix,label);
name += suffix;
// if we're building trees and there is no label, gen a label
// unless we're in a synpred rule.
Rule r = grammar.getRule(currentRuleName);
if ( (grammar.buildAST()||suffix.length()>0) && label==null &&
(r==null || !r.isSynPred) )
{
label = generator.createUniqueLabel(elementName);
CommonToken labelTok = new CommonToken(ANTLRParser.ID, label);
grammar.defineTokenRefLabel(currentRuleName, labelTok, elementAST);
}
StringTemplate elementST = templates.getInstanceOf(name);
if ( label!=null ) {
elementST.setAttribute("label", label);
}
return elementST;
}
public boolean isListLabel(String label) {
boolean hasListLabel=false;
if ( label!=null ) {
Rule r = grammar.getRule(currentRuleName);
String stName = null;
if ( r!=null ) {
Grammar.LabelElementPair pair = r.getLabel(label);
if ( pair!=null &&
(pair.type==Grammar.TOKEN_LIST_LABEL||
pair.type==Grammar.RULE_LIST_LABEL||
pair.type==Grammar.WILDCARD_TREE_LIST_LABEL) )
{
hasListLabel=true;
}
}
}
return hasListLabel;
}
/** Return a non-empty template name suffix if the token is to be
* tracked, added to a tree, or both.
*/
protected String getSTSuffix(GrammarAST elementAST, GrammarAST ast_suffix, String label) {
if ( grammar.type==Grammar.LEXER ) {
return "";
}
// handle list label stuff; make element use "Track"
String operatorPart = "";
String rewritePart = "";
String listLabelPart = "";
Rule ruleDescr = grammar.getRule(currentRuleName);
if ( ast_suffix!=null && !ruleDescr.isSynPred ) {
if ( ast_suffix.getType()==ANTLRParser.ROOT ) {
operatorPart = "RuleRoot";
}
else if ( ast_suffix.getType()==ANTLRParser.BANG ) {
operatorPart = "Bang";
}
}
if ( currentAltHasASTRewrite && elementAST.getType()!=WILDCARD ) {
rewritePart = "Track";
}
if ( isListLabel(label) ) {
listLabelPart = "AndListLabel";
}
String STsuffix = operatorPart+rewritePart+listLabelPart;
//System.out.println("suffix = "+STsuffix);
return STsuffix;
}
/** Convert rewrite AST lists to target labels list */
protected List<String> getTokenTypesAsTargetLabels(Set<GrammarAST> refs) {
if ( refs==null || refs.size()==0 ) {
return null;
}
List<String> labels = new ArrayList<String>(refs.size());
for (GrammarAST t : refs) {
String label;
if ( t.getType()==ANTLRParser.RULE_REF ) {
label = t.getText();
}
else if ( t.getType()==ANTLRParser.LABEL ) {
label = t.getText();
}
else {
// must be char or string literal
label = generator.getTokenTypeAsTargetLabel(
grammar.getTokenType(t.getText()));
}
labels.add(label);
}
return labels;
}
public void init(Grammar g) {
this.grammar = g;
this.generator = grammar.getCodeGenerator();
this.templates = generator.getTemplates();
}
}
grammar[Grammar g,
StringTemplate recognizerST,
StringTemplate outputFileST,
StringTemplate headerFileST]
{
init(g);
this.recognizerST = recognizerST;
this.outputFileST = outputFileST;
this.headerFileST = headerFileST;
String superClass = (String)g.getOption("superClass");
outputOption = (String)g.getOption("output");
recognizerST.setAttribute("superClass", superClass);
if ( g.type!=Grammar.LEXER ) {
recognizerST.setAttribute("ASTLabelType", g.getOption("ASTLabelType"));
}
if ( g.type==Grammar.TREE_PARSER && g.getOption("ASTLabelType")==null ) {
ErrorManager.grammarWarning(ErrorManager.MSG_MISSING_AST_TYPE_IN_TREE_GRAMMAR,
g,
null,
g.name);
}
if ( g.type!=Grammar.TREE_PARSER ) {
recognizerST.setAttribute("labelType", g.getOption("TokenLabelType"));
}
recognizerST.setAttribute("numRules", grammar.getRules().size());
outputFileST.setAttribute("numRules", grammar.getRules().size());
headerFileST.setAttribute("numRules", grammar.getRules().size());
}
: ( #( LEXER_GRAMMAR grammarSpec )
| #( PARSER_GRAMMAR grammarSpec )
| #( TREE_GRAMMAR grammarSpec
)
| #( COMBINED_GRAMMAR grammarSpec )
)
;
attrScope
: #( "scope" ID ACTION )
;
grammarSpec
: name:ID
(cmt:DOC_COMMENT
{
outputFileST.setAttribute("docComment", #cmt.getText());
headerFileST.setAttribute("docComment", #cmt.getText());
}
)?
{
recognizerST.setAttribute("name", grammar.getRecognizerName());
outputFileST.setAttribute("name", grammar.getRecognizerName());
headerFileST.setAttribute("name", grammar.getRecognizerName());
recognizerST.setAttribute("scopes", grammar.getGlobalScopes());
headerFileST.setAttribute("scopes", grammar.getGlobalScopes());
}
( #(OPTIONS .) )?
( #(IMPORT .) )?
( #(TOKENS .) )?
(attrScope)*
(AMPERSAND)*
rules[recognizerST]
;
rules[StringTemplate recognizerST]
{
StringTemplate rST;
}
: ( ( {
String ruleName = _t.getFirstChild().getText();
Rule r = grammar.getRule(ruleName);
}
:
{grammar.generateMethodForRule(ruleName)}?
rST=rule
{
if ( rST!=null ) {
recognizerST.setAttribute("rules", rST);
outputFileST.setAttribute("rules", rST);
headerFileST.setAttribute("rules", rST);
}
}
| RULE
)
)+
;
rule returns [StringTemplate code=null]
{
String r;
String initAction = null;
StringTemplate b;
// get the dfa for the BLOCK
GrammarAST block=#rule.getFirstChildWithType(BLOCK);
DFA dfa=block.getLookaheadDFA();
// init blockNestingLevel so it's block level RULE_BLOCK_NESTING_LEVEL
// for alts of rule
blockNestingLevel = RULE_BLOCK_NESTING_LEVEL-1;
Rule ruleDescr = grammar.getRule(#rule.getFirstChild().getText());
// For syn preds, we don't want any AST code etc... in there.
// Save old templates ptr and restore later. Base templates include Dbg.
StringTemplateGroup saveGroup = templates;
if ( ruleDescr.isSynPred ) {
templates = generator.getBaseTemplates();
}
}
: #( RULE id:ID {r=#id.getText(); currentRuleName = r;}
(mod:modifier)?
#(ARG (ARG_ACTION)?)
#(RET (ARG_ACTION)?)
( #(OPTIONS .) )?
(ruleScopeSpec)?
(AMPERSAND)*
b=block["ruleBlock", dfa]
{
String description =
grammar.grammarTreeToString(#rule.getFirstChildWithType(BLOCK),
false);
description =
generator.target.getTargetStringLiteralFromString(description);
b.setAttribute("description", description);
// do not generate lexer rules in combined grammar
String stName = null;
if ( ruleDescr.isSynPred ) {
stName = "synpredRule";
}
else if ( grammar.type==Grammar.LEXER ) {
if ( r.equals(Grammar.ARTIFICIAL_TOKENS_RULENAME) ) {
stName = "tokensRule";
}
else {
stName = "lexerRule";
}
}
else {
if ( !(grammar.type==Grammar.COMBINED &&
Character.isUpperCase(r.charAt(0))) )
{
stName = "rule";
}
}
code = templates.getInstanceOf(stName);
if ( code.getName().equals("rule") ) {
code.setAttribute("emptyRule",
Boolean.valueOf(grammar.isEmptyRule(block)));
}
code.setAttribute("ruleDescriptor", ruleDescr);
String memo = (String)grammar.getBlockOption(#rule,"memoize");
if ( memo==null ) {
memo = (String)grammar.getOption("memoize");
}
if ( memo!=null && memo.equals("true") &&
(stName.equals("rule")||stName.equals("lexerRule")) )
{
code.setAttribute("memoize",
Boolean.valueOf(memo!=null && memo.equals("true")));
}
}
(exceptionGroup[code])?
EOR
)
{
if ( code!=null ) {
if ( grammar.type==Grammar.LEXER ) {
boolean naked =
r.equals(Grammar.ARTIFICIAL_TOKENS_RULENAME) ||
(mod!=null&&mod.getText().equals(Grammar.FRAGMENT_RULE_MODIFIER));
code.setAttribute("nakedBlock", Boolean.valueOf(naked));
}
else {
description =
grammar.grammarTreeToString(#rule,false);
description =
generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("description", description);
}
Rule theRule = grammar.getRule(r);
generator.translateActionAttributeReferencesForSingleScope(
theRule,
theRule.getActions()
);
code.setAttribute("ruleName", r);
code.setAttribute("block", b);
if ( initAction!=null ) {
code.setAttribute("initAction", initAction);
}
}
templates = saveGroup;
}
;
modifier
: "protected"
| "public"
| "private"
| "fragment"
;
ruleScopeSpec
: #( "scope" (ACTION)? ( ID )* )
;
block[String blockTemplateName, DFA dfa]
returns [StringTemplate code=null]
{
StringTemplate decision = null;
if ( dfa!=null ) {
code = templates.getInstanceOf(blockTemplateName);
decision = generator.genLookaheadDecision(recognizerST,dfa);
code.setAttribute("decision", decision);
code.setAttribute("decisionNumber", dfa.getDecisionNumber());
code.setAttribute("maxK",dfa.getMaxLookaheadDepth());
code.setAttribute("maxAlt",dfa.getNumberOfAlts());
}
else {
code = templates.getInstanceOf(blockTemplateName+"SingleAlt");
}
blockNestingLevel++;
code.setAttribute("blockLevel", blockNestingLevel);
code.setAttribute("enclosingBlockLevel", blockNestingLevel-1);
StringTemplate alt = null;
StringTemplate rew = null;
StringTemplate sb = null;
GrammarAST r = null;
int altNum = 1;
if ( this.blockNestingLevel==RULE_BLOCK_NESTING_LEVEL ) {
this.outerAltNum=1;
}
}
: {#block.getSetValue()!=null}? sb=setBlock
{
code.setAttribute("alts",sb);
blockNestingLevel--;
}
| #( BLOCK
( OPTIONS )? // ignore
( alt=alternative {r=(GrammarAST)_t;} rew=rewrite
{
if ( this.blockNestingLevel==RULE_BLOCK_NESTING_LEVEL ) {
this.outerAltNum++;
}
// add the rewrite code as just another element in the alt :)
// (unless it's a " -> ..." rewrite
// ( -> ... )
boolean etc =
r.getType()==REWRITE &&
r.getFirstChild()!=null &&
r.getFirstChild().getType()==ETC;
if ( rew!=null && !etc ) { alt.setAttribute("rew", rew); }
// add this alt to the list of alts for this block
code.setAttribute("alts",alt);
alt.setAttribute("altNum", Utils.integer(altNum));
alt.setAttribute("outerAlt",
Boolean.valueOf(blockNestingLevel==RULE_BLOCK_NESTING_LEVEL));
altNum++;
}
)+
EOB
)
{blockNestingLevel--;}
;
setBlock returns [StringTemplate code=null]
{
StringTemplate setcode = null;
if ( blockNestingLevel==RULE_BLOCK_NESTING_LEVEL && grammar.buildAST() ) {
Rule r = grammar.getRule(currentRuleName);
currentAltHasASTRewrite = r.hasRewrite(outerAltNum);
if ( currentAltHasASTRewrite ) {
r.trackTokenReferenceInAlt(#setBlock, outerAltNum);
}
}
}
: s:BLOCK
{
int i = ((TokenWithIndex)#s.getToken()).getIndex();
if ( blockNestingLevel==RULE_BLOCK_NESTING_LEVEL ) {
setcode = getTokenElementST("matchRuleBlockSet", "set", #s, null, null);
}
else {
setcode = getTokenElementST("matchSet", "set", #s, null, null);
}
setcode.setAttribute("elementIndex", i);
if ( grammar.type!=Grammar.LEXER ) {
generator.generateLocalFOLLOW(#s,"set",currentRuleName,i);
}
setcode.setAttribute("s",
generator.genSetExpr(templates,#s.getSetValue(),1,false));
StringTemplate altcode=templates.getInstanceOf("alt");
altcode.setAttribute("elements.{el,line,pos}",
setcode,
Utils.integer(#s.getLine()),
Utils.integer(#s.getColumn())
);
altcode.setAttribute("altNum", Utils.integer(1));
altcode.setAttribute("outerAlt",
Boolean.valueOf(blockNestingLevel==RULE_BLOCK_NESTING_LEVEL));
if ( !currentAltHasASTRewrite && grammar.buildAST() ) {
altcode.setAttribute("autoAST", Boolean.valueOf(true));
}
altcode.setAttribute("treeLevel", rewriteTreeNestingLevel);
code = altcode;
}
;
exceptionGroup[StringTemplate ruleST]
: ( exceptionHandler[ruleST] )+ (finallyClause[ruleST])?
| finallyClause[ruleST]
;
exceptionHandler[StringTemplate ruleST]
: #("catch" ARG_ACTION ACTION)
{
List chunks = generator.translateAction(currentRuleName,#ACTION);
ruleST.setAttribute("exceptions.{decl,action}",#ARG_ACTION.getText(),chunks);
}
;
finallyClause[StringTemplate ruleST]
: #("finally" ACTION)
{
List chunks = generator.translateAction(currentRuleName,#ACTION);
ruleST.setAttribute("finally",chunks);
}
;
alternative returns [StringTemplate code=templates.getInstanceOf("alt")]
{
/*
// TODO: can we use Rule.altsWithRewrites???
if ( blockNestingLevel==RULE_BLOCK_NESTING_LEVEL ) {
GrammarAST aRewriteNode = #alternative.findFirstType(REWRITE);
if ( grammar.buildAST() &&
(aRewriteNode!=null||
(#alternative.getNextSibling()!=null &&
#alternative.getNextSibling().getType()==REWRITE)) )
{
currentAltHasASTRewrite = true;
}
else {
currentAltHasASTRewrite = false;
}
}
*/
if ( blockNestingLevel==RULE_BLOCK_NESTING_LEVEL && grammar.buildAST() ) {
Rule r = grammar.getRule(currentRuleName);
currentAltHasASTRewrite = r.hasRewrite(outerAltNum);
}
String description = grammar.grammarTreeToString(#alternative, false);
description = generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("description", description);
code.setAttribute("treeLevel", rewriteTreeNestingLevel);
if ( !currentAltHasASTRewrite && grammar.buildAST() ) {
code.setAttribute("autoAST", Boolean.valueOf(true));
}
StringTemplate e;
}
: #( a:ALT
( {GrammarAST elAST=(GrammarAST)_t;}
e=element[null,null]
{
if ( e!=null ) {
code.setAttribute("elements.{el,line,pos}",
e,
Utils.integer(elAST.getLine()),
Utils.integer(elAST.getColumn())
);
}
}
)+
EOA
)
;
element[GrammarAST label, GrammarAST astSuffix] returns [StringTemplate code=null]
{
IntSet elements=null;
GrammarAST ast = null;
}
: #(ROOT code=element[label,#ROOT])
| #(BANG code=element[label,#BANG])
| #( n:NOT code=notElement[#n, label, astSuffix] )
| #( ASSIGN alabel:ID code=element[#alabel,astSuffix] )
| #( PLUS_ASSIGN label2:ID code=element[#label2,astSuffix] )
| #(CHAR_RANGE a:CHAR_LITERAL b:CHAR_LITERAL)
{code = templates.getInstanceOf("charRangeRef");
String low =
generator.target.getTargetCharLiteralFromANTLRCharLiteral(generator,a.getText());
String high =
generator.target.getTargetCharLiteralFromANTLRCharLiteral(generator,b.getText());
code.setAttribute("a", low);
code.setAttribute("b", high);
if ( label!=null ) {
code.setAttribute("label", label.getText());
}
}
| {#element.getSetValue()==null}? code=ebnf
| code=atom[null, label, astSuffix]
| code=tree
| code=element_action
| (sp:SEMPRED|gsp:GATED_SEMPRED {#sp=#gsp;})
{
code = templates.getInstanceOf("validateSemanticPredicate");
code.setAttribute("pred", generator.translateAction(currentRuleName,#sp));
String description =
generator.target.getTargetStringLiteralFromString(#sp.getText());
code.setAttribute("description", description);
}
| SYN_SEMPRED // used only in lookahead; don't generate validating pred
| BACKTRACK_SEMPRED
| EPSILON
;
element_action returns [StringTemplate code=null]
: act:ACTION
{
code = templates.getInstanceOf("execAction");
code.setAttribute("action", generator.translateAction(currentRuleName,#act));
}
| act2:FORCED_ACTION
{
code = templates.getInstanceOf("execForcedAction");
code.setAttribute("action", generator.translateAction(currentRuleName,#act2));
}
;
notElement[GrammarAST n, GrammarAST label, GrammarAST astSuffix]
returns [StringTemplate code=null]
{
IntSet elements=null;
String labelText = null;
if ( label!=null ) {
labelText = label.getText();
}
}
: (assign_c:CHAR_LITERAL
{
int ttype=0;
if ( grammar.type==Grammar.LEXER ) {
ttype = Grammar.getCharValueFromGrammarCharLiteral(assign_c.getText());
}
else {
ttype = grammar.getTokenType(assign_c.getText());
}
elements = grammar.complement(ttype);
}
| assign_s:STRING_LITERAL
{
int ttype=0;
if ( grammar.type==Grammar.LEXER ) {
// TODO: error!
}
else {
ttype = grammar.getTokenType(assign_s.getText());
}
elements = grammar.complement(ttype);
}
| assign_t:TOKEN_REF
{
int ttype = grammar.getTokenType(assign_t.getText());
elements = grammar.complement(ttype);
}
| assign_st:BLOCK
{
elements = assign_st.getSetValue();
elements = grammar.complement(elements);
}
)
{
code = getTokenElementST("matchSet",
"set",
(GrammarAST)n.getFirstChild(),
astSuffix,
labelText);
code.setAttribute("s",generator.genSetExpr(templates,elements,1,false));
int i = ((TokenWithIndex)n.getToken()).getIndex();
code.setAttribute("elementIndex", i);
if ( grammar.type!=Grammar.LEXER ) {
generator.generateLocalFOLLOW(n,"set",currentRuleName,i);
}
}
;
ebnf returns [StringTemplate code=null]
{
DFA dfa=null;
GrammarAST b = (GrammarAST)#ebnf.getFirstChild();
GrammarAST eob = (GrammarAST)#b.getLastChild(); // loops will use EOB DFA
}
: ( {dfa = #ebnf.getLookaheadDFA();}
code=block["block", dfa]
| {dfa = #ebnf.getLookaheadDFA();}
#( OPTIONAL code=block["optionalBlock", dfa] )
| {dfa = #eob.getLookaheadDFA();}
#( CLOSURE code=block["closureBlock", dfa] )
| {dfa = #eob.getLookaheadDFA();}
#( POSITIVE_CLOSURE code=block["positiveClosureBlock", dfa] )
)
{
String description = grammar.grammarTreeToString(#ebnf, false);
description = generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("description", description);
}
;
tree returns [StringTemplate code=templates.getInstanceOf("tree")]
{
StringTemplate el=null, act=null;
GrammarAST elAST=null, actAST=null;
NFAState afterDOWN = (NFAState)tree_AST_in.NFATreeDownState.transition(0).target;
LookaheadSet s = grammar.LOOK(afterDOWN);
if ( s.member(Label.UP) ) {
// nullable child list if we can see the UP as the next token
// we need an "if ( input.LA(1)==Token.DOWN )" gate around
// the child list.
code.setAttribute("nullableChildList", "true");
}
rewriteTreeNestingLevel++;
code.setAttribute("enclosingTreeLevel", rewriteTreeNestingLevel-1);
code.setAttribute("treeLevel", rewriteTreeNestingLevel);
Rule r = grammar.getRule(currentRuleName);
GrammarAST rootSuffix = null;
if ( grammar.buildAST() && !r.hasRewrite(outerAltNum) ) {
rootSuffix = new GrammarAST(ROOT,"ROOT");
}
}
: #( TREE_BEGIN {elAST=(GrammarAST)_t;}
el=element[null,rootSuffix]
{
code.setAttribute("root.{el,line,pos}",
el,
Utils.integer(elAST.getLine()),
Utils.integer(elAST.getColumn())
);
}
// push all the immediately-following actions out before children
// so actions aren't guarded by the "if (input.LA(1)==Token.DOWN)"
// guard in generated code.
( options {greedy=true;}:
{actAST=(GrammarAST)_t;}
act=element_action
{
code.setAttribute("actionsAfterRoot.{el,line,pos}",
act,
Utils.integer(actAST.getLine()),
Utils.integer(actAST.getColumn())
);
}
)*
( {elAST=(GrammarAST)_t;}
el=element[null,null]
{
code.setAttribute("children.{el,line,pos}",
el,
Utils.integer(elAST.getLine()),
Utils.integer(elAST.getColumn())
);
}
)*
)
{rewriteTreeNestingLevel--;}
;
atom[GrammarAST scope, GrammarAST label, GrammarAST astSuffix]
returns [StringTemplate code=null]
{
String labelText=null;
if ( label!=null ) {
labelText = label.getText();
}
if ( grammar.type!=Grammar.LEXER &&
(#atom.getType()==RULE_REF||#atom.getType()==TOKEN_REF||
#atom.getType()==CHAR_LITERAL||#atom.getType()==STRING_LITERAL) )
{
Rule encRule = grammar.getRule(((GrammarAST)#atom).enclosingRuleName);
if ( encRule!=null && encRule.hasRewrite(outerAltNum) && astSuffix!=null ) {
ErrorManager.grammarError(ErrorManager.MSG_AST_OP_IN_ALT_WITH_REWRITE,
grammar,
((GrammarAST)#atom).getToken(),
((GrammarAST)#atom).enclosingRuleName,
new Integer(outerAltNum));
astSuffix = null;
}
}
}
: #( r:RULE_REF (rarg:ARG_ACTION)? )
{
grammar.checkRuleReference(scope, #r, #rarg, currentRuleName);
String scopeName = null;
if ( scope!=null ) {
scopeName = scope.getText();
}
Rule rdef = grammar.getRule(scopeName, #r.getText());
// don't insert label=r() if $label.attr not used, no ret value, ...
if ( !rdef.getHasReturnValue() ) {
labelText = null;
}
code = getRuleElementST("ruleRef", #r.getText(), #r, astSuffix, labelText);
code.setAttribute("rule", rdef);
if ( scope!=null ) { // scoped rule ref
Grammar scopeG = grammar.composite.getGrammar(scope.getText());
code.setAttribute("scope", scopeG);
}
else if ( rdef.grammar != this.grammar ) { // nonlocal
// if rule definition is not in this grammar, it's nonlocal
List<Grammar> rdefDelegates = rdef.grammar.getDelegates();
if ( rdefDelegates.contains(this.grammar) ) {
code.setAttribute("scope", rdef.grammar);
}
else {
// defining grammar is not a delegate, scope all the
// back to root, which has delegate methods for all
// rules. Don't use scope if we are root.
if ( this.grammar != rdef.grammar.composite.delegateGrammarTreeRoot.grammar ) {
code.setAttribute("scope",
rdef.grammar.composite.delegateGrammarTreeRoot.grammar);
}
}
}
if ( #rarg!=null ) {
List args = generator.translateAction(currentRuleName,#rarg);
code.setAttribute("args", args);
}
int i = ((TokenWithIndex)r.getToken()).getIndex();
code.setAttribute("elementIndex", i);
generator.generateLocalFOLLOW(#r,#r.getText(),currentRuleName,i);
#r.code = code;
}
| #( t:TOKEN_REF (targ:ARG_ACTION)? )
{
if ( currentAltHasASTRewrite && #t.terminalOptions!=null &&
#t.terminalOptions.get(Grammar.defaultTokenOption)!=null ) {
ErrorManager.grammarError(ErrorManager.MSG_HETERO_ILLEGAL_IN_REWRITE_ALT,
grammar,
((GrammarAST)(#t)).getToken(),
#t.getText());
}
grammar.checkRuleReference(scope, #t, #targ, currentRuleName);
if ( grammar.type==Grammar.LEXER ) {
if ( grammar.getTokenType(t.getText())==Label.EOF ) {
code = templates.getInstanceOf("lexerMatchEOF");
}
else {
code = templates.getInstanceOf("lexerRuleRef");
if ( isListLabel(labelText) ) {
code = templates.getInstanceOf("lexerRuleRefAndListLabel");
}
String scopeName = null;
if ( scope!=null ) {
scopeName = scope.getText();
}
Rule rdef2 = grammar.getRule(scopeName, #t.getText());
code.setAttribute("rule", rdef2);
if ( scope!=null ) { // scoped rule ref
Grammar scopeG = grammar.composite.getGrammar(scope.getText());
code.setAttribute("scope", scopeG);
}
else if ( rdef2.grammar != this.grammar ) { // nonlocal
// if rule definition is not in this grammar, it's nonlocal
code.setAttribute("scope", rdef2.grammar);
}
if ( #targ!=null ) {
List args = generator.translateAction(currentRuleName,#targ);
code.setAttribute("args", args);
}
}
int i = ((TokenWithIndex)#t.getToken()).getIndex();
code.setAttribute("elementIndex", i);
if ( label!=null ) code.setAttribute("label", labelText);
}
else {
code = getTokenElementST("tokenRef", #t.getText(), #t, astSuffix, labelText);
String tokenLabel =
generator.getTokenTypeAsTargetLabel(grammar.getTokenType(t.getText()));
code.setAttribute("token",tokenLabel);
if ( !currentAltHasASTRewrite && #t.terminalOptions!=null ) {
code.setAttribute("hetero",#t.terminalOptions.get(Grammar.defaultTokenOption));
}
int i = ((TokenWithIndex)#t.getToken()).getIndex();
code.setAttribute("elementIndex", i);
generator.generateLocalFOLLOW(#t,tokenLabel,currentRuleName,i);
}
#t.code = code;
}
| c:CHAR_LITERAL
{
if ( grammar.type==Grammar.LEXER ) {
code = templates.getInstanceOf("charRef");
code.setAttribute("char",
generator.target.getTargetCharLiteralFromANTLRCharLiteral(generator,c.getText()));
if ( label!=null ) {
code.setAttribute("label", labelText);
}
}
else { // else it's a token type reference
code = getTokenElementST("tokenRef", "char_literal", #c, astSuffix, labelText);
String tokenLabel = generator.getTokenTypeAsTargetLabel(grammar.getTokenType(c.getText()));
code.setAttribute("token",tokenLabel);
if ( #c.terminalOptions!=null ) {
code.setAttribute("hetero",#c.terminalOptions.get(Grammar.defaultTokenOption));
}
int i = ((TokenWithIndex)#c.getToken()).getIndex();
code.setAttribute("elementIndex", i);
generator.generateLocalFOLLOW(#c,tokenLabel,currentRuleName,i);
}
}
| s:STRING_LITERAL
{
if ( grammar.type==Grammar.LEXER ) {
code = templates.getInstanceOf("lexerStringRef");
code.setAttribute("string",
generator.target.getTargetStringLiteralFromANTLRStringLiteral(generator,s.getText()));
if ( label!=null ) {
code.setAttribute("label", labelText);
}
}
else { // else it's a token type reference
code = getTokenElementST("tokenRef", "string_literal", #s, astSuffix, labelText);
String tokenLabel =
generator.getTokenTypeAsTargetLabel(grammar.getTokenType(#s.getText()));
code.setAttribute("token",tokenLabel);
if ( #s.terminalOptions!=null ) {
code.setAttribute("hetero",#s.terminalOptions.get(Grammar.defaultTokenOption));
}
int i = ((TokenWithIndex)#s.getToken()).getIndex();
code.setAttribute("elementIndex", i);
generator.generateLocalFOLLOW(#s,tokenLabel,currentRuleName,i);
}
}
| w:WILDCARD
{
code = getWildcardST(#w,astSuffix,labelText);
code.setAttribute("elementIndex", ((TokenWithIndex)#w.getToken()).getIndex());
}
| #(DOT ID code=atom[#ID, label, astSuffix]) // scope override on rule or token
| code=set[label,astSuffix]
;
ast_suffix
: ROOT
| BANG
;
set[GrammarAST label, GrammarAST astSuffix] returns [StringTemplate code=null]
{
String labelText=null;
if ( label!=null ) {
labelText = label.getText();
}
}
: s:BLOCK // only care that it's a BLOCK with setValue!=null
{
code = getTokenElementST("matchSet", "set", #s, astSuffix, labelText);
int i = ((TokenWithIndex)#s.getToken()).getIndex();
code.setAttribute("elementIndex", i);
if ( grammar.type!=Grammar.LEXER ) {
generator.generateLocalFOLLOW(#s,"set",currentRuleName,i);
}
code.setAttribute("s", generator.genSetExpr(templates,#s.getSetValue(),1,false));
}
;
setElement
: c:CHAR_LITERAL
| t:TOKEN_REF
| s:STRING_LITERAL
| #(CHAR_RANGE c1:CHAR_LITERAL c2:CHAR_LITERAL)
;
// REWRITE stuff
rewrite returns [StringTemplate code=null]
{
StringTemplate alt;
if ( #rewrite.getType()==REWRITE ) {
if ( generator.grammar.buildTemplate() ) {
code = templates.getInstanceOf("rewriteTemplate");
}
else {
code = templates.getInstanceOf("rewriteCode");
code.setAttribute("treeLevel", Utils.integer(OUTER_REWRITE_NESTING_LEVEL));
code.setAttribute("rewriteBlockLevel", Utils.integer(OUTER_REWRITE_NESTING_LEVEL));
code.setAttribute("referencedElementsDeep",
getTokenTypesAsTargetLabels(#rewrite.rewriteRefsDeep));
Set<String> tokenLabels =
grammar.getLabels(#rewrite.rewriteRefsDeep, Grammar.TOKEN_LABEL);
Set<String> tokenListLabels =
grammar.getLabels(#rewrite.rewriteRefsDeep, Grammar.TOKEN_LIST_LABEL);
Set<String> ruleLabels =
grammar.getLabels(#rewrite.rewriteRefsDeep, Grammar.RULE_LABEL);
Set<String> ruleListLabels =
grammar.getLabels(#rewrite.rewriteRefsDeep, Grammar.RULE_LIST_LABEL);
Set<String> wildcardLabels =
grammar.getLabels(#rewrite.rewriteRefsDeep, Grammar.WILDCARD_TREE_LABEL);
Set<String> wildcardListLabels =
grammar.getLabels(#rewrite.rewriteRefsDeep, Grammar.WILDCARD_TREE_LIST_LABEL);
// just in case they ref $r for "previous value", make a stream
// from retval.tree
StringTemplate retvalST = templates.getInstanceOf("prevRuleRootRef");
ruleLabels.add(retvalST.toString());
code.setAttribute("referencedTokenLabels", tokenLabels);
code.setAttribute("referencedTokenListLabels", tokenListLabels);
code.setAttribute("referencedRuleLabels", ruleLabels);
code.setAttribute("referencedRuleListLabels", ruleListLabels);
code.setAttribute("referencedWildcardLabels", wildcardLabels);
code.setAttribute("referencedWildcardListLabels", wildcardListLabels);
}
}
else {
code = templates.getInstanceOf("noRewrite");
code.setAttribute("treeLevel", Utils.integer(OUTER_REWRITE_NESTING_LEVEL));
code.setAttribute("rewriteBlockLevel", Utils.integer(OUTER_REWRITE_NESTING_LEVEL));
}
}
: (
{rewriteRuleRefs = new HashSet();}
#( r:REWRITE (pred:SEMPRED)? alt=rewrite_alternative )
{
rewriteBlockNestingLevel = OUTER_REWRITE_NESTING_LEVEL;
List predChunks = null;
if ( #pred!=null ) {
//predText = #pred.getText();
predChunks = generator.translateAction(currentRuleName,#pred);
}
String description =
grammar.grammarTreeToString(#r,false);
description = generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("alts.{pred,alt,description}",
predChunks,
alt,
description);
pred=null;
}
)*
;
rewrite_block[String blockTemplateName] returns [StringTemplate code=null]
{
rewriteBlockNestingLevel++;
code = templates.getInstanceOf(blockTemplateName);
StringTemplate save_currentBlockST = currentBlockST;
currentBlockST = code;
code.setAttribute("rewriteBlockLevel", rewriteBlockNestingLevel);
StringTemplate alt=null;
}
: #( BLOCK
{
currentBlockST.setAttribute("referencedElementsDeep",
getTokenTypesAsTargetLabels(#BLOCK.rewriteRefsDeep));
currentBlockST.setAttribute("referencedElements",
getTokenTypesAsTargetLabels(#BLOCK.rewriteRefsShallow));
}
alt=rewrite_alternative
EOB
)
{
code.setAttribute("alt", alt);
rewriteBlockNestingLevel--;
currentBlockST = save_currentBlockST;
}
;
rewrite_alternative
returns [StringTemplate code=null]
{
StringTemplate el,st;
}
: {generator.grammar.buildAST()}?
#( a:ALT {code=templates.getInstanceOf("rewriteElementList");}
( ( {GrammarAST elAST=(GrammarAST)_t;}
el=rewrite_element
{code.setAttribute("elements.{el,line,pos}",
el,
Utils.integer(elAST.getLine()),
Utils.integer(elAST.getColumn())
);
}
)+
| EPSILON
{code.setAttribute("elements.{el,line,pos}",
templates.getInstanceOf("rewriteEmptyAlt"),
Utils.integer(#a.getLine()),
Utils.integer(#a.getColumn())
);
}
)
EOA
)
| {generator.grammar.buildTemplate()}? code=rewrite_template
| // reproduce same input (only AST at moment)
ETC
;
rewrite_element returns [StringTemplate code=null]
{
IntSet elements=null;
GrammarAST ast = null;
}
: code=rewrite_atom[false]
| code=rewrite_ebnf
| code=rewrite_tree
;
rewrite_ebnf returns [StringTemplate code=null]
: #( OPTIONAL code=rewrite_block["rewriteOptionalBlock"] )
{
String description = grammar.grammarTreeToString(#rewrite_ebnf, false);
description = generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("description", description);
}
| #( CLOSURE code=rewrite_block["rewriteClosureBlock"] )
{
String description = grammar.grammarTreeToString(#rewrite_ebnf, false);
description = generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("description", description);
}
| #( POSITIVE_CLOSURE code=rewrite_block["rewritePositiveClosureBlock"] )
{
String description = grammar.grammarTreeToString(#rewrite_ebnf, false);
description = generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("description", description);
}
;
rewrite_tree returns [StringTemplate code=templates.getInstanceOf("rewriteTree")]
{
rewriteTreeNestingLevel++;
code.setAttribute("treeLevel", rewriteTreeNestingLevel);
code.setAttribute("enclosingTreeLevel", rewriteTreeNestingLevel-1);
StringTemplate r, el;
GrammarAST elAST=null;
}
: #( TREE_BEGIN {elAST=(GrammarAST)_t;}
r=rewrite_atom[true]
{code.setAttribute("root.{el,line,pos}",
r,
Utils.integer(elAST.getLine()),
Utils.integer(elAST.getColumn())
);
}
( {elAST=(GrammarAST)_t;}
el=rewrite_element
{
code.setAttribute("children.{el,line,pos}",
el,
Utils.integer(elAST.getLine()),
Utils.integer(elAST.getColumn())
);
}
)*
)
{
String description = grammar.grammarTreeToString(#rewrite_tree, false);
description = generator.target.getTargetStringLiteralFromString(description);
code.setAttribute("description", description);
rewriteTreeNestingLevel--;
}
;
rewrite_atom[boolean isRoot] returns [StringTemplate code=null]
: r:RULE_REF
{
String ruleRefName = #r.getText();
String stName = "rewriteRuleRef";
if ( isRoot ) {
stName += "Root";
}
code = templates.getInstanceOf(stName);
code.setAttribute("rule", ruleRefName);
if ( grammar.getRule(ruleRefName)==null ) {
ErrorManager.grammarError(ErrorManager.MSG_UNDEFINED_RULE_REF,
grammar,
((GrammarAST)(#r)).getToken(),
ruleRefName);
code = new StringTemplate(); // blank; no code gen
}
else if ( grammar.getRule(currentRuleName)
.getRuleRefsInAlt(ruleRefName,outerAltNum)==null )
{
ErrorManager.grammarError(ErrorManager.MSG_REWRITE_ELEMENT_NOT_PRESENT_ON_LHS,
grammar,
((GrammarAST)(#r)).getToken(),
ruleRefName);
code = new StringTemplate(); // blank; no code gen
}
else {
// track all rule refs as we must copy 2nd ref to rule and beyond
if ( !rewriteRuleRefs.contains(ruleRefName) ) {
rewriteRuleRefs.add(ruleRefName);
}
}
}
| {GrammarAST term=(GrammarAST)_t;}
( #(tk:TOKEN_REF (arg:ARG_ACTION)?)
| cl:CHAR_LITERAL
| sl:STRING_LITERAL
)
{
String tokenName = #rewrite_atom.getText();
String stName = "rewriteTokenRef";
Rule rule = grammar.getRule(currentRuleName);
Set tokenRefsInAlt = rule.getTokenRefsInAlt(outerAltNum);
boolean createNewNode = !tokenRefsInAlt.contains(tokenName) || #arg!=null;
Object hetero = null;
if ( term.terminalOptions!=null ) {
hetero = term.terminalOptions.get(Grammar.defaultTokenOption);
}
if ( createNewNode ) {
stName = "rewriteImaginaryTokenRef";
}
if ( isRoot ) {
stName += "Root";
}
code = templates.getInstanceOf(stName);
code.setAttribute("hetero", hetero);
if ( #arg!=null ) {
List args = generator.translateAction(currentRuleName,#arg);
code.setAttribute("args", args);
}
code.setAttribute("elementIndex", ((TokenWithIndex)#rewrite_atom.getToken()).getIndex());
int ttype = grammar.getTokenType(tokenName);
String tok = generator.getTokenTypeAsTargetLabel(ttype);
code.setAttribute("token", tok);
if ( grammar.getTokenType(tokenName)==Label.INVALID ) {
ErrorManager.grammarError(ErrorManager.MSG_UNDEFINED_TOKEN_REF_IN_REWRITE,
grammar,
((GrammarAST)(#rewrite_atom)).getToken(),
tokenName);
code = new StringTemplate(); // blank; no code gen
}
}
| LABEL
{
String labelName = #LABEL.getText();
Rule rule = grammar.getRule(currentRuleName);
Grammar.LabelElementPair pair = rule.getLabel(labelName);
if ( labelName.equals(currentRuleName) ) {
// special case; ref to old value via $rule
if ( rule.hasRewrite(outerAltNum) &&
rule.getRuleRefsInAlt(outerAltNum).contains(labelName) )
{
ErrorManager.grammarError(ErrorManager.MSG_RULE_REF_AMBIG_WITH_RULE_IN_ALT,
grammar,
((GrammarAST)(#LABEL)).getToken(),
labelName);
}
StringTemplate labelST = templates.getInstanceOf("prevRuleRootRef");
code = templates.getInstanceOf("rewriteRuleLabelRef"+(isRoot?"Root":""));
code.setAttribute("label", labelST);
}
else if ( pair==null ) {
ErrorManager.grammarError(ErrorManager.MSG_UNDEFINED_LABEL_REF_IN_REWRITE,
grammar,
((GrammarAST)(#LABEL)).getToken(),
labelName);
code = new StringTemplate();
}
else {
String stName = null;
switch ( pair.type ) {
case Grammar.TOKEN_LABEL :
stName = "rewriteTokenLabelRef";
break;
case Grammar.WILDCARD_TREE_LABEL :
stName = "rewriteWildcardLabelRef";
break;
case Grammar.WILDCARD_TREE_LIST_LABEL :
stName = "rewriteRuleListLabelRef"; // acts like rule ref list for ref
break;
case Grammar.RULE_LABEL :
stName = "rewriteRuleLabelRef";
break;
case Grammar.TOKEN_LIST_LABEL :
stName = "rewriteTokenListLabelRef";
break;
case Grammar.RULE_LIST_LABEL :
stName = "rewriteRuleListLabelRef";
break;
}
if ( isRoot ) {
stName += "Root";
}
code = templates.getInstanceOf(stName);
code.setAttribute("label", labelName);
}
}
| ACTION
{
// actions in rewrite rules yield a tree object
String actText = #ACTION.getText();
List chunks = generator.translateAction(currentRuleName,#ACTION);
code = templates.getInstanceOf("rewriteNodeAction"+(isRoot?"Root":""));
code.setAttribute("action", chunks);
}
;
rewrite_template returns [StringTemplate code=null]
: #( ALT EPSILON EOA ) {code=templates.getInstanceOf("rewriteEmptyTemplate");}
| #( TEMPLATE (id:ID|ind:ACTION)
{
if ( #id!=null && #id.getText().equals("template") ) {
code = templates.getInstanceOf("rewriteInlineTemplate");
}
else if ( #id!=null ) {
code = templates.getInstanceOf("rewriteExternalTemplate");
code.setAttribute("name", #id.getText());
}
else if ( #ind!=null ) { // must be %({expr})(args)
code = templates.getInstanceOf("rewriteIndirectTemplate");
List chunks=generator.translateAction(currentRuleName,#ind);
code.setAttribute("expr", chunks);
}
}
#( ARGLIST
( #( ARG arg:ID a:ACTION
{
// must set alt num here rather than in define.g
// because actions like %foo(name={$ID.text}) aren't
// broken up yet into trees.
#a.outerAltNum = this.outerAltNum;
List chunks = generator.translateAction(currentRuleName,#a);
code.setAttribute("args.{name,value}", #arg.getText(), chunks);
}
)
)*
)
( DOUBLE_QUOTE_STRING_LITERAL
{
String sl = #DOUBLE_QUOTE_STRING_LITERAL.getText();
String t = sl.substring(1,sl.length()-1); // strip quotes
t = generator.target.getTargetStringLiteralFromString(t);
code.setAttribute("template",t);
}
| DOUBLE_ANGLE_STRING_LITERAL
{
String sl = #DOUBLE_ANGLE_STRING_LITERAL.getText();
String t = sl.substring(2,sl.length()-2); // strip double angle quotes
t = generator.target.getTargetStringLiteralFromString(t);
code.setAttribute("template",t);
}
)?
)
| act:ACTION
{
// set alt num for same reason as ARGLIST above
#act.outerAltNum = this.outerAltNum;
code=templates.getInstanceOf("rewriteAction");
code.setAttribute("action",
generator.translateAction(currentRuleName,#act));
}
;
antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/v2/antlr.g 0000644 0001750 0001750 00000071123 11256465226 023517 0 ustar twerner twerner header {
/*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.grammar.v2;
import java.util.*;
import java.io.*;
import org.antlr.analysis.*;
import org.antlr.misc.*;
import org.antlr.tool.*;
import antlr.TokenBuffer;
import antlr.TokenStreamException;
import antlr.Token;
import antlr.TokenStream;
import antlr.RecognitionException;
import antlr.NoViableAltException;
import antlr.ParserSharedInputState;
import antlr.collections.impl.BitSet;
import antlr.collections.AST;
import antlr.ASTFactory;
import antlr.ASTPair;
import antlr.TokenWithIndex;
import antlr.collections.impl.ASTArray;
}
/** Read in an ANTLR grammar and build an AST. Try not to do
* any actions, just build the tree.
*
* The phases are:
*
* antlr.g (this file)
* assign.types.g
* define.g
* buildnfa.g
* antlr.print.g (optional)
* codegen.g
*
* Terence Parr
* University of San Francisco
* 2005
*/
class ANTLRParser extends Parser;
options {
buildAST = true;
exportVocab=ANTLR;
ASTLabelType="GrammarAST";
k=3;
}
tokens {
OPTIONS="options";
TOKENS="tokens";
PARSER="parser";
LEXER;
RULE;
BLOCK;
OPTIONAL;
CLOSURE;
POSITIVE_CLOSURE;
SYNPRED;
RANGE;
CHAR_RANGE;
EPSILON;
ALT;
EOR;
EOB;
EOA; // end of alt
ID;
ARG;
ARGLIST;
RET;
LEXER_GRAMMAR;
PARSER_GRAMMAR;
TREE_GRAMMAR;
COMBINED_GRAMMAR;
INITACTION;
FORCED_ACTION; // {{...}} always exec even during syn preds
LABEL; // $x used in rewrite rules
TEMPLATE;
SCOPE="scope";
IMPORT="import";
GATED_SEMPRED; // {p}? =>
SYN_SEMPRED; // (...) => it's a manually-specified synpred converted to sempred
BACKTRACK_SEMPRED; // auto backtracking mode syn pred converted to sempred
FRAGMENT="fragment";
DOT;
}
{
protected Grammar grammar = null;
protected int gtype = 0;
public Grammar getGrammar() {
return grammar;
}
public void setGrammar(Grammar grammar) {
this.grammar = grammar;
}
public int getGtype() {
return gtype;
}
public void setGtype(int gtype) {
this.gtype = gtype;
}
protected String currentRuleName = null;
protected GrammarAST currentBlockAST = null;
protected boolean atTreeRoot; // are we matching a tree root in tree grammar?
protected GrammarAST setToBlockWithSet(GrammarAST b) {
GrammarAST alt = #(#[ALT,"ALT"],#b,#[EOA,"<end-of-alt>"]);
prefixWithSynPred(alt);
return #(#[BLOCK,"BLOCK"],
alt,
#[EOB,"<end-of-block>"]
);
}
/** Create a copy of the alt and make it into a BLOCK; all actions,
* labels, tree operators, rewrites are removed.
*/
protected GrammarAST createBlockFromDupAlt(GrammarAST alt) {
GrammarAST nalt = GrammarAST.dupTreeNoActions(alt, null);
GrammarAST blk = #(#[BLOCK,"BLOCK"],
nalt,
#[EOB,"<end-of-block>"]
);
return blk;
}
/** Rewrite alt to have a synpred as first element;
* (xxx)=>xxx
* but only if they didn't specify one manually.
*/
protected void prefixWithSynPred(GrammarAST alt) {
// if they want backtracking and it's not a lexer rule in combined grammar
String autoBacktrack = (String)grammar.getBlockOption(currentBlockAST, "backtrack");
if ( autoBacktrack==null ) {
autoBacktrack = (String)grammar.getOption("backtrack");
}
if ( autoBacktrack!=null&&autoBacktrack.equals("true") &&
!(gtype==COMBINED_GRAMMAR &&
Character.isUpperCase(currentRuleName.charAt(0))) &&
alt.getFirstChild().getType()!=SYN_SEMPRED )
{
// duplicate alt and make a synpred block around that dup'd alt
GrammarAST synpredBlockAST = createBlockFromDupAlt(alt);
// Create a BACKTRACK_SEMPRED node as if user had typed this in
// Effectively we replace (xxx)=>xxx with {synpredxxx}? xxx
GrammarAST synpredAST = createSynSemPredFromBlock(synpredBlockAST,
BACKTRACK_SEMPRED);
// insert BACKTRACK_SEMPRED as first element of alt
synpredAST.getLastSibling().setNextSibling(alt.getFirstChild());
alt.setFirstChild(synpredAST);
}
}
protected GrammarAST createSynSemPredFromBlock(GrammarAST synpredBlockAST,
int synpredTokenType)
{
// add grammar fragment to a list so we can make fake rules for them
// later.
String predName = grammar.defineSyntacticPredicate(synpredBlockAST,currentRuleName);
// convert (alpha)=> into {synpredN}? where N is some pred count
// during code gen we convert to function call with templates
String synpredinvoke = predName;
GrammarAST p = #[synpredTokenType,synpredinvoke];
// track how many decisions have synpreds
grammar.blocksWithSynPreds.add(currentBlockAST);
return p;
}
public GrammarAST createSimpleRuleAST(String name,
GrammarAST block,
boolean fragment)
{
GrammarAST modifier = null;
if ( fragment ) {
modifier = #[FRAGMENT,"fragment"];
}
GrammarAST EORAST = #[EOR,"<end-of-rule>"];
GrammarAST EOBAST = block.getLastChild();
EORAST.setLine(EOBAST.getLine());
EORAST.setColumn(EOBAST.getColumn());
GrammarAST ruleAST =
#([RULE,"rule"],
[ID,name],modifier,[ARG,"ARG"],[RET,"RET"],
[SCOPE,"scope"],block,EORAST);
ruleAST.setLine(block.getLine());
ruleAST.setColumn(block.getColumn());
return ruleAST;
}
public void reportError(RecognitionException ex) {
Token token = null;
try {
token = LT(1);
}
catch (TokenStreamException tse) {
ErrorManager.internalError("can't get token???", tse);
}
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
grammar,
token,
"antlr: "+ex.toString(),
ex);
}
public void cleanup(GrammarAST root) {
if ( gtype==LEXER_GRAMMAR ) {
String filter = (String)grammar.getOption("filter");
GrammarAST tokensRuleAST =
grammar.addArtificialMatchTokensRule(
root,
grammar.lexerRuleNamesInCombined,
grammar.getDelegateNames(),
filter!=null&&filter.equals("true"));
}
}
}
grammar![Grammar g]
{
this.grammar = g;
GrammarAST opt=null;
Token optionsStartToken = null;
Map opts;
// set to factory that sets enclosing rule
astFactory = new ASTFactory() {
{
setASTNodeClass(GrammarAST.class);
setASTNodeClass("org.antlr.tool.GrammarAST");
}
public AST create(Token token) {
AST t = super.create(token);
((GrammarAST)t).enclosingRuleName = currentRuleName;
return t;
}
public AST create(int i) {
AST t = super.create(i);
((GrammarAST)t).enclosingRuleName = currentRuleName;
return t;
}
};
}
: //hdr:headerSpec
( ACTION )?
( cmt:DOC_COMMENT )?
gr:grammarType gid:id {grammar.setName(#gid.getText());} SEMI
( {optionsStartToken=LT(1);}
opts=optionsSpec {grammar.setOptions(opts, optionsStartToken);}
{opt=(GrammarAST)returnAST;}
)?
(ig:delegateGrammars)?
(ts:tokensSpec!)?
scopes:attrScopes
(a:actions)?
r:rules
EOF
{
#grammar = #(null, #(#gr, #gid, #cmt, opt, #ig, #ts, #scopes, #a, #r));
cleanup(#grammar);
}
;
grammarType
: ( "lexer"! {gtype=LEXER_GRAMMAR; grammar.type = Grammar.LEXER;} // pure lexer
| "parser"! {gtype=PARSER_GRAMMAR; grammar.type = Grammar.PARSER;} // pure parser
| "tree"! {gtype=TREE_GRAMMAR; grammar.type = Grammar.TREE_PARSER;} // a tree parser
| {gtype=COMBINED_GRAMMAR; grammar.type = Grammar.COMBINED;} // merged parser/lexer
)
gr:"grammar" {#gr.setType(gtype);}
;
actions
: (action)+
;
/** Match stuff like @parser::members {int i;} */
action
: AMPERSAND^ (actionScopeName COLON! COLON!)? id ACTION
;
/** Sometimes the scope names will collide with keywords; allow them as
* ids for action scopes.
*/
actionScopeName
: id
| l:"lexer" {#l.setType(ID);}
| p:"parser" {#p.setType(ID);}
;
optionsSpec returns [Map opts=new HashMap()]
: OPTIONS^ (option[opts] SEMI!)+ RCURLY!
;
option[Map opts]
{
Object value=null;
}
: o:id ASSIGN^ value=optionValue
{
opts.put(#o.getText(), value);
}
;
optionValue returns [Object value=null]
: x:id {value = #x.getText();}
| s:STRING_LITERAL {String vs = #s.getText();
value=vs.substring(1,vs.length()-1);}
| c:CHAR_LITERAL {String vs = #c.getText();
value=vs.substring(1,vs.length()-1);}
| i:INT {value = new Integer(#i.getText());}
| ss:STAR {#ss.setType(STRING_LITERAL); value = "*";} // used for k=*
// | cs:charSet {value = #cs;} // return set AST in this case
;
delegateGrammars
: "import"^ delegateGrammar (COMMA! delegateGrammar)* SEMI!
;
delegateGrammar
: lab:id ASSIGN^ g:id {grammar.importGrammar(#g, #lab.getText());}
| g2:id {grammar.importGrammar(#g2,null);}
;
tokensSpec
: TOKENS^
( tokenSpec )+
RCURLY!
;
tokenSpec
: TOKEN_REF ( ASSIGN^ (STRING_LITERAL|CHAR_LITERAL) )? SEMI!
;
attrScopes
: (attrScope)*
;
attrScope
: "scope"^ id ACTION
;
rules
: (
options {
// limitation of appox LL(k) says ambig upon
// DOC_COMMENT TOKEN_REF, but that's an impossible sequence
warnWhenFollowAmbig=false;
}
: //{g.type==PARSER}? (aliasLexerRule)=>aliasLexerRule |
rule
)+
;
rule!
{
GrammarAST modifier=null, blk=null, blkRoot=null, eob=null;
int start = ((TokenWithIndex)LT(1)).getIndex();
int startLine = LT(1).getLine();
GrammarAST opt = null;
Map opts = null;
}
:
( d:DOC_COMMENT
)?
( p1:"protected" {modifier=#p1;}
| p2:"public" {modifier=#p2;}
| p3:"private" {modifier=#p3;}
| p4:"fragment" {modifier=#p4;}
)?
ruleName:id
{currentRuleName=#ruleName.getText();
if ( gtype==LEXER_GRAMMAR && #p4==null ) {
grammar.lexerRuleNamesInCombined.add(currentRuleName);
}
}
( BANG )?
( aa:ARG_ACTION )?
( "returns" rt:ARG_ACTION )?
( throwsSpec )?
( opts=optionsSpec {opt=(GrammarAST)returnAST;} )?
scopes:ruleScopeSpec
(a:ruleActions)?
colon:COLON
{
blkRoot = #[BLOCK,"BLOCK"];
blkRoot.setBlockOptions(opts);
blkRoot.setLine(colon.getLine());
blkRoot.setColumn(colon.getColumn());
eob = #[EOB,"<end-of-block>"];
}
b:altList[opts] {blk = #b;}
semi:SEMI
( ex:exceptionGroup )?
{
int stop = ((TokenWithIndex)LT(1)).getIndex()-1; // point at the semi or exception thingie
eob.setLine(semi.getLine());
eob.setColumn(semi.getColumn());
GrammarAST eor = #[EOR,"<end-of-rule>"];
eor.setLine(semi.getLine());
eor.setColumn(semi.getColumn());
GrammarAST root = #[RULE,"rule"];
root.ruleStartTokenIndex = start;
root.ruleStopTokenIndex = stop;
root.setLine(startLine);
root.setBlockOptions(opts);
#rule = #(root,
#ruleName,modifier,#(#[ARG,"ARG"],#aa),#(#[RET,"RET"],#rt),
opt,#scopes,#a,blk,ex,eor);
currentRuleName=null;
}
;
ruleActions
: (ruleAction)+
;
/** Match stuff like @init {int i;} */
ruleAction
: AMPERSAND^ id ACTION
;
throwsSpec
: "throws" id ( COMMA id )*
;
ruleScopeSpec
{
int line = LT(1).getLine();
int column = LT(1).getColumn();
}
:! ( options {warnWhenFollowAmbig=false;} : "scope" a:ACTION )?
( "scope" ids:idList SEMI! )*
{
GrammarAST scopeRoot = (GrammarAST)#[SCOPE,"scope"];
scopeRoot.setLine(line);
scopeRoot.setColumn(column);
#ruleScopeSpec = #(scopeRoot, #a, #ids);
}
;
/** Build #(BLOCK ( #(ALT ...) EOB )+ ) */
block
{
GrammarAST save = currentBlockAST;
Map opts=null;
}
: lp:LPAREN^ {#lp.setType(BLOCK); #lp.setText("BLOCK");}
(
// 2nd alt and optional branch ambig due to
// linear approx LL(2) issue. COLON ACTION
// matched correctly in 2nd alt.
options {
warnWhenFollowAmbig = false;
}
:
(opts=optionsSpec {#block.setOptions(grammar,opts);})?
( ruleActions )?
COLON!
| ACTION COLON!
)?
{currentBlockAST = #lp;}
a1:alternative rewrite
{if (LA(1)==OR||(LA(2)==QUESTION||LA(2)==PLUS||LA(2)==STAR)) prefixWithSynPred(#a1);}
( OR! a2:alternative rewrite
{if (LA(1)==OR||(LA(2)==QUESTION||LA(2)==PLUS||LA(2)==STAR)) prefixWithSynPred(#a2);}
)*
rp:RPAREN!
{
currentBlockAST = save;
GrammarAST eob = #[EOB,"<end-of-block>"];
eob.setLine(rp.getLine());
eob.setColumn(rp.getColumn());
#block.addChild(eob);
}
;
altList[Map opts]
{
GrammarAST blkRoot = #[BLOCK,"BLOCK"];
blkRoot.setBlockOptions(opts);
blkRoot.setLine(LT(0).getLine()); // set to : or (
blkRoot.setColumn(LT(0).getColumn());
GrammarAST save = currentBlockAST;
currentBlockAST = #blkRoot;
}
: a1:alternative rewrite
{if (LA(1)==OR||(LA(2)==QUESTION||LA(2)==PLUS||LA(2)==STAR)) prefixWithSynPred(#a1);}
( OR! a2:alternative rewrite
{if (LA(1)==OR||(LA(2)==QUESTION||LA(2)==PLUS||LA(2)==STAR)) prefixWithSynPred(#a2);} )*
{
#altList = #(blkRoot,#altList,#[EOB,"<end-of-block>"]);
currentBlockAST = save;
}
;
alternative
{
GrammarAST eoa = #[EOA, "<end-of-alt>"];
GrammarAST altRoot = #[ALT,"ALT"];
altRoot.setLine(LT(1).getLine());
altRoot.setColumn(LT(1).getColumn());
}
: ( el:element )+
{
if ( #alternative==null ) {
#alternative = #(altRoot,#[EPSILON,"epsilon"],eoa);
}
else {
// we have a real list of stuff
#alternative = #(altRoot, #alternative, eoa);
}
}
| {
GrammarAST eps = #[EPSILON,"epsilon"];
eps.setLine(LT(0).getLine()); // get line/col of '|' or ':' (prev token)
eps.setColumn(LT(0).getColumn());
#alternative = #(altRoot,eps,eoa);
}
;
exceptionGroup
: ( exceptionHandler )+ ( finallyClause )?
| finallyClause
;
exceptionHandler
: "catch"^ ARG_ACTION ACTION
;
finallyClause
: "finally"^ ACTION
;
element
: elementNoOptionSpec
;
elementNoOptionSpec
{
IntSet elements=null;
GrammarAST sub, sub2;
}
: ( id (ASSIGN^|PLUS_ASSIGN^) (atom|block)
( sub=ebnfSuffix[(GrammarAST)currentAST.root,false]! {#elementNoOptionSpec=sub;} )?
| atom
( sub2=ebnfSuffix[(GrammarAST)currentAST.root,false]! {#elementNoOptionSpec=sub2;} )?
| ebnf
| FORCED_ACTION
| ACTION
| p:SEMPRED ( IMPLIES! {#p.setType(GATED_SEMPRED);} )?
{
grammar.blocksWithSemPreds.add(currentBlockAST);
}
| t3:tree
)
;
atom
: range (ROOT^|BANG^)?
| ( options {
// TOKEN_REF WILDCARD could match terminal here then WILDCARD next
generateAmbigWarnings=false;
}
: // grammar.rule but ensure no spaces. "A . B" is not a qualified ref
// We do here rather than lexer so we can build a tree
{LT(1).getColumn()+LT(1).getText().length()==LT(2).getColumn()&&
LT(2).getColumn()+1==LT(3).getColumn()}?
id w:WILDCARD^ (terminal|ruleref) {#w.setType(DOT);}
| terminal
| ruleref
)
| notSet (ROOT^|BANG^)?
;
ruleref
: rr:RULE_REF^ ( ARG_ACTION )? (ROOT^|BANG^)?
;
notSet
{
int line = LT(1).getLine();
int col = LT(1).getColumn();
GrammarAST subrule=null;
}
: n:NOT^
( notTerminal
| block
)
{#notSet.setLine(line); #notSet.setColumn(col);}
;
treeRoot
: {atTreeRoot=true;}
( id (ASSIGN^|PLUS_ASSIGN^) (atom|block)
| atom
| block
)
{atTreeRoot=false;}
;
tree: TREE_BEGIN^ treeRoot ( element )+ RPAREN! ;
/** matches ENBF blocks (and sets via block rule) */
ebnf!
{
int line = LT(1).getLine();
int col = LT(1).getColumn();
}
: b:block
( QUESTION {#ebnf=#([OPTIONAL,"?"],#b);}
| STAR {#ebnf=#([CLOSURE,"*"],#b);}
| PLUS {#ebnf=#([POSITIVE_CLOSURE,"+"],#b);}
| IMPLIES! // syntactic predicate
{
if ( gtype==COMBINED_GRAMMAR &&
Character.isUpperCase(currentRuleName.charAt(0)) )
{
// ignore for lexer rules in combined
#ebnf = #(#[SYNPRED,"=>"],#b);
}
else {
// create manually specified (...)=> predicate;
// convert to sempred
#ebnf = createSynSemPredFromBlock(#b, SYN_SEMPRED);
}
}
| ROOT {#ebnf = #(#ROOT, #b);}
| BANG {#ebnf = #(#BANG, #b);}
| {#ebnf = #b;}
)
{#ebnf.setLine(line); #ebnf.setColumn(col);}
;
range!
{
GrammarAST subrule=null, root=null;
}
: c1:CHAR_LITERAL RANGE c2:CHAR_LITERAL
{
GrammarAST r = #[CHAR_RANGE,".."];
r.setLine(c1.getLine());
r.setColumn(c1.getColumn());
#range = #(r, #c1, #c2);
root = #range;
}
// (subrule=ebnfSuffix[root,false] {#range=subrule;})?
;
terminal
{
GrammarAST ebnfRoot=null, subrule=null;
}
: cl:CHAR_LITERAL^ ( elementOptions[#cl]! )? (ROOT^|BANG^)?
| tr:TOKEN_REF^
( elementOptions[#tr]! )?
( ARG_ACTION )? // Args are only valid for lexer rules
(ROOT^|BANG^)?
| sl:STRING_LITERAL^ ( elementOptions[#sl]! )? (ROOT^|BANG^)?
| wi:WILDCARD (ROOT^|BANG^)?
{
if ( atTreeRoot ) {
ErrorManager.syntaxError(
ErrorManager.MSG_WILDCARD_AS_ROOT,grammar,wi,null,null);
}
}
;
elementOptions[GrammarAST terminalAST]
: OPEN_ELEMENT_OPTION^ defaultNodeOption[terminalAST] CLOSE_ELEMENT_OPTION!
| OPEN_ELEMENT_OPTION^ elementOption[terminalAST] (SEMI! elementOption[terminalAST])* CLOSE_ELEMENT_OPTION!
;
defaultNodeOption[GrammarAST terminalAST]
{
StringBuffer buf = new StringBuffer();
}
: i:id {buf.append(#i.getText());} (WILDCARD i2:id {buf.append("."+#i2.getText());})*
{terminalAST.setTerminalOption(grammar,Grammar.defaultTokenOption,buf.toString());}
;
elementOption[GrammarAST terminalAST]
: a:id ASSIGN^ (b:id|s:STRING_LITERAL)
{
Object v = (#b!=null)?#b.getText():#s.getText();
terminalAST.setTerminalOption(grammar,#a.getText(),v);
}
;
ebnfSuffix[GrammarAST elemAST, boolean inRewrite] returns [GrammarAST subrule=null]
{
GrammarAST ebnfRoot=null;
}
:! ( QUESTION {ebnfRoot = #[OPTIONAL,"?"];}
| STAR {ebnfRoot = #[CLOSURE,"*"];}
| PLUS {ebnfRoot = #[POSITIVE_CLOSURE,"+"];}
)
{
GrammarAST save = currentBlockAST;
ebnfRoot.setLine(elemAST.getLine());
ebnfRoot.setColumn(elemAST.getColumn());
GrammarAST blkRoot = #[BLOCK,"BLOCK"];
currentBlockAST = blkRoot;
GrammarAST eob = #[EOB,"<end-of-block>"];
eob.setLine(elemAST.getLine());
eob.setColumn(elemAST.getColumn());
GrammarAST alt = #(#[ALT,"ALT"],elemAST,#[EOA,"<end-of-alt>"]);
if ( !inRewrite ) {
prefixWithSynPred(alt);
}
subrule =
#(ebnfRoot,
#(blkRoot,alt,eob)
);
currentBlockAST = save;
}
;
notTerminal
: cl:CHAR_LITERAL
| tr:TOKEN_REF
| STRING_LITERAL
;
idList
: id (COMMA! id)*
;
id : TOKEN_REF {#id.setType(ID);}
| RULE_REF {#id.setType(ID);}
;
// R E W R I T E S Y N T A X
rewrite
{
GrammarAST root = new GrammarAST();
}
:!
( options { warnWhenFollowAmbig=false;}
: rew:REWRITE pred:SEMPRED alt:rewrite_alternative
{root.addChild( #(#rew, #pred, #alt) );}
)*
rew2:REWRITE alt2:rewrite_alternative
{
root.addChild( #(#rew2, #alt2) );
#rewrite = (GrammarAST)root.getFirstChild();
}
|
;
rewrite_block
: lp:LPAREN^ {#lp.setType(BLOCK); #lp.setText("BLOCK");}
rewrite_alternative
RPAREN!
{
GrammarAST eob = #[EOB,"<end-of-block>"];
eob.setLine(lp.getLine());
eob.setColumn(lp.getColumn());
#rewrite_block.addChild(eob);
}
;
rewrite_alternative
{
GrammarAST eoa = #[EOA, "<end-of-alt>"];
GrammarAST altRoot = #[ALT,"ALT"];
altRoot.setLine(LT(1).getLine());
altRoot.setColumn(LT(1).getColumn());
}
: {grammar.buildTemplate()}? rewrite_template
| {grammar.buildAST()}? ( rewrite_element )+
{
if ( #rewrite_alternative==null ) {
#rewrite_alternative = #(altRoot,#[EPSILON,"epsilon"],eoa);
}
else {
#rewrite_alternative = #(altRoot, #rewrite_alternative,eoa);
}
}
| {#rewrite_alternative = #(altRoot,#[EPSILON,"epsilon"],eoa);}
| {grammar.buildAST()}? ETC
;
rewrite_element
{
GrammarAST subrule=null;
}
: t:rewrite_atom
( subrule=ebnfSuffix[#t,true] {#rewrite_element=subrule;} )?
| rewrite_ebnf
| tr:rewrite_tree
( subrule=ebnfSuffix[#tr,true] {#rewrite_element=subrule;} )?
;
rewrite_atom
{
GrammarAST subrule=null;
}
: tr:TOKEN_REF^ (elementOptions[#tr]!)? (ARG_ACTION)? // for imaginary nodes
| rr:RULE_REF
| cl:CHAR_LITERAL^ (elementOptions[#cl]!)?
| sl:STRING_LITERAL^ (elementOptions[#sl]!)?
|! d:DOLLAR i:id // reference to a label in a rewrite rule
{
#rewrite_atom = #[LABEL,i_AST.getText()];
#rewrite_atom.setLine(#d.getLine());
#rewrite_atom.setColumn(#d.getColumn());
}
| ACTION
;
rewrite_ebnf!
{
int line = LT(1).getLine();
int col = LT(1).getColumn();
}
: b:rewrite_block
( QUESTION {#rewrite_ebnf=#([OPTIONAL,"?"],#b);}
| STAR {#rewrite_ebnf=#([CLOSURE,"*"],#b);}
| PLUS {#rewrite_ebnf=#([POSITIVE_CLOSURE,"+"],#b);}
)
{#rewrite_ebnf.setLine(line); #rewrite_ebnf.setColumn(col);}
;
rewrite_tree :
TREE_BEGIN^
rewrite_atom ( rewrite_element )*
RPAREN!
;
/** Build a tree for a template rewrite:
^(TEMPLATE (ID|ACTION) ^(ARGLIST ^(ARG ID ACTION) ...) )
where ARGLIST is always there even if no args exist.
ID can be "template" keyword. If first child is ACTION then it's
an indirect template ref
-> foo(a={...}, b={...})
-> ({string-e})(a={...}, b={...}) // e evaluates to template name
-> {%{$ID.text}} // create literal template from string (done in ActionTranslator)
-> {st-expr} // st-expr evaluates to ST
*/
rewrite_template
{Token st=null;}
: // -> template(a={...},...) "..."
{LT(1).getText().equals("template")}? // inline
rewrite_template_head {st=LT(1);}
( DOUBLE_QUOTE_STRING_LITERAL! | DOUBLE_ANGLE_STRING_LITERAL! )
{#rewrite_template.addChild(#[st]);}
| // -> foo(a={...}, ...)
rewrite_template_head
| // -> ({expr})(a={...}, ...)
rewrite_indirect_template_head
| // -> {...}
ACTION
;
/** -> foo(a={...}, ...) */
rewrite_template_head
: id lp:LPAREN^ {#lp.setType(TEMPLATE); #lp.setText("TEMPLATE");}
rewrite_template_args
RPAREN!
;
/** -> ({expr})(a={...}, ...) */
rewrite_indirect_template_head
: lp:LPAREN^ {#lp.setType(TEMPLATE); #lp.setText("TEMPLATE");}
ACTION
RPAREN!
LPAREN! rewrite_template_args RPAREN!
;
rewrite_template_args
: rewrite_template_arg (COMMA! rewrite_template_arg)*
{#rewrite_template_args = #(#[ARGLIST,"ARGLIST"], rewrite_template_args);}
| {#rewrite_template_args = #[ARGLIST,"ARGLIST"];}
;
rewrite_template_arg
: id a:ASSIGN^ {#a.setType(ARG); #a.setText("ARG");} ACTION
;
class ANTLRLexer extends Lexer;
options {
k=3;
exportVocab=ANTLR;
testLiterals=false;
interactive=true;
charVocabulary='\003'..'\377';
}
{
/** advance the current column number by one; don't do tabs.
* we want char position in line to be sent to AntlrWorks.
*/
public void tab() {
setColumn( getColumn()+1 );
}
public boolean hasASTOperator = false;
}
WS : ( ' '
| '\t'
| ('\r')? '\n' {newline();}
)
;
COMMENT :
( SL_COMMENT | t:ML_COMMENT {$setType(t.getType());} )
;
protected
SL_COMMENT
: "//"
( (" $ANTLR")=> " $ANTLR " SRC ('\r')? '\n' // src directive
| ( options {greedy=false;} : . )* ('\r')? '\n'
)
{ newline(); }
;
protected
ML_COMMENT :
"/*"
( { LA(2)!='/' }? '*' {$setType(DOC_COMMENT);}
|
)
(
options {
greedy=false; // make it exit upon "*/"
}
: '\r' '\n' {newline();}
| '\n' {newline();}
| ~('\n'|'\r')
)*
"*/"
;
OPEN_ELEMENT_OPTION
: '<'
;
CLOSE_ELEMENT_OPTION
: '>'
;
AMPERSAND : '@';
COMMA : ',';
QUESTION : '?' ;
TREE_BEGIN : "^(" ;
LPAREN: '(' ;
RPAREN: ')' ;
COLON : ':' ;
STAR: '*' ;
PLUS: '+' ;
ASSIGN : '=' ;
PLUS_ASSIGN : "+=" ;
IMPLIES : "=>" ;
REWRITE : "->" ;
SEMI: ';' ;
ROOT : '^' {hasASTOperator=true;} ;
BANG : '!' {hasASTOperator=true;} ;
OR : '|' ;
WILDCARD : '.' ;
ETC : "..." ;
RANGE : ".." ;
NOT : '~' ;
RCURLY: '}' ;
DOLLAR : '$' ;
STRAY_BRACKET
: ']'
{
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
null,
_token,
"antlr: dangling ']'? make sure to escape with \\]",
null);
}
;
CHAR_LITERAL
: '\'' (ESC|'\n'{newline();}|~'\'')* '\''
{
StringBuffer s = Grammar.getUnescapedStringFromGrammarStringLiteral($getText);
if ( s.length()>1 ) {
$setType(STRING_LITERAL);
}
}
;
DOUBLE_QUOTE_STRING_LITERAL
: '"' ('\\'! '"'|'\\' ~'"'|'\n'{newline();}|~'"')* '"'
;
DOUBLE_ANGLE_STRING_LITERAL
: "<<" (options {greedy=false;}:'\n'{newline();}|.)* ">>"
;
protected
ESC : '\\'
( 'n'
| 'r'
| 't'
| 'b'
| 'f'
| '"'
| '\''
| '\\'
| '>'
| 'u' XDIGIT XDIGIT XDIGIT XDIGIT
| . // unknown, leave as it is
)
;
protected
DIGIT
: '0'..'9'
;
protected
XDIGIT :
'0' .. '9'
| 'a' .. 'f'
| 'A' .. 'F'
;
INT : ('0'..'9')+
;
//HETERO_TYPE : '<'! ~'<' (~'>')* '>'! ;
ARG_ACTION
: '['! NESTED_ARG_ACTION ']'!
;
protected
NESTED_ARG_ACTION :
( '\r' '\n' {newline();}
| '\n' {newline();}
| '\\'! ']'
| '\\' ~']'
| ACTION_STRING_LITERAL
| ACTION_CHAR_LITERAL
| ~']'
)*
;
ACTION
{int actionLine=getLine(); int actionColumn = getColumn(); }
: NESTED_ACTION
( '?'! {_ttype = SEMPRED;} )?
{
Token t = makeToken(_ttype);
String action = $getText;
int n = 1; // num delimiter chars
if ( action.startsWith("{{") && action.endsWith("}}") ) {
t.setType(FORCED_ACTION);
n = 2;
}
action = action.substring(n,action.length()-n);
t.setText(action);
t.setLine(actionLine); // set action line to start
t.setColumn(actionColumn);
$setToken(t);
}
;
protected
NESTED_ACTION :
'{'
(
options {
greedy = false; // exit upon '}'
}
:
(
'\r' '\n' {newline();}
| '\n' {newline();}
)
| NESTED_ACTION
| ACTION_CHAR_LITERAL
| COMMENT
| ACTION_STRING_LITERAL
| ACTION_ESC
| .
)*
'}'
;
protected
ACTION_CHAR_LITERAL
: '\'' (ACTION_ESC|'\n'{newline();}|~'\'')* '\''
;
protected
ACTION_STRING_LITERAL
: '"' (ACTION_ESC|'\n'{newline();}|~'"')* '"'
;
protected
ACTION_ESC
: "\\'"
| "\\\""
| '\\' ~('\''|'"')
;
TOKEN_REF
options { testLiterals = true; }
: 'A'..'Z'
( // scarf as many letters/numbers as you can
options {
warnWhenFollowAmbig=false;
}
:
'a'..'z'|'A'..'Z'|'_'|'0'..'9'
)*
;
// we get a warning here when looking for options '{', but it works right
RULE_REF
{
int t=0;
}
: t=INTERNAL_RULE_REF {_ttype=t;}
( {t==OPTIONS}? WS_LOOP ('{' {_ttype = OPTIONS;})?
| {t==TOKENS}? WS_LOOP ('{' {_ttype = TOKENS;})?
|
)
;
protected
WS_LOOP
: ( // grab as much WS as you can
options {
greedy=true;
}
:
WS
| COMMENT
)*
;
protected
INTERNAL_RULE_REF returns [int t]
{
t = RULE_REF;
}
: 'a'..'z'
( // scarf as many letters/numbers as you can
options {
warnWhenFollowAmbig=false;
}
:
'a'..'z'|'A'..'Z'|'_'|'0'..'9'
)*
{t = testLiteralsTable(t);}
;
protected
WS_OPT
: (WS)?
;
/** Reset the file and line information; useful when the grammar
* has been generated so that errors are shown relative to the
* original file like the old C preprocessor used to do.
*/
protected
SRC : "src" ' ' file:ACTION_STRING_LITERAL ' ' line:INT
{
newline();
setFilename(file.getText().substring(1,file.getText().length()-1));
setLine(Integer.parseInt(line.getText())-1); // -1 because SL_COMMENT will increment the line no. KR
$setType(Token.SKIP); // don't let this go to the parser
}
;
antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/v2/define.g 0000644 0001750 0001750 00000041566 11256465226 023641 0 ustar twerner twerner header {
/*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.grammar.v2;
import java.util.*;
import org.antlr.misc.*;
import org.antlr.tool.*;
}
class DefineGrammarItemsWalker extends TreeParser;
options {
importVocab = ANTLR;
ASTLabelType = "GrammarAST";
codeGenBitsetTestThreshold=999;
}
{
protected Grammar grammar;
protected GrammarAST root;
protected String currentRuleName;
protected GrammarAST currentRewriteBlock;
protected GrammarAST currentRewriteRule;
protected int outerAltNum = 0;
protected int blockLevel = 0;
public void reportError(RecognitionException ex) {
Token token = null;
if ( ex instanceof MismatchedTokenException ) {
token = ((MismatchedTokenException)ex).token;
}
else if ( ex instanceof NoViableAltException ) {
token = ((NoViableAltException)ex).token;
}
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
grammar,
token,
"define: "+ex.toString(),
ex);
}
protected void finish() {
trimGrammar();
}
/** Remove any lexer rules from a COMBINED; already passed to lexer */
protected void trimGrammar() {
if ( grammar.type!=Grammar.COMBINED ) {
return;
}
// form is (header ... ) ( grammar ID (scope ...) ... ( rule ... ) ( rule ... ) ... )
GrammarAST p = root;
// find the grammar spec
while ( !p.getText().equals("grammar") ) {
p = (GrammarAST)p.getNextSibling();
}
p = (GrammarAST)p.getFirstChild(); // jump down to first child of grammar
// look for first RULE def
GrammarAST prev = p; // points to the ID (grammar name)
while ( p.getType()!=RULE ) {
prev = p;
p = (GrammarAST)p.getNextSibling();
}
// prev points at last node before first rule subtree at this point
while ( p!=null ) {
String ruleName = p.getFirstChild().getText();
//System.out.println("rule "+ruleName+" prev="+prev.getText());
if ( Character.isUpperCase(ruleName.charAt(0)) ) {
// remove lexer rule
prev.setNextSibling(p.getNextSibling());
}
else {
prev = p; // non-lexer rule; move on
}
p = (GrammarAST)p.getNextSibling();
}
//System.out.println("root after removal is: "+root.toStringList());
}
protected void trackInlineAction(GrammarAST actionAST) {
Rule r = grammar.getRule(currentRuleName);
if ( r!=null ) {
r.trackInlineAction(actionAST);
}
}
}
grammar[Grammar g]
{
grammar = g;
root = #grammar;
}
: ( #( LEXER_GRAMMAR {grammar.type = Grammar.LEXER;} grammarSpec )
| #( PARSER_GRAMMAR {grammar.type = Grammar.PARSER;} grammarSpec )
| #( TREE_GRAMMAR {grammar.type = Grammar.TREE_PARSER;} grammarSpec )
| #( COMBINED_GRAMMAR {grammar.type = Grammar.COMBINED;} grammarSpec )
)
{finish();}
;
attrScope
: #( "scope" name:ID attrs:ACTION )
{
AttributeScope scope = grammar.defineGlobalScope(name.getText(),#attrs.token);
scope.isDynamicGlobalScope = true;
scope.addAttributes(attrs.getText(), ';');
}
;
grammarSpec
{
Map opts=null;
Token optionsStartToken=null;
}
: id:ID
(cmt:DOC_COMMENT)?
//(#(OPTIONS .))? // already parsed these in assign.types.g
( {optionsStartToken=((GrammarAST)_t).getToken();}
optionsSpec
)?
(delegateGrammars)?
(tokensSpec)?
(attrScope)*
(actions)?
rules
;
actions
: ( action )+
;
action
{
String scope=null;
GrammarAST nameAST=null, actionAST=null;
}
: #(amp:AMPERSAND id1:ID
( id2:ID a1:ACTION
{scope=#id1.getText(); nameAST=#id2; actionAST=#a1;}
| a2:ACTION
{scope=null; nameAST=#id1; actionAST=#a2;}
)
)
{
grammar.defineNamedAction(#amp,scope,nameAST,actionAST);
}
;
optionsSpec
: OPTIONS
;
delegateGrammars
: #( "import" ( #(ASSIGN ID ID) | ID )+ )
;
tokensSpec
: #( TOKENS ( tokenSpec )+ )
;
tokenSpec
: t:TOKEN_REF
| #( ASSIGN
t2:TOKEN_REF
( s:STRING_LITERAL
| c:CHAR_LITERAL
)
)
;
rules
: ( rule )+
;
rule
{
String mod=null;
String name=null;
Map opts=null;
Rule r = null;
}
: #( RULE id:ID {opts = #RULE.getBlockOptions();}
(mod=modifier)?
#( ARG (args:ARG_ACTION)? )
#( RET (ret:ARG_ACTION)? )
(optionsSpec)?
{
name = #id.getText();
currentRuleName = name;
if ( Character.isUpperCase(name.charAt(0)) &&
grammar.type==Grammar.COMBINED )
{
// a merged grammar spec, track lexer rules and send to another grammar
grammar.defineLexerRuleFoundInParser(#id.getToken(), #rule);
}
else {
int numAlts = countAltsForRule(#rule);
grammar.defineRule(#id.getToken(), mod, opts, #rule, #args, numAlts);
r = grammar.getRule(name);
if ( #args!=null ) {
r.parameterScope = grammar.createParameterScope(name,#args.token);
r.parameterScope.addAttributes(#args.getText(), ',');
}
if ( #ret!=null ) {
r.returnScope = grammar.createReturnScope(name,#ret.token);
r.returnScope.addAttributes(#ret.getText(), ',');
}
}
}
(ruleScopeSpec[r])?
(ruleAction[r])*
{this.blockLevel=0;}
b:block
(exceptionGroup)?
EOR
{
// copy rule options into the block AST, which is where
// the analysis will look for k option etc...
#b.setBlockOptions(opts);
}
)
;
countAltsForRule returns [int n=0]
: #( RULE id:ID (modifier)? ARG RET (OPTIONS)? ("scope")? (AMPERSAND)*
#( BLOCK (OPTIONS)? (ALT (REWRITE)* {n++;})+ EOB )
(exceptionGroup)?
EOR
)
;
ruleAction[Rule r]
: #(amp:AMPERSAND id:ID a:ACTION ) {if (r!=null) r.defineNamedAction(#amp,#id,#a);}
;
modifier returns [String mod]
{
mod = #modifier.getText();
}
: "protected"
| "public"
| "private"
| "fragment"
;
ruleScopeSpec[Rule r]
: #( "scope"
( attrs:ACTION
{
r.ruleScope = grammar.createRuleScope(r.name,#attrs.token);
r.ruleScope.isDynamicRuleScope = true;
r.ruleScope.addAttributes(#attrs.getText(), ';');
}
)?
( uses:ID
{
if ( grammar.getGlobalScope(#uses.getText())==null ) {
ErrorManager.grammarError(ErrorManager.MSG_UNKNOWN_DYNAMIC_SCOPE,
grammar,
#uses.token,
#uses.getText());
}
else {
if ( r.useScopes==null ) {r.useScopes=new ArrayList();}
r.useScopes.add(#uses.getText());
}
}
)*
)
;
block
{
this.blockLevel++;
if ( this.blockLevel==1 ) {this.outerAltNum=1;}
}
: #( BLOCK
(optionsSpec)?
(blockAction)*
( alternative rewrite
{if ( this.blockLevel==1 ) {this.outerAltNum++;}}
)+
EOB
)
{this.blockLevel--;}
;
// TODO: this does nothing now! subrules cannot have init actions. :(
blockAction
: #(amp:AMPERSAND id:ID a:ACTION ) // {r.defineAction(#amp,#id,#a);}
;
alternative
{
if ( grammar.type!=Grammar.LEXER && grammar.getOption("output")!=null && blockLevel==1 ) {
GrammarAST aRewriteNode = #alternative.findFirstType(REWRITE); // alt itself has rewrite?
GrammarAST rewriteAST = (GrammarAST)#alternative.getNextSibling();
// we have a rewrite if alt uses it inside subrule or this alt has one
// but don't count -> ... rewrites, which mean "do default auto construction"
if ( aRewriteNode!=null||
(rewriteAST!=null &&
rewriteAST.getType()==REWRITE &&
rewriteAST.getFirstChild()!=null &&
rewriteAST.getFirstChild().getType()!=ETC) )
{
Rule r = grammar.getRule(currentRuleName);
r.trackAltsWithRewrites(#alternative,this.outerAltNum);
}
}
}
: #( ALT (element)+ EOA )
;
exceptionGroup
: ( exceptionHandler )+ (finallyClause)?
| finallyClause
;
exceptionHandler
: #("catch" ARG_ACTION ACTION) {trackInlineAction(#ACTION);}
;
finallyClause
: #("finally" ACTION) {trackInlineAction(#ACTION);}
;
element
: #(ROOT element)
| #(BANG element)
| atom[null]
| #(NOT element)
| #(RANGE atom[null] atom[null])
| #(CHAR_RANGE atom[null] atom[null])
| #(ASSIGN id:ID el:element)
{
if ( #el.getType()==ANTLRParser.ROOT ||
#el.getType()==ANTLRParser.BANG )
{
#el = (GrammarAST)#el.getFirstChild();
}
if ( #el.getType()==RULE_REF) {
grammar.defineRuleRefLabel(currentRuleName,#id.getToken(),#el);
}
else if ( #el.getType()==WILDCARD && grammar.type==Grammar.TREE_PARSER ) {
grammar.defineWildcardTreeLabel(currentRuleName,#id.getToken(),#el);
}
else {
grammar.defineTokenRefLabel(currentRuleName,#id.getToken(),#el);
}
}
| #( PLUS_ASSIGN id2:ID a2:element
{
if ( #a2.getType()==ANTLRParser.ROOT ||
#a2.getType()==ANTLRParser.BANG )
{
#a2 = (GrammarAST)#a2.getFirstChild();
}
if ( #a2.getType()==RULE_REF ) {
grammar.defineRuleListLabel(currentRuleName,#id2.getToken(),#a2);
}
else if ( #a2.getType()==WILDCARD && grammar.type==Grammar.TREE_PARSER ) {
grammar.defineWildcardTreeListLabel(currentRuleName,#id2.getToken(),#a2);
}
else {
grammar.defineTokenListLabel(currentRuleName,#id2.getToken(),#a2);
}
}
)
| ebnf
| tree
| #( SYNPRED block )
| act:ACTION
{
#act.outerAltNum = this.outerAltNum;
trackInlineAction(#act);
}
| act2:FORCED_ACTION
{
#act2.outerAltNum = this.outerAltNum;
trackInlineAction(#act2);
}
| SEMPRED
{
#SEMPRED.outerAltNum = this.outerAltNum;
trackInlineAction(#SEMPRED);
}
| SYN_SEMPRED
| BACKTRACK_SEMPRED
| GATED_SEMPRED
{
#GATED_SEMPRED.outerAltNum = this.outerAltNum;
trackInlineAction(#GATED_SEMPRED);
}
| EPSILON
;
ebnf: (dotLoop)=> dotLoop // .* or .+
| block
| #( OPTIONAL block )
| #( CLOSURE block )
| #( POSITIVE_CLOSURE block )
;
/** Track the .* and .+ idioms and make them nongreedy by default.
*/
dotLoop
{
GrammarAST block = (GrammarAST)#dotLoop.getFirstChild();
}
: ( #( CLOSURE dotBlock )
| #( POSITIVE_CLOSURE dotBlock )
)
{
Map opts=new HashMap();
opts.put("greedy", "false");
if ( grammar.type!=Grammar.LEXER ) {
// parser grammars assume k=1 for .* loops
// otherwise they (analysis?) look til EOF!
opts.put("k", Utils.integer(1));
}
block.setOptions(grammar,opts);
}
;
dotBlock
: #( BLOCK #( ALT WILDCARD EOA ) EOB )
;
tree: #(TREE_BEGIN element (element)*)
;
atom[GrammarAST scope]
: #( rr:RULE_REF (rarg:ARG_ACTION)? )
{
grammar.altReferencesRule(currentRuleName, scope, #rr, this.outerAltNum);
if ( #rarg!=null ) {
#rarg.outerAltNum = this.outerAltNum;
trackInlineAction(#rarg);
}
}
| #( t:TOKEN_REF (targ:ARG_ACTION )? )
{
if ( #targ!=null ) {
#targ.outerAltNum = this.outerAltNum;
trackInlineAction(#targ);
}
if ( grammar.type==Grammar.LEXER ) {
grammar.altReferencesRule(currentRuleName, scope, #t, this.outerAltNum);
}
else {
grammar.altReferencesTokenID(currentRuleName, #t, this.outerAltNum);
}
}
| c:CHAR_LITERAL
{
if ( grammar.type!=Grammar.LEXER ) {
Rule rule = grammar.getRule(currentRuleName);
if ( rule!=null ) {
rule.trackTokenReferenceInAlt(#c, outerAltNum);
}
}
}
| s:STRING_LITERAL
{
if ( grammar.type!=Grammar.LEXER ) {
Rule rule = grammar.getRule(currentRuleName);
if ( rule!=null ) {
rule.trackTokenReferenceInAlt(#s, outerAltNum);
}
}
}
| WILDCARD
| #(DOT ID atom[#ID]) // scope override on rule
;
ast_suffix
: ROOT
| BANG
;
rewrite
{
currentRewriteRule = #rewrite; // has to execute during guessing
if ( grammar.buildAST() ) {
#rewrite.rewriteRefsDeep = new HashSet<GrammarAST>();
}
}
: (
#( REWRITE (pred:SEMPRED)? rewrite_alternative )
{
if ( #pred!=null ) {
#pred.outerAltNum = this.outerAltNum;
trackInlineAction(#pred);
}
}
)*
//{System.out.println("-> refs = "+#rewrite.rewriteRefs);}
;
rewrite_block
{
GrammarAST enclosingBlock = currentRewriteBlock;
if ( inputState.guessing==0 ) { // don't do if guessing
currentRewriteBlock=#rewrite_block; // pts to BLOCK node
currentRewriteBlock.rewriteRefsShallow = new HashSet<GrammarAST>();
currentRewriteBlock.rewriteRefsDeep = new HashSet<GrammarAST>();
}
}
: #( BLOCK rewrite_alternative EOB )
//{System.out.println("atoms="+currentRewriteBlock.rewriteRefs);}
{
// copy the element refs in this block to the surrounding block
if ( enclosingBlock!=null ) {
enclosingBlock.rewriteRefsDeep
.addAll(currentRewriteBlock.rewriteRefsShallow);
}
currentRewriteBlock = enclosingBlock; // restore old BLOCK ptr
}
;
rewrite_alternative
: {grammar.buildAST()}?
#( a:ALT ( ( rewrite_element )+ | EPSILON ) EOA )
| {grammar.buildTemplate()}? rewrite_template
| ETC {this.blockLevel==1}? // only valid as outermost rewrite
;
rewrite_element
: rewrite_atom
| rewrite_ebnf
| rewrite_tree
;
rewrite_ebnf
: #( OPTIONAL rewrite_block )
| #( CLOSURE rewrite_block )
| #( POSITIVE_CLOSURE rewrite_block )
;
rewrite_tree
: #( TREE_BEGIN rewrite_atom ( rewrite_element )* )
;
rewrite_atom
{
Rule r = grammar.getRule(currentRuleName);
Set tokenRefsInAlt = r.getTokenRefsInAlt(outerAltNum);
boolean imaginary =
#rewrite_atom.getType()==TOKEN_REF &&
!tokenRefsInAlt.contains(#rewrite_atom.getText());
if ( !imaginary && grammar.buildAST() &&
(#rewrite_atom.getType()==RULE_REF ||
#rewrite_atom.getType()==LABEL ||
#rewrite_atom.getType()==TOKEN_REF ||
#rewrite_atom.getType()==CHAR_LITERAL ||
#rewrite_atom.getType()==STRING_LITERAL) )
{
// track per block and for entire rewrite rule
if ( currentRewriteBlock!=null ) {
currentRewriteBlock.rewriteRefsShallow.add(#rewrite_atom);
currentRewriteBlock.rewriteRefsDeep.add(#rewrite_atom);
}
currentRewriteRule.rewriteRefsDeep.add(#rewrite_atom);
}
}
: RULE_REF
| ( #(TOKEN_REF
(arg:ARG_ACTION)?
)
| CHAR_LITERAL
| STRING_LITERAL
)
{
if ( #arg!=null ) {
#arg.outerAltNum = this.outerAltNum;
trackInlineAction(#arg);
}
}
| LABEL
| ACTION
{
#ACTION.outerAltNum = this.outerAltNum;
trackInlineAction(#ACTION);
}
;
rewrite_template
: #( ALT EPSILON EOA )
| #( TEMPLATE (id:ID|ind:ACTION)
#( ARGLIST
( #( ARG arg:ID a:ACTION )
{
#a.outerAltNum = this.outerAltNum;
trackInlineAction(#a);
}
)*
)
{
if ( #ind!=null ) {
#ind.outerAltNum = this.outerAltNum;
trackInlineAction(#ind);
}
}
( DOUBLE_QUOTE_STRING_LITERAL
| DOUBLE_ANGLE_STRING_LITERAL
)?
)
| act:ACTION
{
#act.outerAltNum = this.outerAltNum;
trackInlineAction(#act);
}
;
antlr-3.2/tool/src/main/antlr2/org/antlr/grammar/v2/antlr.print.g 0000644 0001750 0001750 00000022177 11256465226 024657 0 ustar twerner twerner header {
/*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.grammar.v2;
import java.util.*;
import org.antlr.tool.*;
}
/** Print out a grammar (no pretty printing).
*
* Terence Parr
* University of San Francisco
* August 19, 2003
*/
class ANTLRTreePrinter extends TreeParser;
options {
importVocab = ANTLR;
ASTLabelType = "GrammarAST";
codeGenBitsetTestThreshold=999;
}
{
protected Grammar grammar;
protected boolean showActions;
protected StringBuffer buf = new StringBuffer(300);
public void out(String s) {
buf.append(s);
}
public void reportError(RecognitionException ex) {
Token token = null;
if ( ex instanceof MismatchedTokenException ) {
token = ((MismatchedTokenException)ex).token;
}
else if ( ex instanceof NoViableAltException ) {
token = ((NoViableAltException)ex).token;
}
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
grammar,
token,
"antlr.print: "+ex.toString(),
ex);
}
/** Normalize a grammar print out by removing all double spaces
* and trailing/beginning stuff. FOr example, convert
*
* ( A | B | C )*
*
* to
*
* ( A | B | C )*
*/
public static String normalize(String g) {
StringTokenizer st = new StringTokenizer(g, " ", false);
StringBuffer buf = new StringBuffer();
while ( st.hasMoreTokens() ) {
String w = st.nextToken();
buf.append(w);
buf.append(" ");
}
return buf.toString().trim();
}
}
/** Call this to figure out how to print */
toString[Grammar g, boolean showActions] returns [String s=null]
{
grammar = g;
this.showActions = showActions;
}
: ( grammar
| rule
| alternative
| element
| single_rewrite
| EOR {s="EOR";}
)
{return normalize(buf.toString());}
;
// --------------
grammar
: ( #( LEXER_GRAMMAR grammarSpec["lexer " ] )
| #( PARSER_GRAMMAR grammarSpec["parser "] )
| #( TREE_GRAMMAR grammarSpec["tree "] )
| #( COMBINED_GRAMMAR grammarSpec[""] )
)
;
attrScope
: #( "scope" ID ACTION )
;
grammarSpec[String gtype]
: id:ID {out(gtype+"grammar "+#id.getText());}
(cmt:DOC_COMMENT {out(#cmt.getText()+"\n");} )?
(optionsSpec)? {out(";\n");}
(delegateGrammars)?
(tokensSpec)?
(attrScope)*
(actions)?
rules
;
actions
: ( action )+
;
action
{
String scope=null, name=null;
String action=null;
}
: #(AMPERSAND id1:ID
( id2:ID a1:ACTION
{scope=#id1.getText(); name=#a1.getText(); action=#a1.getText();}
| a2:ACTION
{scope=null; name=#id1.getText(); action=#a2.getText();}
)
)
{
if ( showActions ) {
out("@"+(scope!=null?scope+"::":"")+name+action);
}
}
;
optionsSpec
: #( OPTIONS {out(" options {");}
(option {out("; ");})+
{out("} ");}
)
;
option
: #( ASSIGN id:ID {out(#id.getText()+"=");} optionValue )
;
optionValue
: id:ID {out(#id.getText());}
| s:STRING_LITERAL {out(#s.getText());}
| c:CHAR_LITERAL {out(#c.getText());}
| i:INT {out(#i.getText());}
// | charSet
;
/*
charSet
: #( CHARSET charSetElement )
;
charSetElement
: c:CHAR_LITERAL {out(#c.getText());}
| #( OR c1:CHAR_LITERAL c2:CHAR_LITERAL )
| #( RANGE c3:CHAR_LITERAL c4:CHAR_LITERAL )
;
*/
delegateGrammars
: #( "import" ( #(ASSIGN ID ID) | ID )+ )
;
tokensSpec
: #( TOKENS ( tokenSpec )+ )
;
tokenSpec
: TOKEN_REF
| #( ASSIGN TOKEN_REF (STRING_LITERAL|CHAR_LITERAL) )
;
rules
: ( rule )+
;
rule
: #( RULE id:ID
(modifier)?
{out(#id.getText());}
#(ARG (arg:ARG_ACTION {out("["+#arg.getText()+"]");} )? )
#(RET (ret:ARG_ACTION {out(" returns ["+#ret.getText()+"]");} )? )
(optionsSpec)?
(ruleScopeSpec)?
(ruleAction)*
{out(" : ");}
b:block[false]
(exceptionGroup)?
EOR {out(";\n");}
)
;
ruleAction
: #(AMPERSAND id:ID a:ACTION )
{if ( showActions ) out("@"+#id.getText()+"{"+#a.getText()+"}");}
;
modifier
{out(#modifier.getText()); out(" ");}
: "protected"
| "public"
| "private"
| "fragment"
;
ruleScopeSpec
: #( "scope" (ACTION)? ( ID )* )
;
block[boolean forceParens]
{
int numAlts = countAltsForBlock(#block);
}
: #( BLOCK {if ( forceParens||numAlts>1 ) out(" (");}
(optionsSpec {out(" : ");} )?
alternative rewrite ( {out(" | ");} alternative rewrite )*
EOB {if ( forceParens||numAlts>1 ) out(")");}
)
;
countAltsForBlock returns [int n=0]
: #( BLOCK (OPTIONS)? (ALT (REWRITE)* {n++;})+ EOB )
;
alternative
: #( ALT (element)+ EOA )
;
exceptionGroup
: ( exceptionHandler )+ (finallyClause)?
| finallyClause
;
exceptionHandler
: #("catch" ARG_ACTION ACTION)
;
finallyClause
: #("finally" ACTION)
;
single_rewrite
: #( REWRITE {out(" ->");} (SEMPRED {out(" {"+#SEMPRED.getText()+"}?");})?
( alternative | rewrite_template | ETC {out("...");} | ACTION {out(" {"+#ACTION.getText()+"}");})
)
;
rewrite_template
: #( TEMPLATE
(id:ID {out(" "+#id.getText());}|ind:ACTION {out(" ({"+#ind.getText()+"})");})
#( ARGLIST
{out("(");}
( #( ARG arg:ID {out(#arg.getText()+"=");}
a:ACTION {out(#a.getText());}
)
)*
{out(")");}
)
( DOUBLE_QUOTE_STRING_LITERAL {out(" "+#DOUBLE_QUOTE_STRING_LITERAL.getText());}
| DOUBLE_ANGLE_STRING_LITERAL {out(" "+#DOUBLE_ANGLE_STRING_LITERAL.getText());}
)?
)
;
rewrite
: (single_rewrite)*
;
element
: #(ROOT element)
| #(BANG element)
| atom
| #(NOT {out("~");} element)
| #(RANGE atom {out("..");} atom)
| #(CHAR_RANGE atom {out("..");} atom)
| #(ASSIGN id:ID {out(#id.getText()+"=");} element)
| #(PLUS_ASSIGN id2:ID {out(#id2.getText()+"+=");} element)
| ebnf
| tree
| #( SYNPRED block[true] ) {out("=>");}
| a:ACTION {if ( showActions ) {out("{"); out(a.getText()); out("}");}}
| a2:FORCED_ACTION {if ( showActions ) {out("{{"); out(a2.getText()); out("}}");}}
| pred:SEMPRED
{
if ( showActions ) {out("{"); out(pred.getText()); out("}?");}
else {out("{...}?");}
}
| spred:SYN_SEMPRED
{
String name = spred.getText();
GrammarAST predAST=grammar.getSyntacticPredicate(name);
block(predAST, true);
out("=>");
}
| BACKTRACK_SEMPRED // don't print anything (auto backtrack stuff)
| gpred:GATED_SEMPRED
{
if ( showActions ) {out("{"); out(gpred.getText()); out("}? =>");}
else {out("{...}? =>");}
}
| EPSILON
;
ebnf: block[true] {out(" ");}
| #( OPTIONAL block[true] ) {out("? ");}
| #( CLOSURE block[true] ) {out("* ");}
| #( POSITIVE_CLOSURE block[true] ) {out("+ ");}
;
tree: #(TREE_BEGIN {out(" ^(");} element (element)* {out(") ");} )
;
atom
{out(" ");}
: ( #( RULE_REF {out(#atom.toString());}
(rarg:ARG_ACTION {out("["+#rarg.toString()+"]");})?
(ast_suffix)?
)
| #( TOKEN_REF {out(#atom.toString());}
(targ:ARG_ACTION {out("["+#targ.toString()+"]");} )?
(ast_suffix)?
)
| #( CHAR_LITERAL {out(#atom.toString());}
(ast_suffix)?
)
| #( STRING_LITERAL {out(#atom.toString());}
(ast_suffix)?
)
| #( WILDCARD {out(#atom.toString());}
(ast_suffix)?
)
)
{out(" ");}
| LABEL {out(" $"+#LABEL.getText());} // used in -> rewrites
| #(DOT ID {out(#ID.getText()+".");} atom) // scope override on rule
;
ast_suffix
: ROOT {out("^");}
| BANG {out("!");}
;
antlr-3.2/tool/src/main/resources/ 0000755 0001750 0001750 00000000000 11410174107 017132 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/ 0000755 0001750 0001750 00000000000 11410174107 017721 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/ 0000755 0001750 0001750 00000000000 11410174107 021041 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/ 0000755 0001750 0001750 00000000000 11410174107 022445 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ 0000755 0001750 0001750 00000000000 11410174107 024443 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Ruby/ 0000755 0001750 0001750 00000000000 11410174107 025364 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Ruby/Ruby.stg 0000644 0001750 0001750 00000100340 11256465234 027036 0 ustar twerner twerner /*
[The "BSD license"]
Copyright (c) 2006 Martin Traverso
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group Ruby implements ANTLRCore;
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer, name, tokens, tokenNames, rules,
cyclicDFAs, bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
# <name> (<fileName>)
# Generated by ANTLR <ANTLRVersion> on <generatedTimestamp>
<docComment>
<recognizer>
>>
/**
* Inherits parameters from outputFile(...)
*
* labelType is not used for Ruby (no explicit type declarations)
*/
lexer(grammar, name, tokens, scopes, rules, numRules, labelType, filterMode,
superClass) ::=
<<
class <name>
require 'stringio'
<tokens:{<it.name>=<it.type>}; separator="\n">
def initialize(input)
input = StringIO.new(input) if input.respond_to?(:to_str)
@input = CharStream.new(input)
@backtracking = 0
@failed = false
<actions.lexer.init>
end
def next_token
# TODO: catch exceptions
@token = nil
@channel = nil
@text = nil
@start = @input.index
@line = @input.line
@pos = @input.column
@type = nil
@type_int = nil
return :EOF if <LA(1)> == :EOF
match_Tokens()
if @token == nil
@text ||= @input.substring(@start, @input.index - 1)
@token = Token.new(@type, @type_int, @line, @pos, @text, @channel)
end
<if(trace)>
puts @token.inspect
<endif>
return @token
end
class Token
attr_reader :token_type
attr_reader :int_type
attr_reader :line
attr_reader :pos
attr_reader :text
attr_reader :channel
def initialize(token_type, int_type, line, pos, text, channel = nil)
@token_type = token_type
@int_type = int_type
@line = line
@pos = pos
@text = text
@channel = channel
end
alias :to_i :int_type
end
<actions.lexer.members>
private
class CharStream
attr_reader :line
attr_reader :column
attr_reader :index
def initialize(input)
@buffer = ""
@input = input
@line = 1
@column = 0
@index = 0;
end
# returns a Fixnum between 0 and 0xFFFF or :EOF
def look_ahead(pos)
offset = @index + pos - 1
if @buffer.length \< offset + 1
char = @input.read(offset + 1 - @buffer.length)
@buffer \<\< char if not char.nil?
end
if offset \< @buffer.length
@buffer[offset]
else
:EOF
end
end
def mark
@state = { :index => @index, :line => @line, :column => @column }
return 0
end
def rewind(marker)
@index = @state[:index]
@line = @state[:line]
@column = @state[:column]
end
def consume
look_ahead(1) # force a read from the input if necessary
@column = @column + 1
if @buffer[@index] == ?\n
@line = @line + 1
@column = 0
end
@index = @index + 1
end
def substring(start, stop)
@buffer.slice(start, stop - start + 1)
end
end
def match(value = nil)
@failed = false
case
when value.nil?
@input.consume()
when value.respond_to?(:to_str)
catch(:done) do
value.each_byte do |c|
@failed ||= !(<isolatedLookaheadTest(atom="c", k=1)>)
@input.consume() if !@failed
throw :done if @failed
end
end
else
@failed = !(<isolatedLookaheadTest(atom="value", k=1)>)
@input.consume() if !@failed
end
if @failed && @backtracking \<= 0
raise "Expected #{value.respond_to?(:chr) ? value.chr : value}"
end
end
def match_range(from, to)
char = <LA(1)>
if char != :EOF && (char \>= from || char \<= to)
@failed = false
match()
elsif @backtracking > 0
@failed = true
else
raise "Expected [#{from.chr}..#{to.chr}]"
end
end
<rules; separator="\n\n">
<synpreds: synpred(); separator="\n\n">
<dfaClass()>
<cyclicDFAs: cyclicDFA()>
end
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, ASTLabelType, superClass,
labelType, members) ::=
<<
require '<grammar.name>Lexer'
class <name>
attr_reader :lexer
TOKENS = [
<tokenNames: {[<it>, <i>]}; separator=",\n">
].inject({}) { |hash, pair|
name = pair[0]
index = pair[1] + 3 # hardcoded for now... no way to get this value from ANTLR
if name[0] == ?'
hash[:"T#{index}"] = index
else
hash[:"#{name}"] = index
end
hash
}
TOKENS[:EOF] = -1
def initialize(input)
if input.respond_to?(:to_str) || input.respond_to?(:read)
input = <grammar.name>Lexer.new(input)
end
@lexer = input
@input = TokenStream.new(input)
@backtracking = 0
@failed = false
<actions.parser.init>
<if(trace)>
@indent = 0
<endif>
end
<rules; separator="\n\n">
<actions.parser.members>
private
class TokenStream
attr_reader :index
def initialize(input)
@buffer = []
@input = input
@channel = nil
@index = 0;
end
# returns a Token
def look_ahead(pos)
offset = @index + pos - 1
while @buffer[-1] != :EOF && @buffer.length \< offset + 1
token = @input.next_token
if token == :EOF || token.channel == @channel
@buffer \<\< token
end
end
offset = -1 if offset >= @buffer.length
if offset \< @buffer.length
@buffer[offset]
end
end
def mark
@state = { :index => @index }
return 0
end
def rewind(marker)
@index = @state[:index]
end
def consume
look_ahead(1) # force a read from the input if necessary
@index = @index + 1
end
end
def match(token = nil)
if token.nil? || <LA(1)> == token
@input.consume
@failed = false
return
elsif @backtracking > 0
@failed = true
else
raise "Expected #{token}"
end
end
def look_ahead(k)
token = @input.look_ahead(k)
if token != :EOF
token = token.token_type
end
token
end
<synpreds: synpred(); separator="\n\n">
<dfaClass()>
<cyclicDFAs: cyclicDFA()>
end
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules,
numRules, bitsets, labelType, ASTLabelType,
superClass, members) ::=
<<
raise "treeParser not implemented"
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
# <description>
def <ruleName>_fragment
<block>
end
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::=
<<
# <description>
def <ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>)
<ruleDescriptor.returnScope.attributes:
{ _retval_<it.name> = nil }; separator = "\n"
>
<ruleLabelDefs()>
<if(trace)>
puts " " * @indent + "+<ruleName>"
@indent += 1
<endif>
<ruleDescriptor.actions.init>
<block>
<if(trace)>
@indent -= 1
puts " " * @indent + "-<ruleName>"
<endif>
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasMultipleReturnValues)>
return {<ruleDescriptor.returnScope.attributes:{ a | :<a.name> => _retval_<a.name> }; separator = ",">}
# TODO: need "Attribute.index" for this to work: return <ruleDescriptor.returnScope.attributes:{ a | _retval_<a.name> }; separator = ",">
<else>
return _retval_<ruleDescriptor.singleValueReturnName>
<endif>
<endif>
<endif>
end
>>
ruleLabelDefs() ::= <<
<ruleDescriptor.tokenLabels :{_<it.label.text> = nil}; separator="\n">
<[ruleDescriptor.tokenListLabels, ruleDescriptor.ruleListLabels]
:{list_<it.label.text> = nil}; separator="\n"
>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::=
<<
def match_<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>)
<ruleDescriptor.actions.init>
<lexerRuleLabelDefs()>
<if(nakedBlock)>
<block><\n>
<else>
@type = :<ruleName>
@type_int = <ruleName>
<block>
<endif>
end
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels, ruleDescriptor.ruleLabels]
:{<it.label.text> = nil}; separator="\n"
>
<ruleDescriptor.charLabels:{<it.label.text> = nil}; separator="\n">
<[ruleDescriptor.tokenListLabels, ruleDescriptor.ruleListLabels]
:{list_<it.label.text> = nil}; separator="\n"
>
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::=
<<
<lexerRule(...)>
>>
filteringNextToken() ::=
<<
raise "filteringNextToken not implemented"
>>
filteringActionGate() ::=
<<
raise "filteringActionGate not implemented"
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::=
<<
<switchBlock(...)>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::=
<<
<switchBlock(...)>
>>
/**
* decision, decisionNumber don't seem to be relevant in this template
* alts actually has a single element
*/
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::=
<<
<plainBlock(...)>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::=
<<
<plainBlock(...)>
>>
/** A (..)+ block with 0 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::=
<<
# <description>
matchedOnce<decisionNumber> = false
<decls>
while true
alt<decisionNumber> = <maxAlt>
<decision>
case alt<decisionNumber>
<alts:switchCase(); separator="\n">
else
break
end
matchedOnce<decisionNumber> = true
end
if !matchedOnce<decisionNumber>
raise "Expected at least one match: <description>"
end
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 0 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::=
<<
# <description>
<decls>
while true
alt<decisionNumber> = <maxAlt>
<decision>
case alt<decisionNumber>
<alts:switchCase(); separator="\n">
else
break
end
end
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::=
<<
# <description>
<elements: element(); separator="\n">
>>
// E L E M E N T S
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::=
<<
<if(label)>
_<label> = @input.look_ahead(1)<\n>
<endif>
match(:<token>)
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::=
<<
<tokenRef(...)>
<listLabel(elem=label, ...)>
>>
listLabel(label,elem)::=
<<
list_<label> ||= []
list_<label> \<\< _<elem>
>>
/** match a character */
charRef(char,label)::=
<<
<if(label)>
_<label> = <char><\n>
<endif>
match(<char>)
>>
/** match a character range */
charRangeRef(a,b,label)::=
<<
<if(label)>
_<label> = <LA(1)><\n>
<endif>
match_range(<a>, <b>)
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode)::=
<<
<if(label)>
_<label> = <LA(1)><\n>
<endif>
if <s>
match()
<postmatchCode>
else
raise "Expected set"
end
>>
matchSetAndListLabel(s,label,elementIndex,postmatchCode)::=
<<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label)::=
<<
<if(label)>
_<label> = <string><\n>
<endif>
match(<string>)<\n>
>>
wildcard(label, elementIndex)::=
<<
<if(label)>
_<label> = <LA(1)><\n>
<endif>
match()
>>
wildcardAndListLabel(label,elementIndex)::=
<<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex)::=
<<
<if(label)>
_<label> = <LA(1)><\n>
<endif>
match()
>>
wildcardCharListLabel(label, elementIndex)::=
<<
raise "wildcardCharListLabel not implemented"
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values.
*/
ruleRef(rule,label,elementIndex,args,scope) ::=
<<
<if(label)>
_<label> = <rule>(<args; separator=", ">)<\n>
<else>
<rule>(<args; separator=", ">)<\n>
<endif>
>>
/** ids+=ID */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::=
<<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/**
A: b=B;
B: .;
TODO: Should we use a real token type instead of :invalid? How do we get it?
*/
lexerRuleRef(rule,label,args,elementIndex,scope)::=
<<
<if(label)>
_<label>_start_<elementIndex> = @input.index
_<label>_line_<elementIndex> = @input.line
_<label>_pos_<elementIndex> = @input.column
match_<rule>(<args; separator=", ">)
_<label> = Token.new(:invalid, 0,
_<label>_line_<elementIndex>,
_<label>_pos_<elementIndex>,
@input.substring(_<label>_start_<elementIndex>, @input.index - 1), nil)
<else>
match_<rule>(<args; separator=", ">)
<endif>
>>
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::=
<<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex)::=
<<
<if(label)>
_<label> = :EOF<\n>
<endif>
match(:EOF)
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList, enclosingTreeLevel,
treeLevel) ::=
<<
raise "tree not implemented"
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description)::=
<<
# <description>
if !<evalPredicate(...)>
raise "Semantic predicate failed: #{<description>}"
end
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState)::=
<<
# <description>
look_ahead<decisionNumber>_<stateNumber> = <LA(k)>
<if(LEXER)>
look_ahead<decisionNumber>_<stateNumber> = -1 if look_ahead<decisionNumber>_<stateNumber> == :EOF
<endif>
if <edges; separator="\nelsif ">
else
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt><\n>
<else>
raise "Expected: <description>"<\n>
<endif>
end
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*
* If a semPredState, don't force lookahead lookup; preds might not
* need.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState)::=
<<
# <description>
look_ahead<decisionNumber>_<stateNumber> = <LA(k)>
<if(LEXER)>
look_ahead<decisionNumber>_<stateNumber> = -1 if look_ahead<decisionNumber>_<stateNumber> == :EOF
<endif>
if <edges; separator="\nelsif ">
end
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a
* rule anything other than 'a' predicts exiting.
*
* If a semPredState, don't force lookahead lookup; preds might not
* need.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState)::=
<<
# <description>
look_ahead<decisionNumber>_<stateNumber> = <LA(k)>
<if(LEXER)>
look_ahead<decisionNumber>_<stateNumber> = -1 if look_ahead<decisionNumber>_<stateNumber> == :EOF
<endif>
if <edges; separator="\nelsif ">
<if(eotPredictsAlt)>
else
alt<decisionNumber> = <eotPredictsAlt>
<endif>
end
>>
/** An accept state indicates a unique alternative has been predicted */
/** It is not clear that decisionNumber is available here */
dfaAcceptState(alt) ::= "alt<decisionNumber> = <alt>"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates)::=
<<
<labelExpr> <if(predicates)>&& <predicates><endif>
<targetState>
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState)::=
<<
# <description>
case <LA(k)>
<edges; separator="\n">
else
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt><\n>
<else>
raise "Expected: <description>"<\n>
<endif>
end
>>
/**
* eotPredictsAlt is not relevant here
*/
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState)::=
<<
# <description>
case <LA(k)>
<edges; separator="\n">
end
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState)::=
<<
# <description>
case <LA(k)>
<edges; separator="\n">
<if(eotPredictsAlt)><\n>
else
alt<decisionNumber> = <eotPredictsAlt>
<endif>
end
>>
dfaEdgeSwitch(labels, targetState)::=
<<
<if(PARSER)>
when <labels:{:<it>}; separator=","><\n>
<else>
when <labels:{<it>}; separator=","><\n>
<endif>
<targetState>
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description)::=
<<
alt<decisionNumber> = DFA<decisionNumber>.predict(self, @input)
>>
/** Generate the tables and support code needed for the DFAState object
* argument. Unless there is a semantic predicate (or syn pred, which
* become sem preds), all states should be encoded in the state tables.
* Consequently, cyclicDFAState/cyclicDFAEdge,eotDFAEdge templates are
* not used except for special DFA states that cannot be encoded as
* a transition table.
*/
cyclicDFA(dfa)::=
<<
DFA<dfa.decisionNumber> = DFA.new(
[<dfa.eot; wrap="\n ", separator=",", null="-1">],
[<dfa.eof; wrap="\n ", separator=",", null="-1">],
[<dfa.min; wrap="\n ", separator=",", null="0">],
[<dfa.max; wrap="\n ", separator=",", null="0">],
[<dfa.accept; wrap="\n ", separator=",", null="-1">],
[<dfa.special; wrap="\n ", separator=",", null="-1">],
[
<dfa.transition:{s | [<s; wrap="\n ", separator=",", null="-1">]}; separator=",\n", null="">
])
def special_state_transition(s)
<if(dfa.specialStateSTs)>
case s
<dfa.specialStateSTs:{state |
when <i0>
<state>}; separator="\n">
end
raise "Expected: <dfa.description>"
<else>
-1
<endif>
end
public :special_state_transition
>>
/** A special state in a cyclic DFA; special means has a semantic predicate
* or it's a huge set of symbols to check.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState)::=
<<
<if(semPredState)>
@input.rewind(0)
<else>
look_ahead_<decisionNumber>_<stateNumber> = <LA(1)>
<endif>
s = -1
<edges>
return s if s >= 0
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful. Again, this is for special
* states.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates)::=
<<
return s = <targetStateNumber> if (<labelExpr>) <if(predicates)>&& (<predicates>)<endif><\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates)::=
<<
s = <targetStateNumber><\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right)::= "(<left> && <right>)"
orPredicates(operands)::=
<<
(<operands; separator=" || ">)
>>
notPredicate(pred)::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description)::= "(<pred>)"
evalSynPredicate(pred,description)::= "<pred>()"
/**
* It's not really clear that decisionNumber and stateNumber are available here
*/
lookaheadTest(atom,k,atomAsInt)::=
<<
<if(LEXER)>
look_ahead<decisionNumber>_<stateNumber> == <atom>
<else>
look_ahead<decisionNumber>_<stateNumber> == :<atom>
<endif>
>>
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::=
<<
<if(LEXER)>
<LA(k)> == <atom>
<else>
<LA(k)> == :<atom>
<endif>
>>
/**
* It's not really clear that decisionNumber and stateNumber are available here
*/
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt)::=
<<
<if(LEXER)>
(look_ahead<decisionNumber>_<stateNumber> \>= <lower> && look_ahead<decisionNumber>_<stateNumber> \<= <upper>)
<else>
(TOKENS[look_ahead<decisionNumber>_<stateNumber>] \>= <lowerAsInt> && TOKENS[look_ahead<decisionNumber>_<stateNumber>] \<= <upperAsInt>)
<endif>
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::=
<<
<if(LEXER)>
(<LA(k)> \>= <lower> && <LA(k)> \<= <upper>)
<else>
(TOKENS[<LA(k)>] \>= <lowerAsInt> && TOKENS[<LA(k)>] \<= <upperAsInt>)
<endif>
>>
setTest(ranges) ::=
<<
<ranges; separator=" || ">
>>
// A T T R I B U T E S
parameterAttributeRef(attr)::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> = <expr>"
scopeAttributeRef(scope,attr,index,negIndex)::=
<<
raise "scopeAttributeRef not implemented"
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::=
<<
raise "scopeSetAttributeRef not implemented"
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope)::=
<<
raise "isolatedDynamicScopeRef not implemented"
>>
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr)::=
<<
<if(referencedRule.hasMultipleReturnValues)>
_<scope>[:<attr.name>]
<else>
_<scope>
<endif>
>>
/**
<if(referencedRule.hasMultipleReturnValues)>
<scope>[<attr.index>] # TODO: need "Attribute.index" for this to work
<else>
<scope>
<endif>
>>
**/
returnAttributeRef(ruleDescriptor,attr)::=
<<
_retval_<attr.name>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::=
<<
_retval_<attr.name> = <expr>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label)::= "_<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label)::= "list_<label>"
// not sure the next are the right approach; and they are evaluated early;
// they cannot see TREE_PARSER or PARSER attributes for example. :(
tokenLabelPropertyRef_text(scope,attr)::= "_<scope>.text"
tokenLabelPropertyRef_type(scope,attr)::= "_<scope>.token_type"
tokenLabelPropertyRef_line(scope,attr)::= "_<scope>.line"
tokenLabelPropertyRef_pos(scope,attr) ::= "_<scope>.pos"
tokenLabelPropertyRef_channel(scope,attr)::= "_<scope>.channel"
tokenLabelPropertyRef_index(scope,attr)::= "_<scope>.index"
tokenLabelPropertyRef_tree(scope,attr)::= <<
raise "tokenLabelPropertyRef_tree not implemented"
>>
ruleLabelPropertyRef_start(scope,attr)::=
<<
raise "ruleLabelPropertyRef_start not implemented"
>>
ruleLabelPropertyRef_stop(scope,attr)::=
<<
raise "ruleLabelPropertyRef_stop not implemented"
>>
ruleLabelPropertyRef_tree(scope,attr)::=
<<
raise "ruleLabelPropertyRef_tree not implemented"
>>
ruleLabelPropertyRef_text(scope,attr)::=
<<
raise "ruleLabelPropertyRef_text not implemented"
>>
ruleLabelPropertyRef_st(scope,attr)::=
<<
raise "ruleLabelPropertyRef_st not implemented"
>>
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label)::=
<<
raise "lexerRuleLabel not implemented"
>>
lexerRuleLabelPropertyRef_type(scope,attr)::=
<<
raise "lexerRuleLabelPropertyRef_type not implemented"
>>
lexerRuleLabelPropertyRef_line(scope,attr)::=
<<
raise "lexerRuleLabelPropertyRef_line not implemented"
>>
lexerRuleLabelPropertyRef_pos(scope,attr)::=
<<
raise "lexerRuleLabelPropertyRef_pos not implemented"
>>
lexerRuleLabelPropertyRef_channel(scope,attr)::=
<<
raise "lexerRuleLabelPropertyRef_channel not implemented"
>>
lexerRuleLabelPropertyRef_index(scope,attr)::=
<<
raise "lexerRuleLabelPropertyRef_index not implemented"
>>
lexerRuleLabelPropertyRef_text(scope,attr)::=
<<
raise "lexerRuleLabelPropertyRef_text not implemented"
>>
lexerRulePropertyRef_text(scope,attr) ::= "@text"
lexerRulePropertyRef_type(scope,attr) ::= <<
raise "lexerRulePropertyRef_type not implemented"
>>
lexerRulePropertyRef_line(scope,attr) ::= "@line"
lexerRulePropertyRef_pos(scope,attr) ::= "@pos"
lexerRulePropertyRef_index(scope,attr) ::= <<
raise "lexerRulePropertyRef_index not implemented"
>>
lexerRulePropertyRef_channel(scope,attr) ::= "@channel"
lexerRulePropertyRef_start(scope,attr) ::= "@start"
lexerRulePropertyRef_stop(scope,attr) ::= <<
raise "lexerRulePropertyRef_stop not implemented"
>>
ruleSetPropertyRef_tree(scope,attr,expr) ::= <<
raise "ruleSetPropertyRef_tree not implemented"
>>
ruleSetPropertyRef_st(scope,attr,expr) ::= <<
raise "ruleSetPropertyRef_st not implemented"
>>
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr)::=
<<
raise "rulePropertyRef_start not implemented"
>>
rulePropertyRef_stop(scope,attr)::=
<<
raise "rulePropertyRef_stop not implemented"
>>
rulePropertyRef_tree(scope,attr)::=
<<
raise "rulePropertyRef_tree not implemented"
>>
rulePropertyRef_text(scope,attr)::=
<<
raise "rulePropertyRef_text not implemented"
>>
rulePropertyRef_st(scope,attr)::=
<<
raise "rulePropertyRef_st not implemented"
>>
/** How to execute an action */
/** TODO: add syntactic predicate & bactracking gates **/
execAction(action)::=
<<
<action>
>>
// M I S C (properties, etc...)
codeFileExtension()::=".rb"
true()::= "true"
false()::= "false"
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
///////////// --------------------------- private templates --------------------------------
bitset()::=
<<
raise "bitset not implemented"
>>
element() ::= "<it.el>"
plainBlock(decls, alts, description) ::=
<<
<decls>
<alts>
>>
switchBlock(description, decisionNumber, maxAlt, alts, decls, decision) ::=
<<
# <description>
alt<decisionNumber> = <maxAlt>
<decls>
<decision>
case alt<decisionNumber>
<alts:switchCase(); separator="\n">
end
>>
switchCase() ::=
<<
when <i>
<it>
>>
LA(k) ::=
<<
<if(LEXER)>
@input.look_ahead(<k>)
<else>
look_ahead(<k>)
<endif>
>>
synpred(name) ::= <<
def <name>
start = @input.mark()
@backtracking += 1
<name>_fragment()
@backtracking -= 1
success = !@failed
@input.rewind(start)
@failed = false
return success
end
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
dfaClass() ::= <<
<if(cyclicDFAs)>
class DFA
def initialize(eot, eof, min, max, accept, special, transition)
@eot = eot
@eof = eof
@min = min
@max = max
@accept = accept
@special = special
@transition = transition
end
def predict(parser, input)
mark = input.mark()
s = 0 # we always start at s0
begin
loop do
special_state = @special[s]
if special_state >= 0
s = parser.special_state_transition(special_state)
input.consume()
next
end
if @accept[s] >= 1
return @accept[s]
end
# look for a normal char transition
c = input.look_ahead(1).to_i
if c != :EOF && c >= @min[s] && c \<= @max[s]
next_state = @transition[s][c - @min[s]] # move to next state
if next_state \< 0
# was in range but not a normal transition
# must check EOT, which is like the else clause.
# eot[s]>=0 indicates that an EOT edge goes to another
# state.
if @eot[s] >= 0 # EOT Transition to accept state?
s = @eot[s]
input.consume()
next
end
raise "No viable alt"
end
s = next_state
input.consume()
next
end
if @eot[s] >= 0 # EOT Transition?
s = @eot[s]
input.consume()
next
end
if c == :EOF && @eof[s] >= 0 # EOF Transition to accept state?
return @accept[@eof[s]]
end
# not in range and not EOF/EOT, must be invalid symbol
raise "No viable alt"
end
ensure
input.rewind(mark)
end
end
end
<endif>
>> antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ActionScript/ 0000755 0001750 0001750 00000000000 11410174107 027045 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ActionScript/ActionScript.stg 0000644 0001750 0001750 00000122147 11256465232 032207 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group ActionScript implements ANTLRCore;
asTypeInitMap ::= [
"int":"0",
"uint":"0",
"Number":"0.0",
"Boolean":"false",
default:"null" // anything other than an atomic type
]
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
package<if(actions.(actionScope).package)> <actions.(actionScope).package><endif> {
<actions.(actionScope).header>
<@imports>
import org.antlr.runtime.*;
<if(TREE_PARSER)>
import org.antlr.runtime.tree.*;
<endif>
<@end>
<docComment>
<recognizer>
}
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="Token",
filterMode, superClass="Lexer") ::= <<
public class <grammar.recognizerName> extends <if(actions.(actionScope).superClass)><actions.(actionScope).superClass><else><@superClassName><superClass><@end><endif> {
<tokens:{public static const <it.name>:int=<it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
// delegates
<grammar.delegates:
{g|public var <g:delegateName()>:<g.recognizerName>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public var <g:delegateName()>:<g.recognizerName>;}; separator="\n">
<last(grammar.delegators):{g|public var gParent:<g.recognizerName>;}>
public function <grammar.recognizerName>(<grammar.delegators:{g|<g:delegateName()>:<g.recognizerName>, }>input:CharStream = null, state:RecognizerSharedState = null) {
super(input, state);
<cyclicDFAs:cyclicDFACtor()>
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = new Array(<numRules>+1);<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(<trunc(g.delegators):{p|<p:delegateName()>, }>this, input, this.state);}; separator="\n">
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
public override function get grammarFileName():String { return "<fileName>"; }
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<synpreds:{p | <lexerSynpred(p)>}>
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
}
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
public override function nextToken():Token {
while (true) {
if ( input.LA(1)==CharStreamConstants.EOF ) {
return TokenConstants.EOF_TOKEN;
}
this.state.token = null;
this.state.channel = TokenConstants.DEFAULT_CHANNEL;
this.state.tokenStartCharIndex = input.index;
this.state.tokenStartCharPositionInLine = input.charPositionInLine;
this.state.tokenStartLine = input.line;
this.state.text = null;
try {
var m:int = input.mark();
this.state.backtracking=1; <! means we won't throw slow exception !>
this.state.failed=false;
mTokens();
this.state.backtracking=0;
<! mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1. !>
if ( this.state.failed ) {
input.rewindTo(m);
input.consume(); <! advance one char and try again !>
}
else {
emit();
return this.state.token;
}
}
catch (re:RecognitionException) {
// shouldn't happen in backtracking mode, but...
reportError(re);
recover(re);
}
}
// Not reached - For ActionScript compiler
throw new Error();
}
public override function memoize(input:IntStream,
ruleIndex:int,
ruleStartIndex:int):void
{
if ( this.state.backtracking>1 ) super.memoize(input, ruleIndex, ruleStartIndex);
}
public override function alreadyParsedRule(input:IntStream, ruleIndex:int):Boolean {
if ( this.state.backtracking>1 ) return super.alreadyParsedRule(input, ruleIndex);
return false;
}
>>
actionGate() ::= "this.state.backtracking==0"
filteringActionGate() ::= "this.state.backtracking==1"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass, filterMode,
ASTLabelType="Object", labelType, members, rewriteElementType) ::= <<
public class <grammar.recognizerName> extends <if(actions.(actionScope).superClass)><actions.(actionScope).superClass><else><@superClassName><superClass><@end><endif> {
<if(grammar.grammarIsRoot)>
public static const tokenNames:Array = [
"\<invalid>", "\<EOR>", "\<DOWN>", "\<UP>", <tokenNames; separator=", ">
];<\n>
<endif>
<tokens:{public static const <it.name>:int=<it.type>;}; separator="\n">
// delegates
<grammar.delegates:
{g|public var <g:delegateName()>:<g.recognizerName>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public var <g:delegateName()>:<g.recognizerName>;}; separator="\n">
<last(grammar.delegators):{g|public var gParent:<g.recognizerName>;}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<@members>
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
public function <grammar.recognizerName>(<grammar.delegators:{g|<g:delegateName()>:<g.recognizerName>, }>input:<inputStreamType>, state:RecognizerSharedState = null) {
super(input, state);
<cyclicDFAs:cyclicDFACtor()>
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(<trunc(g.delegators):{p|<p:delegateName()>, }>this, input, this.state);}; separator="\n">
<grammar.indirectDelegates:{g | <g:delegateName()> = <g.delegator:delegateName()>.<g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
<@end>
public override function get tokenNames():Array { return <grammar.composite.rootGrammar.recognizerName>.tokenNames; }
public override function get grammarFileName():String { return "<fileName>"; }
<members>
<rules; separator="\n\n">
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
// Delegated rules
<grammar.delegatedRules:{ruleDescriptor|
public function <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>):<returnType()> \{ <if(ruleDescriptor.hasReturnValue)>return <endif><ruleDescriptor.grammar:delegateName()>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">); \}}; separator="\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
}
>>
parserCtorBody() ::= <<
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = new Array(<length(grammar.allImportedRules)>+1);<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType="Object", superClass="Parser", labelType="Token", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="TokenStream", rewriteElementType="Token", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="Object", superClass="TreeParser", members={<actions.treeparser.members>}, filterMode) ::= <<
<genericParser(inputStreamType="TreeNodeStream", rewriteElementType="Node", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start <ruleName>
public final function <ruleName>_fragment(<ruleDescriptor.parameterScope:parameterScope(scope=it)>):void {
<ruleLabelDefs()>
<if(trace)>
traceIn("<ruleName>_fragment", <ruleDescriptor.index>);
try {
<block>
}
finally {
traceOut("<ruleName>_fragment", <ruleDescriptor.index>);
}
<else>
<block>
<endif>
}
// $ANTLR end <ruleName>
>>
synpred(name) ::= <<
public final function <name>():Boolean {
this.state.backtracking++;
<@start()>
var start:int = input.mark();
try {
<name>_fragment(); // can never throw exception
} catch (re:RecognitionException) {
trace("impossible: "+re);
}
var success:Boolean = !this.state.failed;
input.rewindTo(start);
<@stop()>
this.state.backtracking--;
this.state.failed=false;
return success;
}<\n>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( this.state.backtracking>0 && alreadyParsedRule(input, <ruleDescriptor.index>) ) { return <ruleReturnValue()>; }
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>if (this.state.failed) return <ruleReturnValue()>;<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if (this.state.backtracking>0) {this.state.failed=true; return <ruleReturnValue()>;}<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
// $ANTLR start <ruleName>
// <fileName>:<description>
public final function <ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>):<returnType()> {
<if(trace)>traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try {
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
catch (re:RecognitionException) {
reportError(re);
recoverStream(input,re);
<@setErrorReturnValue()>
}<\n>
<endif>
<endif>
<endif>
finally {
<if(trace)>traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
}
<@postamble()>
return <ruleReturnValue()>;
}
// $ANTLR end <ruleName>
>>
catch(decl,action) ::= <<
catch (<e.decl>) {
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
var retval:<returnType()> = new <returnType()>();
retval.start = input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
var <a.name>:<a.type> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
var <ruleDescriptor.name>_StartIndex:int = input.index;
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.push(new Object());}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.push(new Object());}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.pop();}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels,
ruleDescriptor.wildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{var <it.label.text>:<labelType>=null;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels,ruleDescriptor.wildcardTreeListLabels]
:{var list_<it.label.text>:Array=null;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|var <ll.label.text>:RuleReturnScope = null;}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{var <it.label.text>:<labelType>=null;}; separator="\n"
>
<ruleDescriptor.charLabels:{var <it.label.text>:int;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels]
:{var list_<it.label.text>:Array=null;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
retval.stop = input.LT(-1);<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( this.state.backtracking>0 ) { memoize(input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex); }
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// $ANTLR start <ruleName>
public final function m<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>):void {
<if(trace)>traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
try {
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
var _type:int = <ruleName>;
var _channel:int = DEFAULT_TOKEN_CHANNEL;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
this.state.type = _type;
this.state.channel = _channel;
<(ruleDescriptor.actions.after):execAction()>
<endif>
}
finally {
<if(trace)>traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeCleanUp()>
<memoize()>
}
}
// $ANTLR end <ruleName>
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
public override function mTokens():void {
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
var alt<decisionNumber>:int=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
var alt<decisionNumber>:int=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
var cnt<decisionNumber>:int=0;
<decls>
<@preloop()>
loop<decisionNumber>:
do {
var alt<decisionNumber>:int=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
if ( cnt<decisionNumber> >= 1 ) break loop<decisionNumber>;
<ruleBacktrackFailure()>
throw new EarlyExitException(<decisionNumber>, input);
<! Need to add support for earlyExitException debug hook !>
}
cnt<decisionNumber>++;
} while (true);
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
<decls>
<@preloop()>
loop<decisionNumber>:
do {
var alt<decisionNumber>:int=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
break loop<decisionNumber>;
}
} while (true);
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> :
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
// <fileName>:<description>
{
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
}
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label>=<labelType>(<endif>matchStream(input,<token>,FOLLOW_<token>_in_<ruleName><elementIndex>)<if(label)>)<endif>; <checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label>==null) list_<label>=new Array();
list_<label>.push(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
matchRange(<a>,<b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= input.LA(1);<\n>
<else>
<label>=<labelType>(input.LT(1));<\n>
<endif>
<endif>
if ( <s> ) {
input.consume();
<postmatchCode>
<if(!LEXER)>
this.state.errorRecovery=false;
<endif>
<if(backtracking)>this.state.failed=false;<endif>
}
else {
<ruleBacktrackFailure()>
<@mismatchedSetException()>
<if(LEXER)>
throw recover(new MismatchedSetException(null,input));<\n>
<else>
throw new MismatchedSetException(null,input);
<! use following code to make it recover inline; remove throw mse;
recoverFromMismatchedSet(input,mse,FOLLOW_set_in_<ruleName><elementIndex>);
!>
<endif>
}<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
var <label>Start:int = charIndex;
matchString(<string>); <checkRuleBacktrackFailure()>
<label> = CommonToken.createFromStream(input, TokenConstants.INVALID_TOKEN_TYPE, TokenConstants.DEFAULT_CHANNEL, <label>Start, charIndex-1);
<else>
matchString(<string>); <checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label>=<labelType>(input.LT(1));<\n>
<endif>
matchAny(input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
matchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*
* GMS: Note: do not use post-decrement operator! ASC produces bad code for exceptions in this case.
* See: https://bugs.adobe.com/jira/browse/ASC-3625
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
pushFollow(FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)><label>=<endif><if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
state._fsp = state._fsp - 1;
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference.
*
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
var <label>Start<elementIndex>:int = charIndex;
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> = CommonToken.createFromStream(input, TokenConstants.INVALID_TOKEN_TYPE, TokenConstants.DEFAULT_CHANNEL, <label>Start<elementIndex>, charIndex-1);
<else>
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
var <label>Start<elementIndex>:int = charIndex;
match(EOF); <checkRuleBacktrackFailure()>
var <label>:<labelType> = CommonToken.createFromStream(input, EOF, TokenConstants.DEFAULT_CHANNEL, <label>Start<elementIndex>, charIndex-1);
<else>
match(EOF); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==TokenConstants.DOWN ) {
matchStream(input, TokenConstants.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
matchStream(input, TokenConstants.UP, null); <checkRuleBacktrackFailure()>
}
<else>
matchStream(input, TokenConstants.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
matchStream(input, TokenConstants.UP, null); <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) ) {
<ruleBacktrackFailure()>
throw new FailedPredicateException(input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber>:int = input.LA(<k>);<\n>
<edges; separator="\nelse ">
else {
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
throw new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<! Need to add hook for noViableAltException() !>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber>:int = input.LA(<k>);<\n>
<edges; separator="\nelse ">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber>:int = input.LA(<k>);<\n>
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber>=<eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else {
alt<decisionNumber>=<eotPredictsAlt>;
}<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber>=<alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
throw new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<! Need to add hook for noViableAltException !>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber>=<eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
{
<targetState>
}
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = dfa<decisionNumber>.predict(input);
>>
cyclicDFACtor(dfa) ::= <<
dfa<dfa.decisionNumber> = new DFA(this, <dfa.decisionNumber>,
"<dfa.description>",
DFA<dfa.decisionNumber>_eot, DFA<dfa.decisionNumber>_eof, DFA<dfa.decisionNumber>_min,
DFA<dfa.decisionNumber>_max, DFA<dfa.decisionNumber>_accept, DFA<dfa.decisionNumber>_special,
DFA<dfa.decisionNumber>_transition<if(dfa.specialStateSTs)>, DFA<dfa.decisionNumber>_specialStateTransition<endif>);
>>
/* Dump DFA tables as run-length-encoded Strings of octal values.
* Can't use hex as compiler translates them before compilation.
* These strings are split into multiple, concatenated strings.
* Java puts them back together at compile time thankfully.
* Java cannot handle large static arrays, so we're stuck with this
* encode/decode approach. See analysis and runtime DFA for
* the encoding methods.
*/
cyclicDFA(dfa) ::= <<
private const DFA<dfa.decisionNumber>_eot:Array =
DFA.unpackEncodedString("<dfa.javaCompressedEOT; wrap="\"+\n \"">");
private const DFA<dfa.decisionNumber>_eof:Array =
DFA.unpackEncodedString("<dfa.javaCompressedEOF; wrap="\"+\n \"">");
private const DFA<dfa.decisionNumber>_min:Array =
DFA.unpackEncodedString("<dfa.javaCompressedMin; wrap="\"+\n \"">", true);
private const DFA<dfa.decisionNumber>_max:Array =
DFA.unpackEncodedString("<dfa.javaCompressedMax; wrap="\"+\n \"">", true);
private const DFA<dfa.decisionNumber>_accept:Array =
DFA.unpackEncodedString("<dfa.javaCompressedAccept; wrap="\"+\n \"">");
private const DFA<dfa.decisionNumber>_special:Array =
DFA.unpackEncodedString("<dfa.javaCompressedSpecial; wrap="\"+\n \"">");
private const DFA<dfa.decisionNumber>_transition:Array = [
<dfa.javaCompressedTransition:{s|DFA.unpackEncodedString("<s; wrap="\"+\n\"">")}; separator=",\n">
];
<if(dfa.specialStateSTs)>
private function DFA<dfa.decisionNumber>_specialStateTransition(dfa:DFA, s:int, _input:IntStream):int {
<if(LEXER)>
var input:IntStream = _input;
<endif>
<if(PARSER)>
var input:TokenStream = TokenStream(_input);
<endif>
<if(TREE_PARSER)>
var input:TreeNodeStream = TreeNodeStream(_input);
<endif>
var _s:int = s;
switch ( s ) {
<dfa.specialStateSTs:{state |
case <i0> : <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
<if(backtracking)>
if (this.state.backtracking>0) {this.state.failed=true; return -1;}<\n>
<endif>
throw dfa.error(new NoViableAltException(dfa.description, <dfa.decisionNumber>, _s, input));
}<\n>
<endif>
protected var dfa<dfa.decisionNumber>:DFA; // initialized in constructor
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber>:int = input.LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
var index<decisionNumber>_<stateNumber>:int = input.index;
input.rewind();<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
<if(semPredState)> <! return input cursor to state before we rewound !>
input.seek(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s>=0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {s = <targetStateNumber>;}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left>&&<right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | ||<o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber>==<atomAsInt>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "input.LA(<k>)==<atomAsInt>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber>\>=<lowerAsInt> && LA<decisionNumber>_<stateNumber>\<=<upperAsInt>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(input.LA(<k>)\>=<lowerAsInt> && input.LA(<k>)\<=<upperAsInt>)"
setTest(ranges) ::= "<ranges; separator=\"||\">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected var <scope.name>_stack:Array = new Array();<\n>
<endif>
>>
ruleAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected var <scope.name>_stack:Array = new Array();<\n>
<endif>
>>
returnStructName() ::= "<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope"
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnStructName()>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Generate the Java type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<returnStructName()>
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
<asTypeInitMap.(typeName)>
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
var <label.label.text>:<ruleLabelType(referencedRule=label.referencedRule)> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
public static class <returnType()> extends <if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope {
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
};
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.name>:<it.type>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> =<expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
<scope>_stack[<scope>_stack.length-<negIndex>-1].<attr.name>
<else>
<if(index)>
<scope>_stack[<index>].<attr.name>
<else>
<scope>_stack[<scope>_stack.length-1].<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
<scope>_stack[<scope>_stack.length-<negIndex>-1].<attr.name> =<expr>;
<else>
<if(index)>
<scope>_stack[<index>].<attr.name> =<expr>;
<else>
<scope>_stack[<scope>_stack.length-1].<attr.name> =<expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
(<scope>!=null?<scope>.values.<attr.name>:<initValue(attr.type)>)
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.values.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.values.<attr.name> =<expr>;
<else>
<attr.name> =<expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "(<scope>!=null?<scope>.text:null)"
tokenLabelPropertyRef_type(scope,attr) ::= "(<scope>!=null?<scope>.type:0)"
tokenLabelPropertyRef_line(scope,attr) ::= "(<scope>!=null?<scope>.line:0)"
tokenLabelPropertyRef_pos(scope,attr) ::= "(<scope>!=null?<scope>.charPositionInLine:0)"
tokenLabelPropertyRef_channel(scope,attr) ::= "(<scope>!=null?<scope>.channel:0)"
tokenLabelPropertyRef_index(scope,attr) ::= "(<scope>!=null?<scope>.tokenIndex:0)"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
tokenLabelPropertyRef_int(scope,attr) ::= "(<scope>!=null?int(<scope>.text):0)"
ruleLabelPropertyRef_start(scope,attr) ::= "(<scope>!=null?<labelType>(<scope>.start):null)"
ruleLabelPropertyRef_stop(scope,attr) ::= "(<scope>!=null?<labelType>(<scope>.stop):null)"
ruleLabelPropertyRef_tree(scope,attr) ::= "(<scope>!=null?<ASTLabelType>(<scope>.tree):null)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
(<scope>!=null?(input.tokenStream.toStringWithRange(
input.treeAdaptor.getTokenStartIndex(<scope>.start),
input.treeAdaptor.getTokenStopIndex(<scope>.start))):null)
<else>
(<scope>!=null?input.toStringWithRange(<scope>.start,<scope>.stop):null)
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "(<scope>!=null?<scope>.st:null)"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::=
"(<scope>!=null?<scope>.type:0)"
lexerRuleLabelPropertyRef_line(scope,attr) ::=
"(<scope>!=null?<scope>.lien:0)"
lexerRuleLabelPropertyRef_pos(scope,attr) ::=
"(<scope>!=null?<scope>.charPositionInLine:0)"
lexerRuleLabelPropertyRef_channel(scope,attr) ::=
"(<scope>!=null?<scope>.channel:0)"
lexerRuleLabelPropertyRef_index(scope,attr) ::=
"(<scope>!=null?<scope>.tokenIndex:0)"
lexerRuleLabelPropertyRef_text(scope,attr) ::=
"(<scope>!=null?<scope>.text:null)"
lexerRuleLabelPropertyRef_int(scope,attr) ::=
"(<scope>!=null?int(<scope>.text):0)"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "<labelType>(retval.start)"
rulePropertyRef_stop(scope,attr) ::= "<labelType>(retval.stop)"
rulePropertyRef_tree(scope,attr) ::= "<ASTLabelType>(retval.tree)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
input.tokenStream.toStringWithRange(
input.treeAdaptor.getTokenStartIndex(retval.start),
input.treeAdaptor.getTokenStopIndex(retval.start))
<else>
input.toStringWithRange(retval.start,input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.st"
lexerRulePropertyRef_text(scope,attr) ::= "text"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "state.tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "state.tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "_channel"
lexerRulePropertyRef_start(scope,attr) ::= "state.tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(charIndex-1)"
lexerRulePropertyRef_int(scope,attr) ::= "int(<scope>.text)"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.tree =<expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.st =<expr>;"
/** How to execute an action (only when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {
<action>
}
<else>
<action>
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
public static const <name>:BitSet = new BitSet([<words64:{<it>};separator=", ">]);<\n>
>>
codeFileExtension() ::= ".as"
true() ::= "true"
false() ::= "false"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ActionScript/ASTTreeParser.stg 0000644 0001750 0001750 00000022340 11256465232 032223 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
var _first_0:<ASTLabelType> = null;
var _last:<ASTLabelType> = null;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(rewriteMode)>
retval.tree = <ASTLabelType>(_first_0);
if ( adaptor.getParent(retval.tree)!=null && adaptor.isNil( adaptor.getParent(retval.tree) ) )
retval.tree = <ASTLabelType>(adaptor.getParent(retval.tree));
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = <ASTLabelType>(input.LT(1));
{
var _save_last_<treeLevel>:<ASTLabelType> = _last;
var _first_<treeLevel>:<ASTLabelType> = null;
<if(!rewriteMode)>
var root_<treeLevel>:<ASTLabelType> = <ASTLabelType>(adaptor.nil());
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
<if(root.el.rule)>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>.tree;
<else>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==TokenConstants.DOWN ) {
matchStream(input, TokenConstants.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
matchStream(input, TokenConstants.UP, null); <checkRuleBacktrackFailure()>
}
<else>
matchStream(input, TokenConstants.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
matchStream(input, TokenConstants.UP, null); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = <ASTLabelType>(adaptor.dupNode(<label>));
<endif><\n>
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = <ASTLabelType>(adaptor.dupNode(<label>));
<endif><\n>
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(<label>_tree, root_<treeLevel>));
<if(backtracking)>}<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard and auto dup the node/subtree */
wildcard(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.wildcard(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.dupTree(<label>);
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = <ASTLabelType>(adaptor.dupNode(<label>));
<endif><\n>
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = <ASTLabelType>(adaptor.dupNode(<label>));
<endif><\n>
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(<label>_tree, root_<treeLevel>));
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>
<if(!rewriteMode)>
adaptor.addChild(root_<treeLevel>, <label>.tree);
<else> <! rewrite mode !>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>.tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".tree",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(<label>.tree, root_<treeLevel>));
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".tree",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = <ASTLabelType>(input.LT(1));
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextNode())
<else>
stream_<token>.nextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = <ASTLabelType>(adaptor.rulePostProcessing(root_0));
<if(backtracking)>}<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ActionScript/ASTParser.stg 0000644 0001750 0001750 00000015440 11256465232 031406 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
retval.tree = <ASTLabelType>(adaptor.errorNode(input, Token(retval.start), input.LT(-1), re));
<! trace("<ruleName> returns "+((CommonTree)retval.tree).toStringTree()); !>
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
adaptor.addChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
root_0 = <ASTLabelType>(adaptor.becomeRoot(<label>_tree, root_0));
<if(backtracking)>}<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>adaptor.addChild(root_0, <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label>=<labelType>(input.LT(1));<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = <ASTLabelType>(adaptor.becomeRoot(<createNodeFromToken(...)>, root_0));})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>adaptor.addChild(root_0, <label>.tree);
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = <ASTLabelType>(adaptor.becomeRoot(<label>.tree, root_0));
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".tree",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".tree",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".tree",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <ASTLabelType>(adaptor.create(<label>));
adaptor.addChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <ASTLabelType>(adaptor.create(<label>));
root_0 = <ASTLabelType>(adaptor.becomeRoot(<label>_tree, root_0));
<if(backtracking)>}<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
new <hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
<ASTLabelType>(adaptor.create(<label>))
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = <ASTLabelType>(adaptor.rulePostProcessing(root_0));
adaptor.setTokenBoundaries(retval.tree, Token(retval.start), Token(retval.stop));
<if(backtracking)>}<endif>
>> antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ActionScript/AST.stg 0000644 0001750 0001750 00000031757 11256465232 030242 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()>
<if(!TREE_PARSER)><! tree parser would already have imported !>
import org.antlr.runtime.tree.*;<\n>
<endif>
>>
@genericParser.members() ::= <<
<@super.members()>
<parserMembers()>
>>
/** Add an adaptor property that knows how to build trees */
parserMembers() ::= <<
protected var adaptor:TreeAdaptor = new CommonTreeAdaptor();<\n>
override public function set treeAdaptor(adaptor:TreeAdaptor):void {
this.adaptor = adaptor;
<grammar.directDelegates:{g|<g:delegateName()>.treeAdaptor = this.adaptor;}>
}
override public function get treeAdaptor():TreeAdaptor {
return adaptor;
}
>>
@returnScope.ruleReturnMembers() ::= <<
<ASTLabelType> tree;
public function get tree():Object { return tree; }
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
var root_0:<ASTLabelType> = null;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<[ruleDescriptor.tokenLabels,ruleDescriptor.wildcardTreeLabels,
ruleDescriptor.wildcardTreeListLabels]:{var <it.label.text>_tree:<ASTLabelType>=null;}; separator="\n">
<ruleDescriptor.tokenListLabels:{var <it.label.text>_tree:<ASTLabelType>=null;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{var stream_<it>:RewriteRule<rewriteElementType>Stream=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>");}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{var stream_<it>:RewriteRuleSubtreeStream=new RewriteRuleSubtreeStream(adaptor,"rule <it>");}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = <ASTLabelType>(adaptor.nil());<\n>
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.add(<label>);<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.add(<label>);<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule.name>.add(<label>.tree);
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".tree",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule>.add(<label>.tree);
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".tree",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {<\n>
<endif>
<prevRuleRootRef()>.tree = root_0;
<rewriteCodeLabels()>
root_0 = <ASTLabelType>(adaptor.nil());
<alts:rewriteAlt(); separator="else ">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.tree = <ASTLabelType>(adaptor.rulePostProcessing(root_0));
input.replaceChildren(adaptor.getParent(retval.start),
adaptor.getChildIndex(retval.start),
adaptor.getChildIndex(_last),
retval.tree);
<endif>
<endif>
<! if parser or tree-parser && rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.tree = root_0;
<else>
<if(!rewriteMode)>
<prevRuleRootRef()>.tree = root_0;
<endif>
<endif>
<if(backtracking)>
}
<endif>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{var stream_<it>:RewriteRule<rewriteElementType>Stream=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>",<it>);};
separator="\n"
>
<referencedTokenListLabels
:{var stream_<it>:RewriteRule<rewriteElementType>Stream=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>", list_<it>);};
separator="\n"
>
<referencedWildcardLabels
:{var stream_<it>:RewriteRuleSubtreeStream=new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",<it>);};
separator="\n"
>
<referencedWildcardListLabels
:{var stream_<it>:RewriteRuleSubtreeStream=new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",list_<it>);};
separator="\n"
>
<referencedRuleLabels
:{var stream_<it>:RewriteRuleSubtreeStream=new RewriteRuleSubtreeStream(adaptor,"rule <it>",<it>!=null?<it>.tree:null);};
separator="\n"
>
<referencedRuleListLabels
:{var stream_<it>:RewriteRuleSubtreeStream=new RewriteRuleSubtreeStream(adaptor,"rule <it>",list_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
if ( <referencedElementsDeep:{el | stream_<el>.hasNext}; separator="||"> ) {
<alt>
}
<referencedElementsDeep:{el | stream_<el>.reset();<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
while ( <referencedElements:{el | stream_<el>.hasNext}; separator="||"> ) {
<alt>
}
<referencedElements:{el | stream_<el>.reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if ( !(<referencedElements:{el | stream_<el>.hasNext}; separator="||">) ) {
throw new RewriteEarlyExitException();
}
while ( <referencedElements:{el | stream_<el>.hasNext}; separator="||"> ) {
<alt>
}
<referencedElements:{el | stream_<el>.reset();<\n>}>
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>) {
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = null;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
var root_<treeLevel>:<ASTLabelType> = <ASTLabelType>(adaptor.nil());
<root:rewriteElement()>
<children:rewriteElement()>
adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>);
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
adaptor.addChild(root_<treeLevel>, <createRewriteNodeFromElement(...)>);<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode());<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode());<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>));<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(<createRewriteNodeFromElement(...)>, root_<treeLevel>));<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
adaptor.addChild(root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(<createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>));<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<rule>.nextTree());<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(stream_<rule>.nextNode(), root_<treeLevel>));<\n>
>>
rewriteNodeAction(action) ::= <<
adaptor.addChild(root_<treeLevel>, <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(<action>, root_<treeLevel>));<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>));<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = <ASTLabelType>(adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>));<\n>
>>
rewriteWildcardLabelRef(label) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
new <hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
<ASTLabelType>(adaptor.create(<tokenType>, <args; separator=", "><if(!args)>"<tokenType>"<endif>))
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
adaptor.create(<token>, <args; separator=", ">)
<else>
stream_<token>.nextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/ 0000755 0001750 0001750 00000000000 11410174107 025324 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/ASTTreeParser.stg 0000644 0001750 0001750 00000022242 11256465232 030503 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> _first_0 = null;
<ASTLabelType> _last = null;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(rewriteMode)>
retval.tree = (<ASTLabelType>)_first_0;
if ( adaptor.getParent(retval.tree)!=null && adaptor.isNil( adaptor.getParent(retval.tree) ) )
retval.tree = (<ASTLabelType>)adaptor.getParent(retval.tree);
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = (<ASTLabelType>)input.LT(1);
{
<ASTLabelType> _save_last_<treeLevel> = _last;
<ASTLabelType> _first_<treeLevel> = null;
<if(!rewriteMode)>
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.nil();
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
<if(root.el.rule)>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>.tree;
<else>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==Token.DOWN ) {
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard and auto dup the node/subtree */
wildcard(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.wildcard(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.dupTree(<label>);
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>
<if(!rewriteMode)>
adaptor.addChild(root_<treeLevel>, <label>.getTree());
<else> <! rewrite mode !>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>.tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>.getTree(), root_<treeLevel>);
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextNode())
<else>
stream_<token>.nextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = (<ASTLabelType>)adaptor.rulePostProcessing(root_0);
<if(backtracking)>}<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/ST.stg 0000644 0001750 0001750 00000011611 11256465232 026403 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template subgroup to add template rewrite output
* If debugging, then you'll also get STDbg.stg loaded.
*/
group ST;
@outputFile.imports() ::= <<
<@super.imports()>
import org.antlr.stringtemplate.*;
import org.antlr.stringtemplate.language.*;
import java.util.HashMap;
>>
/** Add this to each rule's return value struct */
@returnScope.ruleReturnMembers() ::= <<
public StringTemplate st;
public Object getTemplate() { return st; }
public String toString() { return st==null?null:st.toString(); }
>>
@genericParser.members() ::= <<
<@super.members()>
protected StringTemplateGroup templateLib =
new StringTemplateGroup("<name>Templates", AngleBracketTemplateLexer.class);
public void setTemplateLib(StringTemplateGroup templateLib) {
this.templateLib = templateLib;
}
public StringTemplateGroup getTemplateLib() {
return templateLib;
}
/** allows convenient multi-value initialization:
* "new STAttrMap().put(...).put(...)"
*/
public static class STAttrMap extends HashMap {
public STAttrMap put(String attrName, Object value) {
super.put(attrName, value);
return this;
}
public STAttrMap put(String attrName, int value) {
super.put(attrName, new Integer(value));
return this;
}
}
>>
/** x+=rule when output=template */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTemplate()",...)>
>>
rewriteTemplate(alts) ::= <<
// TEMPLATE REWRITE
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
}
<else>
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
<endif>
>>
replaceTextInLine() ::= <<
<if(TREE_PARSER)>
((TokenRewriteStream)input.getTokenStream()).replace(
input.getTreeAdaptor().getTokenStartIndex(retval.start),
input.getTreeAdaptor().getTokenStopIndex(retval.start),
retval.st);
<else>
((TokenRewriteStream)input).replace(
((Token)retval.start).getTokenIndex(),
input.LT(-1).getTokenIndex(),
retval.st);
<endif>
>>
rewriteTemplateAlt() ::= <<
// <it.description>
<if(it.pred)>
if (<it.pred>) {
retval.st = <it.alt>;
}<\n>
<else>
{
retval.st = <it.alt>;
}<\n>
<endif>
>>
rewriteEmptyTemplate(alts) ::= <<
null;
>>
/** Invoke a template with a set of attribute name/value pairs.
* Set the value of the rule's template *after* having set
* the attributes because the rule's template might be used as
* an attribute to build a bigger template; you get a self-embedded
* template.
*/
rewriteExternalTemplate(name,args) ::= <<
templateLib.getInstanceOf("<name>"<if(args)>,
new STAttrMap()<args:{a | .put("<a.name>", <a.value>)}>
<endif>)
>>
/** expr is a string expression that says what template to load */
rewriteIndirectTemplate(expr,args) ::= <<
templateLib.getInstanceOf(<expr><if(args)>,
new STAttrMap()<args:{a | .put("<a.name>", <a.value>)}>
<endif>)
>>
/** Invoke an inline template with a set of attribute name/value pairs */
rewriteInlineTemplate(args, template) ::= <<
new StringTemplate(templateLib, "<template>"<if(args)>,
new STAttrMap()<args:{a | .put("<a.name>", <a.value>)}>
<endif>)
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
<action>
>>
/** An action has %st.attrName=expr; or %{st}.attrName=expr; */
actionSetAttribute(st,attrName,expr) ::= <<
(<st>).setAttribute("<attrName>",<expr>);
>>
/** Translate %{stringExpr} */
actionStringConstructor(stringExpr) ::= <<
new StringTemplate(templateLib,<stringExpr>)
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/ASTParser.stg 0000644 0001750 0001750 00000015462 11256465232 027671 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
retval.tree = (<ASTLabelType>)adaptor.errorNode(input, retval.start, input.LT(-1), re);
<! System.out.println("<ruleName> returns "+((CommonTree)retval.tree).toStringTree()); !>
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
adaptor.addChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
root_0 = (<ASTLabelType>)adaptor.becomeRoot(<label>_tree, root_0);
<if(backtracking)>}<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>adaptor.addChild(root_0, <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = (<ASTLabelType>)adaptor.becomeRoot(<createNodeFromToken(...)>, root_0);})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>adaptor.addChild(root_0, <label>.getTree());
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = (<ASTLabelType>)adaptor.becomeRoot(<label>.getTree(), root_0);
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".getTree()",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.create(<label>);
adaptor.addChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.create(<label>);
root_0 = (<ASTLabelType>)adaptor.becomeRoot(<label>_tree, root_0);
<if(backtracking)>}<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
new <hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
(<ASTLabelType>)adaptor.create(<label>)
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = (<ASTLabelType>)adaptor.rulePostProcessing(root_0);
adaptor.setTokenBoundaries(retval.tree, retval.start, retval.stop);
<if(backtracking)>}<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/Dbg.stg 0000644 0001750 0001750 00000021177 11256465232 026561 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to normal Java output;
* If ASTs are built, then you'll also get ASTDbg.stg loaded.
*/
group Dbg;
@outputFile.imports() ::= <<
<@super.imports()>
import org.antlr.runtime.debug.*;
import java.io.IOException;
>>
@genericParser.members() ::= <<
<if(grammar.grammarIsRoot)>
public static final String[] ruleNames = new String[] {
"invalidRule", <grammar.allImportedRules:{rST | "<rST.name>"}; wrap="\n ", separator=", ">
};<\n>
<endif>
<if(grammar.grammarIsRoot)> <! grammar imports other grammar(s) !>
public int ruleLevel = 0;
public int getRuleLevel() { return ruleLevel; }
public void incRuleLevel() { ruleLevel++; }
public void decRuleLevel() { ruleLevel--; }
<if(profile)>
<ctorForProfilingRootGrammar()>
<else>
<ctorForRootGrammar()>
<endif>
<ctorForPredefinedListener()>
<else> <! imported grammar !>
public int getRuleLevel() { return <grammar.delegators:{g| <g:delegateName()>}>.getRuleLevel(); }
public void incRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.incRuleLevel(); }
public void decRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.decRuleLevel(); }
<ctorForDelegateGrammar()>
<endif>
<if(profile)>
public boolean alreadyParsedRule(IntStream input, int ruleIndex) {
((Profiler)dbg).examineRuleMemoization(input, ruleIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
return super.alreadyParsedRule(input, ruleIndex);
}<\n>
public void memoize(IntStream input,
int ruleIndex,
int ruleStartIndex)
{
((Profiler)dbg).memoize(input, ruleIndex, ruleStartIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
super.memoize(input, ruleIndex, ruleStartIndex);
}<\n>
<endif>
protected boolean evalPredicate(boolean result, String predicate) {
dbg.semanticPredicate(result, predicate);
return result;
}<\n>
>>
ctorForRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
<! Same except we add port number and profile stuff if root grammar !>
public <name>(<inputStreamType> input) {
this(input, DebugEventSocketProxy.DEFAULT_DEBUGGER_PORT, new RecognizerSharedState());
}
public <name>(<inputStreamType> input, int port, RecognizerSharedState state) {
super(input, state);
<parserCtorBody()>
<createListenerAndHandshake()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
>>
ctorForProfilingRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
public <name>(<inputStreamType> input) {
this(input, new Profiler(null), new RecognizerSharedState());
}
public <name>(<inputStreamType> input, DebugEventListener dbg, RecognizerSharedState state) {
super(input, dbg, state);
Profiler p = (Profiler)dbg;
p.setParser(this);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}
<\n>
>>
/** Basically we don't want to set any dbg listeners are root will have it. */
ctorForDelegateGrammar() ::= <<
public <name>(<inputStreamType> input, DebugEventListener dbg, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>) {
super(input, dbg, state);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, this, this.state<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
}<\n>
>>
ctorForPredefinedListener() ::= <<
public <name>(<inputStreamType> input, DebugEventListener dbg) {
<@superClassRef>super(input, dbg, new RecognizerSharedState());<@end>
<if(profile)>
Profiler p = (Profiler)dbg;
p.setParser(this);
<endif>
<parserCtorBody()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
>>
createListenerAndHandshake() ::= <<
<if(TREE_PARSER)>
DebugEventSocketProxy proxy =
new DebugEventSocketProxy(this, port, input.getTreeAdaptor());<\n>
<else>
DebugEventSocketProxy proxy =
new DebugEventSocketProxy(this, port, null);<\n>
<endif>
setDebugListener(proxy);
try {
proxy.handshake();
}
catch (IOException ioe) {
reportError(ioe);
}
>>
@genericParser.superClassName() ::= "Debug<@super.superClassName()>"
@rule.preamble() ::= <<
try { dbg.enterRule(getGrammarFileName(), "<ruleName>");
if ( getRuleLevel()==0 ) {dbg.commence();}
incRuleLevel();
dbg.location(<ruleDescriptor.tree.line>, <ruleDescriptor.tree.column>);<\n>
>>
@rule.postamble() ::= <<
dbg.location(<ruleDescriptor.EORNode.line>, <ruleDescriptor.EORNode.column>);<\n>
}
finally {
dbg.exitRule(getGrammarFileName(), "<ruleName>");
decRuleLevel();
if ( getRuleLevel()==0 ) {dbg.terminate();}
}<\n>
>>
@synpred.start() ::= "dbg.beginBacktrack(state.backtracking);"
@synpred.stop() ::= "dbg.endBacktrack(state.backtracking, success);"
// Common debug event triggers used by region overrides below
enterSubRule() ::=
"try { dbg.enterSubRule(<decisionNumber>);<\n>"
exitSubRule() ::=
"} finally {dbg.exitSubRule(<decisionNumber>);}<\n>"
enterDecision() ::=
"try { dbg.enterDecision(<decisionNumber>);<\n>"
exitDecision() ::=
"} finally {dbg.exitDecision(<decisionNumber>);}<\n>"
enterAlt(n) ::= "dbg.enterAlt(<n>);<\n>"
// Region overrides that tell various constructs to add debugging triggers
@block.predecision() ::= "<enterSubRule()><enterDecision()>"
@block.postdecision() ::= "<exitDecision()>"
@block.postbranch() ::= "<exitSubRule()>"
@ruleBlock.predecision() ::= "<enterDecision()>"
@ruleBlock.postdecision() ::= "<exitDecision()>"
@ruleBlockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@blockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@positiveClosureBlock.preloop() ::= "<enterSubRule()>"
@positiveClosureBlock.postloop() ::= "<exitSubRule()>"
@positiveClosureBlock.predecision() ::= "<enterDecision()>"
@positiveClosureBlock.postdecision() ::= "<exitDecision()>"
@positiveClosureBlock.earlyExitException() ::=
"dbg.recognitionException(eee);<\n>"
@closureBlock.preloop() ::= "<enterSubRule()>"
@closureBlock.postloop() ::= "<exitSubRule()>"
@closureBlock.predecision() ::= "<enterDecision()>"
@closureBlock.postdecision() ::= "<exitDecision()>"
@altSwitchCase.prealt() ::= "<enterAlt(n=i)>"
@element.prematch() ::=
"dbg.location(<it.line>,<it.pos>);"
@matchSet.mismatchedSetException() ::=
"dbg.recognitionException(mse);"
@dfaState.noViableAltException() ::= "dbg.recognitionException(nvae);"
@dfaStateSwitch.noViableAltException() ::= "dbg.recognitionException(nvae);"
dfaDecision(decisionNumber,description) ::= <<
try {
isCyclicDecision = true;
<super.dfaDecision(...)>
}
catch (NoViableAltException nvae) {
dbg.recognitionException(nvae);
throw nvae;
}
>>
@cyclicDFA.errorMethod() ::= <<
public void error(NoViableAltException nvae) {
dbg.recognitionException(nvae);
}
>>
/** Force predicate validation to trigger an event */
evalPredicate(pred,description) ::= <<
evalPredicate(<pred>,"<description>")
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/AST.stg 0000644 0001750 0001750 00000032001 11256465232 026500 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()>
<if(!TREE_PARSER)><! tree parser would already have imported !>
import org.antlr.runtime.tree.*;<\n>
<endif>
>>
@genericParser.members() ::= <<
<@super.members()>
<parserMembers()>
>>
/** Add an adaptor property that knows how to build trees */
parserMembers() ::= <<
protected TreeAdaptor adaptor = new CommonTreeAdaptor();<\n>
public void setTreeAdaptor(TreeAdaptor adaptor) {
this.adaptor = adaptor;
<grammar.directDelegates:{g|<g:delegateName()>.setTreeAdaptor(this.adaptor);}>
}
public TreeAdaptor getTreeAdaptor() {
return adaptor;
}
>>
@returnScope.ruleReturnMembers() ::= <<
<ASTLabelType> tree;
public Object getTree() { return tree; }
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> root_0 = null;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<[ruleDescriptor.tokenLabels,ruleDescriptor.wildcardTreeLabels,
ruleDescriptor.wildcardTreeListLabels]:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{RewriteRule<rewriteElementType>Stream stream_<it>=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>");}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"rule <it>");}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = (<ASTLabelType>)adaptor.nil();<\n>
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.add(<label>);<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.add(<label>);<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule.name>.add(<label>.getTree());
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule>.add(<label>.getTree());
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".getTree()",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
// wildcard labels: <[referencedWildcardLabels,referencedWildcardListLabels]; separator=", ">
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {<\n>
<endif>
<prevRuleRootRef()>.tree = root_0;
<rewriteCodeLabels()>
root_0 = (<ASTLabelType>)adaptor.nil();
<alts:rewriteAlt(); separator="else ">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.tree = (<ASTLabelType>)adaptor.rulePostProcessing(root_0);
input.replaceChildren(adaptor.getParent(retval.start),
adaptor.getChildIndex(retval.start),
adaptor.getChildIndex(_last),
retval.tree);
<endif>
<endif>
<! if parser or tree-parser && rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.tree = root_0;
<else>
<if(!rewriteMode)>
<prevRuleRootRef()>.tree = root_0;
<endif>
<endif>
<if(backtracking)>
}
<endif>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{RewriteRule<rewriteElementType>Stream stream_<it>=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>",<it>);};
separator="\n"
>
<referencedTokenListLabels
:{RewriteRule<rewriteElementType>Stream stream_<it>=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>", list_<it>);};
separator="\n"
>
<referencedWildcardLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",<it>);};
separator="\n"
>
<referencedWildcardListLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",list_<it>);};
separator="\n"
>
<referencedRuleLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"rule <it>",<it>!=null?<it>.tree:null);};
separator="\n"
>
<referencedRuleListLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"token <it>",list_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
if ( <referencedElementsDeep:{el | stream_<el>.hasNext()}; separator="||"> ) {
<alt>
}
<referencedElementsDeep:{el | stream_<el>.reset();<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
while ( <referencedElements:{el | stream_<el>.hasNext()}; separator="||"> ) {
<alt>
}
<referencedElements:{el | stream_<el>.reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if ( !(<referencedElements:{el | stream_<el>.hasNext()}; separator="||">) ) {
throw new RewriteEarlyExitException();
}
while ( <referencedElements:{el | stream_<el>.hasNext()}; separator="||"> ) {
<alt>
}
<referencedElements:{el | stream_<el>.reset();<\n>}>
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>) {
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = null;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.nil();
<root:rewriteElement()>
<children:rewriteElement()>
adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>);
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
adaptor.addChild(root_<treeLevel>, <createRewriteNodeFromElement(...)>);<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode());<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode());<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<createRewriteNodeFromElement(...)>, root_<treeLevel>);<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
adaptor.addChild(root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>);<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<rule>.nextTree());<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(stream_<rule>.nextNode(), root_<treeLevel>);<\n>
>>
rewriteNodeAction(action) ::= <<
adaptor.addChild(root_<treeLevel>, <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<action>, root_<treeLevel>);<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>);<\n>
>>
rewriteWildcardLabelRef(label) ::= <<
adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
new <hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
(<ASTLabelType>)adaptor.create(<tokenType>, <args; separator=", "><if(!args)>"<tokenType>"<endif>)
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
adaptor.create(<token>, <args; separator=", ">)
<else>
stream_<token>.nextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/ASTDbg.stg 0000644 0001750 0001750 00000005757 11256465232 027137 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to AST stuff. Dynamic inheritance
* hierarchy is set up as ASTDbg : AST : Dbg : Java by code generator.
*/
group ASTDbg;
parserMembers() ::= <<
protected DebugTreeAdaptor adaptor;
public void setTreeAdaptor(TreeAdaptor adaptor) {
<if(grammar.grammarIsRoot)>
this.adaptor = new DebugTreeAdaptor(dbg,adaptor);
<else>
this.adaptor = (DebugTreeAdaptor)adaptor; // delegator sends dbg adaptor
<endif><\n>
<grammar.directDelegates:{g|<g:delegateName()>.setTreeAdaptor(this.adaptor);}>
}
public TreeAdaptor getTreeAdaptor() {
return adaptor;
}<\n>
>>
parserCtorBody() ::= <<
<super.parserCtorBody()>
>>
createListenerAndHandshake() ::= <<
DebugEventSocketProxy proxy =
new DebugEventSocketProxy(this,port,<if(TREE_PARSER)>input.getTreeAdaptor()<else>adaptor<endif>);
setDebugListener(proxy);
set<inputStreamType>(new Debug<inputStreamType>(input,proxy));
try {
proxy.handshake();
}
catch (IOException ioe) {
reportError(ioe);
}
>>
@ctorForRootGrammar.finally() ::= <<
TreeAdaptor adap = new CommonTreeAdaptor();
setTreeAdaptor(adap);
proxy.setTreeAdaptor(adap);
>>
@ctorForProfilingRootGrammar.finally() ::=<<
TreeAdaptor adap = new CommonTreeAdaptor();
setTreeAdaptor(adap);
proxy.setTreeAdaptor(adap);
>>
@ctorForPredefinedListener.superClassRef() ::= "super(input, dbg);"
@ctorForPredefinedListener.finally() ::=<<
<if(grammar.grammarIsRoot)> <! don't create new adaptor for delegates !>
TreeAdaptor adap = new CommonTreeAdaptor();
setTreeAdaptor(adap);<\n>
<endif>
>>
@rewriteElement.pregen() ::= "dbg.location(<e.line>,<e.pos>);"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Java/Java.stg 0000644 0001750 0001750 00000125324 11256465232 026745 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group Java implements ANTLRCore;
javaTypeInitMap ::= [
"int":"0",
"long":"0",
"float":"0.0f",
"double":"0.0",
"boolean":"false",
"byte":"0",
"short":"0",
"char":"0",
default:"null" // anything other than an atomic type
]
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<actions.(actionScope).header>
<@imports>
import org.antlr.runtime.*;
<if(TREE_PARSER)>
import org.antlr.runtime.tree.*;
<endif>
import java.util.Stack;
import java.util.List;
import java.util.ArrayList;
<if(backtracking)>
import java.util.Map;
import java.util.HashMap;
<endif>
<@end>
<docComment>
<recognizer>
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="CommonToken",
filterMode, superClass="Lexer") ::= <<
public class <grammar.recognizerName> extends <@superClassName><superClass><@end> {
<tokens:{public static final int <it.name>=<it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
public <grammar.recognizerName>() {;} <! needed by subclasses !>
public <grammar.recognizerName>(CharStream input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>) {
this(input, new RecognizerSharedState()<grammar.delegators:{g|, <g:delegateName()>}>);
}
public <grammar.recognizerName>(CharStream input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>) {
super(input,state);
<if(memoize)>
<if(grammar.grammarIsRoot)>
state.ruleMemo = new HashMap[<numRules>+1];<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
public String getGrammarFileName() { return "<fileName>"; }
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<synpreds:{p | <lexerSynpred(p)>}>
<cyclicDFAs:{dfa | protected DFA<dfa.decisionNumber> dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this);}; separator="\n">
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
}
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
public Token nextToken() {
while (true) {
if ( input.LA(1)==CharStream.EOF ) {
return Token.EOF_TOKEN;
}
state.token = null;
state.channel = Token.DEFAULT_CHANNEL;
state.tokenStartCharIndex = input.index();
state.tokenStartCharPositionInLine = input.getCharPositionInLine();
state.tokenStartLine = input.getLine();
state.text = null;
try {
int m = input.mark();
state.backtracking=1; <! means we won't throw slow exception !>
state.failed=false;
mTokens();
state.backtracking=0;
<! mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1. !>
if ( state.failed ) {
input.rewind(m);
input.consume(); <! advance one char and try again !>
}
else {
emit();
return state.token;
}
}
catch (RecognitionException re) {
// shouldn't happen in backtracking mode, but...
reportError(re);
recover(re);
}
}
}
public void memoize(IntStream input,
int ruleIndex,
int ruleStartIndex)
{
if ( state.backtracking>1 ) super.memoize(input, ruleIndex, ruleStartIndex);
}
public boolean alreadyParsedRule(IntStream input, int ruleIndex) {
if ( state.backtracking>1 ) return super.alreadyParsedRule(input, ruleIndex);
return false;
}
>>
actionGate() ::= "state.backtracking==0"
filteringActionGate() ::= "state.backtracking==1"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass,
ASTLabelType="Object", labelType, members, rewriteElementType,
filterMode) ::= <<
public class <grammar.recognizerName> extends <@superClassName><superClass><@end> {
<if(grammar.grammarIsRoot)>
public static final String[] tokenNames = new String[] {
"\<invalid>", "\<EOR>", "\<DOWN>", "\<UP>", <tokenNames; separator=", ">
};<\n>
<endif>
<tokens:{public static final int <it.name>=<it.type>;}; separator="\n">
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<@members>
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
public <grammar.recognizerName>(<inputStreamType> input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>) {
this(input, new RecognizerSharedState()<grammar.delegators:{g|, <g:delegateName()>}>);
}
public <grammar.recognizerName>(<inputStreamType> input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>) {
super(input, state);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.indirectDelegates:{g | <g:delegateName()> = <g.delegator:delegateName()>.<g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
<@end>
public String[] getTokenNames() { return <grammar.composite.rootGrammar.recognizerName>.tokenNames; }
public String getGrammarFileName() { return "<fileName>"; }
<members>
<rules; separator="\n\n">
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
// Delegated rules
<grammar.delegatedRules:{ruleDescriptor|
public <returnType()> <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) throws RecognitionException \{ <if(ruleDescriptor.hasReturnValue)>return <endif><ruleDescriptor.grammar:delegateName()>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">); \}}; separator="\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:{dfa | protected DFA<dfa.decisionNumber> dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this);}; separator="\n">
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
}
>>
parserCtorBody() ::= <<
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = new HashMap[<length(grammar.allImportedRules)>+1];<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets,
ASTLabelType="Object", superClass="Parser", labelType="Token",
members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="TokenStream", rewriteElementType="Token", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules,
numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="Object",
superClass={<if(filterMode)><if(buildAST)>TreeRewriter<else>TreeFilter<endif><else>TreeParser<endif>},
members={<actions.treeparser.members>},
filterMode) ::= <<
<genericParser(inputStreamType="TreeNodeStream", rewriteElementType="Node", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start <ruleName>
public final void <ruleName>_fragment(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) throws RecognitionException {
<ruleLabelDefs()>
<if(trace)>
traceIn("<ruleName>_fragment", <ruleDescriptor.index>);
try {
<block>
}
finally {
traceOut("<ruleName>_fragment", <ruleDescriptor.index>);
}
<else>
<block>
<endif>
}
// $ANTLR end <ruleName>
>>
synpred(name) ::= <<
public final boolean <name>() {
state.backtracking++;
<@start()>
int start = input.mark();
try {
<name>_fragment(); // can never throw exception
} catch (RecognitionException re) {
System.err.println("impossible: "+re);
}
boolean success = !state.failed;
input.rewind(start);
<@stop()>
state.backtracking--;
state.failed=false;
return success;
}<\n>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( state.backtracking>0 && alreadyParsedRule(input, <ruleDescriptor.index>) ) { return <ruleReturnValue()>; }
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>if (state.failed) return <ruleReturnValue()>;<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if (state.backtracking>0) {state.failed=true; return <ruleReturnValue()>;}<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
// $ANTLR start "<ruleName>"
// <fileName>:<description>
public final <returnType()> <ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) throws RecognitionException {
<if(trace)>traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try {
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
catch (RecognitionException re) {
reportError(re);
recover(input,re);
<@setErrorReturnValue()>
}<\n>
<endif>
<endif>
<endif>
finally {
<if(trace)>traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
}
<@postamble()>
return <ruleReturnValue()>;
}
// $ANTLR end "<ruleName>"
>>
catch(decl,action) ::= <<
catch (<e.decl>) {
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnType()> retval = new <returnType()>();
retval.start = input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.type> <a.name> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
int <ruleDescriptor.name>_StartIndex = input.index();
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.push(new <it>_scope());}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.push(new <it.name>_scope());}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.pop();}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels,
ruleDescriptor.wildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{<labelType> <it.label.text>=null;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels,ruleDescriptor.wildcardTreeListLabels]
:{List list_<it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|RuleReturnScope <ll.label.text> = null;}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<labelType> <it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.charLabels:{int <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels]
:{List list_<it.label.text>=null;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
retval.stop = input.LT(-1);<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( state.backtracking>0 ) { memoize(input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex); }
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// $ANTLR start "<ruleName>"
public final void m<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) throws RecognitionException {
<if(trace)>traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
try {
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
int _type = <ruleName>;
int _channel = DEFAULT_TOKEN_CHANNEL;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
state.type = _type;
state.channel = _channel;
<(ruleDescriptor.actions.after):execAction()>
<endif>
}
finally {
<if(trace)>traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeCleanUp()>
<memoize()>
}
}
// $ANTLR end "<ruleName>"
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
public void mTokens() throws RecognitionException {
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int cnt<decisionNumber>=0;
<decls>
<@preloop()>
loop<decisionNumber>:
do {
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
if ( cnt<decisionNumber> >= 1 ) break loop<decisionNumber>;
<ruleBacktrackFailure()>
EarlyExitException eee =
new EarlyExitException(<decisionNumber>, input);
<@earlyExitException()>
throw eee;
}
cnt<decisionNumber>++;
} while (true);
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
<decls>
<@preloop()>
loop<decisionNumber>:
do {
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
break loop<decisionNumber>;
}
} while (true);
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> :
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
// <fileName>:<description>
{
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
}
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label>=(<labelType>)<endif>match(input,<token>,FOLLOW_<token>_in_<ruleName><elementIndex>); <checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label>==null) list_<label>=new ArrayList();
list_<label>.add(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
matchRange(<a>,<b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= input.LA(1);<\n>
<else>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<endif>
if ( <s> ) {
input.consume();
<postmatchCode>
<if(!LEXER)>
state.errorRecovery=false;
<endif>
<if(backtracking)>state.failed=false;<endif>
}
else {
<ruleBacktrackFailure()>
MismatchedSetException mse = new MismatchedSetException(null,input);
<@mismatchedSetException()>
<if(LEXER)>
recover(mse);
throw mse;
<else>
throw mse;
<! use following code to make it recover inline; remove throw mse;
recoverFromMismatchedSet(input,mse,FOLLOW_set_in_<ruleName><elementIndex>);
!>
<endif>
}<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
int <label>Start = getCharIndex();
match(<string>); <checkRuleBacktrackFailure()>
<label> = new <labelType>(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start, getCharIndex()-1);
<else>
match(<string>); <checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
matchAny(input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
matchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
pushFollow(FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)><label>=<endif><if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
state._fsp--;
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference.
*
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
int <label>Start<elementIndex> = getCharIndex();
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> = new <labelType>(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, getCharIndex()-1);
<else>
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
int <label>Start<elementIndex> = getCharIndex();
match(EOF); <checkRuleBacktrackFailure()>
<labelType> <label> = new <labelType>(input, EOF, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, getCharIndex()-1);
<else>
match(EOF); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==Token.DOWN ) {
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) ) {
<ruleBacktrackFailure()>
throw new FailedPredicateException(input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
else {
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae =
new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae;<\n>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber>=<eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else {
alt<decisionNumber>=<eotPredictsAlt>;
}<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber>=<alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae =
new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae;<\n>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber>=<eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
{
<targetState>
}
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = dfa<decisionNumber>.predict(input);
>>
/* Dump DFA tables as run-length-encoded Strings of octal values.
* Can't use hex as compiler translates them before compilation.
* These strings are split into multiple, concatenated strings.
* Java puts them back together at compile time thankfully.
* Java cannot handle large static arrays, so we're stuck with this
* encode/decode approach. See analysis and runtime DFA for
* the encoding methods.
*/
cyclicDFA(dfa) ::= <<
static final String DFA<dfa.decisionNumber>_eotS =
"<dfa.javaCompressedEOT; wrap="\"+\n \"">";
static final String DFA<dfa.decisionNumber>_eofS =
"<dfa.javaCompressedEOF; wrap="\"+\n \"">";
static final String DFA<dfa.decisionNumber>_minS =
"<dfa.javaCompressedMin; wrap="\"+\n \"">";
static final String DFA<dfa.decisionNumber>_maxS =
"<dfa.javaCompressedMax; wrap="\"+\n \"">";
static final String DFA<dfa.decisionNumber>_acceptS =
"<dfa.javaCompressedAccept; wrap="\"+\n \"">";
static final String DFA<dfa.decisionNumber>_specialS =
"<dfa.javaCompressedSpecial; wrap="\"+\n \"">}>";
static final String[] DFA<dfa.decisionNumber>_transitionS = {
<dfa.javaCompressedTransition:{s|"<s; wrap="\"+\n\"">"}; separator=",\n">
};
static final short[] DFA<dfa.decisionNumber>_eot = DFA.unpackEncodedString(DFA<dfa.decisionNumber>_eotS);
static final short[] DFA<dfa.decisionNumber>_eof = DFA.unpackEncodedString(DFA<dfa.decisionNumber>_eofS);
static final char[] DFA<dfa.decisionNumber>_min = DFA.unpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_minS);
static final char[] DFA<dfa.decisionNumber>_max = DFA.unpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_maxS);
static final short[] DFA<dfa.decisionNumber>_accept = DFA.unpackEncodedString(DFA<dfa.decisionNumber>_acceptS);
static final short[] DFA<dfa.decisionNumber>_special = DFA.unpackEncodedString(DFA<dfa.decisionNumber>_specialS);
static final short[][] DFA<dfa.decisionNumber>_transition;
static {
int numStates = DFA<dfa.decisionNumber>_transitionS.length;
DFA<dfa.decisionNumber>_transition = new short[numStates][];
for (int i=0; i\<numStates; i++) {
DFA<dfa.decisionNumber>_transition[i] = DFA.unpackEncodedString(DFA<dfa.decisionNumber>_transitionS[i]);
}
}
class DFA<dfa.decisionNumber> extends DFA {
public DFA<dfa.decisionNumber>(BaseRecognizer recognizer) {
this.recognizer = recognizer;
this.decisionNumber = <dfa.decisionNumber>;
this.eot = DFA<dfa.decisionNumber>_eot;
this.eof = DFA<dfa.decisionNumber>_eof;
this.min = DFA<dfa.decisionNumber>_min;
this.max = DFA<dfa.decisionNumber>_max;
this.accept = DFA<dfa.decisionNumber>_accept;
this.special = DFA<dfa.decisionNumber>_special;
this.transition = DFA<dfa.decisionNumber>_transition;
}
public String getDescription() {
return "<dfa.description>";
}
<@errorMethod()>
<if(dfa.specialStateSTs)>
public int specialStateTransition(int s, IntStream _input) throws NoViableAltException {
<if(LEXER)>
IntStream input = _input;
<endif>
<if(PARSER)>
TokenStream input = (TokenStream)_input;
<endif>
<if(TREE_PARSER)>
TreeNodeStream input = (TreeNodeStream)_input;
<endif>
int _s = s;
switch ( s ) {
<dfa.specialStateSTs:{state |
case <i0> : <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
<if(backtracking)>
if (state.backtracking>0) {state.failed=true; return -1;}<\n>
<endif>
NoViableAltException nvae =
new NoViableAltException(getDescription(), <dfa.decisionNumber>, _s, input);
error(nvae);
throw nvae;
}<\n>
<endif>
}<\n>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
int index<decisionNumber>_<stateNumber> = input.index();
input.rewind();<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
<if(semPredState)> <! return input cursor to state before we rewound !>
input.seek(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s>=0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {s = <targetStateNumber>;}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left>&&<right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | ||<o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber>==<atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "input.LA(<k>)==<atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber>\>=<lower> && LA<decisionNumber>_<stateNumber>\<=<upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(input.LA(<k>)\>=<lower> && input.LA(<k>)\<=<upper>)"
setTest(ranges) ::= "<ranges; separator=\"||\">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected static class <scope.name>_scope {
<scope.attributes:{<it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
ruleAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected static class <scope.name>_scope {
<scope.attributes:{<it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
returnStructName() ::= "<it.name>_return"
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.grammar.recognizerName>.<ruleDescriptor:returnStructName()>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Generate the Java type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.grammar.recognizerName>.<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
<javaTypeInitMap.(typeName)>
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<ruleLabelType(referencedRule=label.referencedRule)> <label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
public static class <ruleDescriptor:returnStructName()> extends <if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope {
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
};
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> =<expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack.elementAt(<scope>_stack.size()-<negIndex>-1)).<attr.name>
<else>
<if(index)>
((<scope>_scope)<scope>_stack.elementAt(<index>)).<attr.name>
<else>
((<scope>_scope)<scope>_stack.peek()).<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack.elementAt(<scope>_stack.size()-<negIndex>-1)).<attr.name> =<expr>;
<else>
<if(index)>
((<scope>_scope)<scope>_stack.elementAt(<index>)).<attr.name> =<expr>;
<else>
((<scope>_scope)<scope>_stack.peek()).<attr.name> =<expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
(<scope>!=null?<scope>.<attr.name>:<initValue(attr.type)>)
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name> =<expr>;
<else>
<attr.name> =<expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "(<scope>!=null?<scope>.getText():null)"
tokenLabelPropertyRef_type(scope,attr) ::= "(<scope>!=null?<scope>.getType():0)"
tokenLabelPropertyRef_line(scope,attr) ::= "(<scope>!=null?<scope>.getLine():0)"
tokenLabelPropertyRef_pos(scope,attr) ::= "(<scope>!=null?<scope>.getCharPositionInLine():0)"
tokenLabelPropertyRef_channel(scope,attr) ::= "(<scope>!=null?<scope>.getChannel():0)"
tokenLabelPropertyRef_index(scope,attr) ::= "(<scope>!=null?<scope>.getTokenIndex():0)"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
tokenLabelPropertyRef_int(scope,attr) ::= "(<scope>!=null?Integer.valueOf(<scope>.getText()):0)"
ruleLabelPropertyRef_start(scope,attr) ::= "(<scope>!=null?((<labelType>)<scope>.start):null)"
ruleLabelPropertyRef_stop(scope,attr) ::= "(<scope>!=null?((<labelType>)<scope>.stop):null)"
ruleLabelPropertyRef_tree(scope,attr) ::= "(<scope>!=null?((<ASTLabelType>)<scope>.tree):null)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
(<scope>!=null?(input.getTokenStream().toString(
input.getTreeAdaptor().getTokenStartIndex(<scope>.start),
input.getTreeAdaptor().getTokenStopIndex(<scope>.start))):null)
<else>
(<scope>!=null?input.toString(<scope>.start,<scope>.stop):null)
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "(<scope>!=null?<scope>.st:null)"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::=
"(<scope>!=null?<scope>.getType():0)"
lexerRuleLabelPropertyRef_line(scope,attr) ::=
"(<scope>!=null?<scope>.getLine():0)"
lexerRuleLabelPropertyRef_pos(scope,attr) ::=
"(<scope>!=null?<scope>.getCharPositionInLine():-1)"
lexerRuleLabelPropertyRef_channel(scope,attr) ::=
"(<scope>!=null?<scope>.getChannel():0)"
lexerRuleLabelPropertyRef_index(scope,attr) ::=
"(<scope>!=null?<scope>.getTokenIndex():0)"
lexerRuleLabelPropertyRef_text(scope,attr) ::=
"(<scope>!=null?<scope>.getText():null)"
lexerRuleLabelPropertyRef_int(scope,attr) ::=
"(<scope>!=null?Integer.valueOf(<scope>.getText()):0)"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "((<labelType>)retval.start)"
rulePropertyRef_stop(scope,attr) ::= "((<labelType>)retval.stop)"
rulePropertyRef_tree(scope,attr) ::= "((<ASTLabelType>)retval.tree)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
input.getTokenStream().toString(
input.getTreeAdaptor().getTokenStartIndex(retval.start),
input.getTreeAdaptor().getTokenStopIndex(retval.start))
<else>
input.toString(retval.start,input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.st"
lexerRulePropertyRef_text(scope,attr) ::= "getText()"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "state.tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "state.tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "_channel"
lexerRulePropertyRef_start(scope,attr) ::= "state.tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(getCharIndex()-1)"
lexerRulePropertyRef_int(scope,attr) ::= "Integer.valueOf(<scope>.getText())"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.tree =<expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.st =<expr>;"
/** How to execute an action (only when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {
<action>
}
<else>
<action>
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
public static final BitSet <name> = new BitSet(new long[]{<words64:{<it>L};separator=",">});<\n>
>>
codeFileExtension() ::= ".java"
true() ::= "true"
false() ::= "false"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CPP/ 0000755 0001750 0001750 00000000000 11410174107 025065 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CPP/CPP.stg 0000644 0001750 0001750 00000112071 11256465232 026242 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group Cpp implements ANTLRCore;
cppTypeInitMap ::= [
"int":"0",
"long":"0",
"float":"0.0",
"double":"0.0",
"bool":"false",
"byte":"0",
"short":"0",
"char":"0",
default:"0" // anything other than an atomic type
]
// What we generate lexer/parser/treeparser, used a suffix in a few places
generatedType() ::= <<
<if(LEXER)>Lexer<endif><if(PARSER)>Parser<endif><if(TREE_PARSER)>TreeParser<endif>
>>
leadIn(type) ::=
<<
/** \file
*
* This <type> file was generated by ANTLR version <ANTLRVersion>
*
* - From the grammar source file : <fileName>
* - On : <generatedTimestamp>
<if(LEXER)>
* - for the lexer : <name><\n>
<endif>
<if(PARSER)>
* - for the parser : <name><\n>
<endif>
<if(TREE_PARSER)>
* - for the tree parser : <name><\n>
<endif>
*
* Edit at your own peril.
*/
>>
standardHeaders() ::=
<<
#include \<antlr3/<generatedType()>.h>
<if(profile)>
#warning "No profiling support.."
<endif>
<if(TREE_PARSER)>
#warning "No tree parsing yet..."
<endif>
>>
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass) ::=
<<
<leadIn("C++ source")>
<@includes>
#include "<name><headerFileExtension()>"
<@end>
<if(actions.(actionScope).header)>
// Header action start ========================================================
<actions.(actionScope).header>
// Header action end ========================================================
<endif>
<headerAction>
<standardHeaders()>
<docComment>
<recognizer>
>>
parserHeaderFile() ::= <<
>>
treeParserHeaderFile() ::= <<
>>
lexerHeaderFile() ::= <<
template\<typename StreamType, typename TokenType, typename TokenBuilder>
class <name> : public antlr3::Lexer\<StreamType,TokenType,TokenBuilder> {
// carry over general types
typedef typename StreamType::position_type position_type;
typedef typename StreamType::char_type char_type;
typedef antlr3::tokenid_type tokenid_type;
typedef antlr3::channel_type channel_type;
typedef antlr3::decision_type decision_type;
// exception shorthands
typedef antlr3::MismatchException\<position_type,char_type> MismatchException;
typedef antlr3::MismatchedRangeException\<position_type,char_type> MismatchedRangeException;
typedef antlr3::MismatchedSetException\<position_type,char_type> MismatchedSetException;
typedef antlr3::EarlyExitException\<position_type> EarlyExitException;
typedef antlr3::NoViableAltException\<position_type> NoViableAltException;
<if(backtracking)>
// @TODO backtracking ruleMemo = new HashMap[<numRules>+1];<\n> <! index from 1..n !>
<endif>
public:
<tokens:{static const tokenid_type <tokenPrefix()><it.name> = <it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
<name>(StreamType* input)
: antlr3::Lexer\<StreamType,TokenType,TokenBuilder>(input)
{
}
<!if(filterMode)!>
<!filteringNextToken()!>
<!endif!>
<rules; separator="\n\n">
// syn preds
<synpreds:{p | <lexerSynpred(p)>}>
// cyclic dfa's
<cyclicDFAs:{dfa | DFA<dfa.decisionNumber> dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this);}; separator="\n">
// dfa tables..
}; // class <name><\n>
>>
headerFile( LEXER,
PARSER,
TREE_PARSER,
actionScope,
actions,
docComment,
recognizer,
name,
tokens,
tokenNames,
rules,
cyclicDFAs,
bitsets,
buildTemplate,
profile,
backtracking,
synpreds,
memoize,
numRules,
fileName,
ANTLRVersion,
generatedTimestamp,
trace,
scopes,
superClass
) ::=
<<
#ifndef _<name>_H
#define _<name>_H
<leadIn("C++ header")>
<actions.(actionScope).headerfile>
<@includes>
<standardHeaders()>
<@end>
<if(LEXER)>
<lexerHeaderFile()>
<endif>
<if(PARSER)>
<parserHeaderFile()>
<endif>
<if(TREE_PARSER)>
<treeParserHeaderFile()>
<endif>
#endif // _<name>_H<\n>
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="Token",
filterMode) ::= <<
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
>>
filteringNextToken() ::= <<
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
public Token nextToken() {
while (true) {
if ( input.LA(1)==CharStream.EOF ) {
return Token.EOF_TOKEN;
}
this->token = 0;
tokenStartCharIndex = getCharIndex();
try {
int m = input.mark();
backtracking=1; <! means we won't throw slow exception !>
failed=false;
mTokens();
backtracking=0;
<! mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1. !>
if ( failed ) {
input.rewind(m);
input.consume(); <! advance one char and try again !>
}
else {
return token;
}
}
catch (RecognitionException re) {
// shouldn't happen in backtracking mode, but...
reportError(re);
recover(re);
}
}
}
public void memoize(IntStream input, int ruleIndex, int ruleStartIndex)
{
if ( backtracking > 1 )
super.memoize(input, ruleIndex, ruleStartIndex);
}
public boolean alreadyParsedRule(IntStream input, int ruleIndex)
{
if ( backtracking > 1 )
return super.alreadyParsedRule(input, ruleIndex);
return false;
}
>>
filteringActionGate() ::= "backtracking == 1"
/** How to generate a parser */
genericParser(
grammar, name, scopes, tokens, tokenNames, rules, numRules, cyclicDFAs,
bitsets, inputStreamType, superClass, ASTLabelType="Object",
labelType, members, filterMode
) ::= <<
// genericParser
class <name> : public <@superClassName><superClass><@end> {
public:
static const char* tokenNames[] = {
"\<invalid>", "\<EOR>", "\<DOWN>", "\<UP>", <tokenNames; separator=", ">
};
<tokens:{static tokenid_type <tokenPrefix()><it.name>=<it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<@members>
<name>(StreamType* input)
: <superClass>\<StreamType,TokenType>(input)
{
<if(backtracking)>
ruleMemo = new HashMap[<numRules>+1];<\n> <! index from 1..n !>
<endif>
}
<@end>
//@TODO public String[] getTokenNames() { return tokenNames; }
//@TODO public String getGrammarFileName() { return "<fileName>"; }
<members>
<rules; separator="\n\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:{dfa | protected DFA<dfa.decisionNumber> dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this);}; separator="\n">
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
};
>>
parser(
grammar, name, scopes, tokens, tokenNames,
rules, numRules, bitsets, ASTLabelType,
superClass="Parser", labelType="Token",
members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="TokenStream", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction,
rules, numRules,
bitsets,
labelType={<ASTLabelType>}, ASTLabelType="Object",
superClass="TreeParser", members={<actions.treeparser.members>}, filterMode
) ::= <<
<genericParser(inputStreamType="TreeNodeStream", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start <ruleName>
public void <ruleName>_fragment(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) throws RecognitionException {
<if(trace)>System.out.println("enter <ruleName> "+input.LT(1)+" failed="+failed+" backtracking="+backtracking);<endif>
<if(trace)>
try {
<block>
}
finally {
System.out.println("exit <ruleName> "+input.LT(1)+" failed="+failed+" backtracking="+backtracking);
}
<else>
<block>
<endif>
}
// $ANTLR end <ruleName>
>>
synpred(name) ::= <<
public boolean <name>() {
this->backtracking++;
<@start()>
int start = input.mark();
try {
<name>_fragment(); // can never throw exception
} catch (RecognitionException re) {
System.err.println("impossible: "+re);
}
boolean success = ! this->failed;
input.rewind(start);
<@stop()>
this->backtracking--;
this->failed = false;
return success;
}<\n>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( backtracking > 0 && alreadyParsedRule(input, <ruleDescriptor.index>) )
return <ruleReturnValue()>;
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>
if (failed)
return <ruleReturnValue()>;
<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>
if (backtracking > 0)
{
failed = true;
return <ruleReturnValue()>;
}
<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
// $ANTLR start <ruleName>
// <fileName>:<description>
public <returnType()> <ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) throw(antlr3::BaseRecognitionException)
{
<if(trace)>
antlr3::Tracer trace(this,"<ruleName>");
System.out.println("enter <ruleName> "+input.LT(1)+" failed="+failed+" backtracking="+backtracking);
<endif>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try {
<ruleMemoization(name=ruleName)>
<block>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
catch (RecognitionException re) {
reportError(re);
recover(input,re);
}<\n>
<endif>
<endif>
<endif>
finally {
<if(trace)>System.out.println("exit <ruleName> "+input.LT(1)+" failed="+failed+" backtracking="+backtracking);<endif>
<ruleCleanUp()>
<(ruleDescriptor.actions.finally):execAction()>
}
<@postamble()>
return <ruleReturnValue()>;
}
// $ANTLR end <ruleName>
>>
catch(decl,action) ::= <<
catch (<e.decl>) {
<e.action>
}
>>
ruleDeclarations() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.push(new <it>_scope());}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.push(new <it.name>_scope());}; separator="\n">
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnType()> retval = new <returnType()>();
retval.start = input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.type> <a.name> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
int <ruleDescriptor.name>_StartIndex = input.index();
<endif>
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]
:{<labelType> <it.label.text>=null;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]
:{List list_<it.label.text>=null;}; separator="\n"
>
<[ruleDescriptor.ruleLabels,ruleDescriptor.ruleListLabels]
:ruleLabelDef(label=it); separator="\n"
>
<[ruleDescriptor.allRuleRefsInAltsWithRewrites,ruleDescriptor.allTokenRefsInAltsWithRewrites]
:{List list_<it>=new ArrayList();}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.pop();}; separator="\n">
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.stop = input.LT(-1);<\n>
<endif>
<if(memoize)>
<if(backtracking)>
if ( backtracking > 0 ) { memoize(input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex); }
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
void m<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) throw(antlr3::BaseRecognitionException)
{
<if(trace)>
antlr3::Tracer trace(this,"<ruleName>");
<endif>
antlr3::CountScope nestingTracker(this->ruleNestingLevel);
StreamType& input(this->getInput());
<if(nakedBlock)>
<ruleDescriptor.actions.init>
<ruleMemoization(name=ruleName)>
<block><\n>
<else>
tokenid_type type = <tokenPrefix()><ruleName>;
channel_type channel = antlr3::Token::DEFAULT_CHANNEL;
position_type start(input.getPosition());
<ruleDescriptor.actions.init>
<ruleMemoization(name=ruleName)>
<block>
<! create token if none exists *and* we are an outermost token rule !>
<execAction({if ( this->token == 0 && this->ruleNestingLevel == 1 ) {
TokenType *tt = TokenBuilder::build(type,start,input,channel);
std::cout \<\< (*tt) \<\< std::endl;
this->emit(tt);
}<\n>
})>
<endif>
}
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
void mTokens() throw(antlr3::BaseRecognitionException)
{
StreamType& input(this->getInput());
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,
maxK,maxAlt,description) ::= <<
// block <fileName>:<description>
decision_type alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// ruleBlock <fileName>:<description>
decision_type alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// ruleBlockSingleAlt <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 0 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// positiveClosureBlock <fileName>:<description>
decision_type cnt<decisionNumber>=0;
<decls>
<@preloop()>
do {
decision_type alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
if ( cnt<decisionNumber> >= 1 )
goto loop<decisionNumber>;
EarlyExitException eee( input.getPosition(), <decisionNumber> );
<@earlyExitException()>
throw eee;
}
cnt<decisionNumber>++;
} while (true);
loop<decisionNumber>: ;
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// closureBlock <fileName>:<description>
<decls>
<@preloop()>
do {
decision_type alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
goto loop<decisionNumber>;
}
} while (true);
loop<decisionNumber>: ;
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> :
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt) ::= <<
// alt <fileName>:<description>
{
<@declarations()>
<elements:element()>
<@cleanup()>
}
>>
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
// element <fileName>:<description>
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex) ::= <<
// tokenRef
<if(label)>
<label> = input.LT(1);<\n>
<endif>
this->match(input,<token>,FOLLOW_<token>_in_<ruleName><elementIndex>);
<checkRuleBacktrackFailure()>
>>
/** ids+=ID no AST building */
tokenRefAndListLabel(token,label,elementIndex) ::= <<
<tokenRef(...)>
<listLabel(...)>
>>
listLabel(label) ::= <<
if (list_<label>==null) list_<label>=new ArrayList();
list_<label>.add(<label>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
// charRef
<if(label)>
<tokenid_type()> <label> = input.LA(1);<\n>
<endif>
this->match(<char>);
<checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b) ::= "this->matchRange(<a>,<b>); <checkRuleBacktrackFailure()>"
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
// matchSet
<if(label)>
<label> = input.LT(1);<\n>
<endif>
if ( <s> )
{
<postmatchCode>
input.consume();
<if(!LEXER)>
errorRecovery=false;
<endif>
<if(backtracking)>failed=false;<endif>
}
else
{
<ruleBacktrackFailure()>
MismatchedSetException mse(input.getPosition(),input.LA(1));
<@mismatchedSetException()>
<if(LEXER)>
this->recover(mse);
<else>
this->recoverFromMismatchedSet(input,mse,FOLLOW_set_in_<ruleName><elementIndex>);
<endif>
throw mse;
}<\n>
>>
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
// lexerStringRef
<if(label)>
position_type <label>Start(input.getPosition());
this->match( <string> );
<checkRuleBacktrackFailure()>
TokenType* <label> = TokenBuilder::build(Token.INVALID_TOKEN_TYPE,<label>Start,input,Token.DEFAULT_CHANNEL);
<else>
this->match( <string> );
<checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label> = input.LT(1);<\n>
<endif>
this->matchAny( input );
<checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(...)>
>>
/** Match . wildcard */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<tokenid_type()> <label> = input.LA(1);<\n>
<endif>
this->matchAny();
<checkRuleBacktrackFailure()>
>>
tokenid_type() ::= "<if(LEXER)>char_type<else>tokenid_type<endif>"
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values.
*/
ruleRef(rule,label,elementIndex,args) ::= <<
following.push(FOLLOW_<rule>_in_<ruleName><elementIndex>);
<if(label)>
<label>=<rule>(<args>);<\n>
<else>
<rule>(<args>);<\n>
<endif>
following.pop();
<checkRuleBacktrackFailure()>
>>
/** ids+=ID */
ruleRefAndListLabel(rule,label,elementIndex,args) ::= <<
<ruleRef(...)>
<listLabel(...)>
>>
/** A lexer rule reference */
lexerRuleRef(rule,label,args) ::= <<
<if(label)>
position_type <label>Start(input.getPosition());
m<rule>(<args>);
<checkRuleBacktrackFailure()>
TokenType* <label> = TokenBuilder::build(Token.INVALID_TOKEN_TYPE,<label>Start,input,Token.DEFAULT_CHANNEL);
<else>
m<rule>(<args>);
<checkRuleBacktrackFailure()>
<endif>
>>
/** EOF in the lexer */
lexerMatchEOF(label) ::= <<
<if(label)>
position_type <label>Start(input.getPosition());
match(EOF);
<checkRuleBacktrackFailure()>
TokenType* <label> = TokenBuilder::build(Token.EOF,<label>Start,input,Token.DEFAULT_CHANNEL);
<else>
match(EOF);
<checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, children, nullableChildList) ::= <<
<root:element()>
<if(nullableChildList)>
if ( input.LA(1)==antlr3::Token::DOWN ) {
match(input, antlr3::Token::DOWN, null);
<checkRuleBacktrackFailure()>
<children:element()>
match(input, antlr3::Token::UP, null);
<checkRuleBacktrackFailure()>
}
<else>
match(input, antlr3::Token::DOWN, null);
<checkRuleBacktrackFailure()>
<children:element()>
match(input, antlr3::Token::UP, null);
<checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) ) {
<ruleBacktrackFailure()>
throw new FailedPredicateException(input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
<if(!semPredState)>
<tokenid_type()> LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<endif>
<edges; separator="\nelse ">
else
{
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;<\n>
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae(input.getPosition(), "<description>", <decisionNumber>, <stateNumber>);<\n>
<@noViableAltException()>
throw nvae;<\n>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
<if(!semPredState)>
<tokenid_type()> LA<decisionNumber>_<stateNumber> = input.LA(<k>);
<endif>
<edges; separator="\nelse ">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
<if(!semPredState)>
<tokenid_type()> LA<decisionNumber>_<stateNumber> = input.LA(<k>);
<endif>
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
else
{
alt<decisionNumber> = <eotPredictsAlt>;
}<\n>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber> = <alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>)
{
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;
<else>
NoViableAltException nvae( input.getPosition(), "<description>", <decisionNumber>, <stateNumber> );<\n>
<@noViableAltException()>
throw nvae;<\n>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) ) {
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber> = <eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n"> {
<targetState>
} break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
// dfaDecision
alt<decisionNumber> = predictDFA<decisionNumber>(input);
>>
/** The overall cyclic DFA chunk; contains all the DFA states */
cyclicDFA(dfa) ::= <<
/* cyclicDFA=<dfa>
*/
// cyclic = <dfa.cyclic>
// numstates = <dfa.numberOfStates>
// startState = <dfa.startState>
// startState.numberOfTransitions = <dfa.startState.NumberOfTransitions>
// startState.lookaheadDepth = <dfa.startState.LookaheadDepth>
const static short <name>dfa<dfa.decisionNumber>_eot[<dfa.numberOfStates>] = {
<dfa.eot; wrap="\n ", separator=",", null="-1">
};
const static short <name>dfa<dfa.decisionNumber>_eof[<dfa.numberOfStates>] = {
<dfa.eof; wrap="\n ", separator=",", null="-1">
};
const static unichar <name>dfa<dfa.decisionNumber>_min[<dfa.numberOfStates>] = {
<dfa.min; wrap="\n ", separator=",", null="0">
};
const static unichar <name>dfa<dfa.decisionNumber>_max[<dfa.numberOfStates>] = {
<dfa.max; wrap="\n ", separator=",", null="0">
};
const static short <name>dfa<dfa.decisionNumber>_accept[<dfa.numberOfStates>] = {
<dfa.accept; wrap="\n ", separator=",", null="-1">
};
const static short <name>dfa<dfa.decisionNumber>_special[<dfa.numberOfStates>] = {
<dfa.special; wrap="\n ", separator=",", null="-1">
};
<dfa.edgeTransitionClassMap.keys:{ table |
const static short <name>dfa<dfa.decisionNumber>_transition<i0>[] = {
<table; separator=", ", wrap="\n ", null="-1">
};
}; null="">
const static short <name>dfa<dfa.decisionNumber>_transition[] = {
<dfa.transitionEdgeTables:{whichTable|<name>dfa<dfa.decisionNumber>_transition<whichTable>,}; separator="\n", null="0 /* fixme? */">
};
<! add attribute for the DFA !>
DFA\<char_type> dfa<dfa.decisionNumber>;
<! this should go in the initializer of the thing
- (id) init
{
if ((self = [super init]) != nil) {
eot = <name>dfa<dfa.decisionNumber>_eot;
eof = <name>dfa<dfa.decisionNumber>_eof;
min = <name>dfa<dfa.decisionNumber>_min;
max = <name>dfa<dfa.decisionNumber>_max;
accept = <name>dfa<dfa.decisionNumber>_accept;
special = <name>dfa<dfa.decisionNumber>_special;
if (!(transition = calloc(<dfa.numberOfStates>, sizeof(void*)))) {
[self release];
return nil;
}
<dfa.transitionEdgeTables:{whichTable|transition[<i0>] = <name>dfa<dfa.decisionNumber>_transition<whichTable>;}; separator="\n", null="">
}
return self;
}
!>
<if(dfa.specialStateSTs)>
int specialStateTransition( int state )
{
int s = state;
switch ( s ) {
<dfa.specialStateSTs:{state |
case <i0> : <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
<if(backtracking)>
if ( recognizer.isBacktracking() ) {
recognizer.setFailed();
return -1;
}<\n>
<endif>
noViableAlt(s, input);
}<\n>
<endif>
<\n>
// <dfa.description>
decision_type predictDFA<dfa.decisionNumber>( StreamType& input )
{
/* mark current location (rewind automatically when the rewinder goes
* out of scope */
antlr3::Rewinder\<position_type> markPoint(input.getPosition());
goto s0; // goto start...
// ...
throw NoViableAltException( input.getPosition(), "<dfa.description>", <dfa.decisionNumber>, 0 /* fixme */ );<\n>
}<\n>
>>
/** A state in a cyclic DFA */
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
// cyclicDFAState
s<stateNumber>: {
<if(semPredState)>
input.rewind();<\n>
<else>
<tokenid_type()> LA<decisionNumber>_<stateNumber> = input.LA(1);
<endif>
<edges>
<if(needErrorClause)>
throw NoViableAltException( input.getPosition(), "<description>", <decisionNumber>, <stateNumber> );<\n>
<endif><\n>
}<\n>
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
// cyclicDFAEdge
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>)
{
input.consume();
goto s<targetStateNumber>;
}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= "goto s<targetStateNumber>;"
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left> && <right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | ||<o>}>)"
notPredicate(pred) ::= "!(<pred>)"
evalPredicate(pred,description) ::= "<pred>"
evalSynPredicate(pred,description) ::= "<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber>==<atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "input.LA(<k>)==<atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber>\>=<lower> && LA<decisionNumber>_<stateNumber>\<=<upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(input.LA(<k>)\>=<lower> && input.LA(<k>)\<=<upper>)"
setTest(ranges) ::= "<ranges; separator=\"||\">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected static class <scope.name> {
<scope.attributes:{<it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
ruleAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected static class <scope.name>_scope {
<scope.attributes:{<it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.name>_return
<else>
<if(ruleDescriptor.singleValueReturnType)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.name>_return
<else>
<if(referencedRule.singleValueReturnType)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
<javaTypeInitMap.(typeName)>
>>
ruleLabelDef(label) ::= <<
<ruleLabelType(referencedRule=label.referencedRule)> <label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
public static class <returnType()> {
<labelType> start, stop;
<if(buildAST)>
<ASTLabelType> tree;
<else>
<if(buildTemplate)>
StringTemplate st;
<endif>
<endif>
<scope.attributes:{<it.decl>;}; separator="\n">
};
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
/** Used in codegen.g to translate $x.y references.
* I could have left actions as StringTemplates to be inserted in
* the output (so they could use attributes inherited from surrounding
* templates), but really wanted to pass in AttributeScope and Attribute
* objects so this translation could query them. So, translation of
* $x.y to executable code occurs before recognizerST.toString() occurs.
* I.e., actions are just text strings during final code generation.
*/
globalAttributeRef(scope,attr) ::= <<
((<scope>)<scope>_stack.peek()).<attr.name>
>>
parameterAttributeRef(attr) ::= "<attr.name>"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack.elementAt(<scope>_stack.size()-<negIndex>-1)).<attr.name>
<else>
<if(index)>
((<scope>_scope)<scope>_stack.elementAt(<index>)).<attr.name>
<else>
((<scope>_scope)<scope>_stack.peek()).<attr.name>
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.singleValueReturnType)>
<scope>
<else>
<scope>.<attr.name>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.singleValueReturnType)>
<attr.name>
<else>
retval.<attr.name>
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach; and they are evaluated early;
// they cannot see TREE_PARSER or PARSER attributes for example. :(
tokenLabelPropertyRef_text(scope,attr) ::= "<scope>.getText()"
tokenLabelPropertyRef_type(scope,attr) ::= "<scope>.getType()"
tokenLabelPropertyRef_line(scope,attr) ::= "<scope>.getLine()"
tokenLabelPropertyRef_pos(scope,attr) ::= "<scope>.getCharPositionInLine()"
tokenLabelPropertyRef_channel(scope,attr) ::= "<scope>.getChannel()"
tokenLabelPropertyRef_index(scope,attr) ::= "<scope>.getTokenIndex()"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
ruleLabelPropertyRef_start(scope,attr) ::= "<scope>.start"
ruleLabelPropertyRef_stop(scope,attr) ::= "<scope>.stop"
ruleLabelPropertyRef_tree(scope,attr) ::= "<scope>.tree"
ruleLabelPropertyRef_text(scope,attr) ::= "input.toString(<scope>.start,<scope>.stop)"
ruleLabelPropertyRef_st(scope,attr) ::= "<scope>.st"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "<scope>.getType()"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "<scope>.getLine()"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "<scope>.getCharPositionInLine()"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "<scope>.getChannel()"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "<scope>.getTokenIndex()"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "<scope>.getText()"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "((<labelType>)retval.start)"
rulePropertyRef_stop(scope,attr) ::= "((<labelType>)retval.stop)"
rulePropertyRef_tree(scope,attr) ::= "((<ASTLabelType>)retval.tree)"
rulePropertyRef_text(scope,attr) ::= "input.toString(retval.start,input.LT(-1))"
rulePropertyRef_st(scope,attr) ::= "retval.st"
// A C T I O N S
emit(type) ::= "emit(<type>);"
setType(type) ::= "setType(<type>);"
/** How to execute an action */
execAction(action) ::= <<
<if(backtracking)>
<if(actions.(actionScope).synpredgate)>
if ( <actions.(actionScope).synpredgate> )
{
<action>
}
<else>
if ( backtracking == 0 )
{
<action>
}
<endif>
<else>
<action>
<endif>
>>
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
public static final BitSet <name> = new BitSet(new long[]{<words64:{<it>L};separator=",">});<\n>
>>
tokenPrefix() ::= "TOK_"
codeFileExtension() ::= ".cpp"
// used in CPPTarget.java to generate the headerfile extension
headerFileExtension() ::= ".h"
true() ::= "true"
false() ::= "false"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Perl5/ 0000755 0001750 0001750 00000000000 11410174107 025432 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Perl5/ASTTreeParser.stg 0000644 0001750 0001750 00000017463 11256465234 030624 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
Copyright (c) 2008 Ronald Blaschke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> _first_0 = null;
<ASTLabelType> _last = null;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<if(rewriteMode)>
retval.tree = (<ASTLabelType>)_first_0;
if ( adaptor.getParent(retval.tree)!=null && adaptor.isNil( adaptor.getParent(retval.tree) ) )
retval.tree = (<ASTLabelType>)adaptor.getParent(retval.tree);
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = (<ASTLabelType>)input.LT(1);
{
<ASTLabelType> _save_last_<treeLevel> = _last;
<ASTLabelType> _first_<treeLevel> = null;
<if(!rewriteMode)>
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.nil();
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( state.backtracking==0 )<endif>
<if(root.el.rule)>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>.tree;
<else>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==Token.DOWN ) {
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( state.backtracking==0 )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(backtracking)>if ( state.backtracking==0 ) <endif>
<if(!rewriteMode)>
adaptor.addChild(root_<treeLevel>, <label>.getTree());
<else> <! rewrite mode !>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>.tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking==0 ) <endif>root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>.getTree(), root_<treeLevel>);
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextNode())
<else>
stream_<token>.nextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking==0 ) {<\n><endif>
retval.tree = (<ASTLabelType>)adaptor.rulePostProcessing(root_0);
<if(backtracking)>}<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Perl5/Perl5.stg 0000644 0001750 0001750 00000115253 11256465234 027163 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
Copyright (c) 2007-2008 Ronald Blaschke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group Perl5 implements ANTLRCore;
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
# $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<actions.(actionScope).header>
<@imports>
<if(TREE_PARSER)>
<endif>
<if(backtracking)>
<endif>
<@end>
<docComment>
<recognizer>
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="Token",
filterMode, superClass="ANTLR::Runtime::Lexer") ::= <<
package <name>;
use ANTLR::Runtime::Class;
use Carp;
use English qw( -no_match_vars ) ;
use Readonly;
use Switch;
use ANTLR::Runtime::BaseRecognizer;
use ANTLR::Runtime::DFA;
use ANTLR::Runtime::NoViableAltException;
extends 'ANTLR::Runtime::Lexer';
use constant {
HIDDEN => ANTLR::Runtime::BaseRecognizer->HIDDEN
};
use constant {
<tokens:{ <it.name> => <it.type>, }; separator="\n">
};
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
sub BUILD {
my ($self, $arg_ref) = @_;
$self->init_dfas();
}
sub get_grammar_file_name {
return "<fileName>";
}
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<synpreds:{p | <lexerSynpred(p)>}>
<cyclicDFAs:{dfa | has 'dfa<dfa.decisionNumber>';}; separator="\n">
sub init_dfas {
my ($self) = @_;
<cyclicDFAs:{dfa |
$self->dfa<dfa.decisionNumber>(<name>::DFA<dfa.decisionNumber>->new({ recognizer => $self }));
}; separator="\n">
return;
}
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
1;
>>
perlTypeInitMap ::= [
"$":"undef",
"@":"()",
"%":"()",
default:"undef"
]
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
public Token nextToken() {
while (true) {
if ( input.LA(1)==CharStream.EOF ) {
return Token.EOF_TOKEN;
}
token = null;
channel = Token.DEFAULT_CHANNEL;
tokenStartCharIndex = input.index();
tokenStartCharPositionInLine = input.getCharPositionInLine();
tokenStartLine = input.getLine();
text = null;
try {
int m = input.mark();
backtracking=1; <! means we won't throw slow exception !>
failed=false;
mTokens();
backtracking=0;
<! mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1. !>
if ( failed ) {
input.rewind(m);
input.consume(); <! advance one char and try again !>
}
else {
emit();
return token;
}
}
catch (RecognitionException re) {
// shouldn't happen in backtracking mode, but...
reportError(re);
recover(re);
}
}
}
public void memoize(IntStream input,
int ruleIndex,
int ruleStartIndex)
{
if ( backtracking>1 ) super.memoize(input, ruleIndex, ruleStartIndex);
}
public boolean alreadyParsedRule(IntStream input, int ruleIndex) {
if ( backtracking>1 ) return super.alreadyParsedRule(input, ruleIndex);
return false;
}
>>
actionGate() ::= "$self->state->backtracking==0"
filteringActionGate() ::= "backtracking==1"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass, filterMode,
ASTLabelType="Object", labelType, members) ::= <<
package <name>;
use ANTLR::Runtime::Class;
use English qw( -no_match_vars ) ;
use Readonly;
use Switch;
use Carp;
use ANTLR::Runtime::BitSet;
extends '<@superClassName><superClass><@end>';
Readonly my $token_names => [
"\<invalid>", "\<EOR>", "\<DOWN>", "\<UP>", <tokenNames; separator=", ">
];
use constant {
<tokens:{ <it.name> => <it.type>, }; separator="\n">
};
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<@members>
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
sub BUILD {
my ($self, $arg_ref) = @_;
<if(backtracking)>
$self->state->rule_memo({});<\n>
<endif>
}
<@end>
sub get_token_names {
return $token_names;
}
sub get_grammar_file_name {
return "<fileName>";
}
<members>
<rules; separator="\n\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:{dfa | dfa<dfa.decisionNumber> = __PACKAGE__::DFA<dfa.decisionNumber>->new($self);}; separator="\n">
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
1;
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType, superClass="ANTLR::Runtime::Parser", labelType="ANTLR::Runtime::Token", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="ANTLR::Runtime::TokenStream", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="Object", superClass="ANTLR::Runtime::TreeParser", members={<actions.treeparser.members>}, filterMode) ::= <<
<genericParser(inputStreamType="TreeNodeStream", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
# $ANTLR start <ruleName>
sub <ruleName>_fragment {
# <ruleDescriptor.parameterScope:parameterScope(scope=it)>
<if(trace)>
$self->traceIn("<ruleName>_fragment", <ruleDescriptor.index>);
eval {
<block>
};
$self->traceOut("<ruleName>_fragment", <ruleDescriptor.index>);
if ($EVAL_ERROR) {
croak $EVAL_ERROR;
}
<else>
<block>
<endif>
}
# $ANTLR end <ruleName>
>>
synpred(name) ::= <<
public final boolean <name>() {
backtracking++;
<@start()>
int start = input.mark();
try {
<name>_fragment(); // can never throw exception
} catch (RecognitionException re) {
System.err.println("impossible: "+re);
}
boolean success = !failed;
input.rewind(start);
<@stop()>
backtracking--;
failed=false;
return success;
}<\n>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( backtracking>0 && alreadyParsedRule(input, <ruleDescriptor.index>) ) { return <ruleReturnValue()>; }
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>
if ($self->state->failed) {
return <ruleReturnValue()>;
}
<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if (backtracking>0) {failed=true; return <ruleReturnValue()>;}<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
# $ANTLR start <ruleName>
# <fileName>:<description>
sub <ruleName>() {
my ($self, <ruleDescriptor.parameterScope:parameterScope(scope=it)>) = @_;
<if(trace)>$self->traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
eval {
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
};
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
my $exception = $EVAL_ERROR;
if (ref $exception && $exception->isa('ANTLR::Runtime::RecognitionException')) {
$self->report_error($exception);
$self->recover($self->input, $exception);
$exception = undef;
}<\n>
<endif>
<endif>
<endif>
<if(trace)>$self->traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
if ($exception) {
croak $exception;
#$exception->rethrow();
}
<@postamble()>
return <ruleReturnValue()>;
}
# $ANTLR end <ruleName>
>>
catch(decl,action) ::= <<
catch (<e.decl>) {
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
my $retval = <returnType()>->new();
$retval->set_start($self->input->LT(1));<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
my $<a.name> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
my $<ruleDescriptor.name>_start_index = $self->input->index();
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.push(new <it>_scope());}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.push(new <it.name>_scope());}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.pop();}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]
:{my $<it.label.text> = undef;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]
:{List list_<it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|RuleReturnScope <ll.label.text> = null;}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<labelType> <it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.charLabels:{my $<it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleListLabels]
:{List list_<it.label.text>=null;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
$<ruleDescriptor.singleValueReturnName>
<else>
$retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
$retval->set_stop($self->input->LT(-1));<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( backtracking>0 ) { memoize(input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex); }
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
# $ANTLR start <ruleName>
sub m_<ruleName> {
# <ruleDescriptor.parameterScope:parameterScope(scope=it)>
my ($self) = @_;
<if(trace)>traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleDeclarations()>
eval {
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
my $_type = <ruleName>;
my $_channel = $self->DEFAULT_TOKEN_CHANNEL;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
$self->state->type($_type);
$self->state->channel($_channel);
<(ruleDescriptor.actions.after):execAction()>
<endif>
};
<if(trace)>traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
if ($EVAL_ERROR) {
croak $EVAL_ERROR;
}
}
# $ANTLR end <ruleName>
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
sub m_tokens {
my ($self) = @_;
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
my $alt<decisionNumber> = <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch ($alt<decisionNumber>) {
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
my $alt<decisionNumber> = <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch ($alt<decisionNumber>) {
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
# <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
# <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
my $cnt<decisionNumber> = 0;
<decls>
<@preloop()>
LOOP<decisionNumber>:
while (1) {
my $alt<decisionNumber> = <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch ($alt<decisionNumber>) {
<alts:altSwitchCase()>
else {
if ( $cnt<decisionNumber> >= 1 ) { last LOOP<decisionNumber> }
<ruleBacktrackFailure()>
my $eee =
ANTLR::Runtime::EarlyExitException->new(<decisionNumber>, $self->input);
<@earlyExitException()>
croak $eee;
}
}
++$cnt<decisionNumber>;
}
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
<decls>
<@preloop()>
LOOP<decisionNumber>:
while (1) {
my $alt<decisionNumber> = <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch ($alt<decisionNumber>) {
<alts:altSwitchCase()>
else { last LOOP<decisionNumber> }
}
}
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> {
<@prealt()>
<it>
}<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
# <fileName>:<description>
{
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
}
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)>$<label> =<endif>$self->match($self->input, <token>, $FOLLOW_<token>_in_<ruleName><elementIndex>);
<checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label>==null) list_<label>=new ArrayList();
list_<label>.add(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = $self->input->LA(1);<\n>
<endif>
$self->match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = $self->input->LA(1);<\n>
<endif>
$self->match_range(<a>,<b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= $self->input->LA(1);<\n>
<else>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<endif>
if ( <s> ) {
$self->input->consume();
<postmatchCode>
<if(!LEXER)>
$self->state->error_recovery(0);
<endif>
<if(backtracking)>failed=false;<endif>
}
else {
<ruleBacktrackFailure()>
my $mse =
ANTLR::Runtime::MismatchedSetException->new(undef, $self->input);
<@mismatchedSetException()>
<if(LEXER)>
$self->recover($mse);
$mse->throw();
<else>
$mse->throw();
<! use following code to make it recover inline; remove throw mse;
$self->recoverFromMismatchedSet($self->input, $mse, $FOLLOW_set_in_<ruleName><elementIndex>);
!>
<endif>
}<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
int <label>Start = getCharIndex();
$self->match(<string>); <checkRuleBacktrackFailure()>
<labelType> <label> = new CommonToken(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start, getCharIndex()-1);
<else>
$self->match(<string>); <checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
matchAny(input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = $self->input->LA(1);<\n>
<endif>
matchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
$self->push_follow($FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)>
$<label> = $self-><rule.name>(<args; separator=", ">);<\n>
<else>
$self-><rule.name>(<args; separator=", ">);<\n>
<endif>
$self->state->_fsp($self->state->_fsp - 1);
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference.
*
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
int <label>Start<elementIndex> = getCharIndex();
$self->m_<rule>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> = new CommonToken(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, getCharIndex()-1);
<else>
$self->m_<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
int <label>Start<elementIndex> = getCharIndex();
match(EOF); <checkRuleBacktrackFailure()>
<labelType> <label> = new CommonToken(input, EOF, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, getCharIndex()-1);
<else>
match(EOF); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==Token.DOWN ) {
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) ) {
<ruleBacktrackFailure()>
throw new FailedPredicateException(input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
my $LA<decisionNumber>_<stateNumber> = $self->input->LA(<k>);<\n>
<edges; separator="\nels">
else {
<if(eotPredictsAlt)>
$alt<decisionNumber> = <eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
my $nvae =
ANTLR::Runtime::NoViableAltException->new({
grammar_decision_description => "<description>",
decision_number => <decisionNumber>,
state_number => <stateNumber>,
input => $self->input,
});<\n>
<@noViableAltException()>
croak $nvae;<\n>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
my $LA<decisionNumber>_<stateNumber> = $self->input->LA(<k>);<\n>
<edges; separator="\nels">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
my $LA<decisionNumber>_<stateNumber> = $self->input->LA(<k>);<\n>
<edges; separator="\nels"><\n>
<if(eotPredictsAlt)>
<if(!edges)>
$alt<decisionNumber> = <eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else {
$alt<decisionNumber> = <eotPredictsAlt>;
}<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "$alt<decisionNumber> = <alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( $self->input->LA(<k>) ) {
<edges; separator="\n">
else {
<if(eotPredictsAlt)>
$alt<decisionNumber> = <eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
my $nvae =
ANTLR::Runtime::NoViableAltException->new({
grammar_decision_description => "<description>",
decision_number => <decisionNumber>,
state_number => <stateNumber>,
input => $self->input,
});<\n>
<@noViableAltException()>
croak $nvae;<\n>
<endif>
}
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( $self->input->LA(<k>) ) {
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( $self->input->LA(<k>) ) {
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
else { $alt<decisionNumber> = <eotPredictsAlt> }<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
case [<labels:{ <it> }; separator=", ">] { <targetState> }
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
$alt<decisionNumber> = $self->dfa<decisionNumber>->predict($self->input);
>>
/* Dump DFA tables as run-length-encoded Strings of octal values.
* Can't use hex as compiler translates them before compilation.
* These strings are split into multiple, concatenated strings.
* Java puts them back together at compile time thankfully.
* Java cannot handle large static arrays, so we're stuck with this
* encode/decode approach. See analysis and runtime DFA for
* the encoding methods.
*/
cyclicDFA(dfa) ::= <<
Readonly my $DFA<dfa.decisionNumber>_eot => ANTLR::Runtime::DFA->unpack_rle([ <dfa.javaCompressedEOT; separator=", "> ]);
Readonly my $DFA<dfa.decisionNumber>_eof => ANTLR::Runtime::DFA->unpack_rle([ <dfa.javaCompressedEOF; separator=", "> ]);
Readonly my $DFA<dfa.decisionNumber>_min => ANTLR::Runtime::DFA->unpack_rle([ <dfa.javaCompressedMin; separator=", "> ]);
Readonly my $DFA<dfa.decisionNumber>_max => ANTLR::Runtime::DFA->unpack_rle([ <dfa.javaCompressedMax; separator=", "> ]);
Readonly my $DFA<dfa.decisionNumber>_accept => ANTLR::Runtime::DFA->unpack_rle([ <dfa.javaCompressedAccept; separator=", "> ]);
Readonly my $DFA<dfa.decisionNumber>_special => ANTLR::Runtime::DFA->unpack_rle([ <dfa.javaCompressedSpecial; separator=", "> ]);
Readonly my $DFA<dfa.decisionNumber>_transition => [ <dfa.javaCompressedTransition:{s|ANTLR::Runtime::DFA->unpack_rle([ <s; separator=", "> ])}; separator=", "> ];
{
package <name>::DFA<dfa.decisionNumber>;
use ANTLR::Runtime::Class;
use strict;
use warnings;
extends 'ANTLR::Runtime::DFA';
sub BUILD {
my $self = shift;
my $param_ref = __PACKAGE__->unpack_params(@_, {
spec => [
{
name => 'recognizer',
isa => 'ANTLR::Runtime::BaseRecognizer'
},
]
});
$self->recognizer($param_ref->{recognizer});
$self->decision_number(<dfa.decisionNumber>);
$self->eot($DFA<dfa.decisionNumber>_eot);
$self->eof($DFA<dfa.decisionNumber>_eof);
$self->min($DFA<dfa.decisionNumber>_min);
$self->max($DFA<dfa.decisionNumber>_max);
$self->accept($DFA<dfa.decisionNumber>_accept);
$self->special($DFA<dfa.decisionNumber>_special);
$self->transition($DFA<dfa.decisionNumber>_transition);
}
sub get_description {
return "<dfa.description>";
}
<@errorMethod()>
<if(dfa.specialStateSTs)>
sub special_state_transition {
my ($self, $param_ref) = unpack_params(@_, {
spec => [
{
name => 's',
type => SCALAR,
},
{
name => 'input',
isa => 'ANTLR::Runtime::IntStream',
}
]
});
my $s = $param_ref->{s};
my $input = $param_ref->{input};
switch ($s) {
<dfa.specialStateSTs:{state |
case <i0> \{ <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
}
<if(backtracking)>
if ($self->state->backtracking > 0) {
$self->state->failed = 1;
return -1;
}<\n>
<endif>
my $nvae =
ANTLR::Runtime::NoViableAltException->new({
grammar_decision_description => $self->get_description(),
decision_number => <dfa.decisionNumber>,
state_number => $s,
input => $input,
});<\n>
$self->error($nvae);
$nvae->throw();
}<\n>
<endif>
}<\n>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
my $input = $self->input;
my $LA<decisionNumber>_<stateNumber> = $input->LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
my $index<decisionNumber>_<stateNumber> = $input->index();
$input->rewind();<\n>
<endif>
s = -1;
<edges; separator="\nels">
<if(semPredState)> <! return input cursor to state before we rewound !>
input.seek(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s>=0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {s = <targetStateNumber>;}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left> && <right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | ||<o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "<pred>"
evalSynPredicate(pred,description) ::= "<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "$LA<decisionNumber>_<stateNumber> eq <atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "$self->input->LA(<k>) eq <atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
($LA<decisionNumber>_<stateNumber> ge <lower> && $LA<decisionNumber>_<stateNumber> le <upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "($self->input->LA(<k>) ge <lower> && $self->input->LA(<k>) le <upper>)"
setTest(ranges) ::= "<ranges; separator=\" || \">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected static class <scope.name>_scope {
<scope.attributes:{<it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
ruleAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected static class <scope.name>_scope {
<scope.attributes:{<it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.name>_return
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Generate the Java type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "undef".
*/
initValue(typeName) ::= <<
<if(typeName)>
<perlTypeInitMap.(typeName)>
<else>
undef
<endif>
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
my $<label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
{
package <returnType()>;
use ANTLR::Runtime::Class;
extends 'ANTLR::Runtime::<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope';
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
}
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{$<it.name>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "$<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "$<attr.name> =<expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack.elementAt(<scope>_stack.size()-<negIndex>-1)).<attr.name>
<else>
<if(index)>
((<scope>_scope)<scope>_stack.elementAt(<index>)).<attr.name>
<else>
((<scope>_scope)<scope>_stack.peek()).<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack.elementAt(<scope>_stack.size()-<negIndex>-1)).<attr.name> =<expr>;
<else>
<if(index)>
((<scope>_scope)<scope>_stack.elementAt(<index>)).<attr.name> =<expr>;
<else>
((<scope>_scope)<scope>_stack.peek()).<attr.name> =<expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
$<scope>.<attr.name>
<else>
$<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
$<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name> =<expr>;
<else>
$<attr.name> =<expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "$<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "$<scope>->get_text()"
tokenLabelPropertyRef_type(scope,attr) ::= "<scope>.getType()"
tokenLabelPropertyRef_line(scope,attr) ::= "<scope>.getLine()"
tokenLabelPropertyRef_pos(scope,attr) ::= "<scope>.getCharPositionInLine()"
tokenLabelPropertyRef_channel(scope,attr) ::= "<scope>.getChannel()"
tokenLabelPropertyRef_index(scope,attr) ::= "<scope>.getTokenIndex()"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
ruleLabelPropertyRef_start(scope,attr) ::= "((<labelType>)<scope>.start)"
ruleLabelPropertyRef_stop(scope,attr) ::= "((<labelType>)<scope>.stop)"
ruleLabelPropertyRef_tree(scope,attr) ::= "((<ASTLabelType>)<scope>.tree)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
input.getTokenStream().toString(
input.getTreeAdaptor().getTokenStartIndex(<scope>.start),
input.getTreeAdaptor().getTokenStopIndex(<scope>.start))
<else>
substr($self->input, $<scope>->start, $<scope>->stop)
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "<scope>.st"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "$<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "<scope>.getType()"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "<scope>.getLine()"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "<scope>.getCharPositionInLine()"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "<scope>.getChannel()"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "<scope>.getTokenIndex()"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "<scope>.getText()"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "((<labelType>)retval.start)"
rulePropertyRef_stop(scope,attr) ::= "((<labelType>)retval.stop)"
rulePropertyRef_tree(scope,attr) ::= "((<ASTLabelType>)retval.tree)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
input.getTokenStream().toString(
input.getTreeAdaptor().getTokenStartIndex(retval.start),
input.getTreeAdaptor().getTokenStopIndex(retval.start))
<else>
input.toString(retval.start,input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.st"
lexerRulePropertyRef_text(scope,attr) ::= "getText()"
lexerRulePropertyRef_type(scope,attr) ::= "$_type"
lexerRulePropertyRef_line(scope,attr) ::= "tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "$_channel"
lexerRulePropertyRef_start(scope,attr) ::= "tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(getCharIndex()-1)"
lexerRulePropertyRef_self(scope,attr) ::= "$self"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.tree =<expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.st =<expr>;"
/** How to execute an action */
execAction(action) ::= <<
<if(backtracking)>
<if(actions.(actionScope).synpredgate)>
if ( <actions.(actionScope).synpredgate> ) {
<action>
}
<else>
if ( backtracking==0 ) {
<action>
}
<endif>
<else>
<action>
<endif>
>>
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
Readonly my $<name> => ANTLR::Runtime::BitSet->new({ words64 => [ <words64:{'<it>'};separator=", "> ] });<\n>
>>
codeFileExtension() ::= ".pm"
true() ::= "1"
false() ::= "0"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/ 0000755 0001750 0001750 00000000000 11410174107 025623 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/ASTTreeParser.stg 0000644 0001750 0001750 00000022362 11256465232 031005 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> _first_0 = null;
<ASTLabelType> _last = null;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(rewriteMode)>
retval.Tree = (<ASTLabelType>)_first_0;
if ( adaptor.GetParent(retval.Tree)!=null && adaptor.IsNil( adaptor.GetParent(retval.Tree) ) )
retval.Tree = (<ASTLabelType>)adaptor.GetParent(retval.Tree);
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = (<ASTLabelType>)input.LT(1);
{
<ASTLabelType> _save_last_<treeLevel> = _last;
<ASTLabelType> _first_<treeLevel> = null;
<if(!rewriteMode)>
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.GetNilNode();
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
<if(root.el.rule)>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = (<ASTLabelType>) <root.el.label>.Tree;
<else>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1) == Token.DOWN )
{
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
adaptor.AddChild(root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>
}
<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>
}
<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard and auto dup the node/subtree */
wildcard(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.wildcard(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.DupTree(<label>);
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>
<if(!rewriteMode)>
adaptor.AddChild(root_<treeLevel>, <label>.Tree);
<else> <! rewrite mode !>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = (<ASTLabelType>) <label>.Tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking == 0 ) <endif>root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>.Tree, root_<treeLevel>);
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.NextNode())
<else>
stream_<token>.NextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<endif>
retval.Tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
<if(backtracking)>
}
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/ST.stg 0000644 0001750 0001750 00000012030 11256465232 026676 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template subgroup to add template rewrite output
* If debugging, then you'll also get STDbg.stg loaded.
*/
group ST;
@outputFile.imports() ::= <<
<@super.imports()>
using Antlr.StringTemplate;
using Antlr.StringTemplate.Language;
<if(!backtracking)>
using Hashtable = System.Collections.Hashtable;
<endif>
>>
/** Add this to each rule's return value struct */
@returnScope.ruleReturnMembers() ::= <<
private StringTemplate st;
public StringTemplate ST { get { return st; } set { st = value; } }
public override object Template { get { return st; } }
public override string ToString() { return (st == null) ? null : st.ToString(); }
>>
@genericParser.members() ::= <<
<@super.members()>
protected StringTemplateGroup templateLib =
new StringTemplateGroup("<name>Templates", typeof(AngleBracketTemplateLexer));
public StringTemplateGroup TemplateLib
{
get { return this.templateLib; }
set { this.templateLib = value; }
}
/// \<summary> Allows convenient multi-value initialization:
/// "new STAttrMap().Add(...).Add(...)"
/// \</summary>
protected class STAttrMap : Hashtable
{
public STAttrMap Add(string attrName, object value)
{
base.Add(attrName, value);
return this;
}
public STAttrMap Add(string attrName, int value)
{
base.Add(attrName, value);
return this;
}
}
>>
/** x+=rule when output=template */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Template",...)>
>>
rewriteTemplate(alts) ::= <<
// TEMPLATE REWRITE
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
}
<else>
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
<endif>
>>
replaceTextInLine() ::= <<
<if(TREE_PARSER)>
((TokenRewriteStream)input.TokenStream).Replace(
input.TreeAdaptor.GetTokenStartIndex(retval.Start),
input.TreeAdaptor.GetTokenStopIndex(retval.Start),
retval.ST);
<else>
((TokenRewriteStream)input).Replace(
((IToken)retval.Start).TokenIndex,
input.LT(-1).TokenIndex,
retval.ST);
<endif>
>>
rewriteTemplateAlt() ::= <<
// <it.description>
<if(it.pred)>
if (<it.pred>) {
retval.ST = <it.alt>;
}<\n>
<else>
{
retval.ST = <it.alt>;
}<\n>
<endif>
>>
rewriteEmptyTemplate(alts) ::= <<
null;
>>
/** Invoke a template with a set of attribute name/value pairs.
* Set the value of the rule's template *after* having set
* the attributes because the rule's template might be used as
* an attribute to build a bigger template; you get a self-embedded
* template.
*/
rewriteExternalTemplate(name,args) ::= <<
templateLib.GetInstanceOf("<name>"<if(args)>,
new STAttrMap()<args:{a | .Add("<a.name>", <a.value>)}>
<endif>)
>>
/** expr is a string expression that says what template to load */
rewriteIndirectTemplate(expr,args) ::= <<
templateLib.GetInstanceOf(<expr><if(args)>,
new STAttrMap()<args:{a | .Add("<a.name>", <a.value>)}>
<endif>)
>>
/** Invoke an inline template with a set of attribute name/value pairs */
rewriteInlineTemplate(args, template) ::= <<
new StringTemplate(templateLib, "<template>"<if(args)>,
new STAttrMap()<args:{a | .Add("<a.name>", <a.value>)}>
<endif>)
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
<action>
>>
/** An action has %st.attrName=expr; or %{st}.attrName=expr; */
actionSetAttribute(st,attrName,expr) ::= <<
(<st>).SetAttribute("<attrName>",<expr>);
>>
/** Translate %{stringExpr} */
actionStringConstructor(stringExpr) ::= <<
new StringTemplate(templateLib,<stringExpr>)
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/ASTParser.stg 0000644 0001750 0001750 00000015635 11256465232 030172 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
// Conversion of the second argument necessary, but harmless
retval.Tree = (<ASTLabelType>)adaptor.ErrorNode(input, (IToken) retval.Start, input.LT(-1), re);
<! System.Console.WriteLine("<ruleName> returns " + ((CommonTree)retval.Tree).ToStringTree()); !>
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = <createNodeFromToken(...)>;
adaptor.AddChild(root_0, <label>_tree);
<if(backtracking)>
}
<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = <createNodeFromToken(...)>;
root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_0);
<if(backtracking)>
}
<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( state.backtracking == 0 ) <endif>adaptor.AddChild(root_0, <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( state.backtracking == 0 ) <endif>root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<createNodeFromToken(...)>, root_0);})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( state.backtracking == 0 ) <endif>adaptor.AddChild(root_0, <label>.Tree);
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( state.backtracking == 0 ) <endif>root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>.Tree, root_0);
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = (<ASTLabelType>)adaptor.Create(<label>);
adaptor.AddChild(root_0, <label>_tree);
<if(backtracking)>
}
<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = (<ASTLabelType>)adaptor.Create(<label>);
root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_0);
<if(backtracking)>
}
<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
new <hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
(<ASTLabelType>)adaptor.Create(<label>)
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<endif>
retval.Tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
<if(!TREE_PARSER)>
adaptor.SetTokenBoundaries(retval.Tree, (IToken) retval.Start, (IToken) retval.Stop);
<endif>
<if(backtracking)>
}
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/Dbg.stg 0000644 0001750 0001750 00000022565 11256465232 027062 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to normal Java output;
* If ASTs are built, then you'll also get ASTDbg.stg loaded.
*/
group Dbg;
@outputFile.debugPreprocessor() ::= "#define ANTLR_DEBUG"
@outputFile.imports() ::= <<
<@super.imports()>
using Antlr.Runtime.Debug;
using IOException = System.IO.IOException;
>>
@genericParser.members() ::= <<
<if(grammar.grammarIsRoot)>
public static readonly string[] ruleNames = new string[] {
"invalidRule", <grammar.allImportedRules:{rST | "<rST.name>"}; wrap="\n ", separator=", ">
};<\n>
<endif>
<if(grammar.grammarIsRoot)> <! grammar imports other grammar(s) !>
private int ruleLevel = 0;
public int RuleLevel {
get { return ruleLevel; }
}
public void IncRuleLevel() { ruleLevel++; }
public void DecRuleLevel() { ruleLevel--; }
<if(profile)>
<ctorForProfilingRootGrammar()>
<else>
<ctorForRootGrammar()>
<endif>
<ctorForPredefinedListener()>
<else> <! imported grammar !>
public int RuleLevel {
get { return <grammar.delegators:{g| <g:delegateName()>}>.RuleLevel; }
}
public void IncRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.IncRuleLevel(); }
public void DecRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.DecRuleLevel(); }
<ctorForDelegateGrammar()>
<endif>
<if(profile)>
override public bool AlreadyParsedRule(IIntStream input, int ruleIndex)
{
((Profiler)dbg).ExamineRuleMemoization(input, ruleIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
return base.AlreadyParsedRule(input, ruleIndex);
}<\n>
override public void Memoize(IIntStream input,
int ruleIndex,
int ruleStartIndex)
{
((Profiler)dbg).Memoize(input, ruleIndex, ruleStartIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
base.Memoize(input, ruleIndex, ruleStartIndex);
}<\n>
<endif>
protected bool EvalPredicate(bool result, string predicate)
{
dbg.SemanticPredicate(result, predicate);
return result;
}<\n>
>>
ctorForRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
<! Same except we add port number and profile stuff if root grammar !>
public <name>(<inputStreamType> input)
: this(input, DebugEventSocketProxy.DEFAULT_DEBUGGER_PORT, new RecognizerSharedState()) {
}
public <name>(<inputStreamType> input, int port, RecognizerSharedState state)
: base(input, state) {
<parserCtorBody()>
<createListenerAndHandshake()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
>>
@parserCtorBody.initializeCyclicDFAs() ::= <<
InitializeCyclicDFAs(dbg);
>>
ctorForProfilingRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
public <name>(<inputStreamType> input) {
this(input, new Profiler(null), new RecognizerSharedState());
}
public <name>(<inputStreamType> input, IDebugEventListener dbg, RecognizerSharedState state)
: base(input, dbg, state) {
Profiler p = (Profiler)dbg;
p.setParser(this);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}
<\n>
>>
/** Basically we don't want to set any dbg listeners are root will have it. */
ctorForDelegateGrammar() ::= <<
public <name>(<inputStreamType> input, IDebugEventListener dbg, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: base(input, dbg, state) {
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, this, this.state<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
}<\n>
>>
ctorForPredefinedListener() ::= <<
public <name>(<inputStreamType> input, IDebugEventListener dbg)
: <@superClassRef>base(input, dbg, new RecognizerSharedState())<@end> {
<if(profile)>
Profiler p = (Profiler)dbg;
p.setParser(this);
<endif>
<parserCtorBody()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
>>
createListenerAndHandshake() ::= <<
<if(TREE_PARSER)>
DebugEventSocketProxy proxy = new DebugEventSocketProxy(this, port, input.TreeAdaptor);
<else>
DebugEventSocketProxy proxy = new DebugEventSocketProxy(this, port, null);
<endif>
DebugListener = proxy;
try
{
proxy.Handshake();
}
catch (IOException ioe)
{
ReportError(ioe);
}
>>
@genericParser.superClassName() ::= "Debug<@super.superClassName()>"
@rule.preamble() ::= <<
try {
dbg.EnterRule(GrammarFileName, "<ruleName>");
if ( RuleLevel==0 ) {dbg.Commence();}
IncRuleLevel();
dbg.Location(<ruleDescriptor.tree.line>, <ruleDescriptor.tree.column>);<\n>
>>
@lexer.debugInitializeCyclicDFAs() ::= "IDebugEventListener dbg"
@lexer.debugAddition() ::= ", dbg"
@genericParser.debugInitializeCyclicDFAs() ::= "IDebugEventListener dbg"
@genericParser.debugAddition() ::= ", dbg"
@rule.postamble() ::= <<
dbg.Location(<ruleDescriptor.EORNode.line>, <ruleDescriptor.EORNode.column>);<\n>
}
finally {
dbg.ExitRule(GrammarFileName, "<ruleName>");
DecRuleLevel();
if ( RuleLevel==0 ) {dbg.Terminate();}
}<\n>
>>
@synpred.start() ::= "dbg.BeginBacktrack(state.backtracking);"
@synpred.stop() ::= "dbg.EndBacktrack(state.backtracking, success);"
// Common debug event triggers used by region overrides below
enterSubRule() ::=
"try { dbg.EnterSubRule(<decisionNumber>);<\n>"
exitSubRule() ::=
"} finally { dbg.ExitSubRule(<decisionNumber>); }<\n>"
enterDecision() ::=
"try { dbg.EnterDecision(<decisionNumber>);<\n>"
exitDecision() ::=
"} finally { dbg.ExitDecision(<decisionNumber>); }<\n>"
enterAlt(n) ::= "dbg.EnterAlt(<n>);<\n>"
// Region overrides that tell various constructs to add debugging triggers
@block.predecision() ::= "<enterSubRule()><enterDecision()>"
@block.postdecision() ::= "<exitDecision()>"
@block.postbranch() ::= "<exitSubRule()>"
@ruleBlock.predecision() ::= "<enterDecision()>"
@ruleBlock.postdecision() ::= "<exitDecision()>"
@ruleBlockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@blockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@positiveClosureBlock.preloop() ::= "<enterSubRule()>"
@positiveClosureBlock.postloop() ::= "<exitSubRule()>"
@positiveClosureBlock.predecision() ::= "<enterDecision()>"
@positiveClosureBlock.postdecision() ::= "<exitDecision()>"
@positiveClosureBlock.earlyExitException() ::=
"dbg.RecognitionException(eee<decisionNumber>);<\n>"
@closureBlock.preloop() ::= "<enterSubRule()>"
@closureBlock.postloop() ::= "<exitSubRule()>"
@closureBlock.predecision() ::= "<enterDecision()>"
@closureBlock.postdecision() ::= "<exitDecision()>"
@altSwitchCase.prealt() ::= "<enterAlt(n=i)>"
@element.prematch() ::=
"dbg.Location(<it.line>,<it.pos>);"
@matchSet.mismatchedSetException() ::=
"dbg.RecognitionException(mse);"
@dfaState.noViableAltException() ::= "dbg.RecognitionException(nvae_d<decisionNumber>s<stateNumber>);"
@dfaStateSwitch.noViableAltException() ::= "dbg.RecognitionException(nvae_d<decisionNumber>s<stateNumber>);"
dfaDecision(decisionNumber,description) ::= <<
try
{
isCyclicDecision = true;
<super.dfaDecision(...)>
}
catch (NoViableAltException nvae)
{
dbg.RecognitionException(nvae);
throw nvae;
}
>>
@cyclicDFA.dbgCtor() ::= <<
public DFA<dfa.decisionNumber>(BaseRecognizer recognizer, IDebugEventListener dbg) : this(recognizer)
{
this.dbg = dbg;
}
>>
@cyclicDFA.debugMember() ::= <<
IDebugEventListener dbg;
>>
@cyclicDFA.errorMethod() ::= <<
public override void Error(NoViableAltException nvae)
{
dbg.RecognitionException(nvae);
}
>>
/** Force predicate validation to trigger an event */
evalPredicate(pred,description) ::= <<
EvalPredicate(<pred>,"<description>")
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/AST.stg 0000644 0001750 0001750 00000032207 11256465232 027007 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()>
<if(!TREE_PARSER)><! tree parser would already have imported !>
using Antlr.Runtime.Tree;<\n>
<endif>
>>
@genericParser.members() ::= <<
<@super.members()>
<parserMembers()>
>>
/** Add an adaptor property that knows how to build trees */
parserMembers() ::= <<
protected ITreeAdaptor adaptor = new CommonTreeAdaptor();<\n>
public ITreeAdaptor TreeAdaptor
{
get { return this.adaptor; }
set {
this.adaptor = value;
<grammar.directDelegates:{g|<g:delegateName()>.TreeAdaptor = this.adaptor;}>
}
}
>>
@returnScope.ruleReturnMembers() ::= <<
private <ASTLabelType> tree;
override public object Tree
{
get { return tree; }
set { tree = (<ASTLabelType>) value; }
}
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> root_0 = null;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<[ruleDescriptor.tokenLabels,ruleDescriptor.wildcardTreeLabels,
ruleDescriptor.wildcardTreeListLabels]:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{RewriteRule<rewriteElementType>Stream stream_<it> = new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>");}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor,"rule <it>");}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = (<ASTLabelType>)adaptor.GetNilNode();<\n>
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.Add(<label>);<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.Add(<label>);<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule.name>.Add(<label>.Tree);
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule>.Add(<label>.Tree);
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
// wildcard labels: <[referencedWildcardLabels,referencedWildcardListLabels]; separator=", ">
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {<\n>
<endif>
<prevRuleRootRef()>.Tree = root_0;
<rewriteCodeLabels()>
root_0 = (<ASTLabelType>)adaptor.GetNilNode();
<alts:rewriteAlt(); separator="else ">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.Tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
input.ReplaceChildren(adaptor.GetParent(retval.Start),
adaptor.GetChildIndex(retval.Start),
adaptor.GetChildIndex(_last),
retval.Tree);
<endif>
<endif>
<! if parser or rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.Tree = root_0;
<endif>
<if(!rewriteMode)>
<prevRuleRootRef()>.Tree = root_0;
<endif>
<if(backtracking)>
}
<endif>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{RewriteRule<rewriteElementType>Stream stream_<it> = new RewriteRule<rewriteElementType>Stream(adaptor, "token <it>", <it>);};
separator="\n"
>
<referencedTokenListLabels
:{RewriteRule<rewriteElementType>Stream stream_<it> = new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>", list_<it>);};
separator="\n"
>
<referencedWildcardLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",<it>);};
separator="\n"
>
<referencedWildcardListLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",list_<it>);};
separator="\n"
>
<referencedRuleLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor, "rule <it>", <it>!=null ? <it>.Tree : null);};
separator="\n"
>
<referencedRuleListLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor, "token <it>", list_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
if ( <referencedElementsDeep:{el | stream_<el>.HasNext()}; separator=" || "> )
{
<alt>
}
<referencedElementsDeep:{el | stream_<el>.Reset();<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
while ( <referencedElements:{el | stream_<el>.HasNext()}; separator=" || "> )
{
<alt>
}
<referencedElements:{el | stream_<el>.Reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if ( !(<referencedElements:{el | stream_<el>.HasNext()}; separator=" || ">) ) {
throw new RewriteEarlyExitException();
}
while ( <referencedElements:{el | stream_<el>.HasNext()}; separator=" || "> )
{
<alt>
}
<referencedElements:{el | stream_<el>.Reset();<\n>}>
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>)
{
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = null;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.GetNilNode();
<root:rewriteElement()>
<children:rewriteElement()>
adaptor.AddChild(root_<enclosingTreeLevel>, root_<treeLevel>);
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
adaptor.AddChild(root_<treeLevel>, <createRewriteNodeFromElement(...)>);<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextNode());<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextNode());<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<createRewriteNodeFromElement(...)>, root_<treeLevel>);<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>);<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.Tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<rule>.NextTree());<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<rule>.NextNode(), root_<treeLevel>);<\n>
>>
rewriteNodeAction(action) ::= <<
adaptor.AddChild(root_<treeLevel>, <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<action>, root_<treeLevel>);<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
rewriteWildcardLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
new <hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
(<ASTLabelType>)adaptor.Create(<tokenType>, <args; separator=", "><if(!args)>"<tokenType>"<endif>)
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.NextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
adaptor.Create(<token>, <args; separator=", ">)
<else>
stream_<token>.NextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/ASTDbg.stg 0000644 0001750 0001750 00000006246 11256465232 027430 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to AST stuff. Dynamic inheritance
* hierarchy is set up as ASTDbg : AST : Dbg : C# by code generator.
*/
group ASTDbg;
parserMembers() ::= <<
protected DebugTreeAdaptor adaptor;
public ITreeAdaptor TreeAdaptor
{
get {
<if(grammar.grammarIsRoot)>
return this.adaptor;
<else>
this.adaptor = (DebugTreeAdaptor)adaptor; // delegator sends dbg adaptor
<endif><\n>
<grammar.directDelegates:{g|<g:delegateName()>.TreeAdaptor = this.adaptor;}>
}
set { this.adaptor = new DebugTreeAdaptor(dbg, value); }
}<\n>
>>
parserCtorBody() ::= <<
<super.parserCtorBody()>
>>
createListenerAndHandshake() ::= <<
DebugEventSocketProxy proxy = new DebugEventSocketProxy(this, port, adaptor);
DebugListener = proxy;
<!
Original line follows, replaced by the next two ifs:
set<inputStreamType>(new Debug<inputStreamType>(input,proxy));
!>
<if(PARSER)>
TokenStream = new DebugTokenStream(input,proxy);<\n>
<endif>
<if(TREE_PARSER)>
TokenStream = new DebugTreeNodeStream(input,proxy);<\n>
<endif>
try {
proxy.Handshake();
} catch (IOException ioe) {
ReportError(ioe);
}
>>
@ctorForRootGrammar.finally() ::= <<
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;
proxy.TreeAdaptor = adap;
>>
@ctorForProfilingRootGrammar.finally() ::=<<
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;
proxy.TreeAdaptor = adap;
>>
@ctorForPredefinedListener.superClassRef() ::= "base(input, dbg)"
@ctorForPredefinedListener.finally() ::=<<
<if(grammar.grammarIsRoot)> <! don't create new adaptor for delegates !>
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;<\n>
<endif>
>>
@rewriteElement.pregen() ::= "dbg.Location(<e.line>,<e.pos>);"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp/CSharp.stg 0000644 0001750 0001750 00000130230 11256465232 027533 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group CSharp implements ANTLRCore;
csharpTypeInitMap ::= [
"int":"0",
"uint":"0",
"long":"0",
"ulong":"0",
"float":"0.0",
"double":"0.0",
"bool":"false",
"byte":"0",
"sbyte":"0",
"short":"0",
"ushort":"0",
"char":"char.MinValue",
default:"null" // anything other than an atomic type
]
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<@debugPreprocessor()>
<actions.(actionScope).header>
<@imports>
using System;
using Antlr.Runtime;
<if(TREE_PARSER)>
using Antlr.Runtime.Tree;
<endif>
using IList = System.Collections.IList;
using ArrayList = System.Collections.ArrayList;
using Stack = Antlr.Runtime.Collections.StackList;
<if(backtracking)>
using IDictionary = System.Collections.IDictionary;
using Hashtable = System.Collections.Hashtable;
<endif>
<@end>
<if(actions.(actionScope).namespace)>
namespace <actions.(actionScope).namespace>
{
<endif>
<docComment>
<recognizer>
<if(actions.(actionScope).namespace)>
}
<endif>
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="IToken",
filterMode, superClass="Lexer") ::= <<
public class <grammar.recognizerName> : <@superClassName><superClass><@end> {
<tokens:{public const int <it.name> = <it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
public <grammar.recognizerName>() <! needed by subclasses !>
{
InitializeCyclicDFAs();
}
public <grammar.recognizerName>(ICharStream input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: this(input, null<grammar.delegators:{g|, <g:delegateName()>}>) {
}
public <grammar.recognizerName>(ICharStream input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: base(input, state) {
InitializeCyclicDFAs(); <! Necessary in C#??? Not removed yet. !>
<if(memoize)>
<if(grammar.grammarIsRoot)>
state.ruleMemo = new Hashtable[<numRules>+1];<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
override public string GrammarFileName
{
get { return "<fileName>";}
}
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<synpreds:{p | <lexerSynpred(p)>}>
<cyclicDFAs:{dfa | protected DFA<dfa.decisionNumber> dfa<dfa.decisionNumber>;}; separator="\n">
private void InitializeCyclicDFAs(<@debugInitializeCyclicDFAs()>)
{
<cyclicDFAs:{dfa | this.dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this<@debugAddition()>);}; separator="\n">
<cyclicDFAs:{dfa | <if(dfa.specialStateSTs)>this.dfa<dfa.decisionNumber>.specialStateTransitionHandler = new DFA.SpecialStateTransitionHandler(DFA<dfa.decisionNumber>_SpecialStateTransition);<endif>}; separator="\n">
}
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
}
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
*
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
override public IToken NextToken()
{
while (true)
{
if ( input.LA(1) == (int)CharStreamConstants.EOF )
{
return Token.EOF_TOKEN;
}
state.token = null;
state.channel = Token.DEFAULT_CHANNEL;
state.tokenStartCharIndex = input.Index();
state.tokenStartCharPositionInLine = input.CharPositionInLine;
state.tokenStartLine = input.Line;
state.text = null;
try
{
int m = input.Mark();
state.backtracking = 1; <! means we won't throw slow exception !>
state.failed = false;
mTokens();
state.backtracking = 0;
<!
mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1.
!>
if ( state.failed )
{
input.Rewind(m);
input.Consume(); <! // advance one char and try again !>
}
else
{
Emit();
return state.token;
}
}
catch (RecognitionException re)
{
// shouldn't happen in backtracking mode, but...
ReportError(re);
Recover(re);
}
}
}
override public void Memoize(IIntStream input, int ruleIndex, int ruleStartIndex)
{
if ( state.backtracking > 1 )
base.Memoize(input, ruleIndex, ruleStartIndex);
}
override public bool AlreadyParsedRule(IIntStream input, int ruleIndex)
{
if ( state.backtracking>1 )
return base.AlreadyParsedRule(input, ruleIndex);
return false;
}
>>
actionGate() ::= "(state.backtracking==0)"
filteringActionGate() ::= "(state.backtracking == 1)"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass, filterMode,
ASTLabelType="object", labelType, members, rewriteElementType) ::= <<
public class <grammar.recognizerName> : <@superClassName><superClass><@end>
{
<if(grammar.grammarIsRoot)>
public static readonly string[] tokenNames = new string[]
{
"\<invalid>",
"\<EOR>",
"\<DOWN>",
"\<UP>",
<tokenNames; separator=", \n">
};<\n>
<endif>
<tokens:{public const int <it.name> = <it.type>;}; separator="\n">
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<@members>
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
public <grammar.recognizerName>(<inputStreamType> input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: this(input, new RecognizerSharedState()<grammar.delegators:{g|, <g:delegateName()>}>) {
}
public <grammar.recognizerName>(<inputStreamType> input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: base(input, state) {
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.indirectDelegates:{g | <g:delegateName()> = <g.delegator:delegateName()>.<g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
<@end>
override public string[] TokenNames {
get { return <grammar.composite.rootGrammar.recognizerName>.tokenNames; }
}
override public string GrammarFileName {
get { return "<fileName>"; }
}
<members>
<rules; separator="\n\n">
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
// Delegated rules
<grammar.delegatedRules:{ruleDescriptor|
public <returnType()> <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) // throws RecognitionException
\{
<if(ruleDescriptor.hasReturnValue)>return <endif><ruleDescriptor.grammar:delegateName()>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">);
\}}; separator="\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:{dfa | protected DFA<dfa.decisionNumber> dfa<dfa.decisionNumber>;}; separator="\n">
private void InitializeCyclicDFAs(<@debugInitializeCyclicDFAs()>)
{
<cyclicDFAs:{dfa | this.dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this<@debugAddition()>);}; separator="\n">
<cyclicDFAs:{dfa | <if(dfa.specialStateSTs)>this.dfa<dfa.decisionNumber>.specialStateTransitionHandler = new DFA.SpecialStateTransitionHandler(DFA<dfa.decisionNumber>_SpecialStateTransition);<endif>}; separator="\n">
}
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
}
>>
parserCtorBody() ::= <<
<@initializeCyclicDFAs>InitializeCyclicDFAs();<@end>
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = new Hashtable[<length(grammar.allImportedRules)>+1];<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType, superClass="Parser", labelType="IToken", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="ITokenStream", rewriteElementType="Token", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="object", superClass="TreeParser", members={<actions.treeparser.members>}, filterMode) ::= <<
<genericParser(inputStreamType="ITreeNodeStream", rewriteElementType="Node", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start "<ruleName>"
public void <ruleName>_fragment(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) {
<ruleLabelDefs()>
<if(trace)>
TraceIn("<ruleName>_fragment", <ruleDescriptor.index>);
try
{
<block>
}
finally
{
TraceOut("<ruleName>_fragment", <ruleDescriptor.index>);
}
<else>
<block>
<endif>
}
// $ANTLR end "<ruleName>"
>>
synpredDecls(name) ::= <<
SynPredPointer <name>;<\n>
>>
synpred(name) ::= <<
public bool <name>()
{
state.backtracking++;
<@start()>
int start = input.Mark();
try
{
<name>_fragment(); // can never throw exception
}
catch (RecognitionException re)
{
Console.Error.WriteLine("impossible: "+re);
}
bool success = !state.failed;
input.Rewind(start);
<@stop()>
state.backtracking--;
state.failed = false;
return success;
}<\n>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( (state.backtracking > 0) && AlreadyParsedRule(input, <ruleDescriptor.index>) )
{
return <ruleReturnValue()>;
}
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>if (state.failed) return <ruleReturnValue()>;<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if ( state.backtracking > 0 ) {state.failed = true; return <ruleReturnValue()>;}<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
// $ANTLR start "<ruleName>"
// <fileName>:<description>
public <returnType()> <ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) // throws RecognitionException [1]
{
<if(trace)>TraceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try
{
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
catch (RecognitionException re)
{
ReportError(re);
Recover(input,re);
<@setErrorReturnValue()>
}<\n>
<endif>
<endif>
<endif>
finally
{
<if(trace)>TraceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
}
<@postamble()>
return <ruleReturnValue()>;
}
// $ANTLR end "<ruleName>"
>>
catch(decl,action) ::= <<
catch (<e.decl>)
{
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnType()> retval = new <returnType()>();
retval.Start = input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.type> <a.name> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
int <ruleDescriptor.name>_StartIndex = input.Index();
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.Push(new <it>_scope());}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.Push(new <it.name>_scope());}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.Pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.Pop();}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels,
ruleDescriptor.wildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{<labelType> <it.label.text> = null;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels,ruleDescriptor.wildcardTreeListLabels]
:{IList list_<it.label.text> = null;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|<ll:ruleLabelDef(label=it)> <ll.label.text> = null;}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleLabels]
:{<labelType> <it.label.text> = null;}; separator="\n"
>
<ruleDescriptor.charLabels:{int <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels]
:{IList list_<it.label.text> = null;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
retval.Stop = input.LT(-1);<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( state.backtracking > 0 )
{
Memoize(input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex);
}
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// $ANTLR start "<ruleName>"
public void m<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) // throws RecognitionException [2]
{
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<if(trace)>TraceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
try
{
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
int _type = <ruleName>;
int _channel = DEFAULT_TOKEN_CHANNEL;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
state.type = _type;
state.channel = _channel;
<(ruleDescriptor.actions.after):execAction()>
<endif>
}
finally
{
<if(trace)>TraceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeCleanUp()>
<memoize()>
}
}
// $ANTLR end "<ruleName>"
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
override public void mTokens() // throws RecognitionException
{
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber> = <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber> = <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int cnt<decisionNumber> = 0;
<decls>
<@preloop()>
do
{
int alt<decisionNumber> = <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
default:
if ( cnt<decisionNumber> >= 1 ) goto loop<decisionNumber>;
<ruleBacktrackFailure()>
EarlyExitException eee<decisionNumber> =
new EarlyExitException(<decisionNumber>, input);
<@earlyExitException()>
throw eee<decisionNumber>;
}
cnt<decisionNumber>++;
} while (true);
loop<decisionNumber>:
; // Stops C# compiler whining that label 'loop<decisionNumber>' has no statements
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
<decls>
<@preloop()>
do
{
int alt<decisionNumber> = <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
default:
goto loop<decisionNumber>;
}
} while (true);
loop<decisionNumber>:
; // Stops C# compiler whining that label 'loop<decisionNumber>' has no statements
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> :
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
// <fileName>:<description>
{
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
}
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label>=(<labelType>)<endif>Match(input,<token>,FOLLOW_<token>_in_<ruleName><elementIndex>); <checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label> == null) list_<label> = new ArrayList();
list_<label>.Add(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
Match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
MatchRange(<a>,<b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= input.LA(1);<\n>
<else>
<label> = (<labelType>)input.LT(1);<\n>
<endif>
<endif>
if ( <s> )
{
input.Consume();
<postmatchCode>
<if(!LEXER)>
state.errorRecovery = false;
<endif>
<if(backtracking)>state.failed = false;<endif>
}
else
{
<ruleBacktrackFailure()>
MismatchedSetException mse = new MismatchedSetException(null,input);
<@mismatchedSetException()>
<if(LEXER)>
Recover(mse);
throw mse;
<else>
throw mse;
<! use following code to make it recover inline; remove throw mse;
RecoverFromMismatchedSet(input,mse,FOLLOW_set_in_<ruleName><elementIndex>);
!>
<endif>
}<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
int <label>Start = CharIndex;
Match(<string>); <checkRuleBacktrackFailure()>
<label> = new CommonToken(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start, CharIndex-1);
<else>
Match(<string>); <checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label> = (<labelType>)input.LT(1);<\n>
<endif>
MatchAny(input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
MatchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
PushFollow(FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)>
<label> = <if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
<else>
<if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
<endif>
state.followingStackPointer--;
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference.
*
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
int <label>Start<elementIndex> = CharIndex;
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> = new CommonToken(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, CharIndex-1);
<else>
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
int <label>Start<elementIndex> = CharIndex;
Match(EOF); <checkRuleBacktrackFailure()>
<labelType> <label> = new CommonToken(input, EOF, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, CharIndex-1);
<else>
Match(EOF); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1) == Token.DOWN )
{
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) )
{
<ruleBacktrackFailure()>
throw new FailedPredicateException(input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
else
{
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae_d<decisionNumber>s<stateNumber> =
new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae_d<decisionNumber>s<stateNumber>;<\n>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber>=<eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else
{
alt<decisionNumber> = <eotPredictsAlt>;
}<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber> = <alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>) <endif>)
{
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;
break;
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae_d<decisionNumber>s<stateNumber> =
new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae_d<decisionNumber>s<stateNumber>;<\n>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber> = <eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
{
<targetState>
}
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = dfa<decisionNumber>.Predict(input);
>>
/* Dump DFA tables.
*/
cyclicDFA(dfa) ::= <<
const string DFA<dfa.decisionNumber>_eotS =
"<dfa.javaCompressedEOT; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_eofS =
"<dfa.javaCompressedEOF; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_minS =
"<dfa.javaCompressedMin; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_maxS =
"<dfa.javaCompressedMax; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_acceptS =
"<dfa.javaCompressedAccept; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_specialS =
"<dfa.javaCompressedSpecial; wrap="\"+\n \"">}>";
static readonly string[] DFA<dfa.decisionNumber>_transitionS = {
<dfa.javaCompressedTransition:{s|"<s; wrap="\"+\n\"">"}; separator=",\n">
};
static readonly short[] DFA<dfa.decisionNumber>_eot = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_eotS);
static readonly short[] DFA<dfa.decisionNumber>_eof = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_eofS);
static readonly char[] DFA<dfa.decisionNumber>_min = DFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_minS);
static readonly char[] DFA<dfa.decisionNumber>_max = DFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_maxS);
static readonly short[] DFA<dfa.decisionNumber>_accept = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_acceptS);
static readonly short[] DFA<dfa.decisionNumber>_special = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_specialS);
static readonly short[][] DFA<dfa.decisionNumber>_transition = DFA.UnpackEncodedStringArray(DFA<dfa.decisionNumber>_transitionS);
protected class DFA<dfa.decisionNumber> : DFA
{
<@debugMember()>
public DFA<dfa.decisionNumber>(BaseRecognizer recognizer)
{
this.recognizer = recognizer;
this.decisionNumber = <dfa.decisionNumber>;
this.eot = DFA<dfa.decisionNumber>_eot;
this.eof = DFA<dfa.decisionNumber>_eof;
this.min = DFA<dfa.decisionNumber>_min;
this.max = DFA<dfa.decisionNumber>_max;
this.accept = DFA<dfa.decisionNumber>_accept;
this.special = DFA<dfa.decisionNumber>_special;
this.transition = DFA<dfa.decisionNumber>_transition;
}
<@dbgCtor()>
override public string Description
{
get { return "<dfa.description>"; }
}
<@errorMethod()>
}<\n>
<if(dfa.specialStateSTs)>
protected internal int DFA<dfa.decisionNumber>_SpecialStateTransition(DFA dfa, int s, IIntStream _input) //throws NoViableAltException
{
<if(LEXER)>
IIntStream input = _input;
<endif>
<if(PARSER)>
ITokenStream input = (ITokenStream)_input;
<endif>
<if(TREE_PARSER)>
ITreeNodeStream input = (ITreeNodeStream)_input;
<endif>
int _s = s;
switch ( s )
{
<dfa.specialStateSTs:{state |
case <i0> : <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
<if(backtracking)>
if (state.backtracking > 0) {state.failed = true; return -1;}<\n>
<endif>
NoViableAltException nvae<dfa.decisionNumber> =
new NoViableAltException(dfa.Description, <dfa.decisionNumber>, _s, input);
dfa.Error(nvae<dfa.decisionNumber>);
throw nvae<dfa.decisionNumber>;
}<\n>
<endif>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
int index<decisionNumber>_<stateNumber> = input.Index();
input.Rewind();<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
<if(semPredState)> <! return input cursor to state before we rewound !>
input.Seek(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s >= 0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>) <endif>) { s = <targetStateNumber>; }<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left> && <right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | || <o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber> == <atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "input.LA(<k>) == <atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber> \>= <lower> && LA<decisionNumber>_<stateNumber> \<= <upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(input.LA(<k>) \>= <lower> && input.LA(<k>) \<= <upper>)"
setTest(ranges) ::= "<ranges; separator=\" || \">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected class <scope.name>_scope
{
<scope.attributes:{protected internal <it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
ruleAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected class <scope.name>_scope
{
<scope.attributes:{protected internal <it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
returnStructName() ::= "<it.name>_return"
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.grammar.recognizerName>.<ruleDescriptor:returnStructName()>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Generate the C# type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.grammar.recognizerName>.<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
<csharpTypeInitMap.(typeName)>
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<ruleLabelType(referencedRule=label.referencedRule)> <label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
public class <ruleDescriptor:returnStructName()> : <if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope
{
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
};
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> = <expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack[<scope>_stack.Count-<negIndex>-1]).<attr.name>
<else>
<if(index)>
((<scope>_scope)<scope>_stack[<index>]).<attr.name>
<else>
((<scope>_scope)<scope>_stack.Peek()).<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack[<scope>_stack.Count-<negIndex>-1]).<attr.name> = <expr>;
<else>
<if(index)>
((<scope>_scope)<scope>_stack[<index>]).<attr.name> = <expr>;
<else>
((<scope>_scope)<scope>_stack.Peek()).<attr.name> = <expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
((<scope> != null) ? <scope>.<attr.name> : <initValue(attr.type)>)
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name> = <expr>;
<else>
<attr.name> = <expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "((<scope> != null) ? <scope>.Text : null)"
tokenLabelPropertyRef_type(scope,attr) ::= "((<scope> != null) ? <scope>.Type : 0)"
tokenLabelPropertyRef_line(scope,attr) ::= "((<scope> != null) ? <scope>.Line : 0)"
tokenLabelPropertyRef_pos(scope,attr) ::= "((<scope> != null) ? <scope>.CharPositionInLine : 0)"
tokenLabelPropertyRef_channel(scope,attr) ::= "((<scope> != null) ? <scope>.Channel : 0)"
tokenLabelPropertyRef_index(scope,attr) ::= "((<scope> != null) ? <scope>.TokenIndex : 0)"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
tokenLabelPropertyRef_int(scope,attr) ::= "(<scope>!=null?int.Parse(<scope>.Text):0)"
ruleLabelPropertyRef_start(scope,attr) ::= "((<scope> != null) ? ((<labelType>)<scope>.Start) : null)"
ruleLabelPropertyRef_stop(scope,attr) ::= "((<scope> != null) ? ((<labelType>)<scope>.Stop) : null)"
ruleLabelPropertyRef_tree(scope,attr) ::= "((<scope> != null) ? ((<ASTLabelType>)<scope>.Tree) : null)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
((<scope> != null) ? input.TokenStream.ToString(
input.TreeAdaptor.GetTokenStartIndex(<scope>.Start),
input.TreeAdaptor.GetTokenStopIndex(<scope>.Start)) : null)
<else>
((<scope> != null) ? input.ToString((IToken)(<scope>.Start),(IToken)(<scope>.Stop)) : null)
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "((<scope> != null) ? <scope>.ST : null)"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "((<scope> != null) ? <scope>.Type : 0)"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "((<scope> != null) ? <scope>.Line : 0)"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "((<scope> != null) ? <scope>.CharPositionInLine : -1)"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "((<scope> != null) ? <scope>.Channel : 0)"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "((<scope> != null) ? <scope>.TokenIndex : 0)"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "((<scope> != null) ? <scope>.Text : null)"
lexerRuleLabelPropertyRef_int(scope,attr) ::= "(<scope>!=null?int.Parse(<scope>.Text):0)"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "((<labelType>)retval.Start)"
rulePropertyRef_stop(scope,attr) ::= "((<labelType>)retval.Stop)"
rulePropertyRef_tree(scope,attr) ::= "((<ASTLabelType>)retval.Tree)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
input.TokenStream.ToString(
input.TreeAdaptor.GetTokenStartIndex(retval.Start),
input.TreeAdaptor.GetTokenStopIndex(retval.Start) )
<else>
input.ToString((IToken)retval.Start,input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.ST"
lexerRulePropertyRef_text(scope,attr) ::= "Text"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "state.tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "state.tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "_channel"
lexerRulePropertyRef_start(scope,attr) ::= "state.tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(CharIndex-1)"
lexerRulePropertyRef_int(scope,attr) ::= "int.Parse(<scope>.Text)"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.Tree = <expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.ST = <expr>;"
/** How to execute an action (only when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<action>
}
<else>
<action>
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
public static readonly BitSet <name> = new BitSet(new ulong[]{<words64:{<it>UL};separator=",">});<\n>
>>
codeFileExtension() ::= ".cs"
true() ::= "true"
false() ::= "false"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Delphi/ 0000755 0001750 0001750 00000000000 11410174107 025650 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Delphi/ASTTreeParser.stg 0000644 0001750 0001750 00000022047 11256465232 031032 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2008 Erik van Bilsen
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
_First[0] := nil;
_Last := nil;<\n>
>>
ruleDeclarationVars() ::= <<
<super.ruleDeclarationVars()>
_First, _Save_Last: array [0..63] of I<ASTLabelType>;
_Last: I<ASTLabelType>;
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if (State.Backtracking = 0) then
begin<endif>
<if(rewriteMode)>
RetVal.Tree := _First[0] as I<ASTLabelType>;
if (Adaptor.GetParent(RetVal.Tree) \<\> nil) and (Adaptor.IsNil(Adaptor.GetParent(RetVal.Tree))) then
RetVal.Tree := Adaptor.GetParent(RetVal.Tree) as I<ASTLabelType>;
<endif>
<if(backtracking)>end;<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
begin
_Save_Last[<treeLevel>] := _Last;
_First[<treeLevel>] := nil;
<if(!rewriteMode)>
Root[<treeLevel>] := Adaptor.GetNilNode as I<ASTLabelType>;<\n>
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>
<if(root.el.rule)>
if (_First[<enclosingTreeLevel>] = nil) then _First[<enclosingTreeLevel>] := <root.el.label>.Tree;
<else>
if (_First[<enclosingTreeLevel>] = nil) then _First[<enclosingTreeLevel>] := <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if (Input.LA(1) = TToken.DOWN) then
begin
Match(Input, TToken.DOWN, nil); <checkRuleBacktrackFailure()>
<children:element()>
Match(Input, TToken.UP, nil); <checkRuleBacktrackFailure()>
end;
<else>
Match(Input, TToken.DOWN, nil); <checkRuleBacktrackFailure()>
<children:element()>
Match(Input, TToken.UP, nil); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
Adaptor.AddChild(Root[<enclosingTreeLevel>], Root[<treeLevel>]);
<endif>
_Last := _Save_Last[<treeLevel>];
end;<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>
if (State.Backtracking = 0) then
begin<\n>
<endif>
<if(hetero)>
<label>_tree := T<hetero>.Create(<label>);
<else>
<label>_tree := Adaptor.DupNode(<label>) as I<ASTLabelType>;
<endif><\n>
Adaptor.AddChild(Root[<treeLevel>], <label>_tree);
<if(backtracking)>
end;
<endif>
<else> <! rewrite mode !>
<if(backtracking)>if (State.Backtracking = 0) then <endif>
if (_First[<treeLevel>] = nil) then _First[<treeLevel>] := <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>
if (State.Backtracking = 0) then
begin
<endif>
<if(hetero)>
<label>_tree := T<hetero>.Create(<label>);
<else>
<label>_tree := Adaptor.DupNode(<label>) as I<ASTLabelType>;
<endif><\n>
Root[<treeLevel>] := Adaptor.BecomeRoot(<label>_tree, Root[<treeLevel>]) as I<ASTLabelType>;
<if(backtracking)>
end;
<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_Llast := Input.LT(1) as I<ASTLabelType>;
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if (State.Backtracking = 0) then begin <endif>
<if(hetero)>
<label>_tree := T<hetero>.Create(<label>);
<else>
<label>_tree := Adaptor.DupNode(<label>) as I<ASTLabelType>;
<endif><\n>
Adaptor.AddChild(Root[<treeLevel>], <label>_tree);
<if(backtracking)>end;<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if (State.Backtracking = 0) then begin <endif>
<if(hetero)>
<label>_tree := T<hetero>.Create(<label>);
<else>
<label>_tree := Adaptor.DupNode(<label>) as I<ASTLabelType>;
<endif><\n>
Root[<treeLevel>] := Adaptor.BecomeRoot(<label>_tree, Root[<treeLevel>]) as I<ASTLabelType>;
<if(backtracking)>end;<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.ruleRef(...)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>
<if(!rewriteMode)>
Adaptor.AddChild(Root[<treeLevel>], <label>.Tree);
<else> <! rewrite mode !>
if (_First[<treeLevel>] = nil) then _First[<treeLevel>] := <label>.Tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>Root[<treeLevel>] := Adaptor.BecomeRoot(<label>.Tree, Root[<treeLevel>]) as I<ASTLabelType>;
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_Last := Input.LT(1) as I<ASTLabelType>;
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
T<hetero>.Create((Locals['Stream_<token>'] as IRewriteRuleElementStream).NextNode)
<else>
(Locals['Stream_<token>'] as IRewriteRuleElementStream).NextNode
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>
if (State.Backtracking = 0) then
begin<\n>
<endif>
RetVal.Tree := Adaptor.RulePostProcessing(Root[0]) as I<ASTLabelType>;
<if(backtracking)>
end;
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Delphi/ASTParser.stg 0000644 0001750 0001750 00000015645 11256465232 030220 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2008 Erik van Bilsen
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
RetVal.Tree := Adaptor.ErrorNode(Input, RetVal.Start as IToken,
Input.LT(-1), RE) as I<ASTLabelType>;
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>
if (State.Backtracking = 0) then
begin<\n>
<endif>
<label>_tree := <createNodeFromToken(...)>;
Adaptor.AddChild(Root[0], <label>_tree);
<if(backtracking)>
end;
<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>
if (State.Backtracking = 0) then
begin
<endif>
<label>_tree := <createNodeFromToken(...)>;
Root[0] := Adaptor.BecomeRoot(<label>_tree, Root[0]) as I<ASTLabelType>;
<if(backtracking)>
end;
<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if (State.Backtracking = 0) then <endif>Adaptor.AddChild(Root[0], <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label> := Input.LT(1) as I<labelType>;<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if (State.Backtracking = 0) then <endif>Root[0] := Adaptor.BecomeRoot(<createNodeFromToken(...)>, Root[0]) as I<ASTLabelType>;})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>Adaptor.AddChild(Root[0], <label>.Tree);
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>Root[0] := Adaptor.BecomeRoot(<label>.Tree, Root[0]) as I<ASTLabelType>;
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>
if (State.Backtracking = 0) then
begin
<endif>
<label>_tree := Adaptor.CreateNode(<label>) as I<ASTLabelType>;
Adaptor.AddChild(Root[0], <label>_tree);
<if(backtracking)>
end;
<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>
if (State.Backtracking = 0) then
begin
<endif>
<label>_tree := Adaptor.CreateNode(<label>) as I<ASTLabelType>;
Root[0] := Adaptor.BecomeRoot(<label>_tree, Root[0]) as I<ASTLabelType>;
<if(backtracking)>
end;
<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
T<hetero>.Create(<label>) <! new MethodNode(IDLabel) !>
<else>
Adaptor.CreateNode(<label>) as I<ASTLabelType>
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(backtracking)>
if (State.Backtracking = 0) then
begin<\n>
<endif>
RetVal.Tree := Adaptor.RulePostProcessing(Root[0]) as I<ASTLabelType>;
<if(!TREE_PARSER)>
Adaptor.SetTokenBoundaries(RetVal.Tree, RetVal.Start as IToken, RetVal.Stop as IToken);
<endif>
<if(backtracking)>
<\n>end;
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Delphi/AST.stg 0000644 0001750 0001750 00000035424 11256465232 027040 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2008 Erik van Bilsen
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()><if(!TREE_PARSER)><! tree parser would already have imported !> Antlr.Runtime.Tree,<\n><endif>
>>
@genericParser.members() ::= <<
<@super.members()>
<parserMembers()>
>>
@genericParser.membersConstructor() ::= <<
<@super.membersConstructor()>
<parserMembersConstructor()>
>>
@genericParser.membersImplementation() ::= <<
<@super.membersImplementation()>
<parserMembersImplementation()>
>>
/** Add an adaptor property that knows how to build trees */
parserMembers() ::= <<
strict protected
FAdaptor: ITreeAdaptor;
procedure SetAdaptor(const Value: ITreeAdaptor);
property Adaptor: ITreeAdaptor read FAdaptor;
public
property TreeAdaptor: ITreeAdaptor read FAdaptor write SetAdaptor;
>>
parserMembersConstructor() ::= <<
FAdaptor := TCommonTreeAdaptor.Create;
>>
parserMembersImplementation() ::= <<
procedure T<grammar.recognizerName>.SetAdaptor(const Value: ITreeAdaptor);
begin
FAdaptor := Value;
<grammar.directDelegates:{g|<g:delegateName()>.TreeAdaptor := FAdaptor;}>
end;
>>
@returnScope.ruleReturnMembers() ::= <<
function T<grammar.recognizerName>.T<ruleDescriptor:returnStructName()>.GetTree: IANTLRInterface;
begin
Result := FTree;
end;
procedure T<grammar.recognizerName>.T<ruleDescriptor:returnStructName()>.SetTree(const Value: IANTLRInterface);
begin
FTree := Value as I<ASTLabelType>;
end;
>>
@returnScopeDeclaration.ruleReturnMembers() ::= <<
strict private
FTree: I<ASTLabelType>;
protected
{ IRuleReturnScope }
function GetTree: IANTLRInterface; override;
procedure SetTree(const Value: IANTLRInterface); override;
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
Root[0] := nil;<\n>
>>
ruleDeclarationVars() ::= <<
<super.ruleDeclarationVars()>
Root: array [0..63] of I<ASTLabelType>;
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<ruleDescriptor.tokenLabels:{<it.label.text>_tree := nil;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<it.label.text>_tree := nil;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites:{Locals['Stream_<it>'] := TRewriteRule<rewriteElementType>Stream.Create(Adaptor,'token <it>');}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites:{Locals['Stream_<it>'] := TRewriteRuleSubtreeStream.Create(Adaptor,'rule <it>');}; separator="\n">
>>
ruleLabelDefVars() ::= <<
<super.ruleLabelDefVars()>
<ruleDescriptor.tokenLabels:{<it.label.text>_tree: I<ASTLabelType>;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<it.label.text>_tree: I<ASTLabelType>;}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
Root[0] := Adaptor.GetNilNode as I<ASTLabelType>;
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if (State.Backtracking = 0) then <endif>(Locals['Stream_<token>'] as IRewriteRuleElementStream).Add(<label>);<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>(Locals['Stream_<token>'] as IRewriteRuleElementStream).Add(<label>);<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
wildcardTrack(label,elementIndex) ::= <<
<super.wildcard(...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>(Locals['Stream_<rule.name>'] as IRewriteRuleElementStream).Add(<label>.Tree);<\n>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if (State.Backtracking = 0) then <endif>(Locals['Stream_<rule>'] as IRewriteRuleElementStream).Add(<label>.Tree);
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
<if(backtracking)>
if (State.Backtracking = 0) then
begin<\n>
<endif>
<prevRuleRootRef()>.Tree := Root[0];
<rewriteCodeLabels()>
Root[0] := Adaptor.GetNilNode as I<ASTLabelType>;
<alts:rewriteAlt(); separator="else ">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.Tree = (<ASTLabelType>)adaptor.rulePostProcessing(root[0]);
input.ReplaceChildren(adaptor.GetParent(retval.Start),
adaptor.GetChildIndex(retval.Start),
adaptor.GetChildIndex(_last),
retval.Tree);
<endif>
<endif>
<! if parser or rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.Tree := Root[0];<\n>
<endif>
<if(!rewriteMode)>
<prevRuleRootRef()>.Tree := Root[0];<\n>
<endif>
<if(backtracking)>
end;
<endif>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{Locals['Stream_<it>'] := TRewriteRule<rewriteElementType>Stream.Create(Adaptor, 'token <it>', <it>);};
separator="\n"
>
<referencedTokenListLabels
:{Locals['Stream_<it>'] := TRewriteRule<rewriteElementType>Stream.Create(Adaptor,'token <it>', list_<it>);};
separator="\n"
>
<referencedRuleLabels:{
if Assigned(<it>) then
Locals['Stream_<it>'] := TRewriteRuleSubtreeStream.Create(Adaptor, 'token <it>', <it>.Tree)
else
Locals['Stream_<it>'] := TRewriteRuleSubtreeStream.Create(Adaptor, 'token <it>', nil);}; separator="\n">
<referencedRuleListLabels
:{Locals['Stream_<it>'] := TRewriteRuleSubtreeStream.Create(Adaptor, 'token <it>', list_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
(* <fileName>:<description> *)
if (<referencedElementsDeep:{el | (Locals['Stream_<el>'] as IRewriteRuleElementStream).HasNext}; separator=" or ">) then
begin
<alt>
end;
<referencedElementsDeep:{el | (Locals['Stream_<el>'] as IRewriteRuleElementStream).Reset;<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
(* <fileName>:<description> *)
while (<referencedElements:{el | (Locals['Stream_<el>'] as IRewriteRuleElementStream).HasNext}; separator=" or ">) do
begin
<alt>
end;
<referencedElements:{el | (Locals['Stream_<el>'] as IRewriteRuleElementStream).Reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if (not (<referencedElements:{el | (Locals['Stream_<el>'] as IRewriteRuleElementStream).HasNext}; separator=" or ">)) then
raise ERewriteEarlyExitException.Create('');
while (<referencedElements:{el | (Locals['Stream_<el>'] as IRewriteRuleElementStream).HasNext}; separator=" or ">) do
begin
<alt>
end;
<referencedElements:{el | (Locals['Stream_<el>'] as IRewriteRuleElementStream).Reset();<\n>}>
>>
rewriteAlt(a) ::= <<
(* <a.description> *)
<if(a.pred)>
if (<a.pred>) then
begin
<a.alt>
end<\n>
<else>
begin
<a.alt>
end;<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "Root[0] = null;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
(* <fileName>:<description> *)
begin
Root[<treeLevel>] := Adaptor.GetNilNode as I<ASTLabelType>;
<root:rewriteElement()>
<children:rewriteElement()>
Adaptor.AddChild(Root[<enclosingTreeLevel>], Root[<treeLevel>]);
end;<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
Adaptor.AddChild(Root[<treeLevel>], <createRewriteNodeFromElement(...)>);<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
Adaptor.AddChild(Root[<treeLevel>], (Locals['Stream_<label>'] as IRewriteRuleElementStream).NextNode());<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
Adaptor.AddChild(Root[<treeLevel>], (Locals['Stream_<label>'] as IRewriteRuleElementStream).NextNode);<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
Root[<treeLevel>] := Adaptor.BecomeRoot((Locals['Stream_<label>'] as IRewriteRuleElementStream).NextNode(), Root[<treeLevel>]) as I<ASTLabelType>;<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
Root[<treeLevel>] := Adaptor.BecomeRoot(<createRewriteNodeFromElement(...)>, Root[<treeLevel>]) as I<ASTLabelType>;<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
Adaptor.AddChild(Root[<treeLevel>], <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
Root[<treeLevel>] := Adaptor.BecomeRoot(<createImaginaryNode(tokenType=token, ...)>, Root[<treeLevel>]) as I<ASTLabelType>;<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
Root[0] = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of Root[0] right
* before I set it during rewrites. The assign will be to retval.Tree.
*/
prevRuleRootRef() ::= "RetVal"
rewriteRuleRef(rule) ::= <<
Adaptor.AddChild(Root[<treeLevel>], (Locals['Stream_<rule>'] as IRewriteRuleElementStream).NextTree());<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
Root[<treeLevel>] := Adaptor.BecomeRoot((Locals['Stream_<rule>'] as IRewriteRuleElementStream).NextNode, Root[<treeLevel>]) as I<ASTLabelType>;<\n>
>>
rewriteNodeAction(action) ::= <<
Adaptor.AddChild(Root[<treeLevel>], <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
Root[<treeLevel>] := Adaptor.BecomeRoot(<action>, Root[<treeLevel>]) as I<ASTLabelType>;<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
Adaptor.AddChild(Root[<treeLevel>], (Locals['Stream_<label>'] as IRewriteRuleElementStream).NextTree());<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
Adaptor.AddChild(Root[<treeLevel>], (Locals['Stream_<label>'] as IRewriteRuleElementStream).NextTree());<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
Root[<treeLevel>] := Adaptor.BecomeRoot((Locals['Stream_<label>'] as IRewriteRuleElementStream).NextNode, Root[<treeLevel>]) as I<ASTLabelType>;<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
Root[<treeLevel>] := Adaptor.BecomeRoot((Locals['Stream_<label>'] as IRewriteRuleElementStream).NextNode, Root[<treeLevel>]) as I<ASTLabelType>;<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
T<hetero>.Create(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
Adaptor.CreateNode(<tokenType>, <args; separator=", "><if(!args)>'<tokenType>'<endif>) as I<ASTLabelType>
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
T<hetero>.Create((Locals['Stream_<token>'] as IRewriteRuleElementStream).NextToken<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
Adaptor.Create(<token>, <args; separator=", ">)
<else>
(Locals['Stream_<token>'] as IRewriteRuleElementStream).NextNode
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Delphi/Delphi.stg 0000644 0001750 0001750 00000155336 11256465232 027623 0 ustar twerner twerner /* [The "BSD licence"]
Copyright (c) 2008 Erik van Bilsen
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group Delphi implements ANTLRCore;
csharpTypeInitMap ::= [
"int":"0",
"uint":"0",
"long":"0",
"ulong":"0",
"float":"0.0",
"double":"0.0",
"bool":"False",
"byte":"0",
"sbyte":"0",
"short":"0",
"ushort":"0",
"char":"#0",
"string":"''",
"String":"''",
default:"nil" // anything other than an atomic type
]
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
* LEXER (Boolean): should we generate lexer code?
* PARSER (Boolean): should we generate parser code?
* TREE_PARSER (Boolean): should we generate tree parser code?
* actionScope (String): 'lexer', 'parser', 'tree_parser' or custom scope
* actions (HashMap):
* docComment (String): document comment
* recognizer (Object): recognizer class generator
* name (String): name of grammar
* tokens (HashMap<name: String, type: Integer>):
* tokenNames:
* rules:
* cyclicDFAs:
* bitsets:
* buildTemplate (Boolean): should we generate a string template?
* buildAST (Boolean): should we generate an AST?
* rewriteMode (Boolean): are we rewriteing nodes?
* profile (Boolean):
* backtracking (Boolean): backtracking mode?
* synpreds (): syntactic predicates
* memoize (Boolean): should we memoize?
* numRules (Integer): number of rules
* fileName (String): fully qualified name of original .g file
* ANTLRVersion (String): ANTLR version in Major.Minor.Build format
* generatedTimestamp (String): date/time when the file is generated
* trace (Boolean): should we trace input/output?
* scopes:
* superClass (String): name of base class, or empty string
* literals:
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
unit <name>;
{$HINTS OFF}
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<actions.(actionScope).header>
interface
<@imports>
uses<\n>
<@end>
<actions.(actionScope).usesInterface>
<if(TREE_PARSER)>
Antlr.Runtime.Tree,<\n>
<endif>
Antlr.Runtime,
Antlr.Runtime.Collections,
Antlr.Runtime.Tools;
<docComment>
<recognizer>
>>
/** Generates source code for the lexer class
* grammar (Grammar object)
*/
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="Token",
filterMode, superClass="Lexer") ::= <<
type
I<grammar.recognizerName> = interface(I<@superClassName><superClass><@end>)
end;
T<grammar.recognizerName> = class(T<@superClassName><superClass><@end>, I<grammar.recognizerName>)
strict private
FCnt: array [0..<grammar.numberOfDecisions>] of Byte;
FLA: array [0..<grammar.numberOfDecisions>, 0..255] of Integer;
FException: ERecognitionException;
procedure InitializeCyclicDFAs;
<cyclicDFAs:cyclicDFADeclaration()>
public
const
<tokens:{<it.name> = <it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
strict private
<actions.(actionScope).memberDeclarations>
public
// delegates
<grammar.delegates: {g|<g:delegateName()>: I<superClass>; {<g.recognizerName>}}; separator="\n">
public
// delegators
<grammar.delegators: {g|<g:delegateName()>: Pointer; {<g.recognizerName>}}; separator="\n">
<last(grammar.delegators):{g|gParent: Pointer; {<g.recognizerName>}}>
protected
{ IBaseRecognizer }
function GetGrammarFileName: String; override;
<if(filterMode)>
function AlreadyParsedRule(const Input: IIntStream;
const RuleIndex: Integer): Boolean; override;
procedure Memoize(const Input: IIntStream; const RuleIndex,
RuleStartIndex: Integer); override;
protected
{ ILexer }
function NextToken: IToken; override;<\n>
<endif>
protected
{ ILexer }
procedure DoTokens; override;
public
constructor Create; overload;
constructor Create(const AInput: ICharStream<grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>); overload;
constructor Create(const AInput: ICharStream; const AState: IRecognizerSharedState<grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>); overload;
<rules: {r | <if(!r.ruleDescriptor.isSynPred)><lexerRuleDeclaration(r)><endif>}>
<synpreds:{p | <lexerSynpredDeclaration(p)>}; separator="\n">
end;
implementation
uses
<grammar.delegates: {g|<g.recognizerName>,}; separator="\n">
<grammar.delegators: {g|<g.recognizerName>,}; separator="\n">
<actions.(actionScope).usesImplementation>
SysUtils,
StrUtils,
Math;
{ T<grammar.recognizerName> }
constructor T<grammar.recognizerName>.Create;
begin
InitializeCyclicDFAs;
end;
constructor T<grammar.recognizerName>.Create(const AInput: ICharStream<grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>);
begin
Create(AInput, nil<grammar.delegators:{g|, A<g:delegateName()>}>);
end;
constructor T<grammar.recognizerName>.Create(const AInput: ICharStream; const AState: IRecognizerSharedState<grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>);
begin
inherited Create(AInput, AState);
InitializeCyclicDFAs; { TODO: Necessary in Delphi??? Not removed yet. }
<if(memoize)>
<if(grammar.grammarIsRoot)>
State.RuleMemoCount := <numRules>+1;<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.directDelegates:
{g|<g:delegateName()> := T<g.recognizerName>.Create(AInput, State<trunc(g.delegators):{p|, <p:delegateName()>}>, Self);}; separator="\n">
<grammar.delegators:
{g|<g:delegateName()> := Pointer(A<g:delegateName()>);}; separator="\n">
<last(grammar.delegators):{g|gParent := Pointer(A<g:delegateName()>);}>
<actions.(actionScope).memberInitializations>
end;
<actions.(actionScope).memberImplementations>
function T<grammar.recognizerName>.GetGrammarFileName: String;
begin
Result := '<fileName>';
end;
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<synpreds:{p | <lexerSynpred(p)>}>
procedure T<grammar.recognizerName>.InitializeCyclicDFAs;
begin
<cyclicDFAs:{dfa | FDFA<dfa.decisionNumber> := TDFA<dfa.decisionNumber>.Create(Self<@debugAddition()>);}; separator="\n">
<cyclicDFAs:{dfa | <if(dfa.specialStateSTs)>FDFA<dfa.decisionNumber>.SpecialStateTransitionHandler := DFA<dfa.decisionNumber>_SpecialStateTransition;<endif>}; separator="\n">
end;
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
end.>>
lexerRuleDeclaration(rule) ::= <<
procedure m<rule.ruleName>(<rule.ruleDescriptor.parameterScope:parameterScope(scope=rule)>);<\n>
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
*
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
function T<grammar.recognizerName>.NextToken: IToken;
var
M: Integer;
begin
while (True) do
begin
if (Input.LA(1) = Integer(cscEOF)) then
Exit(TToken.EOF_TOKEN);
State.Token := nil;
State.Channel := TToken.DEFAULT_CHANNEL;
State.TokenStartCharIndex := Input.Index;
State.TokenStartCharPositionInLine := Input.CharPositionInLine;
State.TokenStartLine := Input.Line;
State.Text := '';
try
M := Input.Mark();
State.Backtracking := 1; <! means we won't throw slow exception !>
State.Failed := False;
mTokens();
State.Backtracking := 0;
<!
mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1.
!>
if (State.Failed) then
begin
Input.Rewind(M);
Input.Consume; <! // advance one char and try again !>
end
else
begin
Emit;
Exit(State.Token);
end;
except
on RE: ERecognitionException do
begin
// shouldn't happen in backtracking mode, but...
ReportError(RE);
Recover(RE);
end;
end;
end;
end;
function T<grammar.recognizerName>.AlreadyParsedRule(const Input: IIntStream;
const RuleIndex: Integer): Boolean;
begin
if (State.Backtracking > 1) then
Result := inherited AlreadyParsedRule(Input, RuleIndex)
else
Result := False;
end;
procedure T<grammar.recognizerName>.Memoize(const Input: IIntStream; const RuleIndex,
RuleStartIndex: Integer);
begin
if (State.Backtracking > 1) then
inherited Memoize(Input, RuleIndex, RuleStartIndex);
end;
>>
filteringActionGate() ::= "(State.Backtracking = 1)"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass, filterMode,
ASTLabelType="ANTLRInterface", labelType, members, rewriteElementType) ::= <<
type
<rules: {r | <genericParserRuleReturnType(rule=r, ruleDescriptor=r.ruleDescriptor)>}>
I<grammar.recognizerName> = interface(I<@superClassName><superClass><@end>)
<rules: {r | <genericParserRuleInterface(rule=r, ruleDescriptor=r.ruleDescriptor)>}>
end;
T<grammar.recognizerName> = class(T<@superClassName><superClass><@end>, I<grammar.recognizerName>)
<if(grammar.grammarIsRoot)>
public
const
TOKEN_NAMES: array [0..<length(tokenNames)>+3] of String = (
'\<invalid>',
'\<EOR>',
'\<DOWN>',
'\<UP>',
<tokenNames; separator=",\n">);<\n>
<endif>
public
const
<tokens:{<it.name> = <it.type>;}; separator="\n">
public
// delegates
<grammar.delegates: {g|<g:delegateName()>: I<superClass>; {<g.recognizerName>}}; separator="\n">
public
// delegators
<grammar.delegators: {g|<g:delegateName()>: Pointer; {<g.recognizerName>}}; separator="\n">
<last(grammar.delegators):{g|gParent: Pointer; {<g.recognizerName>}}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeDeclaration(scope=it)><endif>}>
<@members>
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
public
constructor Create(const AInput: <inputStreamType><grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>); overload;
constructor Create(const AInput: <inputStreamType>; const AState: IRecognizerSharedState<grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>); overload;
<@end>
protected
{ IBaseRecognizer }
function GetTokenNames: TStringArray; override;
function GetGrammarFileName: String; override;
strict private
<actions.(actionScope).memberDeclarations>
<rules: {r | <genericParserRuleDeclaration(rule=r, ruleDescriptor=r.ruleDescriptor)>}>
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
// Delegated rules
<grammar.delegatedRules:{ruleDescriptor| <delegatedRuleDeclaration(ruleDescriptor)>}>
<synpreds:{p | <synpredDeclaration(p)>}; separator="\n">
<cyclicDFAs:cyclicDFADeclaration()>
strict private
FException: ERecognitionException;
FLA: array [0..<grammar.numberOfDecisions>, 0..255] of Integer;
FCnt: array [0..<grammar.numberOfDecisions>] of Byte;
procedure InitializeCyclicDFAs;
<if(bitsets)>
public
class var
<bitsets:bitsetDecl(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>})>
public
class procedure InitializeBitsets; static;<\n>
<endif>
end;
implementation
uses
<grammar.delegates: {g|<g.recognizerName>,}; separator="\n">
<grammar.delegators: {g|<g.recognizerName>,}; separator="\n">
<actions.(actionScope).usesImplementation>
SysUtils,
StrUtils,
Math;
{ T<grammar.recognizerName> }
constructor T<grammar.recognizerName>.Create(const AInput: <inputStreamType><grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>);
begin
Create(AInput, TRecognizerSharedState.Create<grammar.delegators:{g|, A<g:delegateName()>}>);
end;
constructor T<grammar.recognizerName>.Create(const AInput: <inputStreamType>;
const AState: IRecognizerSharedState<grammar.delegators:{g|; const A<g:delegateName()>: IBaseRecognizer{<g.recognizerName>}}>);
begin
inherited Create(AInput, AState);
<@membersConstructor>
<@end>
<parserCtorBody()>
<grammar.directDelegates:{g|<g:delegateName()> := T<g.recognizerName>.Create(Input, State<trunc(g.delegators):{p|, <p:delegateName()>}>, Self);}; separator="\n">
<grammar.indirectDelegates:{g | <g:delegateName()> := <g.delegator:delegateName()>.<g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent := Pointer(A<g:delegateName()>);}>
<rules: {r | <ruleAttributeScopeInit(scope=r.ruleDescriptor.ruleScope)>}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.(actionScope).memberInitializations>
end;
<actions.(actionScope).memberImplementations>
<grammar.delegatedRules:{ruleDescriptor| <delegatedRuleImplementation(ruleDescriptor)>}; separator="\n">
procedure T<grammar.recognizerName>.InitializeCyclicDFAs;
begin
<cyclicDFAs:{dfa | FDFA<dfa.decisionNumber> := TDFA<dfa.decisionNumber>.Create(Self);}; separator="\n">
<cyclicDFAs:{dfa | <if(dfa.specialStateSTs)>FDFA<dfa.decisionNumber>.SpecialStateTransitionHandler := DFA<dfa.decisionNumber>_SpecialStateTransition;<endif>}; separator="\n">
end;
<if(bitsets)>
class procedure T<grammar.recognizerName>.InitializeBitsets;
begin
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>}, words64=it.bits)>
end;
<endif>
<@membersImplementation>
<@end>
function T<grammar.recognizerName>.GetTokenNames: TStringArray;
var
I: Integer;
begin
SetLength(Result,Length(T<grammar.composite.rootGrammar.recognizerName>.TOKEN_NAMES));
for I := 0 to Length(T<grammar.composite.rootGrammar.recognizerName>.TOKEN_NAMES) - 1 do
Result[I] := T<grammar.composite.rootGrammar.recognizerName>.TOKEN_NAMES[I];
end;
function T<grammar.recognizerName>.GetGrammarFileName: String;
begin
Result := '<fileName>';
end;
<rules; separator="\n\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
<if(bitsets)>
initialization
T<grammar.recognizerName>.InitializeBitsets;<\n>
<endif>
end.>>
delegatedRuleDeclaration(ruleDescriptor) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
function <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>): I<returnType()>;<\n>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
function <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>): <returnType()>;<\n>
<else>
procedure <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>);<\n>
<endif>
<endif>
>>
delegatedRuleImplementation(ruleDescriptor) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
function T<grammar.recognizerName>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>): I<returnType()>;<\n>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
function T<grammar.recognizerName>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>): <returnType()>;<\n>
<else>
procedure T<grammar.recognizerName>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>);<\n>
<endif>
<endif>
begin
<if(ruleDescriptor.hasReturnValue)>Result :=<endif> T<ruleDescriptor.grammar.recognizerName>(<ruleDescriptor.grammar:delegateName()>.Implementor).<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">);
end;
>>
parserCtorBody() ::= <<
InitializeCyclicDFAs;
<if(memoize)>
<if(grammar.grammarIsRoot)>
State.RuleMemoCount := <length(grammar.allImportedRules)>+1;<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.delegators: {g|<g:delegateName()> := Pointer(A<g:delegateName()>);}; separator="\n">
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType, superClass="Parser", labelType="Token", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="ITokenStream", rewriteElementType="Token", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="object", superClass="TreeParser", members={<actions.treeparser.members>}, filterMode) ::= <<
<genericParser(inputStreamType="ITreeNodeStream", rewriteElementType="Node", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start "<ruleName>"
procedure T<grammar.recognizerName>.<ruleName>_fragment(<ruleDescriptor.parameterScope:parameterScope(scope=it)>);
var
Alt: array [0..<grammar.numberOfDecisions>] of Integer;
<ruleLabelDefVars()>
begin
<ruleLabelDefs()>
<if(trace)>
TraceIn('<ruleName>_fragment', <ruleDescriptor.index>);
try
<block>
finally
TraceOut('<ruleName>_fragment', <ruleDescriptor.index>);
end;
<else>
<block>
<endif>
end;
// $ANTLR end "<ruleName>"
>>
synpredDecls(name) ::= <<
SynPredPointer <name>;<\n>
>>
synpred(name) ::= <<
function T<grammar.recognizerName>.<name>: Boolean;
var
Start: Integer;
Success: Boolean;
begin
State.Backtracking := State.Backtracking + 1;
<@start()>
Start := Input.Mark;
try
<name>_fragment(); // can never throw exception
except
on RE: ERecognitionException do
WriteLn('Impossible: ' + RE.ToString);
end;
Success := not State.Failed;
Input.Rewind(Start);
<@stop()>
State.Backtracking := State.Backtracking - 1;
State.Failed := False;
Result := Success;
end;<\n>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
lexerSynpredDeclaration(name) ::= <<
function <name>: Boolean;
procedure <name>_fragment;
>>
synpredDeclaration(name) ::= <<
function <name>: Boolean;
procedure <name>_fragment;
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ((State.Backtracking > 0) and AlreadyParsedRule(Input, <ruleDescriptor.index>)) then
Exit(<ruleReturnValue()>);
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)><\n>if (State.Failed) then Exit(<ruleReturnValue()>);<\n><endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if (State.Backtracking > 0) then
begin
State.Failed := True;
Exit(<ruleReturnValue()>);
end;<endif>
>>
genericParserRuleDeclaration(rule, ruleDescriptor) ::= <<
<if(ruleDescriptor.isSynPred)>
<else>
<ruleAttributeScopeDeclaration(scope=ruleDescriptor.ruleScope)>
<returnScopeDeclaration(scope=ruleDescriptor.returnScope)>
public
<if(ruleDescriptor.hasMultipleReturnValues)>
function <rule.ruleName>: I<returnType()>;<\n>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
function <rule.ruleName>: <returnType()>;<\n>
<else>
procedure <rule.ruleName>;<\n>
<endif>
<endif>
<endif>
>>
genericParserRuleInterface(rule, ruleDescriptor) ::= <<
<if(ruleDescriptor.isSynPred)>
<else>
<if(ruleDescriptor.hasMultipleReturnValues)>
function <rule.ruleName>: I<returnType()>;<\n>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
function <rule.ruleName>: <returnType()>;<\n>
<else>
procedure <rule.ruleName>;<\n>
<endif>
<endif>
<endif>
>>
genericParserRuleReturnType(rule, ruleDescriptor) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(ruleDescriptor.isSynPred)>
<else>
I<returnType()> = interface(I<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope)
end;<\n>
<endif>
<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
// $ANTLR start "<ruleName>"
(* <fileName>:<description> *)
<if(ruleDescriptor.hasMultipleReturnValues)>
function T<grammar.recognizerName>.<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>): I<returnType()>;
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
function T<grammar.recognizerName>.<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>): <returnType()>;
<else>
procedure T<grammar.recognizerName>.<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>);
<endif>
<endif>
var
<ruleDescriptor.actions.vars>
Locals: TLocalStorage;
<if(ruleDescriptor.hasMultipleReturnValues)>
RetVal: I<returnType()>;<\n>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
RetVal: <returnType()>;<\n>
<else>
<endif>
<endif>
Alt: array [0..<grammar.numberOfDecisions>] of Integer;
<ruleDeclarationVars()>
<ruleLabelDefVars()>
begin
Locals.Initialize;
try
<if(trace)>TraceIn('<ruleName>', <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try
try
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
except
on RE: ERecognitionException do
begin
ReportError(RE);
Recover(Input,RE);
<@setErrorReturnValue()>
end;<\n>
<endif>
<endif>
<endif>
end;
finally
<if(trace)>TraceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
end;
<@postamble()>
finally
Locals.Finalize;
end;
Exit(<ruleReturnValue()>);
end;
// $ANTLR end "<ruleName>"
>>
catch(decl,action) ::= <<
catch (<e.decl>)
{
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
RetVal := T<returnType()>.Create;
RetVal.Start := Input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.name> := <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
<ruleDescriptor.name>_StartIndex := Input.Index();
<endif>
>>
ruleDeclarationVars() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.name>: <a.type>;
}>
<endif>
<if(memoize)>
<ruleDescriptor.name>_StartIndex: Integer;
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{<it>Stack.Push(T<it>Scope.Create);}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>Stack.Push(T<it.name>Scope.Create);}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>Stack.Pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>Stack.Pop;}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]:{<it.label.text> := nil;}; separator="\n">
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]:{list_<it.label.text> := nil;}; separator="\n">
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|<ll.label.text> := nil;}; separator="\n">
>>
ruleLabelDefVars() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]:{<it.label.text>: I<labelType>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]:{list_<it.label.text>: IList\<IANTLRInterface\>;}; separator="\n">
<ruleDescriptor.ruleLabels:ruleLabelDefVar(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|<ll.label.text>: <ruleLabelType(referencedRule=ll.referencedRule)>;}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<it.label.text> := nil;}; separator="\n"
>
<ruleDescriptor.charLabels:{int <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleListLabels]
:{List_<it.label.text> := nil;}; separator="\n"
>
>>
lexerRuleLabelDefDeclarations() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<it.label.text>: I<labelType>;}; separator="\n"
>
<ruleDescriptor.charLabels:{int <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleListLabels]
:{List_<it.label.text>: IList;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
RetVal
<endif>
<else>
<! nil !>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
RetVal.Stop := Input.LT(-1);
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if (State.Backtracking > 0) then
Memoize(Input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex);
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// $ANTLR start "<ruleName>"
<ruleDescriptor.parameterScope>
procedure T<grammar.recognizerName>.m<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>);
var
<ruleDescriptor.actions.vars>
Locals: TLocalStorage;
TokenType, Channel: Integer;
Alt: array [0..<grammar.numberOfDecisions>] of Integer;
<lexerRuleLabelDefDeclarations()>
begin
Locals.Initialize;
try
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<if(trace)>TraceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
try
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
TokenType := <ruleName>;
Channel := DEFAULT_TOKEN_CHANNEL;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
State.TokenType := TokenType;
State.Channel := Channel;
<(ruleDescriptor.actions.after):execAction()>
<endif>
finally
<if(trace)>TraceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeCleanUp()>
<memoize()>
end;
finally
Locals.Finalize;
end;
end;
// $ANTLR end "<ruleName>"
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
procedure T<grammar.recognizerName>.mTokens;
var
Alt: array [0..<grammar.numberOfDecisions>] of Integer;
begin
<block>
end;
procedure T<grammar.recognizerName>.DoTokens;
begin
mTokens;
end;
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
(* <fileName>:<description> *)
Alt[<decisionNumber>] := <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
case Alt[<decisionNumber>] of
<alts:altSwitchCase()>
end;
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
(* <fileName>:<description> *)
Alt[<decisionNumber>] := <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
case Alt[<decisionNumber>] of
<alts:altSwitchCase()>
end;
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
(* <fileName>:<description> *)
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
(* <fileName>:<description> *)
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
(* <fileName>:<description> *)
FCnt[<decisionNumber>] := 0;
<decls>
<@preloop()>
while (True) do
begin
Alt[<decisionNumber>] := <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
case Alt[<decisionNumber>] of
<alts:altSwitchCase()>
else
begin
if (FCnt[<decisionNumber>] >= 1) then
Break;
<ruleBacktrackFailure()>
raise EEarlyExitException.Create(<decisionNumber>, Input);
<@earlyExitException()>
end;
end;
Inc(FCnt[<decisionNumber>]);
end;
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
(* <fileName>:<description> *)
<decls>
<@preloop()>
while (True) do
begin
Alt[<decisionNumber>] := <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
case Alt[<decisionNumber>] of
<alts:altSwitchCase()>
else
Break;
end;
end;
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
<i>:
<@prealt()>
<it><\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
(* <fileName>:<description> *)
begin
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
end;
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label> := <endif>Match(Input, <token>, FOLLOW_<token>_in_<ruleName><elementIndex>)<if(label)> as I<labelType><endif>;<\n><checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label> = nil) then list_<label> := TList\<IANTLRInterface\>.Create;
list_<label>.Add(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> := Input.LA(1);<\n>
<endif>
Match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> := Input.LA(1);<\n>
<endif>
MatchRange(<a>, <b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label> := Input.LA(1);<\n>
<else>
<label> := Input.LT(1) as I<labelType>;<\n>
<endif>
<endif>
if (<s>) then
begin
Input.Consume;
<postmatchCode>
<if(!LEXER)>
State.ErrorRecovery := False;<endif>
<if(backtracking)>State.Failed := False;<endif>
end
else
begin
<ruleBacktrackFailure()>
FException := EMismatchedSetException.Create(nil, Input);
<@mismatchedSetException()>
<if(LEXER)>
Recover(FException);
raise FException;<\n>
<else>
raise FException;
<! use following code to make it recover inline; remove throw mse;
RecoverFromMismatchedSet(input,mse,FOLLOW_set_in_<ruleName><elementIndex>);
!>
<endif>
end;<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
Locals.AsInteger['<label>Start'] := CharIndex;
Match(<string>); <checkRuleBacktrackFailure()>
<label> := TCommonToken.Create(Input, TToken.INVALID_TOKEN_TYPE, TToken.DEFAULT_CHANNEL, Locals.AsInteger['<label>Start'], CharIndex-1);
<else>
Match(<string>); <checkRuleBacktrackFailure()>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label> := Input.LT(1) as I<labelType>;<\n>
<endif>
MatchAny(input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> := Input.LA(1);<\n>
<endif>
MatchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
PushFollow(FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)>
<label> := <if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
<else>
<if(scope)>T<scope.recognizerName>(IANTLRObject(<scope:delegateName()>).Implementor).<endif><rule.name>(<args; separator=", ">);<\n>
<endif>
State.FollowingStackPointer := State.FollowingStackPointer - 1;
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference.
*
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
Locals.AsInteger['<label>Start<elementIndex>'] := CharIndex;
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> := TCommonToken.Create(Input, TToken.INVALID_TOKEN_TYPE, TToken.DEFAULT_CHANNEL,
Locals.AsInteger['<label>Start<elementIndex>'], CharIndex - 1);
<else>
<if(scope)>(<scope:delegateName()>.Implementor as T<scope.recognizerName>).<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
Locals.AsInteger['<label>Start<elementIndex>'] := CharIndex;
Match(EOF); <checkRuleBacktrackFailure()>
Locals['<label>'] := TCommonToken.Create(Input, EOF, TToken.DEFAULT_CHANNEL, Locals.AsInteger['<label>Start<elementIndex>'], CharIndex-1);
<else>
Match(EOF); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if (Input.LA(1) = TToken.DOWN) then
begin
Match(Input, TToken.DOWN, nil); <checkRuleBacktrackFailure()>
<children:element()>
Match(Input, TToken.UP, nil); <checkRuleBacktrackFailure()>
end;
<else>
Match(Input, TToken.DOWN, nil); <checkRuleBacktrackFailure()>
<children:element()>
Match(Input, TToken.UP, nil);<\n><checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if (not (<evalPredicate(...)>)) then
begin
<ruleBacktrackFailure()>
raise EFailedPredicateException.Create(Input, '<ruleName>', '<description>');
end;<\n>
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
FLA[<decisionNumber>,<stateNumber>] := Input.LA(<k>);<\n>
<edges; separator="\nelse ">
else
begin
<if(eotPredictsAlt)>
Alt[<decisionNumber>] := <eotPredictsAlt>;<\n>
<else>
<ruleBacktrackFailure()>
raise ENoViableAltException.Create('<description>', <decisionNumber>, <stateNumber>, Input);<\n>
<endif>
end;
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
FLA[<decisionNumber>,<stateNumber>] := Input.LA(<k>);<\n>
<edges; separator="\nelse ">;
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
FLA[<decisionNumber>,<stateNumber>] := Input.LA(<k>);
<edges; separator="\nelse ">;<\n>
<if(eotPredictsAlt)>
<if(!edges)>
Alt[<decisionNumber>] := <eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else
begin
Alt[<decisionNumber>] := <eotPredictsAlt>;
end;<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "Alt[<decisionNumber>] := <alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ((<labelExpr>)<if(predicates)> and (<predicates>)<endif>) then
begin
<targetState>
end <! no ; here !>
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
case Input.LA(<k>) of
<edges; separator="\n">
else
begin
<if(eotPredictsAlt)>
Alt[<decisionNumber>] := <eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
<@noViableAltException()>
raise ENoViableAltException.Create('<description>', <decisionNumber>, <stateNumber>, Input);<\n>
<endif>
end;
end;<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
case Input.LA(<k>) of
<edges; separator="\n">
end;<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
case Input.LA(<k>) of
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
else
Alt[<decisionNumber>] := <eotPredictsAlt>;<\n>
<endif>
end;<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{<it>}; separator=",\n">:
begin
<targetState>
end;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
Alt[<decisionNumber>] := FDFA<decisionNumber>.Predict(Input);
>>
/* Dump DFA tables.
*/
cyclicDFADeclaration(dfa) ::= <<
strict protected
type
TDFA<dfa.decisionNumber> = class(TDFA)
protected
{ IDFA }
function Description: String; override;
public
constructor Create(const ARecognizer: IBaseRecognizer);
end;
var
FDFA<dfa.decisionNumber>: IDFA;
<if(dfa.specialStateSTs)>
strict protected
function DFA<dfa.decisionNumber>_SpecialStateTransition(const DFA: IDFA; S: Integer;
const AInput: IIntStream): Integer;<endif>
>>
cyclicDFA(dfa) ::= <<
{ T<grammar.recognizerName>.TDFA<dfa.decisionNumber> }
constructor T<grammar.recognizerName>.TDFA<dfa.decisionNumber>.Create(const ARecognizer: IBaseRecognizer);
const
DFA<dfa.decisionNumber>_EOT = '<dfa.javaCompressedEOT; wrap="'+\n '">';
DFA<dfa.decisionNumber>_EOF = '<dfa.javaCompressedEOF; wrap="'+\n '">';
DFA<dfa.decisionNumber>_MIN = '<dfa.javaCompressedMin; wrap="'+\n '">';
DFA<dfa.decisionNumber>_MAX = '<dfa.javaCompressedMax; wrap="'+\n '">';
DFA<dfa.decisionNumber>_ACCEPT = '<dfa.javaCompressedAccept; wrap="'+\n '">';
DFA<dfa.decisionNumber>_SPECIAL = '<dfa.javaCompressedSpecial; wrap="'+\n '">';
DFA<dfa.decisionNumber>_TRANSITION: array [0..<length(dfa.javaCompressedTransition)>-1] of String = (
<dfa.javaCompressedTransition:{s|'<s; wrap="'+\n'">'}; separator=",\n">);
begin
inherited Create;
Recognizer := ARecognizer;
DecisionNumber := <dfa.decisionNumber>;
EOT := TDFA.UnpackEncodedString(DFA<dfa.decisionNumber>_EOT);
EOF := TDFA.UnpackEncodedString(DFA<dfa.decisionNumber>_EOF);
Min := TDFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_MIN);
Max := TDFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_MAX);
Accept := TDFA.UnpackEncodedString(DFA<dfa.decisionNumber>_ACCEPT);
Special := TDFA.UnpackEncodedString(DFA<dfa.decisionNumber>_SPECIAL);
Transition := TDFA.UnpackEncodedStringArray(DFA<dfa.decisionNumber>_TRANSITION);
end;
function T<grammar.recognizerName>.TDFA<dfa.decisionNumber>.Description: String;
begin
Result := '<dfa.description>';
end;<\n>
<if(dfa.specialStateSTs)>
function T<grammar.recognizerName>.DFA<dfa.decisionNumber>_SpecialStateTransition(const DFA: IDFA; S: Integer;
const AInput: IIntStream): Integer;
var
Locals: TLocalStorage;
<if(LEXER)>
Input: IIntStream;
<endif>
<if(PARSER)>
Input: ITokenStream;
<endif>
<if(TREE_PARSER)>
Input: ITreeNodeStream;
<endif>
_S: Integer;
NVAE: ENoViableAltException;
begin
Result := -1;
Locals.Initialize;
try
<if(LEXER)>
Input := AInput;
<endif>
<if(PARSER)>
Input := AInput as ITokenStream;
<endif>
<if(TREE_PARSER)>
Input := AInput as ITreeNodeStream;
<endif>
_S := S;
case S of
<dfa.specialStateSTs:{state | <i0>: begin<! compressed special state numbers 0..n-1 !>
<state> <\n> end;}; separator="\n">
end;
<if(backtracking)>
if (State.Backtracking > 0) then
begin
State.Failed := True;
Exit(-1);
end;<\n>
<endif>
NVAE := ENoViableAltException.Create(DFA.Description, <dfa.decisionNumber>, _S, Input);
DFA.Error(NVAE);
raise NVAE;
finally
Locals.Finalize;
end;
end;<\n>
<endif>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
FLA[<decisionNumber>,<stateNumber>] := Input.LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
Locals.AsInteger['index<decisionNumber>_<stateNumber>'] := Input.Index;
Input.Rewind;<\n>
<endif>
S := -1;
<edges; separator="\nelse ">;
<if(semPredState)> <! return input cursor to state before we rewound !>
Input.Seek(Locals.AsInteger['index<decisionNumber>_<stateNumber>']);<\n>
<endif>
if (S >= 0) then
Exit(S);
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ((<labelExpr>)<if(predicates)> and (<predicates>)<endif>) then
S := <targetStateNumber>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
S := <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "((<left>) and (<right>))"
orPredicates(operands) ::= "((<first(operands)>)<rest(operands):{o | or (<o>)}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "FLA[<decisionNumber>,<stateNumber>] = <atomAsInt>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "Input.LA(<k>) = <atomAsInt>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
((FLA[<decisionNumber>,<stateNumber>] \>= <lowerAsInt>) and (FLA[<decisionNumber>,<stateNumber>] \<= <upperAsInt>))
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(Input.LA(<k>) \>= <lowerAsInt>) and (Input.LA(<k>) \<= <upperAsInt>)"
setTest(ranges) ::= "<ranges; separator=\") or (\">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<scope.name>Stack := TStackList\<I<scope.name>Scope\>.Create;<\n>
<endif>
>>
globalAttributeScopeDeclaration(scope) ::= <<
<if(scope.attributes)>
strict protected
type
I<scope.name>Scope = interface(IANTLRObject)
end;
T<scope.name>Scope = class(TANTLRObject, I<scope.name>Scope)
protected
<scope.attributes:{<it.name>: <it.type>;}; separator="\n">
end;
strict protected
<scope.name>Stack: IStackList\<I<scope.name>Scope\>;
<endif>
>>
ruleAttributeScopeDeclaration(scope) ::= <<
<if(scope.attributes)>
strict protected
type
I<scope.name>Scope = interface(IANTLRObject)
end;
T<scope.name>Scope = class(TANTLRObject, I<scope.name>Scope)
protected
<scope.attributes:{<it.name>: <it.type>;}; separator="\n">
end;
strict protected
<scope.name>Stack: IStackList\<I<scope.name>Scope\>;
<endif>
>>
ruleAttributeScope(scope) ::= <<
<! protected Stack <scope.name>Stack = new Stack();<\n> !>
>>
ruleAttributeScopeInit(scope) ::= <<
<if(scope)>
<scope.name>Stack := TStackList\<I<scope.name>Scope\>.Create;<\n>
<endif>
>>
returnStructName() ::= "<it.name>_return"
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor:returnStructName()>
<! I<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope !>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
<! Pointer/void !>
<endif>
<endif>
>>
/** Generate the C# type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
I<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
<csharpTypeInitMap.(typeName)>
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<label.label.text> := <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
ruleLabelDefVar(label) ::= <<
<label.label.text>: <ruleLabelType(referencedRule=label.referencedRule)>;
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
{ T<ruleDescriptor:returnStructName()> }
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
<endif>
>>
returnScopeDeclaration(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
public
type
T<ruleDescriptor:returnStructName()> = class(T<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope, I<ruleDescriptor:returnStructName()>)
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
end;
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> := <expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
(<scope>Stack[<scope>Stack.Count-<negIndex>-1] as T<scope>Scope).<attr.name>
<else>
<if(index)>
(<scope>Stack[<index>] as T<scope>Scope).<attr.name>
((<scope>_scope)<scope>_stack[<index>]).<attr.name>
<else>
(<scope>Stack.Peek.Implementor as T<scope>Scope).<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
(<scope>Stack[<scope>Stack.Count-<negIndex>-1] as T<scope>Scope).<attr.name> := <expr>;<\n>
<else>
<if(index)>
(<scope>Stack[<index>] as T<scope>Scope).<attr.name> := <expr>;<\n>
<else>
(<scope>Stack.Peek.Implementor as T<scope>Scope).<attr.name> := <expr>;<\n>
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>Stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
(IfThen(Assigned(<scope>),Def(<scope>).<attr.name>,<initValue(attr.type)>))
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
RetVal.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
RetVal.<attr.name> := <expr>;
<else>
<attr.name> := <expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "(Def(<scope>).Text)"
tokenLabelPropertyRef_type(scope,attr) ::= "(Def(<scope>).TokenType)"
tokenLabelPropertyRef_line(scope,attr) ::= "(Def(<scope>).Line)"
tokenLabelPropertyRef_pos(scope,attr) ::= "(Def(<scope>).CharPositionInLine)"
tokenLabelPropertyRef_channel(scope,attr) ::= "(Def(<scope>).Channel)"
tokenLabelPropertyRef_index(scope,attr) ::= "(Def(<scope>).TokenIndex)"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
tokenLabelPropertyRef_int(scope,attr) ::= "(StrToIntDef(Def(<scope>).Text,0))"
ruleLabelPropertyRef_start(scope,attr) ::= "(IfThen(Assigned(<scope>), Def(<scope>).Start, nil) as I<labelType>)"
ruleLabelPropertyRef_stop(scope,attr) ::= "(Def(<scope>).Stop as I<labelType>)"
ruleLabelPropertyRef_tree(scope,attr) ::= "(Def(Def(<scope>).Tree as I<ASTLabelType>))"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
IfThen(Assigned(<scope>), Input.TokenStream.ToString(
Input.TreeAdaptor.GetTokenStartIndex(Def(<scope>).Start),
Input.TreeAdaptor.GetTokenStopIndex(Def(<scope>).Start)), '')
<else>
IfThen(Assigned(<scope>), Input.ToString(
(Def(<scope>).Start) as IToken,(Def(<scope>).Stop) as IToken), '')
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "((<scope> != null) ? <scope>.ST : null)"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "(Def(<scope>).TokenType)"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "(Def(<scope>).Line)"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "(IfThen(Assigned(<scope>),Def(<scope>).CharPositionInLine,-1))"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "(Def(<scope>).Channel)"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "(Def(<scope>).TokenIndex)"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "(Def(<scope>).Text)"
lexerRuleLabelPropertyRef_int(scope,attr) ::= "(StrToIntDef(Def(<scope>).Text,0))"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "(RetVal.Start as I<labelType>)"
rulePropertyRef_stop(scope,attr) ::= "(RetVal.Stop as I<labelType>)"
rulePropertyRef_tree(scope,attr) ::= "(RetVal.Tree as I<ASTLabelType>)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
Input.TokenStream.ToString(
Input.TreeAdaptor.GetTokenStartIndex(RetVal.Start),
Input.TreeAdaptor.GetTokenStopIndex(RetVal.Start))
<else>
Input.ToString(RetVal.Start as IToken,Input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "RetVal.ST"
lexerRulePropertyRef_text(scope,attr) ::= "Text"
lexerRulePropertyRef_type(scope,attr) ::= "TokenType"
lexerRulePropertyRef_line(scope,attr) ::= "State.TokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "State.TokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "Channel"
lexerRulePropertyRef_start(scope,attr) ::= "State.TokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(CharIndex-1)"
lexerRulePropertyRef_int(scope,attr) ::= "StrToInt(<scope>.Text)"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "RetVal.Tree := <expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "RetVal.ST := <expr>;"
/** How to execute an action (only when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
<if(actions.(actionScope).synpredgate)>
if (<actions.(actionScope).synpredgate>) then
begin
<action>
end;
<else>
if (State.Backtracking = 0) then
begin
<action>
end;<\n>
<endif>
<else>
<action>
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
<name> := TBitSet.Create([<words64:{<it>};separator=",">]);<\n>
>>
bitsetDecl(name) ::= <<
<name>: IBitSet;<\n>
>>
codeFileExtension() ::= ".pas"
true() ::= "True"
false() ::= "False"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/JavaScript/ 0000755 0001750 0001750 00000000000 11410174107 026511 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/JavaScript/JavaScript.stg 0000644 0001750 0001750 00000123431 11256465234 031316 0 ustar twerner twerner group JavaScript implements ANTLRCore;
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<actions.(actionScope).header>
<@imports>
<if(TREE_PARSER)>
<endif>
<@end>
<docComment>
<recognizer>
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="Token",
filterMode, superClass="org.antlr.runtime.Lexer") ::= <<
var <grammar.recognizerName> = function(input, state<grammar.delegators:{g|, <g:delegateName()>}>) {
// alternate constructor @todo
// public <grammar.recognizerName>(CharStream input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
// public <grammar.recognizerName>(CharStream input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>) {
if (!state) {
state = new org.antlr.runtime.RecognizerSharedState();
}
(function(){
<actions.lexer.members>
}).call(this);
<cyclicDFAs:{dfa | this.dfa<dfa.decisionNumber> = new <grammar.recognizerName>.DFA<dfa.decisionNumber>(this);}; separator="\n">
<grammar.recognizerName>.superclass.constructor.call(this, input, state);
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = {};
<endif>
<endif>
<grammar.directDelegates:
{g|this.<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|this.gParent = this.<g:delegateName()>;}>
<actions.lexer.init>
};
org.antlr.lang.augmentObject(<grammar.recognizerName>, {
<tokens:{<it.name>: <it.type>}; separator=",\n">
});
(function(){
var HIDDEN = org.antlr.runtime.Token.HIDDEN_CHANNEL,
EOF = org.antlr.runtime.Token.EOF;
org.antlr.lang.extend(<grammar.recognizerName>, <@superClassName><superClass><@end>, {
<tokens:{<it.name> : <it.type>,}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
getGrammarFileName: function() { return "<fileName>"; }
});
org.antlr.lang.augmentObject(<grammar.recognizerName>.prototype, {
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator=",\n\n">
<synpreds:{p | <lexerSynpred(p)>}; separator=",\n">
}, true); // important to pass true to overwrite default implementations
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
})();
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
nextToken: function() {
while (true) {
if ( this.input.LA(1)==org.antlr.runtime.CharStream.EOF ) {
return org.antlr.runtime.Token.EOF_TOKEN;
}
this.state.token = null;
this.state.channel = org.antlr.runtime.Token.DEFAULT_CHANNEL;
this.state.tokenStartCharIndex = this.input.index();
this.state.tokenStartCharPositionInLine = this.input.getCharPositionInLine();
this.state.tokenStartLine = this.input.getLine();
this.state.text = null;
try {
var m = this.input.mark();
this.state.backtracking=1; <! means we won't throw slow exception !>
this.state.failed=false;
this.mTokens();
this.state.backtracking=0;
<! mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1. !>
if ( this.state.failed ) {
this.input.rewind(m);
this.input.consume(); <! advance one char and try again !>
}
else {
this.emit();
return this.state.token;
}
}
catch (re) {
// shouldn't happen in backtracking mode, but...
if (re instanceof org.antlr.runtime.RecognitionException) {
this.reportError(re);
this.recover(re);
} else {
throw re;
}
}
}
},
memoize: function(input, ruleIndex, ruleStartIndex) {
if (this.state.backtracking>1) {
<grammar.recognizerName>.superclass.prototype.memoize.call(this, input, ruleIndex, ruleStartIndex);
}
},
alreadyParsedRule: function(input, ruleIndex) {
if (this.state.backtracking>1) {
return <grammar.recognizerName>.superclass.prototype.alreadyParsedRule.call(this, input, ruleIndex);
}
return false;
},
>>
actionGate() ::= "this.state.backtracking===0"
filteringActionGate() ::= "this.state.backtracking===1"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass,
ASTLabelType="Object", labelType, members, rewriteElementType) ::= <<
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
var <grammar.recognizerName> = function(input, state<grammar.delegators:{g|, <g:delegateName()>}>) {
if (!state) {
state = new org.antlr.runtime.RecognizerSharedState();
}
(function(){
<members>
}).call(this);
<grammar.recognizerName>.superclass.constructor.call(this, input, state);
<cyclicDFAs:{dfa | this.dfa<dfa.decisionNumber> = new <grammar.recognizerName>.DFA<dfa.decisionNumber>(this);}; separator="\n">
<parserCtorBody()>
<grammar.directDelegates:
{g|this.<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.indirectDelegates:{g | this.<g:delegateName()> = <g.delegator:delegateName()>.<g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|this.gParent = this.<g:delegateName()>;}>
/* @todo only create adaptor if output=AST */
this.adaptor = new org.antlr.runtime.tree.CommonTreeAdaptor();<\n>
};
org.antlr.lang.augmentObject(<grammar.recognizerName>, {
<tokens:{<it.name>: <it.type>}; separator=",\n">
});
(function(){
// public class variables
var <tokens:{<it.name>= <it.type>}; separator=",\n ">;
<if(TREE_PARSER)>
var UP = org.antlr.runtime.Token.UP,
DOWN = org.antlr.runtime.Token.DOWN;
<endif>
// public instance methods/vars
org.antlr.lang.extend(<grammar.recognizerName>, org.antlr.runtime.<@superClassName><superClass><@end>, {
<@members>
<@end>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
getTokenNames: function() { return <grammar.composite.rootGrammar.recognizerName>.tokenNames; },
getGrammarFileName: function() { return "<fileName>"; }
});
org.antlr.lang.augmentObject(<grammar.recognizerName>.prototype, {
<rules; separator=",\n\n">
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
// Delegated rules
<grammar.delegatedRules:{ruleDescriptor|
, <ruleDescriptor.name>: function(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) \{ <if(ruleDescriptor.hasReturnValue)>return <endif>this.<ruleDescriptor.grammar:delegateName()>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">); \}}>
<synpreds:{p | <synpred(p)>}; separator=",\n">
}, true); // important to pass true to overwrite default implementations
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
// public class variables
org.antlr.lang.augmentObject(<grammar.recognizerName>, {
<if(grammar.grammarIsRoot)>
tokenNames: ["\<invalid>", "\<EOR>", "\<DOWN>", "\<UP>", <tokenNames; separator=", ">],<\n>
<endif>
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits); separator=",\n">
});
})();
>>
parserCtorBody() ::= <<
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = {};<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType="Object", superClass="Parser", labelType="Token", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="TokenStream", rewriteElementType="Token", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="var", superClass="tree.TreeParser", members={<actions.treeparser.members>}) ::= <<
<genericParser(inputStreamType="TreeNodeStream", rewriteElementType="Node", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start "<ruleName>"
<ruleName>_fragment: function(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) {
<if(trace)>
this.traceIn("<ruleName>_fragment", <ruleDescriptor.index>);
try {
<block>
}
finally {
this.traceOut("<ruleName>_fragment", <ruleDescriptor.index>);
}
<else>
<block>
<endif>
},
// $ANTLR end "<ruleName>"
>>
synpred(name) ::= <<
<name>: function() {
this.state.backtracking++;
<@start()>
var start = this.input.mark();
try {
this.<name>_fragment(); // can never throw exception
} catch (re) {
alert("impossible: "+re.toString());
}
var success = !this.state.failed;
this.input.rewind(start);
<@stop()>
this.state.backtracking--;
this.state.failed=false;
return success;
}
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( this.state.backtracking>0 && this.alreadyParsedRule(this.input, <ruleDescriptor.index>) ) { return <ruleReturnValue()>; }
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>if (this.state.failed) return <ruleReturnValue()>;<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if (this.state.backtracking>0) {this.state.failed=true; return <ruleReturnValue()>;}<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
// <fileName>:<description>
// $ANTLR start "<ruleName>"
<ruleDescriptor.actions.decorate>
<ruleName>: function(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) {
<if(trace)>this.traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try {
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
catch (re) {
if (re instanceof org.antlr.runtime.RecognitionException) {
this.reportError(re);
this.recover(this.input,re);
<@setErrorReturnValue()>
} else {
throw re;
}
}<\n>
<endif>
<endif>
<endif>
finally {
<if(trace)>this.traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
}
<@postamble()>
return <ruleReturnValue()>;
}
>>
catch(decl,action) ::= <<
catch (<e.decl>) {
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
var retval = new <returnType()>();
retval.start = this.input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
var <a.name> = <if(a.initValue)><a.initValue><else>null<endif>;
}>
<endif>
<if(memoize)>
var <ruleDescriptor.name>_StartIndex = this.input.index();
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{this.<it>_stack.push({});}; separator="\n">
<ruleDescriptor.ruleScope:{this.<it.name>_stack.push({});}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{this.<it>_stack.pop();}; separator="\n">
<ruleDescriptor.ruleScope:{this.<it.name>_stack.pop();}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]
:{var <it.label.text> = null;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]
:{var list_<it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|var <ll.label.text> = null;}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{var <it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.charLabels:{var <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleListLabels]
:{var list_<it.label.text>=null;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
retval.stop = this.input.LT(-1);<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( this.state.backtracking>0 ) { this.memoize(this.input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex); }
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// $ANTLR start <ruleName>
m<ruleName>: function(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) {
<if(trace)>this.traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
try {
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
var _type = this.<ruleName>;
var _channel = org.antlr.runtime.BaseRecognizer.DEFAULT_TOKEN_CHANNEL;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
this.state.type = _type;
this.state.channel = _channel;
<(ruleDescriptor.actions.after):execAction()>
<endif>
}
finally {
<if(trace)>this.traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeCleanUp()>
<memoize()>
}
},
// $ANTLR end "<ruleName>"
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
mTokens: function() {
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
var alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
var alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
var cnt<decisionNumber>=0;
<decls>
<@preloop()>
loop<decisionNumber>:
do {
var alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
if ( cnt<decisionNumber> >= 1 ) {
break loop<decisionNumber>;
}
<ruleBacktrackFailure()>
var eee = new org.antlr.runtime.EarlyExitException(<decisionNumber>, this.input);
<@earlyExitException()>
throw eee;
}
cnt<decisionNumber>++;
} while (true);
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
<decls>
<@preloop()>
loop<decisionNumber>:
do {
var alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
break loop<decisionNumber>;
}
} while (true);
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> :
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
// <fileName>:<description>
<! (function() { /* @todo4 (do we really need a new scope?) */ !>
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
<! }).call(this); !>
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label>=<endif>this.match(this.input,<token>,<grammar.recognizerName>.FOLLOW_<token>_in_<ruleName><elementIndex>); <checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (org.antlr.lang.isNull(list_<label>)) list_<label> = [];
list_<label>.push(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = this.input.LA(1);<\n>
<endif>
this.match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = this.input.LA(1);<\n>
<endif>
this.matchRange(<a>,<b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= this.input.LA(1);<\n>
<else>
<label>=this.input.LT(1);<\n>
<endif>
<endif>
if ( <s> ) {
this.input.consume();
<postmatchCode>
<if(!LEXER)>
this.state.errorRecovery=false;
<endif>
<if(backtracking)>this.state.failed=false;<endif>
}
else {
<ruleBacktrackFailure()>
var mse = new org.antlr.runtime.MismatchedSetException(null,this.input);
<@mismatchedSetException()>
<if(LEXER)>
this.recover(mse);
throw mse;
<else>
throw mse;
<! use following code to make it recover inline; remove throw mse;
this.recoverFromMismatchedSet(this.input,mse,<grammar.recognizerName>.FOLLOW_set_in_<ruleName><elementIndex>);
!>
<endif>
}<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
var <label>Start = this.getCharIndex();
this.match(<string>); <checkRuleBacktrackFailure()>
var <label> = new org.antlr.runtime.CommonToken(this.input, org.antlr.runtime.Token.INVALID_TOKEN_TYPE, org.antlr.runtime.Token.DEFAULT_CHANNEL, <label>Start, this.getCharIndex()-1);
<else>
this.match(<string>); <checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label>=this.input.LT(1);<\n>
<endif>
this.matchAny(this.input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = this.input.LA(1);<\n>
<endif>
this.matchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
// XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
this.pushFollow(<grammar.recognizerName>.FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)><label>=<endif>this.<if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
this.state._fsp--;
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference */
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
var <label>Start<elementIndex> = this.getCharIndex();
this.<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> = new org.antlr.runtime.CommonToken(this.input, org.antlr.runtime.Token.INVALID_TOKEN_TYPE, org.antlr.runtime.Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, this.getCharIndex()-1);
<else>
this.<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
var <label>Start<elementIndex> = this.getCharIndex();
this.match(EOF); <checkRuleBacktrackFailure()>
var <label> = new org.antlr.runtime.CommonToken(this.input, this.EOF, org.antlr.runtime.Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, this.getCharIndex()-1);
<else>
this.match(this.EOF); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( this.input.LA(1)==org.antlr.runtime.Token.DOWN ) {
this.match(this.input, org.antlr.runtime.Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
this.match(this.input, org.antlr.runtime.Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
this.match(this.input, org.antlr.runtime.Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
this.match(this.input, org.antlr.runtime.Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) ) {
<ruleBacktrackFailure()>
throw new org.antlr.runtime.FailedPredicateException(this.input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber> = this.input.LA(<k>);<\n>
<edges; separator="\nelse ">
else {
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
var nvae =
new org.antlr.runtime.NoViableAltException("<description>", <decisionNumber>, <stateNumber>, this.input);<\n>
<@noViableAltException()>
throw nvae;<\n>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber> = this.input.LA(<k>);<\n>
<edges; separator="\nelse ">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber> = this.input.LA(<k>);<\n>
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber>=<eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else {
alt<decisionNumber>=<eotPredictsAlt>;
}<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber>=<alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( this.input.LA(<k>) ) {
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
var nvae =
new org.antlr.runtime.NoViableAltException("<description>", <decisionNumber>, <stateNumber>, this.input);<\n>
<@noViableAltException()>
throw nvae;<\n>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( this.input.LA(<k>) ) {
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( this.input.LA(<k>) ) {
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber>=<eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
<targetState>
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = this.dfa<decisionNumber>.predict(this.input);
>>
/* Dump DFA tables as run-length-encoded Strings of octal values.
* Can't use hex as compiler translates them before compilation.
* These strings are split into multiple, concatenated strings.
* Java puts them back together at compile time thankfully.
* Java cannot handle large static arrays, so we're stuck with this
* encode/decode approach. See analysis and runtime DFA for
* the encoding methods.
*/
cyclicDFA(dfa) ::= <<
org.antlr.lang.augmentObject(<grammar.recognizerName>, {
DFA<dfa.decisionNumber>_eotS:
"<dfa.javaCompressedEOT; wrap="\"+\n \"">",
DFA<dfa.decisionNumber>_eofS:
"<dfa.javaCompressedEOF; wrap="\"+\n \"">",
DFA<dfa.decisionNumber>_minS:
"<dfa.javaCompressedMin; wrap="\"+\n \"">",
DFA<dfa.decisionNumber>_maxS:
"<dfa.javaCompressedMax; wrap="\"+\n \"">",
DFA<dfa.decisionNumber>_acceptS:
"<dfa.javaCompressedAccept; wrap="\"+\n \"">",
DFA<dfa.decisionNumber>_specialS:
"<dfa.javaCompressedSpecial; wrap="\"+\n \"">}>",
DFA<dfa.decisionNumber>_transitionS: [
<dfa.javaCompressedTransition:{s|"<s; wrap="\"+\n\"">"}; separator=",\n">
]
});
org.antlr.lang.augmentObject(<grammar.recognizerName>, {
DFA<dfa.decisionNumber>_eot:
org.antlr.runtime.DFA.unpackEncodedString(<grammar.recognizerName>.DFA<dfa.decisionNumber>_eotS),
DFA<dfa.decisionNumber>_eof:
org.antlr.runtime.DFA.unpackEncodedString(<grammar.recognizerName>.DFA<dfa.decisionNumber>_eofS),
DFA<dfa.decisionNumber>_min:
org.antlr.runtime.DFA.unpackEncodedStringToUnsignedChars(<grammar.recognizerName>.DFA<dfa.decisionNumber>_minS),
DFA<dfa.decisionNumber>_max:
org.antlr.runtime.DFA.unpackEncodedStringToUnsignedChars(<grammar.recognizerName>.DFA<dfa.decisionNumber>_maxS),
DFA<dfa.decisionNumber>_accept:
org.antlr.runtime.DFA.unpackEncodedString(<grammar.recognizerName>.DFA<dfa.decisionNumber>_acceptS),
DFA<dfa.decisionNumber>_special:
org.antlr.runtime.DFA.unpackEncodedString(<grammar.recognizerName>.DFA<dfa.decisionNumber>_specialS),
DFA<dfa.decisionNumber>_transition: (function() {
var a = [],
i,
numStates = <grammar.recognizerName>.DFA<dfa.decisionNumber>_transitionS.length;
for (i=0; i\<numStates; i++) {
a.push(org.antlr.runtime.DFA.unpackEncodedString(<grammar.recognizerName>.DFA<dfa.decisionNumber>_transitionS[i]));
}
return a;
})()
});
<grammar.recognizerName>.DFA<dfa.decisionNumber> = function(recognizer) {
this.recognizer = recognizer;
this.decisionNumber = <dfa.decisionNumber>;
this.eot = <grammar.recognizerName>.DFA<dfa.decisionNumber>_eot;
this.eof = <grammar.recognizerName>.DFA<dfa.decisionNumber>_eof;
this.min = <grammar.recognizerName>.DFA<dfa.decisionNumber>_min;
this.max = <grammar.recognizerName>.DFA<dfa.decisionNumber>_max;
this.accept = <grammar.recognizerName>.DFA<dfa.decisionNumber>_accept;
this.special = <grammar.recognizerName>.DFA<dfa.decisionNumber>_special;
this.transition = <grammar.recognizerName>.DFA<dfa.decisionNumber>_transition;
};
org.antlr.lang.extend(<grammar.recognizerName>.DFA<dfa.decisionNumber>, org.antlr.runtime.DFA, {
getDescription: function() {
return "<dfa.description>";
},
<@errorMethod()>
<if(dfa.specialStateSTs)>
specialStateTransition: function(s, input) {
var _s = s;
/* bind to recognizer so semantic predicates can be evaluated */
var retval = (function(s, input) {
switch ( s ) {
<dfa.specialStateSTs:{state |
case <i0> : <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
}).call(this.recognizer, s, input);
if (!org.antlr.lang.isUndefined(retval)) {
return retval;
}
<if(backtracking)>
if (this.recognizer.state.backtracking>0) {this.recognizer.state.failed=true; return -1;}<\n>
<endif>
var nvae =
new org.antlr.runtime.NoViableAltException(this.getDescription(), <dfa.decisionNumber>, _s, input);
this.error(nvae);
throw nvae;
},<\n>
<endif>
dummy: null
});<\n>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
var LA<decisionNumber>_<stateNumber> = input.LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
var index<decisionNumber>_<stateNumber> = input.index();
input.rewind();<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
<if(semPredState)> <! return input cursor to state before we rewound !>
input.seek(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s>=0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {s = <targetStateNumber>;}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left>&&<right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | ||<o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "this.<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber>==<atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "this.input.LA(<k>)==<atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber>\>=<lower> && LA<decisionNumber>_<stateNumber>\<=<upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(this.input.LA(<k>)\>=<lower> && this.input.LA(<k>)\<=<upper>)"
setTest(ranges) ::= "<ranges; separator=\"||\">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
<scope.name>_stack: [],<\n>
<endif>
>>
ruleAttributeScope(scope) ::= <<
<if(scope.attributes)>
<scope.name>_stack: [],<\n>
<endif>
>>
returnStructName() ::= "<it.name>_return"
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.grammar.recognizerName>.<ruleDescriptor:returnStructName()>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Generate the Java type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.grammar.recognizerName>.<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
null
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<!<ruleLabelType(referencedRule=label.referencedRule)>!> var <label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
// inline static return class
<ruleDescriptor:returnStructName()>: (function() {
<returnType()> = function(){};
org.antlr.lang.extend(<returnType()>,
org.antlr.runtime.<if(TREE_PARSER)>tree.Tree<else>Parser<endif>RuleReturnScope,
{
<@ruleReturnMembers()>
});
return;
})(),
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> =<expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
(this.<scope>_stack[this.<scope>_stack.length-<negIndex>-1]).<attr.name>
<else>
<if(index)>
(this.<scope>_stack[<index>]).<attr.name>
<else>
org.antlr.lang.array.peek(this.<scope>_stack).<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
(this.<scope>_stack[this.<scope>_stack.length-<negIndex>-1]).<attr.name> =<expr>;
<else>
<if(index)>
(this.<scope>_stack[<index>]).<attr.name> =<expr>;
<else>
org.antlr.lang.array.peek(this.<scope>_stack).<attr.name> =<expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "this.<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
(<scope>!==null?<scope>.<attr.name>:<initValue(attr.type)>)
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name> =<expr>;
<else>
<attr.name> =<expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "(<scope>?<scope>.getText():null)"
tokenLabelPropertyRef_type(scope,attr) ::= "(<scope>?<scope>.getType():0)"
tokenLabelPropertyRef_line(scope,attr) ::= "(<scope>?<scope>.getLine():0)"
tokenLabelPropertyRef_pos(scope,attr) ::= "(<scope>?<scope>.getCharPositionInLine():0)"
tokenLabelPropertyRef_channel(scope,attr) ::= "(<scope>?<scope>.getChannel():0)"
tokenLabelPropertyRef_index(scope,attr) ::= "(<scope>?<scope>.getTokenIndex():0)"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
tokenLabelPropertyRef_int(scope,attr) ::= "(<scope>?parseInt(<scope>.getText(), 10):0)"
ruleLabelPropertyRef_start(scope,attr) ::= "(<scope>?<scope>.start:null)"
ruleLabelPropertyRef_stop(scope,attr) ::= "(<scope>?<scope>.stop:null)"
ruleLabelPropertyRef_tree(scope,attr) ::= "(<scope>?<scope>.tree:null)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
(<scope>?(this.input.getTokenStream().toString(
this.input.getTreeAdaptor().getTokenStartIndex(<scope>.start),
this.input.getTreeAdaptor().getTokenStopIndex(<scope>.start))):null)
<else>
(<scope>?this.input.toString(<scope>.start,<scope>.stop):null)
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "<scope>.st"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "(<scope>?<scope>.getType():0)"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "(<scope>?<scope>.getLine():0)"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "(<scope>?<scope>.getCharPositionInLine():-1)"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "(<scope>?<scope>.getChannel():0)"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "(<scope>?<scope>.getTokenIndex():0)"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "(<scope>?<scope>.getText():0)"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "(retval.start)"
rulePropertyRef_stop(scope,attr) ::= "(retval.stop)"
rulePropertyRef_tree(scope,attr) ::= "(retval.tree)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
this.input.getTokenStream().toString(
this.input.getTreeAdaptor().getTokenStartIndex(retval.start),
this.input.getTreeAdaptor().getTokenStopIndex(retval.start))
<else>
this.input.toString(retval.start,this.input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.st"
lexerRulePropertyRef_text(scope,attr) ::= "this.getText()"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "this.state.tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "this.state.tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "_channel"
lexerRulePropertyRef_start(scope,attr) ::= "this.state.tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(this.getCharIndex()-1)"
lexerRulePropertyRef_int(scope,attr) ::= "parseInt(<scope>.getText(),10)"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.tree =<expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.st =<expr>;"
/** How to execute an action */
execAction(action) ::= <<
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {
<action>
}
<else>
<action>
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
<! @todo overflow issue !>
<name>: new org.antlr.runtime.BitSet([<words64:{<it>};separator=",">])
>>
codeFileExtension() ::= ".js"
true() ::= "true"
false() ::= "false"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/JavaScript/ASTTreeParser.stg 0000644 0001750 0001750 00000016052 11256465232 031672 0 ustar twerner twerner /** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
var _first_0 = null;
var _last = null;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(rewriteMode)>
retval.tree = _first_0;
if ( this.adaptor.getParent(retval.tree) && this.adaptor.isNil( this.adaptor.getParent(retval.tree) ) )
retval.tree = this.adaptor.getParent(retval.tree);
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = this.input.LT(1);
{
var _save_last_<treeLevel> = _last;
var _first_<treeLevel> = null;
<if(!rewriteMode)>
var root_<treeLevel> = this.adaptor.nil();
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
<if(root.el.rule)>
if ( !_first_<enclosingTreeLevel> ) _first_<enclosingTreeLevel> = <root.el.label>.tree;
<else>
if ( !_first_<enclosingTreeLevel> ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( this.input.LA(1)==org.antlr.runtime.Token.DOWN ) {
this.match(this.input, org.antlr.runtime.Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
this.match(this.input, org.antlr.runtime.Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
this.match(this.input, org.antlr.runtime.Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
this.match(this.input, org.antlr.runtime.Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
this.adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = this.input.LT(1);
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = this.input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = this.adaptor.dupNode(<label>);
<endif><\n>
this.adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( !_first_<treeLevel> ) _first_<treeLevel> = <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = this.input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = this.adaptor.dupNode(<label>);
<endif><\n>
root_<treeLevel> = this.adaptor.becomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = this.input.LT(1);
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = this.adaptor.dupNode(<label>);
<endif><\n>
this.adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = this.input.LT(1);
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = this.adaptor.dupNode(<label>);
<endif><\n>
root_<treeLevel> = this.adaptor.becomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = this.input.LT(1);
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>
<if(!rewriteMode)>
this.adaptor.addChild(root_<treeLevel>, <label>.getTree());
<else> <! rewrite mode !>
if ( !_first_<treeLevel> ) _first_<treeLevel> = <label>.tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = this.input.LT(1);
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_<treeLevel> = this.adaptor.becomeRoot(<label>.getTree(), root_<treeLevel>);
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = this.input.LT(1);
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = this.input.LT(1);
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = this.input.LT(1);
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = this.input.LT(1);
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextNode())
<else>
stream_<token>.nextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = this.adaptor.rulePostProcessing(root_0);
<if(backtracking)>}<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/JavaScript/ASTParser.stg 0000644 0001750 0001750 00000012345 11256465232 031053 0 ustar twerner twerner /** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
retval.tree = this.adaptor.errorNode(this.input, retval.start, this.input.LT(-1), re);
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
this.adaptor.addChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
root_0 = this.adaptor.becomeRoot(<label>_tree, root_0);
<if(backtracking)>}<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>this.adaptor.addChild(root_0, <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label>=input.LT(1);<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = this.adaptor.becomeRoot(<createNodeFromToken(...)>, root_0);})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>this.adaptor.addChild(root_0, <label>.getTree());
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = this.adaptor.becomeRoot(<label>.getTree(), root_0);
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".getTree()",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = this.adaptor.create(<label>);
this.adaptor.addChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = this.adaptor.create(<label>);
root_0 = this.adaptor.becomeRoot(<label>_tree, root_0);
<if(backtracking)>}<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
new <hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
this.adaptor.create(<label>)
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = this.adaptor.rulePostProcessing(root_0);
this.adaptor.setTokenBoundaries(retval.tree, retval.start, retval.stop);
<if(backtracking)>}<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/JavaScript/AST.stg 0000644 0001750 0001750 00000030415 11256465232 027674 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()>
>>
@genericParser.members() ::= <<
<@super.members()>
<parserMembers()>
>>
/** Add an adaptor property that knows how to build trees */
parserMembers() ::= <<
<!protected TreeAdaptor adaptor = new CommonTreeAdaptor();<\n>!>
setTreeAdaptor: function(adaptor) {
this.adaptor = adaptor;
<grammar.directDelegates:{g|<g:delegateName()>.setTreeAdaptor(this.adaptor);}>
},
getTreeAdaptor: function() {
return this.adaptor;
},
>>
@returnScope.ruleReturnMembers() ::= <<
getTree: function() { return this.tree; }
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
var root_0 = null;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<ruleDescriptor.tokenLabels:{var <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.tokenListLabels:{var <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{var stream_<it>=new org.antlr.runtime.tree.RewriteRuleTokenStream(this.adaptor,"token <it>");}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{var stream_<it>=new org.antlr.runtime.tree.RewriteRuleSubtreeStream(this.adaptor,"rule <it>");}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = this.adaptor.nil();<\n>
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.add(<label>);<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.add(<label>);<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
wildcardTrack(label,elementIndex) ::= <<
<super.wildcard(...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule.name>.add(<label>.getTree());
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule>.add(<label>.getTree());
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".getTree()",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {<\n>
<endif>
<prevRuleRootRef()>.tree = root_0;
<rewriteCodeLabels()>
root_0 = this.adaptor.nil();
<alts:rewriteAlt(); separator="else ">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.tree = this.adaptor.rulePostProcessing(root_0);
this.input.replaceChildren(this.adaptor.getParent(retval.start),
this.adaptor.getChildIndex(retval.start),
this.adaptor.getChildIndex(_last),
retval.tree);
<endif>
<endif>
<! if parser or tree-parser && rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.tree = root_0;
<else>
<if(!rewriteMode)>
<prevRuleRootRef()>.tree = root_0;
<endif>
<endif>
<if(backtracking)>
}
<endif>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{var stream_<it>=new org.antlr.runtime.tree.RewriteRule<rewriteElementType>Stream(this.adaptor,"token <it>",<it>);};
separator="\n"
>
<referencedTokenListLabels
:{var stream_<it>=new org.antlr.runtime.tree.RewriteRule<rewriteElementType>Stream(this.adaptor,"token <it>", list_<it>);};
separator="\n"
>
<referencedRuleLabels
:{var stream_<it>=new org.antlr.runtime.tree.RewriteRuleSubtreeStream(this.adaptor,"token <it>",<it>!=null?<it>.tree:null);};
separator="\n"
>
<referencedRuleListLabels
:{var stream_<it>=new org.antlr.runtime.tree.RewriteRuleSubtreeStream(this.adaptor,"token <it>",list_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
if ( <referencedElementsDeep:{el | stream_<el>.hasNext()}; separator="||"> ) {
<alt>
}
<referencedElementsDeep:{el | stream_<el>.reset();<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
while ( <referencedElements:{el | stream_<el>.hasNext()}; separator="||"> ) {
<alt>
}
<referencedElements:{el | stream_<el>.reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if ( !(<referencedElements:{el | stream_<el>.hasNext()}; separator="||">) ) {
throw new org.antlr.runtime.tree.RewriteEarlyExitException();
}
while ( <referencedElements:{el | stream_<el>.hasNext()}; separator="||"> ) {
<alt>
}
<referencedElements:{el | stream_<el>.reset();<\n>}>
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>) {
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = null;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
var root_<treeLevel> = this.adaptor.nil();
<root:rewriteElement()>
<children:rewriteElement()>
this.adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>);
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
this.adaptor.addChild(root_<treeLevel>, <createRewriteNodeFromElement(...)>);<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
this.adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode());<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
this.adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode());<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = this.adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = this.adaptor.becomeRoot(<createRewriteNodeFromElement(...)>, root_<treeLevel>);<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
this.adaptor.addChild(root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = this.adaptor.becomeRoot(<createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>);<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
this.adaptor.addChild(root_<treeLevel>, stream_<rule>.nextTree());<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = this.adaptor.becomeRoot(stream_<rule>.nextNode(), root_<treeLevel>);<\n>
>>
rewriteNodeAction(action) ::= <<
this.adaptor.addChild(root_<treeLevel>, <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = this.adaptor.becomeRoot(<action>, root_<treeLevel>);<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
this.adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
this.adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree());<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = this.adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = this.adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>);<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
new <hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
this.adaptor.create(<tokenType>, <args; separator=", "><if(!args)>"<tokenType>"<endif>)
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
this.adaptor.create(<token>, <args; separator=", ">)
<else>
stream_<token>.nextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ANTLRCore.sti 0000644 0001750 0001750 00000032064 11256465230 026672 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
interface ANTLRCore;
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals);
/** The header file; make sure to define headerFileExtension() below */
optional
headerFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals);
lexer(grammar, name, tokens, scopes, rules, numRules, labelType,
filterMode, superClass);
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, ASTLabelType, superClass,
labelType, members);
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules,
numRules, bitsets, labelType, ASTLabelType,
superClass, members, filterMode);
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock);
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize);
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize);
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor);
filteringNextToken();
filteringActionGate();
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description);
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description);
/** A (..)+ block with 0 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
positiveClosureBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
/** A (..)* block with 0 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
closureBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
optionalBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description);
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew);
// E L E M E N T S
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero);
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero);
listLabel(label,elem);
/** match a character */
charRef(char,label);
/** match a character range */
charRangeRef(a,b,label);
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode);
matchSetAndListLabel(s,label,elementIndex,postmatchCode);
/** Match a string literal */
lexerStringRef(string,label);
wildcard(label,elementIndex);
wildcardAndListLabel(label,elementIndex);
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex);
wildcardCharListLabel(label, elementIndex);
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values.
*/
ruleRef(rule,label,elementIndex,args,scope);
/** ids+=ID */
ruleRefAndListLabel(rule,label,elementIndex,args,scope);
/** A lexer rule reference */
lexerRuleRef(rule,label,args,elementIndex,scope);
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope);
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex);
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel);
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description);
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState);
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*
* If a semPredState, don't force lookahead lookup; preds might not
* need.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState);
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a
* rule anything other than 'a' predicts exiting.
*
* If a semPredState, don't force lookahead lookup; preds might not
* need.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState);
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt);
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates);
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState);
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState);
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState);
dfaEdgeSwitch(labels, targetState);
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description);
/** Generate the tables and support code needed for the DFAState object
* argument. Unless there is a semantic predicate (or syn pred, which
* become sem preds), all states should be encoded in the state tables.
* Consequently, cyclicDFAState/cyclicDFAEdge,eotDFAEdge templates are
* not used except for special DFA states that cannot be encoded as
* a transition table.
*/
cyclicDFA(dfa);
/** A special state in a cyclic DFA; special means has a semantic predicate
* or it's a huge set of symbols to check.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState);
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful. Again, this is for special
* states.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates);
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates);
// D F A E X P R E S S I O N S
andPredicates(left,right);
orPredicates(operands);
notPredicate(pred);
evalPredicate(pred,description);
evalSynPredicate(pred,description);
lookaheadTest(atom,k,atomAsInt);
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt);
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt);
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt);
setTest(ranges);
// A T T R I B U T E S
parameterAttributeRef(attr);
parameterSetAttributeRef(attr,expr);
scopeAttributeRef(scope,attr,index,negIndex);
scopeSetAttributeRef(scope,attr,expr,index,negIndex);
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope);
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr);
returnAttributeRef(ruleDescriptor,attr);
returnSetAttributeRef(ruleDescriptor,attr,expr);
/** How to translate $tokenLabel */
tokenLabelRef(label);
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label);
// not sure the next are the right approach; and they are evaluated early;
// they cannot see TREE_PARSER or PARSER attributes for example. :(
tokenLabelPropertyRef_text(scope,attr);
tokenLabelPropertyRef_type(scope,attr);
tokenLabelPropertyRef_line(scope,attr);
tokenLabelPropertyRef_pos(scope,attr);
tokenLabelPropertyRef_channel(scope,attr);
tokenLabelPropertyRef_index(scope,attr);
tokenLabelPropertyRef_tree(scope,attr);
ruleLabelPropertyRef_start(scope,attr);
ruleLabelPropertyRef_stop(scope,attr);
ruleLabelPropertyRef_tree(scope,attr);
ruleLabelPropertyRef_text(scope,attr);
ruleLabelPropertyRef_st(scope,attr);
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label);
lexerRuleLabelPropertyRef_type(scope,attr);
lexerRuleLabelPropertyRef_line(scope,attr);
lexerRuleLabelPropertyRef_pos(scope,attr);
lexerRuleLabelPropertyRef_channel(scope,attr);
lexerRuleLabelPropertyRef_index(scope,attr);
lexerRuleLabelPropertyRef_text(scope,attr);
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr);
rulePropertyRef_stop(scope,attr);
rulePropertyRef_tree(scope,attr);
rulePropertyRef_text(scope,attr);
rulePropertyRef_st(scope,attr);
lexerRulePropertyRef_text(scope,attr);
lexerRulePropertyRef_type(scope,attr);
lexerRulePropertyRef_line(scope,attr);
lexerRulePropertyRef_pos(scope,attr);
/** Undefined, but present for consistency with Token attributes; set to -1 */
lexerRulePropertyRef_index(scope,attr);
lexerRulePropertyRef_channel(scope,attr);
lexerRulePropertyRef_start(scope,attr);
lexerRulePropertyRef_stop(scope,attr);
ruleSetPropertyRef_tree(scope,attr,expr);
ruleSetPropertyRef_st(scope,attr,expr);
/** How to execute an action */
execAction(action);
// M I S C (properties, etc...)
codeFileExtension();
/** Your language needs a header file; e.g., ".h" */
optional headerFileExtension();
true();
false();
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/ 0000755 0001750 0001750 00000000000 11410174107 025705 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/ASTTreeParser.stg 0000644 0001750 0001750 00000022362 11256465232 031067 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> _first_0 = null;
<ASTLabelType> _last = null;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(rewriteMode)>
retval.Tree = (<ASTLabelType>)_first_0;
if ( adaptor.GetParent(retval.Tree)!=null && adaptor.IsNil( adaptor.GetParent(retval.Tree) ) )
retval.Tree = (<ASTLabelType>)adaptor.GetParent(retval.Tree);
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = (<ASTLabelType>)input.LT(1);
{
<ASTLabelType> _save_last_<treeLevel> = _last;
<ASTLabelType> _first_<treeLevel> = null;
<if(!rewriteMode)>
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.GetNilNode();
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
<if(root.el.rule)>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = (<ASTLabelType>) <root.el.label>.Tree;
<else>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1) == Token.DOWN )
{
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
adaptor.AddChild(root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>
}
<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>
}
<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard and auto dup the node/subtree */
wildcard(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.wildcard(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.DupTree(<label>);
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>
<if(!rewriteMode)>
adaptor.AddChild(root_<treeLevel>, <label>.Tree);
<else> <! rewrite mode !>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = (<ASTLabelType>) <label>.Tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( state.backtracking == 0 ) <endif>root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>.Tree, root_<treeLevel>);
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.NextNode())
<else>
stream_<token>.NextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<endif>
retval.Tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
<if(backtracking)>
}
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/ST.stg 0000644 0001750 0001750 00000012030 11256465232 026760 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template subgroup to add template rewrite output
* If debugging, then you'll also get STDbg.stg loaded.
*/
group ST;
@outputFile.imports() ::= <<
<@super.imports()>
using Antlr.StringTemplate;
using Antlr.StringTemplate.Language;
<if(!backtracking)>
using Hashtable = System.Collections.Hashtable;
<endif>
>>
/** Add this to each rule's return value struct */
@returnScope.ruleReturnMembers() ::= <<
private StringTemplate st;
public StringTemplate ST { get { return st; } set { st = value; } }
public override object Template { get { return st; } }
public override string ToString() { return (st == null) ? null : st.ToString(); }
>>
@genericParser.members() ::= <<
<@super.members()>
protected StringTemplateGroup templateLib =
new StringTemplateGroup("<name>Templates", typeof(AngleBracketTemplateLexer));
public StringTemplateGroup TemplateLib
{
get { return this.templateLib; }
set { this.templateLib = value; }
}
/// \<summary> Allows convenient multi-value initialization:
/// "new STAttrMap().Add(...).Add(...)"
/// \</summary>
protected class STAttrMap : Hashtable
{
public STAttrMap Add(string attrName, object value)
{
base.Add(attrName, value);
return this;
}
public STAttrMap Add(string attrName, int value)
{
base.Add(attrName, value);
return this;
}
}
>>
/** x+=rule when output=template */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Template",...)>
>>
rewriteTemplate(alts) ::= <<
// TEMPLATE REWRITE
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
}
<else>
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
<endif>
>>
replaceTextInLine() ::= <<
<if(TREE_PARSER)>
((TokenRewriteStream)input.TokenStream).Replace(
input.TreeAdaptor.GetTokenStartIndex(retval.Start),
input.TreeAdaptor.GetTokenStopIndex(retval.Start),
retval.ST);
<else>
((TokenRewriteStream)input).Replace(
((IToken)retval.Start).TokenIndex,
input.LT(-1).TokenIndex,
retval.ST);
<endif>
>>
rewriteTemplateAlt() ::= <<
// <it.description>
<if(it.pred)>
if (<it.pred>) {
retval.ST = <it.alt>;
}<\n>
<else>
{
retval.ST = <it.alt>;
}<\n>
<endif>
>>
rewriteEmptyTemplate(alts) ::= <<
null;
>>
/** Invoke a template with a set of attribute name/value pairs.
* Set the value of the rule's template *after* having set
* the attributes because the rule's template might be used as
* an attribute to build a bigger template; you get a self-embedded
* template.
*/
rewriteExternalTemplate(name,args) ::= <<
templateLib.GetInstanceOf("<name>"<if(args)>,
new STAttrMap()<args:{a | .Add("<a.name>", <a.value>)}>
<endif>)
>>
/** expr is a string expression that says what template to load */
rewriteIndirectTemplate(expr,args) ::= <<
templateLib.GetInstanceOf(<expr><if(args)>,
new STAttrMap()<args:{a | .Add("<a.name>", <a.value>)}>
<endif>)
>>
/** Invoke an inline template with a set of attribute name/value pairs */
rewriteInlineTemplate(args, template) ::= <<
new StringTemplate(templateLib, "<template>"<if(args)>,
new STAttrMap()<args:{a | .Add("<a.name>", <a.value>)}>
<endif>)
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
<action>
>>
/** An action has %st.attrName=expr; or %{st}.attrName=expr; */
actionSetAttribute(st,attrName,expr) ::= <<
(<st>).SetAttribute("<attrName>",<expr>);
>>
/** Translate %{stringExpr} */
actionStringConstructor(stringExpr) ::= <<
new StringTemplate(templateLib,<stringExpr>)
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/ASTParser.stg 0000644 0001750 0001750 00000015635 11256465232 030254 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
// Conversion of the second argument necessary, but harmless
retval.Tree = (<ASTLabelType>)adaptor.ErrorNode(input, (IToken) retval.Start, input.LT(-1), re);
<! System.Console.WriteLine("<ruleName> returns " + ((CommonTree)retval.Tree).ToStringTree()); !>
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = <createNodeFromToken(...)>;
adaptor.AddChild(root_0, <label>_tree);
<if(backtracking)>
}
<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = <createNodeFromToken(...)>;
root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_0);
<if(backtracking)>
}
<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( state.backtracking == 0 ) <endif>adaptor.AddChild(root_0, <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( state.backtracking == 0 ) <endif>root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<createNodeFromToken(...)>, root_0);})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( state.backtracking == 0 ) <endif>adaptor.AddChild(root_0, <label>.Tree);
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( state.backtracking == 0 ) <endif>root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>.Tree, root_0);
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = (<ASTLabelType>)adaptor.Create(<label>);
adaptor.AddChild(root_0, <label>_tree);
<if(backtracking)>
}
<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>
if ( state.backtracking == 0 )
{
<endif>
<label>_tree = (<ASTLabelType>)adaptor.Create(<label>);
root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_0);
<if(backtracking)>
}
<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
new <hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
(<ASTLabelType>)adaptor.Create(<label>)
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<endif>
retval.Tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
<if(!TREE_PARSER)>
adaptor.SetTokenBoundaries(retval.Tree, (IToken) retval.Start, (IToken) retval.Stop);
<endif>
<if(backtracking)>
}
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/Dbg.stg 0000644 0001750 0001750 00000022555 11256465232 027143 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to normal Java output;
* If ASTs are built, then you'll also get ASTDbg.stg loaded.
*/
group Dbg;
@outputFile.debugPreprocessor() ::= "#define ANTLR_DEBUG"
@outputFile.imports() ::= <<
<@super.imports()>
using Antlr.Runtime.Debug;
using IOException = System.IO.IOException;
>>
@genericParser.members() ::= <<
<if(grammar.grammarIsRoot)>
public static readonly string[] ruleNames = new string[] {
"invalidRule", <grammar.allImportedRules:{rST | "<rST.name>"}; wrap="\n ", separator=", ">
};<\n>
<endif>
<if(grammar.grammarIsRoot)> <! grammar imports other grammar(s) !>
private int ruleLevel = 0;
public int RuleLevel {
get { return ruleLevel; }
}
public void IncRuleLevel() { ruleLevel++; }
public void DecRuleLevel() { ruleLevel--; }
<if(profile)>
<ctorForProfilingRootGrammar()>
<else>
<ctorForRootGrammar()>
<endif>
<ctorForPredefinedListener()>
<else> <! imported grammar !>
public int RuleLevel {
get { return <grammar.delegators:{g| <g:delegateName()>}>.RuleLevel; }
}
public void IncRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.IncRuleLevel(); }
public void DecRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.DecRuleLevel(); }
<ctorForDelegateGrammar()>
<endif>
<if(profile)>
override public bool AlreadyParsedRule(IIntStream input, int ruleIndex)
{
((Profiler)dbg).ExamineRuleMemoization(input, ruleIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
return base.AlreadyParsedRule(input, ruleIndex);
}<\n>
override public void Memoize(IIntStream input,
int ruleIndex,
int ruleStartIndex)
{
((Profiler)dbg).Memoize(input, ruleIndex, ruleStartIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
base.Memoize(input, ruleIndex, ruleStartIndex);
}<\n>
<endif>
protected bool EvalPredicate(bool result, string predicate)
{
dbg.SemanticPredicate(result, predicate);
return result;
}<\n>
>>
ctorForRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
<! Same except we add port number and profile stuff if root grammar !>
public <name>(<inputStreamType> input)
: this(input, DebugEventSocketProxy.DEFAULT_DEBUGGER_PORT, new RecognizerSharedState()) {
}
public <name>(<inputStreamType> input, int port, RecognizerSharedState state)
: base(input, state) {
<createListenerAndHandshake()>
<parserCtorBody()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
>>
@parserCtorBody.initializeCyclicDFAs() ::= <<
InitializeCyclicDFAs(dbg);
>>
ctorForProfilingRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
public <name>(<inputStreamType> input) {
this(input, new Profiler(null), new RecognizerSharedState());
}
public <name>(<inputStreamType> input, IDebugEventListener dbg, RecognizerSharedState state)
: base(input, dbg, state) {
Profiler p = (Profiler)dbg;
p.setParser(this);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}
<\n>
>>
/** Basically we don't want to set any dbg listeners are root will have it. */
ctorForDelegateGrammar() ::= <<
public <name>(<inputStreamType> input, IDebugEventListener dbg, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: base(input, dbg, state) {
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, this, this.state<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
}<\n>
>>
ctorForPredefinedListener() ::= <<
public <name>(<inputStreamType> input, IDebugEventListener dbg)
: <@superClassRef>base(input, dbg, new RecognizerSharedState())<@end> {
<if(profile)>
Profiler p = (Profiler)dbg;
p.setParser(this);
<endif>
<parserCtorBody()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
>>
createListenerAndHandshake() ::= <<
<if(TREE_PARSER)>
DebugEventSocketProxy dbg = new DebugEventSocketProxy(this, port, input.TreeAdaptor);
<else>
DebugEventSocketProxy dbg = new DebugEventSocketProxy(this, port, null);
<endif>
DebugListener = dbg;
try
{
dbg.Handshake();
}
catch (IOException ioe)
{
ReportError(ioe);
}
>>
@genericParser.superClassName() ::= "Debug<@super.superClassName()>"
@rule.preamble() ::= <<
try {
dbg.EnterRule(GrammarFileName, "<ruleName>");
if ( RuleLevel==0 ) {dbg.Commence();}
IncRuleLevel();
dbg.Location(<ruleDescriptor.tree.line>, <ruleDescriptor.tree.column>);<\n>
>>
@lexer.debugInitializeCyclicDFAs() ::= "IDebugEventListener dbg"
@lexer.debugAddition() ::= ", dbg"
@genericParser.debugInitializeCyclicDFAs() ::= "IDebugEventListener dbg"
@genericParser.debugAddition() ::= ", dbg"
@rule.postamble() ::= <<
dbg.Location(<ruleDescriptor.EORNode.line>, <ruleDescriptor.EORNode.column>);<\n>
}
finally {
dbg.ExitRule(GrammarFileName, "<ruleName>");
DecRuleLevel();
if ( RuleLevel==0 ) {dbg.Terminate();}
}<\n>
>>
@synpred.start() ::= "dbg.BeginBacktrack(state.backtracking);"
@synpred.stop() ::= "dbg.EndBacktrack(state.backtracking, success);"
// Common debug event triggers used by region overrides below
enterSubRule() ::=
"try { dbg.EnterSubRule(<decisionNumber>);<\n>"
exitSubRule() ::=
"} finally { dbg.ExitSubRule(<decisionNumber>); }<\n>"
enterDecision() ::=
"try { dbg.EnterDecision(<decisionNumber>);<\n>"
exitDecision() ::=
"} finally { dbg.ExitDecision(<decisionNumber>); }<\n>"
enterAlt(n) ::= "dbg.EnterAlt(<n>);<\n>"
// Region overrides that tell various constructs to add debugging triggers
@block.predecision() ::= "<enterSubRule()><enterDecision()>"
@block.postdecision() ::= "<exitDecision()>"
@block.postbranch() ::= "<exitSubRule()>"
@ruleBlock.predecision() ::= "<enterDecision()>"
@ruleBlock.postdecision() ::= "<exitDecision()>"
@ruleBlockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@blockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@positiveClosureBlock.preloop() ::= "<enterSubRule()>"
@positiveClosureBlock.postloop() ::= "<exitSubRule()>"
@positiveClosureBlock.predecision() ::= "<enterDecision()>"
@positiveClosureBlock.postdecision() ::= "<exitDecision()>"
@positiveClosureBlock.earlyExitException() ::=
"dbg.RecognitionException(eee<decisionNumber>);<\n>"
@closureBlock.preloop() ::= "<enterSubRule()>"
@closureBlock.postloop() ::= "<exitSubRule()>"
@closureBlock.predecision() ::= "<enterDecision()>"
@closureBlock.postdecision() ::= "<exitDecision()>"
@altSwitchCase.prealt() ::= "<enterAlt(n=i)>"
@element.prematch() ::=
"dbg.Location(<it.line>,<it.pos>);"
@matchSet.mismatchedSetException() ::=
"dbg.RecognitionException(mse);"
@dfaState.noViableAltException() ::= "dbg.RecognitionException(nvae_d<decisionNumber>s<stateNumber>);"
@dfaStateSwitch.noViableAltException() ::= "dbg.RecognitionException(nvae_d<decisionNumber>s<stateNumber>);"
dfaDecision(decisionNumber,description) ::= <<
try
{
isCyclicDecision = true;
<super.dfaDecision(...)>
}
catch (NoViableAltException nvae)
{
dbg.RecognitionException(nvae);
throw nvae;
}
>>
@cyclicDFA.dbgCtor() ::= <<
public DFA<dfa.decisionNumber>(BaseRecognizer recognizer, IDebugEventListener dbg) : this(recognizer)
{
this.dbg = dbg;
}
>>
@cyclicDFA.debugMember() ::= <<
IDebugEventListener dbg;
>>
@cyclicDFA.errorMethod() ::= <<
public override void Error(NoViableAltException nvae)
{
dbg.RecognitionException(nvae);
}
>>
/** Force predicate validation to trigger an event */
evalPredicate(pred,description) ::= <<
EvalPredicate(<pred>,"<description>")
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/AST.stg 0000644 0001750 0001750 00000032207 11256465232 027071 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()>
<if(!TREE_PARSER)><! tree parser would already have imported !>
using Antlr.Runtime.Tree;<\n>
<endif>
>>
@genericParser.members() ::= <<
<@super.members()>
<parserMembers()>
>>
/** Add an adaptor property that knows how to build trees */
parserMembers() ::= <<
protected ITreeAdaptor adaptor = new CommonTreeAdaptor();<\n>
public ITreeAdaptor TreeAdaptor
{
get { return this.adaptor; }
set {
this.adaptor = value;
<grammar.directDelegates:{g|<g:delegateName()>.TreeAdaptor = this.adaptor;}>
}
}
>>
@returnScope.ruleReturnMembers() ::= <<
private <ASTLabelType> tree;
override public object Tree
{
get { return tree; }
set { tree = (<ASTLabelType>) value; }
}
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> root_0 = null;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<[ruleDescriptor.tokenLabels,ruleDescriptor.wildcardTreeLabels,
ruleDescriptor.wildcardTreeListLabels]:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{RewriteRule<rewriteElementType>Stream stream_<it> = new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>");}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor,"rule <it>");}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = (<ASTLabelType>)adaptor.GetNilNode();<\n>
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.Add(<label>);<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.Add(<label>);<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule.name>.Add(<label>.Tree);
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule>.Add(<label>.Tree);
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
// wildcard labels: <[referencedWildcardLabels,referencedWildcardListLabels]; separator=", ">
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {<\n>
<endif>
<prevRuleRootRef()>.Tree = root_0;
<rewriteCodeLabels()>
root_0 = (<ASTLabelType>)adaptor.GetNilNode();
<alts:rewriteAlt(); separator="else ">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.Tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
input.ReplaceChildren(adaptor.GetParent(retval.Start),
adaptor.GetChildIndex(retval.Start),
adaptor.GetChildIndex(_last),
retval.Tree);
<endif>
<endif>
<! if parser or rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.Tree = root_0;
<endif>
<if(!rewriteMode)>
<prevRuleRootRef()>.Tree = root_0;
<endif>
<if(backtracking)>
}
<endif>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{RewriteRule<rewriteElementType>Stream stream_<it> = new RewriteRule<rewriteElementType>Stream(adaptor, "token <it>", <it>);};
separator="\n"
>
<referencedTokenListLabels
:{RewriteRule<rewriteElementType>Stream stream_<it> = new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>", list_<it>);};
separator="\n"
>
<referencedWildcardLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",<it>);};
separator="\n"
>
<referencedWildcardListLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",list_<it>);};
separator="\n"
>
<referencedRuleLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor, "rule <it>", <it>!=null ? <it>.Tree : null);};
separator="\n"
>
<referencedRuleListLabels
:{RewriteRuleSubtreeStream stream_<it> = new RewriteRuleSubtreeStream(adaptor, "token <it>", list_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
if ( <referencedElementsDeep:{el | stream_<el>.HasNext()}; separator=" || "> )
{
<alt>
}
<referencedElementsDeep:{el | stream_<el>.Reset();<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
while ( <referencedElements:{el | stream_<el>.HasNext()}; separator=" || "> )
{
<alt>
}
<referencedElements:{el | stream_<el>.Reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if ( !(<referencedElements:{el | stream_<el>.HasNext()}; separator=" || ">) ) {
throw new RewriteEarlyExitException();
}
while ( <referencedElements:{el | stream_<el>.HasNext()}; separator=" || "> )
{
<alt>
}
<referencedElements:{el | stream_<el>.Reset();<\n>}>
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>)
{
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = null;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.GetNilNode();
<root:rewriteElement()>
<children:rewriteElement()>
adaptor.AddChild(root_<enclosingTreeLevel>, root_<treeLevel>);
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
adaptor.AddChild(root_<treeLevel>, <createRewriteNodeFromElement(...)>);<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextNode());<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextNode());<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<createRewriteNodeFromElement(...)>, root_<treeLevel>);<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>);<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.Tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<rule>.NextTree());<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<rule>.NextNode(), root_<treeLevel>);<\n>
>>
rewriteNodeAction(action) ::= <<
adaptor.AddChild(root_<treeLevel>, <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<action>, root_<treeLevel>);<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
rewriteWildcardLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
new <hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
(<ASTLabelType>)adaptor.Create(<tokenType>, <args; separator=", "><if(!args)>"<tokenType>"<endif>)
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.NextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
adaptor.Create(<token>, <args; separator=", ">)
<else>
stream_<token>.NextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/ASTDbg.stg 0000644 0001750 0001750 00000006226 11256465232 027510 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to AST stuff. Dynamic inheritance
* hierarchy is set up as ASTDbg : AST : Dbg : C# by code generator.
*/
group ASTDbg;
parserMembers() ::= <<
protected DebugTreeAdaptor adaptor;
public ITreeAdaptor TreeAdaptor
{
get {
<if(grammar.grammarIsRoot)>
return this.adaptor;
<else>
this.adaptor = (DebugTreeAdaptor)adaptor; // delegator sends dbg adaptor
<endif><\n>
<grammar.directDelegates:{g|<g:delegateName()>.TreeAdaptor = this.adaptor;}>
}
set { this.adaptor = new DebugTreeAdaptor(dbg, value); }
}<\n>
>>
parserCtorBody() ::= <<
<super.parserCtorBody()>
>>
createListenerAndHandshake() ::= <<
DebugEventSocketProxy dbg = new DebugEventSocketProxy(this, port, adaptor);
DebugListener = dbg;
<!
Original line follows, replaced by the next two ifs:
set<inputStreamType>(new Debug<inputStreamType>(input,dbg));
!>
<if(PARSER)>
TokenStream = new DebugTokenStream(input,dbg);<\n>
<endif>
<if(TREE_PARSER)>
TokenStream = new DebugTreeNodeStream(input,dbg);<\n>
<endif>
try {
dbg.Handshake();
} catch (IOException ioe) {
ReportError(ioe);
}
>>
@ctorForRootGrammar.finally() ::= <<
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;
dbg.TreeAdaptor = adap;
>>
@ctorForProfilingRootGrammar.finally() ::=<<
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;
dbg.TreeAdaptor = adap;
>>
@ctorForPredefinedListener.superClassRef() ::= "base(input, dbg)"
@ctorForPredefinedListener.finally() ::=<<
<if(grammar.grammarIsRoot)> <! don't create new adaptor for delegates !>
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;<\n>
<endif>
>>
@rewriteElement.pregen() ::= "dbg.Location(<e.line>,<e.pos>);"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp2/CSharp2.stg 0000644 0001750 0001750 00000130066 11256465232 027706 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007-2008 Johannes Luber
Copyright (c) 2005-2007 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group CSharp2 implements ANTLRCore;
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<@debugPreprocessor()>
// The variable 'variable' is assigned but its value is never used.
#pragma warning disable 168, 219
// Unreachable code detected.
#pragma warning disable 162
<actions.(actionScope).header>
<@imports>
using System;
using Antlr.Runtime;
<if(TREE_PARSER)>
using Antlr.Runtime.Tree;
<endif>
using IList = System.Collections.IList;
using ArrayList = System.Collections.ArrayList;
using Stack = Antlr.Runtime.Collections.StackList;
<if(backtracking)>
using IDictionary = System.Collections.IDictionary;
using Hashtable = System.Collections.Hashtable;
<endif>
<@end>
<if(actions.(actionScope).namespace)>
namespace <actions.(actionScope).namespace>
{
<endif>
<docComment>
<recognizer>
<if(actions.(actionScope).namespace)>
}
<endif>
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="IToken",
filterMode, superClass="Lexer") ::= <<
public partial class <grammar.recognizerName> : <@superClassName><superClass><@end> {
<tokens:{public const int <it.name> = <it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
public <grammar.recognizerName>() <! needed by subclasses !>
{
InitializeCyclicDFAs();
}
public <grammar.recognizerName>(ICharStream input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: this(input, null<grammar.delegators:{g|, <g:delegateName()>}>) {
}
public <grammar.recognizerName>(ICharStream input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: base(input, state) {
InitializeCyclicDFAs(); <! Necessary in C#??? Not removed yet. !>
<if(memoize)>
<if(grammar.grammarIsRoot)>
state.ruleMemo = new Hashtable[<numRules>+1];<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
override public string GrammarFileName
{
get { return "<fileName>";}
}
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<synpreds:{p | <lexerSynpred(p)>}>
<cyclicDFAs:{dfa | protected DFA<dfa.decisionNumber> dfa<dfa.decisionNumber>;}; separator="\n">
private void InitializeCyclicDFAs(<@debugInitializeCyclicDFAs()>)
{
<cyclicDFAs:{dfa | this.dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this<@debugAddition()>);}; separator="\n">
<cyclicDFAs:{dfa | <if(dfa.specialStateSTs)>this.dfa<dfa.decisionNumber>.specialStateTransitionHandler = new DFA.SpecialStateTransitionHandler(DFA<dfa.decisionNumber>_SpecialStateTransition);<endif>}; separator="\n">
}
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
}
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
*
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
override public IToken NextToken()
{
while (true)
{
if ( input.LA(1) == (int)CharStreamConstants.EOF )
{
return Token.EOF_TOKEN;
}
state.token = null;
state.channel = Token.DEFAULT_CHANNEL;
state.tokenStartCharIndex = input.Index();
state.tokenStartCharPositionInLine = input.CharPositionInLine;
state.tokenStartLine = input.Line;
state.text = null;
try
{
int m = input.Mark();
state.backtracking = 1; <! means we won't throw slow exception !>
state.failed = false;
mTokens();
state.backtracking = 0;
<!
mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1.
!>
if ( state.failed )
{
input.Rewind(m);
input.Consume(); <! // advance one char and try again !>
}
else
{
Emit();
return state.token;
}
}
catch (RecognitionException re)
{
// shouldn't happen in backtracking mode, but...
ReportError(re);
Recover(re);
}
}
}
override public void Memoize(IIntStream input, int ruleIndex, int ruleStartIndex)
{
if ( state.backtracking > 1 )
base.Memoize(input, ruleIndex, ruleStartIndex);
}
override public bool AlreadyParsedRule(IIntStream input, int ruleIndex)
{
if ( state.backtracking>1 )
return base.AlreadyParsedRule(input, ruleIndex);
return false;
}
>>
actionGate() ::= "(state.backtracking==0)"
filteringActionGate() ::= "(state.backtracking == 1)"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass, filterMode,
ASTLabelType="object", labelType, members, rewriteElementType) ::= <<
public partial class <grammar.recognizerName> : <@superClassName><superClass><@end>
{
<if(grammar.grammarIsRoot)>
public static readonly string[] tokenNames = new string[]
{
"\<invalid>",
"\<EOR>",
"\<DOWN>",
"\<UP>",
<tokenNames; separator=", \n">
};<\n>
<endif>
<tokens:{public const int <it.name> = <it.type>;}; separator="\n">
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<@members>
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
public <grammar.recognizerName>(<inputStreamType> input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: this(input, new RecognizerSharedState()<grammar.delegators:{g|, <g:delegateName()>}>) {
}
public <grammar.recognizerName>(<inputStreamType> input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>)
: base(input, state) {
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.indirectDelegates:{g | <g:delegateName()> = <g.delegator:delegateName()>.<g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
<@end>
override public string[] TokenNames {
get { return <grammar.composite.rootGrammar.recognizerName>.tokenNames; }
}
override public string GrammarFileName {
get { return "<fileName>"; }
}
<members>
<rules; separator="\n\n">
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
// Delegated rules
<grammar.delegatedRules:{ruleDescriptor|
public <returnType()> <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) // throws RecognitionException
\{
<if(ruleDescriptor.hasReturnValue)>return <endif><ruleDescriptor.grammar:delegateName()>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">);
\}}; separator="\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:{dfa | protected DFA<dfa.decisionNumber> dfa<dfa.decisionNumber>;}; separator="\n">
private void InitializeCyclicDFAs(<@debugInitializeCyclicDFAs()>)
{
<cyclicDFAs:{dfa | this.dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>(this<@debugAddition()>);}; separator="\n">
<cyclicDFAs:{dfa | <if(dfa.specialStateSTs)>this.dfa<dfa.decisionNumber>.specialStateTransitionHandler = new DFA.SpecialStateTransitionHandler(DFA<dfa.decisionNumber>_SpecialStateTransition);<endif>}; separator="\n">
}
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
}
>>
parserCtorBody() ::= <<
<@initializeCyclicDFAs>InitializeCyclicDFAs();<@end>
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = new Hashtable[<length(grammar.allImportedRules)>+1];<\n> <! index from 1..n !>
<endif>
<endif>
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType, superClass="Parser", labelType="IToken", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="ITokenStream", rewriteElementType="Token", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="object", superClass="TreeParser", members={<actions.treeparser.members>}, filterMode) ::= <<
<genericParser(inputStreamType="ITreeNodeStream", rewriteElementType="Node", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start "<ruleName>"
public void <ruleName>_fragment(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) {
<ruleLabelDefs()>
<if(trace)>
TraceIn("<ruleName>_fragment", <ruleDescriptor.index>);
try
{
<block>
}
finally
{
TraceOut("<ruleName>_fragment", <ruleDescriptor.index>);
}
<else>
<block>
<endif>
}
// $ANTLR end "<ruleName>"
>>
synpredDecls(name) ::= <<
SynPredPointer <name>;<\n>
>>
synpred(name) ::= <<
public bool <name>()
{
state.backtracking++;
<@start()>
int start = input.Mark();
try
{
<name>_fragment(); // can never throw exception
}
catch (RecognitionException re)
{
Console.Error.WriteLine("impossible: "+re);
}
bool success = !state.failed;
input.Rewind(start);
<@stop()>
state.backtracking--;
state.failed = false;
return success;
}<\n>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( (state.backtracking > 0) && AlreadyParsedRule(input, <ruleDescriptor.index>) )
{
return <ruleReturnValue()>;
}
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>if (state.failed) return <ruleReturnValue()>;<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if ( state.backtracking > 0 ) {state.failed = true; return <ruleReturnValue()>;}<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
// $ANTLR start "<ruleName>"
// <fileName>:<description>
public <returnType()> <ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) // throws RecognitionException [1]
{
<if(trace)>TraceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try
{
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
catch (RecognitionException re)
{
ReportError(re);
Recover(input,re);
<@setErrorReturnValue()>
}<\n>
<endif>
<endif>
<endif>
finally
{
<if(trace)>TraceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
}
<@postamble()>
return <ruleReturnValue()>;
}
// $ANTLR end "<ruleName>"
>>
catch(decl,action) ::= <<
catch (<e.decl>)
{
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnType()> retval = new <returnType()>();
retval.Start = input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.type> <a.name> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
int <ruleDescriptor.name>_StartIndex = input.Index();
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.Push(new <it>_scope());}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.Push(new <it.name>_scope());}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.Pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.Pop();}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels,
ruleDescriptor.wildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{<labelType> <it.label.text> = null;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels,ruleDescriptor.wildcardTreeListLabels]
:{IList list_<it.label.text> = null;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|<ll:ruleLabelDef(label=it)> <ll.label.text> = null;}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<labelType> <it.label.text> = null;}; separator="\n"
>
<ruleDescriptor.charLabels:{int <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels]
:{IList list_<it.label.text> = null;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
retval.Stop = input.LT(-1);<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( state.backtracking > 0 )
{
Memoize(input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex);
}
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// $ANTLR start "<ruleName>"
public void m<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) // throws RecognitionException [2]
{
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<if(trace)>TraceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
try
{
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
int _type = <ruleName>;
int _channel = DEFAULT_TOKEN_CHANNEL;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
state.type = _type;
state.channel = _channel;
<(ruleDescriptor.actions.after):execAction()>
<endif>
}
finally
{
<if(trace)>TraceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeCleanUp()>
<memoize()>
}
}
// $ANTLR end "<ruleName>"
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
override public void mTokens() // throws RecognitionException
{
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber> = <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber> = <maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int cnt<decisionNumber> = 0;
<decls>
<@preloop()>
do
{
int alt<decisionNumber> = <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
default:
if ( cnt<decisionNumber> >= 1 ) goto loop<decisionNumber>;
<ruleBacktrackFailure()>
EarlyExitException eee<decisionNumber> =
new EarlyExitException(<decisionNumber>, input);
<@earlyExitException()>
throw eee<decisionNumber>;
}
cnt<decisionNumber>++;
} while (true);
loop<decisionNumber>:
; // Stops C# compiler whining that label 'loop<decisionNumber>' has no statements
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
<decls>
<@preloop()>
do
{
int alt<decisionNumber> = <maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
default:
goto loop<decisionNumber>;
}
} while (true);
loop<decisionNumber>:
; // Stops C# compiler whining that label 'loop<decisionNumber>' has no statements
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> :
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
// <fileName>:<description>
{
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
}
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label>=(<labelType>)<endif>Match(input,<token>,FOLLOW_<token>_in_<ruleName><elementIndex>); <checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label> == null) list_<label> = new ArrayList();
list_<label>.Add(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
Match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
MatchRange(<a>,<b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= input.LA(1);<\n>
<else>
<label> = (<labelType>)input.LT(1);<\n>
<endif>
<endif>
if ( <s> )
{
input.Consume();
<postmatchCode>
<if(!LEXER)>
state.errorRecovery = false;
<endif>
<if(backtracking)>state.failed = false;<endif>
}
else
{
<ruleBacktrackFailure()>
MismatchedSetException mse = new MismatchedSetException(null,input);
<@mismatchedSetException()>
<if(LEXER)>
Recover(mse);
throw mse;
<else>
throw mse;
<! use following code to make it recover inline; remove throw mse;
RecoverFromMismatchedSet(input,mse,FOLLOW_set_in_<ruleName><elementIndex>);
!>
<endif>
}<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
int <label>Start = CharIndex;
Match(<string>); <checkRuleBacktrackFailure()>
<label> = new CommonToken(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start, CharIndex-1);
<else>
Match(<string>); <checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label> = (<labelType>)input.LT(1);<\n>
<endif>
MatchAny(input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
MatchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
PushFollow(FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)>
<label> = <if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
<else>
<if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
<endif>
state.followingStackPointer--;
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference.
*
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
int <label>Start<elementIndex> = CharIndex;
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> = new CommonToken(input, Token.INVALID_TOKEN_TYPE, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, CharIndex-1);
<else>
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
int <label>Start<elementIndex> = CharIndex;
Match(EOF); <checkRuleBacktrackFailure()>
<labelType> <label> = new CommonToken(input, EOF, Token.DEFAULT_CHANNEL, <label>Start<elementIndex>, CharIndex-1);
<else>
Match(EOF); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1) == Token.DOWN )
{
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
}
<else>
Match(input, Token.DOWN, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, Token.UP, null); <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) )
{
<ruleBacktrackFailure()>
throw new FailedPredicateException(input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
else
{
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae_d<decisionNumber>s<stateNumber> =
new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae_d<decisionNumber>s<stateNumber>;<\n>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber>=<eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else
{
alt<decisionNumber> = <eotPredictsAlt>;
}<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber> = <alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>) <endif>)
{
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;
break;
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae_d<decisionNumber>s<stateNumber> =
new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae_d<decisionNumber>s<stateNumber>;<\n>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber> = <eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
{
<targetState>
}
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = dfa<decisionNumber>.Predict(input);
>>
/* Dump DFA tables.
*/
cyclicDFA(dfa) ::= <<
const string DFA<dfa.decisionNumber>_eotS =
"<dfa.javaCompressedEOT; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_eofS =
"<dfa.javaCompressedEOF; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_minS =
"<dfa.javaCompressedMin; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_maxS =
"<dfa.javaCompressedMax; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_acceptS =
"<dfa.javaCompressedAccept; wrap="\"+\n \"">";
const string DFA<dfa.decisionNumber>_specialS =
"<dfa.javaCompressedSpecial; wrap="\"+\n \"">}>";
static readonly string[] DFA<dfa.decisionNumber>_transitionS = {
<dfa.javaCompressedTransition:{s|"<s; wrap="\"+\n\"">"}; separator=",\n">
};
static readonly short[] DFA<dfa.decisionNumber>_eot = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_eotS);
static readonly short[] DFA<dfa.decisionNumber>_eof = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_eofS);
static readonly char[] DFA<dfa.decisionNumber>_min = DFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_minS);
static readonly char[] DFA<dfa.decisionNumber>_max = DFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_maxS);
static readonly short[] DFA<dfa.decisionNumber>_accept = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_acceptS);
static readonly short[] DFA<dfa.decisionNumber>_special = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_specialS);
static readonly short[][] DFA<dfa.decisionNumber>_transition = DFA.UnpackEncodedStringArray(DFA<dfa.decisionNumber>_transitionS);
protected class DFA<dfa.decisionNumber> : DFA
{
<@debugMember()>
public DFA<dfa.decisionNumber>(BaseRecognizer recognizer)
{
this.recognizer = recognizer;
this.decisionNumber = <dfa.decisionNumber>;
this.eot = DFA<dfa.decisionNumber>_eot;
this.eof = DFA<dfa.decisionNumber>_eof;
this.min = DFA<dfa.decisionNumber>_min;
this.max = DFA<dfa.decisionNumber>_max;
this.accept = DFA<dfa.decisionNumber>_accept;
this.special = DFA<dfa.decisionNumber>_special;
this.transition = DFA<dfa.decisionNumber>_transition;
}
<@dbgCtor()>
override public string Description
{
get { return "<dfa.description>"; }
}
<@errorMethod()>
}<\n>
<if(dfa.specialStateSTs)>
protected internal int DFA<dfa.decisionNumber>_SpecialStateTransition(DFA dfa, int s, IIntStream _input) //throws NoViableAltException
{
<if(LEXER)>
IIntStream input = _input;
<endif>
<if(PARSER)>
ITokenStream input = (ITokenStream)_input;
<endif>
<if(TREE_PARSER)>
ITreeNodeStream input = (ITreeNodeStream)_input;
<endif>
int _s = s;
switch ( s )
{
<dfa.specialStateSTs:{state |
case <i0> : <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
<if(backtracking)>
if (state.backtracking > 0) {state.failed = true; return -1;}<\n>
<endif>
NoViableAltException nvae<dfa.decisionNumber> =
new NoViableAltException(dfa.Description, <dfa.decisionNumber>, _s, input);
dfa.Error(nvae<dfa.decisionNumber>);
throw nvae<dfa.decisionNumber>;
}<\n>
<endif>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
int index<decisionNumber>_<stateNumber> = input.Index();
input.Rewind();<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
<if(semPredState)> <! return input cursor to state before we rewound !>
input.Seek(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s >= 0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>) <endif>) { s = <targetStateNumber>; }<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left> && <right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | || <o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber> == <atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "input.LA(<k>) == <atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber> \>= <lower> && LA<decisionNumber>_<stateNumber> \<= <upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(input.LA(<k>) \>= <lower> && input.LA(<k>) \<= <upper>)"
setTest(ranges) ::= "<ranges; separator=\" || \">"
// A T T R I B U T E S
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected class <scope.name>_scope
{
<scope.attributes:{protected internal <it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
ruleAttributeScope(scope) ::= <<
<if(scope.attributes)>
protected class <scope.name>_scope
{
<scope.attributes:{protected internal <it.decl>;}; separator="\n">
}
protected Stack <scope.name>_stack = new Stack();<\n>
<endif>
>>
returnStructName() ::= "<it.name>_return"
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.grammar.recognizerName>.<ruleDescriptor:returnStructName()>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Generate the C# type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.grammar.recognizerName>.<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
default(<typeName>)
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<ruleLabelType(referencedRule=label.referencedRule)> <label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
public class <ruleDescriptor:returnStructName()> : <if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope
{
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
};
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> = <expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack[<scope>_stack.Count-<negIndex>-1]).<attr.name>
<else>
<if(index)>
((<scope>_scope)<scope>_stack[<index>]).<attr.name>
<else>
((<scope>_scope)<scope>_stack.Peek()).<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack[<scope>_stack.Count-<negIndex>-1]).<attr.name> = <expr>;
<else>
<if(index)>
((<scope>_scope)<scope>_stack[<index>]).<attr.name> = <expr>;
<else>
((<scope>_scope)<scope>_stack.Peek()).<attr.name> = <expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
((<scope> != null) ? <scope>.<attr.name> : <initValue(attr.type)>)
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name> = <expr>;
<else>
<attr.name> = <expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "((<scope> != null) ? <scope>.Text : null)"
tokenLabelPropertyRef_type(scope,attr) ::= "((<scope> != null) ? <scope>.Type : 0)"
tokenLabelPropertyRef_line(scope,attr) ::= "((<scope> != null) ? <scope>.Line : 0)"
tokenLabelPropertyRef_pos(scope,attr) ::= "((<scope> != null) ? <scope>.CharPositionInLine : 0)"
tokenLabelPropertyRef_channel(scope,attr) ::= "((<scope> != null) ? <scope>.Channel : 0)"
tokenLabelPropertyRef_index(scope,attr) ::= "((<scope> != null) ? <scope>.TokenIndex : 0)"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
tokenLabelPropertyRef_int(scope,attr) ::= "(<scope>!=null?int.Parse(<scope>.Text):0)"
ruleLabelPropertyRef_start(scope,attr) ::= "((<scope> != null) ? ((<labelType>)<scope>.Start) : null)"
ruleLabelPropertyRef_stop(scope,attr) ::= "((<scope> != null) ? ((<labelType>)<scope>.Stop) : null)"
ruleLabelPropertyRef_tree(scope,attr) ::= "((<scope> != null) ? ((<ASTLabelType>)<scope>.Tree) : null)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
((<scope> != null) ? input.TokenStream.ToString(
input.TreeAdaptor.GetTokenStartIndex(<scope>.Start),
input.TreeAdaptor.GetTokenStopIndex(<scope>.Start)) : null)
<else>
((<scope> != null) ? input.ToString((IToken)(<scope>.Start),(IToken)(<scope>.Stop)) : null)
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "((<scope> != null) ? <scope>.ST : null)"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "((<scope> != null) ? <scope>.Type : 0)"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "((<scope> != null) ? <scope>.Line : 0)"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "((<scope> != null) ? <scope>.CharPositionInLine : -1)"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "((<scope> != null) ? <scope>.Channel : 0)"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "((<scope> != null) ? <scope>.TokenIndex : 0)"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "((<scope> != null) ? <scope>.Text : null)"
lexerRuleLabelPropertyRef_int(scope,attr) ::= "(<scope>!=null?int.Parse(<scope>.Text):0)"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "((<labelType>)retval.Start)"
rulePropertyRef_stop(scope,attr) ::= "((<labelType>)retval.Stop)"
rulePropertyRef_tree(scope,attr) ::= "((<ASTLabelType>)retval.Tree)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
input.TokenStream.ToString(
input.TreeAdaptor.GetTokenStartIndex(retval.Start),
input.TreeAdaptor.GetTokenStopIndex(retval.Start) )
<else>
input.ToString((IToken)retval.Start,input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.ST"
lexerRulePropertyRef_text(scope,attr) ::= "Text"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "state.tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "state.tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "_channel"
lexerRulePropertyRef_start(scope,attr) ::= "state.tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(CharIndex-1)"
lexerRulePropertyRef_int(scope,attr) ::= "int.Parse(<scope>.Text)"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.Tree = <expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.ST = <expr>;"
/** How to execute an action (only when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<action>
}
<else>
<action>
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
public static readonly BitSet <name> = new BitSet(new ulong[]{<words64:{<it>UL};separator=",">});<\n>
>>
codeFileExtension() ::= ".cs"
true() ::= "true"
false() ::= "false"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/ 0000755 0001750 0001750 00000000000 11410174107 025724 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/ASTTreeParser.stg 0000644 0001750 0001750 00000020255 11256465234 031107 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
finishedBacktracking(block) ::= <<
<if(backtracking)>
if <actions.(actionScope).synpredgate>:
<block>
<else>
<block>
<endif>
>>
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
_first_0 = None
_last = None<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<finishedBacktracking({
<if(rewriteMode)>
retval.tree = _first_0
if self._adaptor.getParent(retval.tree) is not None and self._adaptor.isNil(self._adaptor.getParent(retval.tree)):
retval.tree = self._adaptor.getParent(retval.tree)
<endif>
})>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = self.input.LT(1)
_save_last_<treeLevel> = _last
_first_<treeLevel> = None
<if(!rewriteMode)>
root_<treeLevel> = self._adaptor.nil()<\n>
<endif>
<root:element()>
<if(rewriteMode)>
<finishedBacktracking({
<if(root.el.rule)>
if _first_<enclosingTreeLevel> is None:
_first_<enclosingTreeLevel> = <root.el.label>.tree<\n>
<else>
if _first_<enclosingTreeLevel> is None:
_first_<enclosingTreeLevel> = <root.el.label><\n>
<endif>
})>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if self.input.LA(1) == DOWN:
self.match(self.input, DOWN, None)
<children:element()>
self.match(self.input, UP, None)
<else>
self.match(self.input, DOWN, None)
<children:element()>
self.match(self.input, UP, None)<\n>
<endif>
<if(!rewriteMode)>
self._adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>)<\n>
<endif>
_last = _save_last_<treeLevel>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = self.input.LT(1)
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = self.input.LT(1)
<super.tokenRef(...)>
<if(!rewriteMode)>
<finishedBacktracking({
<if(hetero)>
<label>_tree = <hetero>(<label>)
<else>
<label>_tree = self._adaptor.dupNode(<label>)
<endif><\n>
self._adaptor.addChild(root_<treeLevel>, <label>_tree)
})>
<else> <! rewrite mode !>
<finishedBacktracking({
if _first_<treeLevel> is None:
_first_<treeLevel> = <label><\n>
})>
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = self.input.LT(1)
<super.tokenRef(...)>
<if(!rewriteMode)>
<finishedBacktracking({
<if(hetero)>
<label>_tree = <hetero>(<label>)
<else>
<label>_tree = self._adaptor.dupNode(<label>)
<endif><\n>
root_<treeLevel> = self._adaptor.becomeRoot(<label>_tree, root_<treeLevel>)
})>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard and auto dup the node/subtree */
wildcard(token,label,elementIndex,hetero) ::= <<
_last = self.input.LT(1)
<super.wildcard(...)>
<if(!rewriteMode)>
<finishedBacktracking({
<label>_tree = self._adaptor.dupTree(<label>)
self._adaptor.addChild(root_<treeLevel>, <label>_tree)
})>
<else> <! rewrite mode !>
<finishedBacktracking({
if _first_<treeLevel> is None:
_first_<treeLevel> = <label>
})>
<endif>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = self.input.LT(1)
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<finishedBacktracking({
<if(hetero)>
<label>_tree = <hetero>(<label>)
<else>
<label>_tree = self._adaptor.dupNode(<label>)
<endif><\n>
self._adaptor.addChild(root_<treeLevel>, <label>_tree)
})>
<endif>
})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = self.input.LT(1)
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<finishedBacktracking({
<if(hetero)>
<label>_tree = <hetero>(<label>)
<else>
<label>_tree = self._adaptor.dupNode(<label>)
<endif><\n>
root_<treeLevel> = self._adaptor.becomeRoot(<label>_tree, root_<treeLevel>)
})>
<endif>
})>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = self.input.LT(1)
<super.ruleRef(...)>
<finishedBacktracking({
<if(!rewriteMode)>
self._adaptor.addChild(root_<treeLevel>, <label>.tree)
<else> <! rewrite mode !>
if _first_<treeLevel> is None:
_first_<treeLevel> = <label>.tree<\n>
<endif>
})>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".tree",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = self.input.LT(1)
<super.ruleRef(...)>
<if(!rewriteMode)>
<finishedBacktracking({
root_<treeLevel> = self._adaptor.becomeRoot(<label>.tree, root_<treeLevel>)
})>
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".tree",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = self.input.LT(1)
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = self.input.LT(1)
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = self.input.LT(1)
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = self.input.LT(1)
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
<hetero>(stream_<token>.nextNode())
<else>
stream_<token>.nextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<finishedBacktracking({
retval.tree = self._adaptor.rulePostProcessing(root_0)
})>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/ST.stg 0000644 0001750 0001750 00000011533 11256465234 027010 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template subgroup to add template rewrite output
* If debugging, then you'll also get STDbg.stg loaded.
*/
group ST;
@outputFile.imports() ::= <<
<@super.imports()>
import stringtemplate3
>>
/** Add this to each rule's return value struct */
@returnScope.ruleReturnInit() ::= <<
self.st = None
>>
@returnScope.ruleReturnMembers() ::= <<
def getTemplate(self):
return self.st
def toString(self):
if self.st is not None:
return self.st.toString()
return None
__str__ = toString
>>
@genericParser.init() ::= <<
<@super.init()>
self.templateLib = stringtemplate3.StringTemplateGroup(
'<name>Templates', lexer='angle-bracket'
)
>>
@genericParser.members() ::= <<
<@super.members()>
def setTemplateLib(self, templateLib):
self.templateLib = templateLib
def getTemplateLib(self):
return self.templateLib
>>
/** x+=rule when output=template */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".st",...)>
>>
rewriteTemplate(alts) ::= <<
# TEMPLATE REWRITE
<if(backtracking)>
if <actions.(actionScope).synpredgate>:
<first(alts):rewriteTemplateAltFirst()>
<rest(alts):{el<rewriteTemplateAlt()>}>
<if(rewriteMode)><replaceTextInLine()><endif>
<else>
<first(alts):rewriteTemplateAltFirst()>
<rest(alts):{el<rewriteTemplateAlt()>}>
<if(rewriteMode)><replaceTextInLine()><endif>
<endif>
>>
replaceTextInLine() ::= <<
<if(TREE_PARSER)>
self.input.getTokenStream().replace(
self.input.getTreeAdaptor().getTokenStartIndex(retval.start),
self.input.getTreeAdaptor().getTokenStopIndex(retval.start),
retval.st
)
<else>
self.input.replace(
retval.start.getTokenIndex(),
self.input.LT(-1).getTokenIndex(),
retval.st
)
<endif>
>>
rewriteTemplateAltFirst() ::= <<
<if(it.pred)>
if <it.pred>:
# <it.description>
retval.st = <it.alt>
<\n>
<else>
# <it.description>
retval.st = <it.alt>
<\n>
<endif>
>>
rewriteTemplateAlt() ::= <<
<if(it.pred)>
if <it.pred>:
# <it.description>
retval.st = <it.alt>
<\n>
<else>
se:
# <it.description>
retval.st = <it.alt>
<\n>
<endif>
>>
rewriteEmptyTemplate(alts) ::= <<
None
>>
/** Invoke a template with a set of attribute name/value pairs.
* Set the value of the rule's template *after* having set
* the attributes because the rule's template might be used as
* an attribute to build a bigger template; you get a self-embedded
* template.
*/
rewriteExternalTemplate(name,args) ::= <<
self.templateLib.getInstanceOf("<name>"<if(args)>,
attributes={<args:{a | "<a.name>": <a.value>}; separator=", ">}<endif>)
>>
/** expr is a string expression that says what template to load */
rewriteIndirectTemplate(expr,args) ::= <<
self.templateLib.getInstanceOf(<expr><if(args)>,
attributes={<args:{a | "<a.name>": <a.value>}; separator=", ">}<endif>)
>>
/** Invoke an inline template with a set of attribute name/value pairs */
rewriteInlineTemplate(args, template) ::= <<
stringtemplate3.StringTemplate(
"<template>",
group=self.templateLib<if(args)>,
attributes={<args:{a | "<a.name>": <a.value>}; separator=", ">}
<endif>
)
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
<action>
>>
/** An action has %st.attrName=expr; or %{st}.attrName=expr; */
actionSetAttribute(st,attrName,expr) ::= <<
(<st>)["<attrName>"] = <expr>
>>
/** Translate %{stringExpr} */
actionStringConstructor(stringExpr) ::= <<
stringtemplate3.StringTemplate(<stringExpr>, group=self.templateLib)
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/Python.stg 0000644 0001750 0001750 00000122000 11256465234 027733 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* in sync with Java/Java.stg revision 107 */
group Python implements ANTLRCore;
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
# $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<@imports>
import sys
from antlr3 import *
<if(TREE_PARSER)>
from antlr3.tree import *<\n>
<endif>
from antlr3.compat import set, frozenset
<@end>
<actions.(actionScope).header>
<! <docComment> !>
# for convenience in actions
HIDDEN = BaseRecognizer.HIDDEN
# token types
<tokens:{<it.name>=<it.type>}; separator="\n">
<recognizer>
<if(actions.(actionScope).main)>
<actions.(actionScope).main>
<else>
def main(argv, stdin=sys.stdin, stdout=sys.stdout, stderr=sys.stderr):
<if(LEXER)>
from antlr3.main import LexerMain
main = LexerMain(<recognizer.name>)<\n>
<endif>
<if(PARSER)>
from antlr3.main import ParserMain
main = ParserMain("<recognizer.grammar.name>Lexer", <recognizer.name>)<\n>
<endif>
<if(TREE_PARSER)>
from antlr3.main import WalkerMain
main = WalkerMain(<recognizer.name>)<\n>
<endif>
main.stdin = stdin
main.stdout = stdout
main.stderr = stderr
main.execute(argv)<\n>
<endif>
<actions.(actionScope).footer>
if __name__ == '__main__':
main(sys.argv)
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="Token",
filterMode, superClass="Lexer") ::= <<
<grammar.directDelegates:
{g|from <g.recognizerName> import <g.recognizerName>}; separator="\n">
class <grammar.recognizerName>(<@superClassName><superClass><@end>):
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
grammarFileName = "<fileName>"
antlr_version = version_str_to_tuple("<ANTLRVersion>")
antlr_version_str = "<ANTLRVersion>"
def __init__(self<grammar.delegators:{g|, <g:delegateName()>}>, input=None, state=None):
if state is None:
state = RecognizerSharedState()
super(<grammar.recognizerName>, self).__init__(input, state)
<if(memoize)>
<if(grammar.grammarIsRoot)>
self._state.ruleMemo = {}
<endif>
<endif>
<grammar.directDelegates:
{g|self.<g:delegateName()> = <g.recognizerName>(<trunc(g.delegators):{p|<p:delegateName()>, }>self, input, state)}; separator="\n">
<grammar.delegators:
{g|self.<g:delegateName()> = <g:delegateName()>}; separator="\n">
<last(grammar.delegators):
{g|self.gParent = <g:delegateName()>}; separator="\n">
<cyclicDFAs:{dfa | <cyclicDFAInit(dfa)>}; separator="\n">
<actions.lexer.init>
<actions.lexer.members>
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<synpreds:{p | <lexerSynpred(p)>}>
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
def nextToken(self):
while True:
if self.input.LA(1) == EOF:
return EOF_TOKEN
self._state.token = None
self._state.channel = DEFAULT_CHANNEL
self._state.tokenStartCharIndex = self.input.index()
self._state.tokenStartCharPositionInLine = self.input.charPositionInLine
self._state.tokenStartLine = self.input.line
self._state._text = None
try:
m = self.input.mark()
try:
# means we won't throw slow exception
self._state.backtracking = 1
try:
self.mTokens()
finally:
self._state.backtracking = 0
except BacktrackingFailed:
# mTokens backtracks with synpred at backtracking==2
# and we set the synpredgate to allow actions at level 1.
self.input.rewind(m)
self.input.consume() # advance one char and try again
else:
self.emit()
return self._state.token
except RecognitionException, re:
# shouldn't happen in backtracking mode, but...
self.reportError(re)
self.recover(re)
def memoize(self, input, ruleIndex, ruleStartIndex, success):
if self._state.backtracking > 1:
# is Lexer always superclass?
<@superClassName><superClass><@end>.memoize(self, input, ruleIndex, ruleStartIndex, success)
def alreadyParsedRule(self, input, ruleIndex):
if self._state.backtracking > 1:
return <@superClassName><superClass><@end>.alreadyParsedRule(self, input, ruleIndex)
return False
>>
actionGate() ::= "self._state.backtracking == 0"
filteringActionGate() ::= "self._state.backtracking == 1"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass, filterMode,
ASTLabelType="Object", labelType, members, rewriteElementType,
init) ::= <<
<if(grammar.grammarIsRoot)>
# token names
tokenNames = [
"\<invalid>", "\<EOR>", "\<DOWN>", "\<UP>",
<tokenNames; wrap, separator=", ">
]<\n>
<else>
from <grammar.composite.rootGrammar.recognizerName> import tokenNames<\n>
<endif>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeClass(scope=it)><endif>}>
<grammar.directDelegates:
{g|from <g.recognizerName> import <g.recognizerName>}; separator="\n">
<rules:{<ruleAttributeScopeClass(scope=it.ruleDescriptor.ruleScope)>}>
class <grammar.recognizerName>(<@superClassName><superClass><@end>):
grammarFileName = "<fileName>"
antlr_version = version_str_to_tuple("<ANTLRVersion>")
antlr_version_str = "<ANTLRVersion>"
tokenNames = tokenNames
def __init__(self<grammar.delegators:{g|, <g:delegateName()>}>, input, state=None, *args, **kwargs):
if state is None:
state = RecognizerSharedState()
<@args()>
super(<grammar.recognizerName>, self).__init__(input, state, *args, **kwargs)
<if(memoize)>
<if(grammar.grammarIsRoot)>
self._state.ruleMemo = {}
<endif>
<endif>
<cyclicDFAs:{dfa | <cyclicDFAInit(dfa)>}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeStack(scope=it)><endif>}>
<rules:{<ruleAttributeScopeStack(scope=it.ruleDescriptor.ruleScope)>}>
<init>
<grammar.delegators:
{g|self.<g:delegateName()> = <g:delegateName()>}; separator="\n">
<grammar.directDelegates:
{g|self.<g:delegateName()> = <g.recognizerName>(<trunc(g.delegators):{p|<p:delegateName()>, }>self, input, state)}; separator="\n">
<!grammar.directDelegates:
{g|self.<g:delegateName()> = <g.recognizerName>(self<grammar.delegators:{g|, <g:delegateName()>}>, input, state)}; separator="\n"!>
<last(grammar.delegators):
{g|self.gParent = self.<g:delegateName()>}; separator="\n">
<@init>
<@end>
<@members>
<@end>
<members>
<rules; separator="\n\n">
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
# Delegated rules
<grammar.delegatedRules:{ruleDescriptor| <delegateRule(ruleDescriptor)> }; separator="\n">
<synpreds:{p | <synpred(p)>}>
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
<bitsets:{FOLLOW_<it.name>_in_<it.inName><it.tokenIndex> = frozenset([<it.tokenTypes:{<it>};separator=", ">])<\n>}>
>>
delegateRule(ruleDescriptor) ::= <<
def <ruleDescriptor.name>(self, <ruleDescriptor.parameterScope:parameterScope(scope=it)>):
<\ > <if(ruleDescriptor.hasReturnValue)>return <endif>self.<ruleDescriptor.grammar:delegateName()>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">)
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType="object", superClass="Parser", labelType="Token", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="TokenStream", rewriteElementType="Token", init={<actions.parser.init>}, ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="Object", superClass="TreeParser", members={<actions.treeparser.members>}, filterMode) ::= <<
<genericParser(inputStreamType="TreeNodeStream", rewriteElementType="Node", init={<actions.treeparser.init>}, ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
# $ANTLR start "<ruleName>"
def <ruleName>_fragment(self, <ruleDescriptor.parameterScope:parameterScope(scope=it)>):
<if(trace)>
self.traceIn("<ruleName>_fragment", <ruleDescriptor.index>)
try:
<block>
finally:
self.traceOut("<ruleName>_fragment", <ruleDescriptor.index>)
<else>
<block>
<endif>
# $ANTLR end "<ruleName>"
>>
synpred(name) ::= <<
def <name>(self):
self._state.backtracking += 1
<@start()>
start = self.input.mark()
try:
self.<name>_fragment()
except BacktrackingFailed:
success = False
else:
success = True
self.input.rewind(start)
<@stop()>
self._state.backtracking -= 1
return success
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if self._state.backtracking > 0 and self.alreadyParsedRule(self.input, <ruleDescriptor.index>):
# for cached failed rules, alreadyParsedRule will raise an exception
success = True
return <ruleReturnValue()>
<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>
if self._state.backtracking > 0:
raise BacktrackingFailed
<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<returnScope(scope=ruleDescriptor.returnScope)>
# $ANTLR start "<ruleName>"
# <fileName>:<description>
<ruleDescriptor.actions.decorate>
def <ruleName>(self, <ruleDescriptor.parameterScope:parameterScope(scope=it)>):
<if(trace)>
self.traceIn("<ruleName>", <ruleDescriptor.index>)<\n>
<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
<@body><ruleBody()><@end>
<@postamble()>
return <ruleReturnValue()>
# $ANTLR end "<ruleName>"
>>
ruleBody() ::= <<
<if(memoize)>
<if(backtracking)>
success = False<\n>
<endif>
<endif>
try:
try:
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
<if(memoize)>
<if(backtracking)>
success = True<\n>
<endif>
<endif>
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
except RecognitionException, re:
self.reportError(re)
self.recover(self.input, re)
<@setErrorReturnValue()>
<endif>
<else>
finally:
pass
<endif>
<endif>
finally:
<if(trace)>
self.traceOut("<ruleName>", <ruleDescriptor.index>)<\n>
<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
pass
>>
catch(decl,action) ::= <<
except <e.decl>:
<e.action>
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval = self.<ruleDescriptor.name>_return()
retval.start = self.input.LT(1)<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.name> = <if(a.initValue)><a.initValue><else>None<endif>
}>
<endif>
<if(memoize)>
<ruleDescriptor.name>_StartIndex = self.input.index()
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{self.<it>_stack.append(<it>_scope())}; separator="\n">
<ruleDescriptor.ruleScope:{self.<it.name>_stack.append(<it.name>_scope())}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{self.<it>_stack.pop()}; separator="\n">
<ruleDescriptor.ruleScope:{self.<it.name>_stack.pop()}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels,
ruleDescriptor.wildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{<it.label.text> = None}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels,
ruleDescriptor.wildcardTreeListLabels]
:{list_<it.label.text> = None}; separator="\n"
>
<[ruleDescriptor.ruleLabels,ruleDescriptor.ruleListLabels]
:ruleLabelDef(label=it); separator="\n"
>
<ruleDescriptor.ruleListLabels:{<it.label.text> = None}; separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<it.label.text> = None}; separator="\n"
>
<ruleDescriptor.charLabels:{<it.label.text> = None}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels]
:{list_<it.label.text> = None}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
retval.stop = self.input.LT(-1)<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if self._state.backtracking > 0:
self.memoize(self.input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex, success)
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
# $ANTLR start "<ruleName>"
def m<ruleName>(self, <ruleDescriptor.parameterScope:parameterScope(scope=it)>):
<if(trace)>
self.traceIn("<ruleName>", <ruleDescriptor.index>)<\n>
<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<if(memoize)>
<if(backtracking)>
success = False<\n>
<endif>
<endif>
try:
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
_type = <ruleName>
_channel = DEFAULT_CHANNEL
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
self._state.type = _type
self._state.channel = _channel
<(ruleDescriptor.actions.after):execAction()>
<endif>
<if(memoize)>
<if(backtracking)>
success = True<\n>
<endif>
<endif>
finally:
<if(trace)>
self.traceOut("<ruleName>", <ruleDescriptor.index>)<\n>
<endif>
<ruleScopeCleanUp()>
<memoize()>
pass
# $ANTLR end "<ruleName>"
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
def mTokens(self):
<block><\n>
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
alt<decisionNumber> = <maxAlt>
<decls>
<@body><blockBody()><@end>
>>
blockBody() ::= <<
<@predecision()>
<@decision><decision><@end>
<@postdecision()>
<@prebranch()>
<alts:altSwitchCase(); separator="\nel">
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
alt<decisionNumber> = <maxAlt>
<decls>
<@predecision()>
<@decision><decision><@end>
<@postdecision()>
<alts:altSwitchCase(); separator="\nel">
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
# <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
# <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
cnt<decisionNumber> = 0
<decls>
<@preloop()>
<@loopBody>
<positiveClosureBlockLoop()>
<@end>
<@postloop()>
>>
positiveClosureBlockLoop() ::= <<
while True: #loop<decisionNumber>
alt<decisionNumber> = <maxAlt>
<@predecision()>
<@decisionBody><decision><@end>
<@postdecision()>
<alts:altSwitchCase(); separator="\nel">
else:
if cnt<decisionNumber> >= 1:
break #loop<decisionNumber>
<ruleBacktrackFailure()>
eee = EarlyExitException(<decisionNumber>, self.input)
<@earlyExitException()>
raise eee
cnt<decisionNumber> += 1
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
# <fileName>:<description>
<decls>
<@preloop()>
<@loopBody>
<closureBlockLoop()>
<@end>
<@postloop()>
>>
closureBlockLoop() ::= <<
while True: #loop<decisionNumber>
alt<decisionNumber> = <maxAlt>
<@predecision()>
<@decisionBody><decision><@end>
<@postdecision()>
<alts:altSwitchCase(); separator="\nel">
else:
break #loop<decisionNumber>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
if alt<decisionNumber> == <i>:
<@prealt()>
<it>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt, treeLevel,rew) ::= <<
# <fileName>:<description>
pass <! so empty alternatives are a valid block !>
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label>=<endif>self.match(self.input, <token>, self.FOLLOW_<token>_in_<ruleName><elementIndex>)
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label, elem) ::= <<
if list_<label> is None:
list_<label> = []
list_<label>.append(<elem>)<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = self.input.LA(1)<\n>
<endif>
self.match(<char>)
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = self.input.LA(1)<\n>
<endif>
self.matchRange(<a>, <b>)
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<label> = self.input.LT(1)<\n>
<endif>
if <s>:
self.input.consume()
<postmatchCode>
<if(!LEXER)>
self._state.errorRecovery = False<\n>
<endif>
else:
<ruleBacktrackFailure()>
mse = MismatchedSetException(None, self.input)
<@mismatchedSetException()>
<if(LEXER)>
self.recover(mse)
raise mse
<else>
raise mse
<! use following code to make it recover inline; remove throw mse;
self.recoverFromMismatchedSet(
self.input, mse, self.FOLLOW_set_in_<ruleName><elementIndex>
)
!>
<endif>
<\n>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
<label>Start = self.getCharIndex()
self.match(<string>)
<label> = CommonToken(input=self.input, type=INVALID_TOKEN_TYPE, channel=DEFAULT_CHANNEL, start=<label>Start, stop=self.getCharIndex()-1)
<else>
self.match(<string>)
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label> = self.input.LT(1)<\n>
<endif>
self.matchAny(self.input)
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = self.input.LA(1)<\n>
<endif>
self.matchAny()
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
self._state.following.append(self.FOLLOW_<rule.name>_in_<ruleName><elementIndex>)
<if(label)><label> = <endif>self.<if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">)<\n>
self._state.following.pop()
>>
/** ids+=rule */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
<label>Start<elementIndex> = self.getCharIndex()
self.<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">)
<label> = CommonToken(
input=self.input,
type=INVALID_TOKEN_TYPE,
channel=DEFAULT_CHANNEL,
start=<label>Start<elementIndex>,
stop=self.getCharIndex()-1
)
<else>
self.<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">)
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
<label>Start<elementIndex> = self.getCharIndex()
self.match(EOF)
<label> = CommonToken(input=self.input, type=EOF, channel=DEFAULT_CHANNEL, start=<label>Start<elementIndex>, stop=self.getCharIndex()-1)
<else>
self.match(EOF)
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if self.input.LA(1) == DOWN:
self.match(self.input, DOWN, None)
<children:element()>
self.match(self.input, UP, None)
<else>
self.match(self.input, DOWN, None)
<children:element()>
self.match(self.input, UP, None)
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if not (<evalPredicate(...)>):
<ruleBacktrackFailure()>
raise FailedPredicateException(self.input, "<ruleName>", "<description>")
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
LA<decisionNumber>_<stateNumber> = self.input.LA(<k>)<\n>
<edges; separator="\nel">
else:
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>
<else>
<ruleBacktrackFailure()>
nvae = NoViableAltException("<description>", <decisionNumber>, <stateNumber>, self.input)<\n>
<@noViableAltException()>
raise nvae<\n>
<endif>
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
LA<decisionNumber>_<stateNumber> = self.input.LA(<k>)<\n>
<edges; separator="\nel">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
LA<decisionNumber>_<stateNumber> = self.input.LA(<k>)<\n>
<edges; separator="\nel"><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber> = <eotPredictsAlt> <! if no edges, don't gen ELSE !>
<else>
else:
alt<decisionNumber> = <eotPredictsAlt>
<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber> = <alt>"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if (<labelExpr>) <if(predicates)>and (<predicates>)<endif>:
<targetState>
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
<!
FIXME: this is one of the few occasion, where I miss a switch statement
in Python. ATM this is implemented as a list of if .. elif ..
This may be replaced by faster a dictionary lookup, when I find a solution
for the cases when an edge is not a plain dfaAcceptState.
!>
LA<decisionNumber> = self.input.LA(<k>)
<edges; separator="\nel">
else:
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>
<else>
<ruleBacktrackFailure()>
nvae = NoViableAltException("<description>", <decisionNumber>, <stateNumber>, self.input)<\n>
<@noViableAltException()>
raise nvae<\n>
<endif>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
LA<decisionNumber> = self.input.LA(<k>)
<edges; separator="\nel">
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
LA<decisionNumber> = self.input.LA(<k>)
<edges; separator="\nel">
<if(eotPredictsAlt)>
else:
alt<decisionNumber> = <eotPredictsAlt>
<endif>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
if <labels:{LA<decisionNumber> == <it>}; separator=" or ">:
<targetState>
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = self.dfa<decisionNumber>.predict(self.input)
>>
/* Dump DFA tables as run-length-encoded Strings of octal values.
* Can't use hex as compiler translates them before compilation.
* These strings are split into multiple, concatenated strings.
* Java puts them back together at compile time thankfully.
* Java cannot handle large static arrays, so we're stuck with this
* encode/decode approach. See analysis and runtime DFA for
* the encoding methods.
*/
cyclicDFA(dfa) ::= <<
# lookup tables for DFA #<dfa.decisionNumber>
DFA<dfa.decisionNumber>_eot = DFA.unpack(
u"<dfa.javaCompressedEOT; wrap="\"\n u\"">"
)
DFA<dfa.decisionNumber>_eof = DFA.unpack(
u"<dfa.javaCompressedEOF; wrap="\"\n u\"">"
)
DFA<dfa.decisionNumber>_min = DFA.unpack(
u"<dfa.javaCompressedMin; wrap="\"\n u\"">"
)
DFA<dfa.decisionNumber>_max = DFA.unpack(
u"<dfa.javaCompressedMax; wrap="\"\n u\"">"
)
DFA<dfa.decisionNumber>_accept = DFA.unpack(
u"<dfa.javaCompressedAccept; wrap="\"\n u\"">"
)
DFA<dfa.decisionNumber>_special = DFA.unpack(
u"<dfa.javaCompressedSpecial; wrap="\"\n u\"">"
)
DFA<dfa.decisionNumber>_transition = [
<dfa.javaCompressedTransition:{s|DFA.unpack(u"<s; wrap="\"\nu\"">")}; separator=",\n">
]
# class definition for DFA #<dfa.decisionNumber>
class DFA<dfa.decisionNumber>(DFA):
pass
<@errorMethod()>
<if(dfa.specialStateSTs)>
def specialStateTransition(self_, s, input):
# convince pylint that my self_ magic is ok ;)
# pylint: disable-msg=E0213
# pretend we are a member of the recognizer
# thus semantic predicates can be evaluated
self = self_.recognizer
_s = s
<dfa.specialStateSTs:{state |
if s == <i0>: <! compressed special state numbers 0..n-1 !>
<state>}; separator="\nel">
<if(backtracking)>
if self._state.backtracking >0:
raise BacktrackingFailed
<endif>
nvae = NoViableAltException(self_.getDescription(), <dfa.decisionNumber>, _s, input)
self_.error(nvae)
raise nvae<\n>
<endif>
>>
cyclicDFAInit(dfa) ::= <<
self.dfa<dfa.decisionNumber> = self.DFA<dfa.decisionNumber>(
self, <dfa.decisionNumber>,
eot = self.DFA<dfa.decisionNumber>_eot,
eof = self.DFA<dfa.decisionNumber>_eof,
min = self.DFA<dfa.decisionNumber>_min,
max = self.DFA<dfa.decisionNumber>_max,
accept = self.DFA<dfa.decisionNumber>_accept,
special = self.DFA<dfa.decisionNumber>_special,
transition = self.DFA<dfa.decisionNumber>_transition
)<\n>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
LA<decisionNumber>_<stateNumber> = input.LA(1)<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
index<decisionNumber>_<stateNumber> = input.index()
input.rewind()<\n>
<endif>
s = -1
<edges; separator="\nel">
<if(semPredState)> <! return input cursor to state before we rewound !>
input.seek(index<decisionNumber>_<stateNumber>)<\n>
<endif>
if s >= 0:
return s
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if (<labelExpr>)<if(predicates)> and (<predicates>)<endif>:
s = <targetStateNumber><\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
se:
s = <targetStateNumber><\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "((<left>) and (<right>))"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | or <o>}>)"
notPredicate(pred) ::= "not (<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "self.<pred>()"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber> == <atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "self.input.LA(<k>) == <atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(<lower> \<= LA<decisionNumber>_<stateNumber> \<= <upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(<lower> \<= self.input.LA(<k>) \<= <upper>)"
setTest(ranges) ::= "<ranges; separator=\" or \">"
// A T T R I B U T E S
globalAttributeScopeClass(scope) ::= <<
<if(scope.attributes)>
class <scope.name>_scope(object):
def __init__(self):
<scope.attributes:{self.<it.decl> = None}; separator="\n">
<endif>
>>
globalAttributeScopeStack(scope) ::= <<
<if(scope.attributes)>
self.<scope.name>_stack = []<\n>
<endif>
>>
ruleAttributeScopeClass(scope) ::= <<
<if(scope.attributes)>
class <scope.name>_scope(object):
def __init__(self):
<scope.attributes:{self.<it.decl> = None}; separator="\n">
<endif>
>>
ruleAttributeScopeStack(scope) ::= <<
<if(scope.attributes)>
self.<scope.name>_stack = []<\n>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<label.label.text> = None<\n>
>>
returnStructName() ::= "<it.name>_return"
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
class <ruleDescriptor:returnStructName()>(<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope):
def __init__(self):
super(<grammar.recognizerName>.<ruleDescriptor:returnStructName()>, self).__init__()
<scope.attributes:{self.<it.decl> = None}; separator="\n">
<@ruleReturnInit()>
<@ruleReturnMembers()>
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> = <expr>"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
self.<scope>_stack[-<negIndex>].<attr.name>
<else>
<if(index)>
self.<scope>_stack[<index>].<attr.name>
<else>
self.<scope>_stack[-1].<attr.name>
<endif>
<endif>
>>
/* not applying patch because of bug in action parser!
<if(negIndex)>
((len(self.<scope>_stack) - <negIndex> - 1) >= 0 and [self.<scope>_stack[-<negIndex>].<attr.name>] or [None])[0]
<else>
<if(index)>
((<index> \< len(self.<scope>_stack)) and [self.<scope>_stack[<index>].<attr.name>] or [None])[0]
<else>
((len(self.<scope>_stack) > 0) and [self.<scope>_stack[-1].<attr.name>] or [None])[0]
<endif>
<endif>
*/
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
<!FIXME: this seems not to be used by ActionTranslator...!>
self.<scope>_stack[-<negIndex>].<attr.name> = <expr>
<else>
<if(index)>
<!FIXME: this seems not to be used by ActionTranslator...!>
self.<scope>_stack[<index>].<attr.name> = <expr>
<else>
self.<scope>_stack[-1].<attr.name> = <expr>
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "self.<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
((<scope> is not None) and [<scope>.<attr.name>] or [None])[0]
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name> = <expr>
<else>
<attr.name> = <expr>
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach; and they are evaluated early;
// they cannot see TREE_PARSER or PARSER attributes for example. :(
tokenLabelPropertyRef_text(scope,attr) ::= "<scope>.text"
tokenLabelPropertyRef_type(scope,attr) ::= "<scope>.type"
tokenLabelPropertyRef_line(scope,attr) ::= "<scope>.line"
tokenLabelPropertyRef_pos(scope,attr) ::= "<scope>.charPositionInLine"
tokenLabelPropertyRef_channel(scope,attr) ::= "<scope>.channel"
tokenLabelPropertyRef_index(scope,attr) ::= "<scope>.index"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
ruleLabelPropertyRef_start(scope,attr) ::= "<scope>.start"
ruleLabelPropertyRef_stop(scope,attr) ::= "<scope>.stop"
ruleLabelPropertyRef_tree(scope,attr) ::= "<scope>.tree"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
((<scope> is not None) and [self.input.getTokenStream().toString(
self.input.getTreeAdaptor().getTokenStartIndex(<scope>.start),
self.input.getTreeAdaptor().getTokenStopIndex(<scope>.start)
)] or [None])[0]
<else>
((<scope> is not None) and [self.input.toString(<scope>.start,<scope>.stop)] or [None])[0]
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "((<scope> is not None) and [<scope>.st] or [None])[0]"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "((<scope> is not None) and [<scope>.type] or [0])[0]"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "((<scope> is not None) and [<scope>.line] or [0])[0]"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "((<scope> is not None) and [<scope>.charPositionInLine] or [0])[0]"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "((<scope> is not None) and [<scope>.channel] or [0])[0]"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "((<scope> is not None) and [<scope>.index] or [0])[0]"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "((<scope> is not None) and [<scope>.text] or [None])[0]"
lexerRuleLabelPropertyRef_int(scope,attr) ::= "((<scope> is not None) and [int(<scope>.text)] or [0])[0]"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "retval.start"
rulePropertyRef_stop(scope,attr) ::= "retval.stop" //mmm... or input.LT(-1)??
rulePropertyRef_tree(scope,attr) ::= "retval.tree"
rulePropertyRef_text(scope,attr) ::= "self.input.toString(retval.start, self.input.LT(-1))"
rulePropertyRef_st(scope,attr) ::= "retval.st"
lexerRulePropertyRef_text(scope,attr) ::= "self.text"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "self._state.tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "self._state.tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "_channel"
lexerRulePropertyRef_start(scope,attr) ::= "self._state.tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(self.getCharIndex()-1)"
lexerRulePropertyRef_int(scope,attr) ::= "int(<scope>.text)"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.tree =<expr>"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.st =<expr>"
/** How to execute an action (only when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
<if(actions.(actionScope).synpredgate)>
if <actions.(actionScope).synpredgate>:
<action>
<else>
if <actions.(actionScope).synpredgate>:
<action>
<endif>
<else>
#action start
<action>
#action end
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
codeFileExtension() ::= ".py"
true() ::= "True"
false() ::= "False"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/ASTParser.stg 0000644 0001750 0001750 00000014430 11256465234 030265 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
finishedBacktracking(block) ::= <<
<if(backtracking)>
if <actions.(actionScope).synpredgate>:
<block>
<else>
<block>
<endif>
>>
@ruleBody.setErrorReturnValue() ::= <<
retval.tree = self._adaptor.errorNode(self.input, retval.start, self.input.LT(-1), re)
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<finishedBacktracking({
<label>_tree = <createNodeFromToken(...)>
self._adaptor.addChild(root_0, <label>_tree)
})>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<finishedBacktracking({
<label>_tree = <createNodeFromToken(...)>
root_0 = self._adaptor.becomeRoot(<label>_tree, root_0)
})>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<finishedBacktracking({self._adaptor.addChild(root_0, <createNodeFromToken(...)>)})>})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label> = self.input.LT(1)<\n>
<endif>
<super.matchSet(..., postmatchCode={<finishedBacktracking({root_0 = self._adaptor.becomeRoot(<createNodeFromToken(...)>, root_0)})>})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<finishedBacktracking({self._adaptor.addChild(root_0, <label>.tree)})>
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<finishedBacktracking({root_0 = self._adaptor.becomeRoot(<label>.tree, root_0)})>
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".tree",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".tree",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".tree",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<finishedBacktracking({
<label>_tree = self._adaptor.createWithPayload(<label>)
self._adaptor.addChild(root_0, <label>_tree)
})>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<finishedBacktracking({
<label>_tree = self._adaptor.createWithPayload(<label>)
root_0 = self._adaptor.becomeRoot(<label>_tree, root_0)
})>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
<hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
self._adaptor.createWithPayload(<label>)
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<finishedBacktracking({
retval.tree = self._adaptor.rulePostProcessing(root_0)
self._adaptor.setTokenBoundaries(retval.tree, retval.start, retval.stop)
})>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/Dbg.stg 0000644 0001750 0001750 00000022551 11256465234 027160 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to normal Python output;
* If ASTs are built, then you'll also get ASTDbg.stg loaded.
*/
group Dbg;
@outputFile.imports() ::= <<
<@super.imports()>
from antlr3.debug import *
>>
@genericParser.args() ::= <<
debug_socket = kwargs.pop('debug_socket', None)
port = kwargs.pop('port', None)
>>
@genericParser.init() ::= <<
self.ruleLevel = 0
if self._dbg is None:
<createListenerAndHandshake()>
>>
createListenerAndHandshake() ::= <<
<if(TREE_PARSER)>
proxy = DebugEventSocketProxy(self, adaptor=self.input.getTreeAdaptor(),
debug=debug_socket, port=port)<\n>
<else>
proxy = DebugEventSocketProxy(self, debug=debug_socket, port=port)<\n>
<endif>
self.setDebugListener(proxy)
proxy.handshake()
>>
@genericParser.members() ::= <<
<if(grammar.grammarIsRoot)>
ruleNames = [
"invalidRule", <grammar.allImportedRules:{rST | "<rST.name>"}; wrap="\n ", separator=", ">
]<\n>
<endif>
<if(grammar.grammarIsRoot)> <! grammar imports other grammar(s) !>
def getRuleLevel(self):
return self.ruleLevel
def incRuleLevel(self):
self.ruleLevel += 1
def decRuleLevel(self):
self.ruleLevel -= 1
<if(profile)>
<ctorForProfilingRootGrammar()>
<else>
<ctorForRootGrammar()>
<endif>
<ctorForPredefinedListener()>
<else> <! imported grammar !>
def getRuleLevel(self):
return <grammar.delegators:{g| <g:delegateName()>}>.getRuleLevel()
def incRuleLevel(self):
<grammar.delegators:{g| <g:delegateName()>}>.incRuleLevel()
def decRuleLevel(self):
<grammar.delegators:{g| <g:delegateName()>}>.decRuleLevel()
<ctorForDelegateGrammar()>
<endif>
<if(profile)>
FIXME(2)
public boolean alreadyParsedRule(IntStream input, int ruleIndex) {
((Profiler)self._dbg).examineRuleMemoization(input, ruleIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
return super.alreadyParsedRule(input, ruleIndex);
}<\n>
FIXME(3)
public void memoize(IntStream input,
int ruleIndex,
int ruleStartIndex)
{
((Profiler)self._dbg).memoize(input, ruleIndex, ruleStartIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
super.memoize(input, ruleIndex, ruleStartIndex);
}<\n>
<endif>
def evalPredicate(self, result, predicate):
self._dbg.semanticPredicate(result, predicate)
return result
<\n>
>>
ctorForRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
<! Same except we add port number and profile stuff if root grammar !>
<!
public <name>(<inputStreamType> input) {
this(input, DebugEventSocketProxy.DEFAULT_DEBUGGER_PORT, new RecognizerSharedState());
}
public <name>(<inputStreamType> input, int port, RecognizerSharedState state) {
super(input, state);
<parserCtorBody()>
<createListenerAndHandshake()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, self._dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
!>
>>
ctorForProfilingRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
public <name>(<inputStreamType> input) {
this(input, new Profiler(null), new RecognizerSharedState());
}
public <name>(<inputStreamType> input, DebugEventListener self.dbg, RecognizerSharedState state) {
super(input, self.dbg, state);
Profiler p = (Profiler)self.dbg;
p.setParser(this);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, self.dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}
<\n>
>>
/** Basically we don't want to set any dbg listeners are root will have it. */
ctorForDelegateGrammar() ::= <<
<!
public <name>(<inputStreamType> input, DebugEventListener self.dbg, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}>) {
super(input, dbg, state);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, this, this.state<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
}<\n>
!>
>>
ctorForPredefinedListener() ::= <<
<!
public <name>(<inputStreamType> input, DebugEventListener dbg) {
<@superClassRef>super(input, dbg, new RecognizerSharedState());<@end>
<if(profile)>
Profiler p = (Profiler)dbg;
p.setParser(this);
<endif>
<parserCtorBody()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, self._dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
!>
>>
@genericParser.superClassName() ::= "Debug<@super.superClassName()>"
@rule.body() ::= <<
try:
self._dbg.enterRule(self.getGrammarFileName(), "<ruleName>")
if self.getRuleLevel() == 0:
self._dbg.commence();
self.incRuleLevel()
self._dbg.location(<ruleDescriptor.tree.line>, <ruleDescriptor.tree.column>)
<@super.body()>
self._dbg.location(<ruleDescriptor.EORNode.line>, <ruleDescriptor.EORNode.column>)
finally:
self._dbg.exitRule(self.getGrammarFileName(), "<ruleName>")
self.decRuleLevel()
if self.getRuleLevel() == 0:
self._dbg.terminate()
>>
@synpred.start() ::= "self._dbg.beginBacktrack(self._state.backtracking)"
@synpred.stop() ::= "self._dbg.endBacktrack(self._state.backtracking, success)"
// Common debug event triggers used by region overrides below
enterSubRule() ::=
"try { self._dbg.enterSubRule(<decisionNumber>);<\n>"
exitSubRule() ::=
"} finally {self._dbg.exitSubRule(<decisionNumber>);}<\n>"
enterDecision() ::=
"try { self._dbg.enterDecision(<decisionNumber>);<\n>"
exitDecision() ::=
"} finally {self._dbg.exitDecision(<decisionNumber>);}<\n>"
enterAlt(n) ::= "self._dbg.enterAlt(<n>)<\n>"
// Region overrides that tell various constructs to add debugging triggers
@block.body() ::= <<
try:
self._dbg.enterSubRule(<decisionNumber>)
<@super.body()>
finally:
self._dbg.exitSubRule(<decisionNumber>)
>>
@blockBody.decision() ::= <<
try:
self._dbg.enterDecision(<decisionNumber>)
<@super.decision()>
finally:
self._dbg.exitDecision(<decisionNumber>)
>>
@ruleBlock.decision() ::= <<
try:
self._dbg.enterDecision(<decisionNumber>)
<@super.decision()>
finally:
self._dbg.exitDecision(<decisionNumber>)
>>
@ruleBlockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@blockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@positiveClosureBlock.loopBody() ::= <<
try:
self._dbg.enterSubRule(<decisionNumber>)
<@super.loopBody()>
finally:
self._dbg.exitSubRule(<decisionNumber>)<\n>
>>
@positiveClosureBlockLoop.decisionBody() ::= <<
try:
self._dbg.enterDecision(<decisionNumber>)
<@super.decisionBody()>
finally:
self._dbg.exitDecision(<decisionNumber>)
>>
@positiveClosureBlockLoop.earlyExitException() ::=
"self._dbg.recognitionException(eee)<\n>"
@closureBlock.loopBody() ::= <<
try:
self._dbg.enterSubRule(<decisionNumber>)
<@super.loopBody()>
finally:
self._dbg.exitSubRule(<decisionNumber>)<\n>
>>
@closureBlockLoop.decisionBody() ::= <<
try:
self._dbg.enterDecision(<decisionNumber>)
<@super.decisionBody()>
finally:
self._dbg.exitDecision(<decisionNumber>)
>>
@altSwitchCase.prealt() ::= "<enterAlt(n=i)>"
@element.prematch() ::=
"self._dbg.location(<it.line>, <it.pos>)"
@matchSet.mismatchedSetException() ::=
"self._dbg.recognitionException(mse)"
@dfaState.noViableAltException() ::= "self._dbg.recognitionException(nvae)"
@dfaStateSwitch.noViableAltException() ::= "self._dbg.recognitionException(nvae)"
dfaDecision(decisionNumber,description) ::= <<
try:
self.isCyclicDecision = True
<super.dfaDecision(...)>
except NoViableAltException, nvae:
self._dbg.recognitionException(nvae)
raise
>>
@cyclicDFA.errorMethod() ::= <<
def error(self, nvae):
self._dbg.recognitionException(nvae)
>>
/** Force predicate validation to trigger an event */
evalPredicate(pred,description) ::= <<
self.evalPredicate(<pred>,"<description>")
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/AST.stg 0000644 0001750 0001750 00000033506 11256465234 027115 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* in sync with Java/AST.stg revision 36 */
group AST;
finishedBacktracking(block) ::= <<
<if(backtracking)>
if <actions.(actionScope).synpredgate>:
<block>
<else>
<block>
<endif>
>>
@outputFile.imports() ::= <<
<@super.imports()>
<if(!TREE_PARSER)><! tree parser would already have imported !>
from antlr3.tree import *<\n>
<endif>
>>
/** Add an adaptor property that knows how to build trees */
@genericParser.init() ::= <<
self._adaptor = None
self.adaptor = CommonTreeAdaptor()
<@super.init()>
>>
@genericParser.members() ::= <<
<@super.members()>
<astAccessor()>
>>
astAccessor() ::= <<
def getTreeAdaptor(self):
return self._adaptor
def setTreeAdaptor(self, adaptor):
self._adaptor = adaptor
<grammar.directDelegates:{g|<g:delegateName()>.adaptor = adaptor}; separator="\n">
adaptor = property(getTreeAdaptor, setTreeAdaptor)
>>
@returnScope.ruleReturnInit() ::= <<
self.tree = None
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
root_0 = None<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<[ruleDescriptor.tokenLabels,ruleDescriptor.wildcardTreeLabels,
ruleDescriptor.wildcardTreeListLabels]
:{<it.label.text>_tree = None}; separator="\n">
<ruleDescriptor.tokenListLabels:{<it.label.text>_tree = None}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{stream_<it> = RewriteRule<rewriteElementType>Stream(self._adaptor, "token <it>")}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{stream_<it> = RewriteRuleSubtreeStream(self._adaptor, "rule <it>")}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = self._adaptor.nil()<\n>
<endif>
<endif>
<endif>
>>
// // T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<finishedBacktracking({stream_<token>.add(<label>)})>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<finishedBacktracking({stream_<token>.add(<label>)})>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<finishedBacktracking({stream_<rule.name>.add(<label>.tree)})>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".tree",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<finishedBacktracking({stream_<rule.name>.add(<label>.tree)})>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".tree",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
# AST Rewrite
# elements: <referencedElementsDeep; separator=", ">
# token labels: <referencedTokenLabels; separator=", ">
# rule labels: <referencedRuleLabels; separator=", ">
# token list labels: <referencedTokenListLabels; separator=", ">
# rule list labels: <referencedRuleListLabels; separator=", ">
# wildcard labels: <[referencedWildcardLabels,referencedWildcardListLabels]; separator=", ">
<finishedBacktracking({
<prevRuleRootRef()>.tree = root_0
<rewriteCodeLabels()>
root_0 = self._adaptor.nil()
<first(alts):rewriteAltFirst(); anchor>
<rest(alts):{a | el<rewriteAltRest(a)>}; anchor, separator="\n\n">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.tree = self._adaptor.rulePostProcessing(root_0)
self.input.replaceChildren(
self._adaptor.getParent(retval.start),
self._adaptor.getChildIndex(retval.start),
self._adaptor.getChildIndex(_last),
retval.tree
)<\n>
<endif>
<endif>
<! if parser or tree-parser and rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.tree = root_0<\n>
<else>
<if(!rewriteMode)>
<prevRuleRootRef()>.tree = root_0<\n>
<endif>
<endif>
})>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{stream_<it> = RewriteRule<rewriteElementType>Stream(self._adaptor, "token <it>", <it>)};
separator="\n"
>
<referencedTokenListLabels
:{stream_<it> = RewriteRule<rewriteElementType>Stream(self._adaptor, "token <it>", list_<it>)};
separator="\n"
>
<referencedWildcardLabels
:{stream_<it> = RewriteRuleSubtreeStream(self._adaptor, "wildcard <it>", <it>)};
separator="\n"
>
<referencedWildcardListLabels
:{stream_<it> = RewriteRuleSubtreeStream(self._adaptor, "wildcard <it>", list_<it>)};
separator="\n"
>
<referencedRuleLabels
:{
if <it> is not None:
stream_<it> = RewriteRuleSubtreeStream(self._adaptor, "rule <it>", <it>.tree)
else:
stream_<it> = RewriteRuleSubtreeStream(self._adaptor, "token <it>", None)
};
separator="\n"
>
<referencedRuleListLabels
:{stream_<it> = RewriteRuleSubtreeStream(self._adaptor, "token <it>", list_<it>)};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
# <fileName>:<description>
if <referencedElementsDeep:{el | stream_<el>.hasNext()}; separator=" or ">:
<alt>
<referencedElementsDeep:{el | stream_<el>.reset();<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
# <fileName>:<description>
while <referencedElements:{el | stream_<el>.hasNext()}; separator=" or ">:
<alt>
<referencedElements:{el | stream_<el>.reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
# <fileName>:<description>
if not (<referencedElements:{el | stream_<el>.hasNext()}; separator=" or ">):
raise RewriteEarlyExitException()
while <referencedElements:{el | stream_<el>.hasNext()}; separator=" or ">:
<alt>
<referencedElements:{el | stream_<el>.reset()<\n>}>
>>
rewriteAltRest(a) ::= <<
<if(a.pred)>
if <a.pred>:
# <a.description>
<a.alt>
<else>
se: <! little hack to get if .. elif .. else block right !>
# <a.description>
<a.alt>
<endif>
>>
rewriteAltFirst(a) ::= <<
<if(a.pred)>
if <a.pred>:
# <a.description>
<a.alt>
<else>
# <a.description>
<a.alt>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = None"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
# <fileName>:<description>
root_<treeLevel> = self._adaptor.nil()
<root:rewriteElement()>
<children:rewriteElement()>
self._adaptor.addChild(root_<enclosingTreeLevel>, root_<treeLevel>)<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
self._adaptor.addChild(root_<treeLevel>, <createRewriteNodeFromElement(...)>)<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
self._adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode())<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
self._adaptor.addChild(root_<treeLevel>, stream_<label>.nextNode())<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = self._adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>)<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = self._adaptor.becomeRoot(<createRewriteNodeFromElement(...)>, root_<treeLevel>)<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
self._adaptor.addChild(root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>)<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = self._adaptor.becomeRoot(<createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>)<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
<!FIXME(96,untested)!>
root_0 = <action><\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
self._adaptor.addChild(root_<treeLevel>, stream_<rule>.nextTree())<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = self._adaptor.becomeRoot(stream_<rule>.nextNode(), root_<treeLevel>)<\n>
>>
rewriteNodeAction(action) ::= <<
self._adaptor.addChild(root_<treeLevel>, <action>)<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = self._adaptor.becomeRoot(<action>, root_<treeLevel>)<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
self._adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree())<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
self._adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree())<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = self._adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>)<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = self._adaptor.becomeRoot(stream_<label>.nextNode(), root_<treeLevel>)<\n>
>>
rewriteWildcardLabelRef(label) ::= <<
self._adaptor.addChild(root_<treeLevel>, stream_<label>.nextTree())<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
<hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
<if (!args)>self._adaptor.createFromType(<tokenType>, "<tokenType>")
<else>self._adaptor.create(<tokenType>, <args; separator=", ">)
<endif>
<endif>
>>
//<! need to call different adaptor.create*() methods depending of argument count !>
//<if (!args)>self._adaptor.createFromType(<tokenType>, "<tokenType>")
//<else><if (!rest(args))>self._adaptor.createFromType(<tokenType>, <first(args)>)
//<else><if (!rest(rest(args)))>self._adaptor.createFromToken(<tokenType>, <first(args)>, <first(rest(args))>)
//<endif>
//<endif>
//<endif>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
<hetero>(stream_<token>.nextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
<! need to call different adaptor.create*() methods depending of argument count !>
<if (!args)>self._adaptor.createFromType(<token>, "<token>")
<else><if (!rest(args))>self._adaptor.createFromToken(<token>, <first(args)>)
<else><if (!rest(rest(args)))>self._adaptor.createFromToken(<token>, <first(args)>, <first(rest(args))>)
<endif>
<endif>
<endif>
<else>
stream_<token>.nextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/Python/ASTDbg.stg 0000644 0001750 0001750 00000004702 11256465234 027526 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to AST stuff. Dynamic inheritance
* hierarchy is set up as ASTDbg : AST : Dbg : Python by code generator.
*/
group ASTDbg;
astAccessor() ::= <<
def setTreeAdaptor(self, adaptor):
<if(grammar.grammarIsRoot)>
self._adaptor = DebugTreeAdaptor(self.dbg, adaptor)
<else>
self._adaptor = adaptor # delegator sends dbg adaptor
<endif>
<grammar.directDelegates:{g|<g:delegateName()>.setTreeAdaptor(self._adaptor)}>
def getTreeAdaptor(self):
return self._adaptor
adaptor = property(getTreeAdaptor, setTreeAdaptor)<\n>
>>
createListenerAndHandshake() ::= <<
proxy = DebugEventSocketProxy(self, adaptor=<if(TREE_PARSER)>self.input.getTreeAdaptor()<else>self._adaptor<endif>,
debug=debug_socket, port=port)
self.setDebugListener(proxy)
self.adaptor.setDebugListener(proxy)
self.input.setDebugListener(proxy)
#self.set<inputStreamType>(Debug<inputStreamType>(self.input, proxy))
proxy.handshake()
>>
@rewriteElement.pregen() ::= "self._dbg.location(<e.line>, <e.pos>)"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ObjC/ 0000755 0001750 0001750 00000000000 11410174107 025260 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ObjC/ASTTreeParser.stg 0000644 0001750 0001750 00000011032 11256465234 030434 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group ASTTreeParser;
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
{
<ASTLabelType> root_<treeLevel> = [treeAdapator newEmptyTree];
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( [input LA:1] == ANTLRTokenTypeDOWN ) {
[self match:input tokenType:ANTLRTokenTypeDOWN follow:nil]; <checkRuleBacktrackFailure()>
<children:element()>
[self match:input tokenType:ANTLRTokenTypeUP follow:nil]; <checkRuleBacktrackFailure()>
}
<else>
[self match:input tokenType:ANTLRTokenTypeDOWN follow:nil]; <checkRuleBacktrackFailure()>
<children:element()>
[self match:input tokenType:ANTLRTokenTypeUP follow:nil]; <checkRuleBacktrackFailure()>
<endif>
[root_<treeLevel> release];
}<\n>
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(rewriteMode)>retval.tree = (<ASTLabelType>)retval.start;<endif>
>>
// TOKEN AST STUFF
/** ID auto construct */
tokenRef(token,label,elementIndex) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
adaptor.addChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.dupNode(<label>);
root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( state.backtracking==0 ) <endif>adaptor.addChild(root_<treeLevel>, <label>.getTree());
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( state.backtracking==0 ) <endif>root_<treeLevel> = (<ASTLabelType>)adaptor.becomeRoot(<label>.getTree(), root_<treeLevel>);
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".getTree()",...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
#error Heterogeneous tree support not implemented.
<if(hetero)>
new <hetero>(stream_<token>.nextNode())
<else>
stream_<token>.nextNode()
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ObjC/ASTParser.stg 0000644 0001750 0001750 00000015046 11256465234 027625 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,hetero,elementIndex) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( backtracking == 0 ) {<endif>
_<label>_tree = (<ASTLabelType>)[treeAdaptor newTreeWithToken:_<label>];
[treeAdaptor addChild:_<label>_tree toTree:root_0];
[_<label>_tree release];
<if(backtracking)>}<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,hetero,elementIndex) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( backtracking == 0 ) {<endif>
_<label>_tree = (<ASTLabelType>)[treeAdaptor newTreeWithToken:_<label>];
root_0 = (<ASTLabelType>)[treeAdaptor makeNode:_<label>_tree parentOf:root_0];
[_<label>_tree release];
<if(backtracking)>}<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
// TODO: add support for heterogeneous trees
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={
<if(backtracking)>if (backtracking == 0) {<endif>
_<label>_tree = (<ASTLabelType>)[treeAdaptor newTreeWithToken:_<label>];
[treeAdaptor addChild:_<label>_tree toTree:root_0];
[_<label>_tree release];
<if(backtracking)>}<endif>
})>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(backtracking)>if (backtracking == 0) {<endif>
_<label>_tree = (<ASTLabelType>)[treeAdaptor newTreeWithToken:_<label>];
root_0 = (<ASTLabelType>)[treeAdaptor makeNode:_<label>_tree parentOf:root_0];
[_<label>_tree release];
<if(backtracking)>}<endif>
})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if (backtracking == 0) <endif>[treeAdaptor addChild:[_<label> tree] toTree:root_0];
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if (backtracking == 0) <endif>root_0 = (<ASTLabelType>)[treeAdaptor makeNode:[_<label> tree] parentOf:root_0];
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem="["+label+" tree]",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem="["+label+" tree]",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem="["+label+" tree]",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if (backtracking == 0) {<endif>
_<label>_tree = (<ASTLabelType>)[treeAdaptor newTreeWithToken:_<label>];
[treeAdaptor addChild:_<label>_tree toTree:root_0];
[_<label>_tree release];
<if(backtracking)>}<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if (backtracking == 0) {<endif>
_<label>_tree = (<ASTLabelType>)[treeAdaptor newTreeWithToken:_<label>];
root_0 = (<ASTLabelType>)[treeAdaptor makeNode:_<label>_tree parentOf:root_0];
[_<label>_tree release];
<if(backtracking)>}<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
new <hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
(<ASTLabelType>)adaptor.create(<label>)
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ObjC/Dbg.stg 0000644 0001750 0001750 00000012775 11256465234 026523 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to normal Objective-C output;
* If ASTs are built, then you'll also get ASTDbg.stg loaded.
*/
group Dbg;
@headerFile.imports() ::= <<
<@super.imports()>
#import \<ANTLR/ANTLRDebug.h>
>>
@parserHeaderFile.ivars() ::= <<
int ruleLevel;
NSArray *ruleNames;
>>
@parserHeaderFile.methodsdecl() ::= <<
-(BOOL) evalPredicate:(NSString *)predicate matched:(BOOL)result;<\n>
>>
@genericParser.init() ::= <<
ruleNames = [[NSArray alloc] initWithObjects:<rules:{rST | @"<rST.ruleName>"}; separator=", ", wrap="\n ">, nil];<\n>
>>
@genericParser.dealloc() ::= <<
[ruleNames release];<\n>
>>
@genericParser.methods() ::= <<
-(BOOL) evalPredicate:(NSString *)predicate matched:(BOOL)result
{
[debugListener semanticPredicate:predicate matched:result];
return result;
}<\n>
>>
/* bug: can't use @super.superClassName()> */
@parserHeaderFile.superClassName() ::= "ANTLRDebug<if(TREE_PARSER)>Tree<endif>Parser"
@rule.preamble() ::= <<
@try { [debugListener enterRule:@"<ruleName>"];
if ( ruleLevel==0 ) [debugListener commence];
ruleLevel++;
[debugListener locationLine:<ruleDescriptor.tree.line> column:<ruleDescriptor.tree.column>];<\n>
>>
@rule.postamble() ::= <<
[debugListener locationLine:<ruleDescriptor.EORNode.line> column:<ruleDescriptor.EORNode.column>];<\n>
}
@finally {
[debugListener exitRule:@"<ruleName>"];
ruleLevel--;
if ( ruleLevel==0 ) [debugListener terminate];
}<\n>
>>
/* these are handled in the runtime for now.
* stinks, but that's the easiest way to avoid having to generate two
* methods for each synpred
@synpred.start() ::= "[debugListener beginBacktrack:backtracking];"
@synpred.stop() ::= "[debugListener endBacktrack:backtracking wasSuccessful:success];"
*/
// Common debug event triggers used by region overrides below
enterSubRule() ::=
"@try { [debugListener enterSubRule:<decisionNumber>];<\n>"
exitSubRule() ::=
"} @finally { [debugListener exitSubRule:<decisionNumber>]; }<\n>"
enterDecision() ::=
"@try { [debugListener enterDecision:<decisionNumber>];<\n>"
exitDecision() ::=
"} @finally { [debugListener exitDecision:<decisionNumber>]; }<\n>"
enterAlt(n) ::= "[debugListener enterAlt:<n>];<\n>"
// Region overrides that tell various constructs to add debugging triggers
@block.predecision() ::= "<enterSubRule()><enterDecision()>"
@block.postdecision() ::= "<exitDecision()>"
@block.postbranch() ::= "<exitSubRule()>"
@ruleBlock.predecision() ::= "<enterDecision()>"
@ruleBlock.postdecision() ::= "<exitDecision()>"
@ruleBlockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@blockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@positiveClosureBlock.preloop() ::= "<enterSubRule()>"
@positiveClosureBlock.postloop() ::= "<exitSubRule()>"
@positiveClosureBlock.predecision() ::= "<enterDecision()>"
@positiveClosureBlock.postdecision() ::= "<exitDecision()>"
@positiveClosureBlock.earlyExitException() ::=
"[debugListener recognitionException:eee];<\n>"
@closureBlock.preloop() ::= "<enterSubRule()>"
@closureBlock.postloop() ::= "<exitSubRule()>"
@closureBlock.predecision() ::= "<enterDecision()>"
@closureBlock.postdecision() ::= "<exitDecision()>"
@altSwitchCase.prealt() ::= "<enterAlt(n=i)>"
@element.prematch() ::=
"[debugListener locationLine:<it.line> column:<it.pos>];"
@matchSet.mismatchedSetException() ::=
"[debugListener recognitionException:mse];"
@dfaState.noViableAltException() ::= "[debugListener recognitionException:nvae];"
@dfaStateSwitch.noViableAltException() ::= "[debugListener recognitionException:nvae];"
dfaDecision(decisionNumber,description) ::= <<
@try {
// isCyclicDecision is only necessary for the Profiler. Which I didn't do, yet.
// isCyclicDecision = YES;
<super.dfaDecision(...)>
}
@catch (ANTLRNoViableAltException *nvae) {
[debugListener recognitionException:nvae];
@throw nvae;
}
>>
@cyclicDFA.errorMethod() ::= <<
-(void) error:(ANTLRNoViableAltException *)nvae
{
[[recognizer debugListener] recognitionException:nvae];
}
>>
/** Force predicate validation to trigger an event */
evalPredicate(pred,description) ::= <<
[self evalPredicate:@"<description>" result:<pred>];
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ObjC/AST.stg 0000644 0001750 0001750 00000040450 11256465234 026445 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006, 2007 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()>
<if(!TREE_PARSER)><! tree parser would already have imported !>
#import \<ANTLR/ANTLR.h><\n>
<endif>
>>
@parserHeaderFile.ivars() ::= <<
<@super.ivars()>
<parserIVars()>
>>
@parserHeaderFile.methodsdecl() ::= <<
<@super.methodsdecl()>
<parserMethodsDecl()>
>>
@genericParser.methods() ::= <<
<@super.methods()>
<parserMethods()>
>>
/** additional init code for tree support */
@genericParser.init() ::= <<
<@super.init()>
[self setTreeAdaptor:[[[ANTLRCommonTreeAdaptor alloc] init] autorelease]];
>>
@genericParser.dealloc() ::= <<
[self setTreeAdaptor:nil];
<@super.dealloc()>
>>
/** Add an adaptor property that knows how to build trees */
parserIVars() ::= <<
id\<ANTLRTreeAdaptor> treeAdaptor;
>>
/** Declaration of additional tree support methods - go in interface of parserHeaderFile() */
parserMethodsDecl() ::= <<
- (id\<ANTLRTreeAdaptor>) treeAdaptor;
- (void) setTreeAdaptor:(id\<ANTLRTreeAdaptor>)theTreeAdaptor;
>>
/** Definition of addition tree support methods - go in implementation of genericParser() */
parserMethods() ::= <<
- (id\<ANTLRTreeAdaptor>) treeAdaptor
{
return treeAdaptor;
}
- (void) setTreeAdaptor:(id\<ANTLRTreeAdaptor>)aTreeAdaptor
{
if (aTreeAdaptor != treeAdaptor) {
[aTreeAdaptor retain];
[treeAdaptor release];
treeAdaptor = aTreeAdaptor;
}
}
>>
/** addition ivars for returnscopes */
@returnScopeInterface.ivars() ::= <<
<recognizer.ASTLabelType; null="id"> tree;
>>
/** the interface of returnScope methods */
@returnScopeInterface.methods() ::= <<
- (<recognizer.ASTLabelType; null="id">) tree;
- (void) setTree:(<recognizer.ASTLabelType; null="id">)aTree;
>>
/** the implementation of returnScope methods */
@returnScope.methods() ::= <<
- (<ASTLabelType>) tree
{
return tree;
}
- (void) setTree:(<ASTLabelType>)aTree
{
if (tree != aTree) {
[aTree retain];
[tree release];
tree = aTree;
}
}
- (void) dealloc
{
[self setTree:nil];
[super dealloc];
}
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> root_0 = nil;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<ruleDescriptor.tokenLabels:{<ASTLabelType> _<it.label.text>_tree = nil;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<ASTLabelType> _<it.label.text>_tree = nil;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{ANTLRRewriteRuleTokenStream *_stream_<it>=[[ANTLRRewriteRuleTokenStream alloc] initWithTreeAdaptor:treeAdaptor description:@"token <it>"];}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{ANTLRRewriteRuleSubtreeStream *_stream_<it>=[[ANTLRRewriteRuleSubtreeStream alloc] initWithTreeAdaptor:treeAdaptor description:@"rule <it>"];}; separator="\n">
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<[ruleDescriptor.allTokenRefsInAltsWithRewrites,ruleDescriptor.allRuleRefsInAltsWithRewrites]
:{[_stream_<it> release];}; separator="\n">
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(backtracking)>
if ( ![_state isBacktracking] ) {<\n>
<endif>
[_<prevRuleRootRef()> setTree:(<ASTLabelType>)[treeAdaptor postProcessTree:root_0]];
[treeAdaptor setBoundariesForTree:[_<prevRuleRootRef()> tree] fromToken:[_<prevRuleRootRef()> start] toToken:[_<prevRuleRootRef()> stop]];
<if(backtracking)>
}
<endif>
<endif>
[root_0 release];
>>
rewriteCodeLabelsCleanUp() ::= <<
<referencedTokenLabels
:{[_stream_<it> release];};
separator="\n"
>
<referencedTokenListLabels
:{[_stream_<it> release];};
separator="\n"
>
<referencedRuleLabels
:{[_stream_<it> release];};
separator="\n"
>
<referencedRuleListLabels
:{[_stream_<it> release];};
separator="\n"
>
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
root_0 = (<ASTLabelType>)[treeAdaptor newEmptyTree];<\n>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( ![_state isBacktracking] ) <endif>[_stream_<token> addElement:_<label>];<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( ![_state isBacktracking] ) <endif>[_stream_<token> addElement:_<label>];<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( ![_state isBacktracking] ) <endif>[_stream_<rule.name> addElement:[_<label> tree]];
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRefRuleRoot(...)>
<if(backtracking)>if ( ![_state isBacktracking] ) <endif>[_stream_<rule.name> addElement:[_<label> tree]];<\n>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem="["+label+" tree]",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
<if(backtracking)>
if (![_state isBacktracking]) {<\n>
<endif>
int i_0 = 0;
root_0 = (<ASTLabelType>)[treeAdaptor newEmptyTree];
[_<prevRuleRootRef()> setTree:root_0];
<rewriteCodeLabels()>
<alts:rewriteAlt(); separator="else ">
<rewriteCodeLabelsCleanUp()>
<if(backtracking)>
}
<endif>
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{ANTLRRewriteRuleTokenStream *_stream_<it>=[[ANTLRRewriteRuleTokenStream alloc] initWithTreeAdaptor:treeAdaptor description:@"token <it>" element:_<it>];};
separator="\n"
>
<referencedTokenListLabels
:{ANTLRRewriteRuleTokenStream *_stream_<it>=[[ANTLRRewriteRuleTokenStream alloc] initWithTreeAdaptor:treeAdaptor description:@"token <it>" elements:_<it>_list];};
separator="\n"
>
<referencedRuleLabels
:{ANTLRRewriteRuleSubtreeStream *_stream_<it>=[[ANTLRRewriteRuleSubtreeStream alloc] initWithTreeAdaptor:treeAdaptor description:@"token <it>" element:_<it>!=nil?[_<it> tree]:nil];};
separator="\n"
>
<referencedRuleListLabels
:{ANTLRRewriteRuleSubtreeStream *_stream_<it>=[[ANTLRRewriteRuleSubtreeStream alloc] initWithTreeAdaptor:treeAdaptor description:@"token <it>" elements:_list_<it>];};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
if ( <referencedElementsDeep:{el | [_stream_<el> hasNext]}; separator="||"> ) {
<alt>
}
<referencedElementsDeep:{el | [_stream_<el> reset];<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
while ( <referencedElements:{el | [_stream_<el> hasNext]}; separator="||"> ) {
<alt>
}
<referencedElements:{el | [_stream_<el> reset];<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
{
if ( !(<referencedElements:{el | [_stream_<el> hasNext]}; separator="||">) ) {
@throw [NSException exceptionWithName:@"RewriteEarlyExitException" reason:nil userInfo:nil];
}
while ( <referencedElements:{el | [_stream_<el> hasNext]}; separator="||"> ) {
<alt>
}
<referencedElements:{el | [_stream_<el> reset];<\n>}>
}
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>) {
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = nil;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)[treeAdaptor newEmptyTree];
<root:rewriteElement()>
<children:rewriteElement()>
[treeAdaptor addChild:root_<treeLevel> toTree:root_<enclosingTreeLevel>];
[root_<treeLevel> release];
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
<if(args)>
id\<ANTLRTree> _<token>_tree = [treeAdaptor newTreeWithToken:_<token>]; // TODO: args: <args; separator=", ">
<endif>
[treeAdaptor addChild:<if(args)>_<token>_tree<else>[_stream_<token> next]<endif> toTree:root_<treeLevel>];
<if(args)>
[_<token>_tree release];<\n>
<endif>
<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
[treeAdaptor addChild:[_stream_<label> next] toTree:root_<treeLevel>];<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
[treeAdaptor addChild:[_stream_<label> next] toTree:root_<treeLevel>];<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:[_stream_<label> next] parentOf:root_<treeLevel>];<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:[_stream_<token> next] parentOf:root_<treeLevel>];<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
<if(first(rest(args)))><! got two arguments - means create from token with custom text!>
id\<ANTLRTree> _<token>_tree = [treeAdaptor newTreeWithToken:<first(args)> tokenType:<token> text:@<first(rest(args))>];
[treeAdaptor addChild:_<token>_tree toTree:root_<treeLevel>];
[_<token>_tree release];<\n>
<else><! at most one argument !>
<if(first(args))>
id\<ANTLRTree> _<token>_tree = [treeAdaptor newTreeWithToken:<first(args)> tokenType:<token>];
[treeAdaptor addChild:_<token>_tree toTree:root_<treeLevel>];
[_<token>_tree release];<\n>
<else><! no argument at all !>
id\<ANTLRTree> _<token>_tree = [treeAdaptor newTreeWithTokenType:<token> text:[tokenNames objectAtIndex:<token>]];
[treeAdaptor addChild:_<token>_tree toTree:root_<treeLevel>];
[_<token>_tree release];<\n>
<endif><! one arg !>
<endif><! two args !>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
<if(first(rest(args)))><! got two arguments - means create from token with custom text!>
id\<ANTLRTree> _<token>_tree = [treeAdaptor newTreeWithToken:<first(args)> tokenType:<token> text:@<first(rest(args))>];
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:_<token>_tree parentOf:root_<treeLevel>];
[_<token>_tree release];<\n>
<else><! at most one argument !>
<if(first(args))>
id\<ANTLRTree> _<token>_tree = [treeAdaptor newTreeWithToken:<first(args)> tokenType:<token>];
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:_<token>_tree parentOf:root_<treeLevel>];
[_<token>_tree release];<\n>
<else><! no argument at all !>id\<ANTLRTree> _<token>_tree = [treeAdaptor newTreeWithTokenType:<token> text:[tokenNames objectAtIndex:<token>]];
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:_<token>_tree parentOf:root_<treeLevel>];
[_<token>_tree release];<\n>
<endif><! one arg !>
<endif><! two args !>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
[treeAdaptor addChild:[_stream_<rule> next] toTree:root_<treeLevel>];<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:(id\<ANTLRTree>)[_stream_<rule> next] parentOf:root_<treeLevel>];<\n>
>>
rewriteNodeAction(action) ::= <<
[treeAdaptor addChild:<action> toTree:root_<treeLevel>];<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:<action> parentOf:root_<treeLevel>];<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
[treeAdaptor addChild:[_<label> tree] toTree:root_<treeLevel>];<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
[treeAdaptor addChild:[(ANTLR<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope *)[_stream_<label> next] tree] toTree:root_<treeLevel>];<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:[_<label> tree] parentOf:root_<treeLevel>];<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)[treeAdaptor makeNode:[(ANTLR<if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope *)[_stream_<label> next] tree] parentOf:root_<treeLevel>];<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
new <hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
(<ASTLabelType>)adaptor.create(<tokenType>, <args; separator=", "><if(!args)>"<tokenType>"<endif>)
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.nextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
adaptor.create(<token>, <args; separator=", ">)
<else>
stream_<token>.nextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ObjC/ASTDbg.stg 0000644 0001750 0001750 00000003546 11256465234 027067 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group ASTDbg;
/*
parserMembers() ::= <<
protected TreeAdaptor adaptor =
new DebugTreeAdaptor(dbg,new CommonTreeAdaptor());
public void setTreeAdaptor(TreeAdaptor adaptor) {
this.adaptor = new DebugTreeAdaptor(dbg,adaptor);
}
public TreeAdaptor getTreeAdaptor() {
return adaptor;
}<\n>
>>
*/
@treeParserHeaderFile.superClassName ::= "ANTLRDebugTreeParser"
@rewriteElement.pregen() ::= "[debugListener locationLine:<e.line> column:<e.pos>];"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/ObjC/ObjC.stg 0000644 0001750 0001750 00000131107 11256465234 026633 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006,2007 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
Template group file for the Objective C code generator.
Heavily based on Java.stg
Written by Kay Roepke <kroepke(at)classdump.org>
This file is part of ANTLR and subject to the same license as ANTLR itself.
*/
group ObjC implements ANTLRCore;
objcTypeInitMap ::= [
"int":"0",
"long":"0",
"float":"0.0",
"double":"0.0",
"boolean":"false",
"byte":"0",
"short":"0",
"char":"0",
"id":"nil",
default:"nil" // anything other than an atomic type
]
className() ::= "<name><! if(LEXER)>Lexer<else><if(TREE_PARSER)>Tree<endif>Parser<endif !>"
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass,literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<@imports>
#import "<name><!if(LEXER)>Lexer<else><if(TREE_PARSER)>Tree<endif>Parser<endif!>.h"
<@end>
<docComment>
<recognizer>
>>
headerFile(LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass,literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
<@imports>
#import \<Cocoa/Cocoa.h>
#import \<ANTLR/ANTLR.h>
<@end>
<actions.(actionScope).header>
<if(LEXER)>
<lexerHeaderFile(...)>
<endif>
<if(PARSER)>
<parserHeaderFile(...)>
<endif>
<if(TREE_PARSER)>
<treeParserHeaderFile(...)>
<endif>
>>
lexerHeaderFile() ::=
<<
<cyclicDFAs:cyclicDFAInterface()>
#pragma mark Rule return scopes start
<rules:{rule |
<rule:{ruleDescriptor | <returnScopeInterface(scope=ruleDescriptor.returnScope)>}>
}>
#pragma mark Rule return scopes end
#pragma mark Tokens
<tokens:{#define <name>_<it.name> <it.type>}; separator="\n">
@interface <className()> : ANTLRLexer {
<cyclicDFAs:{dfa | <name>DFA<dfa.decisionNumber> *dfa<dfa.decisionNumber>;}; separator="\n">
<synpreds:{p | SEL <p>SyntacticPredicate;}; separator="\n">
<actions.lexer.ivars>
}
<actions.lexer.methodsdecl>
<rules:{rule |
- (<rule.ruleDescriptor:{ruleDescriptor|<returnType()>}>) <if(!rule.ruleDescriptor.isSynPred)>m<endif><rule.ruleName><if(rule.ruleDescriptor.parameterScope)><rule.ruleDescriptor.parameterScope:parameterScope(scope=it)><endif>;
}>
@end
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="id<ANTLRToken> ",
filterMode) ::= <<
<cyclicDFAs:cyclicDFA()>
/** As per Terence: No returns for lexer rules!
#pragma mark Rule return scopes start
<rules:{rule |
<rule.ruleDescriptor:{ruleDescriptor | <returnScope(scope=ruleDescriptor.returnScope)>}>
}>
#pragma mark Rule return scopes end
*/
@implementation <className()>
static NSArray *tokenNames;
<actions.lexer.methods>
+ (void) initialize
{
// todo: get tokenNames into lexer - requires changes to CodeGenerator.java and ANTLRCore.sti
tokenNames = [[NSArray alloc] init];
}
- (id) initWithCharStream:(id\<ANTLRCharStream>)anInput
{
if (nil!=(self = [super initWithCharStream:anInput])) {
<if(memoize)>
// initialize the memoization cache - the indices are 1-based in the runtime code!
[ruleMemo addObject:[NSNull null]]; // dummy entry to ensure 1-basedness.
for (int i = 0; i \< <numRules>; i++) {
[[state ruleMemo] addObject:[NSMutableDictionary dictionary]];
}
<endif>
<synpreds:{p | <lexerSynpred(name=p)>};separator="\n">
<cyclicDFAs:{dfa | dfa<dfa.decisionNumber> = [[<name>DFA<dfa.decisionNumber> alloc] initWithRecognizer:self];}; separator="\n">
<actions.lexer.init>
}
return self;
}
- (void) dealloc
{
<cyclicDFAs:{dfa | [dfa<dfa.decisionNumber> release];}; separator="\n">
<actions.lexer.dealloc>
[super dealloc];
}
+ (NSString *) tokenNameForType:(int)aTokenType
{
return nil;
}
+ (NSArray *) tokenNames
{
return tokenNames;
}
- (NSString *) grammarFileName
{
return @"<fileName>";
}
<if(actions.lexer.reset)>
- (void) reset
{
<actions.lexer.reset>
[super reset];
}
<endif>
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
@end
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
*/
filteringNextToken() ::= <<
- (id\<ANTLRToken>) nextToken
{
<sharedStateLocalVarDefinition()>
while (YES) {
if ( [input LA:1] == ANTLRCharStreamEOF ) {
return nil; // should really be a +eofToken call here -> go figure
}
[self setToken:nil];
[_state setChannel:ANTLRTokenChannelDefault];
[_state setTokenStartLine:[input line]];
[_state setTokenCharPositionInLine:[input charPositionInLine]];
[_state setTokenStartCharIndex:[self charIndex]];
@try {
int m = [input mark];
[_state setBacktracking:1];
[_state setIsFailed:NO];
[self mTokens];
[_state setBacktracking:0];
if ( [_state isFailed] ) {
[input rewind:m];
[input consume]; <! // advance one char and try again !>
} else {
[self emit];
return token;
}
}
@catch (ANTLRRecognitionException *re) {
// shouldn't happen in backtracking mode, but...
[self reportError:re];
[self recover:re];
}
}
}
>>
filteringActionGate() ::= "[_state backtracking] == 1"
treeParserHeaderFile(LEXER, PARSER, TREE_PARSER, actionScope, actions, docComment,
recognizer, name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, profile, backtracking, synpreds,
memoize, numRules, fileName, ANTLRVersion, generatedTimestamp, trace, scopes, superClass="ANTLRTreeParser") ::= <<
<parserHeaderFile(...)>
>>
parserHeaderFile(LEXER, PARSER, TREE_PARSER, actionScope, actions, docComment,
recognizer, name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, profile, backtracking, synpreds,
memoize, numRules, fileName, ANTLRVersion, generatedTimestamp, trace, scopes, superClass="ANTLRParser") ::=
<<
<cyclicDFAs:cyclicDFAInterface()>
#pragma mark Tokens
<tokens:{#define <name>_<it.name> <it.type>}; separator="\n">
#pragma mark Dynamic Global Scopes
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeDecl(scope=it)><endif>}>
#pragma mark Dynamic Rule Scopes
<rules:{rule |
<rule.ruleDescriptor:{ ruleDescriptor | <ruleAttributeScopeDecl(scope=ruleDescriptor.ruleScope)>}>
}>
#pragma mark Rule Return Scopes
<rules:{rule |
<rule.ruleDescriptor:{ ruleDescriptor | <returnScopeInterface(scope=ruleDescriptor.returnScope)>}>
}>
@interface <className()> : <@superClassName><superClass><@end> {
<cyclicDFAs:{dfa | <name>DFA<dfa.decisionNumber> *dfa<dfa.decisionNumber>;}; separator="\n">
<synpreds:{p | SEL <p>SyntacticPredicate;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeIVar(scope=it)><endif>}>
<rules:{rule |
<rule.ruleDescriptor.ruleScope:ruleAttributeScopeIVar(scope=it)>
}>
<@ivars()>
<actions.(actionScope).ivars>
}
<actions.(actionScope).methodsdecl>
<rules:{rule |
- (<rule.ruleDescriptor:{ruleDescriptor|<returnType()>}>) <rule.ruleName><if(rule.ruleDescriptor.parameterScope)><rule.ruleDescriptor.parameterScope:parameterScope(scope=it)><endif>;
}>
<@methodsdecl()>
@end
>>
/** How to generate a parser */
genericParser(name, scopes, tokens, tokenNames, rules, numRules,
cyclicDFAs, bitsets, inputStreamType, superClass,
ASTLabelType="id", labelType, members) ::= <<
<cyclicDFAs:cyclicDFA()>
#pragma mark Bitsets
<bitsets:bitset(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>}, words64=it.bits)>
#pragma mark Dynamic Global Scopes
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeImpl(scope=it)><endif>}>
#pragma mark Dynamic Rule Scopes
<rules:{rule |
<rule.ruleDescriptor.ruleScope:ruleAttributeScopeImpl(scope=it)>
}>
#pragma mark Rule return scopes start
<rules:{rule |
<rule.ruleDescriptor:{ruleDescriptor | <returnScope(scope=ruleDescriptor.returnScope)>}>
}>
@implementation <className()>
static NSArray *tokenNames;
+ (void) initialize
{
<bitsets:bitsetInit(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>}, words64=it.bits)>
tokenNames = [[NSArray alloc] initWithObjects:@"\<invalid>", @"\<EOR>", @"\<DOWN>", @"\<UP>", <tokenNames:{@<it>}; separator=", ", wrap="\n ">, nil];
}
<if(PARSER)>
- (id) initWithTokenStream:(<inputStreamType>)aStream
{
if ((self = [super initWithTokenStream:aStream])) {
<else><!TREE_PARSER!>
- (id) initWithTreeNodeStream:(<inputStreamType>)aStream
{
if ((self = [super initWithTreeNodeStream:aStream])) {
<endif><\n>
<if(memoize)>
ruleMemo = [[NSMutableDictionary alloc] initWithCapacity:<numRules>+1];
<endif>
<cyclicDFAs:{dfa | dfa<dfa.decisionNumber> = [[<name>DFA<dfa.decisionNumber> alloc] initWithRecognizer:self];}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeInit(scope=it)><endif>}>
<rules:{rule |
<rule.ruleDescriptor.ruleScope:ruleAttributeScopeInit(scope=it)>
}>
<actions.(actionScope).init>
<@init()>
}
return self;
}
- (void) dealloc
{
<if(memoize)>
[ruleMemo release];
<endif>
<cyclicDFAs:{dfa | [dfa<dfa.decisionNumber> release];}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeDealloc(scope=it)><endif>}>
<actions.(actionScope).dealloc>
<@dealloc()>
[super dealloc];
}
- (NSString *) grammarFileName
{
return @"<fileName>";
}
<actions.(actionScope).methods>
<rules; separator="\n\n">
<synpreds:{p | <synpred(p)>}>
<@methods()>
@end
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets, ASTLabelType, superClass="ANTLRParser", labelType="id<ANTLRToken> ", members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="id\<ANTLRTokenStream>", ...)>
>>
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules, numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="id", superClass="ANTLRTreeParser", members={<actions.treeparser.members>}) ::= <<
<genericParser(inputStreamType="id\<ANTLRTreeNodeStream>", ...)>
>>
/** Maintain a local variable for the shared state object to avoid calling the accessor all the time. */
sharedStateLocalVarDefinition() ::= <<
<if(LEXER)>ANTLRLexerState<else>ANTLRBaseRecognizerState<endif> *_state = [self state];
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
- (void) <ruleName>
{
<sharedStateLocalVarDefinition()>
<if(trace)>NSLog(@"enter <ruleName> %d failed=%@ backtracking=%d", [input LA:1], [_state isFailed] ? @"YES" : @"NO", [_state backtracking]);
@try {
<block>
}
@finally {
NSLog(@"exit <ruleName> %d failed=%@ backtracking=%d", [input LA:1], [_state isFailed] ? @"YES" : @"NO", [_state backtracking]);
}
<else>
<block>
<endif>
}
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>if ([_state isFailed]) return <ruleReturnValue()>;<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if ([_state isBacktracking]) {[_state setIsFailed:YES]; return <ruleReturnValue()>;}<endif>
>>
synpred(name) ::= <<
<!name>SyntacticPredicate = @selector(<name>); !>
>>
lexerSynpred(name) ::= <<
<synpred(name)>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ([_state isBacktracking] && [self alreadyParsedRule:input ruleIndex:<ruleDescriptor.index>] ) { return <ruleReturnValue()>; }
<endif>
>>
/** How to generate code for a rule.
* The return type aggregates are declared in the header file (headerFile template)
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
// $ANTLR start <ruleName>
// <fileName>:<description>
- (<returnType()>) <ruleName><ruleDescriptor.parameterScope:parameterScope(scope=it)>
{
<if(trace)>NSLog(@"enter <ruleName> %@ failed=%@ backtracking=%d", [input LT:1], [_state isFailed] ? @"YES" : @"NO", [_state backtracking]);<endif>
<sharedStateLocalVarDefinition()>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
<ruleMemoization(name=ruleName)>
@try {
<block>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
@catch (ANTLRRecognitionException *re) {
[self reportError:re];
[self recover:input exception:re];
}<\n>
<endif>
<endif>
<endif>
@finally {
<if(trace)>NSLog(@"exit <ruleName> %@ failed=%@ backtracking=%d", [input LT:1], [_state isFailed] ? @"YES" : @"NO", [_state backtracking]);<endif>
<ruleCleanUp()>
<(ruleDescriptor.actions.finally):execAction()>
<ruleScopeCleanUp()>
}
<@postamble()>
return <ruleReturnValue()>;
}
// $ANTLR end <ruleName>
>>
catch(decl,action) ::= <<
catch (<e.decl>) {
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnType()> _retval = [[[<returnTypeName()> alloc] init] autorelease];
[_retval setStart:[input LT:1]];<\n>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<returnType()> _<ruleDescriptor.singleValueReturnName> = <initValue(typeName=returnType())>;
<endif>
<endif>
<if(memoize)>
int <ruleDescriptor.name>_StartIndex = [input index];
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{[<name>_<it>_stack addObject:[[[<name><it>Scope alloc] init] autorelease]];}; separator="\n">
<ruleDescriptor.ruleScope:{[<name>_<it.name>_stack addObject:[[[<name><it.name>Scope alloc] init] autorelease]];}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{[<name>_<it>_stack removeLastObject];}; separator="\n">
<ruleDescriptor.ruleScope:{[<name>_<it.name>_stack removeLastObject];}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]
:{<labelType> _<it.label.text> = nil;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]
:{NSMutableArray *_list_<it.label.text> = nil;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:{ll|ANTLRRuleReturnScope <ll.label.text> = nil;}; separator="\n">
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
_<ruleDescriptor.singleValueReturnName>
<else>
_retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
// token+rule list labels
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]:{[_list_<it.label.text> release];}; separator="\n">
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
[_retval setStop:[input LT:-1]];<\n>
<endif>
<endif>
<if(memoize)>
<if(backtracking)>
if ( [_state isBacktracking] ) { [self memoize:input ruleIndex:<ruleDescriptor.index> startIndex:<ruleDescriptor.name>_StartIndex]; }
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block, memoize) ::= <<
- (void) m<ruleName><if(ruleDescriptor.parameterScope)><ruleDescriptor.parameterScope:parameterScope(scope=it)><endif>
{
<if(trace)>NSLog(@"enter <ruleName> %C line=%d:%d failed=%@ backtracking=%d", [input LA:1], [self line], [self charPositionInLine], [_state isFailed] ? @"YES" : @"NO", [_state backtracking]);<endif>
<sharedStateLocalVarDefinition()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleMemoization(name=ruleName)>
@try {
<ruleDescriptor.actions.init>
<if(nakedBlock)>
<block><\n>
<else>
int _type = <name>_<ruleName>;
<block>
[_state setTokenType:_type];<\n>
<endif>
}
@finally {
<if(trace)>NSLog(@"exit <ruleName> %C line=%d:%d failed=%@ backtracking=%d", [input LA:1], [self line], [self charPositionInLine], [_state isFailed] ? @"YES" : @"NO", [_state backtracking]);<endif>
// rule cleanup
<ruleCleanUp()>
<(ruleDescriptor.actions.finally):execAction()>
}
return;
}
// $ANTLR end <ruleName>
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
- (void) mTokens
{
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description> // block
int alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description> //ruleblock
int alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description> // ruleBlockSingleAlt
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description> // blockSingleAlt
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 0 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description> // positiveClosureBlock
int cnt<decisionNumber>=0;
<decls>
<@preloop()>
do {
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
if ( cnt<decisionNumber> >= 1 ) goto loop<decisionNumber>;
<if(backtracking)>
<ruleBacktrackFailure()>
<endif>
ANTLREarlyExitException *eee = [ANTLREarlyExitException exceptionWithStream:input decisionNumber:<decisionNumber>];
<@earlyExitException()>
@throw eee;
}
cnt<decisionNumber>++;
} while (YES); loop<decisionNumber>: ;
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 0 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
<decls>
<@preloop()>
do {
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>) {
<alts:altSwitchCase()>
default :
goto loop<decisionNumber>;
}
} while (YES); loop<decisionNumber>: ;
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i> :
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel) ::= <<
// <fileName>:<description> // alt
{
<@declarations()>
<elements:element()>
<@cleanup()>
}
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)>
_<label>=(<labelType>)[input LT:1];<\n>
<endif>
[self match:input tokenType:<token> follow:FOLLOW_<token>_in_<ruleName><elementIndex>]; <checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (_list_<label> == nil)
_list_<label> = [[NSMutableArray alloc] init];
[_list_<label> addObject:_<elem>];
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
int _<label> = [input LA:1];<\n>
<endif>
[self matchChar:<char>];
<checkRuleBacktrackFailure()><\n>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
_<label> = [input LA:1];<\n>
<endif>
[self matchRangeFromChar:<a> to:<b>];<checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
_<label> = (<labelType>)[input LT:1];<\n>
<endif>
if (<s>) {
<postmatchCode>
[input consume];
<if(!LEXER)>
[_state setIsErrorRecovery:NO];
<endif>
<if(backtracking)>[_state setIsFailed:NO];<endif>
} else {
<ruleBacktrackFailure()>
ANTLRMismatchedSetException *mse = [ANTLRMismatchedSetException exceptionWithSet:nil stream:input];
<@mismatchedSetException()>
<if(LEXER)>
[self recover:mse];
<else>
[self recoverFromMismatchedSet:input exception:mse follow:FOLLOW_set_in_<ruleName><elementIndex>];
<endif>
@throw mse;
}<\n>
>>
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
int _<label>Start = [self charIndex];
[self matchString:<string>];
<checkRuleBacktrackFailure()>
_<label> = [[ANTLRCommonToken alloc] initWithInput:input tokenType:ANTLRTokenTypeInvalid channel:ANTLRTokenChannelDefault start:_<label>Start stop:[self charIndex]];
<else>
[self matchString:<string>];
<checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
_<label> = (<labelType>)[input LT:1];<\n>
<endif>
[self matchAny:input];
<checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
int _<label> = [input LA:1];<\n>
<endif>
[self matchAny];
<checkRuleBacktrackFailure()><\n>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
[[_state following] addObject:FOLLOW_<rule.name>_in_<ruleName><elementIndex>];
<if(label)>
_<label> = [self <rule.name><if(args)>:<args; separator=" :"><endif>];<\n>
<else>
[self <rule.name><if(args)>:<args; separator=" :"><endif>];<\n>
<endif>
[[_state following] removeLastObject];
<checkRuleBacktrackFailure()><\n>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(...)>
>>
/** A lexer rule reference */
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
int _<label>Start<elementIndex> = [self charIndex];
[self m<rule.name><if(args)>:<args; separator=" :"><endif>];
<checkRuleBacktrackFailure()><\n>
_<label> = [[ANTLRCommonToken alloc] initWithInput:input tokenType:ANTLRTokenTypeInvalid channel:ANTLRTokenChannelDefault start:_<label>Start<elementIndex> stop:[self charIndex]-1];
[_<label> setLine:[self line]];
<else>
[self m<rule.name><if(args)>:<args; separator=" :"><endif>];
<checkRuleBacktrackFailure()><\n>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
int _<label>Start<elementIndex> = [self charIndex];
[self matchChar:ANTLRCharStreamEOF];
<checkRuleBacktrackFailure()><\n>
<labelType> _<label> = [[ANTLRCommonToken alloc] initWithInput:input tokenType:ANTLRTokenTypeEOF channel:ANTLRTokenChannelDefault start:_<label>Start<elementIndex> stop:[self charIndex]-1];
[_<label> setLine:[self line]];
<else>
[self matchChar:ANTLRCharStreamEOF];
<checkRuleBacktrackFailure()><\n>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( [input LA:1] == ANTLRTokenTypeDOWN ) {
[self match:input tokenType:ANTLRTokenTypeDOWN follow:nil]; <checkRuleBacktrackFailure()>
<children:element()>
[self match:input tokenType:ANTLRTokenTypeUP follow:nil]; <checkRuleBacktrackFailure()>
}
<else>
[self match:input tokenType:ANTLRTokenTypeDOWN follow:nil]; <checkRuleBacktrackFailure()>
<children:element()>
[self match:input tokenType:ANTLRTokenTypeUP follow:nil]; <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) ) {
<ruleBacktrackFailure()>
@throw [ANTLRFailedPredicateException exceptionWithRuleName:@"<ruleName>" predicate:@"<description>" stream:input];
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
{
int LA<decisionNumber>_<stateNumber> = [input LA:<k>];
<edges; separator="\nelse ">
else {
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
ANTLRNoViableAltException *nvae = [ANTLRNoViableAltException exceptionWithDecision:<decisionNumber> state:<stateNumber> stream:input];
<@noViableAltException()>
@throw nvae;<\n>
<endif>
}
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
{
int LA<decisionNumber>_<stateNumber> = [input LA:<k>];
<edges; separator="\nelse ">
}
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
{
int LA<decisionNumber>_<stateNumber> = [input LA:<k>];
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
else {
alt<decisionNumber> = <eotPredictsAlt>;
}<\n>
<endif>
}
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber> = <alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( <labelExpr> <if(predicates)>&& <predicates><endif>) {
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ([input LA:<k>]) {
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber> = <eotPredictsAlt>;
<else> {
<ruleBacktrackFailure()>
ANTLRNoViableAltException *nvae = [ANTLRNoViableAltException exceptionWithDecision:<decisionNumber> state:<stateNumber> stream:input];
<@noViableAltException()>
@throw nvae;<\n>
}
<endif>
}
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ([input LA:<k>]) {
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ([input LA:<k>]) {
<edges; separator="\n">
<if(eotPredictsAlt)>
default:
alt<decisionNumber> = <eotPredictsAlt>
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
<targetState>
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = [dfa<decisionNumber> predict];
>>
/** Used in headerFile */
cyclicDFAInterface(dfa) ::= <<
#pragma mark Cyclic DFA interface start <name>DFA<dfa.decisionNumber>
@interface <name>DFA<dfa.decisionNumber> : ANTLRDFA {} @end<\n>
#pragma mark Cyclic DFA interface end <name>DFA<dfa.decisionNumber>
>>
/** Used in lexer/parser implementation files */
cyclicDFA(dfa) ::= <<
#pragma mark Cyclic DFA implementation start <name>DFA<dfa.decisionNumber>
@implementation <name>DFA<dfa.decisionNumber>
const static int <name>dfa<dfa.decisionNumber>_eot[<dfa.numberOfStates>] =
{<dfa.eot; wrap="\n ", separator=",", null="-1">};
const static int <name>dfa<dfa.decisionNumber>_eof[<dfa.numberOfStates>] =
{<dfa.eof; wrap="\n ", separator=",", null="-1">};
const static unichar <name>dfa<dfa.decisionNumber>_min[<dfa.numberOfStates>] =
{<dfa.min; wrap="\n ", separator=",", null="0">};
const static unichar <name>dfa<dfa.decisionNumber>_max[<dfa.numberOfStates>] =
{<dfa.max; wrap="\n ", separator=",", null="0">};
const static int <name>dfa<dfa.decisionNumber>_accept[<dfa.numberOfStates>] =
{<dfa.accept; wrap="\n ", separator=",", null="-1">};
const static int <name>dfa<dfa.decisionNumber>_special[<dfa.numberOfStates>] =
{<dfa.special; wrap="\n ", separator=",", null="-1">};
const static int <name>dfa<dfa.decisionNumber>_transition[] = {};
<dfa.edgeTransitionClassMap.keys:{ table |
const static int <name>dfa<dfa.decisionNumber>_transition<i0>[] = {<table; separator=", ", wrap="\n ", null="-1">};
}; null="">
- (id) initWithRecognizer:(ANTLRBaseRecognizer *) theRecognizer
{
if ((self = [super initWithRecognizer:theRecognizer]) != nil) {
eot = <name>dfa<dfa.decisionNumber>_eot;
eof = <name>dfa<dfa.decisionNumber>_eof;
min = <name>dfa<dfa.decisionNumber>_min;
max = <name>dfa<dfa.decisionNumber>_max;
accept = <name>dfa<dfa.decisionNumber>_accept;
special = <name>dfa<dfa.decisionNumber>_special;
if (!(transition = calloc(<dfa.numberOfStates>, sizeof(void*)))) {
[self release];
return nil;
}
<dfa.transitionEdgeTables:{whichTable|transition[<i0>] = <name>dfa<dfa.decisionNumber>_transition<whichTable>;}; separator="\n", null="">
}
return self;
}
<if(dfa.specialStateSTs)>
- (int) specialStateTransition:(int) s
{
int _s = s;
switch ( s ) {
<dfa.specialStateSTs:{state |
case <i0> : <! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
<if(backtracking)>
if ([recognizer isBacktracking]) {
[recognizer setIsFailed:YES];
return -1;
}<\n>
<endif>
ANTLRNoViableAltException *nvae = [ANTLRNoViableAltException exceptionWithDecision:<dfa.decisionNumber> state:_s stream:[recognizer input]];
<! [self error:nvae]; !> <! for debugger - do later !>
@throw nvae;
}<\n>
<endif>
- (void) dealloc
{
free(transition);
[super dealloc];
}
- (NSString *) description
{
return @"<dfa.description>";
}
<@errorMethod()>
@end
#pragma mark Cyclic DFA implementation end <name>DFA<dfa.decisionNumber>
<\n>
>>
cyclicDFAState(decisionNumber, stateNumber,edges,needErrorClause,semPredState) ::= <<
<if(semPredState)>
[[recognizer input] rewind];<\n>
<else>
int LA<decisionNumber>_<stateNumber> = [input LA:1];<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
if ( s>=0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {s = <targetStateNumber>;}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left>&&<right>)"
orPredicates(operands) ::= "(first(operands)<rest(operands):{o | ||<o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "<pred>"
/** synpreds are b0rken in cyclic DFA special states
* Damn! For now, work around with using the selectors directly, and by providing a trampoline evalSynPred method in
* ANTLRDFA
*/
/* evalSynPredicate(pred,description) ::= "[self evaluateSyntacticPredicate:<pred>SyntacticPredicate stream:input]" */
evalSynPredicate(pred,description) ::= "[self evaluateSyntacticPredicate:@selector(<pred>)]"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber>==<atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "[input LA:<k>]==<atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber>\>=<lower> && LA<decisionNumber>_<stateNumber>\<=<upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "([input LA:<k>]\>=<lower> && [input LA:<k>]\<=<upper>)"
setTest(ranges) ::= "<ranges; separator=\"||\">"
// A T T R I B U T E S
globalAttributeScopeDecl(scope) ::= <<
@interface <name><scope.name>Scope : NSObject {
<scope.attributes:{<it.decl>;}; separator="\n">
}
// use KVC to access attributes!
@end
>>
globalAttributeScopeIVar(scope) ::= <<
NSMutableArray *<name>_<scope.name>_stack;
>>
globalAttributeScopeImpl(scope) ::= <<
@implementation <name><scope.name>Scope
@end
>>
globalAttributeScopeInit(scope) ::= <<
<name>_<scope.name>_stack = [[NSMutableArray alloc] init];
>>
globalAttributeScopeDealloc(scope) ::= <<
[<name>_<scope.name>_stack release];
>>
ruleAttributeScopeDecl(scope) ::= <<
<if(scope.attributes)>
@interface <name><scope.name>Scope : NSObject {
<scope.attributes:{<it.decl>;}; separator="\n">
}
// use KVC to access attributes!
@end
<endif>
>>
ruleAttributeScopeIVar(scope) ::= <<
NSMutableArray *<name>_<scope.name>_stack;
>>
ruleAttributeScopeImpl(scope) ::= <<
<if(scope.attributes)>
@implementation <name><scope.name>Scope
@end
<endif>
>>
ruleAttributeScopeInit(scope) ::= <<
<name>_<scope.name>_stack = [[NSMutableArray alloc] init];
>>
ruleAttributeScopeDealloc(scope) ::= <<
[<name>_<scope.name>_stack release];
>>
returnTypeName() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasMultipleReturnValues)>
<className()>_<ruleDescriptor.name>_return
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
<else>
void<! for synpreds this is always void !>
<endif>
>>
returnType() ::= <<
<returnTypeName()><if(!ruleDescriptor.isSynPred)><if(ruleDescriptor.hasMultipleReturnValues)> *<endif><endif>
>>
/** Generate the Objective-C type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<className()>_<referencedRule.name>_return *
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
<objcTypeInitMap.(typeName)>
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<ruleLabelType(referencedRule=label.referencedRule)> _<label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScopeInterface(scope) ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasMultipleReturnValues)>
@interface <returnTypeName()> : ANTLR<if(TREE_PARSER)>Tree<endif>ParserRuleReturnScope {
<scope.attributes:{<it.decl>;}; separator="\n">
<@ivars()>
<actions.(actionScope).ruleReturnIvars>
}
<@methods()>
<actions.(actionScope).ruleReturnMethodsDecl>
@end
<endif>
<endif>
>>
returnScope(scope) ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasMultipleReturnValues)>
@implementation <returnTypeName()>
<@methods()>
<actions.(actionScope).ruleReturnMethods>
@end
<endif>
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{:(<it.type>)<it.name>}; separator=" ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> =<expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
[<scopeAttributeRefStack()> valueForKey:@"<attr.name>"]
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
[<scopeAttributeRefStack()> setValue:<expr> forKey:@"<attr.name>"];
>>
scopeAttributeRefStack() ::= <<
<if(negIndex)>
[<name>_<scope>_stack objectAtIndex:[<name>_<scope>_stack count]-<negIndex>-1]
<else>
<if(index)>
[<name>_<scope>_stack objectAtIndex:<index>]
<else>
[<name>_<scope>_stack lastObject]
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<name>_<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
[<scope> valueForKey:@"<attr.name>"]
<else>
_<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
[_retval setValue: forKey:@"<attr.name>"];
<else>
_<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
[_retval setValue:<expr> forKey:@"<attr.name>"];
<else>
_<attr.name> =<expr>;
<if(LEXER)>
// double check this after beta release!
[_<attr.name> retain];
<endif>
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "_<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "_list_<label>"
// not sure the next are the right approach; and they are evaluated early;
// they cannot see TREE_PARSER or PARSER attributes for example. :(
tokenLabelPropertyRef_text(scope,attr) ::= "[_<scope> text]"
tokenLabelPropertyRef_type(scope,attr) ::= "[_<scope> type]"
tokenLabelPropertyRef_line(scope,attr) ::= "[_<scope> line]"
tokenLabelPropertyRef_pos(scope,attr) ::= "[_<scope> charPositionInLine]"
tokenLabelPropertyRef_channel(scope,attr) ::= "[_<scope> channel]"
tokenLabelPropertyRef_index(scope,attr) ::= "[_<scope> tokenIndex]"
tokenLabelPropertyRef_tree(scope,attr) ::= "_<scope>_tree"
ruleLabelPropertyRef_start(scope,attr) ::= "[_<scope> start]"
ruleLabelPropertyRef_stop(scope,attr) ::= "[_<scope> stop]"
ruleLabelPropertyRef_tree(scope,attr) ::= "[_<scope> tree]"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
//[input textForNode:[_<scope> start]]
[input substringWithRange:NSMakeRange([[input treeAdaptor] startIndex:[_<scope> start]], [[input treeAdaptor] stopIndex:[_<scope> start]])]
<else>
[input substringWithRange:NSMakeRange([_<scope> start], [_<scope> stop]-[_<scope> start])]
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "[_<scope> st]"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "_<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "[_<scope> type]"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "[_<scope> line]"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "[_<scope> charPositionInLine]"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "[_<scope> channel]"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "[_<scope> tokenIndex]"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "[_<scope> text]"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "[_retval start]"
rulePropertyRef_stop(scope,attr) ::= "[_retval stop]"
rulePropertyRef_tree(scope,attr) ::= "[_retval tree]"
rulePropertyRef_text(scope,attr) ::= "[input substringWithRange:NSMakeRange(_start, [input index]-_start)]"
rulePropertyRef_st(scope,attr) ::= "[_retval st]"
ruleSetPropertyRef_tree(scope,attr,expr) ::= "[_retval setValue:<expr> forKey:@\"tree\"]"
ruleSetPropertyRef_st(scope,attr,expr) ::= "<\n>#error StringTemplates are unsupported<\n>"
/* hideous: find a way to cut down on the number of templates to support read/write access */
/* TODO: also, which ones are valid to write to? ask Ter */
lexerRuleSetPropertyRef_text(scope,attr,expr) ::= "[_state setText:<expr>];"
lexerRuleSetPropertyRef_type(scope,attr,expr) ::= "_type"
lexerRuleSetPropertyRef_line(scope,attr,expr) ::= "[_state tokenStartLine]"
lexerRuleSetPropertyRef_pos(scope,attr,expr) ::= "[_state tokenCharPositionInLine]"
lexerRuleSetPropertyRef_index(scope,attr,expr) ::= "-1" // undefined token index in lexer
lexerRuleSetPropertyRef_channel(scope,attr,expr) ::= "[_state setChannel:<expr>];"
lexerRuleSetPropertyRef_start(scope,attr,expr) ::= "[_state tokenStartCharIndex]"
lexerRuleSetPropertyRef_stop(scope,attr,expr) ::= "([self charIndex]-1)"
lexerRulePropertyRef_text(scope,attr) ::= "[self text]"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "[_state tokenStartLine]"
lexerRulePropertyRef_pos(scope,attr) ::= "[_state tokenCharPositionInLine]"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "[_state channel]"
lexerRulePropertyRef_start(scope,attr) ::= "[_state tokenStartCharIndex]"
lexerRulePropertyRef_stop(scope,attr) ::= "([self charIndex]-1)"
/** How to execute an action */
execAction(action) ::= <<
<if(backtracking)>
<if(actions.(actionScope).synpredgate)>
if ( <actions.(actionScope).synpredgate> ) {
<action>
}
<else>
if ( ![_state isBacktracking] ) {
<action>
}
<endif>
<else>
<action>
<endif>
>>
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
const static unsigned long long <name>_data[] = {<words64:{<it>LL};separator=",">};
static ANTLRBitSet *<name>;
<! ANTLRBitSet *<name> = [[ANTLRBitSet alloc] initWithBits:<name>_data count:<length(words64)>];<\n> !>
>>
bitsetInit(name, words64) ::= <<
<name> = [[ANTLRBitSet alloc] initWithBits:<name>_data count:<length(words64)>];<\n>
>>
codeFileExtension() ::= ".m"
headerFileExtension() ::= ".h"
true() ::= "YES"
false() ::= "NO"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/C/ 0000755 0001750 0001750 00000000000 11410174107 024625 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/C/C.stg 0000644 0001750 0001750 00000302306 11256465232 025544 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
http://www.temporal-wave.com
http://www.linkedin.com/in/jimidle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This code generating template and the associated C runtime was produced by:
* Jim Idle jimi|hereisanat|idle|dotgoeshere|ws.
* If it causes the destruction of the Universe, it will be pretty cool so long as
* I am in a different one at the time.
*/
group C implements ANTLRCore ;
cTypeInitMap ::= [
"int" : "0", // Integers start out being 0
"long" : "0", // Longs start out being 0
"float" : "0.0", // Floats start out being 0
"double" : "0.0", // Doubles start out being 0
"ANTLR3_BOOLEAN" : "ANTLR3_FALSE", // Booleans start out being Antlr C for false
"byte" : "0", // Bytes start out being 0
"short" : "0", // Shorts start out being 0
"char" : "0", // Chars start out being 0
default : "NULL" // Anything other than an atomic type (above) is a NULL (probably NULL pointer).
]
leadIn(type) ::=
<<
/** \file
* This <type> file was generated by $ANTLR version <ANTLRVersion>
*
* - From the grammar source file : <fileName>
* - On : <generatedTimestamp>
<if(LEXER)>
* - for the lexer : <name>Lexer
<endif>
<if(PARSER)>
* - for the parser : <name>Parser
<endif>
<if(TREE_PARSER)>
* - for the tree parser : <name>TreeParser
<endif>
*
* Editing it, at least manually, is not wise.
*
* C language generator and runtime by Jim Idle, jimi|hereisanat|idle|dotgoeshere|ws.
*
*
>>
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile( LEXER,
PARSER,
TREE_PARSER,
actionScope,
actions,
docComment,
recognizer,
name,
tokens,
tokenNames,
rules,
cyclicDFAs,
bitsets,
buildTemplate,
buildAST,
rewriteMode,
profile,
backtracking,
synpreds,
memoize,
numRules,
fileName,
ANTLRVersion,
generatedTimestamp,
trace,
scopes,
superClass,
literals
) ::=
<<
<leadIn("C source")>
*/
// [The "BSD licence"]
// Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
// http://www.temporal-wave.com
// http://www.linkedin.com/in/jimidle
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// 3. The name of the author may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
// IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
// OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
// IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
// INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
// NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
// THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
<if(actions.(actionScope).header)>
/* =============================================================================
* This is what the grammar programmer asked us to put at the top of every file.
*/
<actions.(actionScope).header>
/* End of Header action.
* =============================================================================
*/
<endif>
/* -----------------------------------------
* Include the ANTLR3 generated header file.
*/
#include "<name>.h"
<actions.(actionScope).postinclude>
/* ----------------------------------------- */
<docComment>
<if(literals)>
/** String literals used by <name> that we must do things like MATCHS() with.
* C will normally just lay down 8 bit characters, and you can use L"xxx" to
* get wchar_t, but wchar_t is 16 bits on Windows, which is not UTF32 and so
* we perform this little trick of defining the literals as arrays of UINT32
* and passing in the address of these.
*/
<literals:{static ANTLR3_UCHAR lit_<i>[] = <it>;}; separator="\n">
<endif>
/* MACROS that hide the C interface implementations from the
* generated code, which makes it a little more understandable to the human eye.
* I am very much against using C pre-processor macros for function calls and bits
* of code as you cannot see what is happening when single stepping in debuggers
* and so on. The exception (in my book at least) is for generated code, where you are
* not maintaining it, but may wish to read and understand it. If you single step it, you know that input()
* hides some indirect calls, but is always referring to the input stream. This is
* probably more readable than ctx->input->istream->input(snarfle0->blarg) and allows me to rejig
* the runtime interfaces without changing the generated code too often, without
* confusing the reader of the generated output, who may not wish to know the gory
* details of the interface inheritance.
*/
#define CTX ctx
/* Aids in accessing scopes for grammar programmers
*/
#undef SCOPE_TYPE
#undef SCOPE_STACK
#undef SCOPE_TOP
#define SCOPE_TYPE(scope) p<name>_##scope##_SCOPE
#define SCOPE_STACK(scope) p<name>_##scope##Stack
#define SCOPE_TOP(scope) ctx->p<name>_##scope##Top
#define SCOPE_SIZE(scope) ctx->p<name>_##scope##Stack_limit
#define SCOPE_INSTANCE(scope, i) (ctx->SCOPE_STACK(scope)->get(ctx->SCOPE_STACK(scope),i))
<if(LEXER)>
/* Macros for accessing things in a lexer
*/
#undef LEXER
#undef RECOGNIZER
#undef RULEMEMO
#undef GETCHARINDEX
#undef GETLINE
#undef GETCHARPOSITIONINLINE
#undef EMIT
#undef EMITNEW
#undef MATCHC
#undef MATCHS
#undef MATCHRANGE
#undef LTOKEN
#undef HASFAILED
#undef FAILEDFLAG
#undef INPUT
#undef STRSTREAM
#undef LA
#undef HASEXCEPTION
#undef EXCEPTION
#undef CONSTRUCTEX
#undef CONSUME
#undef LRECOVER
#undef MARK
#undef REWIND
#undef REWINDLAST
#undef BACKTRACKING
#undef MATCHANY
#undef MEMOIZE
#undef HAVEPARSEDRULE
#undef GETTEXT
#undef INDEX
#undef SEEK
#undef PUSHSTREAM
#undef POPSTREAM
#undef SETTEXT
#undef SETTEXT8
#define LEXER ctx->pLexer
#define RECOGNIZER LEXER->rec
#define LEXSTATE RECOGNIZER->state
#define TOKSOURCE LEXSTATE->tokSource
#define GETCHARINDEX() LEXER->getCharIndex(LEXER)
#define GETLINE() LEXER->getLine(LEXER)
#define GETTEXT() LEXER->getText(LEXER)
#define GETCHARPOSITIONINLINE() LEXER->getCharPositionInLine(LEXER)
#define EMIT() LEXSTATE->type = _type; LEXER->emit(LEXER)
#define EMITNEW(t) LEXER->emitNew(LEXER, t)
#define MATCHC(c) LEXER->matchc(LEXER, c)
#define MATCHS(s) LEXER->matchs(LEXER, s)
#define MATCHRANGE(c1,c2) LEXER->matchRange(LEXER, c1, c2)
#define MATCHANY() LEXER->matchAny(LEXER)
#define LTOKEN LEXSTATE->token
#define HASFAILED() (LEXSTATE->failed == ANTLR3_TRUE)
#define BACKTRACKING LEXSTATE->backtracking
#define FAILEDFLAG LEXSTATE->failed
#define INPUT LEXER->input
#define STRSTREAM INPUT
#define ISTREAM INPUT->istream
#define INDEX() ISTREAM->index(ISTREAM)
#define SEEK(n) ISTREAM->seek(ISTREAM, n)
#define EOF_TOKEN &(LEXSTATE->tokSource->eofToken)
#define HASEXCEPTION() (LEXSTATE->error == ANTLR3_TRUE)
#define EXCEPTION LEXSTATE->exception
#define CONSTRUCTEX() RECOGNIZER->exConstruct(RECOGNIZER)
#define LRECOVER() LEXER->recover(LEXER)
#define MARK() ISTREAM->mark(ISTREAM)
#define REWIND(m) ISTREAM->rewind(ISTREAM, m)
#define REWINDLAST() ISTREAM->rewindLast(ISTREAM)
#define MEMOIZE(ri,si) RECOGNIZER->memoize(RECOGNIZER, ri, si)
#define HAVEPARSEDRULE(r) RECOGNIZER->alreadyParsedRule(RECOGNIZER, r)
#define PUSHSTREAM(str) LEXER->pushCharStream(LEXER, str)
#define POPSTREAM() LEXER->popCharStream(LEXER)
#define SETTEXT(str) LEXSTATE->text = str
#define SKIP() LEXSTATE->token = &(TOKSOURCE->skipToken)
#define USER1 LEXSTATE->user1
#define USER2 LEXSTATE->user2
#define USER3 LEXSTATE->user3
#define CUSTOM LEXSTATE->custom
#define RULEMEMO LEXSTATE->ruleMemo
#define DBG RECOGNIZER->debugger
/* If we have been told we can rely on the standard 8 bit or 16 bit input
* stream, then we can define our macros to use the direct pointers
* in the input object, which is much faster than indirect calls. This
* is really only significant to lexers with a lot of fragment rules (which
* do not place LA(1) in a temporary at the moment) and even then
* only if there is a lot of input (order of say 1M or so).
*/
#if defined(ANTLR3_INLINE_INPUT_ASCII) || defined(ANTLR3_INLINE_INPUT_UTF16)
# ifdef ANTLR3_INLINE_INPUT_ASCII
/* 8 bit "ASCII" (actually any 8 bit character set) */
# define NEXTCHAR ((pANTLR3_UINT8)(INPUT->nextChar))
# define DATAP ((pANTLR3_UINT8)(INPUT->data))
# else
# define NEXTCHAR ((pANTLR3_UINT16)(INPUT->nextChar))
# define DATAP ((pANTLR3_UINT16)(INPUT->data))
# endif
# define LA(n) ((NEXTCHAR + n) > (DATAP + INPUT->sizeBuf) ? ANTLR3_CHARSTREAM_EOF : (ANTLR3_UCHAR)(*(NEXTCHAR + n - 1)))
# define CONSUME() \
{ \
if (NEXTCHAR \< (DATAP + INPUT->sizeBuf)) \
{ \
INPUT->charPositionInLine++; \
if ((ANTLR3_UCHAR)(*NEXTCHAR) == INPUT->newlineChar) \
{ \
INPUT->line++; \
INPUT->charPositionInLine = 0; \
INPUT->currentLine = (void *)(NEXTCHAR + 1); \
} \
INPUT->nextChar = (void *)(NEXTCHAR + 1); \
} \
}
#else
// Pick up the input character by calling the input stream implementation.
//
#define CONSUME() INPUT->istream->consume(INPUT->istream)
#define LA(n) INPUT->istream->_LA(INPUT->istream, n)
#endif
<endif>
<if(PARSER)>
/* Macros for accessing things in the parser
*/
#undef PARSER
#undef RECOGNIZER
#undef HAVEPARSEDRULE
#undef MEMOIZE
#undef INPUT
#undef STRSTREAM
#undef HASEXCEPTION
#undef EXCEPTION
#undef MATCHT
#undef MATCHANYT
#undef FOLLOWSTACK
#undef FOLLOWPUSH
#undef FOLLOWPOP
#undef PRECOVER
#undef PREPORTERROR
#undef LA
#undef LT
#undef CONSTRUCTEX
#undef CONSUME
#undef MARK
#undef REWIND
#undef REWINDLAST
#undef PERRORRECOVERY
#undef HASFAILED
#undef FAILEDFLAG
#undef RECOVERFROMMISMATCHEDSET
#undef RECOVERFROMMISMATCHEDELEMENT
#undef INDEX
#undef ADAPTOR
#undef SEEK
#undef RULEMEMO
#undef DBG
#define PARSER ctx->pParser
#define RECOGNIZER PARSER->rec
#define PSRSTATE RECOGNIZER->state
#define HAVEPARSEDRULE(r) RECOGNIZER->alreadyParsedRule(RECOGNIZER, r)
#define MEMOIZE(ri,si) RECOGNIZER->memoize(RECOGNIZER, ri, si)
#define INPUT PARSER->tstream
#define STRSTREAM INPUT
#define ISTREAM INPUT->istream
#define INDEX() ISTREAM->index(INPUT->istream)
#define HASEXCEPTION() (PSRSTATE->error == ANTLR3_TRUE)
#define EXCEPTION PSRSTATE->exception
#define MATCHT(t, fs) RECOGNIZER->match(RECOGNIZER, t, fs)
#define MATCHANYT() RECOGNIZER->matchAny(RECOGNIZER)
#define FOLLOWSTACK PSRSTATE->following
#define FOLLOWPUSH(x) FOLLOWSTACK->push(FOLLOWSTACK, ((void *)(&(x))), NULL)
#define FOLLOWPOP() FOLLOWSTACK->pop(FOLLOWSTACK)
#define PRECOVER() RECOGNIZER->recover(RECOGNIZER)
#define PREPORTERROR() RECOGNIZER->reportError(RECOGNIZER)
#define LA(n) INPUT->istream->_LA(ISTREAM, n)
#define LT(n) INPUT->_LT(INPUT, n)
#define CONSTRUCTEX() RECOGNIZER->exConstruct(RECOGNIZER)
#define CONSUME() ISTREAM->consume(ISTREAM)
#define MARK() ISTREAM->mark(ISTREAM)
#define REWIND(m) ISTREAM->rewind(ISTREAM, m)
#define REWINDLAST() ISTREAM->rewindLast(ISTREAM)
#define SEEK(n) ISTREAM->seek(ISTREAM, n)
#define PERRORRECOVERY PSRSTATE->errorRecovery
#define FAILEDFLAG PSRSTATE->failed
#define HASFAILED() (FAILEDFLAG == ANTLR3_TRUE)
#define BACKTRACKING PSRSTATE->backtracking
#define RECOVERFROMMISMATCHEDSET(s) RECOGNIZER->recoverFromMismatchedSet(RECOGNIZER, s)
#define RECOVERFROMMISMATCHEDELEMENT(e) RECOGNIZER->recoverFromMismatchedElement(RECOGNIZER, s)
#define ADAPTOR ctx->adaptor
#define RULEMEMO PSRSTATE->ruleMemo
#define DBG RECOGNIZER->debugger
<endif>
<if(TREE_PARSER)>
/* Macros for accessing things in the parser
*/
#undef PARSER
#undef RECOGNIZER
#undef HAVEPARSEDRULE
#undef INPUT
#undef STRSTREAM
#undef HASEXCEPTION
#undef EXCEPTION
#undef MATCHT
#undef MATCHANYT
#undef FOLLOWSTACK
#undef FOLLOWPUSH
#undef FOLLOWPOP
#undef PRECOVER
#undef PREPORTERROR
#undef LA
#undef LT
#undef CONSTRUCTEX
#undef CONSUME
#undef MARK
#undef REWIND
#undef REWINDLAST
#undef PERRORRECOVERY
#undef HASFAILED
#undef FAILEDFLAG
#undef RECOVERFROMMISMATCHEDSET
#undef RECOVERFROMMISMATCHEDELEMENT
#undef BACKTRACKING
#undef ADAPTOR
#undef RULEMEMO
#undef SEEK
#undef INDEX
#undef DBG
#define PARSER ctx->pTreeParser
#define RECOGNIZER PARSER->rec
#define PSRSTATE RECOGNIZER->state
#define HAVEPARSEDRULE(r) RECOGNIZER->alreadyParsedRule(RECOGNIZER, r)
#define INPUT PARSER->ctnstream
#define ISTREAM INPUT->tnstream->istream
#define STRSTREAM INPUT->tnstream
#define HASEXCEPTION() (PSRSTATE->error == ANTLR3_TRUE)
#define EXCEPTION PSRSTATE->exception
#define MATCHT(t, fs) RECOGNIZER->match(RECOGNIZER, t, fs)
#define MATCHANYT() RECOGNIZER->matchAny(RECOGNIZER)
#define FOLLOWSTACK PSRSTATE->following
#define FOLLOWPUSH(x) FOLLOWSTACK->push(FOLLOWSTACK, ((void *)(&(x))), NULL)
#define FOLLOWPOP() FOLLOWSTACK->pop(FOLLOWSTACK)
#define PRECOVER() RECOGNIZER->recover(RECOGNIZER)
#define PREPORTERROR() RECOGNIZER->reportError(RECOGNIZER)
#define LA(n) ISTREAM->_LA(ISTREAM, n)
#define LT(n) INPUT->tnstream->_LT(INPUT->tnstream, n)
#define CONSTRUCTEX() RECOGNIZER->exConstruct(RECOGNIZER)
#define CONSUME() ISTREAM->consume(ISTREAM)
#define MARK() ISTREAM->mark(ISTREAM)
#define REWIND(m) ISTREAM->rewind(ISTREAM, m)
#define REWINDLAST() ISTREAM->rewindLast(ISTREAM)
#define PERRORRECOVERY PSRSTATE->errorRecovery
#define FAILEDFLAG PSRSTATE->failed
#define HASFAILED() (FAILEDFLAG == ANTLR3_TRUE)
#define BACKTRACKING PSRSTATE->backtracking
#define RECOVERFROMMISMATCHEDSET(s) RECOGNIZER->recoverFromMismatchedSet(RECOGNIZER, s)
#define RECOVERFROMMISMATCHEDELEMENT(e) RECOGNIZER->recoverFromMismatchedElement(RECOGNIZER, s)
#define ADAPTOR INPUT->adaptor
#define RULEMEMO PSRSTATE->ruleMemo
#define SEEK(n) ISTREAM->seek(ISTREAM, n)
#define INDEX() ISTREAM->index(ISTREAM)
#define DBG RECOGNIZER->debugger
<endif>
#define TOKTEXT(tok, txt) tok, (pANTLR3_UINT8)txt
/* The 4 tokens defined below may well clash with your own #defines or token types. If so
* then for the present you must use different names for your defines as these are hard coded
* in the code generator. It would be better not to use such names internally, and maybe
* we can change this in a forthcoming release. I deliberately do not #undef these
* here as this will at least give you a redefined error somewhere if they clash.
*/
#define UP ANTLR3_TOKEN_UP
#define DOWN ANTLR3_TOKEN_DOWN
#define EOR ANTLR3_TOKEN_EOR
#define INVALID ANTLR3_TOKEN_INVALID
/* =============================================================================
* Functions to create and destroy scopes. First come the rule scopes, followed
* by the global declared scopes.
*/
<rules: {r |<if(r.ruleDescriptor.ruleScope)>
<ruleAttributeScopeFuncDecl(scope=r.ruleDescriptor.ruleScope)>
<ruleAttributeScopeFuncs(scope=r.ruleDescriptor.ruleScope)>
<endif>}>
<recognizer.scopes:{<if(it.isDynamicGlobalScope)>
<globalAttributeScopeFuncDecl(scope=it)>
<globalAttributeScopeFuncs(scope=it)>
<endif>}>
/* ============================================================================= */
/* =============================================================================
* Start of recognizer
*/
<recognizer>
/* End of code
* =============================================================================
*/
>>
headerFileExtension() ::= ".h"
headerFile( LEXER,
PARSER,
TREE_PARSER,
actionScope,
actions,
docComment,
recognizer,
name,
tokens,
tokenNames,
rules,
cyclicDFAs,
bitsets,
buildTemplate,
buildAST,
rewriteMode,
profile,
backtracking,
synpreds,
memoize,
numRules,
fileName,
ANTLRVersion,
generatedTimestamp,
trace,
scopes,
superClass,
literals
) ::=
<<
<leadIn("C header")>
<if(PARSER)>
* The parser <mainName()>
<endif>
<if(LEXER)>
* The lexer <mainName()>
<endif>
<if(TREE_PARSER)>
* The tree parser <mainName()>
<endif>
has the callable functions (rules) shown below,
* which will invoke the code for the associated rule in the source grammar
* assuming that the input stream is pointing to a token/text stream that could begin
* this rule.
*
* For instance if you call the first (topmost) rule in a parser grammar, you will
* get the results of a full parse, but calling a rule half way through the grammar will
* allow you to pass part of a full token stream to the parser, such as for syntax checking
* in editors and so on.
*
* The parser entry points are called indirectly (by function pointer to function) via
* a parser context typedef p<name>, which is returned from a call to <name>New().
*
<if(LEXER)>
* As this is a generated lexer, it is unlikely you will call it 'manually'. However
* the methods are provided anyway.
*
<endif>
* The methods in p<name> are as follows:
*
* <rules: {r | <if(!r.ruleDescriptor.isSynPred)> - <headerReturnType(ruleDescriptor=r.ruleDescriptor,...)> p<name>-><r.ruleDescriptor.name>(p<name>)<endif>}; separator="\n * ">
*
* The return type for any particular rule is of course determined by the source
* grammar file.
*/
// [The "BSD licence"]
// Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
// http://www.temporal-wave.com
// http://www.linkedin.com/in/jimidle
//
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// 3. The name of the author may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
// IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
// OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
// IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
// INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
// NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
// THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#ifndef _<name>_H
#define _<name>_H
<actions.(actionScope).preincludes>
/* =============================================================================
* Standard antlr3 C runtime definitions
*/
#include \<antlr3.h>
/* End of standard antlr 3 runtime definitions
* =============================================================================
*/
<actions.(actionScope).includes>
#ifdef __cplusplus
extern "C" {
#endif
// Forward declare the context typedef so that we can use it before it is
// properly defined. Delegators and delegates (from import statements) are
// interdependent and their context structures contain pointers to each other
// C only allows such things to be declared if you pre-declare the typedef.
//
typedef struct <name>_Ctx_struct <name>, * p<name>;
<if(recognizer.grammar.delegates)>
// Include delegate definition header files
//
<recognizer.grammar.delegates: {g|#include \<<g.recognizerName>.h>}; separator="\n">
<endif>
<actions.(actionScope).header>
#ifdef ANTLR3_WINDOWS
// Disable: Unreferenced parameter, - Rules with parameters that are not used
// constant conditional, - ANTLR realizes that a prediction is always true (synpred usually)
// initialized but unused variable - tree rewrite variables declared but not needed
// Unreferenced local variable - lexer rule declares but does not always use _type
// potentially unitialized variable used - retval always returned from a rule
// unreferenced local function has been removed - susually getTokenNames or freeScope, they can go without warnigns
//
// These are only really displayed at warning level /W4 but that is the code ideal I am aiming at
// and the codegen must generate some of these warnings by necessity, apart from 4100, which is
// usually generated when a parser rule is given a parameter that it does not use. Mostly though
// this is a matter of orthogonality hence I disable that one.
//
#pragma warning( disable : 4100 )
#pragma warning( disable : 4101 )
#pragma warning( disable : 4127 )
#pragma warning( disable : 4189 )
#pragma warning( disable : 4505 )
#pragma warning( disable : 4701 )
#endif
<if(backtracking)>
/* ========================
* BACKTRACKING IS ENABLED
* ========================
*/
<endif>
<rules:{r |<headerReturnScope(ruleDescriptor=r.ruleDescriptor,...)>}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeDecl(scope=it)><endif>}>
<rules:{r |<ruleAttributeScopeDecl(scope=r.ruleDescriptor.ruleScope)>}>
<if(recognizer.grammar.delegators)>
// Include delegator definition header files
//
<recognizer.grammar.delegators: {g|#include \<<g.recognizerName>.h>}; separator="\n">
<endif>
/** Context tracking structure for <mainName()>
*/
struct <name>_Ctx_struct
{
/** Built in ANTLR3 context tracker contains all the generic elements
* required for context tracking.
*/
<if(PARSER)>
pANTLR3_PARSER pParser;
<endif>
<if(LEXER)>
pANTLR3_LEXER pLexer;
<endif>
<if(TREE_PARSER)>
pANTLR3_TREE_PARSER pTreeParser;
<endif>
<if(recognizer.grammar.delegates)>
<recognizer.grammar.delegates:
{g|p<g.recognizerName> <g:delegateName()>;}; separator="\n">
<endif>
<if(recognizer.grammar.delegators)>
<recognizer.grammar.delegators:
{g|p<g.recognizerName> <g:delegateName()>;}; separator="\n">
<endif>
<scopes:{<if(it.isDynamicGlobalScope)>
<globalAttributeScopeDef(scope=it)>
<endif>}; separator="\n\n">
<rules: {r |<if(r.ruleDescriptor.ruleScope)>
<ruleAttributeScopeDef(scope=r.ruleDescriptor.ruleScope)>
<endif>}>
<if(LEXER)>
<rules:{r | <if(!r.ruleDescriptor.isSynPred)><headerReturnType(ruleDescriptor=r.ruleDescriptor)> (*m<r.ruleDescriptor.name>) (struct <name>_Ctx_struct * ctx<if(r.ruleDescriptor.parameterScope)>, <endif><r.ruleDescriptor.parameterScope:parameterScope(scope=it)>);<endif>}; separator="\n";>
<endif>
<if(!LEXER)>
<rules:{r | <headerReturnType(ruleDescriptor=r.ruleDescriptor)> (*<r.ruleDescriptor.name>) (struct <name>_Ctx_struct * ctx<if(r.ruleDescriptor.parameterScope)>, <endif><r.ruleDescriptor.parameterScope:parameterScope(scope=it)>);}; separator="\n";>
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
// Delegated rules
<recognizer.grammar.delegatedRules:{ruleDescriptor|
<headerReturnType(ruleDescriptor)> (*<ruleDescriptor.name>)(struct <name>_Ctx_struct * ctx<if(ruleDescriptor.parameterScope)>, <endif><ruleDescriptor.parameterScope:parameterScope(scope=it)>);}; separator="\n";>
<endif>
const char * (*getGrammarFileName)();
void (*free) (struct <name>_Ctx_struct * ctx);
<@members>
<@end>
<actions.(actionScope).context>
};
// Function protoypes for the constructor functions that external translation units
// such as delegators and delegates may wish to call.
//
ANTLR3_API p<name> <name>New (<inputType()> instream<recognizer.grammar.delegators:{g|, p<g.recognizerName> <g:delegateName()>}>);
ANTLR3_API p<name> <name>NewSSD (<inputType()> instream, pANTLR3_RECOGNIZER_SHARED_STATE state<recognizer.grammar.delegators:{g|, p<g.recognizerName> <g:delegateName()>}>);
<if(!recognizer.grammar.grammarIsRoot)>
extern pANTLR3_UINT8 <recognizer.grammar.composite.rootGrammar.recognizerName>TokenNames[];
<endif>
/** Symbolic definitions of all the tokens that the <grammarType()> will work with.
* \{
*
* Antlr will define EOF, but we can't use that as it it is too common in
* in C header files and that would be confusing. There is no way to filter this out at the moment
* so we just undef it here for now. That isn't the value we get back from C recognizers
* anyway. We are looking for ANTLR3_TOKEN_EOF.
*/
#ifdef EOF
#undef EOF
#endif
#ifdef Tokens
#undef Tokens
#endif
<tokens:{#define <it.name> <it.type>}; separator="\n">
#ifdef EOF
#undef EOF
#define EOF ANTLR3_TOKEN_EOF
#endif
#ifndef TOKENSOURCE
#define TOKENSOURCE(lxr) lxr->pLexer->rec->state->tokSource
#endif
/* End of token definitions for <name>
* =============================================================================
*/
/** \} */
#ifdef __cplusplus
}
#endif
#endif
/* END - Note:Keep extra line feed to satisfy UNIX systems */
>>
inputType() ::=<<
<if(LEXER)>
pANTLR3_INPUT_STREAM
<endif>
<if(PARSER)>
pANTLR3_COMMON_TOKEN_STREAM
<endif>
<if(TREE_PARSER)>
pANTLR3_COMMON_TREE_NODE_STREAM
<endif>
>>
grammarType() ::= <<
<if(PARSER)>
parser
<endif>
<if(LEXER)>
lexer
<endif>
<if(TREE_PARSER)>
tree parser
<endif>
>>
mainName() ::= <<
<if(PARSER)>
<name>
<endif>
<if(LEXER)>
<name>
<endif>
<if(TREE_PARSER)>
<name>
<endif>
>>
headerReturnScope(ruleDescriptor) ::= "<returnScope(...)>"
headerReturnType(ruleDescriptor) ::= <<
<if(LEXER)>
<if(!r.ruleDescriptor.isSynPred)>
void
<else>
<ruleDescriptor:returnType()>
<endif>
<else>
<ruleDescriptor:returnType()>
<endif>
>>
// Produce the lexer output
//
lexer( grammar,
name,
tokens,
scopes,
rules,
numRules,
labelType="pANTLR3_COMMON_TOKEN",
filterMode,
superClass) ::= <<
<if(filterMode)>
/* Forward declare implementation function for ANTLR3_TOKEN_SOURCE interface when
* this is a filter mode lexer.
*/
static pANTLR3_COMMON_TOKEN <name>NextToken (pANTLR3_TOKEN_SOURCE toksource);
/* Override the normal MEMOIZE and HAVEALREADYPARSED macros as this is a filtering
* lexer. In filter mode, the memoizing and backtracking are gated at BACKTRACKING > 1 rather
* than just BACKTRACKING. In some cases this might generate code akin to:
* if (BACKTRACKING) if (BACKTRACKING > 1) memoize.
* However, I assume that the C compilers/optimizers are smart enough to work this one out
* these days - Jim
*/
#undef MEMOIZE
#define MEMOIZE(ri,si) if (BACKTRACKING>1) { RECOGNIZER->memoize(RECOGNIZER, ri, si) }
#undef HAVEPARSEDRULE
#define HAVEPARSEDRULE(r) if (BACKTRACKING>1) { RECOGNIZER->alreadyParsedRule(RECOGNIZER, r) }
<endif>
/* Forward declare the locally static matching functions we have generated and any predicate functions.
*/
<rules:{r | static ANTLR3_INLINE <headerReturnType(ruleDescriptor=r.ruleDescriptor)> <if(!r.ruleDescriptor.isSynPred)>m<endif><r.ruleDescriptor.name> (p<name> ctx<if(r.ruleDescriptor.parameterScope)>, <endif><r.ruleDescriptor.parameterScope:parameterScope(scope=it)>);}; separator="\n";>
static void <name>Free(p<name> ctx);
/* =========================================================================
* Lexer matching rules end.
* =========================================================================
*/
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
static void
<name>Free (p<name> ctx)
{
<if(memoize)>
if (RULEMEMO != NULL)
{
RULEMEMO->free(RULEMEMO);
RULEMEMO = NULL;
}
<endif>
<if(grammar.directDelegates)>
// Free the lexers that we delegated to
// functions to. NULL the state so we only free it once.
//
<grammar.directDelegates:
{g|ctx-><g:delegateName()>->pLexer->rec->state = NULL;
ctx-><g:delegateName()>->free(ctx-><g:delegateName()>);}; separator="\n">
<endif>
LEXER->free(LEXER);
ANTLR3_FREE(ctx);
}
/** \brief Name of the grammar file that generated this code
*/
static const char fileName[] = "<fileName>";
/** \brief Return the name of the grammar file that generated this code.
*/
static const char * getGrammarFileName()
{
return fileName;
}
<if(filterMode)>
<filteringNextToken()>
<endif>
/** \brief Create a new lexer called <name>
*
* \param[in] instream Pointer to an initialized input stream
* \return
* - Success p<name> initialized for the lex start
* - Fail NULL
*/
ANTLR3_API p<name> <name>New
(<inputType()> instream<grammar.delegators:{g|, p<g.recognizerName> <g:delegateName()>}>)
{
// See if we can create a new lexer with the standard constructor
//
return <name>NewSSD(instream, NULL<grammar.delegators:{g|, <g:delegateName()>}>);
}
/** \brief Create a new lexer called <name>
*
* \param[in] instream Pointer to an initialized input stream
* \param[state] state Previously created shared recognizer stat
* \return
* - Success p<name> initialized for the lex start
* - Fail NULL
*/
ANTLR3_API p<name> <name>NewSSD
(pANTLR3_INPUT_STREAM instream, pANTLR3_RECOGNIZER_SHARED_STATE state<grammar.delegators:{g|, p<g.recognizerName> <g:delegateName()>}>)
{
p<name> ctx; // Context structure we will build and return
ctx = (p<name>) ANTLR3_CALLOC(1, sizeof(<name>));
if (ctx == NULL)
{
// Failed to allocate memory for lexer context
return NULL;
}
/* -------------------------------------------------------------------
* Memory for basic structure is allocated, now to fill in
* in base ANTLR3 structures. We initialize the function pointers
* for the standard ANTLR3 lexer function set, but upon return
* from here, the programmer may set the pointers to provide custom
* implementations of each function.
*
* We don't use the macros defined in <name>.h here so you can get a sense
* of what goes where.
*/
/* Create a base lexer, using the supplied input stream
*/
ctx->pLexer = antlr3LexerNewStream(ANTLR3_SIZE_HINT, instream, state);
/* Check that we allocated the memory correctly
*/
if (ctx->pLexer == NULL)
{
ANTLR3_FREE(ctx);
return NULL;
}
<if(memoize)>
<if(grammar.grammarIsRoot)>
// Create a LIST for recording rule memos.
//
ctx->pLexer->rec->ruleMemo = antlr3IntTrieNew(15); /* 16 bit depth is enough for 32768 rules! */
<endif>
<endif>
/* Install the implementation of our <name> interface
*/
<rules:{r | <if(!r.ruleDescriptor.isSynPred)>ctx->m<r.ruleDescriptor.name> = m<r.ruleDescriptor.name>;<endif>}; separator="\n";>
/** When the nextToken() call is made to this lexer's pANTLR3_TOKEN_SOURCE
* it will call mTokens() in this generated code, and will pass it the ctx
* pointer of this lexer, not the context of the base lexer, so store that now.
*/
ctx->pLexer->ctx = ctx;
/**Install the token matching function
*/
ctx->pLexer->mTokens = (void (*) (void *))(mTokens);
ctx->getGrammarFileName = getGrammarFileName;
ctx->free = <name>Free;
<if(grammar.directDelegates)>
// Initialize the lexers that we are going to delegate some
// functions to.
//
<grammar.directDelegates:
{g|ctx-><g:delegateName()> = <g.recognizerName>NewSSD(instream, ctx->pLexer->rec->state, ctx<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<endif>
<if(grammar.delegators)>
// Install the pointers back to lexers that will delegate us to perform certain functions
// for them.
//
<grammar.delegators:
{g|ctx-><g:delegateName()> = <g:delegateName()>;}; separator="\n">
<endif>
<if(filterMode)>
/* We have filter mode turned on, so install the filtering nextToken function
*/
ctx->pLexer->rec->state->tokSource->nextToken = <name>NextToken;
<endif>
<actions.lexer.apifuncs>
/* Return the newly built lexer to the caller
*/
return ctx;
}
<if(cyclicDFAs)>
/* =========================================================================
* DFA tables for the lexer
*/
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
/* =========================================================================
* End of DFA tables for the lexer
*/
<endif>
/* =========================================================================
* Functions to match the lexer grammar defined tokens from the input stream
*/
<rules; separator="\n\n">
/* =========================================================================
* Lexer matching rules end.
* =========================================================================
*/
<if(synpreds)>
/* =========================================================================
* Lexer syntactic predicates
*/
<synpreds:{p | <lexerSynpred(predname=p)>}>
/* =========================================================================
* Lexer syntactic predicates end.
* =========================================================================
*/
<endif>
/* End of Lexer code
* ================================================
* ================================================
*/
>>
filteringNextToken() ::= <<
/** An override of the lexer's nextToken() method that backtracks over mTokens() looking
* for matches in lexer filterMode. No error can be generated upon error; just rewind, consume
* a token and then try again. BACKTRACKING needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at BACKTRACKING==1.
*/
static pANTLR3_COMMON_TOKEN
<name>NextToken(pANTLR3_TOKEN_SOURCE toksource)
{
pANTLR3_LEXER lexer;
pANTLR3_RECOGNIZER_SHARED_STATE state;
lexer = (pANTLR3_LEXER)(toksource->super);
state = lexer->rec->state;
/* Get rid of any previous token (token factory takes care of
* any deallocation when this token is finally used up.
*/
state ->token = NULL;
state ->error = ANTLR3_FALSE; /* Start out without an exception */
state ->failed = ANTLR3_FALSE;
/* Record the start of the token in our input stream.
*/
state->tokenStartCharIndex = lexer->input->istream->index(lexer->input->istream);
state->tokenStartCharPositionInLine = lexer->input->getCharPositionInLine(lexer->input);
state->tokenStartLine = lexer->input->getLine(lexer->input);
state->text = NULL;
/* Now call the matching rules and see if we can generate a new token
*/
for (;;)
{
if (lexer->input->istream->_LA(lexer->input->istream, 1) == ANTLR3_CHARSTREAM_EOF)
{
/* Reached the end of the stream, nothing more to do.
*/
pANTLR3_COMMON_TOKEN teof = &(toksource->eofToken);
teof->setStartIndex (teof, lexer->getCharIndex(lexer));
teof->setStopIndex (teof, lexer->getCharIndex(lexer));
teof->setLine (teof, lexer->getLine(lexer));
return teof;
}
state->token = NULL;
state->error = ANTLR3_FALSE; /* Start out without an exception */
{
ANTLR3_MARKER m;
m = lexer->input->istream->mark(lexer->input->istream);
state->backtracking = 1; /* No exceptions */
state->failed = ANTLR3_FALSE;
/* Call the generated lexer, see if it can get a new token together.
*/
lexer->mTokens(lexer->ctx);
state->backtracking = 0;
<! mTokens backtracks with synpred at BACKTRACKING==2
and we set the synpredgate to allow actions at level 1. !>
if (state->failed == ANTLR3_TRUE)
{
lexer->input->istream->rewind(lexer->input->istream, m);
lexer->input->istream->consume(lexer->input->istream); <! advance one char and try again !>
}
else
{
lexer->emit(lexer); /* Assemble the token and emit it to the stream */
return state->token;
}
}
}
}
>>
actionGate() ::= "BACKTRACKING==0"
filteringActionGate() ::= "BACKTRACKING==1"
/** How to generate a parser */
genericParser( grammar,
name,
scopes,
tokens,
tokenNames,
rules,
numRules,
bitsets,
inputStreamType,
superClass,
ASTLabelType="pANTLR3_BASE_TREE",
labelType,
members,
rewriteElementType, filterMode
) ::= <<
<if(grammar.grammarIsRoot)>
/** \brief Table of all token names in symbolic order, mainly used for
* error reporting.
*/
pANTLR3_UINT8 <name>TokenNames[<length(tokenNames)>+4]
= {
(pANTLR3_UINT8) "\<invalid>", /* String to print to indicate an invalid token */
(pANTLR3_UINT8) "\<EOR>",
(pANTLR3_UINT8) "\<DOWN>",
(pANTLR3_UINT8) "\<UP>",
<tokenNames:{(pANTLR3_UINT8) <it>}; separator=",\n">
};
<endif>
<@members>
<@end>
<rules:{r |<ruleAttributeScopeFuncMacro(scope=r.ruleDescriptor.ruleScope)>}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeFuncMacro(scope=it)><endif>}>
// Forward declare the locally static matching functions we have generated.
//
<rules:{r | static <headerReturnType(ruleDescriptor=r.ruleDescriptor)> <r.ruleDescriptor.name> (p<name> ctx<if(r.ruleDescriptor.parameterScope)>, <endif><r.ruleDescriptor.parameterScope:parameterScope(scope=it)>);}; separator="\n";>
static void <name>Free(p<name> ctx);
<if(!LEXER)>
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
<if(recognizer.grammar.delegatedRules)>
// Delegated rules
//
<recognizer.grammar.delegatedRules:{ruleDescriptor|static <headerReturnType(ruleDescriptor)> <ruleDescriptor.name>(p<name> ctx<if(ruleDescriptor.parameterScope)>, <endif><ruleDescriptor.parameterScope:parameterScope(scope=it)>);}; separator="\n";>
<endif>
<endif>
/* For use in tree output where we are accumulating rule labels via label += ruleRef
* we need a function that knows how to free a return scope when the list is destroyed.
* We cannot just use ANTLR3_FREE because in debug tracking mode, this is a macro.
*/
static void ANTLR3_CDECL freeScope(void * scope)
{
ANTLR3_FREE(scope);
}
/** \brief Name of the grammar file that generated this code
*/
static const char fileName[] = "<fileName>";
/** \brief Return the name of the grammar file that generated this code.
*/
static const char * getGrammarFileName()
{
return fileName;
}
/** \brief Create a new <name> parser and return a context for it.
*
* \param[in] instream Pointer to an input stream interface.
*
* \return Pointer to new parser context upon success.
*/
ANTLR3_API p<name>
<name>New (<inputStreamType> instream<grammar.delegators:{g|, p<g.recognizerName> <g:delegateName()>}>)
{
// See if we can create a new parser with the standard constructor
//
return <name>NewSSD(instream, NULL<grammar.delegators:{g|, <g:delegateName()>}>);
}
/** \brief Create a new <name> parser and return a context for it.
*
* \param[in] instream Pointer to an input stream interface.
*
* \return Pointer to new parser context upon success.
*/
ANTLR3_API p<name>
<name>NewSSD (<inputStreamType> instream, pANTLR3_RECOGNIZER_SHARED_STATE state<grammar.delegators:{g|, p<g.recognizerName> <g:delegateName()>}>)
{
p<name> ctx; /* Context structure we will build and return */
ctx = (p<name>) ANTLR3_CALLOC(1, sizeof(<name>));
if (ctx == NULL)
{
// Failed to allocate memory for parser context
//
return NULL;
}
/* -------------------------------------------------------------------
* Memory for basic structure is allocated, now to fill in
* the base ANTLR3 structures. We initialize the function pointers
* for the standard ANTLR3 parser function set, but upon return
* from here, the programmer may set the pointers to provide custom
* implementations of each function.
*
* We don't use the macros defined in <name>.h here, in order that you can get a sense
* of what goes where.
*/
<if(PARSER)>
/* Create a base parser/recognizer, using the supplied token stream
*/
ctx->pParser = antlr3ParserNewStream(ANTLR3_SIZE_HINT, instream->tstream, state);
<endif>
<if(TREE_PARSER)>
/* Create a base Tree parser/recognizer, using the supplied tree node stream
*/
ctx->pTreeParser = antlr3TreeParserNewStream(ANTLR3_SIZE_HINT, instream, state);
<endif>
/* Install the implementation of our <name> interface
*/
<rules:{r | ctx-><r.ruleDescriptor.name> = <r.ruleDescriptor.name>;}; separator="\n";>
<if(grammar.delegatedRules)>
// Install the delegated methods so that they appear to be a part of this
// parser
//
<grammar.delegatedRules:{ruleDescriptor | ctx-><ruleDescriptor.name> = <ruleDescriptor.name>;}; separator="\n";>
<endif>
ctx->free = <name>Free;
ctx->getGrammarFileName = getGrammarFileName;
/* Install the scope pushing methods.
*/
<rules: {r |<if(r.ruleDescriptor.ruleScope)>
<ruleAttributeScope(scope=r.ruleDescriptor.ruleScope)><\n>
<endif>}>
<recognizer.scopes:{<if(it.isDynamicGlobalScope)>
<globalAttributeScope(scope=it)><\n>
<endif>}>
<@apifuncs>
<@end>
<if(grammar.directDelegates)>
// Initialize the parsers that we are going to delegate some
// functions to.
//
<grammar.directDelegates:
{g|ctx-><g:delegateName()> = <g.recognizerName>NewSSD(instream, PSRSTATE, ctx<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<endif>
<if(grammar.delegators)>
// Install the pointers back to parsers that will delegate us to perform certain functions
// for them.
//
<grammar.delegators:
{g|ctx-><g:delegateName()> = <g:delegateName()>;}; separator="\n">
<endif>
<actions.parser.apifuncs>
<actions.treeparser.apifuncs>
<if(memoize)>
<if(grammar.grammarIsRoot)>
/* Create a LIST for recording rule memos.
*/
RULEMEMO = antlr3IntTrieNew(15); /* 16 bit depth is enough for 32768 rules! */<\n>
<endif>
<endif>
/* Install the token table
*/
PSRSTATE->tokenNames = <grammar.composite.rootGrammar.recognizerName>TokenNames;
<@debugStuff()>
/* Return the newly built parser to the caller
*/
return ctx;
}
/** Free the parser resources
*/
static void
<name>Free(p<name> ctx)
{
/* Free any scope memory
*/
<rules: {r |<if(r.ruleDescriptor.ruleScope)><ruleAttributeScopeFree(scope=r.ruleDescriptor.ruleScope)><\n><endif>}>
<recognizer.scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScopeFree(scope=it)><\n><endif>}>
<@cleanup>
<@end>
<if(grammar.directDelegates)>
// Free the parsers that we delegated to
// functions to.NULL the state so we only free it once.
//
<grammar.directDelegates:
{g| ctx-><g:delegateName()>-><if(TREE_PARSER)>pTreeParser<else>pParser<endif>->rec->state = NULL;
ctx-><g:delegateName()>->free(ctx-><g:delegateName()>);}; separator="\n">
<endif>
<if(memoize)>
<if(grammar.grammarIsRoot)>
if (RULEMEMO != NULL)
{
RULEMEMO->free(RULEMEMO);
RULEMEMO = NULL;
}
<endif>
<endif>
// Free this parser
//
<if(TREE_PARSER)>
ctx->pTreeParser->free(ctx->pTreeParser);<\n>
<else>
ctx->pParser->free(ctx->pParser);<\n>
<endif>
ANTLR3_FREE(ctx);
/* Everything is released, so we can return
*/
return;
}
/** Return token names used by this <grammarType()>
*
* The returned pointer is used as an index into the token names table (using the token
* number as the index).
*
* \return Pointer to first char * in the table.
*/
static pANTLR3_UINT8 *getTokenNames()
{
return <grammar.composite.rootGrammar.recognizerName>TokenNames;
}
<members>
/* Declare the bitsets
*/
<bitsets:bitsetDeclare(name={FOLLOW_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
<if(cyclicDFAs)>
/* =========================================================================
* DFA tables for the parser
*/
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
/* =========================================================================
* End of DFA tables for the parser
*/
<endif>
/* ==============================================
* Parsing rules
*/
<rules; separator="\n\n">
<if(grammar.delegatedRules)>
// Delegated methods that appear to be a part of this
// parser
//
<grammar.delegatedRules:{ruleDescriptor|
<returnType()> <ruleDescriptor.name>(p<name> ctx<if(ruleDescriptor.parameterScope.attributes)>, <endif><ruleDescriptor.parameterScope:parameterScope(scope=it)>)
\{
<if(ruleDescriptor.hasReturnValue)>return <endif>ctx-><ruleDescriptor.grammar:delegateName()>-><ruleDescriptor.name>(ctx-><ruleDescriptor.grammar:delegateName()><if(ruleDescriptor.parameterScope.attributes)>, <endif><ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">);
\}}; separator="\n">
<endif>
/* End of parsing rules
* ==============================================
*/
/* ==============================================
* Syntactic predicates
*/
<synpreds:{p | <synpred(predname=p)>}>
/* End of syntactic predicates
* ==============================================
*/
>>
parser( grammar,
name,
scopes,
tokens,
tokenNames,
rules,
numRules,
bitsets,
ASTLabelType,
superClass="Parser",
labelType="pANTLR3_COMMON_TOKEN",
members={<actions.parser.members>}
) ::= <<
<genericParser(inputStreamType="pANTLR3_COMMON_TOKEN_STREAM", rewriteElementType="TOKEN", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser( grammar,
name,
scopes,
tokens,
tokenNames,
globalAction,
rules,
numRules,
bitsets,
labelType={<ASTLabelType>},
ASTLabelType="pANTLR3_BASE_TREE",
superClass="TreeParser",
members={<actions.treeparser.members>}, filterMode
) ::= <<
<genericParser(inputStreamType="pANTLR3_COMMON_TREE_NODE_STREAM", rewriteElementType="NODE", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start <ruleName>
static void <ruleName>_fragment(p<name> ctx <ruleDescriptor.parameterScope:parameterScope(scope=it)>)
{
<ruleLabelDefs()>
<ruleLabelInitializations()>
<if(trace)>
ANTLR3_PRINTF("enter <ruleName> %d failed = %d, backtracking = %d\\n",LT(1),failed,BACKTRACKING);
<block>
ANTLR3_PRINTF("exit <ruleName> %d, failed = %d, backtracking = %d\\n",LT(1),failed,BACKTRACKING);
<else>
<block>
<endif>
<ruleCleanUp()>
}
// $ANTLR end <ruleName>
>>
synpred(predname) ::= <<
static ANTLR3_BOOLEAN <predname>(p<name> ctx)
{
ANTLR3_MARKER start;
ANTLR3_BOOLEAN success;
BACKTRACKING++;
<@start()>
start = MARK();
<predname>_fragment(ctx); // can never throw exception
success = !(FAILEDFLAG);
REWIND(start);
<@stop()>
BACKTRACKING--;
FAILEDFLAG = ANTLR3_FALSE;
return success;
}<\n>
>>
lexerSynpred(predname) ::= <<
<synpred(predname)>
>>
ruleMemoization(rname) ::= <<
<if(memoize)>
if ( (BACKTRACKING>0) && (HAVEPARSEDRULE(<ruleDescriptor.index>)) )
{
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!ruleDescriptor.isSynPred)>
retval.start = 0;<\n>
<endif>
<endif>
<(ruleDescriptor.actions.after):execAfter()>
<finalCode(finalBlock=finally)>
<if(!ruleDescriptor.isSynPred)>
<scopeClean()><\n>
<endif>
return <ruleReturnValue()>;
}
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
if (HASEXCEPTION())
{
goto rule<ruleDescriptor.name>Ex;
}
<if(backtracking)>
if (HASFAILED())
{
<scopeClean()>
return <ruleReturnValue()>;
}
<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>
if (BACKTRACKING>0)
{
FAILEDFLAG = <true()>;
<scopeClean()>
return <ruleReturnValue()>;
}
<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
/**
* $ANTLR start <ruleName>
* <fileName>:<description>
*/
static <returnType()>
<ruleName>(p<name> ctx<if(ruleDescriptor.parameterScope)>, <endif><ruleDescriptor.parameterScope:parameterScope(scope=it)>)
{
<if(trace)>ANTLR3_PRINTF("enter <ruleName> %s failed=%d, backtracking=%d\n", LT(1), BACKTRACKING);<endif>
<ruleDeclarations()>
<ruleDescriptor.actions.declarations>
<ruleLabelDefs()>
<ruleInitializations()>
<ruleDescriptor.actions.init>
<ruleMemoization(rname=ruleName)>
<ruleLabelInitializations()>
<@preamble()>
{
<block>
}
<ruleCleanUp()>
<if(exceptions)>
if (HASEXCEPTION())
{
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
}
else
{
<(ruleDescriptor.actions.after):execAfter()>
}
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
if (HASEXCEPTION())
{
PREPORTERROR();
PRECOVER();
<@setErrorReturnValue()>
}
<if(ruleDescriptor.actions.after)>
else
{
<(ruleDescriptor.actions.after):execAfter()>
}<\n>
<endif>
<endif>
<endif>
<endif>
<if(trace)>ANTLR3_PRINTF("exit <ruleName> %d failed=%s backtracking=%s\n", LT(1), failed, BACKTRACKING);<endif>
<memoize()>
<if(finally)>
<finalCode(finalBlock=finally)>
<endif>
<scopeClean()>
<@postamble()>
return <ruleReturnValue()>;
}
/* $ANTLR end <ruleName> */
>>
finalCode(finalBlock) ::= <<
{
<finalBlock>
}
>>
catch(decl,action) ::= <<
/* catch(decl,action)
*/
{
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnType()> retval;<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.type> <a.name> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
ANTLR3_UINT32 <ruleDescriptor.name>_StartIndex;
<endif>
>>
ruleInitializations() ::= <<
/* Initialize rule variables
*/
<if(memoize)>
<ruleDescriptor.name>_StartIndex = INDEX();<\n>
<endif>
<ruleDescriptor.useScopes:{<scopeTop(sname=it)> = <scopePush(sname=it)>;}; separator="\n">
<ruleDescriptor.ruleScope:{<scopeTop(sname=it.name)> = <scopePush(sname=it.name)>;}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]
:{<labelType> <it.label.text>;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]
:{pANTLR3_VECTOR list_<it.label.text>;}; separator="\n"
>
<[ruleDescriptor.ruleLabels,ruleDescriptor.ruleListLabels]
:ruleLabelDef(label=it); separator="\n"
>
>>
ruleLabelInitializations() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels]
:{<it.label.text> = NULL;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels]
:{list_<it.label.text> = NULL;}; separator="\n"
>
<[ruleDescriptor.ruleLabels,ruleDescriptor.ruleListLabels]
:ruleLabelInitVal(label=it); separator="\n"
>
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!ruleDescriptor.isSynPred)>
retval.start = LT(1); retval.stop = retval.start;<\n>
<endif>
<endif>
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<labelType> <it.label.text>;}; separator="\n"
>
<ruleDescriptor.charLabels:{ANTLR3_UINT32 <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleListLabels]
:{pANTLR3_INT_TRIE list_<it.label.text>;}; separator="\n"
>
>>
lexerRuleLabelInit() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<it.label.text> = NULL;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleListLabels]
:{list_<it.label.text> = antlr3IntTrieNew(31);}; separator="\n"
>
>>
lexerRuleLabelFree() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<it.label.text> = NULL;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels,
ruleDescriptor.ruleListLabels]
:{list_<it.label.text>->free(list_<it.label.text>);}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( BACKTRACKING>0 ) { MEMOIZE(<ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex); }
<endif>
<endif>
>>
ruleCleanUp() ::= <<
// This is where rules clean up and exit
//
goto rule<ruleDescriptor.name>Ex; /* Prevent compiler warnings */
rule<ruleDescriptor.name>Ex: ;
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
<if(!ruleDescriptor.isSynPred)>
retval.stop = LT(-1);<\n>
<endif>
<endif>
<endif>
>>
scopeClean() ::= <<
<ruleDescriptor.useScopes:{<scopePop(sname=it)>}; separator="\n">
<ruleDescriptor.ruleScope:{<scopePop(sname=it.name)>}; separator="\n">
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules, which do not produce tokens.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// Comes from: <block.description>
/** \brief Lexer rule generated by ANTLR3
*
* $ANTLR start <ruleName>
*
* Looks to match the characters the constitute the token <ruleName>
* from the attached input stream.
*
*
* \remark
* - lexer->error == ANTLR3_TRUE if an exception was thrown.
*/
static ANTLR3_INLINE
void m<ruleName>(p<name> ctx<if(ruleDescriptor.parameterScope)>, <endif><ruleDescriptor.parameterScope:parameterScope(scope=it)>)
{
ANTLR3_UINT32 _type;
<ruleDeclarations()>
<ruleDescriptor.actions.declarations>
<lexerRuleLabelDefs()>
<if(trace)>System.out.println("enter <ruleName> '"+(char)LA(1)+"' line="+GETLINE()+":"+GETCHARPOSITIONINLINE()+" failed="+failed+" backtracking="+BACKTRACKING);<endif>
<if(nakedBlock)>
<ruleMemoization(rname=ruleName)>
<lexerRuleLabelInit()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
<ruleMemoization(rname=ruleName)>
<lexerRuleLabelInit()>
_type = <ruleName>;
<ruleDescriptor.actions.init>
<block>
LEXSTATE->type = _type;
<endif>
<if(trace)> ANTLR3_FPRINTF(stderr, "exit <ruleName> '%c' line=%d:%d failed = %d, backtracking =%d\n",LA(1),GETLINE(),GETCHARPOSITIONINLINE(),failed,BACKTRACKING);<endif>
<ruleCleanUp()>
<lexerRuleLabelFree()>
<(ruleDescriptor.actions.after):execAfter()>
<memoize>
}
// $ANTLR end <ruleName>
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
/** This is the entry point in to the lexer from an object that
* wants to generate the next token, such as a pCOMMON_TOKEN_STREAM
*/
static void
mTokens(p<name> ctx)
{
<block><\n>
goto ruleTokensEx; /* Prevent compiler warnings */
ruleTokensEx: ;
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
{
int alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
}
<@postbranch()>
}
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
{
// <fileName>:<description>
ANTLR3_UINT32 alt<decisionNumber>;
alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
}
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
{
int cnt<decisionNumber>=0;
<decls>
<@preloop()>
for (;;)
{
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
default:
if ( cnt<decisionNumber> >= 1 )
{
goto loop<decisionNumber>;
}
<ruleBacktrackFailure()>
<earlyExitEx()>
<@earlyExitException()>
goto rule<ruleDescriptor.name>Ex;
}
cnt<decisionNumber>++;
}
loop<decisionNumber>: ; /* Jump to here if this rule does not match */
<@postloop()>
}
>>
earlyExitEx() ::= <<
/* mismatchedSetEx()
*/
CONSTRUCTEX();
EXCEPTION->type = ANTLR3_EARLY_EXIT_EXCEPTION;
EXCEPTION->name = (void *)ANTLR3_EARLY_EXIT_NAME;
<\n>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
<decls>
<@preloop()>
for (;;)
{
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch (alt<decisionNumber>)
{
<alts:altSwitchCase()>
default:
goto loop<decisionNumber>; /* break out of the loop */
break;
}
}
loop<decisionNumber>: ; /* Jump out to here if this rule does not match */
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by antlr before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i>:
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
// <fileName>:<description>
{
<@declarations()>
<@initializations()>
<elements:element()>
<rew>
<@cleanup()>
}
>>
// E L E M E N T S
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label> = (<labelType>)<endif> MATCHT(<token>, &FOLLOW_<token>_in_<ruleName><elementIndex>);
<checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label> == NULL)
{
list_<label>=ctx->vectors->newVector(ctx->vectors);
}
list_<label>->add(list_<label>, <elem>, NULL);
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = LA(1);<\n>
<endif>
MATCHC(<char>);
<checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = LA(1);<\n>
<endif>
MATCHRANGE(<a>, <b>);
<checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= LA(1);<\n>
<else>
<label>=(<labelType>)LT(1);<\n>
<endif>
<endif>
if ( <s> )
{
CONSUME();
<postmatchCode>
<if(!LEXER)>
PERRORRECOVERY=ANTLR3_FALSE;
<endif>
<if(backtracking)>FAILEDFLAG=ANTLR3_FALSE;<\n><endif>
}
else
{
<ruleBacktrackFailure()>
<mismatchedSetEx()>
<@mismatchedSetException()>
<if(LEXER)>
LRECOVER();
<else>
RECOVERFROMMISMATCHEDSET(&FOLLOW_set_in_<ruleName><elementIndex>);
<endif>
goto rule<ruleDescriptor.name>Ex;
}<\n>
>>
mismatchedSetEx() ::= <<
CONSTRUCTEX();
EXCEPTION->type = ANTLR3_MISMATCHED_SET_EXCEPTION;
EXCEPTION->name = (void *)ANTLR3_MISMATCHED_SET_NAME;
<if(PARSER)>
EXCEPTION->expectingSet = &FOLLOW_set_in_<ruleName><elementIndex>;
<endif>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
<label>Start = GETCHARINDEX();
MATCHS(<string>);
<checkRuleBacktrackFailure()>
<label> = LEXSTATE->tokFactory->newToken(LEXSTATE->tokFactory);
<label>->setType(<label>, ANTLR3_TOKEN_INVALID);
<label>->setStartIndex(<label>, <label>Start);
<label>->setStopIndex(<label>, GETCHARINDEX()-1);
<label>->input = INPUT->tnstream->istream;
<else>
MATCHS(<string>);
<checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label>=(<labelType>)LT(1);<\n>
<endif>
MATCHANYT();
<checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = LA(1);<\n>
<endif>
MATCHANY();
<checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
FOLLOWPUSH(FOLLOW_<rule.name>_in_<ruleName><elementIndex>);
<if(label)><label>=<endif><if(scope)>ctx-><scope:delegateName()>-><endif><rule.name>(ctx<if(scope)>-><scope:delegateName()><endif><if(args)>, <args; separator=", "><endif>);<\n>
FOLLOWPOP();
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
/* <description> */
<if(label)>
{
ANTLR3_MARKER <label>Start<elementIndex> = GETCHARINDEX();
<if(scope)>ctx-><scope:delegateName()>-><endif>m<rule.name>(ctx<if(scope)>-><scope:delegateName()><endif> <if(args)>, <endif><args; separator=", ">);
<checkRuleBacktrackFailure()>
<label> = LEXSTATE->tokFactory->newToken(LEXSTATE->tokFactory);
<label>->setType(<label>, ANTLR3_TOKEN_INVALID);
<label>->setStartIndex(<label>, <label>Start<elementIndex>);
<label>->setStopIndex(<label>, GETCHARINDEX()-1);
<label>->input = INPUT;
}
<else>
<if(scope)>ctx-><scope:delegateName()>-><endif>m<rule.name>(ctx<if(scope)>-><scope:delegateName()><endif> <if(args)>, <endif><args; separator=", ">);
<checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
{
ANTLR3_UINT32 <label>Start<elementIndex>;
<labelType> <label>;
<label>Start<elementIndex> = GETCHARINDEX();
MATCHC(ANTLR3_CHARSTREAM_EOF);
<checkRuleBacktrackFailure()>
<label> = LEXSTATE->tokFactory->newToken(LEXSTATE->tokFactory);
<label>->setType(<label>, ANTLR3_TOKEN_EOF);
<label>->setStartIndex(<label>, <label>Start<elementIndex>);
<label>->setStopIndex(<label>, GETCHARINDEX()-1);
<label>->input = INPUT->tnstream->istream;
}
<else>
MATCHC(ANTLR3_CHARSTREAM_EOF);
<checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList, enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( LA(1)==ANTLR3_TOKEN_DOWN ) {
MATCHT(ANTLR3_TOKEN_DOWN, NULL);
<checkRuleBacktrackFailure()>
<children:element()>
MATCHT(ANTLR3_TOKEN_UP, NULL);
<checkRuleBacktrackFailure()>
}
<else>
MATCHT(ANTLR3_TOKEN_DOWN, NULL);
<checkRuleBacktrackFailure()>
<children:element()>
MATCHT(ANTLR3_TOKEN_UP, NULL);
<checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) )
{
<ruleBacktrackFailure()>
<newFPE(...)>
}
>>
newFPE() ::= <<
CONSTRUCTEX();
EXCEPTION->type = ANTLR3_FAILED_PREDICATE_EXCEPTION;
EXCEPTION->message = (void *)"<description>";
EXCEPTION->ruleName = (void *)"<ruleName>";
<\n>
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
{
int LA<decisionNumber>_<stateNumber> = LA(<k>);
<edges; separator="\nelse ">
else
{
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
<newNVException()>
goto rule<ruleDescriptor.name>Ex;
<endif>
}
}
>>
newNVException() ::= <<
CONSTRUCTEX();
EXCEPTION->type = ANTLR3_NO_VIABLE_ALT_EXCEPTION;
EXCEPTION->message = (void *)"<description>";
EXCEPTION->decisionNum = <decisionNumber>;
EXCEPTION->state = <stateNumber>;
<@noViableAltException()>
<\n>
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
{
int LA<decisionNumber>_<stateNumber> = LA(<k>);
<edges; separator="\nelse ">
}
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackStateDecls()::= <<
ANTLR3_UINT32 LA<decisionNumber>_<stateNumber>;
>>
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
{
/* dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState)
*/
int LA<decisionNumber>_<stateNumber> = LA(<k>);
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber>=<eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else
{
alt<decisionNumber>=<eotPredictsAlt>;
}<\n>
<endif>
<endif>
}
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber>=<alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>)
{
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( LA(<k>) )
{
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
<newNVException()>
goto rule<ruleDescriptor.name>Ex;<\n>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( LA(<k>) )
{
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( LA(<k>) )
{
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber>=<eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
{
<targetState>
}
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = cdfa<decisionNumber>.predict(ctx, RECOGNIZER, ISTREAM, &cdfa<decisionNumber>);
<checkRuleBacktrackFailure()>
>>
/* Dump DFA tables as static initialized arrays of shorts(16 bits)/characters(8 bits)
* which are then used to statically initialize the dfa structure, which means that there
* is no runtime initialization whatsoever, other than anything the C compiler might
* need to generate. In general the C compiler will lay out memory such that there is no
* runtime code required.
*/
cyclicDFA(dfa) ::= <<
/** Static dfa state tables for Cyclic dfa:
* <dfa.description>
*/
static const ANTLR3_INT32 dfa<dfa.decisionNumber>_eot[<dfa.numberOfStates>] =
{
<dfa.eot; wrap="\n", separator=", ", null="-1">
};
static const ANTLR3_INT32 dfa<dfa.decisionNumber>_eof[<dfa.numberOfStates>] =
{
<dfa.eof; wrap="\n", separator=", ", null="-1">
};
static const ANTLR3_INT32 dfa<dfa.decisionNumber>_min[<dfa.numberOfStates>] =
{
<dfa.min; wrap="\n", separator=", ", null="-1">
};
static const ANTLR3_INT32 dfa<dfa.decisionNumber>_max[<dfa.numberOfStates>] =
{
<dfa.max; wrap="\n", separator=", ", null="-1">
};
static const ANTLR3_INT32 dfa<dfa.decisionNumber>_accept[<dfa.numberOfStates>] =
{
<dfa.accept; wrap="\n", separator=", ", null="-1">
};
static const ANTLR3_INT32 dfa<dfa.decisionNumber>_special[<dfa.numberOfStates>] =
{
<dfa.special; wrap="\n", separator=", ", null="-1">
};
/** Used when there is no transition table entry for a particular state */
#define dfa<dfa.decisionNumber>_T_empty NULL
<dfa.edgeTransitionClassMap.keys:{ table |
static const ANTLR3_INT32 dfa<dfa.decisionNumber>_T<i0>[] =
{
<table; separator=", ", wrap="\n", null="-1">
};}; null = "">
/* Transition tables are a table of sub tables, with some tables
* reused for efficiency.
*/
static const ANTLR3_INT32 * const dfa<dfa.decisionNumber>_transitions[] =
{
<dfa.transitionEdgeTables:{xref|dfa<dfa.decisionNumber>_T<xref>}; separator=", ", wrap="\n", null="_empty">
};
<if(dfa.specialStateSTs)>
static ANTLR3_INT32 dfa<dfa.decisionNumber>_sst(p<name> ctx, pANTLR3_BASE_RECOGNIZER recognizer, pANTLR3_INT_STREAM is, pANTLR3_CYCLIC_DFA dfa, ANTLR3_INT32 s)
{
ANTLR3_INT32 _s;
_s = s;
switch (s)
{
<dfa.specialStateSTs:{state |
case <i0>:
<state>}; separator="\n">
}
<if(backtracking)>
if (BACKTRACKING > 0)
{
FAILEDFLAG = ANTLR3_TRUE;
return -1;
}
<endif>
CONSTRUCTEX();
EXCEPTION->type = ANTLR3_NO_VIABLE_ALT_EXCEPTION;
EXCEPTION->message = (void *)"<dfa.description>";
EXCEPTION->decisionNum = <dfa.decisionNumber>;
EXCEPTION->state = _s;
<@noViableAltException()>
return -1;
}
<endif>
<@errorMethod()>
/* Declare tracking structure for Cyclic DFA <dfa.decisionNumber>
*/
static
ANTLR3_CYCLIC_DFA cdfa<dfa.decisionNumber>
= {
<dfa.decisionNumber>, /* Decision number of this dfa */
/* Which decision this represents: */
(const pANTLR3_UCHAR)"<dfa.description>",
<if(dfa.specialStateSTs)>
(CDFA_SPECIAL_FUNC) dfa<dfa.decisionNumber>_sst,
<else>
(CDFA_SPECIAL_FUNC) antlr3dfaspecialStateTransition, /* Default special state transition function */
<endif>
antlr3dfaspecialTransition, /* DFA specialTransition is currently just a default function in the runtime */
antlr3dfapredict, /* DFA simulator function is in the runtime */
dfa<dfa.decisionNumber>_eot, /* EOT table */
dfa<dfa.decisionNumber>_eof, /* EOF table */
dfa<dfa.decisionNumber>_min, /* Minimum tokens for each state */
dfa<dfa.decisionNumber>_max, /* Maximum tokens for each state */
dfa<dfa.decisionNumber>_accept, /* Accept table */
dfa<dfa.decisionNumber>_special, /* Special transition states */
dfa<dfa.decisionNumber>_transitions /* Table of transition tables */
};
/* End of Cyclic DFA <dfa.decisionNumber>
* ---------------------
*/
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
{
ANTLR3_UINT32 LA<decisionNumber>_<stateNumber>;<\n>
ANTLR3_MARKER index<decisionNumber>_<stateNumber>;<\n>
LA<decisionNumber>_<stateNumber> = LA(1);<\n>
<if(semPredState)> <! get next lookahead symbol to test edges, then rewind !>
index<decisionNumber>_<stateNumber> = INDEX();<\n>
REWINDLAST();<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
<if(semPredState)> <! return input cursor to state before we rewound !>
SEEK(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s>=0 )
{
return s;
}
}
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>)
{
s = <targetStateNumber>;
}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "( (<left>) && (<right>) )"
orPredicates(operands) ::= "((<first(operands)>)<rest(operands):{o | ||(<o>)}>)"
notPredicate(pred) ::= "!( <evalPredicate(...)> )"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "<pred>(ctx)"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber> == <atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "LA(<k>) == <atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
((LA<decisionNumber>_<stateNumber> \>= <lower>) && (LA<decisionNumber>_<stateNumber> \<= <upper>))
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "((LA(<k>) \>= <lower>) && (LA(<k>) \<= <upper>))"
setTest(ranges) ::= "<ranges; separator=\" || \">"
// A T T R I B U T E S
makeScopeSet() ::= <<
/* makeScopeSet()
*/
/** Definition of the <scope.name> scope variable tracking
* structure. An instance of this structure is created by calling
* <name>_<scope.name>Push().
*/
typedef struct <scopeStruct(sname=scope.name,...)>_struct
{
/** Function that the user may provide to be called when the
* scope is destroyed (so you can free pANTLR3_HASH_TABLES and so on)
*
* \param POinter to an instance of this typedef/struct
*/
void (ANTLR3_CDECL *free) (struct <scopeStruct(sname=scope.name,...)>_struct * frame);
/* =============================================================================
* Programmer defined variables...
*/
<scope.attributes:{<it.decl>;}; separator="\n">
/* End of programmer defined variables
* =============================================================================
*/
}
<scopeStruct(sname=scope.name,...)>, * <scopeType(sname=scope.name,...)>;
>>
globalAttributeScopeDecl(scope) ::= <<
<if(scope.attributes)>
/* globalAttributeScopeDecl(scope)
*/
<makeScopeSet(...)>
<endif>
>>
ruleAttributeScopeDecl(scope) ::= <<
<if(scope.attributes)>
/* ruleAttributeScopeDecl(scope)
*/
<makeScopeSet(...)>
<endif>
>>
globalAttributeScopeFuncDecl(scope) ::=
<<
/* globalAttributeScopeFuncDecl(scope)
*/
<if(scope.attributes)>
/* -----------------------------------------------------------------------------
* Function declaration for creating a <name>_<scope.name> scope set
*/
static <scopeType(sname=scope.name,...)> <scopePushName(sname=scope.name,...)>(p<name> ctx);
static void ANTLR3_CDECL <scope.name>Free(<scopeType(sname=scope.name)> scope);
/* ----------------------------------------------------------------------------- */
<endif>
>>
globalAttributeScopeFuncMacro(scope) ::= <<
<if(scope.attributes)>
/* globalAttributeScopeFuncMacro(scope)
*/
/** Function for popping the top value from a <scopeStack(sname=scope.name)>
*/
void
<scopePopName(sname=scope.name,...)>(p<name> ctx)
{
// First see if the user defined a function they want to be called when a
// scope is popped/freed.
//
// If the user supplied the scope entries with a free function,then call it first
//
if (SCOPE_TOP(<scope.name>)->free != NULL)
{
SCOPE_TOP(<scope.name>)->free(SCOPE_TOP(<scope.name>));
}
// Now we decrement the scope's upper limit bound. We do not actually pop the scope as
// we want to reuse scope entries if we do continuous push and pops. Most scopes don't
// next too far so we don't want to keep freeing and allocating them
//
ctx-\><scopeStack(sname=scope.name,...)>_limit--;
SCOPE_TOP(<scope.name>) = (<scopeType(sname=scope.name)>)(ctx-\><scopeStack(sname=scope.name,...)>-\>get(ctx-\><scopeStack(sname=scope.name,...)>, ctx-\><scopeStack(sname=scope.name,...)>_limit - 1));
}
<endif>
>>
ruleAttributeScopeFuncDecl(scope) ::= <<
<if(scope.attributes)>
/* ruleAttributeScopeFuncDecl(scope)
*/
/* -----------------------------------------------------------------------------
* Function declarations for creating a <name>_<scope.name> scope set
*/
static <scopeType(sname=scope.name,...)> <scopePushName(sname=scope.name,...)>(p<name> ctx);
static void ANTLR3_CDECL <scope.name>Free(<scopeType(sname=scope.name)> scope);
/* ----------------------------------------------------------------------------- */
<endif>
>>
ruleAttributeScopeFuncMacro(scope) ::= <<
<if(scope.attributes)>
/* ruleAttributeScopeFuncMacro(scope)
*/
/** Function for popping the top value from a <scopeStack(sname=scope.name,...)>
*/
void
<scopePopName(sname=scope.name,...)>(p<name> ctx)
{
// First see if the user defined a function they want to be called when a
// scope is popped/freed.
//
// If the user supplied the scope entries with a free function,then call it first
//
if (SCOPE_TOP(<scope.name>)->free != NULL)
{
SCOPE_TOP(<scope.name>)->free(SCOPE_TOP(<scope.name>));
}
// Now we decrement the scope's upper limit bound. We do not actually pop the scope as
// we want to reuse scope entries if we do continuous push and pops. Most scopes don't
// next too far so we don't want to keep freeing and allocating them
//
ctx-\><scopeStack(sname=scope.name,...)>_limit--;
SCOPE_TOP(<scope.name>) = (<scopeType(sname=scope.name)>)(ctx-\><scopeStack(sname=scope.name,...)>-\>get(ctx-\><scopeStack(sname=scope.name,...)>, ctx-\><scopeStack(sname=scope.name,...)>_limit - 1));
}
<endif>
>>
globalAttributeScopeDef(scope) ::=
<<
/* globalAttributeScopeDef(scope)
*/
<if(scope.attributes)>
/** Pointer to the <scope.name> stack for use by <scopePushName(sname=scope.name)>()
* and <scopePopName(sname=scope.name,...)>()
*/
pANTLR3_STACK <scopeStack(sname=scope.name)>;
ANTLR3_UINT32 <scopeStack(sname=scope.name)>_limit;
/** Pointer to the top of the stack for the global scope <scopeStack(sname=scope.name)>
*/
<scopeType(sname=scope.name,...)> (*<scopePushName(sname=scope.name,...)>)(struct <name>_Ctx_struct * ctx);
<scopeType(sname=scope.name,...)> <scopeTopDecl(sname=scope.name,...)>;
<endif>
>>
ruleAttributeScopeDef(scope) ::= <<
<if(scope.attributes)>
/* ruleAttributeScopeDef(scope)
*/
/** Pointer to the <scope.name> stack for use by <scopePushName(sname=scope.name)>()
* and <scopePopName(sname=scope.name,...)>()
*/
pANTLR3_STACK <scopeStack(sname=scope.name,...)>;
ANTLR3_UINT32 <scopeStack(sname=scope.name,...)>_limit;
<scopeType(sname=scope.name,...)> (*<scopePushName(sname=scope.name,...)>)(struct <name>_Ctx_struct * ctx);
<scopeType(sname=scope.name,...)> <scopeTopDecl(sname=scope.name,...)>;
<endif>
>>
globalAttributeScopeFuncs(scope) ::= <<
<if(scope.attributes)>
/* globalAttributeScopeFuncs(scope)
*/
<attributeFuncs(scope)>
<endif>
>>
ruleAttributeScopeFuncs(scope) ::= <<
<if(scope.attributes)>
/* ruleAttributeScopeFuncs(scope)
*/
<attributeFuncs(scope)>
<endif>
>>
globalAttributeScope(scope) ::= <<
<if(scope.attributes)>
/* globalAttributeScope(scope)
*/
ctx-><scopePushName(sname=scope.name,...)> = <scopePushName(sname=scope.name,...)>;
ctx-><scopeStack(sname=scope.name,...)> = antlr3StackNew(0);
ctx-><scopeStack(sname=scope.name,...)>_limit = 0;
<scopeTop(sname=scope.name,...)> = NULL;
<endif>
>>
ruleAttributeScope(scope) ::=
<<
<if(scope.attributes)>
/* ruleAttributeScope(scope)
*/
ctx-><scopePushName(sname=scope.name,...)> = <scopePushName(sname=scope.name,...)>;
ctx-><scopeStack(sname=scope.name,...)> = antlr3StackNew(0);
ctx-><scopeStack(sname=scope.name,...)>_limit = 0;
<scopeTop(sname=scope.name,...)> = NULL;
<endif>
>>
globalAttributeScopeFree(scope) ::= <<
<if(scope.attributes)>
/* globalAttributeScope(scope)
*/
ctx-><scopeStack(sname=scope.name,...)>-\>free(ctx-><scopeStack(sname=scope.name,...)>);
<endif>
>>
ruleAttributeScopeFree(scope) ::=
<<
<if(scope.attributes)>
/* ruleAttributeScope(scope)
*/
ctx-><scopeStack(sname=scope.name,...)>-\>free(ctx-><scopeStack(sname=scope.name,...)>);
<endif>
>>
scopeTopDecl(sname) ::= <<
p<name>_<sname>Top
>>
scopeTop(sname) ::= <<
ctx-><scopeTopDecl(sname=sname,...)>
>>
scopePop(sname) ::= <<
<scopePopName(sname=sname,...)>(ctx);
>>
scopePush(sname) ::= <<
p<name>_<sname>Push(ctx)
>>
scopePopName(sname) ::= <<
p<name>_<sname>Pop
>>
scopePushName(sname) ::= <<
p<name>_<sname>Push
>>
scopeType(sname) ::= <<
p<name>_<sname>_SCOPE
>>
scopeStruct(sname) ::= <<
<name>_<sname>_SCOPE
>>
scopeStack(sname) ::= <<
p<name>_<sname>Stack
>>
attributeFuncs(scope) ::= <<
<if(scope.attributes)>
/* attributeFuncs(scope)
*/
static void ANTLR3_CDECL <scope.name>Free(<scopeType(sname=scope.name)> scope)
{
ANTLR3_FREE(scope);
}
/** \brief Allocate initial memory for a <name> <scope.name> scope variable stack entry and
* add it to the top of the stack.
*
* \remark
* By default the structure is freed with ANTLR_FREE(), but you can use the
* the \@init action to install a pointer to a custom free() routine by
* adding the code:
* \code
* <scopeTop(sname=scope.name)>->free = myroutine;
* \endcode
*
* With lots of comments of course! The routine should be declared in
* \@members { } as:
* \code
* void ANTLR3_CDECL myfunc( <scopeType(sname=scope.name)> ptr).
* \endcode
*
* It should perform any custom freeing stuff that you need (call ANTLR_FREE3, not free()
* NB: It should not free the pointer it is given, which is the scope stack entry itself
* and will be freed by the function that calls your custom free routine.
*
*/
static <scopeType(sname=scope.name)>
<scopePushName(sname=scope.name)>(p<name> ctx)
{
/* Pointer used to create a new set of attributes
*/
<scopeType(sname=scope.name)> newAttributes;
/* Allocate the memory for a new structure if we need one.
*/
if (ctx-\><scopeStack(sname=scope.name)>-\>size(ctx-\><scopeStack(sname=scope.name)>) \> ctx-\><scopeStack(sname=scope.name)>_limit)
{
// The current limit value was less than the number of scopes available on the stack so
// we can just reuse one. Our limit tracks the stack count, so the index of the entry we want
// is one less than that, or conveniently, the current value of limit.
//
newAttributes = (<scopeType(sname=scope.name)>)ctx-><scopeStack(sname=scope.name)>->get(ctx-><scopeStack(sname=scope.name)>, ctx-\><scopeStack(sname=scope.name)>_limit);
}
else
{
// Need a new allocation
//
newAttributes = (<scopeType(sname=scope.name)>) ANTLR3_MALLOC(sizeof(<scopeStruct(sname=scope.name)>));
if (newAttributes != NULL)
{
/* Standard ANTLR3 library implementation
*/
ctx-\><scopeStack(sname=scope.name)>-\>push(ctx-\><scopeStack(sname=scope.name)>, newAttributes, (void (*)(void *))<scope.name>Free);
}
}
// Blank out any previous free pointer, the user might or might install a new one.
//
newAttributes->free = NULL;
// Indicate the position in the available stack that the current level is at
//
ctx-><scopeStack(sname=scope.name)>_limit++;
/* Return value is the pointer to the new entry, which may be used locally
* without de-referencing via the context.
*/
return newAttributes;
}<\n>
<endif>
>>
returnStructName() ::= "<it.name>_return"
returnType() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.grammar.recognizerName>_<ruleDescriptor:returnStructName()>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
<else>
ANTLR3_BOOLEAN
<endif>
>>
/** Generate the C type associated with a single or multiple return
* value(s).
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.grammar.recognizerName>_<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "0".
*/
initValue(typeName) ::= <<
<cTypeInitMap.(typeName)>
>>
/** Define a rule label */
ruleLabelDef(label) ::= <<
<ruleLabelType(referencedRule=label.referencedRule)> <label.label.text>;
#undef RETURN_TYPE_<label.label.text>
#define RETURN_TYPE_<label.label.text> <ruleLabelType(referencedRule=label.referencedRule)><\n>
>>
/** Rule label default value */
ruleLabelInitVal(label) ::= <<
<if(label.referencedRule.hasSingleReturnValue)>
<label.label.text> = <initValue(label.referencedRule.singleValueReturnType)>;
<endif>
>>
ASTLabelType() ::= "<if(recognizer.ASTLabelType)><recognizer.ASTLabelType><else>pANTLR3_BASE_TREE<endif>"
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasMultipleReturnValues)>
typedef struct <ruleDescriptor.grammar.recognizerName>_<ruleDescriptor:returnStructName()>_struct
{
<if(!TREE_PARSER)>
/** Generic return elements for ANTLR3 rules that are not in tree parsers or returning trees
*/
pANTLR3_COMMON_TOKEN start;
pANTLR3_COMMON_TOKEN stop;
<else>
<recognizer.ASTLabelType> start;
<recognizer.ASTLabelType> stop;
<endif>
<@ruleReturnMembers()>
<ruleDescriptor.returnScope.attributes:{<it.decl>;}; separator="\n">
}
<ruleDescriptor.grammar.recognizerName>_<ruleDescriptor:returnStructName()>;<\n><\n>
<endif>
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name>=<expr>;"
/** Note that the scopeAttributeRef does not have access to the
* grammar name directly
*/
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
((SCOPE_TYPE(<scope>))(ctx->SCOPE_STACK(<scope>)->get( ctx->SCOPE_STACK(<scope>), ctx->SCOPE_STACK(<scope>)->size(ctx->SCOPE_STACK(<scope>)) - <negIndex> - 1) ))-><attr.name>
<else>
<if(index)>
((SCOPE_TYPE(<scope>))(ctx->SCOPE_STACK(<scope>)->get(ctx->SCOPE_STACK(<scope>), (ANTLR3_UINT32)<index> ) ))-><attr.name>
<else>
(SCOPE_TOP(<scope>))-><attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
((SCOPE_TYPE(<scope>))(ctx->SCOPE_STACK(<scope>)->get( ctx->SCOPE_STACK(<scope>), ctx->SCOPE_STACK(<scope>)->size(ctx->SCOPE_STACK(<scope>)) - <negIndex> - 1) ))-><attr.name> = <expr>;
<else>
<if(index)>
((SCOPE_TYPE(<scope>))(ctx->SCOPE_STACK(<scope>)->get(ctx->SCOPE_STACK(<scope>), (ANTLR3_UINT32)<index> ) ))-><attr.name> = <expr>;
<else>
(SCOPE_TOP(<scope>))-><attr.name>=<expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "ctx->SCOPE_STACK(<scope>)"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<scope>.<attr.name>
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>=<expr>;
<else>
<attr.name>=<expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
//
tokenLabelPropertyRef_text(scope,attr) ::= "(<scope>->getText(<scope>))"
tokenLabelPropertyRef_type(scope,attr) ::= "(<scope>->getType(<scope>))"
tokenLabelPropertyRef_line(scope,attr) ::= "(<scope>->getLine(<scope>))"
tokenLabelPropertyRef_pos(scope,attr) ::= "(<scope>->getCharPositionInLine(<scope>))"
tokenLabelPropertyRef_channel(scope,attr) ::= "(<scope>->getChannel(<scope>))"
tokenLabelPropertyRef_index(scope,attr) ::= "(<scope>->getTokenIndex(<scope>))"
tokenLabelPropertyRef_tree(scope,attr) ::= "(<scope>->tree)"
tokenLabelPropertyRef_int(scope,attr) ::= "(<scope>->getText(<scope>)->toInt32(<scope>->getText(<scope>)))"
ruleLabelPropertyRef_start(scope,attr) ::= "(<scope>.start)"
ruleLabelPropertyRef_stop(scope,attr) ::= "(<scope>.stop)"
ruleLabelPropertyRef_tree(scope,attr) ::= "(<scope>.tree)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
(STRSTREAM->toStringSS(STRSTREAM, <scope>.start, <scope>.start))
<else>
(STRSTREAM->toStringTT(STRSTREAM, <scope>.start, <scope>.stop))
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "<scope>.st"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::= "(<scope>->getType(<scope>))"
lexerRuleLabelPropertyRef_line(scope,attr) ::= "(<scope>->getLine(<scope>))"
lexerRuleLabelPropertyRef_pos(scope,attr) ::= "(<scope>->getCharPositionInLine(<scope>))"
lexerRuleLabelPropertyRef_channel(scope,attr) ::= "(<scope>->getChannel(<scope>))"
lexerRuleLabelPropertyRef_index(scope,attr) ::= "(<scope>->getTokenIndex(<scope>))"
lexerRuleLabelPropertyRef_text(scope,attr) ::= "(<scope>->getText(<scope>))"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "retval.start"
rulePropertyRef_stop(scope,attr) ::= "retval.stop"
rulePropertyRef_tree(scope,attr) ::= "retval.tree"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
INPUT->toStringSS(INPUT, ADAPTOR->getTokenStartIndex(ADAPTOR, retval.start), ADAPTOR->getTokenStopIndex(ADAPTOR, retval.start))
<else>
STRSTREAM->toStringTT(STRSTREAM, retval.start, LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.st"
lexerRulePropertyRef_text(scope,attr) ::= "LEXER->getText(LEXER)"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "LEXSTATE->tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "LEXSTATE->tokenStartCharPositionInLine"
lexerRulePropertyRef_channel(scope,attr) ::= "LEXSTATE->channel"
lexerRulePropertyRef_start(scope,attr) ::= "LEXSTATE->tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(LEXER->getCharIndex(LEXER)-1)"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_int(scope,attr) ::= "LEXER->getText(LEXER)->toInt32(LEXER->getText(LEXER))"
// setting $st and $tree is allowed in local rule. everything else is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.tree=<expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.st=<expr>;"
/** How to deal with an @after for C targets. Because we cannot rely on
* any garbage collection, after code is executed even in backtracking
* mode. Must be documented clearly.
*/
execAfter(action) ::= <<
{
<action>
}
>>
/** How to execute an action (when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
<if(actions.(actionScope).synpredgate)>
if ( <actions.(actionScope).synpredgate> )
{
<action>
}
<else>
if ( BACKTRACKING == 0 )
{
<action>
}
<endif>
<else>
{
<action>
}
<endif>
>>
// M I S C (properties, etc...)
bitsetDeclare(name, words64) ::= <<
/** Bitset defining follow set for error recovery in rule state: <name> */
static ANTLR3_BITWORD <name>_bits[] = { <words64:{ANTLR3_UINT64_LIT(<it>)}; separator=", "> };
static ANTLR3_BITSET_LIST <name> = { <name>_bits, <length(words64)> };
>>
bitset(name, words64) ::= <<
antlr3BitsetSetAPI(&<name>);<\n>
>>
codeFileExtension() ::= ".c"
true() ::= "ANTLR3_TRUE"
false() ::= "ANTLR3_FALSE"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/C/ASTTreeParser.stg 0000644 0001750 0001750 00000023355 11256465232 030012 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
http://www.temporal-wave.com
http://www.linkedin.com/in/jimidle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> _last;<\n>
<ASTLabelType> _first_0;<\n>
>>
/** Add a variable to track last element matched */
ruleInitializations() ::= <<
<super.ruleInitializations()>
_last = NULL;<\n>
_first_0 = NULL;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( BACKTRACKING ==0 ) {<endif>
<if(rewriteMode)>
retval.tree = (<ASTLabelType>)_first_0;
if ( ADAPTOR->getParent(ADAPTOR, retval.tree) != NULL && ADAPTOR->isNilNode(ADAPTOR, ADAPTOR->getParent(ADAPTOR, retval.tree) ) )
{
retval.tree = (<ASTLabelType>)ADAPTOR->getParent(ADAPTOR, retval.tree);
}
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = (<ASTLabelType>)LT(1);
{
<ASTLabelType> _save_last_<treeLevel>;
<ASTLabelType> _first_<treeLevel>;
<if(!rewriteMode)>
<ASTLabelType> root_<treeLevel>;
<endif>
_save_last_<treeLevel> = _last;
_first_<treeLevel> = NULL;
<if(!rewriteMode)>
root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->nilNode(ADAPTOR));
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( BACKTRACKING ==0 ) {<endif>
<if(root.el.rule)>
if ( _first_<enclosingTreeLevel> == NULL ) _first_<enclosingTreeLevel> = <root.el.label>.tree;
<else>
if ( _first_<enclosingTreeLevel> == NULL ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<if(backtracking)>}<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( LA(1)==ANTLR3_TOKEN_DOWN ) {
MATCHT(ANTLR3_TOKEN_DOWN, NULL);
<children:element()>
MATCHT(ANTLR3_TOKEN_UP, NULL);
}
<else>
MATCHT(ANTLR3_TOKEN_DOWN, NULL);
<children:element()>
MATCHT(ANTLR3_TOKEN_UP, NULL);
<endif>
<if(!rewriteMode)>
ADAPTOR->addChild(ADAPTOR, root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( BACKTRACKING ==0 ) {<endif>
<if(hetero)>
<label>_tree = <hetero>New(<label>);
<else>
<label>_tree = (<ASTLabelType>)ADAPTOR->dupNode(ADAPTOR, <label>);
<endif>
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else>
<if(backtracking)>if ( BACKTRACKING ==0 ) {<endif>
if ( _first_<treeLevel> == NULL ) _first_<treeLevel> = <label>;
<if(backtracking)>}<endif>
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( BACKTRACKING == 0 ) {<endif>
<if(hetero)>
<label>_tree = <hetero>New(<label>);
<else>
<label>_tree = (<ASTLabelType>)ADAPTOR->dupNode(ADAPTOR, <label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)ADAPTOR->becomeRoot(ADAPTOR, <label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard and auto dup the node/subtree */
wildcard(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.wildcard(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)ADAPTOR->dupTree(ADAPTOR, <label>);
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( BACKTRACKING == 0 ) {<endif>
<if(hetero)>
<label>_tree = <hetero>New(<label>);
<else>
<label>_tree = (<ASTLabelType>)ADAPTOR->dupNode(ADAPTOR, <label>);
<endif><\n>
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ){<endif>
<if(hetero)>
<label>_tree = <hetero>New(<label>);
<else>
<label>_tree = (<ASTLabelType>)ADAPTOR->dupNode(ADAPTOR, <label>);
<endif>
root_<treeLevel> = (<ASTLabelType>)ADAPTOR->becomeRoot(ADAPTOR, <label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )
{
<endif>
<if(!rewriteMode)>
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, <label>.tree);
<else>
if ( _first_<treeLevel> == NULL ) _first_<treeLevel> = <label>.tree;
<endif>
<if(backtracking)>}<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<super.listLabelAST(elem=label,...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( ( <actions.(actionScope).synpredgate> ) ) <endif>root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <label>.tree, root_<treeLevel>));
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<super.listLabelAST(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)LT(1);
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change nextToken to nextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
<hetero>New(stream_<token>->nextNode(stream_<token>))
<else>
stream_<token>->nextNode(stream_<token>)
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp(...)>
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {<\n>
<endif>
<if(!ruleDescriptor.isSynPred)>
retval.stop = LT(-1);<\n>
<endif>
retval.tree = (<ASTLabelType>)ADAPTOR->rulePostProcessing(ADAPTOR, root_0);
<if(backtracking)>
}
<endif>
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>);}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>);}; separator="\n">
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/C/ASTParser.stg 0000644 0001750 0001750 00000016353 11256465232 027172 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
http://www.temporal-wave.com
http://www.linkedin.com/in/jimidle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
retval.tree = (<ASTLabelType>)(ADAPTOR->errorNode(ADAPTOR, INPUT, retval.start, LT(-1), EXCEPTION));
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)(ADAPTOR->create(ADAPTOR, <label>));
ADAPTOR->addChild(ADAPTOR, root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
root_0 = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <label>_tree, root_0));
<if(backtracking)>}<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>ADAPTOR->addChild(ADAPTOR, root_0, <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label>=(<labelType>)LT(1);<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <createNodeFromToken(...)>, root_0));})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>ADAPTOR->addChild(ADAPTOR, root_0, <label>.tree);
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <label>.tree, root_0));
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabelAST(...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabelAST(...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabelAST(...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)(ADAPTOR->create(ADAPTOR, <label>));
ADAPTOR->addChild(ADAPTOR, root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)(ADAPTOR->create(ADAPTOR, <label>));
root_0 = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <label>_tree, root_0));
<if(backtracking)>}<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
<hetero>New(<label>) <! new MethodNode(IDLabel) !>
<else>
(<ASTLabelType>)(ADAPTOR->create(ADAPTOR, <label>))
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp(...)>
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{<\n>
<endif>
<if(!ruleDescriptor.isSynPred)>
retval.stop = LT(-1);<\n>
<endif>
retval.tree = (<ASTLabelType>)(ADAPTOR->rulePostProcessing(ADAPTOR, root_0));
ADAPTOR->setTokenBoundaries(ADAPTOR, retval.tree, retval.start, retval.stop);
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>);}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>);}; separator="\n">
<if(backtracking)>
}<\n>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/C/Dbg.stg 0000644 0001750 0001750 00000014577 11256465232 026070 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
http://www.temporal-wave.com
http://www.linkedin.com/in/jimidle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to normal C output;
* If ASTs are built, then you'll also get ASTDbg.stg loaded.
*/
group Dbg;
@genericParser.members() ::= <<
<if(grammar.grammarIsRoot)>
const char *
ruleNames[] =
{
"invalidRule", <grammar.allImportedRules:{rST | "<rST.name>"}; wrap="\n ", separator=", ">
};<\n>
<endif>
<if(grammar.grammarIsRoot)> <! grammar imports other grammar(s) !>
static ANTLR3_UINT32 ruleLevel = 0;
static ANTLR3_UINT32 getRuleLevel()
{
return ruleLevel;
}
static void incRuleLevel()
{
ruleLevel++;
}
static void decRuleLevel()
{
ruleLevel--;
}
<else> <! imported grammar !>
static ANTLR3_UINT32
getRuleLevel()
{
return <grammar.delegators:{g| <g:delegateName()>}>->getRuleLevel();
}
static void incRuleLevel()
{
<grammar.delegators:{g| <g:delegateName()>}>->incRuleLevel();
}
static void
decRuleLevel()
{
<grammar.delegators:{g| <g:delegateName()>}>.decRuleLevel();
}
<endif>
<if(profile)>
// Profiling not yet implemented for C target
//
<endif>
<if(grammar.grammarIsRoot)>
<ctorForPredefinedListener()>
<else>
<ctorForDelegateGrammar()>
<endif>
static ANTLR3_BOOLEAN
evalPredicate(p<name> ctx, ANTLR3_BOOLEAN result, const char * predicate)
{
DBG->semanticPredicate(DBG, result, predicate);
return result;
}<\n>
>>
@genericParser.debugStuff() ::= <<
<if(grammar.grammarIsRoot)>
<createListenerAndHandshake()>
<endif>
>>
ctorForProfilingRootGrammar() ::= <<
>>
/** Basically we don't want to set any dbg listeners as root will have it. */
ctorForDelegateGrammar() ::= <<
>>
ctorForPredefinedListener() ::= <<
>>
createListenerAndHandshake() ::= <<
{
// DEBUG MODE code
//
pANTLR3_DEBUG_EVENT_LISTENER proxy;
proxy = antlr3DebugListenerNew();
proxy->grammarFileName = INPUT->tokenSource->strFactory->newStr8(INPUT->tokenSource->strFactory, (pANTLR3_UINT8)ctx->getGrammarFileName());
<if(TREE_PARSER)>
proxy->adaptor = ADAPTOR;
<endif>
PARSER->setDebugListener(PARSER, proxy);
// Try to connect to the debugger (waits forever for a connection)
//
proxy->handshake(proxy);
// End DEBUG MODE code
//
}
>>
@rule.preamble() ::= <<
DBG->enterRule(DBG, getGrammarFileName(), (const char *)"<ruleName>");
if ( getRuleLevel()==0 )
{
DBG->commence(DBG);
}
incRuleLevel();
DBG->location(DBG, <ruleDescriptor.tree.line>, <ruleDescriptor.tree.column>);<\n>
>>
@rule.postamble() ::= <<
DBG->location(DBG, <ruleDescriptor.EORNode.line>, <ruleDescriptor.EORNode.column>);<\n>
DBG->exitRule(DBG, getGrammarFileName(), (const char *)"<ruleName>");
decRuleLevel();
if ( getRuleLevel()==0 )
{
DBG->terminate(DBG);
}
<\n>
>>
@synpred.start() ::= "DBG->beginBacktrack(DBG, BACKTRACKING);"
@synpred.stop() ::= "DBG->endBacktrack(DBG, BACKTRACKING, success);"
// Common debug event triggers used by region overrides below
enterSubRule() ::=
"DBG->enterSubRule(DBG, <decisionNumber>);<\n>"
exitSubRule() ::=
"DBG->exitSubRule(DBG, <decisionNumber>);<\n>"
enterDecision() ::=
"DBG->enterDecision(DBG, <decisionNumber>);<\n>"
exitDecision() ::=
"DBG->exitDecision(DBG, <decisionNumber>);<\n>"
enterAlt(n) ::= "DBG->enterAlt(DBG, <n>);<\n>"
// Region overrides that tell various constructs to add debugging triggers
@block.predecision() ::= "<enterSubRule()><enterDecision()>"
@block.postdecision() ::= "<exitDecision()>"
@block.postbranch() ::= "<exitSubRule()>"
@ruleBlock.predecision() ::= "<enterDecision()>"
@ruleBlock.postdecision() ::= "<exitDecision()>"
@ruleBlockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@blockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@positiveClosureBlock.preloop() ::= "<enterSubRule()>"
@positiveClosureBlock.postloop() ::= "<exitSubRule()>"
@positiveClosureBlock.predecision() ::= "<enterDecision()>"
@positiveClosureBlock.postdecision() ::= "<exitDecision()>"
@positiveClosureBlock.earlyExitException() ::=
"DBG->recognitionException(DBG, EXCEPTION);<\n>"
@closureBlock.preloop() ::= "<enterSubRule()>"
@closureBlock.postloop() ::= "<exitSubRule()>"
@closureBlock.predecision() ::= "<enterDecision()>"
@closureBlock.postdecision() ::= "<exitDecision()>"
@altSwitchCase.prealt() ::= "<enterAlt(n=i)>"
@element.prematch() ::=
"DBG->location(DBG, <it.line>, <it.pos>);"
@matchSet.mismatchedSetException() ::=
"DBG->recognitionException(DBG, EXCEPTION);"
@newNVException.noViableAltException() ::= "DBG->recognitionException(DBG, EXCEPTION);"
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = cdfa<decisionNumber>.predict(ctx, RECOGNIZER, ISTREAM, &cdfa<decisionNumber>);
if (HASEXCEPTION())
{
DBG->recognitionException(DBG, EXCEPTION);
goto rule<ruleDescriptor.name>Ex;
}
<checkRuleBacktrackFailure()>
>>
@cyclicDFA.errorMethod() ::= <<
//static void
//dfaError(p<name> ctx)
//{
// DBG->recognitionException(DBG, EXCEPTION);
//}
>>
/** Force predicate validation to trigger an event */
evalPredicate(pred,description) ::= <<
evalPredicate(ctx, <pred>, (const char *)"<description>")
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/C/AST.stg 0000644 0001750 0001750 00000045627 11256465232 026023 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
http://www.temporal-wave.com
http://www.linkedin.com/in/jimidle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
/** Add an adaptor property that knows how to build trees */
@headerFile.members() ::= <<
/* @headerFile.members() */
pANTLR3_BASE_TREE_ADAPTOR adaptor;
pANTLR3_VECTOR_FACTORY vectors;
/* End @headerFile.members() */
>>
/** Install the tree adaptor interface pointer and anything else that
* tree parsers and producers require.
*/
@genericParser.apifuncs() ::= <<
<if(PARSER)>
ADAPTOR = ANTLR3_TREE_ADAPTORNew(instream->tstream->tokenSource->strFactory);<\n>
<endif>
ctx->vectors = antlr3VectorFactoryNew(0);
>>
@genericParser.cleanup() ::= <<
ctx->vectors->close(ctx->vectors);
<if(PARSER)>
/* We created the adaptor so we must free it
*/
ADAPTOR->free(ADAPTOR);
<endif>
>>
@returnScope.ruleReturnMembers() ::= <<
<super.ASTLabelType()> tree;
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> root_0;<\n>
>>
ruleInitializations() ::= <<
<super.ruleInitializations()>
root_0 = NULL;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<ruleDescriptor.tokenLabels:{<ASTLabelType> <it.label.text>_tree;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<ASTLabelType> <it.label.text>_tree;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{pANTLR3_REWRITE_RULE_<rewriteElementType>_STREAM stream_<it>;}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{pANTLR3_REWRITE_RULE_SUBTREE_STREAM stream_<it>;}; separator="\n">
>>
/* Note that we defer the actual creation of any rewrite streams we need here and just initialize
* them to NULL. This saves creating huge numbers of rewrite streams that cannot be used as only
* one alt will be taken in a rule, but we are declaring all the streams here. So we define
* a macro that conatins the create code, then use this macro later to check if the stream
* has been created yet. Checking for NULL is almost free in C.
*/
ruleLabelInitializations() ::= <<
<super.ruleLabelInitializations()>
<ruleDescriptor.tokenLabels:{<it.label.text>_tree = NULL;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<it.label.text>_tree = NULL;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{stream_<it> = NULL;
#define CREATE_stream_<it> if (stream_<it> == NULL) {stream_<it> = antlr3RewriteRule<rewriteElementType>StreamNewAE(ADAPTOR, RECOGNIZER, (pANTLR3_UINT8)"token <it>"); } }; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{stream_<it> = NULL;
#define CREATE_stream_<it> if (stream_<it> == NULL) {stream_<it> = antlr3RewriteRuleSubtreeStreamNewAE(ADAPTOR, RECOGNIZER, (pANTLR3_UINT8)"rule <it>"); }}; separator="\n">
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.tree = NULL;
<endif>
>>
/** a rule label including default value */
ruleLabelInitVal(label) ::= <<
<super.ruleLabelInitVal(...)>
<label.label.text>.tree = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;<\n>
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<endif>
<endif>
>>
@alt.initializations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = (<ASTLabelType>)(ADAPTOR->nilNode(ADAPTOR));<\n>
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
//
/** ID but track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) { <endif>CREATE_stream_<token>; stream_<token>->add(stream_<token>, <label>, NULL);<if(backtracking)> }<endif><\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>CREATE_stream_<token>; stream_<token>->add(stream_<token>, <label>, NULL);<if(backtracking)> }<endif><\n>
>>
wildcardTrack(label,elementIndex) ::= <<
<super.wildcard(...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) { <endif>CREATE_stream_<rule.name>; stream_<rule.name>->add(stream_<rule.name>, <label>.tree, NULL);<if(backtracking)> }<endif>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabelTrack(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) { <endif>CREATE_stream_<rule.name>; stream_<rule.name>->add(stream_<rule.name>, <label>.tree, NULL);<if(backtracking)> }<endif>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabelAST(...)>
>>
// RULE REF AST
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/* How to accumulate lists when we are doing rewrite tracking...
*/
listLabelTrack(label) ::= <<
/* listLabelTrack(label)
*/
if (list_<label> == NULL)
{
list_<label>=ctx->vectors->newVector(ctx->vectors);
}
list_<label>->add(list_<label>, <label>.tree, NULL);
>>
/* How to accumulate lists of rule outputs (only allowed with AST
* option but if the user is going to walk the tree, they will want
* all their custom elements from rule returns.
*
* Normally, we use inline structures (which the compiler lays down
* code to copy from heap allocations. However, here we want to accumulate copies
* of the returned structures because we are adding them to a list. This only makes sense if the
* grammar is not rewriting the tree as a tree rewrite only preserves the tree, not the object/structure
* returned from the rule. The rewrite will extract the tree pointer. However, if we are not going to
* do a tree re-write, then the user may wish to iterate the structures returned by the rule in
* action code and will expect the user defined returns[] elements to be available when they do this.
* Hence we cannot just preserve the tree that was returned. So, we must copy the local structure and provide
* a function that can free the allocated space. We cannot know how to free user allocated elements and
* presume that the user will know to do this using their own factories for the structures they allocate.
*/
listLabelAST(label) ::= <<
if (list_<label> == NULL)
{
list_<label>=ctx->vectors->newVector(ctx->vectors);
}
{
RETURN_TYPE_<label> * tcopy;
tcopy = (RETURN_TYPE_<label> *)ANTLR3_MALLOC(sizeof(RETURN_TYPE_<label>)); /* Note no memory allocation checks! */
ANTLR3_MEMCPY((void *)(tcopy), (const void *)&<label>, sizeof(RETURN_TYPE_<label>));
list_<label>->add(list_<label>, (void *)tcopy, freeScope); /* Add whatever the return type is */<\n>
}
>>
// R e w r i t e
rewriteCode(
alts,
description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel,
enclosingTreeLevel,
treeLevel) ::=
<<
/* AST REWRITE
* elements : <referencedElementsDeep; separator=", ">
* token labels : <referencedTokenLabels; separator=", ">
* rule labels : <referencedRuleLabels; separator=", ">
* token list labels : <referencedTokenListLabels; separator=", ">
* rule list labels : <referencedRuleListLabels; separator=", ">
*/
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) <\n>
<endif>
{
<rewriteCodeLabelsDecl()>
<rewriteCodeLabelsInit()>
root_0 = (<ASTLabelType>)(ADAPTOR->nilNode(ADAPTOR));
<prevRuleRootRef()>.tree = root_0;
<alts:rewriteAlt(); separator="else ">
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.tree = (<ASTLabelType>)(ADAPTOR->rulePostProcessing(ADAPTOR, root_0));
INPUT->replaceChildren(INPUT, ADAPTOR->getParent(ADAPTOR, retval.start),
ADAPTOR->getChildIndex(ADAPTOR, retval.start),
ADAPTOR->getChildIndex(ADAPTOR, _last),
retval.tree);
<endif>
<endif>
<prevRuleRootRef()>.tree = root_0; // set result root
<rewriteCodeLabelsFree()>
}
>>
rewriteCodeLabelsDecl() ::= <<
<referencedTokenLabels
:{pANTLR3_REWRITE_RULE_<rewriteElementType>_STREAM stream_<it>;};
separator="\n"
>
<referencedTokenListLabels
:{pANTLR3_REWRITE_RULE_<rewriteElementType>_STREAM stream_<it>;};
separator="\n"
>
<referencedRuleLabels
:{pANTLR3_REWRITE_RULE_SUBTREE_STREAM stream_<it>;};
separator="\n"
>
<referencedRuleListLabels
:{pANTLR3_REWRITE_RULE_SUBTREE_STREAM stream_<it>;};
separator="\n"
>
>>
rewriteCodeLabelsInit() ::= <<
<referencedTokenLabels
:{stream_<it>=antlr3RewriteRule<rewriteElementType>StreamNewAEE(ADAPTOR, RECOGNIZER, (pANTLR3_UINT8)"token <it>", <it>);};
separator="\n"
>
<referencedTokenListLabels
:{stream_<it>=antlr3RewriteRule<rewriteElementType>StreamNewAEV(ADAPTOR, RECOGNIZER, (pANTLR3_UINT8)"token <it>", list_<it>); };
separator="\n"
>
<referencedRuleLabels
:{stream_<it>=antlr3RewriteRuleSubtreeStreamNewAEE(ADAPTOR, RECOGNIZER, (pANTLR3_UINT8)"token <it>", <it>.tree != NULL ? <it>.tree : NULL);};
separator="\n"
>
<referencedRuleListLabels
:{stream_<it>=antlr3RewriteRuleSubtreeStreamNewAEV(ADAPTOR, RECOGNIZER, (pANTLR3_UINT8)"token <it>", list_<it>);};
separator="\n"
>
>>
rewriteCodeLabelsFree() ::= <<
<referencedTokenLabels
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>); };
separator="\n"
>
<referencedTokenListLabels
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>);};
separator="\n"
>
<referencedRuleLabels
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>);};
separator="\n"
>
<referencedRuleListLabels
:{if (stream_<it> != NULL) stream_<it>->free(stream_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,
rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
{
if ( <referencedElementsDeep:{el | (stream_<el> != NULL && stream_<el>->hasNext(stream_<el>)) }; separator="|| "> )
{
<alt>
}
<referencedElementsDeep:{el | if ( stream_<el> != NULL) stream_<el>->reset(stream_<el>);<\n>}>
}<\n>
>>
rewriteClosureBlock(
alt,
rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
{
while ( <referencedElements:{el | (stream_<el> != NULL && stream_<el>->hasNext(stream_<el>)) }; separator="|| "> )
{
<alt>
}
<referencedElements:{el | if (stream_<el> != NULL) stream_<el>->reset(stream_<el>);<\n>}>
}<\n>
>>
RewriteEarlyExitException() ::=
<<
CONSTRUCTEX();
EXCEPTION->type = ANTLR3_REWRITE_EARLY_EXCEPTION;
EXCEPTION->name = (void *)ANTLR3_REWRITE_EARLY_EXCEPTION_NAME;
>>
rewritePositiveClosureBlock(
alt,
rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if (<referencedElements:{el | (stream_<el> == NULL || !stream_<el>->hasNext(stream_<el>)) }; separator="|| "> )
{
<RewriteEarlyExitException()>
}
else
{
while ( <referencedElements:{el | (stream_<el>->hasNext(stream_<el>)) }; separator="|| "> ) {
<alt>
}
<referencedElements:{el | stream_<el>->reset(stream_<el>);<\n>}>
}
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>)
{
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = NULL; /* \<-- rewriteEmptyAlt()) */"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->nilNode(ADAPTOR));
<root:rewriteElement()>
<children:rewriteElement()>
ADAPTOR->addChild(ADAPTOR, root_<enclosingTreeLevel>, root_<treeLevel>);
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, <createRewriteNodeFromElement(...)>);<\n>
<endif>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, stream_<label> == NULL ? NULL : stream_<label>->nextNode(stream_<label>));<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, stream_<label> == NULL ? NULL : stream_<label>->nextNode(stream_<label>));<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->becomeRootToken(ADAPTOR, stream_<label> == NULL ? NULL : stream_<label>->nextToken(stream_<label>), root_<treeLevel>));<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <createRewriteNodeFromElement(...)>, root_<treeLevel>));<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>));<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule,dup) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, stream_<rule> == NULL ? NULL : stream_<rule>->nextTree(stream_<rule>));<\n>
>>
rewriteRuleRefRoot(rule,dup) ::= <<
root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, stream_<rule> == NULL ? NULL : stream_<rule>->nextNode(stream_<rule>), root_<treeLevel>));<\n>
>>
rewriteNodeAction(action) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = (<ASLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, <action>, root_<treeLevel>));<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, stream_<label> == NULL ? NULL : stream_<label>->nextTree(stream_<label>));<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, stream_<label> == NULL ? NULL : stream_<label>->nextTree(stream_<label>));<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->becomeRoot(ADAPTOR, stream_<label> == NULL ? NULL : stream_<label>->nextNode(stream_<label>), root_<treeLevel>));<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)(ADAPTOR->becomeRoot((<ASTLabelType>)(stream_<label> == NULL ? NULL : stream_<label>->nextNode(stream_<label>), root_<treeLevel>));<\n>
>>
rewriteWildcardLabelRef(label) ::= <<
ADAPTOR->addChild(ADAPTOR, root_<treeLevel>, stream_<label> == NULL ? NULL : stream_<label>->nextTree(stream_<label>));<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
<hetero>New(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
<if(args)>
#if <length(args)> == 2
(<ASTLabelType>)ADAPTOR->createTypeTokenText(ADAPTOR, <tokenType>, TOKTEXT(<args; separator=", ">))
#else
(<ASTLabelType>)ADAPTOR->createTypeText(ADAPTOR, <tokenType>, (pANTLR3_UINT8)<args; separator=", ">)
#endif
<else>
(<ASTLabelType>)ADAPTOR->createTypeText(ADAPTOR, <tokenType>, (pANTLR3_UINT8)"<tokenType>")
<endif>
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
<hetero>New(stream_<token>->nextToken(stream_<token>)<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
#if <length(args)> == 2
ADAPTOR->createTypeTokenText(ADAPTOR, <token>->getType(<token>, TOKTEXT(<token>, <args; separator=", ">)) /* JIMI */
#else
ADAPTOR->createTypeToken(ADAPTOR, <token>->getType(<token>, <token>, <args; separator=", ">)
#endif
<else>
stream_<token> == NULL ? NULL : stream_<token>->nextNode(stream_<token>)
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/C/ASTDbg.stg 0000644 0001750 0001750 00000004160 11256465232 026423 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2009 Jim Idle, Temporal Wave LLC
http://www.temporal-wave.com
http://www.linkedin.com/in/jimidle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to AST stuff. Dynamic inheritance
* hierarchy is set up as ASTDbg : AST : Dbg : Java by code generator.
*/
group ASTDbg;
parserMembers() ::= <<
protected DebugTreeAdaptor adaptor =
new DebugTreeAdaptor(null,new CommonTreeAdaptor());
public void setTreeAdaptor(TreeAdaptor adaptor) {
this.adaptor = new DebugTreeAdaptor(dbg,adaptor);
}
public TreeAdaptor getTreeAdaptor() {
return adaptor;
}<\n>
>>
parserCtorBody() ::= <<
>>
createListenerAndHandshake() ::= <<
<super.createListenerAndHandshake()>
>>
ctorForPredefinedListener() ::= <<
>>
@rewriteElement.pregen() ::= ""
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/ 0000755 0001750 0001750 00000000000 11410174107 025706 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/CSharp3.stg 0000644 0001750 0001750 00000130470 11256465232 027707 0 ustar twerner twerner /*
* [The "BSD licence"]
* Copyright (c) 2005-2008 Terence Parr
* All rights reserved.
*
* Conversion to C#:
* Copyright (c) 2008-2009 Sam Harwell, Pixel Mine, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group CSharp3 implements ANTLRCore;
csharpVisibilityMap ::= [
"private":"private",
"protected":"protected",
"public":"public",
"fragment":"private",
default:"private"
]
/** The overall file structure of a recognizer; stores methods for rules
* and cyclic DFAs plus support code.
*/
outputFile( LEXER,PARSER,TREE_PARSER, actionScope, actions,
docComment, recognizer,
name, tokens, tokenNames, rules, cyclicDFAs,
bitsets, buildTemplate, buildAST, rewriteMode, profile,
backtracking, synpreds, memoize, numRules,
fileName, ANTLRVersion, generatedTimestamp, trace,
scopes, superClass, literals) ::=
<<
// $ANTLR <ANTLRVersion> <fileName> <generatedTimestamp>
// The variable 'variable' is assigned but its value is never used.
#pragma warning disable 219
// Unreachable code detected.
#pragma warning disable 162
<actions.(actionScope).header>
<@imports>
using System.Collections.Generic;
using Antlr.Runtime;
<if(TREE_PARSER)>
using Antlr.Runtime.Tree;
using RewriteRuleITokenStream = Antlr.Runtime.Tree.RewriteRuleTokenStream;
<endif>
using Stack = System.Collections.Generic.Stack\<object>;
using List = System.Collections.IList;
using ArrayList = System.Collections.Generic.List\<object>;
<if(backtracking)>
using Map = System.Collections.IDictionary;
using HashMap = System.Collections.Generic.Dictionary\<object, object>;
<endif>
<@end>
<if(actions.(actionScope).namespace)>
namespace <actions.(actionScope).namespace>
{
<endif>
<docComment>
<recognizer>
<if(actions.(actionScope).namespace)>
} // namespace <actions.(actionScope).namespace>
<endif>
>>
lexer(grammar, name, tokens, scopes, rules, numRules, labelType="CommonToken",
filterMode, superClass="Lexer") ::= <<
[System.CodeDom.Compiler.GeneratedCode("ANTLR", "<ANTLRVersion>")]
[System.CLSCompliant(false)]
public partial class <grammar.recognizerName> : <@superClassName><superClass><@end>
{
<tokens:{public const int <it.name>=<it.type>;}; separator="\n">
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<actions.lexer.members>
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
public <grammar.recognizerName>() {}<! needed by subclasses !>
public <grammar.recognizerName>( ICharStream input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}> )
: this( input, new RecognizerSharedState()<grammar.delegators:{g|, <g:delegateName()>}> )
{
}
public <grammar.recognizerName>( ICharStream input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}> )
: base( input, state )
{
<if(memoize)>
<if(grammar.grammarIsRoot)>
state.ruleMemo = new System.Collections.Generic.Dictionary\<int, int>[<numRules>+1];<\n><! index from 1..n !>
<endif>
<endif>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
public override string GrammarFileName { get { return "<fileName>"; } }
<if(filterMode)>
<filteringNextToken()>
<endif>
<rules; separator="\n\n">
<insertLexerSynpreds(synpreds)>
#region DFA
<cyclicDFAs:{dfa | DFA<dfa.decisionNumber> dfa<dfa.decisionNumber>;}; separator="\n">
protected override void InitDFAs()
{
base.InitDFAs();
<cyclicDFAs:{dfa | dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>( this<if(dfa.specialStateSTs)>, new SpecialStateTransitionHandler( specialStateTransition<dfa.decisionNumber> )<endif> );}; separator="\n">
}
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
#endregion
}
>>
/** A override of Lexer.nextToken() that backtracks over mTokens() looking
* for matches. No error can be generated upon error; just rewind, consume
* a token and then try again. backtracking needs to be set as well.
* Make rule memoization happen only at levels above 1 as we start mTokens
* at backtracking==1.
*/
filteringNextToken() ::= <<
public override IToken NextToken()
{
for ( ; ;)
{
if ( input.LA(1)==CharStreamConstants.EndOfFile )
{
return Tokens.EndOfFile;
}
state.token = null;
state.channel = TokenChannels.Default;
state.tokenStartCharIndex = input.Index;
state.tokenStartCharPositionInLine = input.CharPositionInLine;
state.tokenStartLine = input.Line;
state.text = null;
try
{
int m = input.Mark();
state.backtracking=1;<! means we won't throw slow exception !>
state.failed=false;
mTokens();
state.backtracking=0;
<! mTokens backtracks with synpred at backtracking==2
and we set the synpredgate to allow actions at level 1. !>
if ( state.failed )
{
input.Rewind(m);
input.Consume(); <! advance one char and try again !>
}
else
{
Emit();
return state.token;
}
}
catch ( RecognitionException re )
{
// shouldn't happen in backtracking mode, but...
ReportError(re);
Recover(re);
}
}
}
public override void Memoize( IIntStream input, int ruleIndex, int ruleStartIndex )
{
if ( state.backtracking > 1 )
base.Memoize( input, ruleIndex, ruleStartIndex );
}
public override bool AlreadyParsedRule(IIntStream input, int ruleIndex)
{
if ( state.backtracking > 1 )
return base.AlreadyParsedRule(input, ruleIndex);
return false;
}
>>
actionGate() ::= "state.backtracking == 0"
filteringActionGate() ::= "state.backtracking==1"
/** How to generate a parser */
genericParser(grammar, name, scopes, tokens, tokenNames, rules, numRules,
bitsets, inputStreamType, superClass,
ASTLabelType="object", labelType, members, rewriteElementType,
filterMode) ::= <<
[System.CodeDom.Compiler.GeneratedCode("ANTLR", "<ANTLRVersion>")]
[System.CLSCompliant(false)]
public partial class <grammar.recognizerName> : <@superClassName><superClass><@end>
{
<if(grammar.grammarIsRoot)>
internal static readonly string[] tokenNames = new string[] {
"\<invalid>", "\<EOR>", "\<DOWN>", "\<UP>", <tokenNames; separator=", ">
};<\n>
<endif>
<tokens:{public const int <it.name>=<it.type>;}; separator="\n">
// delegates
<grammar.delegates:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
// delegators
<grammar.delegators:
{g|public <g.recognizerName> <g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|public <g.recognizerName> gParent;}>
<scopes:{<if(it.isDynamicGlobalScope)><globalAttributeScope(scope=it)><endif>}>
<@members>
<! WARNING. bug in ST: this is cut-n-paste into Dbg.stg !>
public <grammar.recognizerName>( <inputStreamType> input<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}> )
: this( input, new RecognizerSharedState()<grammar.delegators:{g|, <g:delegateName()>}> )
{
}
public <grammar.recognizerName>( <inputStreamType> input, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}> )
: base( input, state )
{
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>(input, state<trunc(g.delegators):{p|, <p:delegateName()>}>, this);}; separator="\n">
<grammar.indirectDelegates:{g | <g:delegateName()> = <g.delegator:delegateName()>.<g:delegateName()>;}; separator="\n">
<last(grammar.delegators):{g|gParent = <g:delegateName()>;}>
}
<@end>
public override string[] TokenNames { get { return <grammar.composite.rootGrammar.recognizerName>.tokenNames; } }
public override string GrammarFileName { get { return "<fileName>"; } }
<members>
#region Rules
<rules; separator="\n\n">
#endregion Rules
<if(grammar.delegatedRules)>
<! generate rule/method definitions for imported rules so they
appear to be defined in this recognizer. !>
#region Delegated rules
<grammar.delegatedRules:{ruleDescriptor|
public <returnType()> <ruleDescriptor.name>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>) <!throws RecognitionException !>\{ <if(ruleDescriptor.hasReturnValue)>return <endif><ruleDescriptor.grammar:delegateName()>.<ruleDescriptor.name>(<ruleDescriptor.parameterScope.attributes:{a|<a.name>}; separator=", ">); \}}; separator="\n">
#endregion Delegated rules<\n>
<endif>
<insertSynpreds(synpreds)>
<if(cyclicDFAs)>
#region DFA
<cyclicDFAs:{dfa | DFA<dfa.decisionNumber> dfa<dfa.decisionNumber>;}; separator="\n">
protected override void InitDFAs()
{
base.InitDFAs();
<cyclicDFAs:{dfa | dfa<dfa.decisionNumber> = new DFA<dfa.decisionNumber>( this<if(dfa.specialStateSTs)>, new SpecialStateTransitionHandler( specialStateTransition<dfa.decisionNumber> )<endif> );}; separator="\n">
}
<cyclicDFAs:cyclicDFA()><! dump tables for all DFA !>
#endregion DFA<\n>
<endif>
<if(bitsets)>
#region Follow sets
private static class Follow
{
<bitsets:bitset(name={_<it.name>_in_<it.inName><it.tokenIndex>},
words64=it.bits)>
}
#endregion Follow sets<\n>
<endif>
}
>>
parserCtorBody() ::= <<
<if(memoize)>
<if(grammar.grammarIsRoot)>
this.state.ruleMemo = new System.Collections.Generic.Dictionary\<int, int>[<length(grammar.allImportedRules)>+1];<\n><! index from 1..n !>
<endif>
<endif>
<grammar.delegators:
{g|this.<g:delegateName()> = <g:delegateName()>;}; separator="\n">
>>
parser(grammar, name, scopes, tokens, tokenNames, rules, numRules, bitsets,
ASTLabelType="object", superClass="Parser", labelType="IToken",
members={<actions.parser.members>}) ::= <<
<genericParser(inputStreamType="ITokenStream", rewriteElementType="IToken", ...)>
>>
/** How to generate a tree parser; same as parser except the input
* stream is a different type.
*/
treeParser(grammar, name, scopes, tokens, tokenNames, globalAction, rules,
numRules, bitsets, labelType={<ASTLabelType>}, ASTLabelType="object",
superClass={<if(filterMode)><if(buildAST)>TreeRewriter<else>TreeFilter<endif><else>TreeParser<endif>},
members={<actions.treeparser.members>},
filterMode) ::= <<
<genericParser(inputStreamType="ITreeNodeStream", rewriteElementType="Node", ...)>
>>
/** A simpler version of a rule template that is specific to the imaginary
* rules created for syntactic predicates. As they never have return values
* nor parameters etc..., just give simplest possible method. Don't do
* any of the normal memoization stuff in here either; it's a waste.
* As predicates cannot be inlined into the invoking rule, they need to
* be in a rule by themselves.
*/
synpredRule(ruleName, ruleDescriptor, block, description, nakedBlock) ::=
<<
// $ANTLR start <ruleName>
public <!final !>void <ruleName>_fragment(<ruleDescriptor.parameterScope:parameterScope(scope=it)>)<! throws RecognitionException!>
{
<ruleLabelDefs()>
<if(trace)>
traceIn("<ruleName>_fragment", <ruleDescriptor.index>);
try
{
<block>
}
finally
{
traceOut("<ruleName>_fragment", <ruleDescriptor.index>);
}
<else>
<block>
<endif>
}
// $ANTLR end <ruleName>
>>
insertLexerSynpreds(synpreds) ::= <<
<insertSynpreds(synpreds)>
>>
insertSynpreds(synpreds) ::= <<
<if(synpreds)>
#region Synpreds
bool EvaluatePredicate( System.Action fragment )
{
state.backtracking++;
<@start()>
int start = input.Mark();
try
{
fragment();
}
catch ( RecognitionException re )
{
System.Console.Error.WriteLine("impossible: "+re);
}
bool success = !state.failed;
input.Rewind(start);
<@stop()>
state.backtracking--;
state.failed=false;
return success;
}
#endregion Synpreds<\n>
<endif>
>>
ruleMemoization(name) ::= <<
<if(memoize)>
if ( state.backtracking>0 && AlreadyParsedRule(input, <ruleDescriptor.index>) ) { return <ruleReturnValue()>; }
<endif>
>>
/** How to test for failure and return from rule */
checkRuleBacktrackFailure() ::= <<
<if(backtracking)>if (state.failed) return <ruleReturnValue()>;<endif>
>>
/** This rule has failed, exit indicating failure during backtrack */
ruleBacktrackFailure() ::= <<
<if(backtracking)>if (state.backtracking>0) {state.failed=true; return <ruleReturnValue()>;}<endif>
>>
/** How to generate code for a rule. This includes any return type
* data aggregates required for multiple return values.
*/
rule(ruleName,ruleDescriptor,block,emptyRule,description,exceptions,finally,memoize) ::= <<
<ruleAttributeScope(scope=ruleDescriptor.ruleScope)>
<returnScope(scope=ruleDescriptor.returnScope)>
// $ANTLR start "<ruleName>"
// <fileName>:<description>
<csharpVisibilityMap.(ruleDescriptor.modifier)> <returnType()> <ruleName>( <ruleDescriptor.parameterScope:parameterScope(scope=it)> )<! throws RecognitionException!>
{
<if(trace)>traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
<ruleLabelDefs()>
<ruleDescriptor.actions.init>
<@preamble()>
try
{
<ruleMemoization(name=ruleName)>
<block>
<ruleCleanUp()>
<(ruleDescriptor.actions.after):execAction()>
}
<if(exceptions)>
<exceptions:{e|<catch(decl=e.decl,action=e.action)><\n>}>
<else>
<if(!emptyRule)>
<if(actions.(actionScope).rulecatch)>
<actions.(actionScope).rulecatch>
<else>
catch ( RecognitionException re )
{
ReportError(re);
Recover(input,re);
<@setErrorReturnValue()>
}<\n>
<endif>
<endif>
<endif>
finally
{
<if(trace)>traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<memoize()>
<ruleScopeCleanUp()>
<finally>
}
<@postamble()>
return <ruleReturnValue()>;
}
// $ANTLR end "<ruleName>"
>>
catch(decl,action) ::= <<
catch ( <e.decl> )
{
<e.action>
}
>>
ruleDeclarations() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<returnType()> retval = new <returnType()>();
retval.start = input.LT(1);<\n>
<else>
<ruleDescriptor.returnScope.attributes:{ a |
<a.type> <a.name> = <if(a.initValue)><a.initValue><else><initValue(a.type)><endif>;
}>
<endif>
<if(memoize)>
int <ruleDescriptor.name>_StartIndex = input.Index;
<endif>
>>
ruleScopeSetUp() ::= <<
<ruleDescriptor.useScopes:{<it>_stack.Push(new <it>_scope());<it>_scopeInit(<it>_stack.Peek());}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_stack.Push(new <it.name>_scope());<it.name>_scopeInit(<it.name>_stack.Peek());}; separator="\n">
>>
ruleScopeCleanUp() ::= <<
<ruleDescriptor.useScopes:{<it>_scopeAfter(<it>_stack.Peek());<it>_stack.Pop();}; separator="\n">
<ruleDescriptor.ruleScope:{<it.name>_scopeAfter(<it.name>_stack.Peek());<it.name>_stack.Pop();}; separator="\n">
>>
ruleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,ruleDescriptor.tokenListLabels,ruleDescriptor.wildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{<labelType> <it.label.text>=null;}; separator="\n"
>
<[ruleDescriptor.tokenListLabels,ruleDescriptor.ruleListLabels,ruleDescriptor.wildcardTreeListLabels]
:{List list_<it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.ruleLabels:ruleLabelDef(label=it); separator="\n">
<ruleDescriptor.ruleListLabels:ruleLabelDef(label=it); separator="\n">
>>
lexerRuleLabelDefs() ::= <<
<[ruleDescriptor.tokenLabels,
ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleLabels]
:{<labelType> <it.label.text>=null;}; separator="\n"
>
<ruleDescriptor.charLabels:{int <it.label.text>;}; separator="\n">
<[ruleDescriptor.tokenListLabels,
ruleDescriptor.ruleListLabels]
:{List list_<it.label.text>=null;}; separator="\n"
>
>>
ruleReturnValue() ::= <<
<if(!ruleDescriptor.isSynPred)>
<if(ruleDescriptor.hasReturnValue)>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnName>
<else>
retval
<endif>
<endif>
<endif>
>>
ruleCleanUp() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<if(!TREE_PARSER)>
retval.stop = input.LT(-1);<\n>
<endif>
<endif>
>>
memoize() ::= <<
<if(memoize)>
<if(backtracking)>
if ( state.backtracking>0 ) { Memoize(input, <ruleDescriptor.index>, <ruleDescriptor.name>_StartIndex); }
<endif>
<endif>
>>
/** How to generate a rule in the lexer; naked blocks are used for
* fragment rules.
*/
lexerRule(ruleName,nakedBlock,ruleDescriptor,block,memoize) ::= <<
// $ANTLR start "<ruleName>"
<csharpVisibilityMap.(ruleDescriptor.modifier)> void m<ruleName>(<ruleDescriptor.parameterScope:parameterScope(scope=it)>)
{
<if(trace)>traceIn("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeSetUp()>
<ruleDeclarations()>
try
{
<if(nakedBlock)>
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block><\n>
<else>
int _type = <ruleName>;
int _channel = DefaultTokenChannel;
<ruleMemoization(name=ruleName)>
<lexerRuleLabelDefs()>
<ruleDescriptor.actions.init>
<block>
<ruleCleanUp()>
state.type = _type;
state.channel = _channel;
<(ruleDescriptor.actions.after):execAction()>
<endif>
}
finally
{
<if(trace)>traceOut("<ruleName>", <ruleDescriptor.index>);<endif>
<ruleScopeCleanUp()>
<memoize()>
}
}
// $ANTLR end "<ruleName>"
>>
/** How to generate code for the implicitly-defined lexer grammar rule
* that chooses between lexer rules.
*/
tokensRule(ruleName,nakedBlock,args,block,ruleDescriptor) ::= <<
public override void mTokens()<! throws RecognitionException!>
{
<block><\n>
}
>>
// S U B R U L E S
/** A (...) subrule with multiple alternatives */
block(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
<@prebranch()>
switch ( alt<decisionNumber> )
{
<alts:altSwitchCase()>
}
<@postbranch()>
>>
/** A rule block with multiple alternatives */
ruleBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int alt<decisionNumber>=<maxAlt>;
<decls>
<@predecision()>
<decision>
<@postdecision()>
switch ( alt<decisionNumber> )
{
<alts:altSwitchCase()>
}
>>
ruleBlockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A special case of a (...) subrule with a single alternative */
blockSingleAlt(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,description) ::= <<
// <fileName>:<description>
<decls>
<@prealt()>
<alts>
<@postalt()>
>>
/** A (..)+ block with 1 or more alternatives */
positiveClosureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
int cnt<decisionNumber>=0;
<decls>
<@preloop()>
for ( ; ; )
{
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch ( alt<decisionNumber> )
{
<alts:altSwitchCase()>
default:
if ( cnt<decisionNumber> >= 1 )
goto loop<decisionNumber>;
<ruleBacktrackFailure()>
EarlyExitException eee<decisionNumber> = new EarlyExitException( <decisionNumber>, input );
<@earlyExitException()>
throw eee<decisionNumber>;
}
cnt<decisionNumber>++;
}
loop<decisionNumber>:
;
<@postloop()>
>>
positiveClosureBlockSingleAlt ::= positiveClosureBlock
/** A (..)* block with 1 or more alternatives */
closureBlock(alts,decls,decision,enclosingBlockLevel,blockLevel,decisionNumber,maxK,maxAlt,description) ::= <<
// <fileName>:<description>
<decls>
<@preloop()>
for ( ; ; )
{
int alt<decisionNumber>=<maxAlt>;
<@predecision()>
<decision>
<@postdecision()>
switch ( alt<decisionNumber> )
{
<alts:altSwitchCase()>
default:
goto loop<decisionNumber>;
}
}
loop<decisionNumber>:
;
<@postloop()>
>>
closureBlockSingleAlt ::= closureBlock
/** Optional blocks (x)? are translated to (x|) by before code generation
* so we can just use the normal block template
*/
optionalBlock ::= block
optionalBlockSingleAlt ::= block
/** A case in a switch that jumps to an alternative given the alternative
* number. A DFA predicts the alternative and then a simple switch
* does the jump to the code that actually matches that alternative.
*/
altSwitchCase() ::= <<
case <i>:
<@prealt()>
<it>
break;<\n>
>>
/** An alternative is just a list of elements; at outermost level */
alt(elements,altNum,description,autoAST,outerAlt,treeLevel,rew) ::= <<
// <fileName>:<description>
{
<@declarations()>
<elements:element()>
<rew>
<@cleanup()>
}
>>
/** What to emit when there is no rewrite. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= ""
// E L E M E N T S
/** Dump the elements one per line */
element() ::= <<
<@prematch()>
<it.el><\n>
>>
/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex,hetero) ::= <<
<if(label)><label>=(<labelType>)<endif>Match(input,<token>,Follow._<token>_in_<ruleName><elementIndex>); <checkRuleBacktrackFailure()>
>>
/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
listLabel(label,elem) ::= <<
if (list_<label>==null) list_<label>=new ArrayList();
list_<label>.Add(<elem>);<\n>
>>
/** match a character */
charRef(char,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
Match(<char>); <checkRuleBacktrackFailure()>
>>
/** match a character range */
charRangeRef(a,b,label) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
MatchRange(<a>,<b>); <checkRuleBacktrackFailure()>
>>
/** For now, sets are interval tests and must be tested inline */
matchSet(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= input.LA(1);<\n>
<else>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<endif>
if ( <s> )
{
input.Consume();
<postmatchCode>
<if(!LEXER)>
state.errorRecovery=false;
<endif>
<if(backtracking)>state.failed=false;<endif>
}
else
{
<ruleBacktrackFailure()>
MismatchedSetException mse = new MismatchedSetException(null,input);
<@mismatchedSetException()>
<if(LEXER)>
Recover(mse);
throw mse;
<else>
throw mse;
<! use following code to make it recover inline; remove throw mse;
recoverFromMismatchedSet(input,mse,Follow._set_in_<ruleName><elementIndex>);
!>
<endif>
}<\n>
>>
matchSetUnchecked(s,label,elementIndex,postmatchCode="") ::= <<
<if(label)>
<if(LEXER)>
<label>= input.LA(1);<\n>
<else>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<endif>
input.Consume();
<postmatchCode>
<if(!LEXER)>
state.errorRecovery=false;
<endif>
<if(backtracking)>state.failed=false;<endif>
>>
matchRuleBlockSet ::= matchSet
matchSetAndListLabel(s,label,elementIndex,postmatchCode) ::= <<
<matchSet(...)>
<listLabel(elem=label,...)>
>>
/** Match a string literal */
lexerStringRef(string,label) ::= <<
<if(label)>
int <label>Start = CharIndex;
Match(<string>); <checkRuleBacktrackFailure()>
<label> = new <labelType>(input, TokenTypes.Invalid, TokenChannels.Default, <label>Start, CharIndex-1);
<else>
Match(<string>); <checkRuleBacktrackFailure()><\n>
<endif>
>>
wildcard(label,elementIndex) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
MatchAny(input); <checkRuleBacktrackFailure()>
>>
wildcardAndListLabel(label,elementIndex) ::= <<
<wildcard(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard in lexer */
wildcardChar(label, elementIndex) ::= <<
<if(label)>
<label> = input.LA(1);<\n>
<endif>
MatchAny(); <checkRuleBacktrackFailure()>
>>
wildcardCharListLabel(label, elementIndex) ::= <<
<wildcardChar(...)>
<listLabel(elem=label,...)>
>>
/** Match a rule reference by invoking it possibly with arguments
* and a return value or values. The 'rule' argument was the
* target rule name, but now is type Rule, whose toString is
* same: the rule name. Now though you can access full rule
* descriptor stuff.
*/
ruleRef(rule,label,elementIndex,args,scope) ::= <<
PushFollow(Follow._<rule.name>_in_<ruleName><elementIndex>);
<if(label)><label>=<endif><if(scope)><scope:delegateName()>.<endif><rule.name>(<args; separator=", ">);<\n>
state._fsp--;
<checkRuleBacktrackFailure()>
>>
/** ids+=r */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label,...)>
>>
/** A lexer rule reference.
*
* The 'rule' argument was the target rule name, but now
* is type Rule, whose toString is same: the rule name.
* Now though you can access full rule descriptor stuff.
*/
lexerRuleRef(rule,label,args,elementIndex,scope) ::= <<
<if(label)>
int <label>Start<elementIndex> = CharIndex;
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<label> = new <labelType>(input, TokenTypes.Invalid, TokenChannels.Default, <label>Start<elementIndex>, CharIndex-1);
<else>
<if(scope)><scope:delegateName()>.<endif>m<rule.name>(<args; separator=", ">); <checkRuleBacktrackFailure()>
<endif>
>>
/** i+=INT in lexer */
lexerRuleRefAndListLabel(rule,label,args,elementIndex,scope) ::= <<
<lexerRuleRef(...)>
<listLabel(elem=label,...)>
>>
/** EOF in the lexer */
lexerMatchEOF(label,elementIndex) ::= <<
<if(label)>
int <label>Start<elementIndex> = CharIndex;
Match(EndOfFile); <checkRuleBacktrackFailure()>
<labelType> <label> = new <labelType>(input, EndOfFile, TokenChannels.Default, <label>Start<elementIndex>, CharIndex-1);
<else>
Match(EndOfFile); <checkRuleBacktrackFailure()>
<endif>
>>
/** match ^(root children) in tree parser */
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
<root:element()>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==TokenTypes.Down )
{
Match(input, TokenTypes.Down, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, TokenTypes.Up, null); <checkRuleBacktrackFailure()>
}
<else>
Match(input, TokenTypes.Down, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, TokenTypes.Up, null); <checkRuleBacktrackFailure()>
<endif>
>>
/** Every predicate is used as a validating predicate (even when it is
* also hoisted into a prediction expression).
*/
validateSemanticPredicate(pred,description) ::= <<
if ( !(<evalPredicate(...)>) )
{
<ruleBacktrackFailure()>
throw new FailedPredicateException(input, "<ruleName>", "<description>");
}
>>
// F i x e d D F A (if-then-else)
dfaState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
else
{
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
<else>
<ruleBacktrackFailure()>
NoViableAltException nvae = new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae;<\n>
<endif>
}
>>
/** Same as a normal DFA state except that we don't examine lookahead
* for the bypass alternative. It delays error detection but this
* is faster, smaller, and more what people expect. For (X)? people
* expect "if ( LA(1)==X ) match(X);" and that's it.
*/
dfaOptionalBlockState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse ">
>>
/** A DFA state that is actually the loopback decision of a closure
* loop. If end-of-token (EOT) predicts any of the targets then it
* should act like a default clause (i.e., no error can be generated).
* This is used only in the lexer so that for ('a')* on the end of a rule
* anything other than 'a' predicts exiting.
*/
dfaLoopbackState(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(<k>);<\n>
<edges; separator="\nelse "><\n>
<if(eotPredictsAlt)>
<if(!edges)>
alt<decisionNumber>=<eotPredictsAlt>; <! if no edges, don't gen ELSE !>
<else>
else
{
alt<decisionNumber>=<eotPredictsAlt>;
}<\n>
<endif>
<endif>
>>
/** An accept state indicates a unique alternative has been predicted */
dfaAcceptState(alt) ::= "alt<decisionNumber>=<alt>;"
/** A simple edge with an expression. If the expression is satisfied,
* enter to the target state. To handle gated productions, we may
* have to evaluate some predicates for this edge.
*/
dfaEdge(labelExpr, targetState, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>)
{
<targetState>
}
>>
// F i x e d D F A (switch case)
/** A DFA state where a SWITCH may be generated. The code generator
* decides if this is possible: CodeGenerator.canGenerateSwitch().
*/
dfaStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n">
default:
<if(eotPredictsAlt)>
alt<decisionNumber>=<eotPredictsAlt>;
break;<\n>
<else>
{
<ruleBacktrackFailure()>
NoViableAltException nvae = new NoViableAltException("<description>", <decisionNumber>, <stateNumber>, input);<\n>
<@noViableAltException()>
throw nvae;
}<\n>
<endif>
}<\n>
>>
dfaOptionalBlockStateSwitch(k,edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n">
}<\n>
>>
dfaLoopbackStateSwitch(k, edges,eotPredictsAlt,description,stateNumber,semPredState) ::= <<
switch ( input.LA(<k>) )
{
<edges; separator="\n"><\n>
<if(eotPredictsAlt)>
default:
alt<decisionNumber>=<eotPredictsAlt>;
break;<\n>
<endif>
}<\n>
>>
dfaEdgeSwitch(labels, targetState) ::= <<
<labels:{case <it>:}; separator="\n">
{
<targetState>
}
break;
>>
// C y c l i c D F A
/** The code to initiate execution of a cyclic DFA; this is used
* in the rule to predict an alt just like the fixed DFA case.
* The <name> attribute is inherited via the parser, lexer, ...
*/
dfaDecision(decisionNumber,description) ::= <<
alt<decisionNumber> = dfa<decisionNumber>.Predict(input);
>>
/* Dump DFA tables as run-length-encoded Strings of octal values.
* Can't use hex as compiler translates them before compilation.
* These strings are split into multiple, concatenated strings.
* Java puts them back together at compile time thankfully.
* Java cannot handle large static arrays, so we're stuck with this
* encode/decode approach. See analysis and runtime DFA for
* the encoding methods.
*/
cyclicDFA(dfa) ::= <<
class DFA<dfa.decisionNumber> : DFA
{
const string DFA<dfa.decisionNumber>_eotS =
"<dfa.javaCompressedEOT; wrap="\"+\n\t\t\"">";
const string DFA<dfa.decisionNumber>_eofS =
"<dfa.javaCompressedEOF; wrap="\"+\n\t\t\"">";
const string DFA<dfa.decisionNumber>_minS =
"<dfa.javaCompressedMin; wrap="\"+\n\t\t\"">";
const string DFA<dfa.decisionNumber>_maxS =
"<dfa.javaCompressedMax; wrap="\"+\n\t\t\"">";
const string DFA<dfa.decisionNumber>_acceptS =
"<dfa.javaCompressedAccept; wrap="\"+\n\t\t\"">";
const string DFA<dfa.decisionNumber>_specialS =
"<dfa.javaCompressedSpecial; wrap="\"+\n\t\t\"">}>";
static readonly string[] DFA<dfa.decisionNumber>_transitionS =
{
<dfa.javaCompressedTransition:{s|"<s; wrap="\"+\n\"">"}; separator=",\n">
};
static readonly short[] DFA<dfa.decisionNumber>_eot = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_eotS);
static readonly short[] DFA<dfa.decisionNumber>_eof = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_eofS);
static readonly char[] DFA<dfa.decisionNumber>_min = DFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_minS);
static readonly char[] DFA<dfa.decisionNumber>_max = DFA.UnpackEncodedStringToUnsignedChars(DFA<dfa.decisionNumber>_maxS);
static readonly short[] DFA<dfa.decisionNumber>_accept = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_acceptS);
static readonly short[] DFA<dfa.decisionNumber>_special = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_specialS);
static readonly short[][] DFA<dfa.decisionNumber>_transition;
static DFA<dfa.decisionNumber>()
{
int numStates = DFA<dfa.decisionNumber>_transitionS.Length;
DFA<dfa.decisionNumber>_transition = new short[numStates][];
for ( int i=0; i \< numStates; i++ )
{
DFA<dfa.decisionNumber>_transition[i] = DFA.UnpackEncodedString(DFA<dfa.decisionNumber>_transitionS[i]);
}
}
public DFA<dfa.decisionNumber>( BaseRecognizer recognizer<if(dfa.specialStateSTs)>, SpecialStateTransitionHandler specialStateTransition<endif> )
<if(dfa.specialStateSTs)>
: base( specialStateTransition )
<endif>
{
this.recognizer = recognizer;
this.decisionNumber = <dfa.decisionNumber>;
this.eot = DFA<dfa.decisionNumber>_eot;
this.eof = DFA<dfa.decisionNumber>_eof;
this.min = DFA<dfa.decisionNumber>_min;
this.max = DFA<dfa.decisionNumber>_max;
this.accept = DFA<dfa.decisionNumber>_accept;
this.special = DFA<dfa.decisionNumber>_special;
this.transition = DFA<dfa.decisionNumber>_transition;
}
public override string GetDescription()
{
return "<dfa.description>";
}
<@errorMethod()>
}<\n>
<if(dfa.specialStateSTs)>
int specialStateTransition<dfa.decisionNumber>( DFA dfa, int s, IIntStream _input )<! throws NoViableAltException!>
{
<if(LEXER)>
IIntStream input = _input;
<endif>
<if(PARSER)>
ITokenStream input = (ITokenStream)_input;
<endif>
<if(TREE_PARSER)>
ITreeNodeStream input = (ITreeNodeStream)_input;
<endif>
int _s = s;
switch ( s )
{
<dfa.specialStateSTs:{state |
case <i0>:<! compressed special state numbers 0..n-1 !>
<state>}; separator="\n">
}
<if(backtracking)>
if (state.backtracking>0) {state.failed=true; return -1;}<\n>
<endif>
NoViableAltException nvae = new NoViableAltException(dfa.GetDescription(), <dfa.decisionNumber>, _s, input);
dfa.Error(nvae);
throw nvae;
}<\n>
<endif>
>>
/** A state in a cyclic DFA; it's a special state and part of a big switch on
* state.
*/
cyclicDFAState(decisionNumber,stateNumber,edges,needErrorClause,semPredState) ::= <<
int LA<decisionNumber>_<stateNumber> = input.LA(1);<\n>
<if(semPredState)><! get next lookahead symbol to test edges, then rewind !>
int index<decisionNumber>_<stateNumber> = input.Index;
input.Rewind();<\n>
<endif>
s = -1;
<edges; separator="\nelse ">
<if(semPredState)><! return input cursor to state before we rewound !>
input.Seek(index<decisionNumber>_<stateNumber>);<\n>
<endif>
if ( s>=0 ) return s;
break;
>>
/** Just like a fixed DFA edge, test the lookahead and indicate what
* state to jump to next if successful.
*/
cyclicDFAEdge(labelExpr, targetStateNumber, edgeNumber, predicates) ::= <<
if ( (<labelExpr>) <if(predicates)>&& (<predicates>)<endif>) {s = <targetStateNumber>;}<\n>
>>
/** An edge pointing at end-of-token; essentially matches any char;
* always jump to the target.
*/
eotDFAEdge(targetStateNumber,edgeNumber, predicates) ::= <<
s = <targetStateNumber>;<\n>
>>
// D F A E X P R E S S I O N S
andPredicates(left,right) ::= "(<left>&&<right>)"
orPredicates(operands) ::= "(<first(operands)><rest(operands):{o | ||<o>}>)"
notPredicate(pred) ::= "!(<evalPredicate(...)>)"
evalPredicate(pred,description) ::= "(<pred>)"
evalSynPredicate(pred,description) ::= "EvaluatePredicate(<pred>_fragment)"
lookaheadTest(atom,k,atomAsInt) ::= "LA<decisionNumber>_<stateNumber>==<atom>"
/** Sometimes a lookahead test cannot assume that LA(k) is in a temp variable
* somewhere. Must ask for the lookahead directly.
*/
isolatedLookaheadTest(atom,k,atomAsInt) ::= "input.LA(<k>)==<atom>"
lookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= <<
(LA<decisionNumber>_<stateNumber>\>=<lower> && LA<decisionNumber>_<stateNumber>\<=<upper>)
>>
isolatedLookaheadRangeTest(lower,upper,k,rangeNumber,lowerAsInt,upperAsInt) ::= "(input.LA(<k>)\>=<lower> && input.LA(<k>)\<=<upper>)"
setTest(ranges) ::= "<ranges; separator=\"||\">"
// A T T R I B U T E S
attributeScope(scope) ::= <<
<if(scope.attributes)>
protected class <scope.name>_scope
{
<scope.attributes:{public <it.decl>;}; separator="\n">
}
<if(scope.actions.scopeinit)>
protected void <scope.name>_scopeInit( <scope.name>_scope scope )
{
<scope.actions.scopeinit>
}<\n>
<else>
partial void <scope.name>_scopeInit( <scope.name>_scope scope );
<endif>
<if(scope.actions.scopeafter)>
protected void <scope.name>_scopeAfter( <scope.name>_scope scope )
{
<scope.actions.scopeafter>
}<\n>
<else>
partial void <scope.name>_scopeAfter( <scope.name>_scope scope );
<endif>
protected Stack\<<scope.name>_scope\> <scope.name>_stack = new Stack\<<scope.name>_scope\>();<\n>
<endif>
>>
globalAttributeScope(scope) ::= <<
<attributeScope(...)>
>>
ruleAttributeScope(scope) ::= <<
<attributeScope(...)>
>>
returnStructName() ::= "<it.name>_return"
returnType() ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
<ruleDescriptor.grammar.recognizerName>.<ruleDescriptor:returnStructName()>
<else>
<if(ruleDescriptor.hasSingleReturnValue)>
<ruleDescriptor.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
/** Generate the C# type associated with a single or multiple return
* values.
*/
ruleLabelType(referencedRule) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
<referencedRule.grammar.recognizerName>.<referencedRule.name>_return
<else>
<if(referencedRule.hasSingleReturnValue)>
<referencedRule.singleValueReturnType>
<else>
void
<endif>
<endif>
>>
delegateName() ::= <<
<if(it.label)><it.label><else>g<it.name><endif>
>>
/** Using a type to init value map, try to init a type; if not in table
* must be an object, default value is "null".
*/
initValue(typeName) ::= <<
default(<typeName>)
>>
/** Define a rule label including default value */
ruleLabelDef(label) ::= <<
<ruleLabelType(referencedRule=label.referencedRule)> <label.label.text> = <initValue(typeName=ruleLabelType(referencedRule=label.referencedRule))>;
>>
/** Define a return struct for a rule if the code needs to access its
* start/stop tokens, tree stuff, attributes, ... Leave a hole for
* subgroups to stick in members.
*/
returnScope(scope) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
public <!static !>class <ruleDescriptor:returnStructName()> : <if(TREE_PARSER)>Tree<else>Parser<endif>RuleReturnScope
{
<scope.attributes:{public <it.decl>;}; separator="\n">
<@ruleReturnMembers()>
}
<endif>
>>
parameterScope(scope) ::= <<
<scope.attributes:{<it.decl>}; separator=", ">
>>
parameterAttributeRef(attr) ::= "<attr.name>"
parameterSetAttributeRef(attr,expr) ::= "<attr.name> =<expr>;"
scopeAttributeRef(scope,attr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack.elementAt(<scope>_stack.size()-<negIndex>-1)).<attr.name>
<else>
<if(index)>
((<scope>_scope)<scope>_stack.elementAt(<index>)).<attr.name>
<else>
((<scope>_scope)<scope>_stack.Peek()).<attr.name>
<endif>
<endif>
>>
scopeSetAttributeRef(scope,attr,expr,index,negIndex) ::= <<
<if(negIndex)>
((<scope>_scope)<scope>_stack.elementAt(<scope>_stack.size()-<negIndex>-1)).<attr.name> =<expr>;
<else>
<if(index)>
((<scope>_scope)<scope>_stack.elementAt(<index>)).<attr.name> =<expr>;
<else>
((<scope>_scope)<scope>_stack.Peek()).<attr.name> =<expr>;
<endif>
<endif>
>>
/** $x is either global scope or x is rule with dynamic scope; refers
* to stack itself not top of stack. This is useful for predicates
* like {$function.size()>0 && $function::name.equals("foo")}?
*/
isolatedDynamicScopeRef(scope) ::= "<scope>_stack"
/** reference an attribute of rule; might only have single return value */
ruleLabelRef(referencedRule,scope,attr) ::= <<
<if(referencedRule.hasMultipleReturnValues)>
(<scope>!=null?<scope>.<attr.name>:<initValue(attr.type)>)
<else>
<scope>
<endif>
>>
returnAttributeRef(ruleDescriptor,attr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name>
<else>
<attr.name>
<endif>
>>
returnSetAttributeRef(ruleDescriptor,attr,expr) ::= <<
<if(ruleDescriptor.hasMultipleReturnValues)>
retval.<attr.name> =<expr>;
<else>
<attr.name> =<expr>;
<endif>
>>
/** How to translate $tokenLabel */
tokenLabelRef(label) ::= "<label>"
/** ids+=ID {$ids} or e+=expr {$e} */
listLabelRef(label) ::= "list_<label>"
// not sure the next are the right approach
tokenLabelPropertyRef_text(scope,attr) ::= "(<scope>!=null?<scope>.Text:null)"
tokenLabelPropertyRef_type(scope,attr) ::= "(<scope>!=null?<scope>.Type:0)"
tokenLabelPropertyRef_line(scope,attr) ::= "(<scope>!=null?<scope>.Line:0)"
tokenLabelPropertyRef_pos(scope,attr) ::= "(<scope>!=null?<scope>.CharPositionInLine:0)"
tokenLabelPropertyRef_channel(scope,attr) ::= "(<scope>!=null?<scope>.Channel:0)"
tokenLabelPropertyRef_index(scope,attr) ::= "(<scope>!=null?<scope>.TokenIndex:0)"
tokenLabelPropertyRef_tree(scope,attr) ::= "<scope>_tree"
tokenLabelPropertyRef_int(scope,attr) ::= "(<scope>!=null?Integer.valueOf(<scope>.Text):0)"
ruleLabelPropertyRef_start(scope,attr) ::= "(<scope>!=null?((<labelType>)<scope>.Start):null)"
ruleLabelPropertyRef_stop(scope,attr) ::= "(<scope>!=null?((<labelType>)<scope>.Stop):null)"
ruleLabelPropertyRef_tree(scope,attr) ::= "(<scope>!=null?((<ASTLabelType>)<scope>.Tree):null)"
ruleLabelPropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
(<scope>!=null?(input.TokenStream.ToString(
input.TreeAdaptor.GetTokenStartIndex(<scope>.Start),
input.TreeAdaptor.GetTokenStopIndex(<scope>.Start))):null)
<else>
(<scope>!=null?input.ToString(<scope>.start,<scope>.stop):null)
<endif>
>>
ruleLabelPropertyRef_st(scope,attr) ::= "(<scope>!=null?<scope>.st:null)"
/** Isolated $RULE ref ok in lexer as it's a Token */
lexerRuleLabel(label) ::= "<label>"
lexerRuleLabelPropertyRef_type(scope,attr) ::=
"(<scope>!=null?<scope>.Type:0)"
lexerRuleLabelPropertyRef_line(scope,attr) ::=
"(<scope>!=null?<scope>.Line:0)"
lexerRuleLabelPropertyRef_pos(scope,attr) ::=
"(<scope>!=null?<scope>.CharPositionInLine:-1)"
lexerRuleLabelPropertyRef_channel(scope,attr) ::=
"(<scope>!=null?<scope>.Channel:0)"
lexerRuleLabelPropertyRef_index(scope,attr) ::=
"(<scope>!=null?<scope>.TokenIndex:0)"
lexerRuleLabelPropertyRef_text(scope,attr) ::=
"(<scope>!=null?<scope>.Text:null)"
lexerRuleLabelPropertyRef_int(scope,attr) ::=
"(<scope>!=null?Integer.valueOf(<scope>.Text):0)"
// Somebody may ref $template or $tree or $stop within a rule:
rulePropertyRef_start(scope,attr) ::= "((<labelType>)retval.Start)"
rulePropertyRef_stop(scope,attr) ::= "((<labelType>)retval.Stop)"
rulePropertyRef_tree(scope,attr) ::= "((<ASTLabelType>)retval.Tree)"
rulePropertyRef_text(scope,attr) ::= <<
<if(TREE_PARSER)>
input.TokenStream.ToString(
input.TreeAdaptor.GetTokenStartIndex(retval.Start),
input.TreeAdaptor.GetTokenStopIndex(retval.Start))
<else>
input.ToString(retval.Start,input.LT(-1))
<endif>
>>
rulePropertyRef_st(scope,attr) ::= "retval.st"
lexerRulePropertyRef_text(scope,attr) ::= "Text"
lexerRulePropertyRef_type(scope,attr) ::= "_type"
lexerRulePropertyRef_line(scope,attr) ::= "state.tokenStartLine"
lexerRulePropertyRef_pos(scope,attr) ::= "state.tokenStartCharPositionInLine"
lexerRulePropertyRef_index(scope,attr) ::= "-1" // undefined token index in lexer
lexerRulePropertyRef_channel(scope,attr) ::= "_channel"
lexerRulePropertyRef_start(scope,attr) ::= "state.tokenStartCharIndex"
lexerRulePropertyRef_stop(scope,attr) ::= "(CharIndex-1)"
lexerRulePropertyRef_int(scope,attr) ::= "int.Parse(<scope>.Text)"
// setting $st and $tree is allowed in local rule. everything else
// is flagged as error
ruleSetPropertyRef_tree(scope,attr,expr) ::= "retval.tree =<expr>;"
ruleSetPropertyRef_st(scope,attr,expr) ::= "retval.st =<expr>;"
/** How to execute an action (only when not backtracking) */
execAction(action) ::= <<
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<action>
}
<else>
<action>
<endif>
>>
/** How to always execute an action even when backtracking */
execForcedAction(action) ::= "<action>"
// M I S C (properties, etc...)
bitset(name, words64) ::= <<
public static readonly BitSet <name> = new BitSet(new ulong[]{<words64:{<it>UL};separator=",">});<\n>
>>
codeFileExtension() ::= ".cs"
true() ::= "true"
false() ::= "false"
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/ASTTreeParser.stg 0000644 0001750 0001750 00000022503 11256465232 031065 0 ustar twerner twerner /*
* [The "BSD licence"]
* Copyright (c) 2005-2008 Terence Parr
* All rights reserved.
*
* Conversion to C#:
* Copyright (c) 2008-2009 Sam Harwell, Pixel Mine, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during tree parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* Each combination has its own template except that label/no label
* is combined into tokenRef, ruleRef, ...
*/
group ASTTreeParser;
/** Add a variable to track last element matched */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> _first_0 = null;
<ASTLabelType> _last = null;<\n>
>>
/** What to emit when there is no rewrite rule. For auto build
* mode, does nothing.
*/
noRewrite(rewriteBlockLevel, treeLevel) ::= <<
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(rewriteMode)>
retval.tree = (<ASTLabelType>)_first_0;
if ( adaptor.GetParent(retval.tree)!=null && adaptor.IsNil( adaptor.getParent(retval.tree) ) )
retval.tree = (<ASTLabelType>)adaptor.GetParent(retval.tree);
<endif>
<if(backtracking)>}<endif>
>>
/** match ^(root children) in tree parser; override here to
* add tree construction actions.
*/
tree(root, actionsAfterRoot, children, nullableChildList,
enclosingTreeLevel, treeLevel) ::= <<
_last = (<ASTLabelType>)input.LT(1);
{
<ASTLabelType> _save_last_<treeLevel> = _last;
<ASTLabelType> _first_<treeLevel> = null;
<if(!rewriteMode)>
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.Nil();
<endif>
<root:element()>
<if(rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
<if(root.el.rule)>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>.tree;
<else>
if ( _first_<enclosingTreeLevel>==null ) _first_<enclosingTreeLevel> = <root.el.label>;
<endif>
<endif>
<actionsAfterRoot:element()>
<if(nullableChildList)>
if ( input.LA(1)==TokenTypes.Down ) {
Match(input, TokenTypes.Down, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, TokenTypes.Up, null); <checkRuleBacktrackFailure()>
}
<else>
Match(input, TokenTypes.Down, null); <checkRuleBacktrackFailure()>
<children:element()>
Match(input, TokenTypes.Up, null); <checkRuleBacktrackFailure()>
<endif>
<if(!rewriteMode)>
adaptor.AddChild(root_<enclosingTreeLevel>, root_<treeLevel>);
<endif>
_last = _save_last_<treeLevel>;
}<\n>
>>
// TOKEN AST STUFF
/** ID! and output=AST (same as plain tokenRef) 'cept add
* setting of _last
*/
tokenRefBang(token,label,elementIndex) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
>>
/** ID auto construct */
tokenRef(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
/** label+=TOKEN auto construct */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) auto construct */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.tokenRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
>>
/** Match ^(label+=TOKEN ...) auto construct */
tokenRefRuleRootAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
/** Match . wildcard and auto dup the node/subtree */
wildcard(token,label,elementIndex,hetero) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.wildcard(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.DupTree(<label>);
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<else> <! rewrite mode !>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> )<endif>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>;
<endif>
>>
// SET AST
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
adaptor.AddChild(root_<treeLevel>, <label>_tree);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
<noRewrite()> <! set return tree !>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.matchSet(...)>
>>
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<super.matchSet(..., postmatchCode={
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<if(hetero)>
<label>_tree = new <hetero>(<label>);
<else>
<label>_tree = (<ASTLabelType>)adaptor.DupNode(<label>);
<endif><\n>
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_<treeLevel>);
<if(backtracking)>}<endif>
<endif>
}
)>
>>
// RULE REF AST
/** rule auto construct */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>
<if(!rewriteMode)>
adaptor.AddChild(root_<treeLevel>, <label>.Tree);
<else> <! rewrite mode !>
if ( _first_<treeLevel>==null ) _first_<treeLevel> = <label>.tree;
<endif>
>>
/** x+=rule auto construct */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) auto construct */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRef(...)>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<label>.Tree, root_<treeLevel>);
<endif>
>>
/** ^(x+=rule ...) auto construct */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrack(...)>
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefTrackAndListLabel(...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRootTrack(...)>
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
_last = (<ASTLabelType>)input.LT(1);
<super.ruleRefRuleRootTrackAndListLabel(...)>
>>
/** Streams for token refs are tree nodes now; override to
* change NextToken to NextNode.
*/
createRewriteNodeFromElement(token,hetero,scope) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.NextNode())
<else>
stream_<token>.NextNode()
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(!rewriteMode)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
<if(backtracking)>}<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/ST.stg 0000644 0001750 0001750 00000011646 11256465232 026775 0 ustar twerner twerner /*
* [The "BSD licence"]
* Copyright (c) 2005-2008 Terence Parr
* All rights reserved.
*
* Conversion to C#:
* Copyright (c) 2008-2009 Sam Harwell, Pixel Mine, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template subgroup to add template rewrite output
* If debugging, then you'll also get STDbg.stg loaded.
*/
group ST;
@outputFile.imports() ::= <<
<@super.imports()>
using Antlr3.ST;
using Antlr3.ST.Language;
>>
/** Add this to each rule's return value struct */
@returnScope.ruleReturnMembers() ::= <<
public StringTemplate st;
public object getTemplate() { return st; }
public override string ToString() { return (st==null) ? null : st.ToString(); }
>>
@genericParser.members() ::= <<
<@super.members()>
protected StringTemplateGroup templateLib = new StringTemplateGroup("<name>Templates", typeof(AngleBracketTemplateLexer) );
public StringTemplateGroup TemplateLib
{
get { return templateLib; }
set { templateLib = value; }
}
///** allows convenient multi-value initialization:
// * "new STAttrMap().put(...).put(...)"
// */
//public static class STAttrMap extends HashMap {
// public STAttrMap put(String attrName, object value) {
// super.put(attrName, value);
// return this;
// }
// public STAttrMap put(String attrName, int value) {
// super.put(attrName, new Integer(value));
// return this;
// }
//}
>>
/** x+=rule when output=template */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".getTemplate()",...)>
>>
rewriteTemplate(alts) ::= <<
// TEMPLATE REWRITE
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> )
{
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
}
<else>
<alts:rewriteTemplateAlt(); separator="else ">
<if(rewriteMode)><replaceTextInLine()><endif>
<endif>
>>
replaceTextInLine() ::= <<
<if(TREE_PARSER)>
((TokenRewriteStream)input.TokenStream).replace(
input.TreeAdaptor.GetTokenStartIndex(retval.start),
input.TreeAdaptor.GetTokenStopIndex(retval.start),
retval.st );
<else>
((TokenRewriteStream)input).replace(
((IToken)retval.start).TokenIndex,
input.LT(-1).TokenIndex,
retval.st );
<endif>
>>
rewriteTemplateAlt() ::= <<
// <it.description>
<if(it.pred)>
if (<it.pred>)
{
retval.st = <it.alt>;
}<\n>
<else>
{
retval.st = <it.alt>;
}<\n>
<endif>
>>
rewriteEmptyTemplate(alts) ::= <<
null;
>>
/** Invoke a template with a set of attribute name/value pairs.
* Set the value of the rule's template *after* having set
* the attributes because the rule's template might be used as
* an attribute to build a bigger template; you get a self-embedded
* template.
*/
rewriteExternalTemplate(name,args) ::= <<
templateLib.getInstanceOf("<name>"<if(args)>,
new STAttrMap()<args:{a | .put("<a.name>", <a.value>)}>
<endif>)
>>
/** expr is a string expression that says what template to load */
rewriteIndirectTemplate(expr,args) ::= <<
templateLib.getInstanceOf(<expr><if(args)>,
new STAttrMap()<args:{a | .put("<a.name>", <a.value>)}>
<endif>)
>>
/** Invoke an inline template with a set of attribute name/value pairs */
rewriteInlineTemplate(args, template) ::= <<
new StringTemplate(templateLib, "<template>"<if(args)>,
new STAttrMap()<args:{a | .put("<a.name>", <a.value>)}>
<endif>)
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
<action>
>>
/** An action has %st.attrName=expr; or %{st}.attrName=expr; */
actionSetAttribute(st,attrName,expr) ::= <<
(<st>).setAttribute("<attrName>",<expr>);
>>
/** Translate %{stringExpr} */
actionStringConstructor(stringExpr) ::= <<
new StringTemplate(templateLib,<stringExpr>)
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/ASTParser.stg 0000644 0001750 0001750 00000015651 11256465232 030253 0 ustar twerner twerner /*
* [The "BSD licence"]
* Copyright (c) 2005-2008 Terence Parr
* All rights reserved.
*
* Conversion to C#:
* Copyright (c) 2008-2009 Sam Harwell, Pixel Mine, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Templates for building ASTs during normal parsing.
*
* Deal with many combinations. Dimensions are:
* Auto build or rewrite
* no label, label, list label (label/no-label handled together)
* child, root
* token, set, rule, wildcard
*
* The situation is not too bad as rewrite (->) usage makes ^ and !
* invalid. There is no huge explosion of combinations.
*/
group ASTParser;
@rule.setErrorReturnValue() ::= <<
retval.tree = (<ASTLabelType>)adaptor.ErrorNode(input, retval.start, input.LT(-1), re);
<! System.out.WriteLine("<ruleName> returns "+((CommonTree)retval.tree).toStringTree()); !>
>>
// TOKEN AST STUFF
/** ID and output=AST */
tokenRef(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( state.backtracking==0 ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
adaptor.AddChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
/** ID! and output=AST (same as plain tokenRef) */
tokenRefBang(token,label,elementIndex) ::= "<super.tokenRef(...)>"
/** ID^ and output=AST */
tokenRefRuleRoot(token,label,elementIndex,hetero) ::= <<
<super.tokenRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = <createNodeFromToken(...)>;
root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_0);
<if(backtracking)>}<endif>
>>
/** ids+=ID! and output=AST */
tokenRefBangAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<listLabel(elem=label,...)>
>>
/** label+=TOKEN when output=AST but not rewrite alt */
tokenRefAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRef(...)>
<listLabel(elem=label,...)>
>>
/** Match label+=TOKEN^ when output=AST but not rewrite alt */
tokenRefRuleRootAndListLabel(token,label,hetero,elementIndex) ::= <<
<tokenRefRuleRoot(...)>
<listLabel(elem=label,...)>
>>
// SET AST
// the match set stuff is interesting in that it uses an argument list
// to pass code to the default matchSet; another possible way to alter
// inherited code. I don't use the region stuff because I need to pass
// different chunks depending on the operator. I don't like making
// the template name have the operator as the number of templates gets
// large but this is the most flexible--this is as opposed to having
// the code generator call matchSet then add root code or ruleroot code
// plus list label plus ... The combinations might require complicated
// rather than just added on code. Investigate that refactoring when
// I have more time.
matchSet(s,label,hetero,elementIndex,postmatchCode) ::= <<
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>adaptor.AddChild(root_0, <createNodeFromToken(...)>);})>
>>
matchRuleBlockSet(s,label,hetero,elementIndex,postmatchCode,treeLevel="0") ::= <<
<matchSet(...)>
>>
matchSetBang(s,label,elementIndex,postmatchCode) ::= "<super.matchSet(...)>"
// note there is no matchSetTrack because -> rewrites force sets to be
// plain old blocks of alts: (A|B|...|C)
matchSetRuleRoot(s,label,hetero,elementIndex,debug) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
<super.matchSet(..., postmatchCode={<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<createNodeFromToken(...)>, root_0);})>
>>
// RULE REF AST
/** rule when output=AST */
ruleRef(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>adaptor.AddChild(root_0, <label>.Tree);
>>
/** rule! is same as normal rule ref */
ruleRefBang(rule,label,elementIndex,args,scope) ::= "<super.ruleRef(...)>"
/** rule^ */
ruleRefRuleRoot(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>.Tree, root_0);
>>
/** x+=rule when output=AST */
ruleRefAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRef(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule! when output=AST is a rule ref with list addition */
ruleRefBangAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefBang(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** x+=rule^ */
ruleRefRuleRootAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<listLabel(elem=label+".Tree",...)>
>>
// WILDCARD AST
wildcard(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.Create(<label>);
adaptor.AddChild(root_0, <label>_tree);
<if(backtracking)>}<endif>
>>
wildcardBang(label,elementIndex) ::= "<super.wildcard(...)>"
wildcardRuleRoot(label,elementIndex) ::= <<
<super.wildcard(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<endif>
<label>_tree = (<ASTLabelType>)adaptor.Create(<label>);
root_0 = (<ASTLabelType>)adaptor.BecomeRoot(<label>_tree, root_0);
<if(backtracking)>}<endif>
>>
createNodeFromToken(label,hetero) ::= <<
<if(hetero)>
new <hetero>(<label>) <! new MethodNode(IDLabel) !>
<else>
(<ASTLabelType>)adaptor.Create(<label>)
<endif>
>>
ruleCleanUp() ::= <<
<super.ruleCleanUp()>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) {<\n><endif>
retval.tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
adaptor.SetTokenBoundaries(retval.tree, retval.start, retval.stop);
<if(backtracking)>}<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/Dbg.stg 0000644 0001750 0001750 00000021453 11256465232 027140 0 ustar twerner twerner /*
* [The "BSD licence"]
* Copyright (c) 2005-2008 Terence Parr
* All rights reserved.
*
* Conversion to C#:
* Copyright (c) 2008-2009 Sam Harwell, Pixel Mine, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to normal Java output;
* If ASTs are built, then you'll also get ASTDbg.stg loaded.
*/
group Dbg;
@outputFile.imports() ::= <<
<@super.imports()>
using Antlr.Runtime.Debug;
using IOException = System.IO.IOException;
>>
@genericParser.members() ::= <<
<if(grammar.grammarIsRoot)>
public static readonly string[] ruleNames =
new string[]
{
"invalidRule", <grammar.allImportedRules:{rST | "<rST.name>"}; wrap="\n ", separator=", ">
};<\n>
<endif>
<if(grammar.grammarIsRoot)><! grammar imports other grammar(s) !>
int ruleLevel = 0;
public virtual int RuleLevel { get { return ruleLevel; } }
public virtual void IncRuleLevel() { ruleLevel++; }
public virtual void DecRuleLevel() { ruleLevel--; }
<if(profile)>
<ctorForProfilingRootGrammar()>
<else>
<ctorForRootGrammar()>
<endif>
<ctorForPredefinedListener()>
<else><! imported grammar !>
public int RuleLevel { get { return <grammar.delegators:{g| <g:delegateName()>}>.RuleLevel; } }
public void IncRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.IncRuleLevel(); }
public void DecRuleLevel() { <grammar.delegators:{g| <g:delegateName()>}>.DecRuleLevel(); }
<ctorForDelegateGrammar()>
<endif>
<if(profile)>
public virtual bool AlreadyParsedRule( IIntStream input, int ruleIndex )
{
((Profiler)dbg).ExamineRuleMemoization(input, ruleIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
return super.AlreadyParsedRule(input, ruleIndex);
}<\n>
public virtual void Memoize( IIntStream input, int ruleIndex, int ruleStartIndex )
{
((Profiler)dbg).Memoize(input, ruleIndex, ruleStartIndex, <grammar.composite.rootGrammar.recognizerName>.ruleNames[ruleIndex]);
super.Memoize(input, ruleIndex, ruleStartIndex);
}<\n>
<endif>
protected virtual bool EvalPredicate( bool result, string predicate )
{
dbg.SemanticPredicate( result, predicate );
return result;
}<\n>
>>
ctorForRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
<! Same except we add port number and profile stuff if root grammar !>
public <name>( <inputStreamType> input )
: this( input, DebugEventSocketProxy.DefaultDebuggerPort, new RecognizerSharedState() )
{
}
public <name>( <inputStreamType> input, int port, RecognizerSharedState state )
: base( input, state )
{
<parserCtorBody()>
<createListenerAndHandshake()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>( input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}> );}; separator="\n">
<@finally()>
}<\n>
>>
ctorForProfilingRootGrammar() ::= <<
<! bug: can't use <@super.members()> cut-n-paste instead !>
public <name>( <inputStreamType> input )
: this( input, new Profiler(null), new RecognizerSharedState() )
{
}
public <name>( <inputStreamType> input, IDebugEventListener dbg, RecognizerSharedState state )
: base( input, dbg, state )
{
Profiler p = (Profiler)dbg;
p.setParser(this);
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>( input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}> );}; separator="\n">
<@finally()>
}
<\n>
>>
/** Basically we don't want to set any dbg listeners are root will have it. */
ctorForDelegateGrammar() ::= <<
public <name>( <inputStreamType> input, IDebugEventListener dbg, RecognizerSharedState state<grammar.delegators:{g|, <g.recognizerName> <g:delegateName()>}> )
: base( input, dbg, state )
{
<parserCtorBody()>
<grammar.directDelegates:
{g|<g:delegateName()> = new <g.recognizerName>( input, this, this.state<grammar.delegators:{g|, <g:delegateName()>}> );}; separator="\n">
}<\n>
>>
ctorForPredefinedListener() ::= <<
public <name>( <inputStreamType> input, IDebugEventListener dbg )
<@superClassRef>: base( input, dbg, new RecognizerSharedState() )<@end>
{
<if(profile)>
Profiler p = (Profiler)dbg;
p.setParser(this);
<endif>
<parserCtorBody()>
<grammar.directDelegates:{g|<g:delegateName()> = new <g.recognizerName>(input, dbg, this.state, this<grammar.delegators:{g|, <g:delegateName()>}>);}; separator="\n">
<@finally()>
}<\n>
>>
createListenerAndHandshake() ::= <<
<if(TREE_PARSER)>
DebugEventSocketProxy proxy = new DebugEventSocketProxy( this, port, input.TreeAdaptor );<\n>
<else>
DebugEventSocketProxy proxy = new DebugEventSocketProxy( this, port, null );<\n>
<endif>
DebugListener = proxy;
try
{
proxy.Handshake();
}
catch ( IOException ioe )
{
ReportError( ioe );
}
>>
@genericParser.superClassName() ::= "Debug<@super.superClassName()>"
@rule.preamble() ::= <<
try
{
dbg.EnterRule( GrammarFileName, "<ruleName>" );
if ( RuleLevel == 0 )
{
dbg.Commence();
}
IncRuleLevel();
dbg.Location( <ruleDescriptor.tree.line>, <ruleDescriptor.tree.charPositionInLine> );<\n>
>>
@rule.postamble() ::= <<
dbg.Location(<ruleDescriptor.EORNode.line>, <ruleDescriptor.EORNode.charPositionInLine>);<\n>
}
finally
{
dbg.ExitRule( GrammarFileName, "<ruleName>" );
DecRuleLevel();
if ( RuleLevel == 0 )
{
dbg.Terminate();
}
}<\n>
>>
@insertSynpreds.start() ::= "dbg.BeginBacktrack( state.backtracking );"
@insertSynpreds.stop() ::= "dbg.EndBacktrack( state.backtracking, success );"
// Common debug event triggers used by region overrides below
enterSubRule() ::= <<
try
{
dbg.EnterSubRule( <decisionNumber> );<\n>
>>
exitSubRule() ::= <<
}
finally
{
dbg.ExitSubRule( <decisionNumber> );
}<\n>
>>
enterDecision() ::= <<
try
{
dbg.EnterDecision( <decisionNumber> );<\n>
>>
exitDecision() ::= <<
}
finally
{
dbg.ExitDecision( <decisionNumber> );
}<\n>
>>
enterAlt(n) ::= "dbg.EnterAlt( <n> );<\n>"
// Region overrides that tell various constructs to add debugging triggers
@block.predecision() ::= "<enterSubRule()><enterDecision()>"
@block.postdecision() ::= "<exitDecision()>"
@block.postbranch() ::= "<exitSubRule()>"
@ruleBlock.predecision() ::= "<enterDecision()>"
@ruleBlock.postdecision() ::= "<exitDecision()>"
@ruleBlockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@blockSingleAlt.prealt() ::= "<enterAlt(n=\"1\")>"
@positiveClosureBlock.preloop() ::= "<enterSubRule()>"
@positiveClosureBlock.postloop() ::= "<exitSubRule()>"
@positiveClosureBlock.predecision() ::= "<enterDecision()>"
@positiveClosureBlock.postdecision() ::= "<exitDecision()>"
@positiveClosureBlock.earlyExitException() ::=
"dbg.RecognitionException( eee<decisionNumber> );<\n>"
@closureBlock.preloop() ::= "<enterSubRule()>"
@closureBlock.postloop() ::= "<exitSubRule()>"
@closureBlock.predecision() ::= "<enterDecision()>"
@closureBlock.postdecision() ::= "<exitDecision()>"
@altSwitchCase.prealt() ::= "<enterAlt(n=i)>"
@element.prematch() ::=
"dbg.Location( <it.line>, <it.pos> );"
@matchSet.mismatchedSetException() ::=
"dbg.RecognitionException( mse );"
@dfaState.noViableAltException() ::= "dbg.RecognitionException( nvae );"
@dfaStateSwitch.noViableAltException() ::= "dbg.RecognitionException( nvae );"
dfaDecision(decisionNumber,description) ::= <<
try
{
isCyclicDecision = true;
<super.dfaDecision(...)>
}
catch ( NoViableAltException nvae )
{
dbg.RecognitionException( nvae );
throw nvae;
}
>>
@cyclicDFA.errorMethod() ::= <<
public override void Error( NoViableAltException nvae )
{
((DebugParser)recognizer).dbg.RecognitionException( nvae );
}
>>
/** Force predicate validation to trigger an event */
evalPredicate(pred,description) ::= <<
EvalPredicate( <pred>, "<description>" )
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/AST.stg 0000644 0001750 0001750 00000032562 11256465232 027076 0 ustar twerner twerner /*
* [The "BSD licence"]
* Copyright (c) 2005-2008 Terence Parr
* All rights reserved.
*
* Conversion to C#:
* Copyright (c) 2008-2009 Sam Harwell, Pixel Mine, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
group AST;
@outputFile.imports() ::= <<
<@super.imports()>
<if(!TREE_PARSER)><! tree parser would already have imported !>
using Antlr.Runtime.Tree;
using RewriteRuleITokenStream = Antlr.Runtime.Tree.RewriteRuleTokenStream;<\n>
<endif>
>>
@genericParser.members() ::= <<
<@super.members()>
<parserMembers()>
>>
parserCtorBody() ::= <<
<super.parserCtorBody()>
InitializeTreeAdaptor();
if ( TreeAdaptor == null )
TreeAdaptor = new CommonTreeAdaptor();
>>
/** Add an adaptor property that knows how to build trees */
parserMembers() ::= <<
// Implement this function in your helper file to use a custom tree adaptor
partial void InitializeTreeAdaptor();
ITreeAdaptor adaptor;
public ITreeAdaptor TreeAdaptor
{
get
{
return adaptor;
}
set
{
this.adaptor = value;
<grammar.directDelegates:{g|<g:delegateName()>.TreeAdaptor = this.adaptor;}>
}
}
>>
@returnScope.ruleReturnMembers() ::= <<
internal <ASTLabelType> tree;
public override object Tree { get { return tree; } }
>>
/** Add a variable to track rule's return AST */
ruleDeclarations() ::= <<
<super.ruleDeclarations()>
<ASTLabelType> root_0 = null;<\n>
>>
ruleLabelDefs() ::= <<
<super.ruleLabelDefs()>
<[ruleDescriptor.tokenLabels,ruleDescriptor.wildcardTreeLabels,ruleDescriptor.wildcardTreeListLabels]
:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.tokenListLabels:{<ASTLabelType> <it.label.text>_tree=null;}; separator="\n">
<ruleDescriptor.allTokenRefsInAltsWithRewrites
:{RewriteRule<rewriteElementType>Stream stream_<it>=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>");}; separator="\n">
<ruleDescriptor.allRuleRefsInAltsWithRewrites
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"rule <it>");}; separator="\n">
>>
/** When doing auto AST construction, we must define some variables;
* These should be turned off if doing rewrites. This must be a "mode"
* as a rule could have both rewrite and AST within the same alternative
* block.
*/
@alt.declarations() ::= <<
<if(autoAST)>
<if(outerAlt)>
<if(!rewriteMode)>
root_0 = (<ASTLabelType>)adaptor.Nil();<\n>
<endif>
<endif>
<endif>
>>
// T r a c k i n g R u l e E l e m e n t s
/** ID and track it for use in a rewrite rule */
tokenRefTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)> <! Track implies no auto AST construction!>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.Add(<label>);<\n>
>>
/** ids+=ID and track it for use in a rewrite rule; adds to ids *and*
* to the tracking list stream_ID for use in the rewrite.
*/
tokenRefTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefTrack(...)>
<listLabel(elem=label,...)>
>>
/** ^(ID ...) track for rewrite */
tokenRefRuleRootTrack(token,label,elementIndex,hetero) ::= <<
<tokenRefBang(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<token>.Add(<label>);<\n>
>>
/** Match ^(label+=TOKEN ...) track for rewrite */
tokenRefRuleRootTrackAndListLabel(token,label,elementIndex,hetero) ::= <<
<tokenRefRuleRootTrack(...)>
<listLabel(elem=label,...)>
>>
/** rule when output=AST and tracking for rewrite */
ruleRefTrack(rule,label,elementIndex,args,scope) ::= <<
<super.ruleRef(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule.name>.Add(<label>.Tree);
>>
/** x+=rule when output=AST and tracking for rewrite */
ruleRefTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
/** ^(rule ...) rewrite */
ruleRefRuleRootTrack(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRoot(...)>
<if(backtracking)>if ( <actions.(actionScope).synpredgate> ) <endif>stream_<rule>.Add(<label>.Tree);
>>
/** ^(x+=rule ...) rewrite */
ruleRefRuleRootTrackAndListLabel(rule,label,elementIndex,args,scope) ::= <<
<ruleRefRuleRootTrack(...)>
<listLabel(elem=label+".Tree",...)>
>>
// R e w r i t e
rewriteCode(
alts, description,
referencedElementsDeep, // ALL referenced elements to right of ->
referencedTokenLabels,
referencedTokenListLabels,
referencedRuleLabels,
referencedRuleListLabels,
referencedWildcardLabels,
referencedWildcardListLabels,
rewriteBlockLevel, enclosingTreeLevel, treeLevel) ::=
<<
{
// AST REWRITE
// elements: <referencedElementsDeep; separator=", ">
// token labels: <referencedTokenLabels; separator=", ">
// rule labels: <referencedRuleLabels; separator=", ">
// token list labels: <referencedTokenListLabels; separator=", ">
// rule list labels: <referencedRuleListLabels; separator=", ">
// wildcard labels: <[referencedWildcardLabels,referencedWildcardListLabels]; separator=", ">
<if(backtracking)>
if ( <actions.(actionScope).synpredgate> ) {<\n>
<endif>
<prevRuleRootRef()>.tree = root_0;
<rewriteCodeLabels()>
root_0 = (<ASTLabelType>)adaptor.Nil();
<alts:rewriteAlt(); separator="else ">
<! if tree parser and rewrite=true !>
<if(TREE_PARSER)>
<if(rewriteMode)>
<prevRuleRootRef()>.tree = (<ASTLabelType>)adaptor.RulePostProcessing(root_0);
input.ReplaceChildren(adaptor.GetParent(retval.start),
adaptor.GetChildIndex(retval.start),
adaptor.GetChildIndex(_last),
retval.tree);
<endif>
<endif>
<! if parser or tree-parser && rewrite!=true, we need to set result !>
<if(!TREE_PARSER)>
<prevRuleRootRef()>.tree = root_0;<\n>
<else>
<if(!rewriteMode)>
<prevRuleRootRef()>.tree = root_0;
<endif>
<endif>
<if(backtracking)>
}<\n>
<endif>
}
>>
rewriteCodeLabels() ::= <<
<referencedTokenLabels
:{RewriteRule<rewriteElementType>Stream stream_<it>=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>",<it>);};
separator="\n"
>
<referencedTokenListLabels
:{RewriteRule<rewriteElementType>Stream stream_<it>=new RewriteRule<rewriteElementType>Stream(adaptor,"token <it>", list_<it>);};
separator="\n"
>
<referencedWildcardLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",<it>);};
separator="\n"
>
<referencedWildcardListLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"wildcard <it>",list_<it>);};
separator="\n"
>
<referencedRuleLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"rule <it>",<it>!=null?<it>.tree:null);};
separator="\n"
>
<referencedRuleListLabels
:{RewriteRuleSubtreeStream stream_<it>=new RewriteRuleSubtreeStream(adaptor,"token <it>",list_<it>);};
separator="\n"
>
>>
/** Generate code for an optional rewrite block; note it uses the deep ref'd element
* list rather shallow like other blocks.
*/
rewriteOptionalBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
if ( <referencedElementsDeep:{el | stream_<el>.HasNext}; separator="||"> )
{
<alt>
}
<referencedElementsDeep:{el | stream_<el>.Reset();<\n>}>
>>
rewriteClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
// <fileName>:<description>
while ( <referencedElements:{el | stream_<el>.HasNext}; separator="||"> )
{
<alt>
}
<referencedElements:{el | stream_<el>.Reset();<\n>}>
>>
rewritePositiveClosureBlock(
alt,rewriteBlockLevel,
referencedElementsDeep, // all nested refs
referencedElements, // elements in immediately block; no nested blocks
description) ::=
<<
if ( !(<referencedElements:{el | stream_<el>.HasNext}; separator="||">) )
{
throw new RewriteEarlyExitException();
}
while ( <referencedElements:{el | stream_<el>.HasNext}; separator="||"> )
{
<alt>
}
<referencedElements:{el | stream_<el>.Reset();<\n>}>
>>
rewriteAlt(a) ::= <<
// <a.description>
<if(a.pred)>
if (<a.pred>)
{
<a.alt>
}<\n>
<else>
{
<a.alt>
}<\n>
<endif>
>>
/** For empty rewrites: "r : ... -> ;" */
rewriteEmptyAlt() ::= "root_0 = null;"
rewriteTree(root,children,description,enclosingTreeLevel,treeLevel) ::= <<
// <fileName>:<description>
{
<ASTLabelType> root_<treeLevel> = (<ASTLabelType>)adaptor.Nil();
<root:rewriteElement()>
<children:rewriteElement()>
adaptor.AddChild(root_<enclosingTreeLevel>, root_<treeLevel>);
}<\n>
>>
rewriteElementList(elements) ::= "<elements:rewriteElement()>"
rewriteElement(e) ::= <<
<@pregen()>
<e.el>
>>
/** Gen ID or ID[args] */
rewriteTokenRef(token,elementIndex,hetero,args) ::= <<
adaptor.AddChild(root_<treeLevel>, <createRewriteNodeFromElement(...)>);<\n>
>>
/** Gen $label ... where defined via label=ID */
rewriteTokenLabelRef(label,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextNode());<\n>
>>
/** Gen $label ... where defined via label+=ID */
rewriteTokenListLabelRef(label,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextNode());<\n>
>>
/** Gen ^($label ...) */
rewriteTokenLabelRefRoot(label,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
/** Gen ^(ID ...) or ^(ID[args] ...) */
rewriteTokenRefRoot(token,elementIndex,hetero,args) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<createRewriteNodeFromElement(...)>, root_<treeLevel>);<\n>
>>
rewriteImaginaryTokenRef(args,token,hetero,elementIndex) ::= <<
adaptor.AddChild(root_<treeLevel>, <createImaginaryNode(tokenType=token, ...)>);<\n>
>>
rewriteImaginaryTokenRefRoot(args,token,hetero,elementIndex) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<createImaginaryNode(tokenType=token, ...)>, root_<treeLevel>);<\n>
>>
/** plain -> {foo} action */
rewriteAction(action) ::= <<
root_0 = <action>;<\n>
>>
/** What is the name of the previous value of this rule's root tree? This
* let's us refer to $rule to mean previous value. I am reusing the
* variable 'tree' sitting in retval struct to hold the value of root_0 right
* before I set it during rewrites. The assign will be to retval.tree.
*/
prevRuleRootRef() ::= "retval"
rewriteRuleRef(rule) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<rule>.NextTree());<\n>
>>
rewriteRuleRefRoot(rule) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<rule>.NextNode(), root_<treeLevel>);<\n>
>>
rewriteNodeAction(action) ::= <<
adaptor.AddChild(root_<treeLevel>, <action>);<\n>
>>
rewriteNodeActionRoot(action) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(<action>, root_<treeLevel>);<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel=rule */
rewriteRuleLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
/** Gen $ruleLabel ... where defined via ruleLabel+=rule */
rewriteRuleListLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel=rule */
rewriteRuleLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
/** Gen ^($ruleLabel ...) where ruleLabel+=rule */
rewriteRuleListLabelRefRoot(label) ::= <<
root_<treeLevel> = (<ASTLabelType>)adaptor.BecomeRoot(stream_<label>.NextNode(), root_<treeLevel>);<\n>
>>
rewriteWildcardLabelRef(label) ::= <<
adaptor.AddChild(root_<treeLevel>, stream_<label>.NextTree());<\n>
>>
createImaginaryNode(tokenType,hetero,args) ::= <<
<if(hetero)>
<! new MethodNode(IDLabel, args) !>
new <hetero>(<tokenType><if(args)>, <args; separator=", "><endif>)
<else>
(<ASTLabelType>)adaptor.Create(<tokenType>, <args; separator=", "><if(!args)>"<tokenType>"<endif>)
<endif>
>>
createRewriteNodeFromElement(token,hetero,args) ::= <<
<if(hetero)>
new <hetero>(stream_<token>.NextToken()<if(args)>, <args; separator=", "><endif>)
<else>
<if(args)> <! must create new node from old !>
adaptor.Create(<token>, <args; separator=", ">)
<else>
stream_<token>.NextNode()
<endif>
<endif>
>>
antlr-3.2/tool/src/main/resources/org/antlr/codegen/templates/CSharp3/ASTDbg.stg 0000644 0001750 0001750 00000006306 11256465232 027510 0 ustar twerner twerner /*
* [The "BSD licence"]
* Copyright (c) 2005-2008 Terence Parr
* All rights reserved.
*
* Conversion to C#:
* Copyright (c) 2008-2009 Sam Harwell, Pixel Mine, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/** Template overrides to add debugging to AST stuff. Dynamic inheritance
* hierarchy is set up as ASTDbg : AST : Dbg : Java by code generator.
*/
group ASTDbg;
parserMembers() ::= <<
// Implement this function in your helper file to use a custom tree adaptor
partial void InitializeTreeAdaptor();
protected DebugTreeAdaptor adaptor;
public ITreeAdaptor TreeAdaptor
{
get
{
return adaptor;
}
set
{
<if(grammar.grammarIsRoot)>
this.adaptor = new DebugTreeAdaptor(dbg,adaptor);
<else>
this.adaptor = (DebugTreeAdaptor)adaptor; // delegator sends dbg adaptor
<endif><\n>
<grammar.directDelegates:{g|<g:delegateName()>.TreeAdaptor = this.adaptor;}>
}
}<\n>
>>
parserCtorBody() ::= <<
<super.parserCtorBody()>
>>
createListenerAndHandshake() ::= <<
DebugEventSocketProxy proxy = new DebugEventSocketProxy( this, port, <if(TREE_PARSER)>input.TreeAdaptor<else>adaptor<endif> );
DebugListener = proxy;
<inputStreamType> = new Debug<inputStreamType>( input, proxy );
try
{
proxy.Handshake();
}
catch ( IOException ioe )
{
ReportError( ioe );
}
>>
@ctorForRootGrammar.finally() ::= <<
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;
proxy.TreeAdaptor = adap;
>>
@ctorForProfilingRootGrammar.finally() ::=<<
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;
proxy.TreeAdaptor = adap;
>>
@ctorForPredefinedListener.superClassRef() ::= ": base( input, dbg )"
@ctorForPredefinedListener.finally() ::=<<
<if(grammar.grammarIsRoot)><! don't create new adaptor for delegates !>
ITreeAdaptor adap = new CommonTreeAdaptor();
TreeAdaptor = adap;<\n>
<endif>
>>
@rewriteElement.pregen() ::= "dbg.Location( <e.line>, <e.pos> );"
antlr-3.2/tool/src/main/resources/org/antlr/antlr.properties 0000644 0001750 0001750 00000000526 11256465230 024312 0 ustar twerner twerner # This properties file is used by ANTLR to pick up any resources
# and configuration options that may or may not be produced by
# the build process. For instance the Tool version string is
# filtered into this resource file so that we don't need to
# hard code this into Tool.java and so on.
#
antlr.version=${project.version} ${buildNumber}
antlr-3.2/tool/src/main/resources/org/antlr/tool/ 0000755 0001750 0001750 00000000000 11410174107 022016 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/ 0000755 0001750 0001750 00000000000 11410174107 024014 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/depend.stg 0000644 0001750 0001750 00000000625 11256465234 026011 0 ustar twerner twerner /** templates used to generate make-compatible dependencies */
group depend;
/** Generate "f : x, y, z" dependencies for input
* dependencies and generated files. in and out
* are File objects. For example, you can say
* <f.canonicalPath>
*/
dependencies(grammarFileName,in,out) ::= <<
<if(in)><grammarFileName>: <in; separator=", "><endif>
<out:{f | <f> : <grammarFileName>}; separator="\n">
>>
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/ 0000755 0001750 0001750 00000000000 11410174107 024602 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/epsilon-edge.st 0000644 0001750 0001750 00000000072 11256465234 027540 0 ustar twerner twerner <src> -> <target> [fontname="Times-Italic", label = "e"];
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/edge.st 0000644 0001750 0001750 00000000205 11256465234 026067 0 ustar twerner twerner <src> -> <target> [fontsize=11, fontname="Courier", arrowsize=.7, label = "<label>"<if(arrowhead)>, arrowhead = <arrowhead><endif>];
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/state.st 0000644 0001750 0001750 00000000141 11256465234 026302 0 ustar twerner twerner node [fontsize=11, shape = <if(useBox)>box<else>circle, fixedsize=true, width=.4<endif>]; <name>
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/stopstate.st 0000644 0001750 0001750 00000000201 11256465234 027205 0 ustar twerner twerner node [fontsize=11, shape = <if(useBox)>polygon,sides=4,peripheries=2<else>doublecircle, fixedsize=true, width=.6<endif>]; <name>
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/decision-rank.st 0000644 0001750 0001750 00000000062 11256465234 027712 0 ustar twerner twerner {rank=same; rankdir=TB; <states; separator="; ">}
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/dfa.st 0000644 0001750 0001750 00000000211 11256465234 025712 0 ustar twerner twerner digraph NFA {
<if(rankdir)>rankdir=<rankdir>;<endif>
<decisionRanks; separator="\n">
<states; separator="\n">
<edges; separator="\n">
}
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/nfa.st 0000644 0001750 0001750 00000000155 11256465234 025733 0 ustar twerner twerner digraph NFA {
rankdir=LR;
<decisionRanks; separator="\n">
<states; separator="\n">
<edges; separator="\n">
}
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/dot/action-edge.st 0000644 0001750 0001750 00000000205 11256465234 027342 0 ustar twerner twerner <src> -> <target> [fontsize=11, fontname="Courier", arrowsize=.7, label = "<label>"<if(arrowhead)>, arrowhead = <arrowhead><endif>];
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/messages/ 0000755 0001750 0001750 00000000000 11410174107 025623 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/messages/formats/ 0000755 0001750 0001750 00000000000 11410174107 027276 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/messages/formats/gnu.stg 0000644 0001750 0001750 00000003542 11256465234 030626 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
New style messages. This file contains the actual layout of the messages emitted by ANTLR.
The text itself is coming out of the languages/*stg files, according to the chosen locale.
This file contains the format that mimicks GCC output.
*/
group gnu;
location(file, line, column) ::= "<file>:<line>:"
message(id, text) ::= "<text> (<id>)"
report(location, message, type) ::= "<location> <type>: <message>"
wantsSingleLineMessage() ::= "true" antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/messages/formats/vs2005.stg 0000644 0001750 0001750 00000003604 11256465234 030773 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
New style messages. This file contains the actual layout of the messages emitted by ANTLR.
The text itself is coming out of the languages/*stg files, according to the chosen locale.
This file contains the default format ANTLR uses.
*/
group antlr;
location(file, line, column) ::= "<file>(<line>,<column>)"
message(id, text) ::= "error <id> : <text>"
report(location, message, type) ::= "<location> : <type> <message.id> : <message.text>"
wantsSingleLineMessage() ::= "true"
antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/messages/formats/antlr.stg 0000644 0001750 0001750 00000003574 11256465234 031162 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006 Kay Roepke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
New style messages. This file contains the actual layout of the messages emitted by ANTLR.
The text itself is coming out of the languages/*stg files, according to the chosen locale.
This file contains the default format ANTLR uses.
*/
group antlr;
location(file, line, column) ::= "<file>:<line>:<column>:"
message(id, text) ::= "(<id>) <text>"
report(location, message, type) ::= "<type>(<message.id>): <location> <message.text>"
wantsSingleLineMessage() ::= "false" antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/messages/languages/ 0000755 0001750 0001750 00000000000 11410174107 027571 5 ustar twerner twerner antlr-3.2/tool/src/main/resources/org/antlr/tool/templates/messages/languages/en.stg 0000644 0001750 0001750 00000033721 11256465234 030734 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
New style messages. This file only contains the messages in English, but no
information about which file, line, or column it occurred in.
The location and message ids are taken out of the formats directory.
Kay Roepke
*/
group en_US;
// TOOL ERRORS
// file errors
CANNOT_WRITE_FILE(arg,exception,stackTrace) ::= <<
cannot write file <arg>: <exception>
<stackTrace; separator="\n">
>>
CANNOT_CLOSE_FILE(arg,exception,stackTrace) ::= "cannot close file <arg>: <exception>"
CANNOT_FIND_TOKENS_FILE(arg) ::= "cannot find tokens file <arg>"
ERROR_READING_TOKENS_FILE(arg,exception,stackTrace) ::= <<
problem reading token vocabulary file <arg>: <exception>
<stackTrace; separator="\n">
>>
DIR_NOT_FOUND(arg,exception,stackTrace) ::= "directory not found: <arg>"
OUTPUT_DIR_IS_FILE(arg,exception,stackTrace) ::= "output directory is a file: <arg>"
CANNOT_OPEN_FILE(arg,exception,stackTrace) ::= "cannot find or open file: <arg><if(exception)>; reason: <exception><endif>"
CIRCULAR_DEPENDENCY() ::= "your grammars contain a circular dependency and cannot be sorted into a valid build order."
INTERNAL_ERROR(arg,arg2,exception,stackTrace) ::= <<
internal error: <arg> <arg2><if(exception)>: <exception><endif>
<stackTrace; separator="\n">
>>
INTERNAL_WARNING(arg) ::= "internal warning: <arg>"
ERROR_CREATING_ARTIFICIAL_RULE(arg,exception,stackTrace) ::= <<
problems creating lexer rule listing all tokens: <exception>
<stackTrace; separator="\n">
>>
TOKENS_FILE_SYNTAX_ERROR(arg,arg2) ::=
"problems parsing token vocabulary file <arg> on line <arg2>"
CANNOT_GEN_DOT_FILE(arg,exception,stackTrace) ::=
"cannot write DFA DOT file <arg>: <exception>"
BAD_ACTION_AST_STRUCTURE(exception,stackTrace) ::=
"bad internal tree structure for action '<arg>': <exception>"
BAD_AST_STRUCTURE(arg,exception,stackTrace) ::= <<
bad internal tree structure '<arg>': <exception>
<stackTrace; separator="\n">
>>
FILE_AND_GRAMMAR_NAME_DIFFER(arg,arg2) ::=
"file <arg2> contains grammar <arg>; names must be identical"
FILENAME_EXTENSION_ERROR(arg) ::=
"file <arg> must end in a file extension, normally .g"
// code gen errors
MISSING_CODE_GEN_TEMPLATES(arg) ::=
"cannot find code generation templates <arg>.stg"
MISSING_CYCLIC_DFA_CODE_GEN_TEMPLATES() ::=
"cannot find code generation cyclic DFA templates for language <arg>"
CODE_GEN_TEMPLATES_INCOMPLETE(arg) ::=
"at least one code generation template missing for language <arg>"
CANNOT_CREATE_TARGET_GENERATOR(arg,exception,stackTrace) ::=
"cannot create target <arg> code generator: <exception>"
CANNOT_COMPUTE_SAMPLE_INPUT_SEQ() ::=
"cannot generate a sample input sequence from lookahead DFA"
// grammar interpretation errors
/*
NO_VIABLE_DFA_ALT(arg,arg2) ::=
"no viable transition from state <arg> on <arg2> while interpreting DFA"
*/
// GRAMMAR ERRORS
SYNTAX_ERROR(arg) ::= "syntax error: <arg>"
RULE_REDEFINITION(arg) ::=
"rule <arg> redefinition"
LEXER_RULES_NOT_ALLOWED(arg) ::=
"lexer rule <arg> not allowed in parser"
PARSER_RULES_NOT_ALLOWED(arg) ::=
"parser rule <arg> not allowed in lexer"
CANNOT_FIND_ATTRIBUTE_NAME_IN_DECL(arg) ::=
"cannot find an attribute name in attribute declaration"
NO_TOKEN_DEFINITION(arg) ::=
"no lexer rule corresponding to token: <arg>"
UNDEFINED_RULE_REF(arg) ::=
"reference to undefined rule: <arg>"
LITERAL_NOT_ASSOCIATED_WITH_LEXER_RULE(arg) ::=
"literal has no associated lexer rule: <arg>"
CANNOT_ALIAS_TOKENS_IN_LEXER(arg) ::=
"literals are illegal in lexer tokens{} section: <arg>"
ATTRIBUTE_REF_NOT_IN_RULE(arg,arg2) ::=
"reference to attribute outside of a rule: <arg><if(arg2)>.<arg2><endif>"
UNKNOWN_ATTRIBUTE_IN_SCOPE(arg,arg2) ::=
"unknown attribute for <arg>: <arg2>"
UNKNOWN_RULE_ATTRIBUTE(arg,arg2) ::=
"unknown attribute for rule <arg>: <arg2>"
UNKNOWN_SIMPLE_ATTRIBUTE(arg,args2) ::=
"attribute is not a token, parameter, or return value: <arg>"
ISOLATED_RULE_SCOPE(arg) ::=
"missing attribute access on rule scope: <arg>"
INVALID_RULE_PARAMETER_REF(arg,arg2) ::=
"cannot access rule <arg>'s parameter: <arg2>"
INVALID_RULE_SCOPE_ATTRIBUTE_REF(arg,arg2) ::=
"cannot access rule <arg>'s dynamically-scoped attribute: <arg2>"
SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE(arg) ::=
"symbol <arg> conflicts with global dynamic scope with same name"
WRITE_TO_READONLY_ATTR(arg,arg2,arg3) ::=
"cannot write to read only attribute: $<arg><if(arg2)>.<arg2><endif>"
LABEL_CONFLICTS_WITH_RULE(arg) ::=
"label <arg> conflicts with rule with same name"
LABEL_CONFLICTS_WITH_TOKEN(arg) ::=
"label <arg> conflicts with token with same name"
LABEL_CONFLICTS_WITH_RULE_SCOPE_ATTRIBUTE(arg,arg2) ::=
"label <arg> conflicts with rule <arg2>'s dynamically-scoped attribute with same name"
LABEL_CONFLICTS_WITH_RULE_ARG_RETVAL(arg,arg2) ::=
"label <arg> conflicts with rule <arg2>'s return value or parameter with same name"
ATTRIBUTE_CONFLICTS_WITH_RULE(arg,arg2) ::=
"rule <arg2>'s dynamically-scoped attribute <arg> conflicts with the rule name"
ATTRIBUTE_CONFLICTS_WITH_RULE_ARG_RETVAL(arg,arg2) ::=
"rule <arg2>'s dynamically-scoped attribute <arg> conflicts with<arg2>'s return value or parameter with same name"
LABEL_TYPE_CONFLICT(arg,arg2) ::=
"label <arg> type mismatch with previous definition: <arg2>"
ARG_RETVAL_CONFLICT(arg,arg2) ::=
"rule <arg2>'s argument <arg> conflicts a return value with same name"
NONUNIQUE_REF(arg) ::=
"<arg> is a non-unique reference"
FORWARD_ELEMENT_REF(arg) ::=
"illegal forward reference: <arg>"
MISSING_RULE_ARGS(arg) ::=
"missing parameter(s) on rule reference: <arg>"
RULE_HAS_NO_ARGS(arg) ::=
"rule <arg> has no defined parameters"
ARGS_ON_TOKEN_REF(arg) ::=
"token reference <arg> may not have parameters"
/*
NONCHAR_RANGE() ::=
"range operator can only be used in the lexer"
*/
ILLEGAL_OPTION(arg) ::=
"illegal option <arg>"
LIST_LABEL_INVALID_UNLESS_RETVAL_STRUCT(arg) ::=
"rule '+=' list labels are not allowed w/o output option: <arg>"
UNDEFINED_TOKEN_REF_IN_REWRITE(arg) ::=
"reference to undefined token in rewrite rule: <arg>"
REWRITE_ELEMENT_NOT_PRESENT_ON_LHS(arg) ::=
"reference to rewrite element <arg> without reference on left of ->"
UNDEFINED_LABEL_REF_IN_REWRITE(arg) ::=
"reference to undefined label in rewrite rule: $<arg>"
NO_GRAMMAR_START_RULE (arg) ::=
"grammar <arg>: no start rule (no rule can obviously be followed by EOF)"
EMPTY_COMPLEMENT(arg) ::= <<
<if(arg)>
set complement ~<arg> is empty
<else>
set complement is empty
<endif>
>>
UNKNOWN_DYNAMIC_SCOPE(arg) ::=
"unknown dynamic scope: <arg>"
UNKNOWN_DYNAMIC_SCOPE_ATTRIBUTE(arg,arg2) ::=
"unknown dynamically-scoped attribute for scope <arg>: <arg2>"
RULE_REF_AMBIG_WITH_RULE_IN_ALT(arg) ::=
"reference $<arg> is ambiguous; rule <arg> is enclosing rule and referenced in the production (assuming enclosing rule)"
ISOLATED_RULE_ATTRIBUTE(arg) ::=
"reference to locally-defined rule scope attribute without rule name: <arg>"
INVALID_ACTION_SCOPE(arg,arg2) ::=
"unknown or invalid action scope for <arg2> grammar: <arg>"
ACTION_REDEFINITION(arg) ::=
"redefinition of <arg> action"
DOUBLE_QUOTES_ILLEGAL(arg) ::=
"string literals must use single quotes (such as \'begin\'): <arg>"
INVALID_TEMPLATE_ACTION(arg) ::=
"invalid StringTemplate % shorthand syntax: '<arg>'"
MISSING_ATTRIBUTE_NAME() ::=
"missing attribute name on $ reference"
ARG_INIT_VALUES_ILLEGAL(arg) ::=
"rule parameters may not have init values: <arg>"
REWRITE_OR_OP_WITH_NO_OUTPUT_OPTION(arg) ::=
"<if(arg)>rule <arg> uses <endif>rewrite syntax or operator with no output option; setting output=AST"
AST_OP_WITH_NON_AST_OUTPUT_OPTION(arg) ::=
"AST operator with non-AST output option: <arg>"
NO_RULES(arg) ::= "grammar file <arg> has no rules"
MISSING_AST_TYPE_IN_TREE_GRAMMAR(arg) ::=
"tree grammar <arg> has no ASTLabelType option"
REWRITE_FOR_MULTI_ELEMENT_ALT(arg) ::=
"with rewrite=true, alt <arg> not simple node or obvious tree element; text attribute for rule not guaranteed to be correct"
RULE_INVALID_SET(arg) ::=
"Cannot complement rule <arg>; not a simple set or element"
HETERO_ILLEGAL_IN_REWRITE_ALT(arg) ::=
"alts with rewrites can't use heterogeneous types left of ->"
NO_SUCH_GRAMMAR_SCOPE(arg,arg2) ::=
"reference to undefined grammar in rule reference: <arg>.<arg2>"
NO_SUCH_RULE_IN_SCOPE(arg,arg2) ::=
"rule <arg2> is not defined in grammar <arg>"
TOKEN_ALIAS_CONFLICT(arg,arg2) ::=
"cannot alias <arg>; string already assigned to <arg2>"
TOKEN_ALIAS_REASSIGNMENT(arg,arg2) ::=
"cannot alias <arg>; token name already assigned to <arg2>"
TOKEN_VOCAB_IN_DELEGATE(arg,arg2) ::=
"tokenVocab option ignored in imported grammar <arg>"
INVALID_IMPORT(arg,arg2) ::=
"<arg.grammarTypeString> grammar <arg.name> cannot import <arg2.grammarTypeString> grammar <arg2.name>"
IMPORTED_TOKENS_RULE_EMPTY(arg,arg2) ::=
"no lexer rules contributed to <arg> from imported grammar <arg2>"
IMPORT_NAME_CLASH(arg,arg2) ::=
"combined grammar <arg.name> and imported <arg2.grammarTypeString> grammar <arg2.name> both generate <arg2.recognizerName>; import ignored"
AST_OP_IN_ALT_WITH_REWRITE(arg,arg2) ::=
"rule <arg> alt <arg2> uses rewrite syntax and also an AST operator"
WILDCARD_AS_ROOT(arg) ::= "Wildcard invalid as root; wildcard can itself be a tree"
CONFLICTING_OPTION_IN_TREE_FILTER(arg,arg2) ::= "option <arg>=<arg2> conflicts with tree grammar filter mode"
// GRAMMAR WARNINGS
GRAMMAR_NONDETERMINISM(input,conflictingAlts,paths,disabled,hasPredicateBlockedByAction) ::=
<<
<if(paths)>
Decision can match input such as "<input>" using multiple alternatives:
<paths:{ alt <it.alt> via NFA path <it.states; separator=","><\n>}>
<else>
Decision can match input such as "<input>" using multiple alternatives: <conflictingAlts; separator=", ">
<endif>
<if(disabled)><\n>As a result, alternative(s) <disabled; separator=","> were disabled for that input<endif><if(hasPredicateBlockedByAction)><\n>Semantic predicates were present but were hidden by actions.<endif>
>>
DANGLING_STATE(danglingAlts,input) ::= <<
the decision cannot distinguish between alternative(s) <danglingAlts; separator=","> for input such as "<input>"
>>
UNREACHABLE_ALTS(alts) ::= <<
The following alternatives can never be matched: <alts; separator=","><\n>
>>
INSUFFICIENT_PREDICATES(upon,altToLocations,hasPredicateBlockedByAction) ::= <<
Input such as "<upon>" is insufficiently covered with predicates at locations: <altToLocations.keys:{alt|alt <alt>: <altToLocations.(alt):{loc| line <loc.line>:<loc.column> at <loc.text>}; separator=", ">}; separator=", "><if(hasPredicateBlockedByAction)><\n>Semantic predicates were present but were hidden by actions.<endif>
>>
DUPLICATE_SET_ENTRY(arg) ::=
"duplicate token type <arg> when collapsing subrule into set"
ANALYSIS_ABORTED(enclosingRule) ::= <<
ANTLR could not analyze this decision in rule <enclosingRule>; often this is because of recursive rule references visible from the left edge of alternatives. ANTLR will re-analyze the decision with a fixed lookahead of k=1. Consider using "options {k=1;}" for that decision and possibly adding a syntactic predicate.
>>
RECURSION_OVERLOW(alt,input,targetRules,callSiteStates) ::= <<
Alternative <alt>: after matching input such as <input> decision cannot predict what comes next due to recursion overflow <targetRules,callSiteStates:{t,c|to <t> from <c:{s|<s.enclosingRule.name>};separator=", ">}; separator=" and ">
>>
LEFT_RECURSION(targetRules,alt,callSiteStates) ::= <<
Alternative <alt> discovers infinite left-recursion <targetRules,callSiteStates:{t,c|to <t> from <c:{s|<s.enclosingRule>};separator=", ">}; separator=" and ">
>>
UNREACHABLE_TOKENS(tokens) ::= <<
The following token definitions can never be matched because prior tokens match the same input: <tokens; separator=",">
>>
TOKEN_NONDETERMINISM(input,conflictingTokens,paths,disabled,hasPredicateBlockedByAction) ::=
<<
<if(paths)>
Decision can match input such as "<input>" using multiple alternatives:
<paths:{ alt <it.alt> via NFA path <it.states; separator=","><\n>}>
<else>
Multiple token rules can match input such as "<input>": <conflictingTokens; separator=", "><\n>
<endif>
<if(disabled)><\n>As a result, token(s) <disabled; separator=","> were disabled for that input<endif><if(hasPredicateBlockedByAction)><\n>Semantic predicates were present but were hidden by actions.<endif>
>>
LEFT_RECURSION_CYCLES(listOfCycles) ::= <<
The following sets of rules are mutually left-recursive <listOfCycles:{c| [<c:{r|<r.name>}; separator=", ">]}; separator=" and ">
>>
NONREGULAR_DECISION(ruleName,alts) ::= <<
[fatal] rule <ruleName> has non-LL(*) decision due to recursive rule invocations reachable from alts <alts; separator=",">. Resolve by left-factoring or using syntactic predicates or using backtrack=true option.
>>
/* l10n for message levels */
warning() ::= "warning"
error() ::= "error"
antlr-3.2/tool/src/main/java/ 0000755 0001750 0001750 00000000000 11410174110 016033 5 ustar twerner twerner antlr-3.2/tool/src/main/java/org/ 0000755 0001750 0001750 00000000000 11410174107 016630 5 ustar twerner twerner antlr-3.2/tool/src/main/java/org/antlr/ 0000755 0001750 0001750 00000000000 11410174110 017742 5 ustar twerner twerner antlr-3.2/tool/src/main/java/org/antlr/misc/ 0000755 0001750 0001750 00000000000 11410174107 020703 5 ustar twerner twerner antlr-3.2/tool/src/main/java/org/antlr/misc/MutableInteger.java 0000644 0001750 0001750 00000003420 11256465230 024464 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
/** Java won't let you modify an Integer; not sure how that's more
* efficient, but...here's one that let's you modify it.
* Frightening I have to implement this myself. Blech.
*/
public class MutableInteger {
public int value;
public MutableInteger() {
this(0);
}
public MutableInteger(int value) {
this.value = value;
}
}
antlr-3.2/tool/src/main/java/org/antlr/misc/Graph.java 0000644 0001750 0001750 00000007604 11256465230 022626 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
import java.util.*;
/** A generic graph with edges; Each node as a single Object payload.
* This is only used to topologically sort a list of file dependencies
* at the moment.
*/
public class Graph {
public static class Node {
Object payload;
List<Node> edges; // points at which nodes?
public Node(Object payload) { this.payload = payload; }
public void addEdge(Node n) {
if ( edges==null ) edges = new ArrayList<Node>();
if ( !edges.contains(n) ) edges.add(n);
}
public String toString() { return payload.toString(); }
}
/** Map from node payload to node containing it */
protected Map<Object,Node> nodes = new HashMap<Object,Node>();
public void addEdge(Object a, Object b) {
//System.out.println("add edge "+a+" to "+b);
Node a_node = getNode(a);
Node b_node = getNode(b);
a_node.addEdge(b_node);
}
protected Node getNode(Object a) {
Node existing = nodes.get(a);
if ( existing!=null ) return existing;
Node n = new Node(a);
nodes.put(a, n);
return n;
}
/** DFS-based topological sort. A valid sort is the reverse of
* the post-order DFA traversal. Amazingly simple but true.
* For sorting, I'm not following convention here since ANTLR
* needs the opposite. Here's what I assume for sorting:
*
* If there exists an edge u -> v then u depends on v and v
* must happen before u.
*
* So if this gives nonreversed postorder traversal, I get the order
* I want.
*/
public List<Object> sort() {
Set<Node> visited = new HashSet<Node>();
ArrayList<Object> sorted = new ArrayList<Object>();
while ( visited.size() < nodes.size() ) {
// pick any unvisited node, n
Node n = null;
for (Iterator it = nodes.values().iterator(); it.hasNext();) {
n = (Node)it.next();
if ( !visited.contains(n) ) break;
}
DFS(n, visited, sorted);
}
return sorted;
}
public void DFS(Node n, Set<Node> visited, ArrayList<Object> sorted) {
if ( visited.contains(n) ) return;
visited.add(n);
if ( n.edges!=null ) {
for (Iterator it = n.edges.iterator(); it.hasNext();) {
Node target = (Node) it.next();
DFS(target, visited, sorted);
}
}
sorted.add(n.payload);
}
} antlr-3.2/tool/src/main/java/org/antlr/misc/Utils.java 0000644 0001750 0001750 00000005550 11256465230 022663 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
public class Utils {
public static final int INTEGER_POOL_MAX_VALUE = 1000;
static Integer[] ints = new Integer[INTEGER_POOL_MAX_VALUE+1];
/** Integer objects are immutable so share all Integers with the
* same value up to some max size. Use an array as a perfect hash.
* Return shared object for 0..INTEGER_POOL_MAX_VALUE or a new
* Integer object with x in it.
*/
public static Integer integer(int x) {
if ( x<0 || x>INTEGER_POOL_MAX_VALUE ) {
return new Integer(x);
}
if ( ints[x]==null ) {
ints[x] = new Integer(x);
}
return ints[x];
}
/** Given a source string, src,
a string to replace, replacee,
and a string to replace with, replacer,
return a new string w/ the replacing done.
You can use replacer==null to remove replacee from the string.
This should be faster than Java's String.replaceAll as that one
uses regex (I only want to play with strings anyway).
*/
public static String replace(String src, String replacee, String replacer) {
StringBuffer result = new StringBuffer(src.length() + 50);
int startIndex = 0;
int endIndex = src.indexOf(replacee);
while(endIndex != -1) {
result.append(src.substring(startIndex,endIndex));
if ( replacer!=null ) {
result.append(replacer);
}
startIndex = endIndex + replacee.length();
endIndex = src.indexOf(replacee,startIndex);
}
result.append(src.substring(startIndex,src.length()));
return result.toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/misc/IntervalSet.java 0000644 0001750 0001750 00000055207 11256465230 024027 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
import org.antlr.analysis.Label;
import org.antlr.tool.Grammar;
import java.util.*;
/** A set of integers that relies on ranges being common to do
* "run-length-encoded" like compression (if you view an IntSet like
* a BitSet with runs of 0s and 1s). Only ranges are recorded so that
* a few ints up near value 1000 don't cause massive bitsets, just two
* integer intervals.
*
* element values may be negative. Useful for sets of EPSILON and EOF.
*
* 0..9 char range is index pair ['\u0030','\u0039'].
* Multiple ranges are encoded with multiple index pairs. Isolated
* elements are encoded with an index pair where both intervals are the same.
*
* The ranges are ordered and disjoint so that 2..6 appears before 101..103.
*/
public class IntervalSet implements IntSet {
public static final IntervalSet COMPLETE_SET = IntervalSet.of(0,Label.MAX_CHAR_VALUE);
/** The list of sorted, disjoint intervals. */
protected List<Interval> intervals;
/** Create a set with no elements */
public IntervalSet() {
intervals = new ArrayList<Interval>(2); // most sets are 1 or 2 elements
}
public IntervalSet(List<Interval> intervals) {
this.intervals = intervals;
}
/** Create a set with a single element, el. */
public static IntervalSet of(int a) {
IntervalSet s = new IntervalSet();
s.add(a);
return s;
}
/** Create a set with all ints within range [a..b] (inclusive) */
public static IntervalSet of(int a, int b) {
IntervalSet s = new IntervalSet();
s.add(a,b);
return s;
}
/** Add a single element to the set. An isolated element is stored
* as a range el..el.
*/
public void add(int el) {
add(el,el);
}
/** Add interval; i.e., add all integers from a to b to set.
* If b<a, do nothing.
* Keep list in sorted order (by left range value).
* If overlap, combine ranges. For example,
* If this is {1..5, 10..20}, adding 6..7 yields
* {1..5, 6..7, 10..20}. Adding 4..8 yields {1..8, 10..20}.
*/
public void add(int a, int b) {
add(Interval.create(a,b));
}
// copy on write so we can cache a..a intervals and sets of that
protected void add(Interval addition) {
//System.out.println("add "+addition+" to "+intervals.toString());
if ( addition.b<addition.a ) {
return;
}
// find position in list
// Use iterators as we modify list in place
for (ListIterator iter = intervals.listIterator(); iter.hasNext();) {
Interval r = (Interval) iter.next();
if ( addition.equals(r) ) {
return;
}
if ( addition.adjacent(r) || !addition.disjoint(r) ) {
// next to each other, make a single larger interval
Interval bigger = addition.union(r);
iter.set(bigger);
// make sure we didn't just create an interval that
// should be merged with next interval in list
if ( iter.hasNext() ) {
Interval next = (Interval) iter.next();
if ( bigger.adjacent(next)||!bigger.disjoint(next) ) {
// if we bump up against or overlap next, merge
iter.remove(); // remove this one
iter.previous(); // move backwards to what we just set
iter.set(bigger.union(next)); // set to 3 merged ones
}
}
return;
}
if ( addition.startsBeforeDisjoint(r) ) {
// insert before r
iter.previous();
iter.add(addition);
return;
}
// if disjoint and after r, a future iteration will handle it
}
// ok, must be after last interval (and disjoint from last interval)
// just add it
intervals.add(addition);
}
/*
protected void add(Interval addition) {
//System.out.println("add "+addition+" to "+intervals.toString());
if ( addition.b<addition.a ) {
return;
}
// find position in list
//for (ListIterator iter = intervals.listIterator(); iter.hasNext();) {
int n = intervals.size();
for (int i=0; i<n; i++) {
Interval r = (Interval)intervals.get(i);
if ( addition.equals(r) ) {
return;
}
if ( addition.adjacent(r) || !addition.disjoint(r) ) {
// next to each other, make a single larger interval
Interval bigger = addition.union(r);
intervals.set(i, bigger);
// make sure we didn't just create an interval that
// should be merged with next interval in list
if ( (i+1)<n ) {
i++;
Interval next = (Interval)intervals.get(i);
if ( bigger.adjacent(next)||!bigger.disjoint(next) ) {
// if we bump up against or overlap next, merge
intervals.remove(i); // remove next one
i--;
intervals.set(i, bigger.union(next)); // set to 3 merged ones
}
}
return;
}
if ( addition.startsBeforeDisjoint(r) ) {
// insert before r
intervals.add(i, addition);
return;
}
// if disjoint and after r, a future iteration will handle it
}
// ok, must be after last interval (and disjoint from last interval)
// just add it
intervals.add(addition);
}
*/
public void addAll(IntSet set) {
if ( set==null ) {
return;
}
if ( !(set instanceof IntervalSet) ) {
throw new IllegalArgumentException("can't add non IntSet ("+
set.getClass().getName()+
") to IntervalSet");
}
IntervalSet other = (IntervalSet)set;
// walk set and add each interval
int n = other.intervals.size();
for (int i = 0; i < n; i++) {
Interval I = (Interval) other.intervals.get(i);
this.add(I.a,I.b);
}
}
public IntSet complement(int minElement, int maxElement) {
return this.complement(IntervalSet.of(minElement,maxElement));
}
/** Given the set of possible values (rather than, say UNICODE or MAXINT),
* return a new set containing all elements in vocabulary, but not in
* this. The computation is (vocabulary - this).
*
* 'this' is assumed to be either a subset or equal to vocabulary.
*/
public IntSet complement(IntSet vocabulary) {
if ( vocabulary==null ) {
return null; // nothing in common with null set
}
if ( !(vocabulary instanceof IntervalSet ) ) {
throw new IllegalArgumentException("can't complement with non IntervalSet ("+
vocabulary.getClass().getName()+")");
}
IntervalSet vocabularyIS = ((IntervalSet)vocabulary);
int maxElement = vocabularyIS.getMaxElement();
IntervalSet compl = new IntervalSet();
int n = intervals.size();
if ( n ==0 ) {
return compl;
}
Interval first = (Interval)intervals.get(0);
// add a range from 0 to first.a constrained to vocab
if ( first.a > 0 ) {
IntervalSet s = IntervalSet.of(0, first.a-1);
IntervalSet a = (IntervalSet)s.and(vocabularyIS);
compl.addAll(a);
}
for (int i=1; i<n; i++) { // from 2nd interval .. nth
Interval previous = (Interval)intervals.get(i-1);
Interval current = (Interval)intervals.get(i);
IntervalSet s = IntervalSet.of(previous.b+1, current.a-1);
IntervalSet a = (IntervalSet)s.and(vocabularyIS);
compl.addAll(a);
}
Interval last = (Interval)intervals.get(n -1);
// add a range from last.b to maxElement constrained to vocab
if ( last.b < maxElement ) {
IntervalSet s = IntervalSet.of(last.b+1, maxElement);
IntervalSet a = (IntervalSet)s.and(vocabularyIS);
compl.addAll(a);
}
return compl;
}
/** Compute this-other via this&~other.
* Return a new set containing all elements in this but not in other.
* other is assumed to be a subset of this;
* anything that is in other but not in this will be ignored.
*/
public IntSet subtract(IntSet other) {
// assume the whole unicode range here for the complement
// because it doesn't matter. Anything beyond the max of this' set
// will be ignored since we are doing this & ~other. The intersection
// will be empty. The only problem would be when this' set max value
// goes beyond MAX_CHAR_VALUE, but hopefully the constant MAX_CHAR_VALUE
// will prevent this.
return this.and(((IntervalSet)other).complement(COMPLETE_SET));
}
/** return a new set containing all elements in this but not in other.
* Intervals may have to be broken up when ranges in this overlap
* with ranges in other. other is assumed to be a subset of this;
* anything that is in other but not in this will be ignored.
*
* Keep around, but 10-20-2005, I decided to make complement work w/o
* subtract and so then subtract can simply be a&~b
*
public IntSet subtract(IntSet other) {
if ( other==null || !(other instanceof IntervalSet) ) {
return null; // nothing in common with null set
}
IntervalSet diff = new IntervalSet();
// iterate down both interval lists
ListIterator thisIter = this.intervals.listIterator();
ListIterator otherIter = ((IntervalSet)other).intervals.listIterator();
Interval mine=null;
Interval theirs=null;
if ( thisIter.hasNext() ) {
mine = (Interval)thisIter.next();
}
if ( otherIter.hasNext() ) {
theirs = (Interval)otherIter.next();
}
while ( mine!=null ) {
//System.out.println("mine="+mine+", theirs="+theirs);
// CASE 1: nothing in theirs removes a chunk from mine
if ( theirs==null || mine.disjoint(theirs) ) {
// SUBCASE 1a: finished traversing theirs; keep adding mine now
if ( theirs==null ) {
// add everything in mine to difference since theirs done
diff.add(mine);
mine = null;
if ( thisIter.hasNext() ) {
mine = (Interval)thisIter.next();
}
}
else {
// SUBCASE 1b: mine is completely to the left of theirs
// so we can add to difference; move mine, but not theirs
if ( mine.startsBeforeDisjoint(theirs) ) {
diff.add(mine);
mine = null;
if ( thisIter.hasNext() ) {
mine = (Interval)thisIter.next();
}
}
// SUBCASE 1c: theirs is completely to the left of mine
else {
// keep looking in theirs
theirs = null;
if ( otherIter.hasNext() ) {
theirs = (Interval)otherIter.next();
}
}
}
}
else {
// CASE 2: theirs breaks mine into two chunks
if ( mine.properlyContains(theirs) ) {
// must add two intervals: stuff to left and stuff to right
diff.add(mine.a, theirs.a-1);
// don't actually add stuff to right yet as next 'theirs'
// might overlap with it
// The stuff to the right might overlap with next "theirs".
// so it is considered next
Interval right = new Interval(theirs.b+1, mine.b);
mine = right;
// move theirs forward
theirs = null;
if ( otherIter.hasNext() ) {
theirs = (Interval)otherIter.next();
}
}
// CASE 3: theirs covers mine; nothing to add to diff
else if ( theirs.properlyContains(mine) ) {
// nothing to add, theirs forces removal totally of mine
// just move mine looking for an overlapping interval
mine = null;
if ( thisIter.hasNext() ) {
mine = (Interval)thisIter.next();
}
}
// CASE 4: non proper overlap
else {
// overlap, but not properly contained
diff.add(mine.differenceNotProperlyContained(theirs));
// update iterators
boolean moveTheirs = true;
if ( mine.startsBeforeNonDisjoint(theirs) ||
theirs.b > mine.b )
{
// uh oh, right of theirs extends past right of mine
// therefore could overlap with next of mine so don't
// move theirs iterator yet
moveTheirs = false;
}
// always move mine
mine = null;
if ( thisIter.hasNext() ) {
mine = (Interval)thisIter.next();
}
if ( moveTheirs ) {
theirs = null;
if ( otherIter.hasNext() ) {
theirs = (Interval)otherIter.next();
}
}
}
}
}
return diff;
}
*/
/** TODO: implement this! */
public IntSet or(IntSet a) {
IntervalSet o = new IntervalSet();
o.addAll(this);
o.addAll(a);
//throw new NoSuchMethodError();
return o;
}
/** Return a new set with the intersection of this set with other. Because
* the intervals are sorted, we can use an iterator for each list and
* just walk them together. This is roughly O(min(n,m)) for interval
* list lengths n and m.
*/
public IntSet and(IntSet other) {
if ( other==null ) { //|| !(other instanceof IntervalSet) ) {
return null; // nothing in common with null set
}
ArrayList myIntervals = (ArrayList)this.intervals;
ArrayList theirIntervals = (ArrayList)((IntervalSet)other).intervals;
IntervalSet intersection = null;
int mySize = myIntervals.size();
int theirSize = theirIntervals.size();
int i = 0;
int j = 0;
// iterate down both interval lists looking for nondisjoint intervals
while ( i<mySize && j<theirSize ) {
Interval mine = (Interval)myIntervals.get(i);
Interval theirs = (Interval)theirIntervals.get(j);
//System.out.println("mine="+mine+" and theirs="+theirs);
if ( mine.startsBeforeDisjoint(theirs) ) {
// move this iterator looking for interval that might overlap
i++;
}
else if ( theirs.startsBeforeDisjoint(mine) ) {
// move other iterator looking for interval that might overlap
j++;
}
else if ( mine.properlyContains(theirs) ) {
// overlap, add intersection, get next theirs
if ( intersection==null ) {
intersection = new IntervalSet();
}
intersection.add(mine.intersection(theirs));
j++;
}
else if ( theirs.properlyContains(mine) ) {
// overlap, add intersection, get next mine
if ( intersection==null ) {
intersection = new IntervalSet();
}
intersection.add(mine.intersection(theirs));
i++;
}
else if ( !mine.disjoint(theirs) ) {
// overlap, add intersection
if ( intersection==null ) {
intersection = new IntervalSet();
}
intersection.add(mine.intersection(theirs));
// Move the iterator of lower range [a..b], but not
// the upper range as it may contain elements that will collide
// with the next iterator. So, if mine=[0..115] and
// theirs=[115..200], then intersection is 115 and move mine
// but not theirs as theirs may collide with the next range
// in thisIter.
// move both iterators to next ranges
if ( mine.startsAfterNonDisjoint(theirs) ) {
j++;
}
else if ( theirs.startsAfterNonDisjoint(mine) ) {
i++;
}
}
}
if ( intersection==null ) {
return new IntervalSet();
}
return intersection;
}
/** Is el in any range of this set? */
public boolean member(int el) {
int n = intervals.size();
for (int i = 0; i < n; i++) {
Interval I = (Interval) intervals.get(i);
int a = I.a;
int b = I.b;
if ( el<a ) {
break; // list is sorted and el is before this interval; not here
}
if ( el>=a && el<=b ) {
return true; // found in this interval
}
}
return false;
/*
for (ListIterator iter = intervals.listIterator(); iter.hasNext();) {
Interval I = (Interval) iter.next();
if ( el<I.a ) {
break; // list is sorted and el is before this interval; not here
}
if ( el>=I.a && el<=I.b ) {
return true; // found in this interval
}
}
return false;
*/
}
/** return true if this set has no members */
public boolean isNil() {
return intervals==null || intervals.size()==0;
}
/** If this set is a single integer, return it otherwise Label.INVALID */
public int getSingleElement() {
if ( intervals!=null && intervals.size()==1 ) {
Interval I = (Interval)intervals.get(0);
if ( I.a == I.b ) {
return I.a;
}
}
return Label.INVALID;
}
public int getMaxElement() {
if ( isNil() ) {
return Label.INVALID;
}
Interval last = (Interval)intervals.get(intervals.size()-1);
return last.b;
}
/** Return minimum element >= 0 */
public int getMinElement() {
if ( isNil() ) {
return Label.INVALID;
}
int n = intervals.size();
for (int i = 0; i < n; i++) {
Interval I = (Interval) intervals.get(i);
int a = I.a;
int b = I.b;
for (int v=a; v<=b; v++) {
if ( v>=0 ) return v;
}
}
return Label.INVALID;
}
/** Return a list of Interval objects. */
public List<Interval> getIntervals() {
return intervals;
}
/** Are two IntervalSets equal? Because all intervals are sorted
* and disjoint, equals is a simple linear walk over both lists
* to make sure they are the same. Interval.equals() is used
* by the List.equals() method to check the ranges.
*/
public boolean equals(Object obj) {
if ( obj==null || !(obj instanceof IntervalSet) ) {
return false;
}
IntervalSet other = (IntervalSet)obj;
return this.intervals.equals(other.intervals);
}
public String toString() {
return toString(null);
}
public String toString(Grammar g) {
StringBuffer buf = new StringBuffer();
if ( this.intervals==null || this.intervals.size()==0 ) {
return "{}";
}
if ( this.intervals.size()>1 ) {
buf.append("{");
}
Iterator iter = this.intervals.iterator();
while (iter.hasNext()) {
Interval I = (Interval) iter.next();
int a = I.a;
int b = I.b;
if ( a==b ) {
if ( g!=null ) {
buf.append(g.getTokenDisplayName(a));
}
else {
buf.append(a);
}
}
else {
if ( g!=null ) {
buf.append(g.getTokenDisplayName(a)+".."+g.getTokenDisplayName(b));
}
else {
buf.append(a+".."+b);
}
}
if ( iter.hasNext() ) {
buf.append(", ");
}
}
if ( this.intervals.size()>1 ) {
buf.append("}");
}
return buf.toString();
}
public int size() {
int n = 0;
int numIntervals = intervals.size();
if ( numIntervals==1 ) {
Interval firstInterval = this.intervals.get(0);
return firstInterval.b-firstInterval.a+1;
}
for (int i = 0; i < numIntervals; i++) {
Interval I = (Interval) intervals.get(i);
n += (I.b-I.a+1);
}
return n;
}
public List toList() {
List values = new ArrayList();
int n = intervals.size();
for (int i = 0; i < n; i++) {
Interval I = (Interval) intervals.get(i);
int a = I.a;
int b = I.b;
for (int v=a; v<=b; v++) {
values.add(Utils.integer(v));
}
}
return values;
}
/** Get the ith element of ordered set. Used only by RandomPhrase so
* don't bother to implement if you're not doing that for a new
* ANTLR code gen target.
*/
public int get(int i) {
int n = intervals.size();
int index = 0;
for (int j = 0; j < n; j++) {
Interval I = (Interval) intervals.get(j);
int a = I.a;
int b = I.b;
for (int v=a; v<=b; v++) {
if ( index==i ) {
return v;
}
index++;
}
}
return -1;
}
public int[] toArray() {
int[] values = new int[size()];
int n = intervals.size();
int j = 0;
for (int i = 0; i < n; i++) {
Interval I = (Interval) intervals.get(i);
int a = I.a;
int b = I.b;
for (int v=a; v<=b; v++) {
values[j] = v;
j++;
}
}
return values;
}
public org.antlr.runtime.BitSet toRuntimeBitSet() {
org.antlr.runtime.BitSet s =
new org.antlr.runtime.BitSet(getMaxElement()+1);
int n = intervals.size();
for (int i = 0; i < n; i++) {
Interval I = (Interval) intervals.get(i);
int a = I.a;
int b = I.b;
for (int v=a; v<=b; v++) {
s.add(v);
}
}
return s;
}
public void remove(int el) {
throw new NoSuchMethodError("IntervalSet.remove() unimplemented");
}
/*
protected void finalize() throws Throwable {
super.finalize();
System.out.println("size "+intervals.size()+" "+size());
}
*/
}
antlr-3.2/tool/src/main/java/org/antlr/misc/MultiMap.java 0000644 0001750 0001750 00000003554 11256465230 023315 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
import java.util.HashMap;
import java.util.List;
import java.util.ArrayList;
/** A hash table that maps a key to a list of elements not just a single. */
public class MultiMap<K, V> extends HashMap<K, List<V>> {
public void map(K key, V value) {
List<V> elementsForKey = get(key);
if ( elementsForKey==null ) {
elementsForKey = new ArrayList<V>();
super.put(key, elementsForKey);
}
elementsForKey.add(value);
}
}
antlr-3.2/tool/src/main/java/org/antlr/misc/BitSet.java 0000644 0001750 0001750 00000040054 11256465230 022753 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
import org.antlr.analysis.Label;
import org.antlr.tool.Grammar;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
/**A BitSet to replace java.util.BitSet.
*
* Primary differences are that most set operators return new sets
* as opposed to oring and anding "in place". Further, a number of
* operations were added. I cannot contain a BitSet because there
* is no way to access the internal bits (which I need for speed)
* and, because it is final, I cannot subclass to add functionality.
* Consider defining set degree. Without access to the bits, I must
* call a method n times to test the ith bit...ack!
*
* Also seems like or() from util is wrong when size of incoming set is bigger
* than this.bits.length.
*
* @author Terence Parr
*/
public class BitSet implements IntSet, Cloneable {
protected final static int BITS = 64; // number of bits / long
protected final static int LOG_BITS = 6; // 2^6 == 64
/* We will often need to do a mod operator (i mod nbits). Its
* turns out that, for powers of two, this mod operation is
* same as (i & (nbits-1)). Since mod is slow, we use a
* precomputed mod mask to do the mod instead.
*/
protected final static int MOD_MASK = BITS - 1;
/** The actual data bits */
protected long bits[];
/** Construct a bitset of size one word (64 bits) */
public BitSet() {
this(BITS);
}
/** Construction from a static array of longs */
public BitSet(long[] bits_) {
bits = bits_;
}
/** Construct a bitset given the size
* @param nbits The size of the bitset in bits
*/
public BitSet(int nbits) {
bits = new long[((nbits - 1) >> LOG_BITS) + 1];
}
/** or this element into this set (grow as necessary to accommodate) */
public void add(int el) {
//System.out.println("add("+el+")");
int n = wordNumber(el);
//System.out.println("word number is "+n);
//System.out.println("bits.length "+bits.length);
if (n >= bits.length) {
growToInclude(el);
}
bits[n] |= bitMask(el);
}
public void addAll(IntSet set) {
if ( set instanceof BitSet ) {
this.orInPlace((BitSet)set);
}
else if ( set instanceof IntervalSet ) {
IntervalSet other = (IntervalSet)set;
// walk set and add each interval
for (Iterator iter = other.intervals.iterator(); iter.hasNext();) {
Interval I = (Interval) iter.next();
this.orInPlace(BitSet.range(I.a,I.b));
}
}
else {
throw new IllegalArgumentException("can't add "+
set.getClass().getName()+
" to BitSet");
}
}
public void addAll(int[] elements) {
if ( elements==null ) {
return;
}
for (int i = 0; i < elements.length; i++) {
int e = elements[i];
add(e);
}
}
public void addAll(Iterable elements) {
if ( elements==null ) {
return;
}
Iterator it = elements.iterator();
while (it.hasNext()) {
Object o = (Object) it.next();
if ( !(o instanceof Integer) ) {
throw new IllegalArgumentException();
}
Integer eI = (Integer)o;
add(eI.intValue());
}
/*
int n = elements.size();
for (int i = 0; i < n; i++) {
Object o = elements.get(i);
if ( !(o instanceof Integer) ) {
throw new IllegalArgumentException();
}
Integer eI = (Integer)o;
add(eI.intValue());
}
*/
}
public IntSet and(IntSet a) {
BitSet s = (BitSet)this.clone();
s.andInPlace((BitSet)a);
return s;
}
public void andInPlace(BitSet a) {
int min = Math.min(bits.length, a.bits.length);
for (int i = min - 1; i >= 0; i--) {
bits[i] &= a.bits[i];
}
// clear all bits in this not present in a (if this bigger than a).
for (int i = min; i < bits.length; i++) {
bits[i] = 0;
}
}
private final static long bitMask(int bitNumber) {
int bitPosition = bitNumber & MOD_MASK; // bitNumber mod BITS
return 1L << bitPosition;
}
public void clear() {
for (int i = bits.length - 1; i >= 0; i--) {
bits[i] = 0;
}
}
public void clear(int el) {
int n = wordNumber(el);
if (n >= bits.length) { // grow as necessary to accommodate
growToInclude(el);
}
bits[n] &= ~bitMask(el);
}
public Object clone() {
BitSet s;
try {
s = (BitSet)super.clone();
s.bits = new long[bits.length];
System.arraycopy(bits, 0, s.bits, 0, bits.length);
}
catch (CloneNotSupportedException e) {
throw new InternalError();
}
return s;
}
public int size() {
int deg = 0;
for (int i = bits.length - 1; i >= 0; i--) {
long word = bits[i];
if (word != 0L) {
for (int bit = BITS - 1; bit >= 0; bit--) {
if ((word & (1L << bit)) != 0) {
deg++;
}
}
}
}
return deg;
}
public boolean equals(Object other) {
if ( other == null || !(other instanceof BitSet) ) {
return false;
}
BitSet otherSet = (BitSet)other;
int n = Math.min(this.bits.length, otherSet.bits.length);
// for any bits in common, compare
for (int i=0; i<n; i++) {
if (this.bits[i] != otherSet.bits[i]) {
return false;
}
}
// make sure any extra bits are off
if (this.bits.length > n) {
for (int i = n+1; i<this.bits.length; i++) {
if (this.bits[i] != 0) {
return false;
}
}
}
else if (otherSet.bits.length > n) {
for (int i = n+1; i<otherSet.bits.length; i++) {
if (otherSet.bits[i] != 0) {
return false;
}
}
}
return true;
}
/**
* Grows the set to a larger number of bits.
* @param bit element that must fit in set
*/
public void growToInclude(int bit) {
int newSize = Math.max(bits.length << 1, numWordsToHold(bit));
long newbits[] = new long[newSize];
System.arraycopy(bits, 0, newbits, 0, bits.length);
bits = newbits;
}
public boolean member(int el) {
int n = wordNumber(el);
if (n >= bits.length) return false;
return (bits[n] & bitMask(el)) != 0;
}
/** Get the first element you find and return it. Return Label.INVALID
* otherwise.
*/
public int getSingleElement() {
for (int i = 0; i < (bits.length << LOG_BITS); i++) {
if (member(i)) {
return i;
}
}
return Label.INVALID;
}
public boolean isNil() {
for (int i = bits.length - 1; i >= 0; i--) {
if (bits[i] != 0) return false;
}
return true;
}
public IntSet complement() {
BitSet s = (BitSet)this.clone();
s.notInPlace();
return s;
}
public IntSet complement(IntSet set) {
if ( set==null ) {
return this.complement();
}
return set.subtract(this);
}
public void notInPlace() {
for (int i = bits.length - 1; i >= 0; i--) {
bits[i] = ~bits[i];
}
}
/** complement bits in the range 0..maxBit. */
public void notInPlace(int maxBit) {
notInPlace(0, maxBit);
}
/** complement bits in the range minBit..maxBit.*/
public void notInPlace(int minBit, int maxBit) {
// make sure that we have room for maxBit
growToInclude(maxBit);
for (int i = minBit; i <= maxBit; i++) {
int n = wordNumber(i);
bits[n] ^= bitMask(i);
}
}
private final int numWordsToHold(int el) {
return (el >> LOG_BITS) + 1;
}
public static BitSet of(int el) {
BitSet s = new BitSet(el + 1);
s.add(el);
return s;
}
public static BitSet of(Collection elements) {
BitSet s = new BitSet();
Iterator iter = elements.iterator();
while (iter.hasNext()) {
Integer el = (Integer) iter.next();
s.add(el.intValue());
}
return s;
}
public static BitSet of(IntSet set) {
if ( set==null ) {
return null;
}
if ( set instanceof BitSet ) {
return (BitSet)set;
}
if ( set instanceof IntervalSet ) {
BitSet s = new BitSet();
s.addAll(set);
return s;
}
throw new IllegalArgumentException("can't create BitSet from "+set.getClass().getName());
}
public static BitSet of(Map elements) {
return BitSet.of(elements.keySet());
}
public static BitSet range(int a, int b) {
BitSet s = new BitSet(b + 1);
for (int i = a; i <= b; i++) {
int n = wordNumber(i);
s.bits[n] |= bitMask(i);
}
return s;
}
/** return this | a in a new set */
public IntSet or(IntSet a) {
if ( a==null ) {
return this;
}
BitSet s = (BitSet)this.clone();
s.orInPlace((BitSet)a);
return s;
}
public void orInPlace(BitSet a) {
if ( a==null ) {
return;
}
// If this is smaller than a, grow this first
if (a.bits.length > bits.length) {
setSize(a.bits.length);
}
int min = Math.min(bits.length, a.bits.length);
for (int i = min - 1; i >= 0; i--) {
bits[i] |= a.bits[i];
}
}
// remove this element from this set
public void remove(int el) {
int n = wordNumber(el);
if (n >= bits.length) {
growToInclude(el);
}
bits[n] &= ~bitMask(el);
}
/**
* Sets the size of a set.
* @param nwords how many words the new set should be
*/
private void setSize(int nwords) {
long newbits[] = new long[nwords];
int n = Math.min(nwords, bits.length);
System.arraycopy(bits, 0, newbits, 0, n);
bits = newbits;
}
public int numBits() {
return bits.length << LOG_BITS; // num words * bits per word
}
/** return how much space is being used by the bits array not
* how many actually have member bits on.
*/
public int lengthInLongWords() {
return bits.length;
}
/**Is this contained within a? */
public boolean subset(BitSet a) {
if (a == null) return false;
return this.and(a).equals(this);
}
/**Subtract the elements of 'a' from 'this' in-place.
* Basically, just turn off all bits of 'this' that are in 'a'.
*/
public void subtractInPlace(BitSet a) {
if (a == null) return;
// for all words of 'a', turn off corresponding bits of 'this'
for (int i = 0; i < bits.length && i < a.bits.length; i++) {
bits[i] &= ~a.bits[i];
}
}
public IntSet subtract(IntSet a) {
if (a == null || !(a instanceof BitSet)) return null;
BitSet s = (BitSet)this.clone();
s.subtractInPlace((BitSet)a);
return s;
}
public List toList() {
throw new NoSuchMethodError("BitSet.toList() unimplemented");
}
public int[] toArray() {
int[] elems = new int[size()];
int en = 0;
for (int i = 0; i < (bits.length << LOG_BITS); i++) {
if (member(i)) {
elems[en++] = i;
}
}
return elems;
}
public long[] toPackedArray() {
return bits;
}
public String toString() {
return toString(null);
}
/** Transform a bit set into a string by formatting each element as an integer
* separator The string to put in between elements
* @return A commma-separated list of values
*/
public String toString(Grammar g) {
StringBuffer buf = new StringBuffer();
String separator = ",";
boolean havePrintedAnElement = false;
buf.append('{');
for (int i = 0; i < (bits.length << LOG_BITS); i++) {
if (member(i)) {
if (i > 0 && havePrintedAnElement ) {
buf.append(separator);
}
if ( g!=null ) {
buf.append(g.getTokenDisplayName(i));
}
else {
buf.append(i);
}
havePrintedAnElement = true;
}
}
buf.append('}');
return buf.toString();
}
/**Create a string representation where instead of integer elements, the
* ith element of vocabulary is displayed instead. Vocabulary is a Vector
* of Strings.
* separator The string to put in between elements
* @return A commma-separated list of character constants.
*/
public String toString(String separator, List vocabulary) {
if (vocabulary == null) {
return toString(null);
}
String str = "";
for (int i = 0; i < (bits.length << LOG_BITS); i++) {
if (member(i)) {
if (str.length() > 0) {
str += separator;
}
if (i >= vocabulary.size()) {
str += "'" + (char)i + "'";
}
else if (vocabulary.get(i) == null) {
str += "'" + (char)i + "'";
}
else {
str += (String)vocabulary.get(i);
}
}
}
return str;
}
/**
* Dump a comma-separated list of the words making up the bit set.
* Split each 64 bit number into two more manageable 32 bit numbers.
* This generates a comma-separated list of C++-like unsigned long constants.
*/
public String toStringOfHalfWords() {
StringBuffer s = new StringBuffer();
for (int i = 0; i < bits.length; i++) {
if (i != 0) s.append(", ");
long tmp = bits[i];
tmp &= 0xFFFFFFFFL;
s.append(tmp);
s.append("UL");
s.append(", ");
tmp = bits[i] >>> 32;
tmp &= 0xFFFFFFFFL;
s.append(tmp);
s.append("UL");
}
return s.toString();
}
/**
* Dump a comma-separated list of the words making up the bit set.
* This generates a comma-separated list of Java-like long int constants.
*/
public String toStringOfWords() {
StringBuffer s = new StringBuffer();
for (int i = 0; i < bits.length; i++) {
if (i != 0) s.append(", ");
s.append(bits[i]);
s.append("L");
}
return s.toString();
}
public String toStringWithRanges() {
return toString();
}
private final static int wordNumber(int bit) {
return bit >> LOG_BITS; // bit / BITS
}
}
antlr-3.2/tool/src/main/java/org/antlr/misc/Barrier.java 0000644 0001750 0001750 00000004551 11256465230 023151 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
/**A very simple barrier wait. Once a thread has requested a
* wait on the barrier with waitForRelease, it cannot fool the
* barrier into releasing by "hitting" the barrier multiple times--
* the thread is blocked on the wait().
*/
public class Barrier {
protected int threshold;
protected int count = 0;
public Barrier(int t) {
threshold = t;
}
public synchronized void waitForRelease()
throws InterruptedException
{
count++;
// The final thread to reach barrier resets barrier and
// releases all threads
if ( count==threshold ) {
// notify blocked threads that threshold has been reached
action(); // perform the requested operation
notifyAll();
}
else while ( count<threshold ) {
wait();
}
}
/** What to do when everyone reaches barrier */
public void action() {
}
}
antlr-3.2/tool/src/main/java/org/antlr/misc/IntSet.java 0000644 0001750 0001750 00000005406 11256465230 022771 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
import org.antlr.tool.Grammar;
import java.util.List;
/** A generic set of ints that has an efficient implementation, BitSet,
* which is a compressed bitset and is useful for ints that
* are small, for example less than 500 or so, and w/o many ranges. For
* ranges with large values like unicode char sets, this is not very efficient.
* Consider using IntervalSet. Not all methods in IntervalSet are implemented.
*
* @see org.antlr.misc.BitSet
* @see org.antlr.misc.IntervalSet
*/
public interface IntSet {
/** Add an element to the set */
void add(int el);
/** Add all elements from incoming set to this set. Can limit
* to set of its own type.
*/
void addAll(IntSet set);
/** Return the intersection of this set with the argument, creating
* a new set.
*/
IntSet and(IntSet a);
IntSet complement(IntSet elements);
IntSet or(IntSet a);
IntSet subtract(IntSet a);
/** Return the size of this set (not the underlying implementation's
* allocated memory size, for example).
*/
int size();
boolean isNil();
boolean equals(Object obj);
int getSingleElement();
boolean member(int el);
/** remove this element from this set */
void remove(int el);
List toList();
String toString();
String toString(Grammar g);
}
antlr-3.2/tool/src/main/java/org/antlr/misc/OrderedHashSet.java 0000644 0001750 0001750 00000006761 11256465230 024434 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
/** A HashMap that remembers the order that the elements were added.
* You can alter the ith element with set(i,value) too :) Unique list.
* I need the replace/set-element-i functionality so I'm subclassing
* OrderedHashSet.
*/
public class OrderedHashSet<T> extends HashSet {
/** Track the elements as they are added to the set */
protected List<T> elements = new ArrayList<T>();
public T get(int i) {
return elements.get(i);
}
/** Replace an existing value with a new value; updates the element
* list and the hash table, but not the key as that has not changed.
*/
public T set(int i, T value) {
T oldElement = elements.get(i);
elements.set(i,value); // update list
super.remove(oldElement); // now update the set: remove/add
super.add(value);
return oldElement;
}
/** Add a value to list; keep in hashtable for consistency also;
* Key is object itself. Good for say asking if a certain string is in
* a list of strings.
*/
public boolean add(Object value) {
boolean result = super.add(value);
if ( result ) { // only track if new element not in set
elements.add((T)value);
}
return result;
}
public boolean remove(Object o) {
throw new UnsupportedOperationException();
/*
elements.remove(o);
return super.remove(o);
*/
}
public void clear() {
elements.clear();
super.clear();
}
/** Return the List holding list of table elements. Note that you are
* NOT getting a copy so don't write to the list.
*/
public List<T> elements() {
return elements;
}
public int size() {
/*
if ( elements.size()!=super.size() ) {
ErrorManager.internalError("OrderedHashSet: elements and set size differs; "+
elements.size()+"!="+super.size());
}
*/
return elements.size();
}
public String toString() {
return elements.toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/misc/Interval.java 0000644 0001750 0001750 00000011455 11256465230 023350 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
/** An immutable inclusive interval a..b */
public class Interval {
public static final int INTERVAL_POOL_MAX_VALUE = 1000;
static Interval[] cache = new Interval[INTERVAL_POOL_MAX_VALUE+1];
public int a;
public int b;
public static int creates = 0;
public static int misses = 0;
public static int hits = 0;
public static int outOfRange = 0;
public Interval(int a, int b) { this.a=a; this.b=b; }
/** Interval objects are used readonly so share all with the
* same single value a==b up to some max size. Use an array as a perfect hash.
* Return shared object for 0..INTERVAL_POOL_MAX_VALUE or a new
* Interval object with a..a in it. On Java.g, 218623 IntervalSets
* have a..a (set with 1 element).
*/
public static Interval create(int a, int b) {
//return new Interval(a,b);
// cache just a..a
if ( a!=b || a<0 || a>INTERVAL_POOL_MAX_VALUE ) {
return new Interval(a,b);
}
if ( cache[a]==null ) {
cache[a] = new Interval(a,a);
}
return cache[a];
}
public boolean equals(Object o) {
if ( o==null ) {
return false;
}
Interval other = (Interval)o;
return this.a==other.a && this.b==other.b;
}
/** Does this start completely before other? Disjoint */
public boolean startsBeforeDisjoint(Interval other) {
return this.a<other.a && this.b<other.a;
}
/** Does this start at or before other? Nondisjoint */
public boolean startsBeforeNonDisjoint(Interval other) {
return this.a<=other.a && this.b>=other.a;
}
/** Does this.a start after other.b? May or may not be disjoint */
public boolean startsAfter(Interval other) { return this.a>other.a; }
/** Does this start completely after other? Disjoint */
public boolean startsAfterDisjoint(Interval other) {
return this.a>other.b;
}
/** Does this start after other? NonDisjoint */
public boolean startsAfterNonDisjoint(Interval other) {
return this.a>other.a && this.a<=other.b; // this.b>=other.b implied
}
/** Are both ranges disjoint? I.e., no overlap? */
public boolean disjoint(Interval other) {
return startsBeforeDisjoint(other) || startsAfterDisjoint(other);
}
/** Are two intervals adjacent such as 0..41 and 42..42? */
public boolean adjacent(Interval other) {
return this.a == other.b+1 || this.b == other.a-1;
}
public boolean properlyContains(Interval other) {
return other.a >= this.a && other.b <= this.b;
}
/** Return the interval computed from combining this and other */
public Interval union(Interval other) {
return Interval.create(Math.min(a,other.a), Math.max(b,other.b));
}
/** Return the interval in common between this and o */
public Interval intersection(Interval other) {
return Interval.create(Math.max(a,other.a), Math.min(b,other.b));
}
/** Return the interval with elements from this not in other;
* other must not be totally enclosed (properly contained)
* within this, which would result in two disjoint intervals
* instead of the single one returned by this method.
*/
public Interval differenceNotProperlyContained(Interval other) {
Interval diff = null;
// other.a to left of this.a (or same)
if ( other.startsBeforeNonDisjoint(this) ) {
diff = Interval.create(Math.max(this.a,other.b+1),
this.b);
}
// other.a to right of this.a
else if ( other.startsAfterNonDisjoint(this) ) {
diff = Interval.create(this.a, other.a-1);
}
return diff;
}
public String toString() {
return a+".."+b;
}
}
antlr-3.2/tool/src/main/java/org/antlr/misc/IntArrayList.java 0000644 0001750 0001750 00000010004 11256465230 024136 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.misc;
import java.util.AbstractList;
/** An ArrayList based upon int members. Not quite a real implementation of a
* modifiable list as I don't do, for example, add(index,element).
* TODO: unused?
*/
public class IntArrayList extends AbstractList implements Cloneable {
private static final int DEFAULT_CAPACITY = 10;
protected int n = 0;
protected int[] elements = null;
public IntArrayList() {
this(DEFAULT_CAPACITY);
}
public IntArrayList(int initialCapacity) {
elements = new int[initialCapacity];
}
/** Set the ith element. Like ArrayList, this does NOT affect size. */
public int set(int i, int newValue) {
if ( i>=n ) {
setSize(i); // unlike definition of set in ArrayList, set size
}
int v = elements[i];
elements[i] = newValue;
return v;
}
public boolean add(int o) {
if ( n>=elements.length ) {
grow();
}
elements[n] = o;
n++;
return true;
}
public void setSize(int newSize) {
if ( newSize>=elements.length ) {
ensureCapacity(newSize);
}
n = newSize;
}
protected void grow() {
ensureCapacity((elements.length * 3)/2 + 1);
}
public boolean contains(int v) {
for (int i = 0; i < n; i++) {
int element = elements[i];
if ( element == v ) {
return true;
}
}
return false;
}
public void ensureCapacity(int newCapacity) {
int oldCapacity = elements.length;
if (n>=oldCapacity) {
int oldData[] = elements;
elements = new int[newCapacity];
System.arraycopy(oldData, 0, elements, 0, n);
}
}
public Object get(int i) {
return Utils.integer(element(i));
}
public int element(int i) {
return elements[i];
}
public int[] elements() {
int[] a = new int[n];
System.arraycopy(elements, 0, a, 0, n);
return a;
}
public int size() {
return n;
}
public int capacity() {
return elements.length;
}
public boolean equals(Object o) {
if ( o==null ) {
return false;
}
IntArrayList other = (IntArrayList)o;
if ( this.size()!=other.size() ) {
return false;
}
for (int i = 0; i < n; i++) {
if ( elements[i] != other.elements[i] ) {
return false;
}
}
return true;
}
public Object clone() throws CloneNotSupportedException {
IntArrayList a = (IntArrayList)super.clone();
a.n = this.n;
System.arraycopy(this.elements, 0, a.elements, 0, this.elements.length);
return a;
}
public String toString() {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < n; i++) {
if ( i>0 ) {
buf.append(", ");
}
buf.append(elements[i]);
}
return buf.toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/ 0000755 0001750 0001750 00000000000 11410174107 021354 5 ustar twerner twerner antlr-3.2/tool/src/main/java/org/antlr/codegen/DelphiTarget.java 0000644 0001750 0001750 00000011473 11256465230 024611 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2008 Erik van Bilsen
Copyright (c) 2006 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.Tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
import org.antlr.misc.Utils;
import org.antlr.analysis.Label;
public class DelphiTarget extends Target
{
public DelphiTarget() {
targetCharValueEscape['\n'] = "'#10'";
targetCharValueEscape['\r'] = "'#13'";
targetCharValueEscape['\t'] = "'#9'";
targetCharValueEscape['\b'] = "\\b";
targetCharValueEscape['\f'] = "\\f";
targetCharValueEscape['\\'] = "\\";
targetCharValueEscape['\''] = "''";
targetCharValueEscape['"'] = "'";
}
protected StringTemplate chooseWhereCyclicDFAsGo(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate recognizerST,
StringTemplate cyclicDFAST)
{
return recognizerST;
}
public String encodeIntAsCharEscape(int v)
{
if (v <= 127)
{
String hex1 = Integer.toHexString(v | 0x10000).substring(3, 5);
return "'#$" + hex1 + "'";
}
String hex = Integer.toHexString(v | 0x10000).substring(1, 5);
return "'#$" + hex + "'";
}
public String getTargetCharLiteralFromANTLRCharLiteral(
CodeGenerator generator,
String literal)
{
StringBuffer buf = new StringBuffer();
int c = Grammar.getCharValueFromGrammarCharLiteral(literal);
if ( c<Label.MIN_CHAR_VALUE ) {
return "0";
}
// normal char
buf.append(c);
return buf.toString();
}
public String getTargetStringLiteralFromString(String s, boolean quoted) {
if ( s==null ) {
return null;
}
StringBuffer buf = new StringBuffer();
if ( quoted ) {
buf.append('\'');
}
for (int i=0; i<s.length(); i++) {
int c = s.charAt(i);
if ( c!='"' && // don't escape double quotes in strings for Delphi
c<targetCharValueEscape.length &&
targetCharValueEscape[c]!=null )
{
buf.append(targetCharValueEscape[c]);
}
else {
buf.append((char)c);
}
if ((i & 127) == 127)
{
// Concatenate string literals because Delphi doesn't support literals over 255 characters,
// and the code editor doesn't support lines over 1023 characters
buf.append("\' + \r\n \'");
}
}
if ( quoted ) {
buf.append('\'');
}
return buf.toString();
}
public String getTargetStringLiteralFromANTLRStringLiteral(
CodeGenerator generator,
String literal)
{
literal = Utils.replace(literal,"\\\'","''"); // \' to ' to normalize
literal = Utils.replace(literal,"\\r\\n","'#13#10'");
literal = Utils.replace(literal,"\\r","'#13'");
literal = Utils.replace(literal,"\\n","'#10'");
StringBuffer buf = new StringBuffer(literal);
buf.setCharAt(0,'\'');
buf.setCharAt(literal.length()-1,'\'');
return buf.toString();
}
public String getTarget64BitStringFromValue(long word) {
int numHexDigits = 8*2;
StringBuffer buf = new StringBuffer(numHexDigits+2);
buf.append("$");
String digits = Long.toHexString(word);
digits = digits.toUpperCase();
int padding = numHexDigits - digits.length();
// pad left with zeros
for (int i=1; i<=padding; i++) {
buf.append('0');
}
buf.append(digits);
return buf.toString();
}
} antlr-3.2/tool/src/main/java/org/antlr/codegen/RubyTarget.java 0000644 0001750 0001750 00000005046 11256465230 024324 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005 Martin Traverso
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
public class RubyTarget
extends Target
{
public String getTargetCharLiteralFromANTLRCharLiteral(
CodeGenerator generator,
String literal)
{
literal = literal.substring(1, literal.length() - 1);
String result = "?";
if (literal.equals("\\")) {
result += "\\\\";
}
else if (literal.equals(" ")) {
result += "\\s";
}
else if (literal.startsWith("\\u")) {
result = "0x" + literal.substring(2);
}
else {
result += literal;
}
return result;
}
public int getMaxCharValue(CodeGenerator generator)
{
// we don't support unicode, yet.
return 0xFF;
}
public String getTokenTypeAsTargetLabel(CodeGenerator generator, int ttype)
{
String name = generator.grammar.getTokenDisplayName(ttype);
// If name is a literal, return the token type instead
if ( name.charAt(0)=='\'' ) {
return generator.grammar.computeTokenNameFromLiteral(ttype, name);
}
return name;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/CSharp2Target.java 0000644 0001750 0001750 00000004147 11256465230 024646 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.Tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
public class CSharp2Target extends Target
{
protected StringTemplate chooseWhereCyclicDFAsGo(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate recognizerST,
StringTemplate cyclicDFAST)
{
return recognizerST;
}
public String encodeIntAsCharEscape(int v)
{
if (v <= 127)
{
String hex1 = Integer.toHexString(v | 0x10000).substring(3, 5);
return "\\x" + hex1;
}
String hex = Integer.toHexString(v | 0x10000).substring(1, 5);
return "\\u" + hex;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/PythonTarget.java 0000644 0001750 0001750 00000014114 11256465230 024660 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005 Martin Traverso
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
Please excuse my obvious lack of Java experience. The code here is probably
full of WTFs - though IMHO Java is the Real WTF(TM) here...
*/
package org.antlr.codegen;
import org.antlr.tool.Grammar;
import java.util.*;
public class PythonTarget extends Target {
/** Target must be able to override the labels used for token types */
public String getTokenTypeAsTargetLabel(CodeGenerator generator,
int ttype) {
// use ints for predefined types;
// <invalid> <EOR> <DOWN> <UP>
if ( ttype >= 0 && ttype <= 3 ) {
return String.valueOf(ttype);
}
String name = generator.grammar.getTokenDisplayName(ttype);
// If name is a literal, return the token type instead
if ( name.charAt(0)=='\'' ) {
return String.valueOf(ttype);
}
return name;
}
public String getTargetCharLiteralFromANTLRCharLiteral(
CodeGenerator generator,
String literal) {
int c = Grammar.getCharValueFromGrammarCharLiteral(literal);
return String.valueOf(c);
}
private List splitLines(String text) {
ArrayList l = new ArrayList();
int idx = 0;
while ( true ) {
int eol = text.indexOf("\n", idx);
if ( eol == -1 ) {
l.add(text.substring(idx));
break;
}
else {
l.add(text.substring(idx, eol+1));
idx = eol+1;
}
}
return l;
}
public List postProcessAction(List chunks, antlr.Token actionToken) {
/* TODO
- check for and report TAB usage
*/
//System.out.println("\n*** Action at " + actionToken.getLine() + ":" + actionToken.getColumn());
/* First I create a new list of chunks. String chunks are splitted into
lines and some whitespace my be added at the beginning.
As a result I get a list of chunks
- where the first line starts at column 0
- where every LF is at the end of a string chunk
*/
List nChunks = new ArrayList();
for (int i = 0; i < chunks.size(); i++) {
Object chunk = chunks.get(i);
if ( chunk instanceof String ) {
String text = (String)chunks.get(i);
if ( nChunks.size() == 0 && actionToken.getColumn() > 0 ) {
// first chunk and some 'virtual' WS at beginning
// prepend to this chunk
String ws = "";
for ( int j = 0 ; j < actionToken.getColumn() ; j++ ) {
ws += " ";
}
text = ws + text;
}
List parts = splitLines(text);
for ( int j = 0 ; j < parts.size() ; j++ ) {
chunk = parts.get(j);
nChunks.add(chunk);
}
}
else {
if ( nChunks.size() == 0 && actionToken.getColumn() > 0 ) {
// first chunk and some 'virtual' WS at beginning
// add as a chunk of its own
String ws = "";
for ( int j = 0 ; j < actionToken.getColumn() ; j++ ) {
ws += " ";
}
nChunks.add(ws);
}
nChunks.add(chunk);
}
}
int lineNo = actionToken.getLine();
int col = 0;
// strip trailing empty lines
int lastChunk = nChunks.size() - 1;
while ( lastChunk > 0
&& nChunks.get(lastChunk) instanceof String
&& ((String)nChunks.get(lastChunk)).trim().length() == 0 )
lastChunk--;
// string leading empty lines
int firstChunk = 0;
while ( firstChunk <= lastChunk
&& nChunks.get(firstChunk) instanceof String
&& ((String)nChunks.get(firstChunk)).trim().length() == 0
&& ((String)nChunks.get(firstChunk)).endsWith("\n") ) {
lineNo++;
firstChunk++;
}
int indent = -1;
for ( int i = firstChunk ; i <= lastChunk ; i++ ) {
Object chunk = nChunks.get(i);
//System.out.println(lineNo + ":" + col + " " + quote(chunk.toString()));
if ( chunk instanceof String ) {
String text = (String)chunk;
if ( col == 0 ) {
if ( indent == -1 ) {
// first non-blank line
// count number of leading whitespaces
indent = 0;
for ( int j = 0; j < text.length(); j++ ) {
if ( !Character.isWhitespace(text.charAt(j)) )
break;
indent++;
}
}
if ( text.length() >= indent ) {
int j;
for ( j = 0; j < indent ; j++ ) {
if ( !Character.isWhitespace(text.charAt(j)) ) {
// should do real error reporting here...
System.err.println("Warning: badly indented line " + lineNo + " in action:");
System.err.println(text);
break;
}
}
nChunks.set(i, text.substring(j));
}
else if ( text.trim().length() > 0 ) {
// should do real error reporting here...
System.err.println("Warning: badly indented line " + lineNo + " in action:");
System.err.println(text);
}
}
if ( text.endsWith("\n") ) {
lineNo++;
col = 0;
}
else {
col += text.length();
}
}
else {
// not really correct, but all I need is col to increment...
col += 1;
}
}
return nChunks;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/CTarget.java 0000644 0001750 0001750 00000027312 11256465230 023565 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.Tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
import java.io.IOException;
import java.util.ArrayList;
public class CTarget extends Target {
ArrayList strings = new ArrayList();
@Override
protected void genRecognizerFile(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate outputFileST)
throws IOException {
// Before we write this, and cause it to generate its string,
// we need to add all the string literals that we are going to match
//
outputFileST.setAttribute("literals", strings);
String fileName = generator.getRecognizerFileName(grammar.name, grammar.type);
generator.write(outputFileST, fileName);
}
@Override
protected void genRecognizerHeaderFile(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate headerFileST,
String extName)
throws IOException {
// Pick up the file name we are generating. This method will return a
// a file suffixed with .c, so we must substring and add the extName
// to it as we cannot assign into strings in Java.
///
String fileName = generator.getRecognizerFileName(grammar.name, grammar.type);
fileName = fileName.substring(0, fileName.length() - 2) + extName;
generator.write(headerFileST, fileName);
}
protected StringTemplate chooseWhereCyclicDFAsGo(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate recognizerST,
StringTemplate cyclicDFAST) {
return recognizerST;
}
/** Is scope in @scope::name {action} valid for this kind of grammar?
* Targets like C++ may want to allow new scopes like headerfile or
* some such. The action names themselves are not policed at the
* moment so targets can add template actions w/o having to recompile
* ANTLR.
*/
@Override
public boolean isValidActionScope(int grammarType, String scope) {
switch (grammarType) {
case Grammar.LEXER:
if (scope.equals("lexer")) {
return true;
}
if (scope.equals("header")) {
return true;
}
if (scope.equals("includes")) {
return true;
}
if (scope.equals("preincludes")) {
return true;
}
if (scope.equals("overrides")) {
return true;
}
break;
case Grammar.PARSER:
if (scope.equals("parser")) {
return true;
}
if (scope.equals("header")) {
return true;
}
if (scope.equals("includes")) {
return true;
}
if (scope.equals("preincludes")) {
return true;
}
if (scope.equals("overrides")) {
return true;
}
break;
case Grammar.COMBINED:
if (scope.equals("parser")) {
return true;
}
if (scope.equals("lexer")) {
return true;
}
if (scope.equals("header")) {
return true;
}
if (scope.equals("includes")) {
return true;
}
if (scope.equals("preincludes")) {
return true;
}
if (scope.equals("overrides")) {
return true;
}
break;
case Grammar.TREE_PARSER:
if (scope.equals("treeparser")) {
return true;
}
if (scope.equals("header")) {
return true;
}
if (scope.equals("includes")) {
return true;
}
if (scope.equals("preincludes")) {
return true;
}
if (scope.equals("overrides")) {
return true;
}
break;
}
return false;
}
@Override
public String getTargetCharLiteralFromANTLRCharLiteral(
CodeGenerator generator,
String literal) {
if (literal.startsWith("'\\u")) {
literal = "0x" + literal.substring(3, 7);
} else {
int c = literal.charAt(1);
if (c < 32 || c > 127) {
literal = "0x" + Integer.toHexString(c);
}
}
return literal;
}
/** Convert from an ANTLR string literal found in a grammar file to
* an equivalent string literal in the C target.
* Because we must support Unicode character sets and have chosen
* to have the lexer match UTF32 characters, then we must encode
* string matches to use 32 bit character arrays. Here then we
* must produce the C array and cater for the case where the
* lexer has been encoded with a string such as 'xyz\n',
*/
@Override
public String getTargetStringLiteralFromANTLRStringLiteral(
CodeGenerator generator,
String literal) {
int index;
String bytes;
StringBuffer buf = new StringBuffer();
buf.append("{ ");
// We need ot lose any escaped characters of the form \x and just
// replace them with their actual values as well as lose the surrounding
// quote marks.
//
for (int i = 1; i < literal.length() - 1; i++) {
buf.append("0x");
if (literal.charAt(i) == '\\') {
i++; // Assume that there is a next character, this will just yield
// invalid strings if not, which is what the input would be of course - invalid
switch (literal.charAt(i)) {
case 'u':
case 'U':
buf.append(literal.substring(i + 1, i + 5)); // Already a hex string
i = i + 5; // Move to next string/char/escape
break;
case 'n':
case 'N':
buf.append("0A");
break;
case 'r':
case 'R':
buf.append("0D");
break;
case 't':
case 'T':
buf.append("09");
break;
case 'b':
case 'B':
buf.append("08");
break;
case 'f':
case 'F':
buf.append("0C");
break;
default:
// Anything else is what it is!
//
buf.append(Integer.toHexString((int) literal.charAt(i)).toUpperCase());
break;
}
} else {
buf.append(Integer.toHexString((int) literal.charAt(i)).toUpperCase());
}
buf.append(", ");
}
buf.append(" ANTLR3_STRING_TERMINATOR}");
bytes = buf.toString();
index = strings.indexOf(bytes);
if (index == -1) {
strings.add(bytes);
index = strings.indexOf(bytes);
}
String strref = "lit_" + String.valueOf(index + 1);
return strref;
}
/**
* Overrides the standard grammar analysis so we can prepare the analyser
* a little differently from the other targets.
*
* In particular we want to influence the way the code generator makes assumptions about
* switchs vs ifs, vs table driven DFAs. In general, C code should be generated that
* has the minimum use of tables, and tha meximum use of large switch statements. This
* allows the optimizers to generate very efficient code, it can reduce object code size
* by about 30% and give about a 20% performance improvement over not doing this. Hence,
* for the C target only, we change the defaults here, but only if they are still set to the
* defaults.
*
* @param generator An instance of the generic code generator class.
* @param grammar The grammar that we are currently analyzing
*/
@Override
protected void performGrammarAnalysis(CodeGenerator generator, Grammar grammar) {
// Check to see if the maximum inline DFA states is still set to
// the default size. If it is then whack it all the way up to the maximum that
// we can sensibly get away with.
//
if (CodeGenerator.MAX_ACYCLIC_DFA_STATES_INLINE == CodeGenerator.MADSI_DEFAULT) {
CodeGenerator.MAX_ACYCLIC_DFA_STATES_INLINE = 65535;
}
// Check to see if the maximum switch size is still set to the default
// and bring it up much higher if it is. Modern C compilers can handle
// much bigger switch statements than say Java can and if anyone finds a compiler
// that cannot deal with such big switches, all the need do is generate the
// code with a reduced -Xmaxswitchcaselabels nnn
//
if (CodeGenerator.MAX_SWITCH_CASE_LABELS == CodeGenerator.MSCL_DEFAULT) {
CodeGenerator.MAX_SWITCH_CASE_LABELS = 3000;
}
// Check to see if the number of transitions considered a miminum for using
// a switch is still at the default. Because a switch is still generally faster than
// an if even with small sets, and given that the optimizer will do the best thing with it
// anyway, then we simply want to generate a switch for any number of states.
//
if (CodeGenerator.MIN_SWITCH_ALTS == CodeGenerator.MSA_DEFAULT) {
CodeGenerator.MIN_SWITCH_ALTS = 1;
}
// Now we allow the superclass implementation to do whatever it feels it
// must do.
//
super.performGrammarAnalysis(generator, grammar);
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/CodeGenerator.java 0000644 0001750 0001750 00000133422 11256465230 024755 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2007 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import antlr.ANTLRLexer;
import antlr.RecognitionException;
import antlr.TokenStreamRewriteEngine;
import antlr.collections.AST;
import org.antlr.Tool;
import org.antlr.analysis.*;
import org.antlr.misc.*;
import org.antlr.stringtemplate.*;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import org.antlr.tool.*;
import java.io.IOException;
import java.io.StringReader;
import java.io.Writer;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.antlr.grammar.v2.*;
import org.antlr.grammar.v3.ActionTranslator;
/** ANTLR's code generator.
*
* Generate recognizers derived from grammars. Language independence
* achieved through the use of StringTemplateGroup objects. All output
* strings are completely encapsulated in the group files such as Java.stg.
* Some computations are done that are unused by a particular language.
* This generator just computes and sets the values into the templates;
* the templates are free to use or not use the information.
*
* To make a new code generation target, define X.stg for language X
* by copying from existing Y.stg most closely releated to your language;
* e.g., to do CSharp.stg copy Java.stg. The template group file has a
* bunch of templates that are needed by the code generator. You can add
* a new target w/o even recompiling ANTLR itself. The language=X option
* in a grammar file dictates which templates get loaded/used.
*
* Some language like C need both parser files and header files. Java needs
* to have a separate file for the cyclic DFA as ANTLR generates bytecodes
* directly (which cannot be in the generated parser Java file). To facilitate
* this,
*
* cyclic can be in same file, but header, output must be searpate. recognizer
* is in outptufile.
*/
public class CodeGenerator {
/** When generating SWITCH statements, some targets might need to limit
* the size (based upon the number of case labels). Generally, this
* limit will be hit only for lexers where wildcard in a UNICODE
* vocabulary environment would generate a SWITCH with 65000 labels.
*/
public final static int MSCL_DEFAULT = 300;
public static int MAX_SWITCH_CASE_LABELS = MSCL_DEFAULT;
public final static int MSA_DEFAULT = 3;
public static int MIN_SWITCH_ALTS = MSA_DEFAULT;
public boolean GENERATE_SWITCHES_WHEN_POSSIBLE = true;
//public static boolean GEN_ACYCLIC_DFA_INLINE = true;
public static boolean EMIT_TEMPLATE_DELIMITERS = false;
public final static int MADSI_DEFAULT = 10;
public static int MAX_ACYCLIC_DFA_STATES_INLINE = 10;
public String classpathTemplateRootDirectoryName =
"org/antlr/codegen/templates";
/** Which grammar are we generating code for? Each generator
* is attached to a specific grammar.
*/
public Grammar grammar;
/** What language are we generating? */
protected String language;
/** The target specifies how to write out files and do other language
* specific actions.
*/
public Target target = null;
/** Where are the templates this generator should use to generate code? */
protected StringTemplateGroup templates;
/** The basic output templates without AST or templates stuff; this will be
* the templates loaded for the language such as Java.stg *and* the Dbg
* stuff if turned on. This is used for generating syntactic predicates.
*/
protected StringTemplateGroup baseTemplates;
protected StringTemplate recognizerST;
protected StringTemplate outputFileST;
protected StringTemplate headerFileST;
/** Used to create unique labels */
protected int uniqueLabelNumber = 1;
/** A reference to the ANTLR tool so we can learn about output directories
* and such.
*/
protected Tool tool;
/** Generate debugging event method calls */
protected boolean debug;
/** Create a Tracer object and make the recognizer invoke this. */
protected boolean trace;
/** Track runtime parsing information about decisions etc...
* This requires the debugging event mechanism to work.
*/
protected boolean profile;
protected int lineWidth = 72;
/** I have factored out the generation of acyclic DFAs to separate class */
public ACyclicDFACodeGenerator acyclicDFAGenerator =
new ACyclicDFACodeGenerator(this);
/** I have factored out the generation of cyclic DFAs to separate class */
/*
public CyclicDFACodeGenerator cyclicDFAGenerator =
new CyclicDFACodeGenerator(this);
*/
public static final String VOCAB_FILE_EXTENSION = ".tokens";
protected final static String vocabFilePattern =
"<tokens:{<attr.name>=<attr.type>\n}>" +
"<literals:{<attr.name>=<attr.type>\n}>";
public CodeGenerator(Tool tool, Grammar grammar, String language) {
this.tool = tool;
this.grammar = grammar;
this.language = language;
loadLanguageTarget(language);
}
protected void loadLanguageTarget(String language) {
String targetName = "org.antlr.codegen."+language+"Target";
try {
Class c = Class.forName(targetName);
target = (Target)c.newInstance();
}
catch (ClassNotFoundException cnfe) {
target = new Target(); // use default
}
catch (InstantiationException ie) {
ErrorManager.error(ErrorManager.MSG_CANNOT_CREATE_TARGET_GENERATOR,
targetName,
ie);
}
catch (IllegalAccessException cnfe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_CREATE_TARGET_GENERATOR,
targetName,
cnfe);
}
}
/** load the main language.stg template group file */
public void loadTemplates(String language) {
// get a group loader containing main templates dir and target subdir
String templateDirs =
classpathTemplateRootDirectoryName+":"+
classpathTemplateRootDirectoryName+"/"+language;
//System.out.println("targets="+templateDirs.toString());
StringTemplateGroupLoader loader =
new CommonGroupLoader(templateDirs,
ErrorManager.getStringTemplateErrorListener());
StringTemplateGroup.registerGroupLoader(loader);
StringTemplateGroup.registerDefaultLexer(AngleBracketTemplateLexer.class);
// first load main language template
StringTemplateGroup coreTemplates =
StringTemplateGroup.loadGroup(language);
baseTemplates = coreTemplates;
if ( coreTemplates ==null ) {
ErrorManager.error(ErrorManager.MSG_MISSING_CODE_GEN_TEMPLATES,
language);
return;
}
// dynamically add subgroups that act like filters to apply to
// their supergroup. E.g., Java:Dbg:AST:ASTParser::ASTDbg.
String outputOption = (String)grammar.getOption("output");
if ( outputOption!=null && outputOption.equals("AST") ) {
if ( debug && grammar.type!=Grammar.LEXER ) {
StringTemplateGroup dbgTemplates =
StringTemplateGroup.loadGroup("Dbg", coreTemplates);
baseTemplates = dbgTemplates;
StringTemplateGroup astTemplates =
StringTemplateGroup.loadGroup("AST",dbgTemplates);
StringTemplateGroup astParserTemplates = astTemplates;
//if ( !grammar.rewriteMode() ) {
if ( grammar.type==Grammar.TREE_PARSER ) {
astParserTemplates =
StringTemplateGroup.loadGroup("ASTTreeParser", astTemplates);
}
else {
astParserTemplates =
StringTemplateGroup.loadGroup("ASTParser", astTemplates);
}
//}
StringTemplateGroup astDbgTemplates =
StringTemplateGroup.loadGroup("ASTDbg", astParserTemplates);
templates = astDbgTemplates;
}
else {
StringTemplateGroup astTemplates =
StringTemplateGroup.loadGroup("AST", coreTemplates);
StringTemplateGroup astParserTemplates = astTemplates;
//if ( !grammar.rewriteMode() ) {
if ( grammar.type==Grammar.TREE_PARSER ) {
astParserTemplates =
StringTemplateGroup.loadGroup("ASTTreeParser", astTemplates);
}
else {
astParserTemplates =
StringTemplateGroup.loadGroup("ASTParser", astTemplates);
}
//}
templates = astParserTemplates;
}
}
else if ( outputOption!=null && outputOption.equals("template") ) {
if ( debug && grammar.type!=Grammar.LEXER ) {
StringTemplateGroup dbgTemplates =
StringTemplateGroup.loadGroup("Dbg", coreTemplates);
baseTemplates = dbgTemplates;
StringTemplateGroup stTemplates =
StringTemplateGroup.loadGroup("ST",dbgTemplates);
templates = stTemplates;
}
else {
templates = StringTemplateGroup.loadGroup("ST", coreTemplates);
}
}
else if ( debug && grammar.type!=Grammar.LEXER ) {
templates = StringTemplateGroup.loadGroup("Dbg", coreTemplates);
baseTemplates = templates;
}
else {
templates = coreTemplates;
}
if ( EMIT_TEMPLATE_DELIMITERS ) {
templates.emitDebugStartStopStrings(true);
templates.doNotEmitDebugStringsForTemplate("codeFileExtension");
templates.doNotEmitDebugStringsForTemplate("headerFileExtension");
}
}
/** Given the grammar to which we are attached, walk the AST associated
* with that grammar to create NFAs. Then create the DFAs for all
* decision points in the grammar by converting the NFAs to DFAs.
* Finally, walk the AST again to generate code.
*
* Either 1 or 2 files are written:
*
* recognizer: the main parser/lexer/treewalker item
* header file: language like C/C++ need extern definitions
*
* The target, such as JavaTarget, dictates which files get written.
*/
public StringTemplate genRecognizer() {
//System.out.println("### generate "+grammar.name+" recognizer");
// LOAD OUTPUT TEMPLATES
loadTemplates(language);
if ( templates==null ) {
return null;
}
// CREATE NFA FROM GRAMMAR, CREATE DFA FROM NFA
if ( ErrorManager.doNotAttemptAnalysis() ) {
return null;
}
target.performGrammarAnalysis(this, grammar);
// some grammar analysis errors will not yield reliable DFA
if ( ErrorManager.doNotAttemptCodeGen() ) {
return null;
}
// OPTIMIZE DFA
DFAOptimizer optimizer = new DFAOptimizer(grammar);
optimizer.optimize();
// OUTPUT FILE (contains recognizerST)
outputFileST = templates.getInstanceOf("outputFile");
// HEADER FILE
if ( templates.isDefined("headerFile") ) {
headerFileST = templates.getInstanceOf("headerFile");
}
else {
// create a dummy to avoid null-checks all over code generator
headerFileST = new StringTemplate(templates,"");
headerFileST.setName("dummy-header-file");
}
boolean filterMode = grammar.getOption("filter")!=null &&
grammar.getOption("filter").equals("true");
boolean canBacktrack = grammar.getSyntacticPredicates()!=null ||
grammar.composite.getRootGrammar().atLeastOneBacktrackOption ||
filterMode;
// TODO: move this down further because generating the recognizer
// alters the model with info on who uses predefined properties etc...
// The actions here might refer to something.
// The only two possible output files are available at this point.
// Verify action scopes are ok for target and dump actions into output
// Templates can say <actions.parser.header> for example.
Map actions = grammar.getActions();
verifyActionScopesOkForTarget(actions);
// translate $x::y references
translateActionAttributeReferences(actions);
StringTemplate gateST = templates.getInstanceOf("actionGate");
if ( filterMode ) {
// if filtering, we need to set actions to execute at backtracking
// level 1 not 0.
gateST = templates.getInstanceOf("filteringActionGate");
}
grammar.setSynPredGateIfNotAlready(gateST);
headerFileST.setAttribute("actions", actions);
outputFileST.setAttribute("actions", actions);
headerFileST.setAttribute("buildTemplate", new Boolean(grammar.buildTemplate()));
outputFileST.setAttribute("buildTemplate", new Boolean(grammar.buildTemplate()));
headerFileST.setAttribute("buildAST", new Boolean(grammar.buildAST()));
outputFileST.setAttribute("buildAST", new Boolean(grammar.buildAST()));
outputFileST.setAttribute("rewriteMode", Boolean.valueOf(grammar.rewriteMode()));
headerFileST.setAttribute("rewriteMode", Boolean.valueOf(grammar.rewriteMode()));
outputFileST.setAttribute("backtracking", Boolean.valueOf(canBacktrack));
headerFileST.setAttribute("backtracking", Boolean.valueOf(canBacktrack));
// turn on memoize attribute at grammar level so we can create ruleMemo.
// each rule has memoize attr that hides this one, indicating whether
// it needs to save results
String memoize = (String)grammar.getOption("memoize");
outputFileST.setAttribute("memoize",
(grammar.atLeastOneRuleMemoizes||
Boolean.valueOf(memoize!=null&&memoize.equals("true"))&&
canBacktrack));
headerFileST.setAttribute("memoize",
(grammar.atLeastOneRuleMemoizes||
Boolean.valueOf(memoize!=null&&memoize.equals("true"))&&
canBacktrack));
outputFileST.setAttribute("trace", Boolean.valueOf(trace));
headerFileST.setAttribute("trace", Boolean.valueOf(trace));
outputFileST.setAttribute("profile", Boolean.valueOf(profile));
headerFileST.setAttribute("profile", Boolean.valueOf(profile));
// RECOGNIZER
if ( grammar.type==Grammar.LEXER ) {
recognizerST = templates.getInstanceOf("lexer");
outputFileST.setAttribute("LEXER", Boolean.valueOf(true));
headerFileST.setAttribute("LEXER", Boolean.valueOf(true));
recognizerST.setAttribute("filterMode",
Boolean.valueOf(filterMode));
}
else if ( grammar.type==Grammar.PARSER ||
grammar.type==Grammar.COMBINED )
{
recognizerST = templates.getInstanceOf("parser");
outputFileST.setAttribute("PARSER", Boolean.valueOf(true));
headerFileST.setAttribute("PARSER", Boolean.valueOf(true));
}
else {
recognizerST = templates.getInstanceOf("treeParser");
outputFileST.setAttribute("TREE_PARSER", Boolean.valueOf(true));
headerFileST.setAttribute("TREE_PARSER", Boolean.valueOf(true));
recognizerST.setAttribute("filterMode",
Boolean.valueOf(filterMode));
}
outputFileST.setAttribute("recognizer", recognizerST);
headerFileST.setAttribute("recognizer", recognizerST);
outputFileST.setAttribute("actionScope",
grammar.getDefaultActionScope(grammar.type));
headerFileST.setAttribute("actionScope",
grammar.getDefaultActionScope(grammar.type));
String targetAppropriateFileNameString =
target.getTargetStringLiteralFromString(grammar.getFileName());
outputFileST.setAttribute("fileName", targetAppropriateFileNameString);
headerFileST.setAttribute("fileName", targetAppropriateFileNameString);
outputFileST.setAttribute("ANTLRVersion", tool.VERSION);
headerFileST.setAttribute("ANTLRVersion", tool.VERSION);
outputFileST.setAttribute("generatedTimestamp", Tool.getCurrentTimeStamp());
headerFileST.setAttribute("generatedTimestamp", Tool.getCurrentTimeStamp());
// GENERATE RECOGNIZER
// Walk the AST holding the input grammar, this time generating code
// Decisions are generated by using the precomputed DFAs
// Fill in the various templates with data
CodeGenTreeWalker gen = new CodeGenTreeWalker();
try {
gen.grammar((AST)grammar.getGrammarTree(),
grammar,
recognizerST,
outputFileST,
headerFileST);
}
catch (RecognitionException re) {
ErrorManager.error(ErrorManager.MSG_BAD_AST_STRUCTURE,
re);
}
genTokenTypeConstants(recognizerST);
genTokenTypeConstants(outputFileST);
genTokenTypeConstants(headerFileST);
if ( grammar.type!=Grammar.LEXER ) {
genTokenTypeNames(recognizerST);
genTokenTypeNames(outputFileST);
genTokenTypeNames(headerFileST);
}
// Now that we know what synpreds are used, we can set into template
Set synpredNames = null;
if ( grammar.synPredNamesUsedInDFA.size()>0 ) {
synpredNames = grammar.synPredNamesUsedInDFA;
}
outputFileST.setAttribute("synpreds", synpredNames);
headerFileST.setAttribute("synpreds", synpredNames);
// all recognizers can see Grammar object
recognizerST.setAttribute("grammar", grammar);
// WRITE FILES
try {
target.genRecognizerFile(tool,this,grammar,outputFileST);
if ( templates.isDefined("headerFile") ) {
StringTemplate extST = templates.getInstanceOf("headerFileExtension");
target.genRecognizerHeaderFile(tool,this,grammar,headerFileST,extST.toString());
}
// write out the vocab interchange file; used by antlr,
// does not change per target
StringTemplate tokenVocabSerialization = genTokenVocabOutput();
String vocabFileName = getVocabFileName();
if ( vocabFileName!=null ) {
write(tokenVocabSerialization, vocabFileName);
}
//System.out.println(outputFileST.getDOTForDependencyGraph(false));
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE,
getVocabFileName(),
ioe);
}
/*
System.out.println("num obj.prop refs: "+ ASTExpr.totalObjPropRefs);
System.out.println("num reflection lookups: "+ ASTExpr.totalReflectionLookups);
*/
return outputFileST;
}
/** Some targets will have some extra scopes like C++ may have
* '@headerfile:name {action}' or something. Make sure the
* target likes the scopes in action table.
*/
protected void verifyActionScopesOkForTarget(Map actions) {
Set actionScopeKeySet = actions.keySet();
for (Iterator it = actionScopeKeySet.iterator(); it.hasNext();) {
String scope = (String)it.next();
if ( !target.isValidActionScope(grammar.type, scope) ) {
// get any action from the scope to get error location
Map scopeActions = (Map)actions.get(scope);
GrammarAST actionAST =
(GrammarAST)scopeActions.values().iterator().next();
ErrorManager.grammarError(
ErrorManager.MSG_INVALID_ACTION_SCOPE,grammar,
actionAST.getToken(),scope,
grammar.getGrammarTypeString());
}
}
}
/** Actions may reference $x::y attributes, call translateAction on
* each action and replace that action in the Map.
*/
protected void translateActionAttributeReferences(Map actions) {
Set actionScopeKeySet = actions.keySet();
for (Iterator it = actionScopeKeySet.iterator(); it.hasNext();) {
String scope = (String)it.next();
Map scopeActions = (Map)actions.get(scope);
translateActionAttributeReferencesForSingleScope(null,scopeActions);
}
}
/** Use for translating rule @init{...} actions that have no scope */
public void translateActionAttributeReferencesForSingleScope(
Rule r,
Map scopeActions)
{
String ruleName=null;
if ( r!=null ) {
ruleName = r.name;
}
Set actionNameSet = scopeActions.keySet();
for (Iterator nameIT = actionNameSet.iterator(); nameIT.hasNext();) {
String name = (String) nameIT.next();
GrammarAST actionAST = (GrammarAST)scopeActions.get(name);
List chunks = translateAction(ruleName,actionAST);
scopeActions.put(name, chunks); // replace with translation
}
}
/** Error recovery in ANTLR recognizers.
*
* Based upon original ideas:
*
* Algorithms + Data Structures = Programs by Niklaus Wirth
*
* and
*
* A note on error recovery in recursive descent parsers:
* http://portal.acm.org/citation.cfm?id=947902.947905
*
* Later, Josef Grosch had some good ideas:
* Efficient and Comfortable Error Recovery in Recursive Descent Parsers:
* ftp://www.cocolab.com/products/cocktail/doca4.ps/ell.ps.zip
*
* Like Grosch I implemented local FOLLOW sets that are combined at run-time
* upon error to avoid parsing overhead.
*/
public void generateLocalFOLLOW(GrammarAST referencedElementNode,
String referencedElementName,
String enclosingRuleName,
int elementIndex)
{
/*
System.out.println("compute FOLLOW "+grammar.name+"."+referencedElementNode.toString()+
" for "+referencedElementName+"#"+elementIndex +" in "+
enclosingRuleName+
" line="+referencedElementNode.getLine());
*/
NFAState followingNFAState = referencedElementNode.followingNFAState;
LookaheadSet follow = null;
if ( followingNFAState!=null ) {
// compute follow for this element and, as side-effect, track
// the rule LOOK sensitivity.
follow = grammar.FIRST(followingNFAState);
}
if ( follow==null ) {
ErrorManager.internalError("no follow state or cannot compute follow");
follow = new LookaheadSet();
}
if ( follow.member(Label.EOF) ) {
// TODO: can we just remove? Seems needed here:
// compilation_unit : global_statement* EOF
// Actually i guess we resync to EOF regardless
follow.remove(Label.EOF);
}
//System.out.println(" "+follow);
List tokenTypeList = null;
long[] words = null;
if ( follow.tokenTypeSet==null ) {
words = new long[1];
tokenTypeList = new ArrayList();
}
else {
BitSet bits = BitSet.of(follow.tokenTypeSet);
words = bits.toPackedArray();
tokenTypeList = follow.tokenTypeSet.toList();
}
// use the target to convert to hex strings (typically)
String[] wordStrings = new String[words.length];
for (int j = 0; j < words.length; j++) {
long w = words[j];
wordStrings[j] = target.getTarget64BitStringFromValue(w);
}
recognizerST.setAttribute("bitsets.{name,inName,bits,tokenTypes,tokenIndex}",
referencedElementName,
enclosingRuleName,
wordStrings,
tokenTypeList,
Utils.integer(elementIndex));
outputFileST.setAttribute("bitsets.{name,inName,bits,tokenTypes,tokenIndex}",
referencedElementName,
enclosingRuleName,
wordStrings,
tokenTypeList,
Utils.integer(elementIndex));
headerFileST.setAttribute("bitsets.{name,inName,bits,tokenTypes,tokenIndex}",
referencedElementName,
enclosingRuleName,
wordStrings,
tokenTypeList,
Utils.integer(elementIndex));
}
// L O O K A H E A D D E C I S I O N G E N E R A T I O N
/** Generate code that computes the predicted alt given a DFA. The
* recognizerST can be either the main generated recognizerTemplate
* for storage in the main parser file or a separate file. It's up to
* the code that ultimately invokes the codegen.g grammar rule.
*
* Regardless, the output file and header file get a copy of the DFAs.
*/
public StringTemplate genLookaheadDecision(StringTemplate recognizerST,
DFA dfa)
{
StringTemplate decisionST;
// If we are doing inline DFA and this one is acyclic and LL(*)
// I have to check for is-non-LL(*) because if non-LL(*) the cyclic
// check is not done by DFA.verify(); that is, verify() avoids
// doesStateReachAcceptState() if non-LL(*)
if ( dfa.canInlineDecision() ) {
decisionST =
acyclicDFAGenerator.genFixedLookaheadDecision(getTemplates(), dfa);
}
else {
// generate any kind of DFA here (cyclic or acyclic)
dfa.createStateTables(this);
outputFileST.setAttribute("cyclicDFAs", dfa);
headerFileST.setAttribute("cyclicDFAs", dfa);
decisionST = templates.getInstanceOf("dfaDecision");
String description = dfa.getNFADecisionStartState().getDescription();
description = target.getTargetStringLiteralFromString(description);
if ( description!=null ) {
decisionST.setAttribute("description", description);
}
decisionST.setAttribute("decisionNumber",
Utils.integer(dfa.getDecisionNumber()));
}
return decisionST;
}
/** A special state is huge (too big for state tables) or has a predicated
* edge. Generate a simple if-then-else. Cannot be an accept state as
* they have no emanating edges. Don't worry about switch vs if-then-else
* because if you get here, the state is super complicated and needs an
* if-then-else. This is used by the new DFA scheme created June 2006.
*/
public StringTemplate generateSpecialState(DFAState s) {
StringTemplate stateST;
stateST = templates.getInstanceOf("cyclicDFAState");
stateST.setAttribute("needErrorClause", Boolean.valueOf(true));
stateST.setAttribute("semPredState",
Boolean.valueOf(s.isResolvedWithPredicates()));
stateST.setAttribute("stateNumber", s.stateNumber);
stateST.setAttribute("decisionNumber", s.dfa.decisionNumber);
boolean foundGatedPred = false;
StringTemplate eotST = null;
for (int i = 0; i < s.getNumberOfTransitions(); i++) {
Transition edge = (Transition) s.transition(i);
StringTemplate edgeST;
if ( edge.label.getAtom()==Label.EOT ) {
// this is the default clause; has to held until last
edgeST = templates.getInstanceOf("eotDFAEdge");
stateST.removeAttribute("needErrorClause");
eotST = edgeST;
}
else {
edgeST = templates.getInstanceOf("cyclicDFAEdge");
StringTemplate exprST =
genLabelExpr(templates,edge,1);
edgeST.setAttribute("labelExpr", exprST);
}
edgeST.setAttribute("edgeNumber", Utils.integer(i+1));
edgeST.setAttribute("targetStateNumber",
Utils.integer(edge.target.stateNumber));
// stick in any gated predicates for any edge if not already a pred
if ( !edge.label.isSemanticPredicate() ) {
DFAState t = (DFAState)edge.target;
SemanticContext preds = t.getGatedPredicatesInNFAConfigurations();
if ( preds!=null ) {
foundGatedPred = true;
StringTemplate predST = preds.genExpr(this,
getTemplates(),
t.dfa);
edgeST.setAttribute("predicates", predST.toString());
}
}
if ( edge.label.getAtom()!=Label.EOT ) {
stateST.setAttribute("edges", edgeST);
}
}
if ( foundGatedPred ) {
// state has >= 1 edge with a gated pred (syn or sem)
// must rewind input first, set flag.
stateST.setAttribute("semPredState", new Boolean(foundGatedPred));
}
if ( eotST!=null ) {
stateST.setAttribute("edges", eotST);
}
return stateST;
}
/** Generate an expression for traversing an edge. */
protected StringTemplate genLabelExpr(StringTemplateGroup templates,
Transition edge,
int k)
{
Label label = edge.label;
if ( label.isSemanticPredicate() ) {
return genSemanticPredicateExpr(templates, edge);
}
if ( label.isSet() ) {
return genSetExpr(templates, label.getSet(), k, true);
}
// must be simple label
StringTemplate eST = templates.getInstanceOf("lookaheadTest");
eST.setAttribute("atom", getTokenTypeAsTargetLabel(label.getAtom()));
eST.setAttribute("atomAsInt", Utils.integer(label.getAtom()));
eST.setAttribute("k", Utils.integer(k));
return eST;
}
protected StringTemplate genSemanticPredicateExpr(StringTemplateGroup templates,
Transition edge)
{
DFA dfa = ((DFAState)edge.target).dfa; // which DFA are we in
Label label = edge.label;
SemanticContext semCtx = label.getSemanticContext();
return semCtx.genExpr(this,templates,dfa);
}
/** For intervals such as [3..3, 30..35], generate an expression that
* tests the lookahead similar to LA(1)==3 || (LA(1)>=30&&LA(1)<=35)
*/
public StringTemplate genSetExpr(StringTemplateGroup templates,
IntSet set,
int k,
boolean partOfDFA)
{
if ( !(set instanceof IntervalSet) ) {
throw new IllegalArgumentException("unable to generate expressions for non IntervalSet objects");
}
IntervalSet iset = (IntervalSet)set;
if ( iset.getIntervals()==null || iset.getIntervals().size()==0 ) {
StringTemplate emptyST = new StringTemplate(templates, "");
emptyST.setName("empty-set-expr");
return emptyST;
}
String testSTName = "lookaheadTest";
String testRangeSTName = "lookaheadRangeTest";
if ( !partOfDFA ) {
testSTName = "isolatedLookaheadTest";
testRangeSTName = "isolatedLookaheadRangeTest";
}
StringTemplate setST = templates.getInstanceOf("setTest");
Iterator iter = iset.getIntervals().iterator();
int rangeNumber = 1;
while (iter.hasNext()) {
Interval I = (Interval) iter.next();
int a = I.a;
int b = I.b;
StringTemplate eST;
if ( a==b ) {
eST = templates.getInstanceOf(testSTName);
eST.setAttribute("atom", getTokenTypeAsTargetLabel(a));
eST.setAttribute("atomAsInt", Utils.integer(a));
//eST.setAttribute("k",Utils.integer(k));
}
else {
eST = templates.getInstanceOf(testRangeSTName);
eST.setAttribute("lower",getTokenTypeAsTargetLabel(a));
eST.setAttribute("lowerAsInt", Utils.integer(a));
eST.setAttribute("upper",getTokenTypeAsTargetLabel(b));
eST.setAttribute("upperAsInt", Utils.integer(b));
eST.setAttribute("rangeNumber",Utils.integer(rangeNumber));
}
eST.setAttribute("k",Utils.integer(k));
setST.setAttribute("ranges", eST);
rangeNumber++;
}
return setST;
}
// T O K E N D E F I N I T I O N G E N E R A T I O N
/** Set attributes tokens and literals attributes in the incoming
* code template. This is not the token vocab interchange file, but
* rather a list of token type ID needed by the recognizer.
*/
protected void genTokenTypeConstants(StringTemplate code) {
// make constants for the token types
Iterator tokenIDs = grammar.getTokenIDs().iterator();
while (tokenIDs.hasNext()) {
String tokenID = (String) tokenIDs.next();
int tokenType = grammar.getTokenType(tokenID);
if ( tokenType==Label.EOF ||
tokenType>=Label.MIN_TOKEN_TYPE )
{
// don't do FAUX labels 'cept EOF
code.setAttribute("tokens.{name,type}", tokenID, Utils.integer(tokenType));
}
}
}
/** Generate a token names table that maps token type to a printable
* name: either the label like INT or the literal like "begin".
*/
protected void genTokenTypeNames(StringTemplate code) {
for (int t=Label.MIN_TOKEN_TYPE; t<=grammar.getMaxTokenType(); t++) {
String tokenName = grammar.getTokenDisplayName(t);
if ( tokenName!=null ) {
tokenName=target.getTargetStringLiteralFromString(tokenName, true);
code.setAttribute("tokenNames", tokenName);
}
}
}
/** Get a meaningful name for a token type useful during code generation.
* Literals without associated names are converted to the string equivalent
* of their integer values. Used to generate x==ID and x==34 type comparisons
* etc... Essentially we are looking for the most obvious way to refer
* to a token type in the generated code. If in the lexer, return the
* char literal translated to the target language. For example, ttype=10
* will yield '\n' from the getTokenDisplayName method. That must
* be converted to the target languages literals. For most C-derived
* languages no translation is needed.
*/
public String getTokenTypeAsTargetLabel(int ttype) {
if ( grammar.type==Grammar.LEXER ) {
String name = grammar.getTokenDisplayName(ttype);
return target.getTargetCharLiteralFromANTLRCharLiteral(this,name);
}
return target.getTokenTypeAsTargetLabel(this,ttype);
}
/** Generate a token vocab file with all the token names/types. For example:
* ID=7
* FOR=8
* 'for'=8
*
* This is independent of the target language; used by antlr internally
*/
protected StringTemplate genTokenVocabOutput() {
StringTemplate vocabFileST =
new StringTemplate(vocabFilePattern,
AngleBracketTemplateLexer.class);
vocabFileST.setName("vocab-file");
// make constants for the token names
Iterator tokenIDs = grammar.getTokenIDs().iterator();
while (tokenIDs.hasNext()) {
String tokenID = (String) tokenIDs.next();
int tokenType = grammar.getTokenType(tokenID);
if ( tokenType>=Label.MIN_TOKEN_TYPE ) {
vocabFileST.setAttribute("tokens.{name,type}", tokenID, Utils.integer(tokenType));
}
}
// now dump the strings
Iterator literals = grammar.getStringLiterals().iterator();
while (literals.hasNext()) {
String literal = (String) literals.next();
int tokenType = grammar.getTokenType(literal);
if ( tokenType>=Label.MIN_TOKEN_TYPE ) {
vocabFileST.setAttribute("tokens.{name,type}", literal, Utils.integer(tokenType));
}
}
return vocabFileST;
}
public List translateAction(String ruleName,
GrammarAST actionTree)
{
if ( actionTree.getType()==ANTLRParser.ARG_ACTION ) {
return translateArgAction(ruleName, actionTree);
}
ActionTranslator translator = new ActionTranslator(this,ruleName,actionTree);
List chunks = translator.translateToChunks();
chunks = target.postProcessAction(chunks, actionTree.token);
return chunks;
}
/** Translate an action like [3,"foo",a[3]] and return a List of the
* translated actions. Because actions are themselves translated to a list
* of chunks, must cat together into a StringTemplate>. Don't translate
* to strings early as we need to eval templates in context.
*/
public List<StringTemplate> translateArgAction(String ruleName,
GrammarAST actionTree)
{
String actionText = actionTree.token.getText();
List<String> args = getListOfArgumentsFromAction(actionText,',');
List<StringTemplate> translatedArgs = new ArrayList<StringTemplate>();
for (String arg : args) {
if ( arg!=null ) {
antlr.Token actionToken =
new antlr.CommonToken(ANTLRParser.ACTION,arg);
ActionTranslator translator =
new ActionTranslator(this,ruleName,
actionToken,
actionTree.outerAltNum);
List chunks = translator.translateToChunks();
chunks = target.postProcessAction(chunks, actionToken);
StringTemplate catST = new StringTemplate(templates, "<chunks>");
catST.setAttribute("chunks", chunks);
templates.createStringTemplate();
translatedArgs.add(catST);
}
}
if ( translatedArgs.size()==0 ) {
return null;
}
return translatedArgs;
}
public static List<String> getListOfArgumentsFromAction(String actionText,
int separatorChar)
{
List<String> args = new ArrayList<String>();
getListOfArgumentsFromAction(actionText, 0, -1, separatorChar, args);
return args;
}
/** Given an arg action like
*
* [x, (*a).foo(21,33), 3.2+1, '\n',
* "a,oo\nick", {bl, "fdkj"eck}, ["cat\n,", x, 43]]
*
* convert to a list of arguments. Allow nested square brackets etc...
* Set separatorChar to ';' or ',' or whatever you want.
*/
public static int getListOfArgumentsFromAction(String actionText,
int start,
int targetChar,
int separatorChar,
List<String> args)
{
if ( actionText==null ) {
return -1;
}
actionText = actionText.replaceAll("//.*\n", "");
int n = actionText.length();
//System.out.println("actionText@"+start+"->"+(char)targetChar+"="+actionText.substring(start,n));
int p = start;
int last = p;
while ( p<n && actionText.charAt(p)!=targetChar ) {
int c = actionText.charAt(p);
switch ( c ) {
case '\'' :
p++;
while ( p<n && actionText.charAt(p)!='\'' ) {
if ( actionText.charAt(p)=='\\' && (p+1)<n &&
actionText.charAt(p+1)=='\'' )
{
p++; // skip escaped quote
}
p++;
}
p++;
break;
case '"' :
p++;
while ( p<n && actionText.charAt(p)!='\"' ) {
if ( actionText.charAt(p)=='\\' && (p+1)<n &&
actionText.charAt(p+1)=='\"' )
{
p++; // skip escaped quote
}
p++;
}
p++;
break;
case '(' :
p = getListOfArgumentsFromAction(actionText,p+1,')',separatorChar,args);
break;
case '{' :
p = getListOfArgumentsFromAction(actionText,p+1,'}',separatorChar,args);
break;
case '<' :
if ( actionText.indexOf('>',p+1)>=p ) {
// do we see a matching '>' ahead? if so, hope it's a generic
// and not less followed by expr with greater than
p = getListOfArgumentsFromAction(actionText,p+1,'>',separatorChar,args);
}
else {
p++; // treat as normal char
}
break;
case '[' :
p = getListOfArgumentsFromAction(actionText,p+1,']',separatorChar,args);
break;
default :
if ( c==separatorChar && targetChar==-1 ) {
String arg = actionText.substring(last, p);
//System.out.println("arg="+arg);
args.add(arg.trim());
last = p+1;
}
p++;
break;
}
}
if ( targetChar==-1 && p<=n ) {
String arg = actionText.substring(last, p).trim();
//System.out.println("arg="+arg);
if ( arg.length()>0 ) {
args.add(arg.trim());
}
}
p++;
return p;
}
/** Given a template constructor action like %foo(a={...}) in
* an action, translate it to the appropriate template constructor
* from the templateLib. This translates a *piece* of the action.
*/
public StringTemplate translateTemplateConstructor(String ruleName,
int outerAltNum,
antlr.Token actionToken,
String templateActionText)
{
// first, parse with antlr.g
//System.out.println("translate template: "+templateActionText);
org.antlr.grammar.v2.ANTLRLexer lexer = new org.antlr.grammar.v2.ANTLRLexer(new StringReader(templateActionText));
lexer.setFilename(grammar.getFileName());
lexer.setTokenObjectClass("antlr.TokenWithIndex");
TokenStreamRewriteEngine tokenBuffer = new TokenStreamRewriteEngine(lexer);
tokenBuffer.discard(ANTLRParser.WS);
tokenBuffer.discard(ANTLRParser.ML_COMMENT);
tokenBuffer.discard(ANTLRParser.COMMENT);
tokenBuffer.discard(ANTLRParser.SL_COMMENT);
ANTLRParser parser = new ANTLRParser(tokenBuffer);
parser.setFilename(grammar.getFileName());
parser.setASTNodeClass("org.antlr.tool.GrammarAST");
try {
parser.rewrite_template();
}
catch (RecognitionException re) {
ErrorManager.grammarError(ErrorManager.MSG_INVALID_TEMPLATE_ACTION,
grammar,
actionToken,
templateActionText);
}
catch (Exception tse) {
ErrorManager.internalError("can't parse template action",tse);
}
GrammarAST rewriteTree = (GrammarAST)parser.getAST();
// then translate via codegen.g
CodeGenTreeWalker gen = new CodeGenTreeWalker();
gen.init(grammar);
gen.setCurrentRuleName(ruleName);
gen.setOuterAltNum(outerAltNum);
StringTemplate st = null;
try {
st = gen.rewrite_template((AST)rewriteTree);
}
catch (RecognitionException re) {
ErrorManager.error(ErrorManager.MSG_BAD_AST_STRUCTURE,
re);
}
return st;
}
public void issueInvalidScopeError(String x,
String y,
Rule enclosingRule,
antlr.Token actionToken,
int outerAltNum)
{
//System.out.println("error $"+x+"::"+y);
Rule r = grammar.getRule(x);
AttributeScope scope = grammar.getGlobalScope(x);
if ( scope==null ) {
if ( r!=null ) {
scope = r.ruleScope; // if not global, might be rule scope
}
}
if ( scope==null ) {
ErrorManager.grammarError(ErrorManager.MSG_UNKNOWN_DYNAMIC_SCOPE,
grammar,
actionToken,
x);
}
else if ( scope.getAttribute(y)==null ) {
ErrorManager.grammarError(ErrorManager.MSG_UNKNOWN_DYNAMIC_SCOPE_ATTRIBUTE,
grammar,
actionToken,
x,
y);
}
}
public void issueInvalidAttributeError(String x,
String y,
Rule enclosingRule,
antlr.Token actionToken,
int outerAltNum)
{
//System.out.println("error $"+x+"."+y);
if ( enclosingRule==null ) {
// action not in a rule
ErrorManager.grammarError(ErrorManager.MSG_ATTRIBUTE_REF_NOT_IN_RULE,
grammar,
actionToken,
x,
y);
return;
}
// action is in a rule
Grammar.LabelElementPair label = enclosingRule.getRuleLabel(x);
if ( label!=null || enclosingRule.getRuleRefsInAlt(x, outerAltNum)!=null ) {
// $rulelabel.attr or $ruleref.attr; must be unknown attr
String refdRuleName = x;
if ( label!=null ) {
refdRuleName = enclosingRule.getRuleLabel(x).referencedRuleName;
}
Rule refdRule = grammar.getRule(refdRuleName);
AttributeScope scope = refdRule.getAttributeScope(y);
if ( scope==null ) {
ErrorManager.grammarError(ErrorManager.MSG_UNKNOWN_RULE_ATTRIBUTE,
grammar,
actionToken,
refdRuleName,
y);
}
else if ( scope.isParameterScope ) {
ErrorManager.grammarError(ErrorManager.MSG_INVALID_RULE_PARAMETER_REF,
grammar,
actionToken,
refdRuleName,
y);
}
else if ( scope.isDynamicRuleScope ) {
ErrorManager.grammarError(ErrorManager.MSG_INVALID_RULE_SCOPE_ATTRIBUTE_REF,
grammar,
actionToken,
refdRuleName,
y);
}
}
}
public void issueInvalidAttributeError(String x,
Rule enclosingRule,
antlr.Token actionToken,
int outerAltNum)
{
//System.out.println("error $"+x);
if ( enclosingRule==null ) {
// action not in a rule
ErrorManager.grammarError(ErrorManager.MSG_ATTRIBUTE_REF_NOT_IN_RULE,
grammar,
actionToken,
x);
return;
}
// action is in a rule
Grammar.LabelElementPair label = enclosingRule.getRuleLabel(x);
AttributeScope scope = enclosingRule.getAttributeScope(x);
if ( label!=null ||
enclosingRule.getRuleRefsInAlt(x, outerAltNum)!=null ||
enclosingRule.name.equals(x) )
{
ErrorManager.grammarError(ErrorManager.MSG_ISOLATED_RULE_SCOPE,
grammar,
actionToken,
x);
}
else if ( scope!=null && scope.isDynamicRuleScope ) {
ErrorManager.grammarError(ErrorManager.MSG_ISOLATED_RULE_ATTRIBUTE,
grammar,
actionToken,
x);
}
else {
ErrorManager.grammarError(ErrorManager.MSG_UNKNOWN_SIMPLE_ATTRIBUTE,
grammar,
actionToken,
x);
}
}
// M I S C
public StringTemplateGroup getTemplates() {
return templates;
}
public StringTemplateGroup getBaseTemplates() {
return baseTemplates;
}
public void setDebug(boolean debug) {
this.debug = debug;
}
public void setTrace(boolean trace) {
this.trace = trace;
}
public void setProfile(boolean profile) {
this.profile = profile;
if ( profile ) {
setDebug(true); // requires debug events
}
}
public StringTemplate getRecognizerST() {
return outputFileST;
}
/** Generate TParser.java and TLexer.java from T.g if combined, else
* just use T.java as output regardless of type.
*/
public String getRecognizerFileName(String name, int type) {
StringTemplate extST = templates.getInstanceOf("codeFileExtension");
String recognizerName = grammar.getRecognizerName();
return recognizerName+extST.toString();
/*
String suffix = "";
if ( type==Grammar.COMBINED ||
(type==Grammar.LEXER && !grammar.implicitLexer) )
{
suffix = Grammar.grammarTypeToFileNameSuffix[type];
}
return name+suffix+extST.toString();
*/
}
/** What is the name of the vocab file generated for this grammar?
* Returns null if no .tokens file should be generated.
*/
public String getVocabFileName() {
if ( grammar.isBuiltFromString() ) {
return null;
}
return grammar.name+VOCAB_FILE_EXTENSION;
}
public void write(StringTemplate code, String fileName) throws IOException {
long start = System.currentTimeMillis();
Writer w = tool.getOutputFile(grammar, fileName);
// Write the output to a StringWriter
StringTemplateWriter wr = templates.getStringTemplateWriter(w);
wr.setLineWidth(lineWidth);
code.write(wr);
w.close();
long stop = System.currentTimeMillis();
//System.out.println("render time for "+fileName+": "+(int)(stop-start)+"ms");
}
/** You can generate a switch rather than if-then-else for a DFA state
* if there are no semantic predicates and the number of edge label
* values is small enough; e.g., don't generate a switch for a state
* containing an edge label such as 20..52330 (the resulting byte codes
* would overflow the method 65k limit probably).
*/
protected boolean canGenerateSwitch(DFAState s) {
if ( !GENERATE_SWITCHES_WHEN_POSSIBLE ) {
return false;
}
int size = 0;
for (int i = 0; i < s.getNumberOfTransitions(); i++) {
Transition edge = (Transition) s.transition(i);
if ( edge.label.isSemanticPredicate() ) {
return false;
}
// can't do a switch if the edges are going to require predicates
if ( edge.label.getAtom()==Label.EOT ) {
int EOTPredicts = ((DFAState)edge.target).getUniquelyPredictedAlt();
if ( EOTPredicts==NFA.INVALID_ALT_NUMBER ) {
// EOT target has to be a predicate then; no unique alt
return false;
}
}
// if target is a state with gated preds, we need to use preds on
// this edge then to reach it.
if ( ((DFAState)edge.target).getGatedPredicatesInNFAConfigurations()!=null ) {
return false;
}
size += edge.label.getSet().size();
}
if ( s.getNumberOfTransitions()<MIN_SWITCH_ALTS ||
size>MAX_SWITCH_CASE_LABELS ) {
return false;
}
return true;
}
/** Create a label to track a token / rule reference's result.
* Technically, this is a place where I break model-view separation
* as I am creating a variable name that could be invalid in a
* target language, however, label ::= <ID><INT> is probably ok in
* all languages we care about.
*/
public String createUniqueLabel(String name) {
return new StringBuffer()
.append(name).append(uniqueLabelNumber++).toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/JavaScriptTarget.java 0000644 0001750 0001750 00000003635 11256465230 025453 0 ustar twerner twerner package org.antlr.codegen;
import java.util.*;
public class JavaScriptTarget extends Target {
/** Convert an int to a JavaScript Unicode character literal.
*
* The current JavaScript spec (ECMA-262) doesn't provide for octal
* notation in String literals, although some implementations support it.
* This method overrides the parent class so that characters will always
* be encoded as Unicode literals (e.g. \u0011).
*/
public String encodeIntAsCharEscape(int v) {
String hex = Integer.toHexString(v|0x10000).substring(1,5);
return "\\u"+hex;
}
/** Convert long to two 32-bit numbers separted by a comma.
* JavaScript does not support 64-bit numbers, so we need to break
* the number into two 32-bit literals to give to the Bit. A number like
* 0xHHHHHHHHLLLLLLLL is broken into the following string:
* "0xLLLLLLLL, 0xHHHHHHHH"
* Note that the low order bits are first, followed by the high order bits.
* This is to match how the BitSet constructor works, where the bits are
* passed in in 32-bit chunks with low-order bits coming first.
*
* Note: stole the following two methods from the ActionScript target.
*/
public String getTarget64BitStringFromValue(long word) {
StringBuffer buf = new StringBuffer(22); // enough for the two "0x", "," and " "
buf.append("0x");
writeHexWithPadding(buf, Integer.toHexString((int)(word & 0x00000000ffffffffL)));
buf.append(", 0x");
writeHexWithPadding(buf, Integer.toHexString((int)(word >> 32)));
return buf.toString();
}
private void writeHexWithPadding(StringBuffer buf, String digits) {
digits = digits.toUpperCase();
int padding = 8 - digits.length();
// pad left with zeros
for (int i=1; i<=padding; i++) {
buf.append('0');
}
buf.append(digits);
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/JavaTarget.java 0000644 0001750 0001750 00000003511 11256465230 024257 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.Tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
public class JavaTarget extends Target {
protected StringTemplate chooseWhereCyclicDFAsGo(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate recognizerST,
StringTemplate cyclicDFAST)
{
return recognizerST;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/Target.java 0000644 0001750 0001750 00000030257 11256465230 023464 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.Tool;
import org.antlr.analysis.Label;
import org.antlr.misc.Utils;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
import java.io.IOException;
import java.util.List;
/** The code generator for ANTLR can usually be retargeted just by providing
* a new X.stg file for language X, however, sometimes the files that must
* be generated vary enough that some X-specific functionality is required.
* For example, in C, you must generate header files whereas in Java you do not.
* Other languages may want to keep DFA separate from the main
* generated recognizer file.
*
* The notion of a Code Generator target abstracts out the creation
* of the various files. As new language targets get added to the ANTLR
* system, this target class may have to be altered to handle more
* functionality. Eventually, just about all language generation issues
* will be expressible in terms of these methods.
*
* If org.antlr.codegen.XTarget class exists, it is used else
* Target base class is used. I am using a superclass rather than an
* interface for this target concept because I can add functionality
* later without breaking previously written targets (extra interface
* methods would force adding dummy functions to all code generator
* target classes).
*
*/
public class Target {
/** For pure strings of Java 16-bit unicode char, how can we display
* it in the target language as a literal. Useful for dumping
* predicates and such that may refer to chars that need to be escaped
* when represented as strings. Also, templates need to be escaped so
* that the target language can hold them as a string.
*
* I have defined (via the constructor) the set of typical escapes,
* but your Target subclass is free to alter the translated chars or
* add more definitions. This is nonstatic so each target can have
* a different set in memory at same time.
*/
protected String[] targetCharValueEscape = new String[255];
public Target() {
targetCharValueEscape['\n'] = "\\n";
targetCharValueEscape['\r'] = "\\r";
targetCharValueEscape['\t'] = "\\t";
targetCharValueEscape['\b'] = "\\b";
targetCharValueEscape['\f'] = "\\f";
targetCharValueEscape['\\'] = "\\\\";
targetCharValueEscape['\''] = "\\'";
targetCharValueEscape['"'] = "\\\"";
}
protected void genRecognizerFile(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate outputFileST)
throws IOException
{
String fileName =
generator.getRecognizerFileName(grammar.name, grammar.type);
generator.write(outputFileST, fileName);
}
protected void genRecognizerHeaderFile(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate headerFileST,
String extName) // e.g., ".h"
throws IOException
{
// no header file by default
}
protected void performGrammarAnalysis(CodeGenerator generator,
Grammar grammar)
{
// Build NFAs from the grammar AST
grammar.buildNFA();
// Create the DFA predictors for each decision
grammar.createLookaheadDFAs();
}
/** Is scope in @scope::name {action} valid for this kind of grammar?
* Targets like C++ may want to allow new scopes like headerfile or
* some such. The action names themselves are not policed at the
* moment so targets can add template actions w/o having to recompile
* ANTLR.
*/
public boolean isValidActionScope(int grammarType, String scope) {
switch (grammarType) {
case Grammar.LEXER :
if ( scope.equals("lexer") ) {return true;}
break;
case Grammar.PARSER :
if ( scope.equals("parser") ) {return true;}
break;
case Grammar.COMBINED :
if ( scope.equals("parser") ) {return true;}
if ( scope.equals("lexer") ) {return true;}
break;
case Grammar.TREE_PARSER :
if ( scope.equals("treeparser") ) {return true;}
break;
}
return false;
}
/** Target must be able to override the labels used for token types */
public String getTokenTypeAsTargetLabel(CodeGenerator generator, int ttype) {
String name = generator.grammar.getTokenDisplayName(ttype);
// If name is a literal, return the token type instead
if ( name.charAt(0)=='\'' ) {
return String.valueOf(ttype);
}
return name;
}
/** Convert from an ANTLR char literal found in a grammar file to
* an equivalent char literal in the target language. For most
* languages, this means leaving 'x' as 'x'. Actually, we need
* to escape '\u000A' so that it doesn't get converted to \n by
* the compiler. Convert the literal to the char value and then
* to an appropriate target char literal.
*
* Expect single quotes around the incoming literal.
*/
public String getTargetCharLiteralFromANTLRCharLiteral(
CodeGenerator generator,
String literal)
{
StringBuffer buf = new StringBuffer();
buf.append('\'');
int c = Grammar.getCharValueFromGrammarCharLiteral(literal);
if ( c<Label.MIN_CHAR_VALUE ) {
return "'\u0000'";
}
if ( c<targetCharValueEscape.length &&
targetCharValueEscape[c]!=null )
{
buf.append(targetCharValueEscape[c]);
}
else if ( Character.UnicodeBlock.of((char)c)==
Character.UnicodeBlock.BASIC_LATIN &&
!Character.isISOControl((char)c) )
{
// normal char
buf.append((char)c);
}
else {
// must be something unprintable...use \\uXXXX
// turn on the bit above max "\\uFFFF" value so that we pad with zeros
// then only take last 4 digits
String hex = Integer.toHexString(c|0x10000).toUpperCase().substring(1,5);
buf.append("\\u");
buf.append(hex);
}
buf.append('\'');
return buf.toString();
}
/** Convert from an ANTLR string literal found in a grammar file to
* an equivalent string literal in the target language. For Java, this
* is the translation 'a\n"' -> "a\n\"". Expect single quotes
* around the incoming literal. Just flip the quotes and replace
* double quotes with \"
*
* Note that we have decided to allow poeple to use '\"' without
* penalty, so we must build the target string in a loop as Utils.replae
* cannot handle both \" and " without a lot of messing around.
*
*/
public String getTargetStringLiteralFromANTLRStringLiteral(
CodeGenerator generator,
String literal)
{
StringBuilder sb = new StringBuilder();
StringBuffer is = new StringBuffer(literal);
// Opening quote
//
sb.append('"');
for (int i = 1; i < is.length() -1; i++) {
if (is.charAt(i) == '\\') {
// Anything escaped is what it is! We assume that
// people know how to escape characters correctly. However
// we catch anything that does not need an escape in Java (which
// is what the default implementation is dealing with and remove
// the escape. The C target does this for instance.
//
switch (is.charAt(i+1)) {
// Pass through any escapes that Java also needs
//
case '"':
case 'n':
case 'r':
case 't':
case 'b':
case 'f':
case '\\':
case 'u': // Assume unnnn
sb.append('\\'); // Pass the escape through
break;
default:
// Remove the escape by virtue of not adding it here
// Thus \' becomes ' and so on
//
break;
}
// Go past the \ character
//
i++;
} else {
// Chracters that don't need \ in ANTLR 'strings' but do in Java
//
if (is.charAt(i) == '"') {
// We need to escape " in Java
//
sb.append('\\');
}
}
// Add in the next character, which may have been escaped
//
sb.append(is.charAt(i));
}
// Append closing " and return
//
sb.append('"');
return sb.toString();
}
/** Given a random string of Java unicode chars, return a new string with
* optionally appropriate quote characters for target language and possibly
* with some escaped characters. For example, if the incoming string has
* actual newline characters, the output of this method would convert them
* to the two char sequence \n for Java, C, C++, ... The new string has
* double-quotes around it as well. Example String in memory:
*
* a"[newlinechar]b'c[carriagereturnchar]d[tab]e\f
*
* would be converted to the valid Java s:
*
* "a\"\nb'c\rd\te\\f"
*
* or
*
* a\"\nb'c\rd\te\\f
*
* depending on the quoted arg.
*/
public String getTargetStringLiteralFromString(String s, boolean quoted) {
if ( s==null ) {
return null;
}
StringBuffer buf = new StringBuffer();
if ( quoted ) {
buf.append('"');
}
for (int i=0; i<s.length(); i++) {
int c = s.charAt(i);
if ( c!='\'' && // don't escape single quotes in strings for java
c<targetCharValueEscape.length &&
targetCharValueEscape[c]!=null )
{
buf.append(targetCharValueEscape[c]);
}
else {
buf.append((char)c);
}
}
if ( quoted ) {
buf.append('"');
}
return buf.toString();
}
public String getTargetStringLiteralFromString(String s) {
return getTargetStringLiteralFromString(s, false);
}
/** Convert long to 0xNNNNNNNNNNNNNNNN by default for spitting out
* with bitsets. I.e., convert bytes to hex string.
*/
public String getTarget64BitStringFromValue(long word) {
int numHexDigits = 8*2;
StringBuffer buf = new StringBuffer(numHexDigits+2);
buf.append("0x");
String digits = Long.toHexString(word);
digits = digits.toUpperCase();
int padding = numHexDigits - digits.length();
// pad left with zeros
for (int i=1; i<=padding; i++) {
buf.append('0');
}
buf.append(digits);
return buf.toString();
}
public String encodeIntAsCharEscape(int v) {
if ( v<=127 ) {
return "\\"+Integer.toOctalString(v);
}
String hex = Integer.toHexString(v|0x10000).substring(1,5);
return "\\u"+hex;
}
/** Some targets only support ASCII or 8-bit chars/strings. For example,
* C++ will probably want to return 0xFF here.
*/
public int getMaxCharValue(CodeGenerator generator) {
return Label.MAX_CHAR_VALUE;
}
/** Give target a chance to do some postprocessing on actions.
* Python for example will have to fix the indention.
*/
public List postProcessAction(List chunks, antlr.Token actionToken) {
return chunks;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/CPPTarget.java 0000644 0001750 0001750 00000011656 11256465230 024031 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.tool.Grammar;
import org.antlr.Tool;
import java.io.IOException;
public class CPPTarget extends Target {
public String escapeChar( int c ) {
// System.out.println("CPPTarget.escapeChar("+c+")");
switch (c) {
case '\n' : return "\\n";
case '\t' : return "\\t";
case '\r' : return "\\r";
case '\\' : return "\\\\";
case '\'' : return "\\'";
case '"' : return "\\\"";
default :
if ( c < ' ' || c > 126 )
{
if (c > 255)
{
String s = Integer.toString(c,16);
// put leading zeroes in front of the thing..
while( s.length() < 4 )
s = '0' + s;
return "\\u" + s;
}
else {
return "\\" + Integer.toString(c,8);
}
}
else {
return String.valueOf((char)c);
}
}
}
/** Converts a String into a representation that can be use as a literal
* when surrounded by double-quotes.
*
* Used for escaping semantic predicate strings for exceptions.
*
* @param s The String to be changed into a literal
*/
public String escapeString(String s)
{
StringBuffer retval = new StringBuffer();
for (int i = 0; i < s.length(); i++) {
retval.append(escapeChar(s.charAt(i)));
}
return retval.toString();
}
protected void genRecognizerHeaderFile(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate headerFileST,
String extName)
throws IOException
{
StringTemplateGroup templates = generator.getTemplates();
generator.write(headerFileST, grammar.name+extName);
}
/** Convert from an ANTLR char literal found in a grammar file to
* an equivalent char literal in the target language. For Java, this
* is the identify translation; i.e., '\n' -> '\n'. Most languages
* will be able to use this 1-to-1 mapping. Expect single quotes
* around the incoming literal.
* Depending on the charvocabulary the charliteral should be prefixed with a 'L'
*/
public String getTargetCharLiteralFromANTLRCharLiteral( CodeGenerator codegen, String literal) {
int c = Grammar.getCharValueFromGrammarCharLiteral(literal);
String prefix = "'";
if( codegen.grammar.getMaxCharValue() > 255 )
prefix = "L'";
else if( (c & 0x80) != 0 ) // if in char mode prevent sign extensions
return ""+c;
return prefix+escapeChar(c)+"'";
}
/** Convert from an ANTLR string literal found in a grammar file to
* an equivalent string literal in the target language. For Java, this
* is the identify translation; i.e., "\"\n" -> "\"\n". Most languages
* will be able to use this 1-to-1 mapping. Expect double quotes
* around the incoming literal.
* Depending on the charvocabulary the string should be prefixed with a 'L'
*/
public String getTargetStringLiteralFromANTLRStringLiteral( CodeGenerator codegen, String literal) {
StringBuffer buf = Grammar.getUnescapedStringFromGrammarStringLiteral(literal);
String prefix = "\"";
if( codegen.grammar.getMaxCharValue() > 255 )
prefix = "L\"";
return prefix+escapeString(buf.toString())+"\"";
}
/** Character constants get truncated to this value.
* TODO: This should be derived from the charVocabulary. Depending on it
* being 255 or 0xFFFF the templates should generate normal character
* constants or multibyte ones.
*/
public int getMaxCharValue( CodeGenerator codegen ) {
int maxval = 255; // codegen.grammar.get????();
if ( maxval <= 255 )
return 255;
else
return maxval;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/ObjCTarget.java 0000644 0001750 0001750 00000010341 11256465230 024212 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005 Terence Parr
Copyright (c) 2006 Kay Roepke (Objective-C runtime)
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
import org.antlr.Tool;
import org.antlr.misc.Utils;
import java.io.IOException;
public class ObjCTarget extends Target {
protected void genRecognizerHeaderFile(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate headerFileST,
String extName)
throws IOException
{
generator.write(headerFileST, grammar.name + Grammar.grammarTypeToFileNameSuffix[grammar.type] + extName);
}
public String getTargetCharLiteralFromANTLRCharLiteral(CodeGenerator generator,
String literal)
{
if (literal.startsWith("'\\u") ) {
literal = "0x" +literal.substring(3, 7);
} else {
int c = literal.charAt(1); // TJP
if (c < 32 || c > 127) {
literal = "0x" + Integer.toHexString(c);
}
}
return literal;
}
/** Convert from an ANTLR string literal found in a grammar file to
* an equivalent string literal in the target language. For Java, this
* is the translation 'a\n"' -> "a\n\"". Expect single quotes
* around the incoming literal. Just flip the quotes and replace
* double quotes with \"
*/
public String getTargetStringLiteralFromANTLRStringLiteral(CodeGenerator generator,
String literal)
{
literal = Utils.replace(literal,"\"","\\\"");
StringBuffer buf = new StringBuffer(literal);
buf.setCharAt(0,'"');
buf.setCharAt(literal.length()-1,'"');
buf.insert(0,'@');
return buf.toString();
}
/** If we have a label, prefix it with the recognizer's name */
public String getTokenTypeAsTargetLabel(CodeGenerator generator, int ttype) {
String name = generator.grammar.getTokenDisplayName(ttype);
// If name is a literal, return the token type instead
if ( name.charAt(0)=='\'' ) {
return String.valueOf(ttype);
}
return generator.grammar.name + Grammar.grammarTypeToFileNameSuffix[generator.grammar.type] + "_" + name;
//return super.getTokenTypeAsTargetLabel(generator, ttype);
//return this.getTokenTextAndTypeAsTargetLabel(generator, null, ttype);
}
/** Target must be able to override the labels used for token types. Sometimes also depends on the token text.*/
public String getTokenTextAndTypeAsTargetLabel(CodeGenerator generator, String text, int tokenType) {
String name = generator.grammar.getTokenDisplayName(tokenType);
// If name is a literal, return the token type instead
if ( name.charAt(0)=='\'' ) {
return String.valueOf(tokenType);
}
String textEquivalent = text == null ? name : text;
if (textEquivalent.charAt(0) >= '0' && textEquivalent.charAt(0) <= '9') {
return textEquivalent;
} else {
return generator.grammar.name + Grammar.grammarTypeToFileNameSuffix[generator.grammar.type] + "_" + textEquivalent;
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/ACyclicDFACodeGenerator.java 0000644 0001750 0001750 00000016761 11256465230 026526 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.analysis.*;
import org.antlr.misc.Utils;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import java.util.List;
public class ACyclicDFACodeGenerator {
protected CodeGenerator parentGenerator;
public ACyclicDFACodeGenerator(CodeGenerator parent) {
this.parentGenerator = parent;
}
public StringTemplate genFixedLookaheadDecision(StringTemplateGroup templates,
DFA dfa)
{
return walkFixedDFAGeneratingStateMachine(templates, dfa, dfa.startState, 1);
}
protected StringTemplate walkFixedDFAGeneratingStateMachine(
StringTemplateGroup templates,
DFA dfa,
DFAState s,
int k)
{
//System.out.println("walk "+s.stateNumber+" in dfa for decision "+dfa.decisionNumber);
if ( s.isAcceptState() ) {
StringTemplate dfaST = templates.getInstanceOf("dfaAcceptState");
dfaST.setAttribute("alt", Utils.integer(s.getUniquelyPredictedAlt()));
return dfaST;
}
// the default templates for generating a state and its edges
// can be an if-then-else structure or a switch
String dfaStateName = "dfaState";
String dfaLoopbackStateName = "dfaLoopbackState";
String dfaOptionalBlockStateName = "dfaOptionalBlockState";
String dfaEdgeName = "dfaEdge";
if ( parentGenerator.canGenerateSwitch(s) ) {
dfaStateName = "dfaStateSwitch";
dfaLoopbackStateName = "dfaLoopbackStateSwitch";
dfaOptionalBlockStateName = "dfaOptionalBlockStateSwitch";
dfaEdgeName = "dfaEdgeSwitch";
}
StringTemplate dfaST = templates.getInstanceOf(dfaStateName);
if ( dfa.getNFADecisionStartState().decisionStateType==NFAState.LOOPBACK ) {
dfaST = templates.getInstanceOf(dfaLoopbackStateName);
}
else if ( dfa.getNFADecisionStartState().decisionStateType==NFAState.OPTIONAL_BLOCK_START ) {
dfaST = templates.getInstanceOf(dfaOptionalBlockStateName);
}
dfaST.setAttribute("k", Utils.integer(k));
dfaST.setAttribute("stateNumber", Utils.integer(s.stateNumber));
dfaST.setAttribute("semPredState",
Boolean.valueOf(s.isResolvedWithPredicates()));
/*
String description = dfa.getNFADecisionStartState().getDescription();
description = parentGenerator.target.getTargetStringLiteralFromString(description);
//System.out.println("DFA: "+description+" associated with AST "+dfa.getNFADecisionStartState());
if ( description!=null ) {
dfaST.setAttribute("description", description);
}
*/
int EOTPredicts = NFA.INVALID_ALT_NUMBER;
DFAState EOTTarget = null;
//System.out.println("DFA state "+s.stateNumber);
for (int i = 0; i < s.getNumberOfTransitions(); i++) {
Transition edge = (Transition) s.transition(i);
//System.out.println("edge "+s.stateNumber+"-"+edge.label.toString()+"->"+edge.target.stateNumber);
if ( edge.label.getAtom()==Label.EOT ) {
// don't generate a real edge for EOT; track alt EOT predicts
// generate that prediction in the else clause as default case
EOTTarget = (DFAState)edge.target;
EOTPredicts = EOTTarget.getUniquelyPredictedAlt();
/*
System.out.println("DFA s"+s.stateNumber+" EOT goes to s"+
edge.target.stateNumber+" predicates alt "+
EOTPredicts);
*/
continue;
}
StringTemplate edgeST = templates.getInstanceOf(dfaEdgeName);
// If the template wants all the label values delineated, do that
if ( edgeST.getFormalArgument("labels")!=null ) {
List labels = edge.label.getSet().toList();
for (int j = 0; j < labels.size(); j++) {
Integer vI = (Integer) labels.get(j);
String label =
parentGenerator.getTokenTypeAsTargetLabel(vI.intValue());
labels.set(j, label); // rewrite List element to be name
}
edgeST.setAttribute("labels", labels);
}
else { // else create an expression to evaluate (the general case)
edgeST.setAttribute("labelExpr",
parentGenerator.genLabelExpr(templates,edge,k));
}
// stick in any gated predicates for any edge if not already a pred
if ( !edge.label.isSemanticPredicate() ) {
DFAState target = (DFAState)edge.target;
SemanticContext preds =
target.getGatedPredicatesInNFAConfigurations();
if ( preds!=null ) {
//System.out.println("preds="+target.getGatedPredicatesInNFAConfigurations());
StringTemplate predST = preds.genExpr(parentGenerator,
parentGenerator.getTemplates(),
dfa);
edgeST.setAttribute("predicates", predST);
}
}
StringTemplate targetST =
walkFixedDFAGeneratingStateMachine(templates,
dfa,
(DFAState)edge.target,
k+1);
edgeST.setAttribute("targetState", targetST);
dfaST.setAttribute("edges", edgeST);
/*
System.out.println("back to DFA "+
dfa.decisionNumber+"."+s.stateNumber);
*/
}
// HANDLE EOT EDGE
if ( EOTPredicts!=NFA.INVALID_ALT_NUMBER ) {
// EOT unique predicts an alt
dfaST.setAttribute("eotPredictsAlt", Utils.integer(EOTPredicts));
}
else if ( EOTTarget!=null && EOTTarget.getNumberOfTransitions()>0 ) {
// EOT state has transitions so must split on predicates.
// Generate predicate else-if clauses and then generate
// NoViableAlt exception as else clause.
// Note: these predicates emanate from the EOT target state
// rather than the current DFAState s so the error message
// might be slightly misleading if you are looking at the
// state number. Predicates emanating from EOT targets are
// hoisted up to the state that has the EOT edge.
for (int i = 0; i < EOTTarget.getNumberOfTransitions(); i++) {
Transition predEdge = (Transition)EOTTarget.transition(i);
StringTemplate edgeST = templates.getInstanceOf(dfaEdgeName);
edgeST.setAttribute("labelExpr",
parentGenerator.genSemanticPredicateExpr(templates,predEdge));
// the target must be an accept state
//System.out.println("EOT edge");
StringTemplate targetST =
walkFixedDFAGeneratingStateMachine(templates,
dfa,
(DFAState)predEdge.target,
k+1);
edgeST.setAttribute("targetState", targetST);
dfaST.setAttribute("edges", edgeST);
}
}
return dfaST;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/ActionScriptTarget.java 0000644 0001750 0001750 00000013016 11256465230 026001 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.Tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
public class ActionScriptTarget extends Target {
public String getTargetCharLiteralFromANTLRCharLiteral(
CodeGenerator generator,
String literal) {
int c = Grammar.getCharValueFromGrammarCharLiteral(literal);
return String.valueOf(c);
}
public String getTokenTypeAsTargetLabel(CodeGenerator generator,
int ttype) {
// use ints for predefined types;
// <invalid> <EOR> <DOWN> <UP>
if (ttype >= 0 && ttype <= 3) {
return String.valueOf(ttype);
}
String name = generator.grammar.getTokenDisplayName(ttype);
// If name is a literal, return the token type instead
if (name.charAt(0) == '\'') {
return String.valueOf(ttype);
}
return name;
}
/**
* ActionScript doesn't support Unicode String literals that are considered "illegal"
* or are in the surrogate pair ranges. For example "/uffff" will not encode properly
* nor will "/ud800". To keep things as compact as possible we use the following encoding
* if the int is below 255, we encode as hex literal
* If the int is between 255 and 0x7fff we use a single unicode literal with the value
* If the int is above 0x7fff, we use a unicode literal of 0x80hh, where hh is the high-order
* bits followed by \xll where ll is the lower order bits of a 16-bit number.
*
* Ideally this should be improved at a future date. The most optimal way to encode this
* may be a compressed AMF encoding that is embedded using an Embed tag in ActionScript.
*
* @param v
* @return
*/
public String encodeIntAsCharEscape(int v) {
// encode as hex
if ( v<=255 ) {
return "\\x"+ Integer.toHexString(v|0x100).substring(1,3);
}
if (v <= 0x7fff) {
String hex = Integer.toHexString(v|0x10000).substring(1,5);
return "\\u"+hex;
}
if (v > 0xffff) {
System.err.println("Warning: character literal out of range for ActionScript target " + v);
return "";
}
StringBuffer buf = new StringBuffer("\\u80");
buf.append(Integer.toHexString((v >> 8) | 0x100).substring(1, 3)); // high - order bits
buf.append("\\x");
buf.append(Integer.toHexString((v & 0xff) | 0x100).substring(1, 3)); // low -order bits
return buf.toString();
}
/** Convert long to two 32-bit numbers separted by a comma.
* ActionScript does not support 64-bit numbers, so we need to break
* the number into two 32-bit literals to give to the Bit. A number like
* 0xHHHHHHHHLLLLLLLL is broken into the following string:
* "0xLLLLLLLL, 0xHHHHHHHH"
* Note that the low order bits are first, followed by the high order bits.
* This is to match how the BitSet constructor works, where the bits are
* passed in in 32-bit chunks with low-order bits coming first.
*/
public String getTarget64BitStringFromValue(long word) {
StringBuffer buf = new StringBuffer(22); // enough for the two "0x", "," and " "
buf.append("0x");
writeHexWithPadding(buf, Integer.toHexString((int)(word & 0x00000000ffffffffL)));
buf.append(", 0x");
writeHexWithPadding(buf, Integer.toHexString((int)(word >> 32)));
return buf.toString();
}
private void writeHexWithPadding(StringBuffer buf, String digits) {
digits = digits.toUpperCase();
int padding = 8 - digits.length();
// pad left with zeros
for (int i=1; i<=padding; i++) {
buf.append('0');
}
buf.append(digits);
}
protected StringTemplate chooseWhereCyclicDFAsGo(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate recognizerST,
StringTemplate cyclicDFAST) {
return recognizerST;
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/Perl5Target.java 0000644 0001750 0001750 00000007203 11256465230 024367 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2007 Ronald Blaschke
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.analysis.Label;
import org.antlr.tool.AttributeScope;
import org.antlr.tool.Grammar;
import org.antlr.tool.RuleLabelScope;
public class Perl5Target extends Target {
public Perl5Target() {
targetCharValueEscape['$'] = "\\$";
targetCharValueEscape['@'] = "\\@";
targetCharValueEscape['%'] = "\\%";
AttributeScope.tokenScope.addAttribute("self", null);
RuleLabelScope.predefinedLexerRulePropertiesScope.addAttribute("self", null);
}
public String getTargetCharLiteralFromANTLRCharLiteral(final CodeGenerator generator,
final String literal) {
final StringBuffer buf = new StringBuffer(10);
final int c = Grammar.getCharValueFromGrammarCharLiteral(literal);
if (c < Label.MIN_CHAR_VALUE) {
buf.append("\\x{0000}");
} else if (c < targetCharValueEscape.length &&
targetCharValueEscape[c] != null) {
buf.append(targetCharValueEscape[c]);
} else if (Character.UnicodeBlock.of((char) c) ==
Character.UnicodeBlock.BASIC_LATIN &&
!Character.isISOControl((char) c)) {
// normal char
buf.append((char) c);
} else {
// must be something unprintable...use \\uXXXX
// turn on the bit above max "\\uFFFF" value so that we pad with zeros
// then only take last 4 digits
String hex = Integer.toHexString(c | 0x10000).toUpperCase().substring(1, 5);
buf.append("\\x{");
buf.append(hex);
buf.append("}");
}
if (buf.indexOf("\\") == -1) {
// no need for interpolation, use single quotes
buf.insert(0, '\'');
buf.append('\'');
} else {
// need string interpolation
buf.insert(0, '\"');
buf.append('\"');
}
return buf.toString();
}
public String encodeIntAsCharEscape(final int v) {
final int intValue;
if ((v & 0x8000) == 0) {
intValue = v;
} else {
intValue = -(0x10000 - v);
}
return String.valueOf(intValue);
}
}
antlr-3.2/tool/src/main/java/org/antlr/codegen/CSharpTarget.java 0000644 0001750 0001750 00000004237 11256465230 024564 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2006 Kunle Odutola
Copyright (c) 2005 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.codegen;
import org.antlr.Tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.Grammar;
public class CSharpTarget extends Target
{
protected StringTemplate chooseWhereCyclicDFAsGo(Tool tool,
CodeGenerator generator,
Grammar grammar,
StringTemplate recognizerST,
StringTemplate cyclicDFAST)
{
return recognizerST;
}
public String encodeIntAsCharEscape(int v)
{
if (v <= 127)
{
String hex1 = Integer.toHexString(v | 0x10000).substring(3, 5);
return "\\x" + hex1;
}
String hex = Integer.toHexString(v | 0x10000).substring(1, 5);
return "\\u" + hex;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/ 0000755 0001750 0001750 00000000000 11410174110 021565 5 ustar twerner twerner antlr-3.2/tool/src/main/java/org/antlr/analysis/NFAConfiguration.java 0000644 0001750 0001750 00000013400 11256465230 025600 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.misc.Utils;
/** An NFA state, predicted alt, and syntactic/semantic context.
* The syntactic context is a pointer into the rule invocation
* chain used to arrive at the state. The semantic context is
* the unordered set semantic predicates encountered before reaching
* an NFA state.
*/
public class NFAConfiguration {
/** The NFA state associated with this configuration */
public int state;
/** What alt is predicted by this configuration */
public int alt;
/** What is the stack of rule invocations that got us to state? */
public NFAContext context;
/** The set of semantic predicates associated with this NFA
* configuration. The predicates were found on the way to
* the associated NFA state in this syntactic context.
* Set<AST>: track nodes in grammar containing the predicate
* for error messages and such (nice to know where the predicate
* came from in case of duplicates etc...). By using a set,
* the equals() method will correctly show {pred1,pred2} as equals()
* to {pred2,pred1}.
*/
public SemanticContext semanticContext = SemanticContext.EMPTY_SEMANTIC_CONTEXT;
/** Indicate that this configuration has been resolved and no further
* DFA processing should occur with it. Essentially, this is used
* as an "ignore" bit so that upon a set of nondeterministic configurations
* such as (s|2) and (s|3), I can set (s|3) to resolved=true (and any
* other configuration associated with alt 3).
*/
protected boolean resolved;
/** This bit is used to indicate a semantic predicate will be
* used to resolve the conflict. Method
* DFA.findNewDFAStatesAndAddDFATransitions will add edges for
* the predicates after it performs the reach operation. The
* nondeterminism resolver sets this when it finds a set of
* nondeterministic configurations (as it does for "resolved" field)
* that have enough predicates to resolve the conflit.
*/
protected boolean resolveWithPredicate;
/** Lots of NFA states have only epsilon edges (1 or 2). We can
* safely consider only n>0 during closure.
*/
protected int numberEpsilonTransitionsEmanatingFromState;
/** Indicates that the NFA state associated with this configuration
* has exactly one transition and it's an atom (not epsilon etc...).
*/
protected boolean singleAtomTransitionEmanating;
//protected boolean addedDuringClosure = true;
public NFAConfiguration(int state,
int alt,
NFAContext context,
SemanticContext semanticContext)
{
this.state = state;
this.alt = alt;
this.context = context;
this.semanticContext = semanticContext;
}
/** An NFA configuration is equal to another if both have
* the same state, the predict the same alternative, and
* syntactic/semantic contexts are the same. I don't think
* the state|alt|ctx could be the same and have two different
* semantic contexts, but might as well define equals to be
* everything.
*/
public boolean equals(Object o) {
if ( o==null ) {
return false;
}
NFAConfiguration other = (NFAConfiguration)o;
return this.state==other.state &&
this.alt==other.alt &&
this.context.equals(other.context)&&
this.semanticContext.equals(other.semanticContext);
}
public int hashCode() {
int h = state + alt + context.hashCode();
return h;
}
public String toString() {
return toString(true);
}
public String toString(boolean showAlt) {
StringBuffer buf = new StringBuffer();
buf.append(state);
if ( showAlt ) {
buf.append("|");
buf.append(alt);
}
if ( context.parent!=null ) {
buf.append("|");
buf.append(context);
}
if ( semanticContext!=null &&
semanticContext!=SemanticContext.EMPTY_SEMANTIC_CONTEXT ) {
buf.append("|");
String escQuote = Utils.replace(semanticContext.toString(), "\"", "\\\"");
buf.append(escQuote);
}
if ( resolved ) {
buf.append("|resolved");
}
if ( resolveWithPredicate ) {
buf.append("|resolveWithPredicate");
}
return buf.toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/DFA.java 0000644 0001750 0001750 00000111732 11256465226 023052 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.codegen.CodeGenerator;
import org.antlr.misc.IntSet;
import org.antlr.misc.IntervalSet;
import org.antlr.misc.Utils;
import org.antlr.runtime.IntStream;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.tool.*;
import java.util.*;
/** A DFA (converted from a grammar's NFA).
* DFAs are used as prediction machine for alternative blocks in all kinds
* of recognizers (lexers, parsers, tree walkers).
*/
public class DFA {
public static final int REACHABLE_UNKNOWN = -2;
public static final int REACHABLE_BUSY = -1; // in process of computing
public static final int REACHABLE_NO = 0;
public static final int REACHABLE_YES = 1;
/** Prevent explosion of DFA states during conversion. The max number
* of states per alt in a single decision's DFA.
public static final int MAX_STATES_PER_ALT_IN_DFA = 450;
*/
/** Set to 0 to not terminate early (time in ms) */
public static int MAX_TIME_PER_DFA_CREATION = 1*1000;
/** How many edges can each DFA state have before a "special" state
* is created that uses IF expressions instead of a table?
*/
public static int MAX_STATE_TRANSITIONS_FOR_TABLE = 65534;
/** What's the start state for this DFA? */
public DFAState startState;
/** This DFA is being built for which decision? */
public int decisionNumber = 0;
/** From what NFAState did we create the DFA? */
public NFAState decisionNFAStartState;
/** The printable grammar fragment associated with this DFA */
public String description;
/** A set of all uniquely-numbered DFA states. Maps hash of DFAState
* to the actual DFAState object. We use this to detect
* existing DFA states. Map<DFAState,DFAState>. Use Map so
* we can get old state back (Set only allows you to see if it's there).
* Not used during fixed k lookahead as it's a waste to fill it with
* a dup of states array.
*/
protected Map<DFAState, DFAState> uniqueStates = new HashMap<DFAState, DFAState>();
/** Maps the state number to the actual DFAState. Use a Vector as it
* grows automatically when I set the ith element. This contains all
* states, but the states are not unique. s3 might be same as s1 so
* s3 -> s1 in this table. This is how cycles occur. If fixed k,
* then these states will all be unique as states[i] always points
* at state i when no cycles exist.
*
* This is managed in parallel with uniqueStates and simply provides
* a way to go from state number to DFAState rather than via a
* hash lookup.
*/
protected Vector<DFAState> states = new Vector<DFAState>();
/** Unique state numbers per DFA */
protected int stateCounter = 0;
/** count only new states not states that were rejected as already present */
protected int numberOfStates = 0;
/** User specified max fixed lookahead. If 0, nothing specified. -1
* implies we have not looked at the options table yet to set k.
*/
protected int user_k = -1;
/** While building the DFA, track max lookahead depth if not cyclic */
protected int max_k = -1;
/** Is this DFA reduced? I.e., can all states lead to an accept state? */
protected boolean reduced = true;
/** Are there any loops in this DFA?
* Computed by doesStateReachAcceptState()
*/
protected boolean cyclic = false;
/** Track whether this DFA has at least one sem/syn pred encountered
* during a closure operation. This is useful for deciding whether
* to retry a non-LL(*) with k=1. If no pred, it will not work w/o
* a pred so don't bother. It would just give another error message.
*/
public boolean predicateVisible = false;
public boolean hasPredicateBlockedByAction = false;
/** Each alt in an NFA derived from a grammar must have a DFA state that
* predicts it lest the parser not know what to do. Nondeterminisms can
* lead to this situation (assuming no semantic predicates can resolve
* the problem) and when for some reason, I cannot compute the lookahead
* (which might arise from an error in the algorithm or from
* left-recursion etc...). This list starts out with all alts contained
* and then in method doesStateReachAcceptState() I remove the alts I
* know to be uniquely predicted.
*/
protected List<Integer> unreachableAlts;
protected int nAlts = 0;
/** We only want one accept state per predicted alt; track here */
protected DFAState[] altToAcceptState;
/** Track whether an alt discovers recursion for each alt during
* NFA to DFA conversion; >1 alt with recursion implies nonregular.
*/
public IntSet recursiveAltSet = new IntervalSet();
/** Which NFA are we converting (well, which piece of the NFA)? */
public NFA nfa;
protected NFAToDFAConverter nfaConverter;
/** This probe tells you a lot about a decision and is useful even
* when there is no error such as when a syntactic nondeterminism
* is solved via semantic predicates. Perhaps a GUI would want
* the ability to show that.
*/
public DecisionProbe probe = new DecisionProbe(this);
/** Track absolute time of the conversion so we can have a failsafe:
* if it takes too long, then terminate. Assume bugs are in the
* analysis engine.
*/
protected long conversionStartTime;
/** Map an edge transition table to a unique set number; ordered so
* we can push into the output template as an ordered list of sets
* and then ref them from within the transition[][] table. Like this
* for C# target:
* public static readonly DFA30_transition0 =
* new short[] { 46, 46, -1, 46, 46, -1, -1, -1, -1, -1, -1, -1,...};
* public static readonly DFA30_transition1 =
* new short[] { 21 };
* public static readonly short[][] DFA30_transition = {
* DFA30_transition0,
* DFA30_transition0,
* DFA30_transition1,
* ...
* };
*/
public Map edgeTransitionClassMap = new LinkedHashMap();
/** The unique edge transition class number; every time we see a new
* set of edges emanating from a state, we number it so we can reuse
* if it's every seen again for another state. For Java grammar,
* some of the big edge transition tables are seen about 57 times.
*/
protected int edgeTransitionClass =0;
/* This DFA can be converted to a transition[state][char] table and
* the following tables are filled by createStateTables upon request.
* These are injected into the templates for code generation.
* See March 25, 2006 entry for description:
* http://www.antlr.org/blog/antlr3/codegen.tml
* Often using Vector as can't set ith position in a List and have
* it extend list size; bizarre.
*/
/** List of special DFAState objects */
public List specialStates;
/** List of ST for special states. */
public List specialStateSTs;
public Vector accept;
public Vector eot;
public Vector eof;
public Vector min;
public Vector max;
public Vector special;
public Vector transition;
/** just the Vector<Integer> indicating which unique edge table is at
* position i.
*/
public Vector transitionEdgeTables; // not used by java yet
protected int uniqueCompressedSpecialStateNum = 0;
/** Which generator to use if we're building state tables */
protected CodeGenerator generator = null;
protected DFA() {;}
public DFA(int decisionNumber, NFAState decisionStartState) {
this.decisionNumber = decisionNumber;
this.decisionNFAStartState = decisionStartState;
nfa = decisionStartState.nfa;
nAlts = nfa.grammar.getNumberOfAltsForDecisionNFA(decisionStartState);
//setOptions( nfa.grammar.getDecisionOptions(getDecisionNumber()) );
initAltRelatedInfo();
//long start = System.currentTimeMillis();
nfaConverter = new NFAToDFAConverter(this);
try {
nfaConverter.convert();
// figure out if there are problems with decision
verify();
if ( !probe.isDeterministic() || probe.analysisOverflowed() ) {
probe.issueWarnings();
}
// must be after verify as it computes cyclic, needed by this routine
// should be after warnings because early termination or something
// will not allow the reset to operate properly in some cases.
resetStateNumbersToBeContiguous();
//long stop = System.currentTimeMillis();
//System.out.println("verify cost: "+(int)(stop-start)+" ms");
}
catch (AnalysisTimeoutException at) {
probe.reportAnalysisTimeout();
if ( !okToRetryDFAWithK1() ) {
probe.issueWarnings();
}
}
catch (NonLLStarDecisionException nonLL) {
probe.reportNonLLStarDecision(this);
// >1 alt recurses, k=* and no auto backtrack nor manual sem/syn
if ( !okToRetryDFAWithK1() ) {
probe.issueWarnings();
}
}
}
/** Walk all states and reset their numbers to be a contiguous sequence
* of integers starting from 0. Only cyclic DFA can have unused positions
* in states list. State i might be identical to a previous state j and
* will result in states[i] == states[j]. We don't want to waste a state
* number on this. Useful mostly for code generation in tables.
*
* At the start of this routine, states[i].stateNumber <= i by definition.
* If states[50].stateNumber is 50 then a cycle during conversion may
* try to add state 103, but we find that an identical DFA state, named
* 50, already exists, hence, states[103]==states[50] and both have
* stateNumber 50 as they point at same object. Afterwards, the set
* of state numbers from all states should represent a contiguous range
* from 0..n-1 where n is the number of unique states.
*/
public void resetStateNumbersToBeContiguous() {
if ( getUserMaxLookahead()>0 ) {
// all numbers are unique already; no states are thrown out.
return;
}
// walk list of DFAState objects by state number,
// setting state numbers to 0..n-1
int snum=0;
for (int i = 0; i <= getMaxStateNumber(); i++) {
DFAState s = getState(i);
// some states are unused after creation most commonly due to cycles
// or conflict resolution.
if ( s==null ) {
continue;
}
// state i is mapped to DFAState with state number set to i originally
// so if it's less than i, then we renumbered it already; that
// happens when states have been merged or cycles occurred I think.
// states[50] will point to DFAState with s50 in it but
// states[103] might also point at this same DFAState. Since
// 50 < 103 then it's already been renumbered as it points downwards.
boolean alreadyRenumbered = s.stateNumber<i;
if ( !alreadyRenumbered ) {
// state i is a valid state, reset it's state number
s.stateNumber = snum; // rewrite state numbers to be 0..n-1
snum++;
}
}
if ( snum!=getNumberOfStates() ) {
ErrorManager.internalError("DFA "+decisionNumber+": "+
decisionNFAStartState.getDescription()+" num unique states "+getNumberOfStates()+
"!= num renumbered states "+snum);
}
}
// JAVA-SPECIFIC Accessors!!!!! It is so impossible to get arrays
// or even consistently formatted strings acceptable to java that
// I am forced to build the individual char elements here
public List getJavaCompressedAccept() { return getRunLengthEncoding(accept); }
public List getJavaCompressedEOT() { return getRunLengthEncoding(eot); }
public List getJavaCompressedEOF() { return getRunLengthEncoding(eof); }
public List getJavaCompressedMin() { return getRunLengthEncoding(min); }
public List getJavaCompressedMax() { return getRunLengthEncoding(max); }
public List getJavaCompressedSpecial() { return getRunLengthEncoding(special); }
public List getJavaCompressedTransition() {
if ( transition==null || transition.size()==0 ) {
return null;
}
List encoded = new ArrayList(transition.size());
// walk Vector<Vector<FormattedInteger>> which is the transition[][] table
for (int i = 0; i < transition.size(); i++) {
Vector transitionsForState = (Vector) transition.elementAt(i);
encoded.add(getRunLengthEncoding(transitionsForState));
}
return encoded;
}
/** Compress the incoming data list so that runs of same number are
* encoded as number,value pair sequences. 3 -1 -1 -1 28 is encoded
* as 1 3 3 -1 1 28. I am pretty sure this is the lossless compression
* that GIF files use. Transition tables are heavily compressed by
* this technique. I got the idea from JFlex http://jflex.de/
*
* Return List<String> where each string is either \xyz for 8bit char
* and \uFFFF for 16bit. Hideous and specific to Java, but it is the
* only target bad enough to need it.
*/
public List getRunLengthEncoding(List data) {
if ( data==null || data.size()==0 ) {
// for states with no transitions we want an empty string ""
// to hold its place in the transitions array.
List empty = new ArrayList();
empty.add("");
return empty;
}
int size = Math.max(2,data.size()/2);
List encoded = new ArrayList(size); // guess at size
// scan values looking for runs
int i = 0;
Integer emptyValue = Utils.integer(-1);
while ( i < data.size() ) {
Integer I = (Integer)data.get(i);
if ( I==null ) {
I = emptyValue;
}
// count how many v there are?
int n = 0;
for (int j = i; j < data.size(); j++) {
Integer v = (Integer)data.get(j);
if ( v==null ) {
v = emptyValue;
}
if ( I.equals(v) ) {
n++;
}
else {
break;
}
}
encoded.add(generator.target.encodeIntAsCharEscape((char)n));
encoded.add(generator.target.encodeIntAsCharEscape((char)I.intValue()));
i+=n;
}
return encoded;
}
public void createStateTables(CodeGenerator generator) {
//System.out.println("createTables:\n"+this);
this.generator = generator;
description = getNFADecisionStartState().getDescription();
description =
generator.target.getTargetStringLiteralFromString(description);
// create all the tables
special = new Vector(this.getNumberOfStates()); // Vector<short>
special.setSize(this.getNumberOfStates());
specialStates = new ArrayList(); // List<DFAState>
specialStateSTs = new ArrayList(); // List<ST>
accept = new Vector(this.getNumberOfStates()); // Vector<int>
accept.setSize(this.getNumberOfStates());
eot = new Vector(this.getNumberOfStates()); // Vector<int>
eot.setSize(this.getNumberOfStates());
eof = new Vector(this.getNumberOfStates()); // Vector<int>
eof.setSize(this.getNumberOfStates());
min = new Vector(this.getNumberOfStates()); // Vector<int>
min.setSize(this.getNumberOfStates());
max = new Vector(this.getNumberOfStates()); // Vector<int>
max.setSize(this.getNumberOfStates());
transition = new Vector(this.getNumberOfStates()); // Vector<Vector<int>>
transition.setSize(this.getNumberOfStates());
transitionEdgeTables = new Vector(this.getNumberOfStates()); // Vector<Vector<int>>
transitionEdgeTables.setSize(this.getNumberOfStates());
// for each state in the DFA, fill relevant tables.
Iterator it = null;
if ( getUserMaxLookahead()>0 ) {
it = states.iterator();
}
else {
it = getUniqueStates().values().iterator();
}
while ( it.hasNext() ) {
DFAState s = (DFAState)it.next();
if ( s==null ) {
// ignore null states; some acylic DFA see this condition
// when inlining DFA (due to lacking of exit branch pruning?)
continue;
}
if ( s.isAcceptState() ) {
// can't compute min,max,special,transition on accepts
accept.set(s.stateNumber,
Utils.integer(s.getUniquelyPredictedAlt()));
}
else {
createMinMaxTables(s);
createTransitionTableEntryForState(s);
createSpecialTable(s);
createEOTAndEOFTables(s);
}
}
// now that we have computed list of specialStates, gen code for 'em
for (int i = 0; i < specialStates.size(); i++) {
DFAState ss = (DFAState) specialStates.get(i);
StringTemplate stateST =
generator.generateSpecialState(ss);
specialStateSTs.add(stateST);
}
// check that the tables are not messed up by encode/decode
/*
testEncodeDecode(min);
testEncodeDecode(max);
testEncodeDecode(accept);
testEncodeDecode(special);
System.out.println("min="+min);
System.out.println("max="+max);
System.out.println("eot="+eot);
System.out.println("eof="+eof);
System.out.println("accept="+accept);
System.out.println("special="+special);
System.out.println("transition="+transition);
*/
}
/*
private void testEncodeDecode(List data) {
System.out.println("data="+data);
List encoded = getRunLengthEncoding(data);
StringBuffer buf = new StringBuffer();
for (int i = 0; i < encoded.size(); i++) {
String I = (String)encoded.get(i);
int v = 0;
if ( I.startsWith("\\u") ) {
v = Integer.parseInt(I.substring(2,I.length()), 16);
}
else {
v = Integer.parseInt(I.substring(1,I.length()), 8);
}
buf.append((char)v);
}
String encodedS = buf.toString();
short[] decoded = org.antlr.runtime.DFA.unpackEncodedString(encodedS);
//System.out.println("decoded:");
for (int i = 0; i < decoded.length; i++) {
short x = decoded[i];
if ( x!=((Integer)data.get(i)).intValue() ) {
System.err.println("problem with encoding");
}
//System.out.print(", "+x);
}
//System.out.println();
}
*/
protected void createMinMaxTables(DFAState s) {
int smin = Label.MAX_CHAR_VALUE + 1;
int smax = Label.MIN_ATOM_VALUE - 1;
for (int j = 0; j < s.getNumberOfTransitions(); j++) {
Transition edge = (Transition) s.transition(j);
Label label = edge.label;
if ( label.isAtom() ) {
if ( label.getAtom()>=Label.MIN_CHAR_VALUE ) {
if ( label.getAtom()<smin ) {
smin = label.getAtom();
}
if ( label.getAtom()>smax ) {
smax = label.getAtom();
}
}
}
else if ( label.isSet() ) {
IntervalSet labels = (IntervalSet)label.getSet();
int lmin = labels.getMinElement();
// if valid char (don't do EOF) and less than current min
if ( lmin<smin && lmin>=Label.MIN_CHAR_VALUE ) {
smin = labels.getMinElement();
}
if ( labels.getMaxElement()>smax ) {
smax = labels.getMaxElement();
}
}
}
if ( smax<0 ) {
// must be predicates or pure EOT transition; just zero out min, max
smin = Label.MIN_CHAR_VALUE;
smax = Label.MIN_CHAR_VALUE;
}
min.set(s.stateNumber, Utils.integer((char)smin));
max.set(s.stateNumber, Utils.integer((char)smax));
if ( smax<0 || smin>Label.MAX_CHAR_VALUE || smin<0 ) {
ErrorManager.internalError("messed up: min="+min+", max="+max);
}
}
protected void createTransitionTableEntryForState(DFAState s) {
/*
System.out.println("createTransitionTableEntryForState s"+s.stateNumber+
" dec "+s.dfa.decisionNumber+" cyclic="+s.dfa.isCyclic());
*/
int smax = ((Integer)max.get(s.stateNumber)).intValue();
int smin = ((Integer)min.get(s.stateNumber)).intValue();
Vector stateTransitions = new Vector(smax-smin+1);
stateTransitions.setSize(smax-smin+1);
transition.set(s.stateNumber, stateTransitions);
for (int j = 0; j < s.getNumberOfTransitions(); j++) {
Transition edge = (Transition) s.transition(j);
Label label = edge.label;
if ( label.isAtom() && label.getAtom()>=Label.MIN_CHAR_VALUE ) {
int labelIndex = label.getAtom()-smin; // offset from 0
stateTransitions.set(labelIndex,
Utils.integer(edge.target.stateNumber));
}
else if ( label.isSet() ) {
IntervalSet labels = (IntervalSet)label.getSet();
int[] atoms = labels.toArray();
for (int a = 0; a < atoms.length; a++) {
// set the transition if the label is valid (don't do EOF)
if ( atoms[a]>=Label.MIN_CHAR_VALUE ) {
int labelIndex = atoms[a]-smin; // offset from 0
stateTransitions.set(labelIndex,
Utils.integer(edge.target.stateNumber));
}
}
}
}
// track unique state transition tables so we can reuse
Integer edgeClass = (Integer)edgeTransitionClassMap.get(stateTransitions);
if ( edgeClass!=null ) {
//System.out.println("we've seen this array before; size="+stateTransitions.size());
transitionEdgeTables.set(s.stateNumber, edgeClass);
}
else {
edgeClass = Utils.integer(edgeTransitionClass);
transitionEdgeTables.set(s.stateNumber, edgeClass);
edgeTransitionClassMap.put(stateTransitions, edgeClass);
edgeTransitionClass++;
}
}
/** Set up the EOT and EOF tables; we cannot put -1 min/max values so
* we need another way to test that in the DFA transition function.
*/
protected void createEOTAndEOFTables(DFAState s) {
for (int j = 0; j < s.getNumberOfTransitions(); j++) {
Transition edge = (Transition) s.transition(j);
Label label = edge.label;
if ( label.isAtom() ) {
if ( label.getAtom()==Label.EOT ) {
// eot[s] points to accept state
eot.set(s.stateNumber, Utils.integer(edge.target.stateNumber));
}
else if ( label.getAtom()==Label.EOF ) {
// eof[s] points to accept state
eof.set(s.stateNumber, Utils.integer(edge.target.stateNumber));
}
}
else if ( label.isSet() ) {
IntervalSet labels = (IntervalSet)label.getSet();
int[] atoms = labels.toArray();
for (int a = 0; a < atoms.length; a++) {
if ( atoms[a]==Label.EOT ) {
// eot[s] points to accept state
eot.set(s.stateNumber, Utils.integer(edge.target.stateNumber));
}
else if ( atoms[a]==Label.EOF ) {
eof.set(s.stateNumber, Utils.integer(edge.target.stateNumber));
}
}
}
}
}
protected void createSpecialTable(DFAState s) {
// number all special states from 0...n-1 instead of their usual numbers
boolean hasSemPred = false;
// TODO this code is very similar to canGenerateSwitch. Refactor to share
for (int j = 0; j < s.getNumberOfTransitions(); j++) {
Transition edge = (Transition) s.transition(j);
Label label = edge.label;
// can't do a switch if the edges have preds or are going to
// require gated predicates
if ( label.isSemanticPredicate() ||
((DFAState)edge.target).getGatedPredicatesInNFAConfigurations()!=null)
{
hasSemPred = true;
break;
}
}
// if has pred or too big for table, make it special
int smax = ((Integer)max.get(s.stateNumber)).intValue();
int smin = ((Integer)min.get(s.stateNumber)).intValue();
if ( hasSemPred || smax-smin>MAX_STATE_TRANSITIONS_FOR_TABLE ) {
special.set(s.stateNumber,
Utils.integer(uniqueCompressedSpecialStateNum));
uniqueCompressedSpecialStateNum++;
specialStates.add(s);
}
else {
special.set(s.stateNumber, Utils.integer(-1)); // not special
}
}
public int predict(IntStream input) {
Interpreter interp = new Interpreter(nfa.grammar, input);
return interp.predict(this);
}
/** Add a new DFA state to this DFA if not already present.
* To force an acyclic, fixed maximum depth DFA, just always
* return the incoming state. By not reusing old states,
* no cycles can be created. If we're doing fixed k lookahead
* don't updated uniqueStates, just return incoming state, which
* indicates it's a new state.
*/
protected DFAState addState(DFAState d) {
if ( getUserMaxLookahead()>0 ) {
return d;
}
// does a DFA state exist already with everything the same
// except its state number?
DFAState existing = (DFAState)uniqueStates.get(d);
if ( existing != null ) {
/*
System.out.println("state "+d.stateNumber+" exists as state "+
existing.stateNumber);
*/
// already there...get the existing DFA state
return existing;
}
// if not there, then add new state.
uniqueStates.put(d,d);
numberOfStates++;
return d;
}
public void removeState(DFAState d) {
DFAState it = (DFAState)uniqueStates.remove(d);
if ( it!=null ) {
numberOfStates--;
}
}
public Map<DFAState, DFAState> getUniqueStates() {
return uniqueStates;
}
/** What is the max state number ever created? This may be beyond
* getNumberOfStates().
*/
public int getMaxStateNumber() {
return states.size()-1;
}
public DFAState getState(int stateNumber) {
return (DFAState)states.get(stateNumber);
}
public void setState(int stateNumber, DFAState d) {
states.set(stateNumber, d);
}
/** Is the DFA reduced? I.e., does every state have a path to an accept
* state? If not, don't delete as we need to generate an error indicating
* which paths are "dead ends". Also tracks list of alts with no accept
* state in the DFA. Must call verify() first before this makes sense.
*/
public boolean isReduced() {
return reduced;
}
/** Is this DFA cyclic? That is, are there any loops? If not, then
* the DFA is essentially an LL(k) predictor for some fixed, max k value.
* We can build a series of nested IF statements to match this. In the
* presence of cycles, we need to build a general DFA and interpret it
* to distinguish between alternatives.
*/
public boolean isCyclic() {
return cyclic && getUserMaxLookahead()==0;
}
public boolean canInlineDecision() {
return !isCyclic() &&
!probe.isNonLLStarDecision() &&
getNumberOfStates() < CodeGenerator.MAX_ACYCLIC_DFA_STATES_INLINE;
}
/** Is this DFA derived from the NFA for the Tokens rule? */
public boolean isTokensRuleDecision() {
if ( nfa.grammar.type!=Grammar.LEXER ) {
return false;
}
NFAState nfaStart = getNFADecisionStartState();
Rule r = nfa.grammar.getLocallyDefinedRule(Grammar.ARTIFICIAL_TOKENS_RULENAME);
NFAState TokensRuleStart = r.startState;
NFAState TokensDecisionStart =
(NFAState)TokensRuleStart.transition[0].target;
return nfaStart == TokensDecisionStart;
}
/** The user may specify a max, acyclic lookahead for any decision. No
* DFA cycles are created when this value, k, is greater than 0.
* If this decision has no k lookahead specified, then try the grammar.
*/
public int getUserMaxLookahead() {
if ( user_k>=0 ) { // cache for speed
return user_k;
}
user_k = nfa.grammar.getUserMaxLookahead(decisionNumber);
return user_k;
}
public boolean getAutoBacktrackMode() {
return nfa.grammar.getAutoBacktrackMode(decisionNumber);
}
public void setUserMaxLookahead(int k) {
this.user_k = k;
}
/** Return k if decision is LL(k) for some k else return max int */
public int getMaxLookaheadDepth() {
if ( isCyclic() ) {
return Integer.MAX_VALUE;
}
return max_k;
}
/** Return a list of Integer alt numbers for which no lookahead could
* be computed or for which no single DFA accept state predicts those
* alts. Must call verify() first before this makes sense.
*/
public List<Integer> getUnreachableAlts() {
return unreachableAlts;
}
/** Once this DFA has been built, need to verify that:
*
* 1. it's reduced
* 2. all alts have an accept state
*
* Elsewhere, in the NFA converter, we need to verify that:
*
* 3. alts i and j have disjoint lookahead if no sem preds
* 4. if sem preds, nondeterministic alts must be sufficiently covered
*
* This is avoided if analysis bails out for any reason.
*/
public void verify() {
doesStateReachAcceptState(startState);
}
/** figure out if this state eventually reaches an accept state and
* modify the instance variable 'reduced' to indicate if we find
* at least one state that cannot reach an accept state. This implies
* that the overall DFA is not reduced. This algorithm should be
* linear in the number of DFA states.
*
* The algorithm also tracks which alternatives have no accept state,
* indicating a nondeterminism.
*
* Also computes whether the DFA is cyclic.
*
* TODO: I call getUniquelyPredicatedAlt too much; cache predicted alt
*/
protected boolean doesStateReachAcceptState(DFAState d) {
if ( d.isAcceptState() ) {
// accept states have no edges emanating from them so we can return
d.setAcceptStateReachable(REACHABLE_YES);
// this alt is uniquely predicted, remove from nondeterministic list
int predicts = d.getUniquelyPredictedAlt();
unreachableAlts.remove(Utils.integer(predicts));
return true;
}
// avoid infinite loops
d.setAcceptStateReachable(REACHABLE_BUSY);
boolean anEdgeReachesAcceptState = false;
// Visit every transition, track if at least one edge reaches stop state
// Cannot terminate when we know this state reaches stop state since
// all transitions must be traversed to set status of each DFA state.
for (int i=0; i<d.getNumberOfTransitions(); i++) {
Transition t = d.transition(i);
DFAState edgeTarget = (DFAState)t.target;
int targetStatus = edgeTarget.getAcceptStateReachable();
if ( targetStatus==REACHABLE_BUSY ) { // avoid cycles; they say nothing
cyclic = true;
continue;
}
if ( targetStatus==REACHABLE_YES ) { // avoid unnecessary work
anEdgeReachesAcceptState = true;
continue;
}
if ( targetStatus==REACHABLE_NO ) { // avoid unnecessary work
continue;
}
// target must be REACHABLE_UNKNOWN (i.e., unvisited)
if ( doesStateReachAcceptState(edgeTarget) ) {
anEdgeReachesAcceptState = true;
// have to keep looking so don't break loop
// must cover all states even if we find a path for this state
}
}
if ( anEdgeReachesAcceptState ) {
d.setAcceptStateReachable(REACHABLE_YES);
}
else {
d.setAcceptStateReachable(REACHABLE_NO);
reduced = false;
}
return anEdgeReachesAcceptState;
}
/** Walk all accept states and find the manually-specified synpreds.
* Gated preds are not always hoisted
* I used to do this in the code generator, but that is too late.
* This converter tries to avoid computing DFA for decisions in
* syntactic predicates that are not ever used such as those
* created by autobacktrack mode.
*/
public void findAllGatedSynPredsUsedInDFAAcceptStates() {
int nAlts = getNumberOfAlts();
for (int i=1; i<=nAlts; i++) {
DFAState a = getAcceptState(i);
//System.out.println("alt "+i+": "+a);
if ( a!=null ) {
Set synpreds = a.getGatedSyntacticPredicatesInNFAConfigurations();
if ( synpreds!=null ) {
// add all the predicates we find (should be just one, right?)
for (Iterator it = synpreds.iterator(); it.hasNext();) {
SemanticContext semctx = (SemanticContext) it.next();
// System.out.println("synpreds: "+semctx);
nfa.grammar.synPredUsedInDFA(this, semctx);
}
}
}
}
}
public NFAState getNFADecisionStartState() {
return decisionNFAStartState;
}
public DFAState getAcceptState(int alt) {
return altToAcceptState[alt];
}
public void setAcceptState(int alt, DFAState acceptState) {
altToAcceptState[alt] = acceptState;
}
public String getDescription() {
return description;
}
public int getDecisionNumber() {
return decisionNFAStartState.getDecisionNumber();
}
/** If this DFA failed to finish during construction, we might be
* able to retry with k=1 but we need to know whether it will
* potentially succeed. Can only succeed if there is a predicate
* to resolve the issue. Don't try if k=1 already as it would
* cycle forever. Timeout can retry with k=1 even if no predicate
* if k!=1.
*/
public boolean okToRetryDFAWithK1() {
boolean nonLLStarOrOverflowAndPredicateVisible =
(probe.isNonLLStarDecision()||probe.analysisOverflowed()) &&
predicateVisible; // auto backtrack or manual sem/syn
return getUserMaxLookahead()!=1 &&
(analysisTimedOut() || nonLLStarOrOverflowAndPredicateVisible);
}
public String getReasonForFailure() {
StringBuffer buf = new StringBuffer();
if ( probe.isNonLLStarDecision() ) {
buf.append("non-LL(*)");
if ( predicateVisible ) {
buf.append(" && predicate visible");
}
}
if ( probe.analysisOverflowed() ) {
buf.append("recursion overflow");
if ( predicateVisible ) {
buf.append(" && predicate visible");
}
}
if ( analysisTimedOut() ) {
if ( buf.length()>0 ) {
buf.append(" && ");
}
buf.append("timed out (>");
buf.append(DFA.MAX_TIME_PER_DFA_CREATION);
buf.append("ms)");
}
buf.append("\n");
return buf.toString();
}
/** What GrammarAST node (derived from the grammar) is this DFA
* associated with? It will point to the start of a block or
* the loop back of a (...)+ block etc...
*/
public GrammarAST getDecisionASTNode() {
return decisionNFAStartState.associatedASTNode;
}
public boolean isGreedy() {
GrammarAST blockAST = nfa.grammar.getDecisionBlockAST(decisionNumber);
Object v = nfa.grammar.getBlockOption(blockAST,"greedy");
if ( v!=null && v.equals("false") ) {
return false;
}
return true;
}
public DFAState newState() {
DFAState n = new DFAState(this);
n.stateNumber = stateCounter;
stateCounter++;
states.setSize(n.stateNumber+1);
states.set(n.stateNumber, n); // track state num to state
return n;
}
public int getNumberOfStates() {
if ( getUserMaxLookahead()>0 ) {
// if using fixed lookahead then uniqueSets not set
return states.size();
}
return numberOfStates;
}
public int getNumberOfAlts() {
return nAlts;
}
public boolean analysisTimedOut() {
return probe.analysisTimedOut();
}
protected void initAltRelatedInfo() {
unreachableAlts = new LinkedList();
for (int i = 1; i <= nAlts; i++) {
unreachableAlts.add(Utils.integer(i));
}
altToAcceptState = new DFAState[nAlts+1];
}
public String toString() {
FASerializer serializer = new FASerializer(nfa.grammar);
if ( startState==null ) {
return "";
}
return serializer.serialize(startState, false);
}
/** EOT (end of token) is a label that indicates when the DFA conversion
* algorithm would "fall off the end of a lexer rule". It normally
* means the default clause. So for ('a'..'z')+ you would see a DFA
* with a state that has a..z and EOT emanating from it. a..z would
* jump to a state predicting alt 1 and EOT would jump to a state
* predicting alt 2 (the exit loop branch). EOT implies anything other
* than a..z. If for some reason, the set is "all char" such as with
* the wildcard '.', then EOT cannot match anything. For example,
*
* BLOCK : '{' (.)* '}'
*
* consumes all char until EOF when greedy=true. When all edges are
* combined for the DFA state after matching '}', you will find that
* it is all char. The EOT transition has nothing to match and is
* unreachable. The findNewDFAStatesAndAddDFATransitions() method
* must know to ignore the EOT, so we simply remove it from the
* reachable labels. Later analysis will find that the exit branch
* is not predicted by anything. For greedy=false, we leave only
* the EOT label indicating that the DFA should stop immediately
* and predict the exit branch. The reachable labels are often a
* set of disjoint values like: [<EOT>, 42, {0..41, 43..65534}]
* due to DFA conversion so must construct a pure set to see if
* it is same as Label.ALLCHAR.
*
* Only do this for Lexers.
*
* If EOT coexists with ALLCHAR:
* 1. If not greedy, modify the labels parameter to be EOT
* 2. If greedy, remove EOT from the labels set
protected boolean reachableLabelsEOTCoexistsWithAllChar(OrderedHashSet labels)
{
Label eot = new Label(Label.EOT);
if ( !labels.containsKey(eot) ) {
return false;
}
System.out.println("### contains EOT");
boolean containsAllChar = false;
IntervalSet completeVocab = new IntervalSet();
int n = labels.size();
for (int i=0; i<n; i++) {
Label rl = (Label)labels.get(i);
if ( !rl.equals(eot) ) {
completeVocab.addAll(rl.getSet());
}
}
System.out.println("completeVocab="+completeVocab);
if ( completeVocab.equals(Label.ALLCHAR) ) {
System.out.println("all char");
containsAllChar = true;
}
return containsAllChar;
}
*/
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/LL1Analyzer.java 0000644 0001750 0001750 00000034744 11256465230 024560 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.Rule;
import org.antlr.grammar.v2.ANTLRParser;
import org.antlr.tool.Grammar;
import org.antlr.misc.IntervalSet;
import org.antlr.misc.IntSet;
import java.util.*;
/**
* Created by IntelliJ IDEA.
* User: parrt
* Date: Dec 31, 2007
* Time: 1:31:16 PM
* To change this template use File | Settings | File Templates.
*/
public class LL1Analyzer {
/** 0 if we hit end of rule and invoker should keep going (epsilon) */
public static final int DETECT_PRED_EOR = 0;
/** 1 if we found a nonautobacktracking pred */
public static final int DETECT_PRED_FOUND = 1;
/** 2 if we didn't find such a pred */
public static final int DETECT_PRED_NOT_FOUND = 2;
public Grammar grammar;
/** Used during LOOK to detect computation cycles */
protected Set<NFAState> lookBusy = new HashSet<NFAState>();
public Map<NFAState, LookaheadSet> FIRSTCache = new HashMap<NFAState, LookaheadSet>();
public Map<Rule, LookaheadSet> FOLLOWCache = new HashMap<Rule, LookaheadSet>();
public LL1Analyzer(Grammar grammar) {
this.grammar = grammar;
}
/*
public void computeRuleFIRSTSets() {
if ( getNumberOfDecisions()==0 ) {
createNFAs();
}
for (Iterator it = getRules().iterator(); it.hasNext();) {
Rule r = (Rule)it.next();
if ( r.isSynPred ) {
continue;
}
LookaheadSet s = FIRST(r);
System.out.println("FIRST("+r.name+")="+s);
}
}
*/
/*
public Set<String> getOverriddenRulesWithDifferentFIRST() {
// walk every rule in this grammar and compare FIRST set with
// those in imported grammars.
Set<String> rules = new HashSet();
for (Iterator it = getRules().iterator(); it.hasNext();) {
Rule r = (Rule)it.next();
//System.out.println(r.name+" FIRST="+r.FIRST);
for (int i = 0; i < delegates.size(); i++) {
Grammar g = delegates.get(i);
Rule importedRule = g.getRule(r.name);
if ( importedRule != null ) { // exists in imported grammar
// System.out.println(r.name+" exists in imported grammar: FIRST="+importedRule.FIRST);
if ( !r.FIRST.equals(importedRule.FIRST) ) {
rules.add(r.name);
}
}
}
}
return rules;
}
public Set<Rule> getImportedRulesSensitiveToOverriddenRulesDueToLOOK() {
Set<String> diffFIRSTs = getOverriddenRulesWithDifferentFIRST();
Set<Rule> rules = new HashSet();
for (Iterator it = diffFIRSTs.iterator(); it.hasNext();) {
String r = (String) it.next();
for (int i = 0; i < delegates.size(); i++) {
Grammar g = delegates.get(i);
Set<Rule> callers = g.ruleSensitivity.get(r);
// somebody invokes rule whose FIRST changed in subgrammar?
if ( callers!=null ) {
rules.addAll(callers);
//System.out.println(g.name+" rules "+callers+" sensitive to "+r+"; dup 'em");
}
}
}
return rules;
}
*/
/*
public LookaheadSet LOOK(Rule r) {
if ( r.FIRST==null ) {
r.FIRST = FIRST(r.startState);
}
return r.FIRST;
}
*/
/** From an NFA state, s, find the set of all labels reachable from s.
* Used to compute follow sets for error recovery. Never computes
* a FOLLOW operation. FIRST stops at end of rules, returning EOR, unless
* invoked from another rule. I.e., routine properly handles
*
* a : b A ;
*
* where b is nullable.
*
* We record with EOR_TOKEN_TYPE if we hit the end of a rule so we can
* know at runtime (when these sets are used) to start walking up the
* follow chain to compute the real, correct follow set (as opposed to
* the FOLLOW, which is a superset).
*
* This routine will only be used on parser and tree parser grammars.
*/
public LookaheadSet FIRST(NFAState s) {
//System.out.println("> FIRST("+s.enclosingRule.name+") in rule "+s.enclosingRule);
lookBusy.clear();
LookaheadSet look = _FIRST(s, false);
//System.out.println("< FIRST("+s.enclosingRule.name+") in rule "+s.enclosingRule+"="+look.toString(this.grammar));
return look;
}
public LookaheadSet FOLLOW(Rule r) {
//System.out.println("> FOLLOW("+r.name+") in rule "+r.startState.enclosingRule);
LookaheadSet f = FOLLOWCache.get(r);
if ( f!=null ) {
return f;
}
f = _FIRST(r.stopState, true);
FOLLOWCache.put(r, f);
//System.out.println("< FOLLOW("+r+") in rule "+r.startState.enclosingRule+"="+f.toString(this.grammar));
return f;
}
public LookaheadSet LOOK(NFAState s) {
if ( NFAToDFAConverter.debug ) {
System.out.println("> LOOK("+s+")");
}
lookBusy.clear();
LookaheadSet look = _FIRST(s, true);
// FOLLOW makes no sense (at the moment!) for lexical rules.
if ( grammar.type!=Grammar.LEXER && look.member(Label.EOR_TOKEN_TYPE) ) {
// avoid altering FIRST reset as it is cached
LookaheadSet f = FOLLOW(s.enclosingRule);
f.orInPlace(look);
f.remove(Label.EOR_TOKEN_TYPE);
look = f;
//look.orInPlace(FOLLOW(s.enclosingRule));
}
else if ( grammar.type==Grammar.LEXER && look.member(Label.EOT) ) {
// if this has EOT, lookahead is all char (all char can follow rule)
//look = new LookaheadSet(Label.EOT);
look = new LookaheadSet(IntervalSet.COMPLETE_SET);
}
if ( NFAToDFAConverter.debug ) {
System.out.println("< LOOK("+s+")="+look.toString(grammar));
}
return look;
}
protected LookaheadSet _FIRST(NFAState s, boolean chaseFollowTransitions) {
/*
System.out.println("_LOOK("+s+") in rule "+s.enclosingRule);
if ( s.transition[0] instanceof RuleClosureTransition ) {
System.out.println("go to rule "+((NFAState)s.transition[0].target).enclosingRule);
}
*/
if ( !chaseFollowTransitions && s.isAcceptState() ) {
if ( grammar.type==Grammar.LEXER ) {
// FOLLOW makes no sense (at the moment!) for lexical rules.
// assume all char can follow
return new LookaheadSet(IntervalSet.COMPLETE_SET);
}
return new LookaheadSet(Label.EOR_TOKEN_TYPE);
}
if ( lookBusy.contains(s) ) {
// return a copy of an empty set; we may modify set inline
return new LookaheadSet();
}
lookBusy.add(s);
Transition transition0 = s.transition[0];
if ( transition0==null ) {
return null;
}
if ( transition0.label.isAtom() ) {
int atom = transition0.label.getAtom();
return new LookaheadSet(atom);
}
if ( transition0.label.isSet() ) {
IntSet sl = transition0.label.getSet();
return new LookaheadSet(sl);
}
// compute FIRST of transition 0
LookaheadSet tset = null;
// if transition 0 is a rule call and we don't want FOLLOW, check cache
if ( !chaseFollowTransitions && transition0 instanceof RuleClosureTransition ) {
LookaheadSet prev = FIRSTCache.get((NFAState)transition0.target);
if ( prev!=null ) {
tset = new LookaheadSet(prev);
}
}
// if not in cache, must compute
if ( tset==null ) {
tset = _FIRST((NFAState)transition0.target, chaseFollowTransitions);
// save FIRST cache for transition 0 if rule call
if ( !chaseFollowTransitions && transition0 instanceof RuleClosureTransition ) {
FIRSTCache.put((NFAState)transition0.target, tset);
}
}
// did we fall off the end?
if ( grammar.type!=Grammar.LEXER && tset.member(Label.EOR_TOKEN_TYPE) ) {
if ( transition0 instanceof RuleClosureTransition ) {
// we called a rule that found the end of the rule.
// That means the rule is nullable and we need to
// keep looking at what follows the rule ref. E.g.,
// a : b A ; where b is nullable means that LOOK(a)
// should include A.
RuleClosureTransition ruleInvocationTrans =
(RuleClosureTransition)transition0;
// remove the EOR and get what follows
//tset.remove(Label.EOR_TOKEN_TYPE);
NFAState following = (NFAState) ruleInvocationTrans.followState;
LookaheadSet fset = _FIRST(following, chaseFollowTransitions);
fset.orInPlace(tset); // tset cached; or into new set
fset.remove(Label.EOR_TOKEN_TYPE);
tset = fset;
}
}
Transition transition1 = s.transition[1];
if ( transition1!=null ) {
LookaheadSet tset1 =
_FIRST((NFAState)transition1.target, chaseFollowTransitions);
tset1.orInPlace(tset); // tset cached; or into new set
tset = tset1;
}
return tset;
}
/** Is there a non-syn-pred predicate visible from s that is not in
* the rule enclosing s? This accounts for most predicate situations
* and lets ANTLR do a simple LL(1)+pred computation.
*
* TODO: what about gated vs regular preds?
*/
public boolean detectConfoundingPredicates(NFAState s) {
lookBusy.clear();
Rule r = s.enclosingRule;
return _detectConfoundingPredicates(s, r, false) == DETECT_PRED_FOUND;
}
protected int _detectConfoundingPredicates(NFAState s,
Rule enclosingRule,
boolean chaseFollowTransitions)
{
//System.out.println("_detectNonAutobacktrackPredicates("+s+")");
if ( !chaseFollowTransitions && s.isAcceptState() ) {
if ( grammar.type==Grammar.LEXER ) {
// FOLLOW makes no sense (at the moment!) for lexical rules.
// assume all char can follow
return DETECT_PRED_NOT_FOUND;
}
return DETECT_PRED_EOR;
}
if ( lookBusy.contains(s) ) {
// return a copy of an empty set; we may modify set inline
return DETECT_PRED_NOT_FOUND;
}
lookBusy.add(s);
Transition transition0 = s.transition[0];
if ( transition0==null ) {
return DETECT_PRED_NOT_FOUND;
}
if ( !(transition0.label.isSemanticPredicate()||
transition0.label.isEpsilon()) ) {
return DETECT_PRED_NOT_FOUND;
}
if ( transition0.label.isSemanticPredicate() ) {
//System.out.println("pred "+transition0.label);
SemanticContext ctx = transition0.label.getSemanticContext();
SemanticContext.Predicate p = (SemanticContext.Predicate)ctx;
if ( p.predicateAST.getType() != ANTLRParser.BACKTRACK_SEMPRED ) {
return DETECT_PRED_FOUND;
}
}
/*
if ( transition0.label.isSemanticPredicate() ) {
System.out.println("pred "+transition0.label);
SemanticContext ctx = transition0.label.getSemanticContext();
SemanticContext.Predicate p = (SemanticContext.Predicate)ctx;
// if a non-syn-pred found not in enclosingRule, say we found one
if ( p.predicateAST.getType() != ANTLRParser.BACKTRACK_SEMPRED &&
!p.predicateAST.enclosingRuleName.equals(enclosingRule.name) )
{
System.out.println("found pred "+p+" not in "+enclosingRule.name);
return DETECT_PRED_FOUND;
}
}
*/
int result = _detectConfoundingPredicates((NFAState)transition0.target,
enclosingRule,
chaseFollowTransitions);
if ( result == DETECT_PRED_FOUND ) {
return DETECT_PRED_FOUND;
}
if ( result == DETECT_PRED_EOR ) {
if ( transition0 instanceof RuleClosureTransition ) {
// we called a rule that found the end of the rule.
// That means the rule is nullable and we need to
// keep looking at what follows the rule ref. E.g.,
// a : b A ; where b is nullable means that LOOK(a)
// should include A.
RuleClosureTransition ruleInvocationTrans =
(RuleClosureTransition)transition0;
NFAState following = (NFAState) ruleInvocationTrans.followState;
int afterRuleResult =
_detectConfoundingPredicates(following,
enclosingRule,
chaseFollowTransitions);
if ( afterRuleResult == DETECT_PRED_FOUND ) {
return DETECT_PRED_FOUND;
}
}
}
Transition transition1 = s.transition[1];
if ( transition1!=null ) {
int t1Result =
_detectConfoundingPredicates((NFAState)transition1.target,
enclosingRule,
chaseFollowTransitions);
if ( t1Result == DETECT_PRED_FOUND ) {
return DETECT_PRED_FOUND;
}
}
return DETECT_PRED_NOT_FOUND;
}
/** Return predicate expression found via epsilon edges from s. Do
* not look into other rules for now. Do something simple. Include
* backtracking synpreds.
*/
public SemanticContext getPredicates(NFAState altStartState) {
lookBusy.clear();
return _getPredicates(altStartState, altStartState);
}
protected SemanticContext _getPredicates(NFAState s, NFAState altStartState) {
//System.out.println("_getPredicates("+s+")");
if ( s.isAcceptState() ) {
return null;
}
// avoid infinite loops from (..)* etc...
if ( lookBusy.contains(s) ) {
return null;
}
lookBusy.add(s);
Transition transition0 = s.transition[0];
// no transitions
if ( transition0==null ) {
return null;
}
// not a predicate and not even an epsilon
if ( !(transition0.label.isSemanticPredicate()||
transition0.label.isEpsilon()) ) {
return null;
}
SemanticContext p = null;
SemanticContext p0 = null;
SemanticContext p1 = null;
if ( transition0.label.isSemanticPredicate() ) {
//System.out.println("pred "+transition0.label);
p = transition0.label.getSemanticContext();
// ignore backtracking preds not on left edge for this decision
if ( ((SemanticContext.Predicate)p).predicateAST.getType() ==
ANTLRParser.BACKTRACK_SEMPRED &&
s == altStartState.transition[0].target )
{
p = null; // don't count
}
}
// get preds from beyond this state
p0 = _getPredicates((NFAState)transition0.target, altStartState);
// get preds from other transition
Transition transition1 = s.transition[1];
if ( transition1!=null ) {
p1 = _getPredicates((NFAState)transition1.target, altStartState);
}
// join this&following-right|following-down
return SemanticContext.and(p,SemanticContext.or(p0,p1));
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/AnalysisTimeoutException.java 0000644 0001750 0001750 00000003267 11256465226 027474 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
/** Analysis took too long; bail out of entire DFA construction. */
public class AnalysisTimeoutException extends RuntimeException {
public DFA abortedDFA;
public AnalysisTimeoutException(DFA abortedDFA) {
this.abortedDFA = abortedDFA;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/ActionLabel.java 0000644 0001750 0001750 00000003701 11256465226 024631 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.GrammarAST;
import org.antlr.tool.Grammar;
public class ActionLabel extends Label {
public GrammarAST actionAST;
public ActionLabel(GrammarAST actionAST) {
super(ACTION);
this.actionAST = actionAST;
}
public boolean isEpsilon() {
return true; // we are to be ignored by analysis 'cept for predicates
}
public boolean isAction() {
return true;
}
public String toString() {
return "{"+actionAST+"}";
}
public String toString(Grammar g) {
return toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/LL1DFA.java 0000644 0001750 0001750 00000014753 11256465230 023363 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.misc.IntervalSet;
import org.antlr.misc.MultiMap;
import org.antlr.grammar.v2.ANTLRParser;
import java.util.Iterator;
import java.util.List;
import java.util.Collections;
/** A special DFA that is exactly LL(1) or LL(1) with backtracking mode
* predicates to resolve edge set collisions.
*/
public class LL1DFA extends DFA {
/** From list of lookahead sets (one per alt in decision), create
* an LL(1) DFA. One edge per set.
*
* s0-{alt1}->:o=>1
* | \
* | -{alt2}->:o=>2
* |
* ...
*/
public LL1DFA(int decisionNumber, NFAState decisionStartState, LookaheadSet[] altLook) {
DFAState s0 = newState();
startState = s0;
nfa = decisionStartState.nfa;
nAlts = nfa.grammar.getNumberOfAltsForDecisionNFA(decisionStartState);
this.decisionNumber = decisionNumber;
this.decisionNFAStartState = decisionStartState;
initAltRelatedInfo();
unreachableAlts = null;
for (int alt=1; alt<altLook.length; alt++) {
DFAState acceptAltState = newState();
acceptAltState.acceptState = true;
setAcceptState(alt, acceptAltState);
acceptAltState.k = 1;
acceptAltState.cachedUniquelyPredicatedAlt = alt;
Label e = getLabelForSet(altLook[alt].tokenTypeSet);
s0.addTransition(acceptAltState, e);
}
}
/** From a set of edgeset->list-of-alts mappings, create a DFA
* that uses syn preds for all |list-of-alts|>1.
*/
public LL1DFA(int decisionNumber,
NFAState decisionStartState,
MultiMap<IntervalSet, Integer> edgeMap)
{
DFAState s0 = newState();
startState = s0;
nfa = decisionStartState.nfa;
nAlts = nfa.grammar.getNumberOfAltsForDecisionNFA(decisionStartState);
this.decisionNumber = decisionNumber;
this.decisionNFAStartState = decisionStartState;
initAltRelatedInfo();
unreachableAlts = null;
for (Iterator it = edgeMap.keySet().iterator(); it.hasNext();) {
IntervalSet edge = (IntervalSet)it.next();
List<Integer> alts = edgeMap.get(edge);
Collections.sort(alts); // make sure alts are attempted in order
//System.out.println(edge+" -> "+alts);
DFAState s = newState();
s.k = 1;
Label e = getLabelForSet(edge);
s0.addTransition(s, e);
if ( alts.size()==1 ) {
s.acceptState = true;
int alt = alts.get(0);
setAcceptState(alt, s);
s.cachedUniquelyPredicatedAlt = alt;
}
else {
// resolve with syntactic predicates. Add edges from
// state s that test predicates.
s.resolvedWithPredicates = true;
for (int i = 0; i < alts.size(); i++) {
int alt = (int)alts.get(i);
s.cachedUniquelyPredicatedAlt = NFA.INVALID_ALT_NUMBER;
DFAState predDFATarget = getAcceptState(alt);
if ( predDFATarget==null ) {
predDFATarget = newState(); // create if not there.
predDFATarget.acceptState = true;
predDFATarget.cachedUniquelyPredicatedAlt = alt;
setAcceptState(alt, predDFATarget);
}
// add a transition to pred target from d
/*
int walkAlt =
decisionStartState.translateDisplayAltToWalkAlt(alt);
NFAState altLeftEdge = nfa.grammar.getNFAStateForAltOfDecision(decisionStartState, walkAlt);
NFAState altStartState = (NFAState)altLeftEdge.transition[0].target;
SemanticContext ctx = nfa.grammar.ll1Analyzer.getPredicates(altStartState);
System.out.println("sem ctx = "+ctx);
if ( ctx == null ) {
ctx = new SemanticContext.TruePredicate();
}
s.addTransition(predDFATarget, new Label(ctx));
*/
SemanticContext.Predicate synpred =
getSynPredForAlt(decisionStartState, alt);
if ( synpred == null ) {
synpred = new SemanticContext.TruePredicate();
}
s.addTransition(predDFATarget, new PredicateLabel(synpred));
}
}
}
//System.out.println("dfa for preds=\n"+this);
}
protected Label getLabelForSet(IntervalSet edgeSet) {
Label e = null;
int atom = edgeSet.getSingleElement();
if ( atom != Label.INVALID ) {
e = new Label(atom);
}
else {
e = new Label(edgeSet);
}
return e;
}
protected SemanticContext.Predicate getSynPredForAlt(NFAState decisionStartState,
int alt)
{
int walkAlt =
decisionStartState.translateDisplayAltToWalkAlt(alt);
NFAState altLeftEdge =
nfa.grammar.getNFAStateForAltOfDecision(decisionStartState, walkAlt);
NFAState altStartState = (NFAState)altLeftEdge.transition[0].target;
//System.out.println("alt "+alt+" start state = "+altStartState.stateNumber);
if ( altStartState.transition[0].isSemanticPredicate() ) {
SemanticContext ctx = altStartState.transition[0].label.getSemanticContext();
if ( ctx.isSyntacticPredicate() ) {
SemanticContext.Predicate p = (SemanticContext.Predicate)ctx;
if ( p.predicateAST.getType() == ANTLRParser.BACKTRACK_SEMPRED ) {
/*
System.out.println("syn pred for alt "+walkAlt+" "+
((SemanticContext.Predicate)altStartState.transition[0].label.getSemanticContext()).predicateAST);
*/
if ( ctx.isSyntacticPredicate() ) {
nfa.grammar.synPredUsedInDFA(this, ctx);
}
return (SemanticContext.Predicate)altStartState.transition[0].label.getSemanticContext();
}
}
}
return null;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/State.java 0000644 0001750 0001750 00000004060 11256465230 023526 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
/** A generic state machine state. */
public abstract class State {
public static final int INVALID_STATE_NUMBER = -1;
public int stateNumber = INVALID_STATE_NUMBER;
/** An accept state is an end of rule state for lexers and
* parser grammar rules.
*/
protected boolean acceptState = false;
public abstract int getNumberOfTransitions();
public abstract void addTransition(Transition e);
public abstract Transition transition(int i);
public boolean isAcceptState() {
return acceptState;
}
public void setAcceptState(boolean acceptState) {
this.acceptState = acceptState;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/DecisionProbe.java 0000644 0001750 0001750 00000102644 11256465230 025202 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.ErrorManager;
import org.antlr.tool.Grammar;
import org.antlr.tool.GrammarAST;
import org.antlr.grammar.v2.ANTLRParser;
import org.antlr.misc.Utils;
import org.antlr.misc.MultiMap;
import java.util.*;
import antlr.Token;
/** Collection of information about what is wrong with a decision as
* discovered while building the DFA predictor.
*
* The information is collected during NFA->DFA conversion and, while
* some of this is available elsewhere, it is nice to have it all tracked
* in one spot so a great error message can be easily had. I also like
* the fact that this object tracks it all for later perusing to make an
* excellent error message instead of lots of imprecise on-the-fly warnings
* (during conversion).
*
* A decision normally only has one problem; e.g., some input sequence
* can be matched by multiple alternatives. Unfortunately, some decisions
* such as
*
* a : ( A | B ) | ( A | B ) | A ;
*
* have multiple problems. So in general, you should approach a decision
* as having multiple flaws each one uniquely identified by a DFAState.
* For example, statesWithSyntacticallyAmbiguousAltsSet tracks the set of
* all DFAStates where ANTLR has discovered a problem. Recall that a decision
* is represented internall with a DFA comprised of multiple states, each of
* which could potentially have problems.
*
* Because of this, you need to iterate over this list of DFA states. You'll
* note that most of the informational methods like
* getSampleNonDeterministicInputSequence() require a DFAState. This state
* will be one of the iterated states from stateToSyntacticallyAmbiguousAltsSet.
*
* This class is not thread safe due to shared use of visited maps etc...
* Only one thread should really need to access one DecisionProbe anyway.
*/
public class DecisionProbe {
public DFA dfa;
/** Track all DFA states with nondeterministic alternatives.
* By reaching the same DFA state, a path through the NFA for some input
* is able to reach the same NFA state by starting at more than one
* alternative's left edge. Though, later, we may find that predicates
* resolve the issue, but track info anyway.
* Note that from the DFA state, you can ask for
* which alts are nondeterministic.
*/
protected Set<DFAState> statesWithSyntacticallyAmbiguousAltsSet = new HashSet<DFAState>();
/** Track just like stateToSyntacticallyAmbiguousAltsMap, but only
* for nondeterminisms that arise in the Tokens rule such as keyword vs
* ID rule. The state maps to the list of Tokens rule alts that are
* in conflict.
*/
protected Map<DFAState, Set<Integer>> stateToSyntacticallyAmbiguousTokensRuleAltsMap =
new HashMap<DFAState, Set<Integer>>();
/** Was a syntactic ambiguity resolved with predicates? Any DFA
* state that predicts more than one alternative, must be resolved
* with predicates or it should be reported to the user.
*/
protected Set<DFAState> statesResolvedWithSemanticPredicatesSet = new HashSet<DFAState>();
/** Track the predicates for each alt per DFA state;
* more than one DFA state might have syntactically ambig alt prediction.
* Maps DFA state to another map, mapping alt number to a
* SemanticContext (pred(s) to execute to resolve syntactic ambiguity).
*/
protected Map<DFAState, Map<Integer,SemanticContext>> stateToAltSetWithSemanticPredicatesMap =
new HashMap<DFAState, Map<Integer,SemanticContext>>();
/** Tracks alts insufficiently covered.
* For example, p1||true gets reduced to true and so leaves
* whole alt uncovered. This maps DFA state to the set of alts
*/
protected Map<DFAState,Map<Integer, Set<Token>>> stateToIncompletelyCoveredAltsMap =
new HashMap<DFAState,Map<Integer, Set<Token>>>();
/** The set of states w/o emanating edges and w/o resolving sem preds. */
protected Set<DFAState> danglingStates = new HashSet<DFAState>();
/** The overall list of alts within the decision that have at least one
* conflicting input sequence.
*/
protected Set<Integer> altsWithProblem = new HashSet<Integer>();
/** If decision with > 1 alt has recursion in > 1 alt, it's nonregular
* lookahead. The decision cannot be made with a DFA.
* the alts are stored in altsWithProblem.
*/
protected boolean nonLLStarDecision = false;
/** Recursion is limited to a particular depth. If that limit is exceeded
* the proposed new NFAConfiguration is recorded for the associated DFA state.
*/
protected MultiMap<Integer, NFAConfiguration> stateToRecursionOverflowConfigurationsMap =
new MultiMap<Integer, NFAConfiguration>();
/*
protected Map<Integer, List<NFAConfiguration>> stateToRecursionOverflowConfigurationsMap =
new HashMap<Integer, List<NFAConfiguration>>();
*/
/** Left recursion discovered. The proposed new NFAConfiguration
* is recorded for the associated DFA state.
protected Map<Integer,List<NFAConfiguration>> stateToLeftRecursiveConfigurationsMap =
new HashMap<Integer,List<NFAConfiguration>>();
*/
/** Did ANTLR have to terminate early on the analysis of this decision? */
protected boolean timedOut = false;
/** Used to find paths through syntactically ambiguous DFA. If we've
* seen statement number before, what did we learn?
*/
protected Map<Integer, Integer> stateReachable;
public static final Integer REACHABLE_BUSY = Utils.integer(-1);
public static final Integer REACHABLE_NO = Utils.integer(0);
public static final Integer REACHABLE_YES = Utils.integer(1);
/** Used while finding a path through an NFA whose edge labels match
* an input sequence. Tracks the input position
* we were at the last time at this node. If same input position, then
* we'd have reached same state without consuming input...probably an
* infinite loop. Stop. Set<String>. The strings look like
* stateNumber_labelIndex.
*/
protected Set<String> statesVisitedAtInputDepth;
protected Set<Integer> statesVisitedDuringSampleSequence;
public static boolean verbose = false;
public DecisionProbe(DFA dfa) {
this.dfa = dfa;
}
// I N F O R M A T I O N A B O U T D E C I S I O N
/** Return a string like "3:22: ( A {;} | B )" that describes this
* decision.
*/
public String getDescription() {
return dfa.getNFADecisionStartState().getDescription();
}
public boolean isReduced() {
return dfa.isReduced();
}
public boolean isCyclic() {
return dfa.isCyclic();
}
/** If no states are dead-ends, no alts are unreachable, there are
* no nondeterminisms unresolved by syn preds, all is ok with decision.
*/
public boolean isDeterministic() {
if ( danglingStates.size()==0 &&
statesWithSyntacticallyAmbiguousAltsSet.size()==0 &&
dfa.getUnreachableAlts().size()==0 )
{
return true;
}
if ( statesWithSyntacticallyAmbiguousAltsSet.size()>0 ) {
Iterator it =
statesWithSyntacticallyAmbiguousAltsSet.iterator();
while ( it.hasNext() ) {
DFAState d = (DFAState) it.next();
if ( !statesResolvedWithSemanticPredicatesSet.contains(d) ) {
return false;
}
}
// no syntactically ambig alts were left unresolved by predicates
return true;
}
return false;
}
/** Did the analysis complete it's work? */
public boolean analysisTimedOut() {
return timedOut;
}
/** Took too long to analyze a DFA */
public boolean analysisOverflowed() {
return stateToRecursionOverflowConfigurationsMap.size()>0;
}
/** Found recursion in > 1 alt */
public boolean isNonLLStarDecision() {
return nonLLStarDecision;
}
/** How many states does the DFA predictor have? */
public int getNumberOfStates() {
return dfa.getNumberOfStates();
}
/** Get a list of all unreachable alternatives for this decision. There
* may be multiple alternatives with ambiguous input sequences, but this
* is the overall list of unreachable alternatives (either due to
* conflict resolution or alts w/o accept states).
*/
public List<Integer> getUnreachableAlts() {
return dfa.getUnreachableAlts();
}
/** return set of states w/o emanating edges and w/o resolving sem preds.
* These states come about because the analysis algorithm had to
* terminate early to avoid infinite recursion for example (due to
* left recursion perhaps).
*/
public Set getDanglingStates() {
return danglingStates;
}
public Set getNonDeterministicAlts() {
return altsWithProblem;
}
/** Return the sorted list of alts that conflict within a single state.
* Note that predicates may resolve the conflict.
*/
public List getNonDeterministicAltsForState(DFAState targetState) {
Set nondetAlts = targetState.getNonDeterministicAlts();
if ( nondetAlts==null ) {
return null;
}
List sorted = new LinkedList();
sorted.addAll(nondetAlts);
Collections.sort(sorted); // make sure it's 1, 2, ...
return sorted;
}
/** Return all DFA states in this DFA that have NFA configurations that
* conflict. You must report a problem for each state in this set
* because each state represents a different input sequence.
*/
public Set getDFAStatesWithSyntacticallyAmbiguousAlts() {
return statesWithSyntacticallyAmbiguousAltsSet;
}
/** Which alts were specifically turned off to resolve nondeterminisms?
* This is different than the unreachable alts. Disabled doesn't mean that
* the alternative is totally unreachable necessarily, it just means
* that for this DFA state, that alt is disabled. There may be other
* accept states for that alt that make an alt reachable.
*/
public Set getDisabledAlternatives(DFAState d) {
return d.getDisabledAlternatives();
}
/** If a recursion overflow is resolve with predicates, then we need
* to shut off the warning that would be generated.
*/
public void removeRecursiveOverflowState(DFAState d) {
Integer stateI = Utils.integer(d.stateNumber);
stateToRecursionOverflowConfigurationsMap.remove(stateI);
}
/** Return a List<Label> indicating an input sequence that can be matched
* from the start state of the DFA to the targetState (which is known
* to have a problem).
*/
public List<Label> getSampleNonDeterministicInputSequence(DFAState targetState) {
Set dfaStates = getDFAPathStatesToTarget(targetState);
statesVisitedDuringSampleSequence = new HashSet<Integer>();
List<Label> labels = new ArrayList<Label>(); // may access ith element; use array
if ( dfa==null || dfa.startState==null ) {
return labels;
}
getSampleInputSequenceUsingStateSet(dfa.startState,
targetState,
dfaStates,
labels);
return labels;
}
/** Given List<Label>, return a String with a useful representation
* of the associated input string. One could show something different
* for lexers and parsers, for example.
*/
public String getInputSequenceDisplay(List labels) {
Grammar g = dfa.nfa.grammar;
StringBuffer buf = new StringBuffer();
for (Iterator it = labels.iterator(); it.hasNext();) {
Label label = (Label) it.next();
buf.append(label.toString(g));
if ( it.hasNext() && g.type!=Grammar.LEXER ) {
buf.append(' ');
}
}
return buf.toString();
}
/** Given an alternative associated with a nondeterministic DFA state,
* find the path of NFA states associated with the labels sequence.
* Useful tracing where in the NFA, a single input sequence can be
* matched. For different alts, you should get different NFA paths.
*
* The first NFA state for all NFA paths will be the same: the starting
* NFA state of the first nondeterministic alt. Imagine (A|B|A|A):
*
* 5->9-A->o
* |
* 6->10-B->o
* |
* 7->11-A->o
* |
* 8->12-A->o
*
* There are 3 nondeterministic alts. The paths should be:
* 5 9 ...
* 5 6 7 11 ...
* 5 6 7 8 12 ...
*
* The NFA path matching the sample input sequence (labels) is computed
* using states 9, 11, and 12 rather than 5, 7, 8 because state 5, for
* example can get to all ambig paths. Must isolate for each alt (hence,
* the extra state beginning each alt in my NFA structures). Here,
* firstAlt=1.
*/
public List getNFAPathStatesForAlt(int firstAlt,
int alt,
List labels)
{
NFAState nfaStart = dfa.getNFADecisionStartState();
List path = new LinkedList();
// first add all NFA states leading up to altStart state
for (int a=firstAlt; a<=alt; a++) {
NFAState s =
dfa.nfa.grammar.getNFAStateForAltOfDecision(nfaStart,a);
path.add(s);
}
// add first state of actual alt
NFAState altStart = dfa.nfa.grammar.getNFAStateForAltOfDecision(nfaStart,alt);
NFAState isolatedAltStart = (NFAState)altStart.transition[0].target;
path.add(isolatedAltStart);
// add the actual path now
statesVisitedAtInputDepth = new HashSet();
getNFAPath(isolatedAltStart,
0,
labels,
path);
return path;
}
/** Each state in the DFA represents a different input sequence for an
* alt of the decision. Given a DFA state, what is the semantic
* predicate context for a particular alt.
*/
public SemanticContext getSemanticContextForAlt(DFAState d, int alt) {
Map altToPredMap = (Map)stateToAltSetWithSemanticPredicatesMap.get(d);
if ( altToPredMap==null ) {
return null;
}
return (SemanticContext)altToPredMap.get(Utils.integer(alt));
}
/** At least one alt refs a sem or syn pred */
public boolean hasPredicate() {
return stateToAltSetWithSemanticPredicatesMap.size()>0;
}
public Set getNondeterministicStatesResolvedWithSemanticPredicate() {
return statesResolvedWithSemanticPredicatesSet;
}
/** Return a list of alts whose predicate context was insufficient to
* resolve a nondeterminism for state d.
*/
public Map<Integer, Set<Token>> getIncompletelyCoveredAlts(DFAState d) {
return stateToIncompletelyCoveredAltsMap.get(d);
}
public void issueWarnings() {
// NONREGULAR DUE TO RECURSION > 1 ALTS
// Issue this before aborted analysis, which might also occur
// if we take too long to terminate
if ( nonLLStarDecision && !dfa.getAutoBacktrackMode() ) {
ErrorManager.nonLLStarDecision(this);
}
if ( analysisTimedOut() ) {
// only report early termination errors if !backtracking
if ( !dfa.getAutoBacktrackMode() ) {
ErrorManager.analysisAborted(this);
}
// now just return...if we bailed out, don't spew other messages
return;
}
issueRecursionWarnings();
// generate a separate message for each problem state in DFA
Set resolvedStates = getNondeterministicStatesResolvedWithSemanticPredicate();
Set problemStates = getDFAStatesWithSyntacticallyAmbiguousAlts();
if ( problemStates.size()>0 ) {
Iterator it =
problemStates.iterator();
while ( it.hasNext() && !dfa.nfa.grammar.NFAToDFAConversionExternallyAborted() ) {
DFAState d = (DFAState) it.next();
Map<Integer, Set<Token>> insufficientAltToLocations = getIncompletelyCoveredAlts(d);
if ( insufficientAltToLocations!=null && insufficientAltToLocations.size()>0 ) {
ErrorManager.insufficientPredicates(this,d,insufficientAltToLocations);
}
// don't report problem if resolved
if ( resolvedStates==null || !resolvedStates.contains(d) ) {
// first strip last alt from disableAlts if it's wildcard
// then don't print error if no more disable alts
Set disabledAlts = getDisabledAlternatives(d);
stripWildCardAlts(disabledAlts);
if ( disabledAlts.size()>0 ) {
ErrorManager.nondeterminism(this,d);
}
}
}
}
Set danglingStates = getDanglingStates();
if ( danglingStates.size()>0 ) {
//System.err.println("no emanating edges for states: "+danglingStates);
for (Iterator it = danglingStates.iterator(); it.hasNext();) {
DFAState d = (DFAState) it.next();
ErrorManager.danglingState(this,d);
}
}
if ( !nonLLStarDecision ) {
List<Integer> unreachableAlts = dfa.getUnreachableAlts();
if ( unreachableAlts!=null && unreachableAlts.size()>0 ) {
// give different msg if it's an empty Tokens rule from delegate
boolean isInheritedTokensRule = false;
if ( dfa.isTokensRuleDecision() ) {
for (Integer altI : unreachableAlts) {
GrammarAST decAST = dfa.getDecisionASTNode();
GrammarAST altAST = decAST.getChild(altI-1);
GrammarAST delegatedTokensAlt =
altAST.getFirstChildWithType(ANTLRParser.DOT);
if ( delegatedTokensAlt !=null ) {
isInheritedTokensRule = true;
ErrorManager.grammarWarning(ErrorManager.MSG_IMPORTED_TOKENS_RULE_EMPTY,
dfa.nfa.grammar,
null,
dfa.nfa.grammar.name,
delegatedTokensAlt.getFirstChild().getText());
}
}
}
if ( isInheritedTokensRule ) {
}
else {
ErrorManager.unreachableAlts(this,unreachableAlts);
}
}
}
}
/** Get the last disabled alt number and check in the grammar to see
* if that alt is a simple wildcard. If so, treat like an else clause
* and don't emit the error. Strip out the last alt if it's wildcard.
*/
protected void stripWildCardAlts(Set disabledAlts) {
List sortedDisableAlts = new ArrayList(disabledAlts);
Collections.sort(sortedDisableAlts);
Integer lastAlt =
(Integer)sortedDisableAlts.get(sortedDisableAlts.size()-1);
GrammarAST blockAST =
dfa.nfa.grammar.getDecisionBlockAST(dfa.decisionNumber);
//System.out.println("block with error = "+blockAST.toStringTree());
GrammarAST lastAltAST = null;
if ( blockAST.getChild(0).getType()==ANTLRParser.OPTIONS ) {
// if options, skip first child: ( options { ( = greedy false ) )
lastAltAST = blockAST.getChild(lastAlt.intValue());
}
else {
lastAltAST = blockAST.getChild(lastAlt.intValue()-1);
}
//System.out.println("last alt is "+lastAltAST.toStringTree());
// if last alt looks like ( ALT . <end-of-alt> ) then wildcard
// Avoid looking at optional blocks etc... that have last alt
// as the EOB:
// ( BLOCK ( ALT 'else' statement <end-of-alt> ) <end-of-block> )
if ( lastAltAST.getType()!=ANTLRParser.EOB &&
lastAltAST.getChild(0).getType()== ANTLRParser.WILDCARD &&
lastAltAST.getChild(1).getType()== ANTLRParser.EOA )
{
//System.out.println("wildcard");
disabledAlts.remove(lastAlt);
}
}
protected void issueRecursionWarnings() {
// RECURSION OVERFLOW
Set dfaStatesWithRecursionProblems =
stateToRecursionOverflowConfigurationsMap.keySet();
// now walk truly unique (unaliased) list of dfa states with inf recur
// Goal: create a map from alt to map<target,List<callsites>>
// Map<Map<String target, List<NFAState call sites>>
Map altToTargetToCallSitesMap = new HashMap();
// track a single problem DFA state for each alt
Map altToDFAState = new HashMap();
computeAltToProblemMaps(dfaStatesWithRecursionProblems,
stateToRecursionOverflowConfigurationsMap,
altToTargetToCallSitesMap, // output param
altToDFAState); // output param
// walk each alt with recursion overflow problems and generate error
Set alts = altToTargetToCallSitesMap.keySet();
List sortedAlts = new ArrayList(alts);
Collections.sort(sortedAlts);
for (Iterator altsIt = sortedAlts.iterator(); altsIt.hasNext();) {
Integer altI = (Integer) altsIt.next();
Map targetToCallSiteMap =
(Map)altToTargetToCallSitesMap.get(altI);
Set targetRules = targetToCallSiteMap.keySet();
Collection callSiteStates = targetToCallSiteMap.values();
DFAState sampleBadState = (DFAState)altToDFAState.get(altI);
ErrorManager.recursionOverflow(this,
sampleBadState,
altI.intValue(),
targetRules,
callSiteStates);
}
}
private void computeAltToProblemMaps(Set dfaStatesUnaliased,
Map configurationsMap,
Map altToTargetToCallSitesMap,
Map altToDFAState)
{
for (Iterator it = dfaStatesUnaliased.iterator(); it.hasNext();) {
Integer stateI = (Integer) it.next();
// walk this DFA's config list
List configs = (List)configurationsMap.get(stateI);
for (int i = 0; i < configs.size(); i++) {
NFAConfiguration c = (NFAConfiguration) configs.get(i);
NFAState ruleInvocationState = dfa.nfa.getState(c.state);
Transition transition0 = ruleInvocationState.transition[0];
RuleClosureTransition ref = (RuleClosureTransition)transition0;
String targetRule = ((NFAState) ref.target).enclosingRule.name;
Integer altI = Utils.integer(c.alt);
Map targetToCallSiteMap =
(Map)altToTargetToCallSitesMap.get(altI);
if ( targetToCallSiteMap==null ) {
targetToCallSiteMap = new HashMap();
altToTargetToCallSitesMap.put(altI, targetToCallSiteMap);
}
Set callSites =
(HashSet)targetToCallSiteMap.get(targetRule);
if ( callSites==null ) {
callSites = new HashSet();
targetToCallSiteMap.put(targetRule, callSites);
}
callSites.add(ruleInvocationState);
// track one problem DFA state per alt
if ( altToDFAState.get(altI)==null ) {
DFAState sampleBadState = dfa.getState(stateI.intValue());
altToDFAState.put(altI, sampleBadState);
}
}
}
}
private Set getUnaliasedDFAStateSet(Set dfaStatesWithRecursionProblems) {
Set dfaStatesUnaliased = new HashSet();
for (Iterator it = dfaStatesWithRecursionProblems.iterator(); it.hasNext();) {
Integer stateI = (Integer) it.next();
DFAState d = dfa.getState(stateI.intValue());
dfaStatesUnaliased.add(Utils.integer(d.stateNumber));
}
return dfaStatesUnaliased;
}
// T R A C K I N G M E T H O D S
/** Report the fact that DFA state d is not a state resolved with
* predicates and yet it has no emanating edges. Usually this
* is a result of the closure/reach operations being unable to proceed
*/
public void reportDanglingState(DFAState d) {
danglingStates.add(d);
}
public void reportAnalysisTimeout() {
timedOut = true;
dfa.nfa.grammar.setOfDFAWhoseAnalysisTimedOut.add(dfa);
}
/** Report that at least 2 alts have recursive constructs. There is
* no way to build a DFA so we terminated.
*/
public void reportNonLLStarDecision(DFA dfa) {
/*
System.out.println("non-LL(*) DFA "+dfa.decisionNumber+", alts: "+
dfa.recursiveAltSet.toList());
*/
nonLLStarDecision = true;
altsWithProblem.addAll(dfa.recursiveAltSet.toList());
}
public void reportRecursionOverflow(DFAState d,
NFAConfiguration recursionNFAConfiguration)
{
// track the state number rather than the state as d will change
// out from underneath us; hash wouldn't return any value
// left-recursion is detected in start state. Since we can't
// call resolveNondeterminism() on the start state (it would
// not look k=1 to get min single token lookahead), we must
// prevent errors derived from this state. Avoid start state
if ( d.stateNumber > 0 ) {
Integer stateI = Utils.integer(d.stateNumber);
stateToRecursionOverflowConfigurationsMap.map(stateI, recursionNFAConfiguration);
}
}
public void reportNondeterminism(DFAState d, Set<Integer> nondeterministicAlts) {
altsWithProblem.addAll(nondeterministicAlts); // track overall list
statesWithSyntacticallyAmbiguousAltsSet.add(d);
dfa.nfa.grammar.setOfNondeterministicDecisionNumbers.add(
Utils.integer(dfa.getDecisionNumber())
);
}
/** Currently the analysis reports issues between token definitions, but
* we don't print out warnings in favor of just picking the first token
* definition found in the grammar ala lex/flex.
*/
public void reportLexerRuleNondeterminism(DFAState d, Set<Integer> nondeterministicAlts) {
stateToSyntacticallyAmbiguousTokensRuleAltsMap.put(d,nondeterministicAlts);
}
public void reportNondeterminismResolvedWithSemanticPredicate(DFAState d) {
// First, prevent a recursion warning on this state due to
// pred resolution
if ( d.abortedDueToRecursionOverflow ) {
d.dfa.probe.removeRecursiveOverflowState(d);
}
statesResolvedWithSemanticPredicatesSet.add(d);
//System.out.println("resolved with pred: "+d);
dfa.nfa.grammar.setOfNondeterministicDecisionNumbersResolvedWithPredicates.add(
Utils.integer(dfa.getDecisionNumber())
);
}
/** Report the list of predicates found for each alternative; copy
* the list because this set gets altered later by the method
* tryToResolveWithSemanticPredicates() while flagging NFA configurations
* in d as resolved.
*/
public void reportAltPredicateContext(DFAState d, Map altPredicateContext) {
Map copy = new HashMap();
copy.putAll(altPredicateContext);
stateToAltSetWithSemanticPredicatesMap.put(d,copy);
}
public void reportIncompletelyCoveredAlts(DFAState d,
Map<Integer, Set<Token>> altToLocationsReachableWithoutPredicate)
{
stateToIncompletelyCoveredAltsMap.put(d, altToLocationsReachableWithoutPredicate);
}
// S U P P O R T
/** Given a start state and a target state, return true if start can reach
* target state. Also, compute the set of DFA states
* that are on a path from start to target; return in states parameter.
*/
protected boolean reachesState(DFAState startState,
DFAState targetState,
Set states) {
if ( startState==targetState ) {
states.add(targetState);
//System.out.println("found target DFA state "+targetState.getStateNumber());
stateReachable.put(startState.stateNumber, REACHABLE_YES);
return true;
}
DFAState s = startState;
// avoid infinite loops
stateReachable.put(s.stateNumber, REACHABLE_BUSY);
// look for a path to targetState among transitions for this state
// stop when you find the first one; I'm pretty sure there is
// at most one path to any DFA state with conflicting predictions
for (int i=0; i<s.getNumberOfTransitions(); i++) {
Transition t = s.transition(i);
DFAState edgeTarget = (DFAState)t.target;
Integer targetStatus = stateReachable.get(edgeTarget.stateNumber);
if ( targetStatus==REACHABLE_BUSY ) { // avoid cycles; they say nothing
continue;
}
if ( targetStatus==REACHABLE_YES ) { // return success!
stateReachable.put(s.stateNumber, REACHABLE_YES);
return true;
}
if ( targetStatus==REACHABLE_NO ) { // try another transition
continue;
}
// if null, target must be REACHABLE_UNKNOWN (i.e., unvisited)
if ( reachesState(edgeTarget, targetState, states) ) {
states.add(s);
stateReachable.put(s.stateNumber, REACHABLE_YES);
return true;
}
}
stateReachable.put(s.stateNumber, REACHABLE_NO);
return false; // no path to targetState found.
}
protected Set getDFAPathStatesToTarget(DFAState targetState) {
Set dfaStates = new HashSet();
stateReachable = new HashMap();
if ( dfa==null || dfa.startState==null ) {
return dfaStates;
}
boolean reaches = reachesState(dfa.startState, targetState, dfaStates);
return dfaStates;
}
/** Given a start state and a final state, find a list of edge labels
* between the two ignoring epsilon. Limit your scan to a set of states
* passed in. This is used to show a sample input sequence that is
* nondeterministic with respect to this decision. Return List<Label> as
* a parameter. The incoming states set must be all states that lead
* from startState to targetState and no others so this algorithm doesn't
* take a path that eventually leads to a state other than targetState.
* Don't follow loops, leading to short (possibly shortest) path.
*/
protected void getSampleInputSequenceUsingStateSet(State startState,
State targetState,
Set states,
List<Label> labels)
{
statesVisitedDuringSampleSequence.add(startState.stateNumber);
// pick the first edge in states as the one to traverse
for (int i=0; i<startState.getNumberOfTransitions(); i++) {
Transition t = startState.transition(i);
DFAState edgeTarget = (DFAState)t.target;
if ( states.contains(edgeTarget) &&
!statesVisitedDuringSampleSequence.contains(edgeTarget.stateNumber) )
{
labels.add(t.label); // traverse edge and track label
if ( edgeTarget!=targetState ) {
// get more labels if not at target
getSampleInputSequenceUsingStateSet(edgeTarget,
targetState,
states,
labels);
}
// done with this DFA state as we've found a good path to target
return;
}
}
labels.add(new Label(Label.EPSILON)); // indicate no input found
// this happens on a : {p1}? a | A ;
//ErrorManager.error(ErrorManager.MSG_CANNOT_COMPUTE_SAMPLE_INPUT_SEQ);
}
/** Given a sample input sequence, you usually would like to know the
* path taken through the NFA. Return the list of NFA states visited
* while matching a list of labels. This cannot use the usual
* interpreter, which does a deterministic walk. We need to be able to
* take paths that are turned off during nondeterminism resolution. So,
* just do a depth-first walk traversing edges labeled with the current
* label. Return true if a path was found emanating from state s.
*/
protected boolean getNFAPath(NFAState s, // starting where?
int labelIndex, // 0..labels.size()-1
List labels, // input sequence
List path) // output list of NFA states
{
// track a visit to state s at input index labelIndex if not seen
String thisStateKey = getStateLabelIndexKey(s.stateNumber,labelIndex);
if ( statesVisitedAtInputDepth.contains(thisStateKey) ) {
/*
System.out.println("### already visited "+s.stateNumber+" previously at index "+
labelIndex);
*/
return false;
}
statesVisitedAtInputDepth.add(thisStateKey);
/*
System.out.println("enter state "+s.stateNumber+" visited states: "+
statesVisitedAtInputDepth);
*/
// pick the first edge whose target is in states and whose
// label is labels[labelIndex]
for (int i=0; i<s.getNumberOfTransitions(); i++) {
Transition t = s.transition[i];
NFAState edgeTarget = (NFAState)t.target;
Label label = (Label)labels.get(labelIndex);
/*
System.out.println(s.stateNumber+"-"+
t.label.toString(dfa.nfa.grammar)+"->"+
edgeTarget.stateNumber+" =="+
label.toString(dfa.nfa.grammar)+"?");
*/
if ( t.label.isEpsilon() || t.label.isSemanticPredicate() ) {
// nondeterministically backtrack down epsilon edges
path.add(edgeTarget);
boolean found =
getNFAPath(edgeTarget, labelIndex, labels, path);
if ( found ) {
statesVisitedAtInputDepth.remove(thisStateKey);
return true; // return to "calling" state
}
path.remove(path.size()-1); // remove; didn't work out
continue; // look at the next edge
}
if ( t.label.matches(label) ) {
path.add(edgeTarget);
/*
System.out.println("found label "+
t.label.toString(dfa.nfa.grammar)+
" at state "+s.stateNumber+"; labelIndex="+labelIndex);
*/
if ( labelIndex==labels.size()-1 ) {
// found last label; done!
statesVisitedAtInputDepth.remove(thisStateKey);
return true;
}
// otherwise try to match remaining input
boolean found =
getNFAPath(edgeTarget, labelIndex+1, labels, path);
if ( found ) {
statesVisitedAtInputDepth.remove(thisStateKey);
return true;
}
/*
System.out.println("backtrack; path from "+s.stateNumber+"->"+
t.label.toString(dfa.nfa.grammar)+" didn't work");
*/
path.remove(path.size()-1); // remove; didn't work out
continue; // keep looking for a path for labels
}
}
//System.out.println("no epsilon or matching edge; removing "+thisStateKey);
// no edge was found matching label; is ok, some state will have it
statesVisitedAtInputDepth.remove(thisStateKey);
return false;
}
protected String getStateLabelIndexKey(int s, int i) {
StringBuffer buf = new StringBuffer();
buf.append(s);
buf.append('_');
buf.append(i);
return buf.toString();
}
/** From an alt number associated with artificial Tokens rule, return
* the name of the token that is associated with that alt.
*/
public String getTokenNameForTokensRuleAlt(int alt) {
NFAState decisionState = dfa.getNFADecisionStartState();
NFAState altState =
dfa.nfa.grammar.getNFAStateForAltOfDecision(decisionState,alt);
NFAState decisionLeft = (NFAState)altState.transition[0].target;
RuleClosureTransition ruleCallEdge =
(RuleClosureTransition)decisionLeft.transition[0];
NFAState ruleStartState = (NFAState)ruleCallEdge.target;
//System.out.println("alt = "+decisionLeft.getEnclosingRule());
return ruleStartState.enclosingRule.name;
}
public void reset() {
stateToRecursionOverflowConfigurationsMap.clear();
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/PredicateLabel.java 0000644 0001750 0001750 00000005453 11256465230 025315 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.GrammarAST;
import org.antlr.tool.Grammar;
public class PredicateLabel extends Label {
/** A tree of semantic predicates from the grammar AST if label==SEMPRED.
* In the NFA, labels will always be exactly one predicate, but the DFA
* may have to combine a bunch of them as it collects predicates from
* multiple NFA configurations into a single DFA state.
*/
protected SemanticContext semanticContext;
/** Make a semantic predicate label */
public PredicateLabel(GrammarAST predicateASTNode) {
super(SEMPRED);
this.semanticContext = new SemanticContext.Predicate(predicateASTNode);
}
/** Make a semantic predicates label */
public PredicateLabel(SemanticContext semCtx) {
super(SEMPRED);
this.semanticContext = semCtx;
}
public int hashCode() {
return semanticContext.hashCode();
}
public boolean equals(Object o) {
if ( o==null ) {
return false;
}
if ( this == o ) {
return true; // equals if same object
}
if ( !(o instanceof PredicateLabel) ) {
return false;
}
return semanticContext.equals(((PredicateLabel)o).semanticContext);
}
public boolean isSemanticPredicate() {
return true;
}
public SemanticContext getSemanticContext() {
return semanticContext;
}
public String toString() {
return "{"+semanticContext+"}?";
}
public String toString(Grammar g) {
return toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/NFAState.java 0000644 0001750 0001750 00000021204 11256465230 024052 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.GrammarAST;
import org.antlr.tool.Rule;
import org.antlr.tool.ErrorManager;
/** A state within an NFA. At most 2 transitions emanate from any NFA state. */
public class NFAState extends State {
// I need to distinguish between NFA decision states for (...)* and (...)+
// during NFA interpretation.
public static final int LOOPBACK = 1;
public static final int BLOCK_START = 2;
public static final int OPTIONAL_BLOCK_START = 3;
public static final int BYPASS = 4;
public static final int RIGHT_EDGE_OF_BLOCK = 5;
public static final int MAX_TRANSITIONS = 2;
/** How many transitions; 0, 1, or 2 transitions */
int numTransitions = 0;
public Transition[] transition = new Transition[MAX_TRANSITIONS];
/** For o-A->o type NFA tranitions, record the label that leads to this
* state. Useful for creating rich error messages when we find
* insufficiently (with preds) covered states.
*/
public Label incidentEdgeLabel;
/** Which NFA are we in? */
public NFA nfa = null;
/** What's its decision number from 1..n? */
protected int decisionNumber = 0;
/** Subrules (...)* and (...)+ have more than one decision point in
* the NFA created for them. They both have a loop-exit-or-stay-in
* decision node (the loop back node). They both have a normal
* alternative block decision node at the left edge. The (...)* is
* worse as it even has a bypass decision (2 alts: stay in or bypass)
* node at the extreme left edge. This is not how they get generated
* in code as a while-loop or whatever deals nicely with either. For
* error messages (where I need to print the nondeterministic alts)
* and for interpretation, I need to use the single DFA that is created
* (for efficiency) but interpret the results differently depending
* on which of the 2 or 3 decision states uses the DFA. For example,
* the DFA will always report alt n+1 as the exit branch for n real
* alts, so I need to translate that depending on the decision state.
*
* If decisionNumber>0 then this var tells you what kind of decision
* state it is.
*/
public int decisionStateType;
/** What rule do we live in? */
public Rule enclosingRule;
/** During debugging and for nondeterminism warnings, it's useful
* to know what relationship this node has to the original grammar.
* For example, "start of alt 1 of rule a".
*/
protected String description;
/** Associate this NFAState with the corresponding GrammarAST node
* from which this node was created. This is useful not only for
* associating the eventual lookahead DFA with the associated
* Grammar position, but also for providing users with
* nondeterminism warnings. Mainly used by decision states to
* report line:col info. Could also be used to track line:col
* for elements such as token refs.
*/
public GrammarAST associatedASTNode;
/** Is this state the sole target of an EOT transition? */
protected boolean EOTTargetState = false;
/** Jean Bovet needs in the GUI to know which state pairs correspond
* to the start/stop of a block.
*/
public int endOfBlockStateNumber = State.INVALID_STATE_NUMBER;
public NFAState(NFA nfa) {
this.nfa = nfa;
}
public int getNumberOfTransitions() {
return numTransitions;
}
public void addTransition(Transition e) {
if ( e==null ) {
throw new IllegalArgumentException("You can't add a null transition");
}
if ( numTransitions>transition.length ) {
throw new IllegalArgumentException("You can only have "+transition.length+" transitions");
}
if ( e!=null ) {
transition[numTransitions] = e;
numTransitions++;
// Set the "back pointer" of the target state so that it
// knows about the label of the incoming edge.
Label label = e.label;
if ( label.isAtom() || label.isSet() ) {
if ( ((NFAState)e.target).incidentEdgeLabel!=null ) {
ErrorManager.internalError("Clobbered incident edge");
}
((NFAState)e.target).incidentEdgeLabel = e.label;
}
}
}
/** Used during optimization to reset a state to have the (single)
* transition another state has.
*/
public void setTransition0(Transition e) {
if ( e==null ) {
throw new IllegalArgumentException("You can't use a solitary null transition");
}
transition[0] = e;
transition[1] = null;
numTransitions = 1;
}
public Transition transition(int i) {
return transition[i];
}
/** The DFA decision for this NFA decision state always has
* an exit path for loops as n+1 for n alts in the loop.
* That is really useful for displaying nondeterministic alts
* and so on, but for walking the NFA to get a sequence of edge
* labels or for actually parsing, we need to get the real alt
* number. The real alt number for exiting a loop is always 1
* as transition 0 points at the exit branch (we compute DFAs
* always for loops at the loopback state).
*
* For walking/parsing the loopback state:
* 1 2 3 display alt (for human consumption)
* 2 3 1 walk alt
*
* For walking the block start:
* 1 2 3 display alt
* 1 2 3
*
* For walking the bypass state of a (...)* loop:
* 1 2 3 display alt
* 1 1 2 all block alts map to entering loop exit means take bypass
*
* Non loop EBNF do not need to be translated; they are ignored by
* this method as decisionStateType==0.
*
* Return same alt if we can't translate.
*/
public int translateDisplayAltToWalkAlt(int displayAlt) {
NFAState nfaStart = this;
if ( decisionNumber==0 || decisionStateType==0 ) {
return displayAlt;
}
int walkAlt = 0;
// find the NFA loopback state associated with this DFA
// and count number of alts (all alt numbers are computed
// based upon the loopback's NFA state.
/*
DFA dfa = nfa.grammar.getLookaheadDFA(decisionNumber);
if ( dfa==null ) {
ErrorManager.internalError("can't get DFA for decision "+decisionNumber);
}
*/
int nAlts = nfa.grammar.getNumberOfAltsForDecisionNFA(nfaStart);
switch ( nfaStart.decisionStateType ) {
case LOOPBACK :
walkAlt = displayAlt % nAlts + 1; // rotate right mod 1..3
break;
case BLOCK_START :
case OPTIONAL_BLOCK_START :
walkAlt = displayAlt; // identity transformation
break;
case BYPASS :
if ( displayAlt == nAlts ) {
walkAlt = 2; // bypass
}
else {
walkAlt = 1; // any non exit branch alt predicts entering
}
break;
}
return walkAlt;
}
// Setter/Getters
/** What AST node is associated with this NFAState? When you
* set the AST node, I set the node to point back to this NFA state.
*/
public void setDecisionASTNode(GrammarAST decisionASTNode) {
decisionASTNode.setNFAStartState(this);
this.associatedASTNode = decisionASTNode;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public int getDecisionNumber() {
return decisionNumber;
}
public void setDecisionNumber(int decisionNumber) {
this.decisionNumber = decisionNumber;
}
public boolean isEOTTargetState() {
return EOTTargetState;
}
public void setEOTTargetState(boolean eot) {
EOTTargetState = eot;
}
public boolean isDecisionState() {
return decisionStateType>0;
}
public String toString() {
return String.valueOf(stateNumber);
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/NonLLStarDecisionException.java 0000644 0001750 0001750 00000003402 11256465230 027616 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
/** Used to abort DFA construction when we find non-LL(*) decision; i.e.,
* a decision that has recursion in more than a single alt.
*/
public class NonLLStarDecisionException extends RuntimeException {
public DFA abortedDFA;
public NonLLStarDecisionException(DFA abortedDFA) {
this.abortedDFA = abortedDFA;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/NFA.java 0000644 0001750 0001750 00000004754 11256465230 023064 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.Grammar;
import org.antlr.tool.NFAFactory;
/** An NFA (collection of NFAStates) constructed from a grammar. This
* NFA is one big machine for entire grammar. Decision points are recorded
* by the Grammar object so we can, for example, convert to DFA or simulate
* the NFA (interpret a decision).
*/
public class NFA {
public static final int INVALID_ALT_NUMBER = -1;
/** This NFA represents which grammar? */
public Grammar grammar;
/** Which factory created this NFA? */
protected NFAFactory factory = null;
public boolean complete;
public NFA(Grammar g) {
this.grammar = g;
}
public int getNewNFAStateNumber() {
return grammar.composite.getNewNFAStateNumber();
}
public void addState(NFAState state) {
grammar.composite.addState(state);
}
public NFAState getState(int s) {
return grammar.composite.getState(s);
}
public NFAFactory getFactory() {
return factory;
}
public void setFactory(NFAFactory factory) {
this.factory = factory;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/NFAConversionThread.java 0000644 0001750 0001750 00000004505 11256465230 026254 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.misc.Barrier;
import org.antlr.tool.Grammar;
import org.antlr.tool.ErrorManager;
/** Convert all decisions i..j inclusive in a thread */
public class NFAConversionThread implements Runnable {
Grammar grammar;
int i, j;
Barrier barrier;
public NFAConversionThread(Grammar grammar,
Barrier barrier,
int i,
int j)
{
this.grammar = grammar;
this.barrier = barrier;
this.i = i;
this.j = j;
}
public void run() {
for (int decision=i; decision<=j; decision++) {
NFAState decisionStartState = grammar.getDecisionNFAStartState(decision);
if ( decisionStartState.getNumberOfTransitions()>1 ) {
grammar.createLookaheadDFA(decision,true);
}
}
// now wait for others to finish
try {
barrier.waitForRelease();
}
catch(InterruptedException e) {
ErrorManager.internalError("what the hell? DFA interruptus", e);
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/NFAContext.java 0000644 0001750 0001750 00000027315 11256465230 024427 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
/** A tree node for tracking the call chains for NFAs that invoke
* other NFAs. These trees only have to point upwards to their parents
* so we can walk back up the tree (i.e., pop stuff off the stack). We
* never walk from stack down down through the children.
*
* Each alt predicted in a decision has its own context tree,
* representing all possible return nodes. The initial stack has
* EOF ("$") in it. So, for m alternative productions, the lookahead
* DFA will have m NFAContext trees.
*
* To "push" a new context, just do "new NFAContext(context-parent, state)"
* which will add itself to the parent. The root is NFAContext(null, null).
*
* The complete context for an NFA configuration is the set of invoking states
* on the path from this node thru the parent pointers to the root.
*/
public class NFAContext {
/** This is similar to Bermudez's m constant in his LAR(m) where
* you bound the stack so your states don't explode. The main difference
* is that I bound only recursion on the stack, not the simple stack size.
* This looser constraint will let the conversion roam further to find
* lookahead to resolve a decision.
*
* Bermudez's m operates differently as it is his LR stack depth
* I'm pretty sure it therefore includes all stack symbols. Here I
* restrict the size of an NFA configuration to be finite because a
* stack component may mention the same NFA invocation state at
* most m times. Hence, the number of DFA states will not grow forever.
* With recursive rules like
*
* e : '(' e ')' | INT ;
*
* you could chase your tail forever if somebody said "s : e '.' | e ';' ;"
* This constant prevents new states from being created after a stack gets
* "too big". Actually (12/14/2007) I realize that this example is
* trapped by the non-LL(*) detector for recursion in > 1 alt. Here is
* an example that trips stack overflow:
*
* s : a Y | A A A A A X ; // force recursion past m=4
* a : A a | Q;
*
* If that were:
*
* s : a Y | A+ X ;
*
* it could loop forever.
*
* Imagine doing a depth-first search on the e DFA...as you chase an input
* sequence you can recurse to same rule such as e above. You'd have a
* chain of ((((. When you get do some point, you have to give up. The
* states in the chain will have longer and longer NFA config stacks.
* Must limit size.
*
* max=0 implies you cannot ever jump to another rule during closure.
* max=1 implies you can make as many calls as you want--you just
* can't ever visit a state that is on your rule invocation stack.
* I.e., you cannot ever recurse.
* max=2 implies you are able to recurse once (i.e., call a rule twice
* from the same place).
*
* This tracks recursion to a rule specific to an invocation site!
* It does not detect multiple calls to a rule from different rule
* invocation states. We are guaranteed to terminate because the
* stack can only grow as big as the number of NFA states * max.
*
* I noticed that the Java grammar didn't work with max=1, but did with
* max=4. Let's set to 4. Recursion is sometimes needed to resolve some
* fixed lookahead decisions.
*/
public static int MAX_SAME_RULE_INVOCATIONS_PER_NFA_CONFIG_STACK = 4;
public NFAContext parent;
/** The NFA state that invoked another rule's start state is recorded
* on the rule invocation context stack.
*/
public NFAState invokingState;
/** Computing the hashCode is very expensive and closureBusy()
* uses it to track when it's seen a state|ctx before to avoid
* infinite loops. As we add new contexts, record the hash code
* as this.invokingState + parent.cachedHashCode. Avoids walking
* up the tree for every hashCode(). Note that this caching works
* because a context is a monotonically growing tree of context nodes
* and nothing on the stack is ever modified...ctx just grows
* or shrinks.
*/
protected int cachedHashCode;
public NFAContext(NFAContext parent, NFAState invokingState) {
this.parent = parent;
this.invokingState = invokingState;
if ( invokingState!=null ) {
this.cachedHashCode = invokingState.stateNumber;
}
if ( parent!=null ) {
this.cachedHashCode += parent.cachedHashCode;
}
}
/** Two contexts are equals() if both have
* same call stack; walk upwards to the root.
* Recall that the root sentinel node has no invokingStates and no parent.
* Note that you may be comparing contexts in different alt trees.
*
* The hashCode is now cheap as it's computed once upon each context
* push on the stack. Use it to make equals() more efficient.
*/
public boolean equals(Object o) {
NFAContext other = ((NFAContext)o);
if ( this.cachedHashCode != other.cachedHashCode ) {
return false; // can't be same if hash is different
}
if ( this==other ) {
return true;
}
// System.out.println("comparing "+this+" with "+other);
NFAContext sp = this;
while ( sp.parent!=null && other.parent!=null ) {
if ( sp.invokingState != other.invokingState ) {
return false;
}
sp = sp.parent;
other = other.parent;
}
if ( !(sp.parent==null && other.parent==null) ) {
return false; // both pointers must be at their roots after walk
}
return true;
}
/** Two contexts conflict() if they are equals() or one is a stack suffix
* of the other. For example, contexts [21 12 $] and [21 9 $] do not
* conflict, but [21 $] and [21 12 $] do conflict. Note that I should
* probably not show the $ in this case. There is a dummy node for each
* stack that just means empty; $ is a marker that's all.
*
* This is used in relation to checking conflicts associated with a
* single NFA state's configurations within a single DFA state.
* If there are configurations s and t within a DFA state such that
* s.state=t.state && s.alt != t.alt && s.ctx conflicts t.ctx then
* the DFA state predicts more than a single alt--it's nondeterministic.
* Two contexts conflict if they are the same or if one is a suffix
* of the other.
*
* When comparing contexts, if one context has a stack and the other
* does not then they should be considered the same context. The only
* way for an NFA state p to have an empty context and a nonempty context
* is the case when closure falls off end of rule without a call stack
* and re-enters the rule with a context. This resolves the issue I
* discussed with Sriram Srinivasan Feb 28, 2005 about not terminating
* fast enough upon nondeterminism.
*/
public boolean conflictsWith(NFAContext other) {
return this.suffix(other); // || this.equals(other);
}
/** [$] suffix any context
* [21 $] suffix [21 12 $]
* [21 12 $] suffix [21 $]
* [21 18 $] suffix [21 18 12 9 $]
* [21 18 12 9 $] suffix [21 18 $]
* [21 12 $] not suffix [21 9 $]
*
* Example "[21 $] suffix [21 12 $]" means: rule r invoked current rule
* from state 21. Rule s invoked rule r from state 12 which then invoked
* current rule also via state 21. While the context prior to state 21
* is different, the fact that both contexts emanate from state 21 implies
* that they are now going to track perfectly together. Once they
* converged on state 21, there is no way they can separate. In other
* words, the prior stack state is not consulted when computing where to
* go in the closure operation. ?$ and ??$ are considered the same stack.
* If ? is popped off then $ and ?$ remain; they are now an empty and
* nonempty context comparison. So, if one stack is a suffix of
* another, then it will still degenerate to the simple empty stack
* comparison case.
*/
protected boolean suffix(NFAContext other) {
NFAContext sp = this;
// if one of the contexts is empty, it never enters loop and returns true
while ( sp.parent!=null && other.parent!=null ) {
if ( sp.invokingState != other.invokingState ) {
return false;
}
sp = sp.parent;
other = other.parent;
}
//System.out.println("suffix");
return true;
}
/** Walk upwards to the root of the call stack context looking
* for a particular invoking state.
public boolean contains(int state) {
NFAContext sp = this;
int n = 0; // track recursive invocations of state
System.out.println("this.context is "+sp);
while ( sp.parent!=null ) {
if ( sp.invokingState.stateNumber == state ) {
return true;
}
sp = sp.parent;
}
return false;
}
*/
/** Given an NFA state number, how many times has the NFA-to-DFA
* conversion pushed that state on the stack? In other words,
* the NFA state must be a rule invocation state and this method
* tells you how many times you've been to this state. If none,
* then you have not called the target rule from this state before
* (though another NFA state could have called that target rule).
* If n=1, then you've been to this state before during this
* DFA construction and are going to invoke that rule again.
*
* Note that many NFA states can invoke rule r, but we ignore recursion
* unless you hit the same rule invocation state again.
*/
public int recursionDepthEmanatingFromState(int state) {
NFAContext sp = this;
int n = 0; // track recursive invocations of target from this state
//System.out.println("this.context is "+sp);
while ( sp.parent!=null ) {
if ( sp.invokingState.stateNumber == state ) {
n++;
}
sp = sp.parent;
}
return n;
}
public int hashCode() {
return cachedHashCode;
/*
int h = 0;
NFAContext sp = this;
while ( sp.parent!=null ) {
h += sp.invokingState.getStateNumber();
sp = sp.parent;
}
return h;
*/
}
/** A context is empty if there is no parent; meaning nobody pushed
* anything on the call stack.
*/
public boolean isEmpty() {
return parent==null;
}
public String toString() {
StringBuffer buf = new StringBuffer();
NFAContext sp = this;
buf.append("[");
while ( sp.parent!=null ) {
buf.append(sp.invokingState.stateNumber);
buf.append(" ");
sp = sp.parent;
}
buf.append("$]");
return buf.toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/RuleClosureTransition.java 0000644 0001750 0001750 00000004425 11256465230 026772 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.Grammar;
import org.antlr.tool.Rule;
/** A transition used to reference another rule. It tracks two targets
* really: the actual transition target and the state following the
* state that refers to the other rule. Conversion of an NFA that
* falls off the end of a rule will be able to figure out who invoked
* that rule because of these special transitions.
*/
public class RuleClosureTransition extends Transition {
/** Ptr to the rule definition object for this rule ref */
public Rule rule;
/** What node to begin computations following ref to rule */
public NFAState followState;
public RuleClosureTransition(Rule rule,
NFAState ruleStart,
NFAState followState)
{
super(Label.EPSILON, ruleStart);
this.rule = rule;
this.followState = followState;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/SemanticContext.java 0000644 0001750 0001750 00000034505 11256465230 025565 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.codegen.CodeGenerator;
import org.antlr.grammar.v2.ANTLRParser;
import org.antlr.tool.GrammarAST;
import org.antlr.tool.Grammar;
import java.util.Set;
import java.util.HashSet;
import java.util.Iterator;
/** A binary tree structure used to record the semantic context in which
* an NFA configuration is valid. It's either a single predicate or
* a tree representing an operation tree such as: p1&&p2 or p1||p2.
*
* For NFA o-p1->o-p2->o, create tree AND(p1,p2).
* For NFA (1)-p1->(2)
* | ^
* | |
* (3)-p2----
* we will have to combine p1 and p2 into DFA state as we will be
* adding NFA configurations for state 2 with two predicates p1,p2.
* So, set context for combined NFA config for state 2: OR(p1,p2).
*
* I have scoped the AND, NOT, OR, and Predicate subclasses of
* SemanticContext within the scope of this outer class.
*
* July 7, 2006: TJP altered OR to be set of operands. the Binary tree
* made it really hard to reduce complicated || sequences to their minimum.
* Got huge repeated || conditions.
*/
public abstract class SemanticContext {
/** Create a default value for the semantic context shared among all
* NFAConfigurations that do not have an actual semantic context.
* This prevents lots of if!=null type checks all over; it represents
* just an empty set of predicates.
*/
public static final SemanticContext EMPTY_SEMANTIC_CONTEXT = new Predicate();
/** Given a semantic context expression tree, return a tree with all
* nongated predicates set to true and then reduced. So p&&(q||r) would
* return p&&r if q is nongated but p and r are gated.
*/
public abstract SemanticContext getGatedPredicateContext();
/** Generate an expression that will evaluate the semantic context,
* given a set of output templates.
*/
public abstract StringTemplate genExpr(CodeGenerator generator,
StringTemplateGroup templates,
DFA dfa);
public abstract boolean isSyntacticPredicate();
/** Notify the indicated grammar of any syn preds used within this context */
public void trackUseOfSyntacticPredicates(Grammar g) {
}
public static class Predicate extends SemanticContext {
/** The AST node in tree created from the grammar holding the predicate */
public GrammarAST predicateAST;
/** Is this a {...}?=> gating predicate or a normal disambiguating {..}?
* If any predicate in expression is gated, then expression is considered
* gated.
*
* The simple Predicate object's predicate AST's type is used to set
* gated to true if type==GATED_SEMPRED.
*/
protected boolean gated = false;
/** syntactic predicates are converted to semantic predicates
* but synpreds are generated slightly differently.
*/
protected boolean synpred = false;
public static final int INVALID_PRED_VALUE = -1;
public static final int FALSE_PRED = 0;
public static final int TRUE_PRED = 1;
/** sometimes predicates are known to be true or false; we need
* a way to represent this without resorting to a target language
* value like true or TRUE.
*/
protected int constantValue = INVALID_PRED_VALUE;
public Predicate() {
predicateAST = new GrammarAST();
this.gated=false;
}
public Predicate(GrammarAST predicate) {
this.predicateAST = predicate;
this.gated =
predicate.getType()==ANTLRParser.GATED_SEMPRED ||
predicate.getType()==ANTLRParser.SYN_SEMPRED ;
this.synpred =
predicate.getType()==ANTLRParser.SYN_SEMPRED ||
predicate.getType()==ANTLRParser.BACKTRACK_SEMPRED;
}
public Predicate(Predicate p) {
this.predicateAST = p.predicateAST;
this.gated = p.gated;
this.synpred = p.synpred;
this.constantValue = p.constantValue;
}
/** Two predicates are the same if they are literally the same
* text rather than same node in the grammar's AST.
* Or, if they have the same constant value, return equal.
* As of July 2006 I'm not sure these are needed.
*/
public boolean equals(Object o) {
if ( !(o instanceof Predicate) ) {
return false;
}
return predicateAST.getText().equals(((Predicate)o).predicateAST.getText());
}
public int hashCode() {
if ( predicateAST ==null ) {
return 0;
}
return predicateAST.getText().hashCode();
}
public StringTemplate genExpr(CodeGenerator generator,
StringTemplateGroup templates,
DFA dfa)
{
StringTemplate eST = null;
if ( templates!=null ) {
if ( synpred ) {
eST = templates.getInstanceOf("evalSynPredicate");
}
else {
eST = templates.getInstanceOf("evalPredicate");
generator.grammar.decisionsWhoseDFAsUsesSemPreds.add(dfa);
}
String predEnclosingRuleName = predicateAST.enclosingRuleName;
/*
String decisionEnclosingRuleName =
dfa.getNFADecisionStartState().getEnclosingRule();
// if these rulenames are diff, then pred was hoisted out of rule
// Currently I don't warn you about this as it could be annoying.
// I do the translation anyway.
*/
//eST.setAttribute("pred", this.toString());
if ( generator!=null ) {
eST.setAttribute("pred",
generator.translateAction(predEnclosingRuleName,predicateAST));
}
}
else {
eST = new StringTemplate("$pred$");
eST.setAttribute("pred", this.toString());
return eST;
}
if ( generator!=null ) {
String description =
generator.target.getTargetStringLiteralFromString(this.toString());
eST.setAttribute("description", description);
}
return eST;
}
public SemanticContext getGatedPredicateContext() {
if ( gated ) {
return this;
}
return null;
}
public boolean isSyntacticPredicate() {
return predicateAST !=null &&
( predicateAST.getType()==ANTLRParser.SYN_SEMPRED ||
predicateAST.getType()==ANTLRParser.BACKTRACK_SEMPRED );
}
public void trackUseOfSyntacticPredicates(Grammar g) {
if ( synpred ) {
g.synPredNamesUsedInDFA.add(predicateAST.getText());
}
}
public String toString() {
if ( predicateAST ==null ) {
return "<nopred>";
}
return predicateAST.getText();
}
}
public static class TruePredicate extends Predicate {
public TruePredicate() {
super();
this.constantValue = TRUE_PRED;
}
public StringTemplate genExpr(CodeGenerator generator,
StringTemplateGroup templates,
DFA dfa)
{
if ( templates!=null ) {
return templates.getInstanceOf("true");
}
return new StringTemplate("true");
}
public String toString() {
return "true"; // not used for code gen, just DOT and print outs
}
}
/*
public static class FalsePredicate extends Predicate {
public FalsePredicate() {
super();
this.constantValue = FALSE_PRED;
}
public StringTemplate genExpr(CodeGenerator generator,
StringTemplateGroup templates,
DFA dfa)
{
if ( templates!=null ) {
return templates.getInstanceOf("false");
}
return new StringTemplate("false");
}
public String toString() {
return "false"; // not used for code gen, just DOT and print outs
}
}
*/
public static class AND extends SemanticContext {
protected SemanticContext left,right;
public AND(SemanticContext a, SemanticContext b) {
this.left = a;
this.right = b;
}
public StringTemplate genExpr(CodeGenerator generator,
StringTemplateGroup templates,
DFA dfa)
{
StringTemplate eST = null;
if ( templates!=null ) {
eST = templates.getInstanceOf("andPredicates");
}
else {
eST = new StringTemplate("($left$&&$right$)");
}
eST.setAttribute("left", left.genExpr(generator,templates,dfa));
eST.setAttribute("right", right.genExpr(generator,templates,dfa));
return eST;
}
public SemanticContext getGatedPredicateContext() {
SemanticContext gatedLeft = left.getGatedPredicateContext();
SemanticContext gatedRight = right.getGatedPredicateContext();
if ( gatedLeft==null ) {
return gatedRight;
}
if ( gatedRight==null ) {
return gatedLeft;
}
return new AND(gatedLeft, gatedRight);
}
public boolean isSyntacticPredicate() {
return left.isSyntacticPredicate()||right.isSyntacticPredicate();
}
public void trackUseOfSyntacticPredicates(Grammar g) {
left.trackUseOfSyntacticPredicates(g);
right.trackUseOfSyntacticPredicates(g);
}
public String toString() {
return "("+left+"&&"+right+")";
}
}
public static class OR extends SemanticContext {
protected Set operands;
public OR(SemanticContext a, SemanticContext b) {
operands = new HashSet();
if ( a instanceof OR ) {
operands.addAll(((OR)a).operands);
}
else if ( a!=null ) {
operands.add(a);
}
if ( b instanceof OR ) {
operands.addAll(((OR)b).operands);
}
else if ( b!=null ) {
operands.add(b);
}
}
public StringTemplate genExpr(CodeGenerator generator,
StringTemplateGroup templates,
DFA dfa)
{
StringTemplate eST = null;
if ( templates!=null ) {
eST = templates.getInstanceOf("orPredicates");
}
else {
eST = new StringTemplate("($first(operands)$$rest(operands):{o | ||$o$}$)");
}
for (Iterator it = operands.iterator(); it.hasNext();) {
SemanticContext semctx = (SemanticContext) it.next();
eST.setAttribute("operands", semctx.genExpr(generator,templates,dfa));
}
return eST;
}
public SemanticContext getGatedPredicateContext() {
SemanticContext result = null;
for (Iterator it = operands.iterator(); it.hasNext();) {
SemanticContext semctx = (SemanticContext) it.next();
SemanticContext gatedPred = semctx.getGatedPredicateContext();
if ( gatedPred!=null ) {
result = or(result, gatedPred);
// result = new OR(result, gatedPred);
}
}
return result;
}
public boolean isSyntacticPredicate() {
for (Iterator it = operands.iterator(); it.hasNext();) {
SemanticContext semctx = (SemanticContext) it.next();
if ( semctx.isSyntacticPredicate() ) {
return true;
}
}
return false;
}
public void trackUseOfSyntacticPredicates(Grammar g) {
for (Iterator it = operands.iterator(); it.hasNext();) {
SemanticContext semctx = (SemanticContext) it.next();
semctx.trackUseOfSyntacticPredicates(g);
}
}
public String toString() {
StringBuffer buf = new StringBuffer();
buf.append("(");
int i = 0;
for (Iterator it = operands.iterator(); it.hasNext();) {
SemanticContext semctx = (SemanticContext) it.next();
if ( i>0 ) {
buf.append("||");
}
buf.append(semctx.toString());
i++;
}
buf.append(")");
return buf.toString();
}
}
public static class NOT extends SemanticContext {
protected SemanticContext ctx;
public NOT(SemanticContext ctx) {
this.ctx = ctx;
}
public StringTemplate genExpr(CodeGenerator generator,
StringTemplateGroup templates,
DFA dfa)
{
StringTemplate eST = null;
if ( templates!=null ) {
eST = templates.getInstanceOf("notPredicate");
}
else {
eST = new StringTemplate("?!($pred$)");
}
eST.setAttribute("pred", ctx.genExpr(generator,templates,dfa));
return eST;
}
public SemanticContext getGatedPredicateContext() {
SemanticContext p = ctx.getGatedPredicateContext();
if ( p==null ) {
return null;
}
return new NOT(p);
}
public boolean isSyntacticPredicate() {
return ctx.isSyntacticPredicate();
}
public void trackUseOfSyntacticPredicates(Grammar g) {
ctx.trackUseOfSyntacticPredicates(g);
}
public boolean equals(Object object) {
if ( !(object instanceof NOT) ) {
return false;
}
return this.ctx.equals(((NOT)object).ctx);
}
public String toString() {
return "!("+ctx+")";
}
}
public static SemanticContext and(SemanticContext a, SemanticContext b) {
//System.out.println("AND: "+a+"&&"+b);
if ( a==EMPTY_SEMANTIC_CONTEXT || a==null ) {
return b;
}
if ( b==EMPTY_SEMANTIC_CONTEXT || b==null ) {
return a;
}
if ( a.equals(b) ) {
return a; // if same, just return left one
}
//System.out.println("## have to AND");
return new AND(a,b);
}
public static SemanticContext or(SemanticContext a, SemanticContext b) {
//System.out.println("OR: "+a+"||"+b);
if ( a==EMPTY_SEMANTIC_CONTEXT || a==null ) {
return b;
}
if ( b==EMPTY_SEMANTIC_CONTEXT || b==null ) {
return a;
}
if ( a instanceof TruePredicate ) {
return a;
}
if ( b instanceof TruePredicate ) {
return b;
}
if ( a instanceof NOT && b instanceof Predicate ) {
NOT n = (NOT)a;
// check for !p||p
if ( n.ctx.equals(b) ) {
return new TruePredicate();
}
}
else if ( b instanceof NOT && a instanceof Predicate ) {
NOT n = (NOT)b;
// check for p||!p
if ( n.ctx.equals(a) ) {
return new TruePredicate();
}
}
else if ( a.equals(b) ) {
return a;
}
//System.out.println("## have to OR");
return new OR(a,b);
}
public static SemanticContext not(SemanticContext a) {
return new NOT(a);
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/LookaheadSet.java 0000644 0001750 0001750 00000006517 11256465230 025022 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.misc.IntervalSet;
import org.antlr.misc.IntSet;
import org.antlr.tool.Grammar;
/** An LL(1) lookahead set; contains a set of token types and a "hasEOF"
* condition when the set contains EOF. Since EOF is -1 everywhere and -1
* cannot be stored in my BitSet, I set a condition here. There may be other
* reasons in the future to abstract a LookaheadSet over a raw BitSet.
*/
public class LookaheadSet {
public IntervalSet tokenTypeSet;
public LookaheadSet() {
tokenTypeSet = new IntervalSet();
}
public LookaheadSet(IntSet s) {
this();
tokenTypeSet.addAll(s);
}
public LookaheadSet(int atom) {
tokenTypeSet = IntervalSet.of(atom);
}
public LookaheadSet(LookaheadSet other) {
this();
this.tokenTypeSet.addAll(other.tokenTypeSet);
}
public void orInPlace(LookaheadSet other) {
this.tokenTypeSet.addAll(other.tokenTypeSet);
}
public LookaheadSet or(LookaheadSet other) {
return new LookaheadSet(tokenTypeSet.or(other.tokenTypeSet));
}
public LookaheadSet subtract(LookaheadSet other) {
return new LookaheadSet(this.tokenTypeSet.subtract(other.tokenTypeSet));
}
public boolean member(int a) {
return tokenTypeSet.member(a);
}
public LookaheadSet intersection(LookaheadSet s) {
IntSet i = this.tokenTypeSet.and(s.tokenTypeSet);
LookaheadSet intersection = new LookaheadSet(i);
return intersection;
}
public boolean isNil() {
return tokenTypeSet.isNil();
}
public void remove(int a) {
tokenTypeSet = (IntervalSet)tokenTypeSet.subtract(IntervalSet.of(a));
}
public int hashCode() {
return tokenTypeSet.hashCode();
}
public boolean equals(Object other) {
return tokenTypeSet.equals(((LookaheadSet)other).tokenTypeSet);
}
public String toString(Grammar g) {
if ( tokenTypeSet==null ) {
return "";
}
String r = tokenTypeSet.toString(g);
return r;
}
public String toString() {
return toString(null);
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/Label.java 0000644 0001750 0001750 00000026533 11256465230 023476 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.Grammar;
import org.antlr.tool.GrammarAST;
import org.antlr.misc.IntervalSet;
import org.antlr.misc.IntSet;
/** A state machine transition label. A label can be either a simple
* label such as a token or character. A label can be a set of char or
* tokens. It can be an epsilon transition. It can be a semantic predicate
* (which assumes an epsilon transition) or a tree of predicates (in a DFA).
*/
public class Label implements Comparable, Cloneable {
public static final int INVALID = -7;
public static final int ACTION = -6;
public static final int EPSILON = -5;
public static final String EPSILON_STR = "<EPSILON>";
/** label is a semantic predicate; implies label is epsilon also */
public static final int SEMPRED = -4;
/** label is a set of tokens or char */
public static final int SET = -3;
/** End of Token is like EOF for lexer rules. It implies that no more
* characters are available and that NFA conversion should terminate
* for this path. For example
*
* A : 'a' 'b' | 'a' ;
*
* yields a DFA predictor:
*
* o-a->o-b->1 predict alt 1
* |
* |-EOT->o predict alt 2
*
* To generate code for EOT, treat it as the "default" path, which
* implies there is no way to mismatch a char for the state from
* which the EOT emanates.
*/
public static final int EOT = -2;
public static final int EOF = -1;
/** We have labels like EPSILON that are below 0; it's hard to
* store them in an array with negative index so use this
* constant as an index shift when accessing arrays based upon
* token type. If real token type is i, then array index would be
* NUM_FAUX_LABELS + i.
*/
public static final int NUM_FAUX_LABELS = -INVALID;
/** Anything at this value or larger can be considered a simple atom int
* for easy comparison during analysis only; faux labels are not used
* during parse time for real token types or char values.
*/
public static final int MIN_ATOM_VALUE = EOT;
// TODO: is 0 a valid unicode char? max is FFFF -1, right?
public static final int MIN_CHAR_VALUE = '\u0000';
public static final int MAX_CHAR_VALUE = '\uFFFF';
/** End of rule token type; imaginary token type used only for
* local, partial FOLLOW sets to indicate that the local FOLLOW
* hit the end of rule. During error recovery, the local FOLLOW
* of a token reference may go beyond the end of the rule and have
* to use FOLLOW(rule). I have to just shift the token types to 2..n
* rather than 1..n to accommodate this imaginary token in my bitsets.
* If I didn't use a bitset implementation for runtime sets, I wouldn't
* need this. EOF is another candidate for a run time token type for
* parsers. Follow sets are not computed for lexers so we do not have
* this issue.
*/
public static final int EOR_TOKEN_TYPE =
org.antlr.runtime.Token.EOR_TOKEN_TYPE;
public static final int DOWN = org.antlr.runtime.Token.DOWN;
public static final int UP = org.antlr.runtime.Token.UP;
/** tokens and char range overlap; tokens are MIN_TOKEN_TYPE..n */
public static final int MIN_TOKEN_TYPE =
org.antlr.runtime.Token.MIN_TOKEN_TYPE;
/** The wildcard '.' char atom implies all valid characters==UNICODE */
//public static final IntSet ALLCHAR = IntervalSet.of(MIN_CHAR_VALUE,MAX_CHAR_VALUE);
/** The token type or character value; or, signifies special label. */
protected int label;
/** A set of token types or character codes if label==SET */
// TODO: try IntervalSet for everything
protected IntSet labelSet;
public Label(int label) {
this.label = label;
}
/** Make a set label */
public Label(IntSet labelSet) {
if ( labelSet==null ) {
this.label = SET;
this.labelSet = IntervalSet.of(INVALID);
return;
}
int singleAtom = labelSet.getSingleElement();
if ( singleAtom!=INVALID ) {
// convert back to a single atomic element if |labelSet|==1
label = singleAtom;
return;
}
this.label = SET;
this.labelSet = labelSet;
}
public Object clone() {
Label l;
try {
l = (Label)super.clone();
l.label = this.label;
l.labelSet = new IntervalSet();
l.labelSet.addAll(this.labelSet);
}
catch (CloneNotSupportedException e) {
throw new InternalError();
}
return l;
}
public void add(Label a) {
if ( isAtom() ) {
labelSet = IntervalSet.of(label);
label=SET;
if ( a.isAtom() ) {
labelSet.add(a.getAtom());
}
else if ( a.isSet() ) {
labelSet.addAll(a.getSet());
}
else {
throw new IllegalStateException("can't add element to Label of type "+label);
}
return;
}
if ( isSet() ) {
if ( a.isAtom() ) {
labelSet.add(a.getAtom());
}
else if ( a.isSet() ) {
labelSet.addAll(a.getSet());
}
else {
throw new IllegalStateException("can't add element to Label of type "+label);
}
return;
}
throw new IllegalStateException("can't add element to Label of type "+label);
}
public boolean isAtom() {
return label>=MIN_ATOM_VALUE;
}
public boolean isEpsilon() {
return label==EPSILON;
}
public boolean isSemanticPredicate() {
return false;
}
public boolean isAction() {
return false;
}
public boolean isSet() {
return label==SET;
}
/** return the single atom label or INVALID if not a single atom */
public int getAtom() {
if ( isAtom() ) {
return label;
}
return INVALID;
}
public IntSet getSet() {
if ( label!=SET ) {
// convert single element to a set if they ask for it.
return IntervalSet.of(label);
}
return labelSet;
}
public void setSet(IntSet set) {
label=SET;
labelSet = set;
}
public SemanticContext getSemanticContext() {
return null;
}
public boolean matches(int atom) {
if ( label==atom ) {
return true; // handle the single atom case efficiently
}
if ( isSet() ) {
return labelSet.member(atom);
}
return false;
}
public boolean matches(IntSet set) {
if ( isAtom() ) {
return set.member(getAtom());
}
if ( isSet() ) {
// matches if intersection non-nil
return !getSet().and(set).isNil();
}
return false;
}
public boolean matches(Label other) {
if ( other.isSet() ) {
return matches(other.getSet());
}
if ( other.isAtom() ) {
return matches(other.getAtom());
}
return false;
}
public int hashCode() {
if (label==SET) {
return labelSet.hashCode();
}
else {
return label;
}
}
// TODO: do we care about comparing set {A} with atom A? Doesn't now.
public boolean equals(Object o) {
if ( o==null ) {
return false;
}
if ( this == o ) {
return true; // equals if same object
}
// labels must be the same even if epsilon or set or sempred etc...
if ( label!=((Label)o).label ) {
return false;
}
if ( label==SET ) {
return this.labelSet.equals(((Label)o).labelSet);
}
return true; // label values are same, so true
}
public int compareTo(Object o) {
return this.label-((Label)o).label;
}
/** Predicates are lists of AST nodes from the NFA created from the
* grammar, but the same predicate could be cut/paste into multiple
* places in the grammar. I must compare the text of all the
* predicates to truly answer whether {p1,p2} .equals {p1,p2}.
* Unfortunately, I cannot rely on the AST.equals() to work properly
* so I must do a brute force O(n^2) nested traversal of the Set
* doing a String compare.
*
* At this point, Labels are not compared for equals when they are
* predicates, but here's the code for future use.
*/
/*
protected boolean predicatesEquals(Set others) {
Iterator iter = semanticContext.iterator();
while (iter.hasNext()) {
AST predAST = (AST) iter.next();
Iterator inner = semanticContext.iterator();
while (inner.hasNext()) {
AST otherPredAST = (AST) inner.next();
if ( !predAST.getText().equals(otherPredAST.getText()) ) {
return false;
}
}
}
return true;
}
*/
public String toString() {
switch (label) {
case SET :
return labelSet.toString();
default :
return String.valueOf(label);
}
}
public String toString(Grammar g) {
switch (label) {
case SET :
return labelSet.toString(g);
default :
return g.getTokenDisplayName(label);
}
}
/*
public String predicatesToString() {
if ( semanticContext==NFAConfiguration.DEFAULT_CLAUSE_SEMANTIC_CONTEXT ) {
return "!other preds";
}
StringBuffer buf = new StringBuffer();
Iterator iter = semanticContext.iterator();
while (iter.hasNext()) {
AST predAST = (AST) iter.next();
buf.append(predAST.getText());
if ( iter.hasNext() ) {
buf.append("&");
}
}
return buf.toString();
}
*/
public static boolean intersect(Label label, Label edgeLabel) {
boolean hasIntersection = false;
boolean labelIsSet = label.isSet();
boolean edgeIsSet = edgeLabel.isSet();
if ( !labelIsSet && !edgeIsSet && edgeLabel.label==label.label ) {
hasIntersection = true;
}
else if ( labelIsSet && edgeIsSet &&
!edgeLabel.getSet().and(label.getSet()).isNil() ) {
hasIntersection = true;
}
else if ( labelIsSet && !edgeIsSet &&
label.getSet().member(edgeLabel.label) ) {
hasIntersection = true;
}
else if ( !labelIsSet && edgeIsSet &&
edgeLabel.getSet().member(label.label) ) {
hasIntersection = true;
}
return hasIntersection;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/StateCluster.java 0000644 0001750 0001750 00000003401 11256465230 025066 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
/** A Cluster object points to the left/right (start and end) states of a
* state machine. Used to build NFAs.
*/
public class StateCluster {
public NFAState left;
public NFAState right;
public StateCluster(NFAState left, NFAState right) {
this.left = left;
this.right = right;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/AnalysisRecursionOverflowException.java 0000644 0001750 0001750 00000003547 11256465226 031544 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
/** An NFA configuration context stack overflowed. */
public class AnalysisRecursionOverflowException extends RuntimeException {
public DFAState ovfState;
public NFAConfiguration proposedNFAConfiguration;
public AnalysisRecursionOverflowException(DFAState ovfState,
NFAConfiguration proposedNFAConfiguration)
{
this.ovfState = ovfState;
this.proposedNFAConfiguration = proposedNFAConfiguration;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/NFAToDFAConverter.java 0000644 0001750 0001750 00000213171 11256465230 025565 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.misc.OrderedHashSet;
import org.antlr.misc.Utils;
import org.antlr.tool.ErrorManager;
import java.util.*;
import antlr.Token;
/** Code that embodies the NFA conversion to DFA. A new object is needed
* per DFA (also required for thread safety if multiple conversions
* launched).
*/
public class NFAToDFAConverter {
/** A list of DFA states we still need to process during NFA conversion */
protected List work = new LinkedList();
/** While converting NFA, we must track states that
* reference other rule's NFAs so we know what to do
* at the end of a rule. We need to know what context invoked
* this rule so we can know where to continue looking for NFA
* states. I'm tracking a context tree (record of rule invocation
* stack trace) for each alternative that could be predicted.
*/
protected NFAContext[] contextTrees;
/** We are converting which DFA? */
protected DFA dfa;
public static boolean debug = false;
/** Should ANTLR launch multiple threads to convert NFAs to DFAs?
* With a 2-CPU box, I note that it's about the same single or
* multithreaded. Both CPU meters are going even when single-threaded
* so I assume the GC is killing us. Could be the compiler. When I
* run java -Xint mode, I get about 15% speed improvement with multiple
* threads.
*/
public static boolean SINGLE_THREADED_NFA_CONVERSION = true;
protected boolean computingStartState = false;
public NFAToDFAConverter(DFA dfa) {
this.dfa = dfa;
int nAlts = dfa.getNumberOfAlts();
initContextTrees(nAlts);
}
public void convert() {
dfa.conversionStartTime = System.currentTimeMillis();
// create the DFA start state
dfa.startState = computeStartState();
// while more DFA states to check, process them
while ( work.size()>0 &&
!dfa.nfa.grammar.NFAToDFAConversionExternallyAborted() )
{
DFAState d = (DFAState) work.get(0);
if ( dfa.nfa.grammar.composite.watchNFAConversion ) {
System.out.println("convert DFA state "+d.stateNumber+
" ("+d.nfaConfigurations.size()+" nfa states)");
}
int k = dfa.getUserMaxLookahead();
if ( k>0 && k==d.getLookaheadDepth() ) {
// we've hit max lookahead, make this a stop state
//System.out.println("stop state @k="+k+" (terminated early)");
/*
List<Label> sampleInputLabels = d.dfa.probe.getSampleNonDeterministicInputSequence(d);
String input = d.dfa.probe.getInputSequenceDisplay(sampleInputLabels);
System.out.println("sample input: "+input);
*/
resolveNonDeterminisms(d);
// Check to see if we need to add any semantic predicate transitions
if ( d.isResolvedWithPredicates() ) {
addPredicateTransitions(d);
}
else {
d.setAcceptState(true); // must convert to accept state at k
}
}
else {
findNewDFAStatesAndAddDFATransitions(d);
}
work.remove(0); // done with it; remove from work list
}
// Find all manual syn preds (gated). These are not discovered
// in tryToResolveWithSemanticPredicates because they are implicitly
// added to every edge by code gen, DOT generation etc...
dfa.findAllGatedSynPredsUsedInDFAAcceptStates();
}
/** From this first NFA state of a decision, create a DFA.
* Walk each alt in decision and compute closure from the start of that
* rule, making sure that the closure does not include other alts within
* that same decision. The idea is to associate a specific alt number
* with the starting closure so we can trace the alt number for all states
* derived from this. At a stop state in the DFA, we can return this alt
* number, indicating which alt is predicted.
*
* If this DFA is derived from an loop back NFA state, then the first
* transition is actually the exit branch of the loop. Rather than make
* this alternative one, let's make this alt n+1 where n is the number of
* alts in this block. This is nice to keep the alts of the block 1..n;
* helps with error messages.
*
* I handle nongreedy in findNewDFAStatesAndAddDFATransitions
* when nongreedy and EOT transition. Make state with EOT emanating
* from it the accept state.
*/
protected DFAState computeStartState() {
NFAState alt = dfa.decisionNFAStartState;
DFAState startState = dfa.newState();
computingStartState = true;
int i = 0;
int altNum = 1;
while ( alt!=null ) {
// find the set of NFA states reachable without consuming
// any input symbols for each alt. Keep adding to same
// overall closure that will represent the DFA start state,
// but track the alt number
NFAContext initialContext = contextTrees[i];
// if first alt is derived from loopback/exit branch of loop,
// make alt=n+1 for n alts instead of 1
if ( i==0 &&
dfa.getNFADecisionStartState().decisionStateType==NFAState.LOOPBACK )
{
int numAltsIncludingExitBranch = dfa.nfa.grammar
.getNumberOfAltsForDecisionNFA(dfa.decisionNFAStartState);
altNum = numAltsIncludingExitBranch;
closure((NFAState)alt.transition[0].target,
altNum,
initialContext,
SemanticContext.EMPTY_SEMANTIC_CONTEXT,
startState,
true
);
altNum = 1; // make next alt the first
}
else {
closure((NFAState)alt.transition[0].target,
altNum,
initialContext,
SemanticContext.EMPTY_SEMANTIC_CONTEXT,
startState,
true
);
altNum++;
}
i++;
// move to next alternative
if ( alt.transition[1] ==null ) {
break;
}
alt = (NFAState)alt.transition[1].target;
}
// now DFA start state has the complete closure for the decision
// but we have tracked which alt is associated with which
// NFA states.
dfa.addState(startState); // make sure dfa knows about this state
work.add(startState);
computingStartState = false;
return startState;
}
/** From this node, add a d--a-->t transition for all
* labels 'a' where t is a DFA node created
* from the set of NFA states reachable from any NFA
* state in DFA state d.
*/
protected void findNewDFAStatesAndAddDFATransitions(DFAState d) {
//System.out.println("work on DFA state "+d);
OrderedHashSet labels = d.getReachableLabels();
//System.out.println("reachable labels="+labels);
/*
System.out.println("|reachable|/|nfaconfigs|="+
labels.size()+"/"+d.getNFAConfigurations().size()+"="+
labels.size()/(float)d.getNFAConfigurations().size());
*/
// normally EOT is the "default" clause and decisions just
// choose that last clause when nothing else matches. DFA conversion
// continues searching for a unique sequence that predicts the
// various alts or until it finds EOT. So this rule
//
// DUH : ('x'|'y')* "xy!";
//
// does not need a greedy indicator. The following rule works fine too
//
// A : ('x')+ ;
//
// When the follow branch could match what is in the loop, by default,
// the nondeterminism is resolved in favor of the loop. You don't
// get a warning because the only way to get this condition is if
// the DFA conversion hits the end of the token. In that case,
// we're not *sure* what will happen next, but it could be anything.
// Anyway, EOT is the default case which means it will never be matched
// as resolution goes to the lowest alt number. Exit branches are
// always alt n+1 for n alts in a block.
//
// When a loop is nongreedy and we find an EOT transition, the DFA
// state should become an accept state, predicting exit of loop. It's
// just reversing the resolution of ambiguity.
// TODO: should this be done in the resolveAmbig method?
Label EOTLabel = new Label(Label.EOT);
boolean containsEOT = labels!=null && labels.contains(EOTLabel);
if ( !dfa.isGreedy() && containsEOT ) {
convertToEOTAcceptState(d);
return; // no more work to do on this accept state
}
// if in filter mode for lexer, want to match shortest not longest
// string so if we see an EOT edge emanating from this state, then
// convert this state to an accept state. This only counts for
// The Tokens rule as all other decisions must continue to look for
// longest match.
// [Taking back out a few days later on Jan 17, 2006. This could
// be an option for the future, but this was wrong soluion for
// filtering.]
/*
if ( dfa.nfa.grammar.type==Grammar.LEXER && containsEOT ) {
String filterOption = (String)dfa.nfa.grammar.getOption("filter");
boolean filterMode = filterOption!=null && filterOption.equals("true");
if ( filterMode && d.dfa.isTokensRuleDecision() ) {
DFAState t = reach(d, EOTLabel);
if ( t.getNFAConfigurations().size()>0 ) {
convertToEOTAcceptState(d);
//System.out.println("state "+d+" has EOT target "+t.stateNumber);
return;
}
}
}
*/
int numberOfEdgesEmanating = 0;
Map targetToLabelMap = new HashMap();
// for each label that could possibly emanate from NFAStates of d
int numLabels = 0;
if ( labels!=null ) {
numLabels = labels.size();
}
for (int i=0; i<numLabels; i++) {
Label label = (Label)labels.get(i);
DFAState t = reach(d, label);
if ( debug ) {
System.out.println("DFA state after reach "+label+" "+d+"-" +
label.toString(dfa.nfa.grammar)+"->"+t);
}
if ( t==null ) {
// nothing was reached by label due to conflict resolution
// EOT also seems to be in here occasionally probably due
// to an end-of-rule state seeing it even though we'll pop
// an invoking state off the state; don't bother to conflict
// as this labels set is a covering approximation only.
continue;
}
//System.out.println("dfa.k="+dfa.getUserMaxLookahead());
if ( t.getUniqueAlt()==NFA.INVALID_ALT_NUMBER ) {
// Only compute closure if a unique alt number is not known.
// If a unique alternative is mentioned among all NFA
// configurations then there is no possibility of needing to look
// beyond this state; also no possibility of a nondeterminism.
// This optimization May 22, 2006 just dropped -Xint time
// for analysis of Java grammar from 11.5s to 2s! Wow.
closure(t); // add any NFA states reachable via epsilon
}
/*
System.out.println("DFA state after closure "+d+"-"+
label.toString(dfa.nfa.grammar)+
"->"+t);
*/
// add if not in DFA yet and then make d-label->t
DFAState targetState = addDFAStateToWorkList(t);
numberOfEdgesEmanating +=
addTransition(d, label, targetState, targetToLabelMap);
// lookahead of target must be one larger than d's k
// We are possibly setting the depth of a pre-existing state
// that is equal to one we just computed...not sure if that's
// ok.
targetState.setLookaheadDepth(d.getLookaheadDepth() + 1);
}
//System.out.println("DFA after reach / closures:\n"+dfa);
if ( !d.isResolvedWithPredicates() && numberOfEdgesEmanating==0 ) {
//System.out.println("dangling DFA state "+d+"\nAfter reach / closures:\n"+dfa);
// TODO: can fixed lookahead hit a dangling state case?
// TODO: yes, with left recursion
//System.err.println("dangling state alts: "+d.getAltSet());
dfa.probe.reportDanglingState(d);
// turn off all configurations except for those associated with
// min alt number; somebody has to win else some input will not
// predict any alt.
int minAlt = resolveByPickingMinAlt(d, null);
// force it to be an accept state
// don't call convertToAcceptState() which merges stop states.
// other states point at us; don't want them pointing to dead states
d.setAcceptState(true); // might be adding new accept state for alt
dfa.setAcceptState(minAlt, d);
//convertToAcceptState(d, minAlt); // force it to be an accept state
}
// Check to see if we need to add any semantic predicate transitions
if ( d.isResolvedWithPredicates() ) {
addPredicateTransitions(d);
}
}
/** Add a transition from state d to targetState with label in normal case.
* if COLLAPSE_ALL_INCIDENT_EDGES, however, try to merge all edges from
* d to targetState; this means merging their labels. Another optimization
* is to reduce to a single EOT edge any set of edges from d to targetState
* where there exists an EOT state. EOT is like the wildcard so don't
* bother to test any other edges. Example:
*
* NUM_INT
* : '1'..'9' ('0'..'9')* ('l'|'L')?
* | '0' ('x'|'X') ('0'..'9'|'a'..'f'|'A'..'F')+ ('l'|'L')?
* | '0' ('0'..'7')* ('l'|'L')?
* ;
*
* The normal decision to predict alts 1, 2, 3 is:
*
* if ( (input.LA(1)>='1' && input.LA(1)<='9') ) {
* alt7=1;
* }
* else if ( input.LA(1)=='0' ) {
* if ( input.LA(2)=='X'||input.LA(2)=='x' ) {
* alt7=2;
* }
* else if ( (input.LA(2)>='0' && input.LA(2)<='7') ) {
* alt7=3;
* }
* else if ( input.LA(2)=='L'||input.LA(2)=='l' ) {
* alt7=3;
* }
* else {
* alt7=3;
* }
* }
* else error
*
* Clearly, alt 3 is predicted with extra work since it tests 0..7
* and [lL] before finally realizing that any character is actually
* ok at k=2.
*
* A better decision is as follows:
*
* if ( (input.LA(1)>='1' && input.LA(1)<='9') ) {
* alt7=1;
* }
* else if ( input.LA(1)=='0' ) {
* if ( input.LA(2)=='X'||input.LA(2)=='x' ) {
* alt7=2;
* }
* else {
* alt7=3;
* }
* }
*
* The DFA originally has 3 edges going to the state the predicts alt 3,
* but upon seeing the EOT edge (the "else"-clause), this method
* replaces the old merged label (which would have (0..7|l|L)) with EOT.
* The code generator then leaves alt 3 predicted with a simple else-
* clause. :)
*
* The only time the EOT optimization makes no sense is in the Tokens
* rule. We want EOT to truly mean you have matched an entire token
* so don't bother actually rewinding to execute that rule unless there
* are actions in that rule. For now, since I am not preventing
* backtracking from Tokens rule, I will simply allow the optimization.
*/
protected static int addTransition(DFAState d,
Label label,
DFAState targetState,
Map targetToLabelMap)
{
//System.out.println(d.stateNumber+"-"+label.toString(dfa.nfa.grammar)+"->"+targetState.stateNumber);
int n = 0;
if ( DFAOptimizer.COLLAPSE_ALL_PARALLEL_EDGES ) {
// track which targets we've hit
Integer tI = Utils.integer(targetState.stateNumber);
Transition oldTransition = (Transition)targetToLabelMap.get(tI);
if ( oldTransition!=null ) {
//System.out.println("extra transition to "+tI+" upon "+label.toString(dfa.nfa.grammar));
// already seen state d to target transition, just add label
// to old label unless EOT
if ( label.getAtom()==Label.EOT ) {
// merge with EOT means old edge can go away
oldTransition.label = new Label(Label.EOT);
}
else {
// don't add anything to EOT, it's essentially the wildcard
if ( oldTransition.label.getAtom()!=Label.EOT ) {
// ok, not EOT, add in this label to old label
oldTransition.label.add(label);
}
//System.out.println("label updated to be "+oldTransition.label.toString(dfa.nfa.grammar));
}
}
else {
// make a transition from d to t upon 'a'
n = 1;
label = (Label)label.clone(); // clone in case we alter later
int transitionIndex = d.addTransition(targetState, label);
Transition trans = d.getTransition(transitionIndex);
// track target/transition pairs
targetToLabelMap.put(tI, trans);
}
}
else {
n = 1;
d.addTransition(targetState, label);
}
return n;
}
/** For all NFA states (configurations) merged in d,
* compute the epsilon closure; that is, find all NFA states reachable
* from the NFA states in d via purely epsilon transitions.
*/
public void closure(DFAState d) {
if ( debug ) {
System.out.println("closure("+d+")");
}
List<NFAConfiguration> configs = new ArrayList<NFAConfiguration>();
// Because we are adding to the configurations in closure
// must clone initial list so we know when to stop doing closure
configs.addAll(d.nfaConfigurations);
// for each NFA configuration in d (abort if we detect non-LL(*) state)
int numConfigs = configs.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration c = (NFAConfiguration)configs.get(i);
if ( c.singleAtomTransitionEmanating ) {
continue; // ignore NFA states w/o epsilon transitions
}
//System.out.println("go do reach for NFA state "+c.state);
// figure out reachable NFA states from each of d's nfa states
// via epsilon transitions.
// Fill configsInClosure rather than altering d configs inline
closure(dfa.nfa.getState(c.state),
c.alt,
c.context,
c.semanticContext,
d,
false);
}
//System.out.println("after closure d="+d);
d.closureBusy = null; // wack all that memory used during closure
}
/** Where can we get from NFA state p traversing only epsilon transitions?
* Add new NFA states + context to DFA state d. Also add semantic
* predicates to semantic context if collectPredicates is set. We only
* collect predicates at hoisting depth 0, meaning before any token/char
* have been recognized. This corresponds, during analysis, to the
* initial DFA start state construction closure() invocation.
*
* There are four cases of interest (the last being the usual transition):
*
* 1. Traverse an edge that takes us to the start state of another
* rule, r. We must push this state so that if the DFA
* conversion hits the end of rule r, then it knows to continue
* the conversion at state following state that "invoked" r. By
* construction, there is a single transition emanating from a rule
* ref node.
*
* 2. Reach an NFA state associated with the end of a rule, r, in the
* grammar from which it was built. We must add an implicit (i.e.,
* don't actually add an epsilon transition) epsilon transition
* from r's end state to the NFA state following the NFA state
* that transitioned to rule r's start state. Because there are
* many states that could reach r, the context for a rule invocation
* is part of a call tree not a simple stack. When we fall off end
* of rule, "pop" a state off the call tree and add that state's
* "following" node to d's NFA configuration list. The context
* for this new addition will be the new "stack top" in the call tree.
*
* 3. Like case 2, we reach an NFA state associated with the end of a
* rule, r, in the grammar from which NFA was built. In this case,
* however, we realize that during this NFA->DFA conversion, no state
* invoked the current rule's NFA. There is no choice but to add
* all NFA states that follow references to r's start state. This is
* analogous to computing the FOLLOW(r) in the LL(k) world. By
* construction, even rule stop state has a chain of nodes emanating
* from it that points to every possible following node. This case
* is conveniently handled then by the 4th case.
*
* 4. Normal case. If p can reach another NFA state q, then add
* q to d's configuration list, copying p's context for q's context.
* If there is a semantic predicate on the transition, then AND it
* with any existing semantic context.
*
* Current state p is always added to d's configuration list as it's part
* of the closure as well.
*
* When is a closure operation in a cycle condition? While it is
* very possible to have the same NFA state mentioned twice
* within the same DFA state, there are two situations that
* would lead to nontermination of closure operation:
*
* o Whenever closure reaches a configuration where the same state
* with same or a suffix context already exists. This catches
* the IF-THEN-ELSE tail recursion cycle and things like
*
* a : A a | B ;
*
* the context will be $ (empty stack).
*
* We have to check
* larger context stacks because of (...)+ loops. For
* example, the context of a (...)+ can be nonempty if the
* surrounding rule is invoked by another rule:
*
* a : b A | X ;
* b : (B|)+ ; // nondeterministic by the way
*
* The context of the (B|)+ loop is "invoked from item
* a : . b A ;" and then the empty alt of the loop can reach back
* to itself. The context stack will have one "return
* address" element and so we must check for same state, same
* context for arbitrary context stacks.
*
* Idea: If we've seen this configuration before during closure, stop.
* We also need to avoid reaching same state with conflicting context.
* Ultimately analysis would stop and we'd find the conflict, but we
* should stop the computation. Previously I only checked for
* exact config. Need to check for same state, suffix context
* not just exact context.
*
* o Whenever closure reaches a configuration where state p
* is present in its own context stack. This means that
* p is a rule invocation state and the target rule has
* been called before. NFAContext.MAX_RECURSIVE_INVOCATIONS
* (See the comment there also) determines how many times
* it's possible to recurse; clearly we cannot recurse forever.
* Some grammars such as the following actually require at
* least one recursive call to correctly compute the lookahead:
*
* a : L ID R
* | b
* ;
* b : ID
* | L a R
* ;
*
* Input L ID R is ambiguous but to figure this out, ANTLR
* needs to go a->b->a->b to find the L ID sequence.
*
* Do not allow closure to add a configuration that would
* allow too much recursion.
*
* This case also catches infinite left recursion.
*/
public void closure(NFAState p,
int alt,
NFAContext context,
SemanticContext semanticContext,
DFAState d,
boolean collectPredicates)
{
if ( debug ){
System.out.println("closure at "+p.enclosingRule.name+" state "+p.stateNumber+"|"+
alt+" filling DFA state "+d.stateNumber+" with context "+context
);
}
if ( DFA.MAX_TIME_PER_DFA_CREATION>0 &&
System.currentTimeMillis() - d.dfa.conversionStartTime >=
DFA.MAX_TIME_PER_DFA_CREATION )
{
// bail way out; we've blown up somehow
throw new AnalysisTimeoutException(d.dfa);
}
NFAConfiguration proposedNFAConfiguration =
new NFAConfiguration(p.stateNumber,
alt,
context,
semanticContext);
// Avoid infinite recursion
if ( closureIsBusy(d, proposedNFAConfiguration) ) {
if ( debug ) {
System.out.println("avoid visiting exact closure computation NFA config: "+
proposedNFAConfiguration+" in "+p.enclosingRule.name);
System.out.println("state is "+d.dfa.decisionNumber+"."+d.stateNumber);
}
return;
}
// set closure to be busy for this NFA configuration
d.closureBusy.add(proposedNFAConfiguration);
// p itself is always in closure
d.addNFAConfiguration(p, proposedNFAConfiguration);
// Case 1: are we a reference to another rule?
Transition transition0 = p.transition[0];
if ( transition0 instanceof RuleClosureTransition ) {
int depth = context.recursionDepthEmanatingFromState(p.stateNumber);
// Detect recursion by more than a single alt, which indicates
// that the decision's lookahead language is non-regular; terminate
if ( depth == 1 && d.dfa.getUserMaxLookahead()==0 ) { // k=* only
d.dfa.recursiveAltSet.add(alt); // indicate that this alt is recursive
if ( d.dfa.recursiveAltSet.size()>1 ) {
//System.out.println("recursive alts: "+d.dfa.recursiveAltSet.toString());
d.abortedDueToMultipleRecursiveAlts = true;
throw new NonLLStarDecisionException(d.dfa);
}
/*
System.out.println("alt "+alt+" in rule "+p.enclosingRule+" dec "+d.dfa.decisionNumber+
" ctx: "+context);
System.out.println("d="+d);
*/
}
// Detect an attempt to recurse too high
// if this context has hit the max recursions for p.stateNumber,
// don't allow it to enter p.stateNumber again
if ( depth >= NFAContext.MAX_SAME_RULE_INVOCATIONS_PER_NFA_CONFIG_STACK ) {
/*
System.out.println("OVF state "+d);
System.out.println("proposed "+proposedNFAConfiguration);
*/
d.abortedDueToRecursionOverflow = true;
d.dfa.probe.reportRecursionOverflow(d, proposedNFAConfiguration);
if ( debug ) {
System.out.println("analysis overflow in closure("+d.stateNumber+")");
}
return;
}
// otherwise, it's cool to (re)enter target of this rule ref
RuleClosureTransition ref = (RuleClosureTransition)transition0;
// first create a new context and push onto call tree,
// recording the fact that we are invoking a rule and
// from which state (case 2 below will get the following state
// via the RuleClosureTransition emanating from the invoking state
// pushed on the stack).
// Reset the context to reflect the fact we invoked rule
NFAContext newContext = new NFAContext(context, p);
//System.out.println("invoking rule "+ref.rule.name);
// System.out.println(" context="+context);
// traverse epsilon edge to new rule
NFAState ruleTarget = (NFAState)ref.target;
closure(ruleTarget, alt, newContext, semanticContext, d, collectPredicates);
}
// Case 2: end of rule state, context (i.e., an invoker) exists
else if ( p.isAcceptState() && context.parent!=null ) {
NFAState whichStateInvokedRule = context.invokingState;
RuleClosureTransition edgeToRule =
(RuleClosureTransition)whichStateInvokedRule.transition[0];
NFAState continueState = edgeToRule.followState;
NFAContext newContext = context.parent; // "pop" invoking state
closure(continueState, alt, newContext, semanticContext, d, collectPredicates);
}
// Case 3: end of rule state, nobody invoked this rule (no context)
// Fall thru to be handled by case 4 automagically.
// Case 4: ordinary NFA->DFA conversion case: simple epsilon transition
else {
// recurse down any epsilon transitions
if ( transition0!=null && transition0.isEpsilon() ) {
boolean collectPredicatesAfterAction = collectPredicates;
if ( transition0.isAction() && collectPredicates ) {
collectPredicatesAfterAction = false;
/*
if ( computingStartState ) {
System.out.println("found action during prediction closure "+((ActionLabel)transition0.label).actionAST.token);
}
*/
}
closure((NFAState)transition0.target,
alt,
context,
semanticContext,
d,
collectPredicatesAfterAction
);
}
else if ( transition0!=null && transition0.isSemanticPredicate() ) {
SemanticContext labelContext = transition0.label.getSemanticContext();
if ( computingStartState ) {
if ( collectPredicates ) {
// only indicate we can see a predicate if we're collecting preds
// Could be computing start state & seen an action before this.
dfa.predicateVisible = true;
}
else {
// this state has a pred, but we can't see it.
dfa.hasPredicateBlockedByAction = true;
// System.out.println("found pred during prediction but blocked by action found previously");
}
}
// continue closure here too, but add the sem pred to ctx
SemanticContext newSemanticContext = semanticContext;
if ( collectPredicates ) {
// AND the previous semantic context with new pred
// do not hoist syn preds from other rules; only get if in
// starting state's rule (i.e., context is empty)
int walkAlt =
dfa.decisionNFAStartState.translateDisplayAltToWalkAlt(alt);
NFAState altLeftEdge =
dfa.nfa.grammar.getNFAStateForAltOfDecision(dfa.decisionNFAStartState,walkAlt);
/*
System.out.println("state "+p.stateNumber+" alt "+alt+" walkAlt "+walkAlt+" trans to "+transition0.target);
System.out.println("DFA start state "+dfa.decisionNFAStartState.stateNumber);
System.out.println("alt left edge "+altLeftEdge.stateNumber+
", epsilon target "+
altLeftEdge.transition(0).target.stateNumber);
*/
if ( !labelContext.isSyntacticPredicate() ||
p==altLeftEdge.transition[0].target )
{
//System.out.println("&"+labelContext+" enclosingRule="+p.enclosingRule);
newSemanticContext =
SemanticContext.and(semanticContext, labelContext);
}
}
closure((NFAState)transition0.target,
alt,
context,
newSemanticContext,
d,
collectPredicates);
}
Transition transition1 = p.transition[1];
if ( transition1!=null && transition1.isEpsilon() ) {
closure((NFAState)transition1.target,
alt,
context,
semanticContext,
d,
collectPredicates);
}
}
// don't remove "busy" flag as we want to prevent all
// references to same config of state|alt|ctx|semCtx even
// if resulting from another NFA state
}
/** A closure operation should abort if that computation has already
* been done or a computation with a conflicting context has already
* been done. If proposed NFA config's state and alt are the same
* there is potentially a problem. If the stack context is identical
* then clearly the exact same computation is proposed. If a context
* is a suffix of the other, then again the computation is in an
* identical context. ?$ and ??$ are considered the same stack.
* We could walk configurations linearly doing the comparison instead
* of a set for exact matches but it's much slower because you can't
* do a Set lookup. I use exact match as ANTLR
* always detect the conflict later when checking for context suffixes...
* I check for left-recursive stuff and terminate before analysis to
* avoid need to do this more expensive computation.
*
* 12-31-2007: I had to use the loop again rather than simple
* closureBusy.contains(proposedNFAConfiguration) lookup. The
* semantic context should not be considered when determining if
* a closure operation is busy. I saw a FOLLOW closure operation
* spin until time out because the predicate context kept increasing
* in size even though it's same boolean value. This seems faster also
* because I'm not doing String.equals on the preds all the time.
*
* 05-05-2008: Hmm...well, i think it was a mistake to remove the sem
* ctx check below...adding back in. Coincides with report of ANTLR
* getting super slow: http://www.antlr.org:8888/browse/ANTLR-235
* This could be because it doesn't properly compute then resolve
* a predicate expression. Seems to fix unit test:
* TestSemanticPredicates.testSemanticContextPreventsEarlyTerminationOfClosure()
* Changing back to Set from List. Changed a large grammar from 8 minutes
* to 11 seconds. Cool. Closing ANTLR-235.
*/
public static boolean closureIsBusy(DFAState d,
NFAConfiguration proposedNFAConfiguration)
{
return d.closureBusy.contains(proposedNFAConfiguration);
/*
int numConfigs = d.closureBusy.size();
// Check epsilon cycle (same state, same alt, same context)
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration c = (NFAConfiguration) d.closureBusy.get(i);
if ( proposedNFAConfiguration.state==c.state &&
proposedNFAConfiguration.alt==c.alt &&
proposedNFAConfiguration.semanticContext.equals(c.semanticContext) &&
proposedNFAConfiguration.context.suffix(c.context) )
{
return true;
}
}
return false;
*/
}
/** Given the set of NFA states in DFA state d, find all NFA states
* reachable traversing label arcs. By definition, there can be
* only one DFA state reachable by an atom from DFA state d so we must
* find and merge all NFA states reachable via label. Return a new
* DFAState that has all of those NFA states with their context (i.e.,
* which alt do they predict and where to return to if they fall off
* end of a rule).
*
* Because we cannot jump to another rule nor fall off the end of a rule
* via a non-epsilon transition, NFA states reachable from d have the
* same configuration as the NFA state in d. So if NFA state 7 in d's
* configurations can reach NFA state 13 then 13 will be added to the
* new DFAState (labelDFATarget) with the same configuration as state
* 7 had.
*
* This method does not see EOT transitions off the end of token rule
* accept states if the rule was invoked by somebody.
*/
public DFAState reach(DFAState d, Label label) {
//System.out.println("reach "+label.toString(dfa.nfa.grammar)+" from "+d.stateNumber);
DFAState labelDFATarget = dfa.newState();
// for each NFA state in d with a labeled edge,
// add in target states for label
//System.out.println("size(d.state="+d.stateNumber+")="+d.nfaConfigurations.size());
//System.out.println("size(labeled edge states)="+d.configurationsWithLabeledEdges.size());
List<NFAConfiguration> configs = d.configurationsWithLabeledEdges;
int numConfigs = configs.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration c = configs.get(i);
if ( c.resolved || c.resolveWithPredicate ) {
continue; // the conflict resolver indicates we must leave alone
}
NFAState p = dfa.nfa.getState(c.state);
// by design of the grammar->NFA conversion, only transition 0
// may have a non-epsilon edge.
Transition edge = p.transition[0];
if ( edge==null || !c.singleAtomTransitionEmanating ) {
continue;
}
Label edgeLabel = edge.label;
// SPECIAL CASE
// if it's an EOT transition on end of lexer rule, but context
// stack is not empty, then don't see the EOT; the closure
// will have added in the proper states following the reference
// to this rule in the invoking rule. In other words, if
// somebody called this rule, don't see the EOT emanating from
// this accept state.
if ( c.context.parent!=null && edgeLabel.label==Label.EOT ) {
continue;
}
// Labels not unique at this point (not until addReachableLabels)
// so try simple int label match before general set intersection
//System.out.println("comparing "+edgeLabel+" with "+label);
if ( Label.intersect(label, edgeLabel) ) {
// found a transition with label;
// add NFA target to (potentially) new DFA state
NFAConfiguration newC = labelDFATarget.addNFAConfiguration(
(NFAState)edge.target,
c.alt,
c.context,
c.semanticContext);
}
}
if ( labelDFATarget.nfaConfigurations.size()==0 ) {
// kill; it's empty
dfa.setState(labelDFATarget.stateNumber, null);
labelDFATarget = null;
}
return labelDFATarget;
}
/** Walk the configurations of this DFA state d looking for the
* configuration, c, that has a transition on EOT. State d should
* be converted to an accept state predicting the c.alt. Blast
* d's current configuration set and make it just have config c.
*
* TODO: can there be more than one config with EOT transition?
* That would mean that two NFA configurations could reach the
* end of the token with possibly different predicted alts.
* Seems like that would be rare or impossible. Perhaps convert
* this routine to find all such configs and give error if >1.
*/
protected void convertToEOTAcceptState(DFAState d) {
Label eot = new Label(Label.EOT);
int numConfigs = d.nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration c = (NFAConfiguration)d.nfaConfigurations.get(i);
if ( c.resolved || c.resolveWithPredicate ) {
continue; // the conflict resolver indicates we must leave alone
}
NFAState p = dfa.nfa.getState(c.state);
Transition edge = p.transition[0];
Label edgeLabel = edge.label;
if ( edgeLabel.equals(eot) ) {
//System.out.println("config with EOT: "+c);
d.setAcceptState(true);
//System.out.println("d goes from "+d);
d.nfaConfigurations.clear();
d.addNFAConfiguration(p,c.alt,c.context,c.semanticContext);
//System.out.println("to "+d);
return; // assume only one EOT transition
}
}
}
/** Add a new DFA state to the DFA if not already present.
* If the DFA state uniquely predicts a single alternative, it
* becomes a stop state; don't add to work list. Further, if
* there exists an NFA state predicted by > 1 different alternatives
* and with the same syn and sem context, the DFA is nondeterministic for
* at least one input sequence reaching that NFA state.
*/
protected DFAState addDFAStateToWorkList(DFAState d) {
DFAState existingState = dfa.addState(d);
if ( d != existingState ) {
// already there...use/return the existing DFA state.
// But also set the states[d.stateNumber] to the existing
// DFA state because the closureIsBusy must report
// infinite recursion on a state before it knows
// whether or not the state will already be
// found after closure on it finishes. It could be
// referring to a state that will ultimately not make it
// into the reachable state space and the error
// reporting must be able to compute the path from
// start to the error state with infinite recursion
dfa.setState(d.stateNumber, existingState);
return existingState;
}
// if not there, then examine new state.
// resolve syntactic conflicts by choosing a single alt or
// by using semantic predicates if present.
resolveNonDeterminisms(d);
// If deterministic, don't add this state; it's an accept state
// Just return as a valid DFA state
int alt = d.getUniquelyPredictedAlt();
if ( alt!=NFA.INVALID_ALT_NUMBER ) { // uniquely predicts an alt?
d = convertToAcceptState(d, alt);
/*
System.out.println("convert to accept; DFA "+d.dfa.decisionNumber+" state "+d.stateNumber+" uniquely predicts alt "+
d.getUniquelyPredictedAlt());
*/
}
else {
// unresolved, add to work list to continue NFA conversion
work.add(d);
}
return d;
}
protected DFAState convertToAcceptState(DFAState d, int alt) {
// only merge stop states if they are deterministic and no
// recursion problems and only if they have the same gated pred
// context!
// Later, the error reporting may want to trace the path from
// the start state to the nondet state
if ( DFAOptimizer.MERGE_STOP_STATES &&
d.getNonDeterministicAlts()==null &&
!d.abortedDueToRecursionOverflow &&
!d.abortedDueToMultipleRecursiveAlts )
{
// check to see if we already have an accept state for this alt
// [must do this after we resolve nondeterminisms in general]
DFAState acceptStateForAlt = dfa.getAcceptState(alt);
if ( acceptStateForAlt!=null ) {
// we already have an accept state for alt;
// Are their gate sem pred contexts the same?
// For now we assume a braindead version: both must not
// have gated preds or share exactly same single gated pred.
// The equals() method is only defined on Predicate contexts not
// OR etc...
SemanticContext gatedPreds = d.getGatedPredicatesInNFAConfigurations();
SemanticContext existingStateGatedPreds =
acceptStateForAlt.getGatedPredicatesInNFAConfigurations();
if ( (gatedPreds==null && existingStateGatedPreds==null) ||
((gatedPreds!=null && existingStateGatedPreds!=null) &&
gatedPreds.equals(existingStateGatedPreds)) )
{
// make this d.statenumber point at old DFA state
dfa.setState(d.stateNumber, acceptStateForAlt);
dfa.removeState(d); // remove this state from unique DFA state set
d = acceptStateForAlt; // use old accept state; throw this one out
return d;
}
// else consider it a new accept state; fall through.
}
}
d.setAcceptState(true); // new accept state for alt
dfa.setAcceptState(alt, d);
return d;
}
/** If > 1 NFA configurations within this DFA state have identical
* NFA state and context, but differ in their predicted
* TODO update for new context suffix stuff 3-9-2005
* alternative then a single input sequence predicts multiple alts.
* The NFA decision is therefore syntactically indistinguishable
* from the left edge upon at least one input sequence. We may
* terminate the NFA to DFA conversion for these paths since no
* paths emanating from those NFA states can possibly separate
* these conjoined twins once interwined to make things
* deterministic (unless there are semantic predicates; see below).
*
* Upon a nondeterministic set of NFA configurations, we should
* report a problem to the grammar designer and resolve the issue
* by aribitrarily picking the first alternative (this usually
* ends up producing the most natural behavior). Pick the lowest
* alt number and just turn off all NFA configurations
* associated with the other alts. Rather than remove conflicting
* NFA configurations, I set the "resolved" bit so that future
* computations will ignore them. In this way, we maintain the
* complete DFA state with all its configurations, but prevent
* future DFA conversion operations from pursuing undesirable
* paths. Remember that we want to terminate DFA conversion as
* soon as we know the decision is deterministic *or*
* nondeterministic.
*
* [BTW, I have convinced myself that there can be at most one
* set of nondeterministic configurations in a DFA state. Only NFA
* configurations arising from the same input sequence can appear
* in a DFA state. There is no way to have another complete set
* of nondeterministic NFA configurations without another input
* sequence, which would reach a different DFA state. Therefore,
* the two nondeterministic NFA configuration sets cannot collide
* in the same DFA state.]
*
* Consider DFA state {(s|1),(s|2),(s|3),(t|3),(v|4)} where (s|a)
* is state 's' and alternative 'a'. Here, configuration set
* {(s|1),(s|2),(s|3)} predicts 3 different alts. Configurations
* (s|2) and (s|3) are "resolved", leaving {(s|1),(t|3),(v|4)} as
* items that must still be considered by the DFA conversion
* algorithm in DFA.findNewDFAStatesAndAddDFATransitions().
*
* Consider the following grammar where alts 1 and 2 are no
* problem because of the 2nd lookahead symbol. Alts 3 and 4 are
* identical and will therefore reach the rule end NFA state but
* predicting 2 different alts (no amount of future lookahead
* will render them deterministic/separable):
*
* a : A B
* | A C
* | A
* | A
* ;
*
* Here is a (slightly reduced) NFA of this grammar:
*
* (1)-A->(2)-B->(end)-EOF->(8)
* | ^
* (2)-A->(3)-C----|
* | ^
* (4)-A->(5)------|
* | ^
* (6)-A->(7)------|
*
* where (n) is NFA state n. To begin DFA conversion, the start
* state is created:
*
* {(1|1),(2|2),(4|3),(6|4)}
*
* Upon A, all NFA configurations lead to new NFA states yielding
* new DFA state:
*
* {(2|1),(3|2),(5|3),(7|4),(end|3),(end|4)}
*
* where the configurations with state end in them are added
* during the epsilon closure operation. State end predicts both
* alts 3 and 4. An error is reported, the latter configuration is
* flagged as resolved leaving the DFA state as:
*
* {(2|1),(3|2),(5|3),(7|4|resolved),(end|3),(end|4|resolved)}
*
* As NFA configurations are added to a DFA state during its
* construction, the reachable set of labels is computed. Here
* reachable is {B,C,EOF} because there is at least one NFA state
* in the DFA state that can transition upon those symbols.
*
* The final DFA looks like:
*
* {(1|1),(2|2),(4|3),(6|4)}
* |
* v
* {(2|1),(3|2),(5|3),(7|4),(end|3),(end|4)} -B-> (end|1)
* | |
* C ----EOF-> (8,3)
* |
* v
* (end|2)
*
* Upon AB, alt 1 is predicted. Upon AC, alt 2 is predicted.
* Upon A EOF, alt 3 is predicted. Alt 4 is not a viable
* alternative.
*
* The algorithm is essentially to walk all the configurations
* looking for a conflict of the form (s|i) and (s|j) for i!=j.
* Use a hash table to track state+context pairs for collisions
* so that we have O(n) to walk the n configurations looking for
* a conflict. Upon every conflict, track the alt number so
* we have a list of all nondeterministically predicted alts. Also
* track the minimum alt. Next go back over the configurations, setting
* the "resolved" bit for any that have an alt that is a member of
* the nondeterministic set. This will effectively remove any alts
* but the one we want from future consideration.
*
* See resolveWithSemanticPredicates()
*
* AMBIGUOUS TOKENS
*
* With keywords and ID tokens, there is an inherit ambiguity in that
* "int" can be matched by ID also. Each lexer rule has an EOT
* transition emanating from it which is used whenever the end of
* a rule is reached and another token rule did not invoke it. EOT
* is the only thing that can be seen next. If two rules are identical
* like "int" and "int" then the 2nd def is unreachable and you'll get
* a warning. We prevent a warning though for the keyword/ID issue as
* ID is still reachable. This can be a bit weird. '+' rule then a
* '+'|'+=' rule will fail to match '+' for the 2nd rule.
*
* If all NFA states in this DFA state are targets of EOT transitions,
* (and there is more than one state plus no unique alt is predicted)
* then DFA conversion will leave this state as a dead state as nothing
* can be reached from this state. To resolve the ambiguity, just do
* what flex and friends do: pick the first rule (alt in this case) to
* win. This means you should put keywords before the ID rule.
* If the DFA state has only one NFA state then there is no issue:
* it uniquely predicts one alt. :) Problem
* states will look like this during conversion:
*
* DFA 1:{9|1, 19|2, 14|3, 20|2, 23|2, 24|2, ...}-<EOT>->5:{41|3, 42|2}
*
* Worse, when you have two identical literal rules, you will see 3 alts
* in the EOT state (one for ID and one each for the identical rules).
*/
public void resolveNonDeterminisms(DFAState d) {
if ( debug ) {
System.out.println("resolveNonDeterminisms "+d.toString());
}
boolean conflictingLexerRules = false;
Set nondeterministicAlts = d.getNonDeterministicAlts();
if ( debug && nondeterministicAlts!=null ) {
System.out.println("nondet alts="+nondeterministicAlts);
}
// CHECK FOR AMBIGUOUS EOT (if |allAlts|>1 and EOT state, resolve)
// grab any config to see if EOT state; any other configs must
// transition on EOT to get to this DFA state as well so all
// states in d must be targets of EOT. These are the end states
// created in NFAFactory.build_EOFState
NFAConfiguration anyConfig = d.nfaConfigurations.get(0);
NFAState anyState = dfa.nfa.getState(anyConfig.state);
// if d is target of EOT and more than one predicted alt
// indicate that d is nondeterministic on all alts otherwise
// it looks like state has no problem
if ( anyState.isEOTTargetState() ) {
Set allAlts = d.getAltSet();
// is more than 1 alt predicted?
if ( allAlts!=null && allAlts.size()>1 ) {
nondeterministicAlts = allAlts;
// track Tokens rule issues differently than other decisions
if ( d.dfa.isTokensRuleDecision() ) {
dfa.probe.reportLexerRuleNondeterminism(d,allAlts);
//System.out.println("Tokens rule DFA state "+d+" nondeterministic");
conflictingLexerRules = true;
}
}
}
// if no problems return unless we aborted work on d to avoid inf recursion
if ( !d.abortedDueToRecursionOverflow && nondeterministicAlts==null ) {
return; // no problems, return
}
// if we're not a conflicting lexer rule and we didn't abort, report ambig
// We should get a report for abort so don't give another
if ( !d.abortedDueToRecursionOverflow && !conflictingLexerRules ) {
// TODO: with k=x option set, this is called twice for same state
dfa.probe.reportNondeterminism(d, nondeterministicAlts);
// TODO: how to turn off when it's only the FOLLOW that is
// conflicting. This used to shut off even alts i,j < n
// conflict warnings. :(
}
// ATTEMPT TO RESOLVE WITH SEMANTIC PREDICATES
boolean resolved =
tryToResolveWithSemanticPredicates(d, nondeterministicAlts);
if ( resolved ) {
if ( debug ) {
System.out.println("resolved DFA state "+d.stateNumber+" with pred");
}
d.resolvedWithPredicates = true;
dfa.probe.reportNondeterminismResolvedWithSemanticPredicate(d);
return;
}
// RESOLVE SYNTACTIC CONFLICT BY REMOVING ALL BUT ONE ALT
resolveByChoosingFirstAlt(d, nondeterministicAlts);
//System.out.println("state "+d.stateNumber+" resolved to alt "+winningAlt);
}
protected int resolveByChoosingFirstAlt(DFAState d, Set nondeterministicAlts) {
int winningAlt = 0;
if ( dfa.isGreedy() ) {
winningAlt = resolveByPickingMinAlt(d,nondeterministicAlts);
}
else {
// If nongreedy, the exit alt shout win, but only if it's
// involved in the nondeterminism!
/*
System.out.println("resolving exit alt for decision="+
dfa.decisionNumber+" state="+d);
System.out.println("nondet="+nondeterministicAlts);
System.out.println("exit alt "+exitAlt);
*/
int exitAlt = dfa.getNumberOfAlts();
if ( nondeterministicAlts.contains(Utils.integer(exitAlt)) ) {
// if nongreedy and exit alt is one of those nondeterministic alts
// predicted, resolve in favor of what follows block
winningAlt = resolveByPickingExitAlt(d,nondeterministicAlts);
}
else {
winningAlt = resolveByPickingMinAlt(d,nondeterministicAlts);
}
}
return winningAlt;
}
/** Turn off all configurations associated with the
* set of incoming nondeterministic alts except the min alt number.
* There may be many alts among the configurations but only turn off
* the ones with problems (other than the min alt of course).
*
* If nondeterministicAlts is null then turn off all configs 'cept those
* associated with the minimum alt.
*
* Return the min alt found.
*/
protected int resolveByPickingMinAlt(DFAState d, Set nondeterministicAlts) {
int min = Integer.MAX_VALUE;
if ( nondeterministicAlts!=null ) {
min = getMinAlt(nondeterministicAlts);
}
else {
min = d.minAltInConfigurations;
}
turnOffOtherAlts(d, min, nondeterministicAlts);
return min;
}
/** Resolve state d by choosing exit alt, which is same value as the
* number of alternatives. Return that exit alt.
*/
protected int resolveByPickingExitAlt(DFAState d, Set nondeterministicAlts) {
int exitAlt = dfa.getNumberOfAlts();
turnOffOtherAlts(d, exitAlt, nondeterministicAlts);
return exitAlt;
}
/** turn off all states associated with alts other than the good one
* (as long as they are one of the nondeterministic ones)
*/
protected static void turnOffOtherAlts(DFAState d, int min, Set<Integer> nondeterministicAlts) {
int numConfigs = d.nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration)d.nfaConfigurations.get(i);
if ( configuration.alt!=min ) {
if ( nondeterministicAlts==null ||
nondeterministicAlts.contains(Utils.integer(configuration.alt)) )
{
configuration.resolved = true;
}
}
}
}
protected static int getMinAlt(Set<Integer> nondeterministicAlts) {
int min = Integer.MAX_VALUE;
for (Integer altI : nondeterministicAlts) {
int alt = altI.intValue();
if ( alt < min ) {
min = alt;
}
}
return min;
}
/** See if a set of nondeterministic alternatives can be disambiguated
* with the semantic predicate contexts of the alternatives.
*
* Without semantic predicates, syntactic conflicts are resolved
* by simply choosing the first viable alternative. In the
* presence of semantic predicates, you can resolve the issue by
* evaluating boolean expressions at run time. During analysis,
* this amounts to suppressing grammar error messages to the
* developer. NFA configurations are always marked as "to be
* resolved with predicates" so that
* DFA.findNewDFAStatesAndAddDFATransitions() will know to ignore
* these configurations and add predicate transitions to the DFA
* after adding token/char labels.
*
* During analysis, we can simply make sure that for n
* ambiguously predicted alternatives there are at least n-1
* unique predicate sets. The nth alternative can be predicted
* with "not" the "or" of all other predicates. NFA configurations without
* predicates are assumed to have the default predicate of
* "true" from a user point of view. When true is combined via || with
* another predicate, the predicate is a tautology and must be removed
* from consideration for disambiguation:
*
* a : b | B ; // hoisting p1||true out of rule b, yields no predicate
* b : {p1}? B | B ;
*
* This is done down in getPredicatesPerNonDeterministicAlt().
*/
protected boolean tryToResolveWithSemanticPredicates(DFAState d,
Set nondeterministicAlts)
{
Map<Integer, SemanticContext> altToPredMap =
getPredicatesPerNonDeterministicAlt(d, nondeterministicAlts);
if ( altToPredMap.size()==0 ) {
return false;
}
//System.out.println("nondeterministic alts with predicates: "+altToPredMap);
dfa.probe.reportAltPredicateContext(d, altToPredMap);
if ( nondeterministicAlts.size()-altToPredMap.size()>1 ) {
// too few predicates to resolve; just return
// TODO: actually do we need to gen error here?
return false;
}
// Handle case where 1 predicate is missing
// Case 1. Semantic predicates
// If the missing pred is on nth alt, !(union of other preds)==true
// so we can avoid that computation. If naked alt is ith, then must
// test it with !(union) since semantic predicated alts are order
// independent
// Case 2: Syntactic predicates
// The naked alt is always assumed to be true as the order of
// alts is the order of precedence. The naked alt will be a tautology
// anyway as it's !(union of other preds). This implies
// that there is no such thing as noviable alt for synpred edges
// emanating from a DFA state.
if ( altToPredMap.size()==nondeterministicAlts.size()-1 ) {
// if there are n-1 predicates for n nondeterministic alts, can fix
org.antlr.misc.BitSet ndSet = org.antlr.misc.BitSet.of(nondeterministicAlts);
org.antlr.misc.BitSet predSet = org.antlr.misc.BitSet.of(altToPredMap);
int nakedAlt = ndSet.subtract(predSet).getSingleElement();
SemanticContext nakedAltPred = null;
if ( nakedAlt == max(nondeterministicAlts) ) {
// the naked alt is the last nondet alt and will be the default clause
nakedAltPred = new SemanticContext.TruePredicate();
}
else {
// pretend naked alternative is covered with !(union other preds)
// unless it's a synpred since those have precedence same
// as alt order
SemanticContext unionOfPredicatesFromAllAlts =
getUnionOfPredicates(altToPredMap);
//System.out.println("all predicates "+unionOfPredicatesFromAllAlts);
if ( unionOfPredicatesFromAllAlts.isSyntacticPredicate() ) {
nakedAltPred = new SemanticContext.TruePredicate();
}
else {
nakedAltPred =
SemanticContext.not(unionOfPredicatesFromAllAlts);
}
}
//System.out.println("covering naked alt="+nakedAlt+" with "+nakedAltPred);
altToPredMap.put(Utils.integer(nakedAlt), nakedAltPred);
// set all config with alt=nakedAlt to have the computed predicate
int numConfigs = d.nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration)d.nfaConfigurations.get(i);
if ( configuration.alt == nakedAlt ) {
configuration.semanticContext = nakedAltPred;
}
}
}
if ( altToPredMap.size()==nondeterministicAlts.size() ) {
// RESOLVE CONFLICT by picking one NFA configuration for each alt
// and setting its resolvedWithPredicate flag
// First, prevent a recursion warning on this state due to
// pred resolution
if ( d.abortedDueToRecursionOverflow ) {
d.dfa.probe.removeRecursiveOverflowState(d);
}
int numConfigs = d.nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration)d.nfaConfigurations.get(i);
SemanticContext semCtx = (SemanticContext)
altToPredMap.get(Utils.integer(configuration.alt));
if ( semCtx!=null ) {
// resolve (first found) with pred
// and remove alt from problem list
configuration.resolveWithPredicate = true;
configuration.semanticContext = semCtx; // reset to combined
altToPredMap.remove(Utils.integer(configuration.alt));
// notify grammar that we've used the preds contained in semCtx
if ( semCtx.isSyntacticPredicate() ) {
dfa.nfa.grammar.synPredUsedInDFA(dfa, semCtx);
}
}
else if ( nondeterministicAlts.contains(Utils.integer(configuration.alt)) ) {
// resolve all configurations for nondeterministic alts
// for which there is no predicate context by turning it off
configuration.resolved = true;
}
}
return true;
}
return false; // couldn't fix the problem with predicates
}
/** Return a mapping from nondeterministc alt to combined list of predicates.
* If both (s|i|semCtx1) and (t|i|semCtx2) exist, then the proper predicate
* for alt i is semCtx1||semCtx2 because you have arrived at this single
* DFA state via two NFA paths, both of which have semantic predicates.
* We ignore deterministic alts because syntax alone is sufficient
* to predict those. Do not include their predicates.
*
* Alts with no predicate are assumed to have {true}? pred.
*
* When combining via || with "true", all predicates are removed from
* consideration since the expression will always be true and hence
* not tell us how to resolve anything. So, if any NFA configuration
* in this DFA state does not have a semantic context, the alt cannot
* be resolved with a predicate.
*
* If nonnull, incidentEdgeLabel tells us what NFA transition label
* we did a reach on to compute state d. d may have insufficient
* preds, so we really want this for the error message.
*/
protected Map<Integer, SemanticContext> getPredicatesPerNonDeterministicAlt(DFAState d,
Set nondeterministicAlts)
{
// map alt to combined SemanticContext
Map<Integer, SemanticContext> altToPredicateContextMap =
new HashMap<Integer, SemanticContext>();
// init the alt to predicate set map
Map<Integer, Set<SemanticContext>> altToSetOfContextsMap =
new HashMap<Integer, Set<SemanticContext>>();
for (Iterator it = nondeterministicAlts.iterator(); it.hasNext();) {
Integer altI = (Integer) it.next();
altToSetOfContextsMap.put(altI, new HashSet<SemanticContext>());
}
/*
List<Label> sampleInputLabels = d.dfa.probe.getSampleNonDeterministicInputSequence(d);
String input = d.dfa.probe.getInputSequenceDisplay(sampleInputLabels);
System.out.println("sample input: "+input);
*/
// for each configuration, create a unique set of predicates
// Also, track the alts with at least one uncovered configuration
// (one w/o a predicate); tracks tautologies like p1||true
Map<Integer, Set<Token>> altToLocationsReachableWithoutPredicate = new HashMap<Integer, Set<Token>>();
Set<Integer> nondetAltsWithUncoveredConfiguration = new HashSet<Integer>();
//System.out.println("configs="+d.nfaConfigurations);
//System.out.println("configs with preds?"+d.atLeastOneConfigurationHasAPredicate);
//System.out.println("configs with preds="+d.configurationsWithPredicateEdges);
int numConfigs = d.nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration)d.nfaConfigurations.get(i);
Integer altI = Utils.integer(configuration.alt);
// if alt is nondeterministic, combine its predicates
if ( nondeterministicAlts.contains(altI) ) {
// if there is a predicate for this NFA configuration, OR in
if ( configuration.semanticContext !=
SemanticContext.EMPTY_SEMANTIC_CONTEXT )
{
Set<SemanticContext> predSet = altToSetOfContextsMap.get(altI);
predSet.add(configuration.semanticContext);
}
else {
// if no predicate, but it's part of nondeterministic alt
// then at least one path exists not covered by a predicate.
// must remove predicate for this alt; track incomplete alts
nondetAltsWithUncoveredConfiguration.add(altI);
/*
NFAState s = dfa.nfa.getState(configuration.state);
System.out.println("###\ndec "+dfa.decisionNumber+" alt "+configuration.alt+
" enclosing rule for nfa state not covered "+
s.enclosingRule);
if ( s.associatedASTNode!=null ) {
System.out.println("token="+s.associatedASTNode.token);
}
System.out.println("nfa state="+s);
if ( s.incidentEdgeLabel!=null && Label.intersect(incidentEdgeLabel, s.incidentEdgeLabel) ) {
Set<Token> locations = altToLocationsReachableWithoutPredicate.get(altI);
if ( locations==null ) {
locations = new HashSet<Token>();
altToLocationsReachableWithoutPredicate.put(altI, locations);
}
locations.add(s.associatedASTNode.token);
}
*/
}
}
}
// For each alt, OR together all unique predicates associated with
// all configurations
// Also, track the list of incompletely covered alts: those alts
// with at least 1 predicate and at least one configuration w/o a
// predicate. We want this in order to report to the decision probe.
List<Integer> incompletelyCoveredAlts = new ArrayList<Integer>();
for (Iterator it = nondeterministicAlts.iterator(); it.hasNext();) {
Integer altI = (Integer) it.next();
Set<SemanticContext> contextsForThisAlt = altToSetOfContextsMap.get(altI);
if ( nondetAltsWithUncoveredConfiguration.contains(altI) ) { // >= 1 config has no ctx
if ( contextsForThisAlt.size()>0 ) { // && at least one pred
incompletelyCoveredAlts.add(altI); // this alt incompleted covered
}
continue; // don't include at least 1 config has no ctx
}
SemanticContext combinedContext = null;
for (Iterator itrSet = contextsForThisAlt.iterator(); itrSet.hasNext();) {
SemanticContext ctx = (SemanticContext) itrSet.next();
combinedContext =
SemanticContext.or(combinedContext,ctx);
}
altToPredicateContextMap.put(altI, combinedContext);
}
if ( incompletelyCoveredAlts.size()>0 ) {
/*
System.out.println("prob in dec "+dfa.decisionNumber+" state="+d);
FASerializer serializer = new FASerializer(dfa.nfa.grammar);
String result = serializer.serialize(dfa.startState);
System.out.println("dfa: "+result);
System.out.println("incomplete alts: "+incompletelyCoveredAlts);
System.out.println("nondet="+nondeterministicAlts);
System.out.println("nondetAltsWithUncoveredConfiguration="+ nondetAltsWithUncoveredConfiguration);
System.out.println("altToCtxMap="+altToSetOfContextsMap);
System.out.println("altToPredicateContextMap="+altToPredicateContextMap);
*/
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration)d.nfaConfigurations.get(i);
Integer altI = Utils.integer(configuration.alt);
if ( incompletelyCoveredAlts.contains(altI) &&
configuration.semanticContext == SemanticContext.EMPTY_SEMANTIC_CONTEXT )
{
NFAState s = dfa.nfa.getState(configuration.state);
/*
System.out.print("nondet config w/o context "+configuration+
" incident "+(s.incidentEdgeLabel!=null?s.incidentEdgeLabel.toString(dfa.nfa.grammar):null));
if ( s.associatedASTNode!=null ) {
System.out.print(" token="+s.associatedASTNode.token);
}
else System.out.println();
*/
// We want to report getting to an NFA state with an
// incoming label, unless it's EOF, w/o a predicate.
if ( s.incidentEdgeLabel!=null && s.incidentEdgeLabel.label != Label.EOF ) {
if ( s.associatedASTNode==null || s.associatedASTNode.token==null ) {
ErrorManager.internalError("no AST/token for nonepsilon target w/o predicate");
}
else {
Set<Token> locations = altToLocationsReachableWithoutPredicate.get(altI);
if ( locations==null ) {
locations = new HashSet<Token>();
altToLocationsReachableWithoutPredicate.put(altI, locations);
}
locations.add(s.associatedASTNode.token);
}
}
}
}
dfa.probe.reportIncompletelyCoveredAlts(d,
altToLocationsReachableWithoutPredicate);
}
return altToPredicateContextMap;
}
/** OR together all predicates from the alts. Note that the predicate
* for an alt could itself be a combination of predicates.
*/
protected static SemanticContext getUnionOfPredicates(Map altToPredMap) {
Iterator iter;
SemanticContext unionOfPredicatesFromAllAlts = null;
iter = altToPredMap.values().iterator();
while ( iter.hasNext() ) {
SemanticContext semCtx = (SemanticContext)iter.next();
if ( unionOfPredicatesFromAllAlts==null ) {
unionOfPredicatesFromAllAlts = semCtx;
}
else {
unionOfPredicatesFromAllAlts =
SemanticContext.or(unionOfPredicatesFromAllAlts,semCtx);
}
}
return unionOfPredicatesFromAllAlts;
}
/** for each NFA config in d, look for "predicate required" sign set
* during nondeterminism resolution.
*
* Add the predicate edges sorted by the alternative number; I'm fairly
* sure that I could walk the configs backwards so they are added to
* the predDFATarget in the right order, but it's best to make sure.
* Predicates succeed in the order they are specifed. Alt i wins
* over alt i+1 if both predicates are true.
*/
protected void addPredicateTransitions(DFAState d) {
List configsWithPreds = new ArrayList();
// get a list of all configs with predicates
int numConfigs = d.nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration c = (NFAConfiguration)d.nfaConfigurations.get(i);
if ( c.resolveWithPredicate ) {
configsWithPreds.add(c);
}
}
// Sort ascending according to alt; alt i has higher precedence than i+1
Collections.sort(configsWithPreds,
new Comparator() {
public int compare(Object a, Object b) {
NFAConfiguration ca = (NFAConfiguration)a;
NFAConfiguration cb = (NFAConfiguration)b;
if ( ca.alt < cb.alt ) return -1;
else if ( ca.alt > cb.alt ) return 1;
return 0;
}
});
List predConfigsSortedByAlt = configsWithPreds;
// Now, we can add edges emanating from d for these preds in right order
for (int i = 0; i < predConfigsSortedByAlt.size(); i++) {
NFAConfiguration c = (NFAConfiguration)predConfigsSortedByAlt.get(i);
DFAState predDFATarget = d.dfa.getAcceptState(c.alt);
if ( predDFATarget==null ) {
predDFATarget = dfa.newState(); // create if not there.
// create a new DFA state that is a target of the predicate from d
predDFATarget.addNFAConfiguration(dfa.nfa.getState(c.state),
c.alt,
c.context,
c.semanticContext);
predDFATarget.setAcceptState(true);
dfa.setAcceptState(c.alt, predDFATarget);
DFAState existingState = dfa.addState(predDFATarget);
if ( predDFATarget != existingState ) {
// already there...use/return the existing DFA state that
// is a target of this predicate. Make this state number
// point at the existing state
dfa.setState(predDFATarget.stateNumber, existingState);
predDFATarget = existingState;
}
}
// add a transition to pred target from d
d.addTransition(predDFATarget, new PredicateLabel(c.semanticContext));
}
}
protected void initContextTrees(int numberOfAlts) {
contextTrees = new NFAContext[numberOfAlts];
for (int i = 0; i < contextTrees.length; i++) {
int alt = i+1;
// add a dummy root node so that an NFA configuration can
// always point at an NFAContext. If a context refers to this
// node then it implies there is no call stack for
// that configuration
contextTrees[i] = new NFAContext(null, null);
}
}
public static int max(Set s) {
if ( s==null ) {
return Integer.MIN_VALUE;
}
int i = 0;
int m = 0;
for (Iterator it = s.iterator(); it.hasNext();) {
i++;
Integer I = (Integer) it.next();
if ( i==1 ) { // init m with first value
m = I.intValue();
continue;
}
if ( I.intValue()>m ) {
m = I.intValue();
}
}
return m;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/Transition.java 0000644 0001750 0001750 00000005777 11256465230 024620 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
/** A generic transition between any two state machine states. It defines
* some special labels that indicate things like epsilon transitions and
* that the label is actually a set of labels or a semantic predicate.
* This is a one way link. It emanates from a state (usually via a list of
* transitions) and has a label/target pair. I have abstracted the notion
* of a Label to handle the various kinds of things it can be.
*/
public class Transition implements Comparable {
/** What label must be consumed to transition to target */
public Label label;
/** The target of this transition */
public State target;
public Transition(Label label, State target) {
this.label = label;
this.target = target;
}
public Transition(int label, State target) {
this.label = new Label(label);
this.target = target;
}
public boolean isEpsilon() {
return label.isEpsilon();
}
public boolean isAction() {
return label.isAction();
}
public boolean isSemanticPredicate() {
return label.isSemanticPredicate();
}
public int hashCode() {
return label.hashCode() + target.stateNumber;
}
public boolean equals(Object o) {
Transition other = (Transition)o;
return this.label.equals(other.label) &&
this.target.equals(other.target);
}
public int compareTo(Object o) {
Transition other = (Transition)o;
return this.label.compareTo(other.label);
}
public String toString() {
return label+"->"+target.stateNumber;
}
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/DFAOptimizer.java 0000644 0001750 0001750 00000022430 11256465226 024751 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.tool.Grammar;
import org.antlr.misc.Utils;
import java.util.HashSet;
import java.util.Set;
/** A module to perform optimizations on DFAs.
*
* I could more easily (and more quickly) do some optimizations (such as
* PRUNE_EBNF_EXIT_BRANCHES) during DFA construction, but then it
* messes up the determinism checking. For example, it looks like
* loop exit branches are unreachable if you prune exit branches
* during DFA construction and before determinism checks.
*
* In general, ANTLR's NFA->DFA->codegen pipeline seems very robust
* to me which I attribute to a uniform and consistent set of data
* structures. Regardless of what I want to "say"/implement, I do so
* within the confines of, for example, a DFA. The code generator
* can then just generate code--it doesn't have to do much thinking.
* Putting optimizations in the code gen code really starts to make
* it a spagetti factory (uh oh, now I'm hungry!). The pipeline is
* very testable; each stage has well defined input/output pairs.
*
* ### Optimization: PRUNE_EBNF_EXIT_BRANCHES
*
* There is no need to test EBNF block exit branches. Not only is it
* an unneeded computation, but counter-intuitively, you actually get
* better errors. You can report an error at the missing or extra
* token rather than as soon as you've figured out you will fail.
*
* Imagine optional block "( DOT CLASS )? SEMI". ANTLR generates:
*
* int alt=0;
* if ( input.LA(1)==DOT ) {
* alt=1;
* }
* else if ( input.LA(1)==SEMI ) {
* alt=2;
* }
*
* Clearly, since Parser.match() will ultimately find the error, we
* do not want to report an error nor do we want to bother testing
* lookahead against what follows the (...)? We want to generate
* simply "should I enter the subrule?":
*
* int alt=2;
* if ( input.LA(1)==DOT ) {
* alt=1;
* }
*
* NOTE 1. Greedy loops cannot be optimized in this way. For example,
* "(greedy=false:'x'|.)* '\n'". You specifically need the exit branch
* to tell you when to terminate the loop as the same input actually
* predicts one of the alts (i.e., staying in the loop).
*
* NOTE 2. I do not optimize cyclic DFAs at the moment as it doesn't
* seem to work. ;) I'll have to investigate later to see what work I
* can do on cyclic DFAs to make them have fewer edges. Might have
* something to do with the EOT token.
*
* ### PRUNE_SUPERFLUOUS_EOT_EDGES
*
* When a token is a subset of another such as the following rules, ANTLR
* quietly assumes the first token to resolve the ambiguity.
*
* EQ : '=' ;
* ASSIGNOP : '=' | '+=' ;
*
* It can yield states that have only a single edge on EOT to an accept
* state. This is a waste and messes up my code generation. ;) If
* Tokens rule DFA goes
*
* s0 -'='-> s3 -EOT-> s5 (accept)
*
* then s5 should be pruned and s3 should be made an accept. Do NOT do this
* for keyword versus ID as the state with EOT edge emanating from it will
* also have another edge.
*
* ### Optimization: COLLAPSE_ALL_INCIDENT_EDGES
*
* Done during DFA construction. See method addTransition() in
* NFAToDFAConverter.
*
* ### Optimization: MERGE_STOP_STATES
*
* Done during DFA construction. See addDFAState() in NFAToDFAConverter.
*/
public class DFAOptimizer {
public static boolean PRUNE_EBNF_EXIT_BRANCHES = true;
public static boolean PRUNE_TOKENS_RULE_SUPERFLUOUS_EOT_EDGES = true;
public static boolean COLLAPSE_ALL_PARALLEL_EDGES = true;
public static boolean MERGE_STOP_STATES = true;
/** Used by DFA state machine generator to avoid infinite recursion
* resulting from cycles int the DFA. This is a set of int state #s.
* This is a side-effect of calling optimize; can't clear after use
* because code gen needs it.
*/
protected Set visited = new HashSet();
protected Grammar grammar;
public DFAOptimizer(Grammar grammar) {
this.grammar = grammar;
}
public void optimize() {
// optimize each DFA in this grammar
for (int decisionNumber=1;
decisionNumber<=grammar.getNumberOfDecisions();
decisionNumber++)
{
DFA dfa = grammar.getLookaheadDFA(decisionNumber);
optimize(dfa);
}
}
protected void optimize(DFA dfa) {
if ( dfa==null ) {
return; // nothing to do
}
/*
System.out.println("Optimize DFA "+dfa.decisionNFAStartState.decisionNumber+
" num states="+dfa.getNumberOfStates());
*/
//long start = System.currentTimeMillis();
if ( PRUNE_EBNF_EXIT_BRANCHES && dfa.canInlineDecision() ) {
visited.clear();
int decisionType =
dfa.getNFADecisionStartState().decisionStateType;
if ( dfa.isGreedy() &&
(decisionType==NFAState.OPTIONAL_BLOCK_START ||
decisionType==NFAState.LOOPBACK) )
{
optimizeExitBranches(dfa.startState);
}
}
// If the Tokens rule has syntactically ambiguous rules, try to prune
if ( PRUNE_TOKENS_RULE_SUPERFLUOUS_EOT_EDGES &&
dfa.isTokensRuleDecision() &&
dfa.probe.stateToSyntacticallyAmbiguousTokensRuleAltsMap.size()>0 )
{
visited.clear();
optimizeEOTBranches(dfa.startState);
}
/* ack...code gen needs this, cannot optimize
visited.clear();
unlinkUnneededStateData(dfa.startState);
*/
//long stop = System.currentTimeMillis();
//System.out.println("minimized in "+(int)(stop-start)+" ms");
}
protected void optimizeExitBranches(DFAState d) {
Integer sI = Utils.integer(d.stateNumber);
if ( visited.contains(sI) ) {
return; // already visited
}
visited.add(sI);
int nAlts = d.dfa.getNumberOfAlts();
for (int i = 0; i < d.getNumberOfTransitions(); i++) {
Transition edge = (Transition) d.transition(i);
DFAState edgeTarget = ((DFAState)edge.target);
/*
System.out.println(d.stateNumber+"-"+
edge.label.toString(d.dfa.nfa.grammar)+"->"+
edgeTarget.stateNumber);
*/
// if target is an accept state and that alt is the exit alt
if ( edgeTarget.isAcceptState() &&
edgeTarget.getUniquelyPredictedAlt()==nAlts)
{
/*
System.out.println("ignoring transition "+i+" to max alt "+
d.dfa.getNumberOfAlts());
*/
d.removeTransition(i);
i--; // back up one so that i++ of loop iteration stays within bounds
}
optimizeExitBranches(edgeTarget);
}
}
protected void optimizeEOTBranches(DFAState d) {
Integer sI = Utils.integer(d.stateNumber);
if ( visited.contains(sI) ) {
return; // already visited
}
visited.add(sI);
for (int i = 0; i < d.getNumberOfTransitions(); i++) {
Transition edge = (Transition) d.transition(i);
DFAState edgeTarget = ((DFAState)edge.target);
/*
System.out.println(d.stateNumber+"-"+
edge.label.toString(d.dfa.nfa.grammar)+"->"+
edgeTarget.stateNumber);
*/
// if only one edge coming out, it is EOT, and target is accept prune
if ( PRUNE_TOKENS_RULE_SUPERFLUOUS_EOT_EDGES &&
edgeTarget.isAcceptState() &&
d.getNumberOfTransitions()==1 &&
edge.label.isAtom() &&
edge.label.getAtom()==Label.EOT )
{
//System.out.println("state "+d+" can be pruned");
// remove the superfluous EOT edge
d.removeTransition(i);
d.setAcceptState(true); // make it an accept state
// force it to uniquely predict the originally predicted state
d.cachedUniquelyPredicatedAlt =
edgeTarget.getUniquelyPredictedAlt();
i--; // back up one so that i++ of loop iteration stays within bounds
}
optimizeEOTBranches(edgeTarget);
}
}
/** Walk DFA states, unlinking the nfa configs and whatever else I
* can to reduce memory footprint.
protected void unlinkUnneededStateData(DFAState d) {
Integer sI = Utils.integer(d.stateNumber);
if ( visited.contains(sI) ) {
return; // already visited
}
visited.add(sI);
d.nfaConfigurations = null;
for (int i = 0; i < d.getNumberOfTransitions(); i++) {
Transition edge = (Transition) d.transition(i);
DFAState edgeTarget = ((DFAState)edge.target);
unlinkUnneededStateData(edgeTarget);
}
}
*/
}
antlr-3.2/tool/src/main/java/org/antlr/analysis/DFAState.java 0000644 0001750 0001750 00000074470 11256465226 024062 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.analysis;
import org.antlr.misc.IntSet;
import org.antlr.misc.MultiMap;
import org.antlr.misc.OrderedHashSet;
import org.antlr.misc.Utils;
import org.antlr.tool.Grammar;
import java.util.*;
/** A DFA state represents a set of possible NFA configurations.
* As Aho, Sethi, Ullman p. 117 says "The DFA uses its state
* to keep track of all possible states the NFA can be in after
* reading each input symbol. That is to say, after reading
* input a1a2..an, the DFA is in a state that represents the
* subset T of the states of the NFA that are reachable from the
* NFA's start state along some path labeled a1a2..an."
* In conventional NFA->DFA conversion, therefore, the subset T
* would be a bitset representing the set of states the
* NFA could be in. We need to track the alt predicted by each
* state as well, however. More importantly, we need to maintain
* a stack of states, tracking the closure operations as they
* jump from rule to rule, emulating rule invocations (method calls).
* Recall that NFAs do not normally have a stack like a pushdown-machine
* so I have to add one to simulate the proper lookahead sequences for
* the underlying LL grammar from which the NFA was derived.
*
* I use a list of NFAConfiguration objects. An NFAConfiguration
* is both a state (ala normal conversion) and an NFAContext describing
* the chain of rules (if any) followed to arrive at that state. There
* is also the semantic context, which is the "set" of predicates found
* on the path to this configuration.
*
* A DFA state may have multiple references to a particular state,
* but with different NFAContexts (with same or different alts)
* meaning that state was reached via a different set of rule invocations.
*/
public class DFAState extends State {
public static final int INITIAL_NUM_TRANSITIONS = 4;
public static final int PREDICTED_ALT_UNSET = NFA.INVALID_ALT_NUMBER-1;
/** We are part of what DFA? Use this ref to get access to the
* context trees for an alt.
*/
public DFA dfa;
/** Track the transitions emanating from this DFA state. The List
* elements are Transition objects.
*/
protected List<Transition> transitions =
new ArrayList<Transition>(INITIAL_NUM_TRANSITIONS);
/** When doing an acyclic DFA, this is the number of lookahead symbols
* consumed to reach this state. This value may be nonzero for most
* dfa states, but it is only a valid value if the user has specified
* a max fixed lookahead.
*/
protected int k;
/** The NFA->DFA algorithm may terminate leaving some states
* without a path to an accept state, implying that upon certain
* input, the decision is not deterministic--no decision about
* predicting a unique alternative can be made. Recall that an
* accept state is one in which a unique alternative is predicted.
*/
protected int acceptStateReachable = DFA.REACHABLE_UNKNOWN;
/** Rather than recheck every NFA configuration in a DFA state (after
* resolving) in findNewDFAStatesAndAddDFATransitions just check
* this boolean. Saves a linear walk perhaps DFA state creation.
* Every little bit helps.
*/
protected boolean resolvedWithPredicates = false;
/** If a closure operation finds that we tried to invoke the same
* rule too many times (stack would grow beyond a threshold), it
* marks the state has aborted and notifies the DecisionProbe.
*/
public boolean abortedDueToRecursionOverflow = false;
/** If we detect recursion on more than one alt, decision is non-LL(*),
* but try to isolate it to only those states whose closure operations
* detect recursion. There may be other alts that are cool:
*
* a : recur '.'
* | recur ';'
* | X Y // LL(2) decision; don't abort and use k=1 plus backtracking
* | X Z
* ;
*
* 12/13/2007: Actually this has caused problems. If k=*, must terminate
* and throw out entire DFA; retry with k=1. Since recursive, do not
* attempt more closure ops as it may take forever. Exception thrown
* now and we simply report the problem. If synpreds exist, I'll retry
* with k=1.
*/
protected boolean abortedDueToMultipleRecursiveAlts = false;
/** Build up the hash code for this state as NFA configurations
* are added as it's monotonically increasing list of configurations.
*/
protected int cachedHashCode;
protected int cachedUniquelyPredicatedAlt = PREDICTED_ALT_UNSET;
public int minAltInConfigurations=Integer.MAX_VALUE;
public boolean atLeastOneConfigurationHasAPredicate = false;
/** The set of NFA configurations (state,alt,context) for this DFA state */
public OrderedHashSet<NFAConfiguration> nfaConfigurations =
new OrderedHashSet<NFAConfiguration>();
public List<NFAConfiguration> configurationsWithLabeledEdges =
new ArrayList<NFAConfiguration>();
/** Used to prevent the closure operation from looping to itself and
* hence looping forever. Sensitive to the NFA state, the alt, and
* the stack context. This just the nfa config set because we want to
* prevent closures only on states contributed by closure not reach
* operations.
*
* Two configurations identical including semantic context are
* considered the same closure computation. @see NFAToDFAConverter.closureBusy().
*/
protected Set<NFAConfiguration> closureBusy = new HashSet<NFAConfiguration>();
/** As this state is constructed (i.e., as NFA states are added), we
* can easily check for non-epsilon transitions because the only
* transition that could be a valid label is transition(0). When we
* process this node eventually, we'll have to walk all states looking
* for all possible transitions. That is of the order: size(label space)
* times size(nfa states), which can be pretty damn big. It's better
* to simply track possible labels.
*/
protected OrderedHashSet<Label> reachableLabels;
public DFAState(DFA dfa) {
this.dfa = dfa;
}
public void reset() {
//nfaConfigurations = null; // getGatedPredicatesInNFAConfigurations needs
configurationsWithLabeledEdges = null;
closureBusy = null;
reachableLabels = null;
}
public Transition transition(int i) {
return (Transition)transitions.get(i);
}
public int getNumberOfTransitions() {
return transitions.size();
}
public void addTransition(Transition t) {
transitions.add(t);
}
/** Add a transition from this state to target with label. Return
* the transition number from 0..n-1.
*/
public int addTransition(DFAState target, Label label) {
transitions.add( new Transition(label, target) );
return transitions.size()-1;
}
public Transition getTransition(int trans) {
return transitions.get(trans);
}
public void removeTransition(int trans) {
transitions.remove(trans);
}
/** Add an NFA configuration to this DFA node. Add uniquely
* an NFA state/alt/syntactic&semantic context (chain of invoking state(s)
* and semantic predicate contexts).
*
* I don't see how there could be two configurations with same
* state|alt|synCtx and different semantic contexts because the
* semantic contexts are computed along the path to a particular state
* so those two configurations would have to have the same predicate.
* Nonetheless, the addition of configurations is unique on all
* configuration info. I guess I'm saying that syntactic context
* implies semantic context as the latter is computed according to the
* former.
*
* As we add configurations to this DFA state, track the set of all possible
* transition labels so we can simply walk it later rather than doing a
* loop over all possible labels in the NFA.
*/
public void addNFAConfiguration(NFAState state, NFAConfiguration c) {
if ( nfaConfigurations.contains(c) ) {
return;
}
nfaConfigurations.add(c);
// track min alt rather than compute later
if ( c.alt < minAltInConfigurations ) {
minAltInConfigurations = c.alt;
}
if ( c.semanticContext!=SemanticContext.EMPTY_SEMANTIC_CONTEXT ) {
atLeastOneConfigurationHasAPredicate = true;
}
// update hashCode; for some reason using context.hashCode() also
// makes the GC take like 70% of the CPU and is slow!
cachedHashCode += c.state + c.alt;
// update reachableLabels
// We're adding an NFA state; check to see if it has a non-epsilon edge
if ( state.transition[0] != null ) {
Label label = state.transition[0].label;
if ( !(label.isEpsilon()||label.isSemanticPredicate()) ) {
// this NFA state has a non-epsilon edge, track for fast
// walking later when we do reach on this DFA state we're
// building.
configurationsWithLabeledEdges.add(c);
if ( state.transition[1] ==null ) {
// later we can check this to ignore o-A->o states in closure
c.singleAtomTransitionEmanating = true;
}
addReachableLabel(label);
}
}
}
public NFAConfiguration addNFAConfiguration(NFAState state,
int alt,
NFAContext context,
SemanticContext semanticContext)
{
NFAConfiguration c = new NFAConfiguration(state.stateNumber,
alt,
context,
semanticContext);
addNFAConfiguration(state, c);
return c;
}
/** Add label uniquely and disjointly; intersection with
* another set or int/char forces breaking up the set(s).
*
* Example, if reachable list of labels is [a..z, {k,9}, 0..9],
* the disjoint list will be [{a..j,l..z}, k, 9, 0..8].
*
* As we add NFA configurations to a DFA state, we might as well track
* the set of all possible transition labels to make the DFA conversion
* more efficient. W/o the reachable labels, we'd need to check the
* whole vocabulary space (could be 0..\uFFFF)! The problem is that
* labels can be sets, which may overlap with int labels or other sets.
* As we need a deterministic set of transitions from any
* state in the DFA, we must make the reachable labels set disjoint.
* This operation amounts to finding the character classes for this
* DFA state whereas with tools like flex, that need to generate a
* homogeneous DFA, must compute char classes across all states.
* We are going to generate DFAs with heterogeneous states so we
* only care that the set of transitions out of a single state are
* unique. :)
*
* The idea for adding a new set, t, is to look for overlap with the
* elements of existing list s. Upon overlap, replace
* existing set s[i] with two new disjoint sets, s[i]-t and s[i]&t.
* (if s[i]-t is nil, don't add). The remainder is t-s[i], which is
* what you want to add to the set minus what was already there. The
* remainder must then be compared against the i+1..n elements in s
* looking for another collision. Each collision results in a smaller
* and smaller remainder. Stop when you run out of s elements or
* remainder goes to nil. If remainder is non nil when you run out of
* s elements, then add remainder to the end.
*
* Single element labels are treated as sets to make the code uniform.
*/
protected void addReachableLabel(Label label) {
if ( reachableLabels==null ) {
reachableLabels = new OrderedHashSet<Label>();
}
/*
System.out.println("addReachableLabel to state "+dfa.decisionNumber+"."+stateNumber+": "+label.getSet().toString(dfa.nfa.grammar));
System.out.println("start of add to state "+dfa.decisionNumber+"."+stateNumber+": " +
"reachableLabels="+reachableLabels.toString());
*/
if ( reachableLabels.contains(label) ) { // exact label present
return;
}
IntSet t = label.getSet();
IntSet remainder = t; // remainder starts out as whole set to add
int n = reachableLabels.size(); // only look at initial elements
// walk the existing list looking for the collision
for (int i=0; i<n; i++) {
Label rl = reachableLabels.get(i);
/*
System.out.println("comparing ["+i+"]: "+label.toString(dfa.nfa.grammar)+" & "+
rl.toString(dfa.nfa.grammar)+"="+
intersection.toString(dfa.nfa.grammar));
*/
if ( !Label.intersect(label, rl) ) {
continue;
}
//System.out.println(label+" collides with "+rl);
// For any (s_i, t) with s_i&t!=nil replace with (s_i-t, s_i&t)
// (ignoring s_i-t if nil; don't put in list)
// Replace existing s_i with intersection since we
// know that will always be a non nil character class
IntSet s_i = rl.getSet();
IntSet intersection = s_i.and(t);
reachableLabels.set(i, new Label(intersection));
// Compute s_i-t to see what is in current set and not in incoming
IntSet existingMinusNewElements = s_i.subtract(t);
//System.out.println(s_i+"-"+t+"="+existingMinusNewElements);
if ( !existingMinusNewElements.isNil() ) {
// found a new character class, add to the end (doesn't affect
// outer loop duration due to n computation a priori.
Label newLabel = new Label(existingMinusNewElements);
reachableLabels.add(newLabel);
}
/*
System.out.println("after collision, " +
"reachableLabels="+reachableLabels.toString());
*/
// anything left to add to the reachableLabels?
remainder = t.subtract(s_i);
if ( remainder.isNil() ) {
break; // nothing left to add to set. done!
}
t = remainder;
}
if ( !remainder.isNil() ) {
/*
System.out.println("before add remainder to state "+dfa.decisionNumber+"."+stateNumber+": " +
"reachableLabels="+reachableLabels.toString());
System.out.println("remainder state "+dfa.decisionNumber+"."+stateNumber+": "+remainder.toString(dfa.nfa.grammar));
*/
Label newLabel = new Label(remainder);
reachableLabels.add(newLabel);
}
/*
System.out.println("#END of add to state "+dfa.decisionNumber+"."+stateNumber+": " +
"reachableLabels="+reachableLabels.toString());
*/
}
public OrderedHashSet getReachableLabels() {
return reachableLabels;
}
public void setNFAConfigurations(OrderedHashSet<NFAConfiguration> configs) {
this.nfaConfigurations = configs;
}
/** A decent hash for a DFA state is the sum of the NFA state/alt pairs.
* This is used when we add DFAState objects to the DFA.states Map and
* when we compare DFA states. Computed in addNFAConfiguration()
*/
public int hashCode() {
if ( cachedHashCode==0 ) {
// LL(1) algorithm doesn't use NFA configurations, which
// dynamically compute hashcode; must have something; use super
return super.hashCode();
}
return cachedHashCode;
}
/** Two DFAStates are equal if their NFA configuration sets are the
* same. This method is used to see if a DFA state already exists.
*
* Because the number of alternatives and number of NFA configurations are
* finite, there is a finite number of DFA states that can be processed.
* This is necessary to show that the algorithm terminates.
*
* Cannot test the DFA state numbers here because in DFA.addState we need
* to know if any other state exists that has this exact set of NFA
* configurations. The DFAState state number is irrelevant.
*/
public boolean equals(Object o) {
// compare set of NFA configurations in this set with other
DFAState other = (DFAState)o;
return this.nfaConfigurations.equals(other.nfaConfigurations);
}
/** Walk each configuration and if they are all the same alt, return
* that alt else return NFA.INVALID_ALT_NUMBER. Ignore resolved
* configurations, but don't ignore resolveWithPredicate configs
* because this state should not be an accept state. We need to add
* this to the work list and then have semantic predicate edges
* emanating from it.
*/
public int getUniquelyPredictedAlt() {
if ( cachedUniquelyPredicatedAlt!=PREDICTED_ALT_UNSET ) {
return cachedUniquelyPredicatedAlt;
}
int alt = NFA.INVALID_ALT_NUMBER;
int numConfigs = nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
// ignore anything we resolved; predicates will still result
// in transitions out of this state, so must count those
// configurations; i.e., don't ignore resolveWithPredicate configs
if ( configuration.resolved ) {
continue;
}
if ( alt==NFA.INVALID_ALT_NUMBER ) {
alt = configuration.alt; // found first nonresolved alt
}
else if ( configuration.alt!=alt ) {
return NFA.INVALID_ALT_NUMBER;
}
}
this.cachedUniquelyPredicatedAlt = alt;
return alt;
}
/** Return the uniquely mentioned alt from the NFA configurations;
* Ignore the resolved bit etc... Return INVALID_ALT_NUMBER
* if there is more than one alt mentioned.
*/
public int getUniqueAlt() {
int alt = NFA.INVALID_ALT_NUMBER;
int numConfigs = nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
if ( alt==NFA.INVALID_ALT_NUMBER ) {
alt = configuration.alt; // found first alt
}
else if ( configuration.alt!=alt ) {
return NFA.INVALID_ALT_NUMBER;
}
}
return alt;
}
/** When more than one alternative can match the same input, the first
* alternative is chosen to resolve the conflict. The other alts
* are "turned off" by setting the "resolved" flag in the NFA
* configurations. Return the set of disabled alternatives. For
*
* a : A | A | A ;
*
* this method returns {2,3} as disabled. This does not mean that
* the alternative is totally unreachable, it just means that for this
* DFA state, that alt is disabled. There may be other accept states
* for that alt.
*/
public Set getDisabledAlternatives() {
Set disabled = new LinkedHashSet();
int numConfigs = nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
if ( configuration.resolved ) {
disabled.add(Utils.integer(configuration.alt));
}
}
return disabled;
}
protected Set getNonDeterministicAlts() {
int user_k = dfa.getUserMaxLookahead();
if ( user_k>0 && user_k==k ) {
// if fixed lookahead, then more than 1 alt is a nondeterminism
// if we have hit the max lookahead
return getAltSet();
}
else if ( abortedDueToMultipleRecursiveAlts || abortedDueToRecursionOverflow ) {
// if we had to abort for non-LL(*) state assume all alts are a problem
return getAltSet();
}
else {
return getConflictingAlts();
}
}
/** Walk each NFA configuration in this DFA state looking for a conflict
* where (s|i|ctx) and (s|j|ctx) exist, indicating that state s with
* context conflicting ctx predicts alts i and j. Return an Integer set
* of the alternative numbers that conflict. Two contexts conflict if
* they are equal or one is a stack suffix of the other or one is
* the empty context.
*
* Use a hash table to record the lists of configs for each state
* as they are encountered. We need only consider states for which
* there is more than one configuration. The configurations' predicted
* alt must be different or must have different contexts to avoid a
* conflict.
*
* Don't report conflicts for DFA states that have conflicting Tokens
* rule NFA states; they will be resolved in favor of the first rule.
*/
protected Set<Integer> getConflictingAlts() {
// TODO this is called multiple times: cache result?
//System.out.println("getNondetAlts for DFA state "+stateNumber);
Set<Integer> nondeterministicAlts = new HashSet<Integer>();
// If only 1 NFA conf then no way it can be nondeterministic;
// save the overhead. There are many o-a->o NFA transitions
// and so we save a hash map and iterator creation for each
// state.
int numConfigs = nfaConfigurations.size();
if ( numConfigs <=1 ) {
return null;
}
// First get a list of configurations for each state.
// Most of the time, each state will have one associated configuration.
MultiMap<Integer, NFAConfiguration> stateToConfigListMap =
new MultiMap<Integer, NFAConfiguration>();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
Integer stateI = Utils.integer(configuration.state);
stateToConfigListMap.map(stateI, configuration);
}
// potential conflicts are states with > 1 configuration and diff alts
Set states = stateToConfigListMap.keySet();
int numPotentialConflicts = 0;
for (Iterator it = states.iterator(); it.hasNext();) {
Integer stateI = (Integer) it.next();
boolean thisStateHasPotentialProblem = false;
List configsForState = (List)stateToConfigListMap.get(stateI);
int alt=0;
int numConfigsForState = configsForState.size();
for (int i = 0; i < numConfigsForState && numConfigsForState>1 ; i++) {
NFAConfiguration c = (NFAConfiguration) configsForState.get(i);
if ( alt==0 ) {
alt = c.alt;
}
else if ( c.alt!=alt ) {
/*
System.out.println("potential conflict in state "+stateI+
" configs: "+configsForState);
*/
// 11/28/2005: don't report closures that pinch back
// together in Tokens rule. We want to silently resolve
// to the first token definition ala lex/flex by ignoring
// these conflicts.
// Also this ensures that lexers look for more and more
// characters (longest match) before resorting to predicates.
// TestSemanticPredicates.testLexerMatchesLongestThenTestPred()
// for example would terminate at state s1 and test predicate
// meaning input "ab" would test preds to decide what to
// do but it should match rule C w/o testing preds.
if ( dfa.nfa.grammar.type!=Grammar.LEXER ||
!dfa.decisionNFAStartState.enclosingRule.name.equals(Grammar.ARTIFICIAL_TOKENS_RULENAME) )
{
numPotentialConflicts++;
thisStateHasPotentialProblem = true;
}
}
}
if ( !thisStateHasPotentialProblem ) {
// remove NFA state's configurations from
// further checking; no issues with it
// (can't remove as it's concurrent modification; set to null)
stateToConfigListMap.put(stateI, null);
}
}
// a fast check for potential issues; most states have none
if ( numPotentialConflicts==0 ) {
return null;
}
// we have a potential problem, so now go through config lists again
// looking for different alts (only states with potential issues
// are left in the states set). Now we will check context.
// For example, the list of configs for NFA state 3 in some DFA
// state might be:
// [3|2|[28 18 $], 3|1|[28 $], 3|1, 3|2]
// I want to create a map from context to alts looking for overlap:
// [28 18 $] -> 2
// [28 $] -> 1
// [$] -> 1,2
// Indeed a conflict exists as same state 3, same context [$], predicts
// alts 1 and 2.
// walk each state with potential conflicting configurations
for (Iterator it = states.iterator(); it.hasNext();) {
Integer stateI = (Integer) it.next();
List configsForState = (List)stateToConfigListMap.get(stateI);
// compare each configuration pair s, t to ensure:
// s.ctx different than t.ctx if s.alt != t.alt
int numConfigsForState = 0;
if ( configsForState!=null ) {
numConfigsForState = configsForState.size();
}
for (int i = 0; i < numConfigsForState; i++) {
NFAConfiguration s = (NFAConfiguration) configsForState.get(i);
for (int j = i+1; j < numConfigsForState; j++) {
NFAConfiguration t = (NFAConfiguration)configsForState.get(j);
// conflicts means s.ctx==t.ctx or s.ctx is a stack
// suffix of t.ctx or vice versa (if alts differ).
// Also a conflict if s.ctx or t.ctx is empty
if ( s.alt != t.alt && s.context.conflictsWith(t.context) ) {
nondeterministicAlts.add(Utils.integer(s.alt));
nondeterministicAlts.add(Utils.integer(t.alt));
}
}
}
}
if ( nondeterministicAlts.size()==0 ) {
return null;
}
return nondeterministicAlts;
}
/** Get the set of all alts mentioned by all NFA configurations in this
* DFA state.
*/
public Set getAltSet() {
int numConfigs = nfaConfigurations.size();
Set alts = new HashSet();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
alts.add(Utils.integer(configuration.alt));
}
if ( alts.size()==0 ) {
return null;
}
return alts;
}
public Set getGatedSyntacticPredicatesInNFAConfigurations() {
int numConfigs = nfaConfigurations.size();
Set<SemanticContext> synpreds = new HashSet<SemanticContext>();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
SemanticContext gatedPredExpr =
configuration.semanticContext.getGatedPredicateContext();
// if this is a manual syn pred (gated and syn pred), add
if ( gatedPredExpr!=null &&
configuration.semanticContext.isSyntacticPredicate() )
{
synpreds.add(configuration.semanticContext);
}
}
if ( synpreds.size()==0 ) {
return null;
}
return synpreds;
}
/** For gated productions, we need an OR'd list of all predicates for the
* target of an edge so we can gate the edge based upon the predicates
* associated with taking that path (if any).
*
* For syntactic predicates, we only want to generate predicate
* evaluations as it transitions to an accept state; waste to
* do it earlier. So, only add gated preds derived from manually-
* specified syntactic predicates if this is an accept state.
*
* Also, since configurations w/o gated predicates are like true
* gated predicates, finding a configuration whose alt has no gated
* predicate implies we should evaluate the predicate to true. This
* means the whole edge has to be ungated. Consider:
*
* X : ('a' | {p}?=> 'a')
* | 'a' 'b'
* ;
*
* Here, you 'a' gets you from s0 to s1 but you can't test p because
* plain 'a' is ok. It's also ok for starting alt 2. Hence, you can't
* test p. Even on the edge going to accept state for alt 1 of X, you
* can't test p. You can get to the same place with and w/o the context.
* Therefore, it is never ok to test p in this situation.
*
* TODO: cache this as it's called a lot; or at least set bit if >1 present in state
*/
public SemanticContext getGatedPredicatesInNFAConfigurations() {
SemanticContext unionOfPredicatesFromAllAlts = null;
int numConfigs = nfaConfigurations.size();
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
SemanticContext gatedPredExpr =
configuration.semanticContext.getGatedPredicateContext();
if ( gatedPredExpr==null ) {
// if we ever find a configuration w/o a gated predicate
// (even if it's a nongated predicate), we cannot gate
// the indident edges.
return null;
}
else if ( acceptState || !configuration.semanticContext.isSyntacticPredicate() ) {
// at this point we have a gated predicate and, due to elseif,
// we know it's an accept and not a syn pred. In this case,
// it's safe to add the gated predicate to the union. We
// only want to add syn preds if it's an accept state. Other
// gated preds can be used with edges leading to accept states.
if ( unionOfPredicatesFromAllAlts==null ) {
unionOfPredicatesFromAllAlts = gatedPredExpr;
}
else {
unionOfPredicatesFromAllAlts =
SemanticContext.or(unionOfPredicatesFromAllAlts,gatedPredExpr);
}
}
}
if ( unionOfPredicatesFromAllAlts instanceof SemanticContext.TruePredicate ) {
return null;
}
return unionOfPredicatesFromAllAlts;
}
/** Is an accept state reachable from this state? */
public int getAcceptStateReachable() {
return acceptStateReachable;
}
public void setAcceptStateReachable(int acceptStateReachable) {
this.acceptStateReachable = acceptStateReachable;
}
public boolean isResolvedWithPredicates() {
return resolvedWithPredicates;
}
/** Print all NFA states plus what alts they predict */
public String toString() {
StringBuffer buf = new StringBuffer();
buf.append(stateNumber+":{");
for (int i = 0; i < nfaConfigurations.size(); i++) {
NFAConfiguration configuration = (NFAConfiguration) nfaConfigurations.get(i);
if ( i>0 ) {
buf.append(", ");
}
buf.append(configuration);
}
buf.append("}");
return buf.toString();
}
public int getLookaheadDepth() {
return k;
}
public void setLookaheadDepth(int k) {
this.k = k;
if ( k > dfa.max_k ) { // track max k for entire DFA
dfa.max_k = k;
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/ 0000755 0001750 0001750 00000000000 11410174110 020717 5 ustar twerner twerner antlr-3.2/tool/src/main/java/org/antlr/tool/BuildDependencyGenerator.java 0000644 0001750 0001750 00000024206 11256465230 026511 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.Tool;
import org.antlr.misc.Utils;
import org.antlr.codegen.CodeGenerator;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import java.util.List;
import java.util.ArrayList;
import java.io.*;
/** Given a grammar file, show the dependencies on .tokens etc...
* Using ST, emit a simple "make compatible" list of dependencies.
* For example, combined grammar T.g (no token import) generates:
*
* TParser.java : T.g
* T.tokens : T.g
* T__g : T.g
*
* For tree grammar TP with import of T.tokens:
*
* TP.g : T.tokens
* TP.java : TP.g
*
* If "-lib libdir" is used on command-line with -depend, then include the
* path like
*
* TP.g : libdir/T.tokens
*
* Pay attention to -o as well:
*
* outputdir/TParser.java : T.g
*
* So this output shows what the grammar depends on *and* what it generates.
*
* Operate on one grammar file at a time. If given a list of .g on the
* command-line with -depend, just emit the dependencies. The grammars
* may depend on each other, but the order doesn't matter. Build tools,
* reading in this output, will know how to organize it.
*
* This is a wee bit slow probably because the code generator has to load
* all of its template files in order to figure out the file extension
* for the generated recognizer.
*
* This code was obvious until I removed redundant "./" on front of files
* and had to escape spaces in filenames :(
*/
public class BuildDependencyGenerator {
protected String grammarFileName;
protected String tokenVocab;
protected Tool tool;
protected Grammar grammar;
protected CodeGenerator generator;
protected StringTemplateGroup templates;
public BuildDependencyGenerator(Tool tool, String grammarFileName)
throws IOException, antlr.TokenStreamException, antlr.RecognitionException {
this.tool = tool;
this.grammarFileName = grammarFileName;
grammar = tool.getRootGrammar(grammarFileName);
String language = (String) grammar.getOption("language");
generator = new CodeGenerator(tool, grammar, language);
generator.loadTemplates(language);
}
/** From T.g return a list of File objects that
* name files ANTLR will emit from T.g.
*/
public List<File> getGeneratedFileList() {
List<File> files = new ArrayList<File>();
File outputDir = tool.getOutputDirectory(grammarFileName);
if (outputDir.getName().equals(".")) {
outputDir = null;
} else if (outputDir.getName().indexOf(' ') >= 0) { // has spaces?
String escSpaces = Utils.replace(outputDir.toString(),
" ",
"\\ ");
outputDir = new File(escSpaces);
}
// add generated recognizer; e.g., TParser.java
String recognizer =
generator.getRecognizerFileName(grammar.name, grammar.type);
files.add(new File(outputDir, recognizer));
// add output vocab file; e.g., T.tokens. This is always generated to
// the base output directory, which will be just . if there is no -o option
//
files.add(new File(tool.getOutputDirectory(), generator.getVocabFileName()));
// are we generating a .h file?
StringTemplate headerExtST = null;
StringTemplate extST = generator.getTemplates().getInstanceOf("codeFileExtension");
if (generator.getTemplates().isDefined("headerFile")) {
headerExtST = generator.getTemplates().getInstanceOf("headerFileExtension");
String suffix = Grammar.grammarTypeToFileNameSuffix[grammar.type];
String fileName = grammar.name + suffix + headerExtST.toString();
files.add(new File(outputDir, fileName));
}
if (grammar.type == Grammar.COMBINED) {
// add autogenerated lexer; e.g., TLexer.java TLexer.h TLexer.tokens
// don't add T__.g (just a temp file)
String suffix = Grammar.grammarTypeToFileNameSuffix[Grammar.LEXER];
String lexer = grammar.name + suffix + extST.toString();
files.add(new File(outputDir, lexer));
// TLexer.h
if (headerExtST != null) {
String header = grammar.name + suffix + headerExtST.toString();
files.add(new File(outputDir, header));
}
// for combined, don't generate TLexer.tokens
}
// handle generated files for imported grammars
List<Grammar> imports =
grammar.composite.getDelegates(grammar.composite.getRootGrammar());
for (Grammar g : imports) {
outputDir = tool.getOutputDirectory(g.getFileName());
String fname = groomQualifiedFileName(outputDir.toString(), g.getRecognizerName() + extST.toString());
files.add(new File(fname));
}
if (files.size() == 0) {
return null;
}
return files;
}
/**
* Return a list of File objects that name files ANTLR will read
* to process T.g; This can be .tokens files if the grammar uses the tokenVocab option
* as well as any imported grammar files.
*/
public List<File> getDependenciesFileList() {
// Find all the things other than imported grammars
List<File> files = getNonImportDependenciesFileList();
// Handle imported grammars
List<Grammar> imports =
grammar.composite.getDelegates(grammar.composite.getRootGrammar());
for (Grammar g : imports) {
String libdir = tool.getLibraryDirectory();
String fileName = groomQualifiedFileName(libdir, g.fileName);
files.add(new File(fileName));
}
if (files.size() == 0) {
return null;
}
return files;
}
/**
* Return a list of File objects that name files ANTLR will read
* to process T.g; This can only be .tokens files and only
* if they use the tokenVocab option.
*
* @return List of dependencies other than imported grammars
*/
public List<File> getNonImportDependenciesFileList() {
List<File> files = new ArrayList<File>();
// handle token vocabulary loads
tokenVocab = (String) grammar.getOption("tokenVocab");
if (tokenVocab != null) {
File vocabFile = tool.getImportedVocabFile(tokenVocab);
files.add(vocabFile);
}
return files;
}
public StringTemplate getDependencies() {
loadDependencyTemplates();
StringTemplate dependenciesST = templates.getInstanceOf("dependencies");
dependenciesST.setAttribute("in", getDependenciesFileList());
dependenciesST.setAttribute("out", getGeneratedFileList());
dependenciesST.setAttribute("grammarFileName", grammar.fileName);
return dependenciesST;
}
public void loadDependencyTemplates() {
if (templates != null) {
return;
}
String fileName = "org/antlr/tool/templates/depend.stg";
ClassLoader cl = Thread.currentThread().getContextClassLoader();
InputStream is = cl.getResourceAsStream(fileName);
if (is == null) {
cl = ErrorManager.class.getClassLoader();
is = cl.getResourceAsStream(fileName);
}
if (is == null) {
ErrorManager.internalError("Can't load dependency templates: " + fileName);
return;
}
BufferedReader br = null;
try {
br = new BufferedReader(new InputStreamReader(is));
templates = new StringTemplateGroup(br,
AngleBracketTemplateLexer.class);
br.close();
} catch (IOException ioe) {
ErrorManager.internalError("error reading dependency templates file " + fileName, ioe);
} finally {
if (br != null) {
try {
br.close();
} catch (IOException ioe) {
ErrorManager.internalError("cannot close dependency templates file " + fileName, ioe);
}
}
}
}
public String getTokenVocab() {
return tokenVocab;
}
public CodeGenerator getGenerator() {
return generator;
}
public String groomQualifiedFileName(String outputDir, String fileName) {
if (outputDir.equals(".")) {
return fileName;
} else if (outputDir.indexOf(' ') >= 0) { // has spaces?
String escSpaces = Utils.replace(outputDir.toString(),
" ",
"\\ ");
return escSpaces + File.separator + fileName;
} else {
return outputDir + File.separator + fileName;
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarSpelunker.java 0000644 0001750 0001750 00000022045 11256465230 025062 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import java.io.Reader;
import java.io.IOException;
import java.io.FileReader;
import java.io.BufferedReader;
import java.io.File;
import java.util.List;
import java.util.ArrayList;
/** Load a grammar file and scan it just until we learn a few items
* of interest. Currently: name, type, imports, tokenVocab, language option.
*
* GrammarScanner (at bottom of this class) converts grammar to stuff like:
*
* grammar Java ; options { backtrack true memoize true }
* import JavaDecl JavaAnnotations JavaExpr ;
* ... : ...
*
* First ':' or '@' indicates we can stop looking for imports/options.
*
* Then we just grab interesting grammar properties.
*/
public class GrammarSpelunker {
protected String grammarFileName;
protected String token;
protected Scanner scanner;
// grammar info / properties
protected String grammarModifier;
protected String grammarName;
protected String tokenVocab;
protected String language = "Java"; // default
protected String inputDirectory;
protected List<String> importedGrammars;
public GrammarSpelunker(String inputDirectory, String grammarFileName) {
this.inputDirectory = inputDirectory;
this.grammarFileName = grammarFileName;
}
void consume() throws IOException { token = scanner.nextToken(); }
protected void match(String expecting) throws IOException {
//System.out.println("match "+expecting+"; is "+token);
if ( token.equals(expecting) ) consume();
else throw new Error("Error parsing "+grammarFileName+": '"+token+
"' not expected '"+expecting+"'");
}
public void parse() throws IOException {
Reader r = new FileReader((inputDirectory != null ? inputDirectory + File.separator : "") + grammarFileName);
BufferedReader br = new BufferedReader(r);
try {
scanner = new Scanner(br);
consume();
grammarHeader();
// scan until imports or options
while ( token!=null && !token.equals("@") && !token.equals(":") &&
!token.equals("import") && !token.equals("options") )
{
consume();
}
if ( token.equals("options") ) options();
// scan until options or first rule
while ( token!=null && !token.equals("@") && !token.equals(":") &&
!token.equals("import") )
{
consume();
}
if ( token.equals("import") ) imports();
// ignore rest of input; close up shop
}
finally {
if ( br!=null ) br.close();
}
}
protected void grammarHeader() throws IOException {
if ( token==null ) return;
if ( token.equals("tree") || token.equals("parser") || token.equals("lexer") ) {
grammarModifier=token;
consume();
}
match("grammar");
grammarName = token;
consume(); // move beyond name
}
// looks like "options { backtrack true ; tokenVocab MyTokens ; }"
protected void options() throws IOException {
match("options");
match("{");
while ( token!=null && !token.equals("}") ) {
String name = token;
consume();
String value = token;
consume();
match(";");
if ( name.equals("tokenVocab") ) tokenVocab = value;
if ( name.equals("language") ) language = value;
}
match("}");
}
// looks like "import JavaDecl JavaAnnotations JavaExpr ;"
protected void imports() throws IOException {
match("import");
importedGrammars = new ArrayList<String>();
while ( token!=null && !token.equals(";") ) {
importedGrammars.add(token);
consume();
}
match(";");
if ( importedGrammars.size()==0 ) importedGrammars = null;
}
public String getGrammarModifier() { return grammarModifier; }
public String getGrammarName() { return grammarName; }
public String getTokenVocab() { return tokenVocab; }
public String getLanguage() { return language; }
public List<String> getImportedGrammars() { return importedGrammars; }
/** Strip comments and then return stream of words and
* tokens {';', ':', '{', '}'}
*/
public static class Scanner {
public static final int EOF = -1;
Reader input;
int c;
public Scanner(Reader input) throws IOException {
this.input = input;
consume();
}
boolean isDIGIT() { return c>='0'&&c<='9'; }
boolean isID_START() { return c>='a'&&c<='z' || c>='A'&&c<='Z'; }
boolean isID_LETTER() { return isID_START() || c>='0'&&c<='9' || c=='_'; }
void consume() throws IOException { c = input.read(); }
public String nextToken() throws IOException {
while ( c!=EOF ) {
//System.out.println("check "+(char)c);
switch ( c ) {
case ';' : consume(); return ";";
case '{' : consume(); return "{";
case '}' : consume(); return "}";
case ':' : consume(); return ":";
case '@' : consume(); return "@";
case '/' : COMMENT(); break;
case '\'': return STRING();
default:
if ( isID_START() ) return ID();
else if ( isDIGIT() ) return INT();
consume(); // ignore anything else
}
}
return null;
}
/** NAME : LETTER+ ; // NAME is sequence of >=1 letter */
String ID() throws IOException {
StringBuffer buf = new StringBuffer();
while ( c!=EOF && isID_LETTER() ) { buf.append((char)c); consume(); }
return buf.toString();
}
String INT() throws IOException {
StringBuffer buf = new StringBuffer();
while ( c!=EOF && isDIGIT() ) { buf.append((char)c); consume(); }
return buf.toString();
}
String STRING() throws IOException {
StringBuffer buf = new StringBuffer();
consume();
while ( c!=EOF && c!='\'' ) {
if ( c=='\\' ) {
buf.append((char)c);
consume();
}
buf.append((char)c);
consume();
}
consume(); // scan past '
return buf.toString();
}
void COMMENT() throws IOException {
if ( c=='/' ) {
consume();
if ( c=='*' ) {
consume();
scarf:
while ( true ) {
if ( c=='*' ) {
consume();
if ( c=='/' ) { consume(); break scarf; }
}
else {
while ( c!=EOF && c!='*' ) consume();
}
}
}
else if ( c=='/' ) {
while ( c!=EOF && c!='\n' ) consume();
}
}
}
}
/** Tester; Give grammar filename as arg */
public static void main(String[] args) throws IOException {
GrammarSpelunker g = new GrammarSpelunker(".", args[0]);
g.parse();
System.out.println(g.grammarModifier+" grammar "+g.grammarName);
System.out.println("language="+g.language);
System.out.println("tokenVocab="+g.tokenVocab);
System.out.println("imports="+g.importedGrammars);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/FASerializer.java 0000644 0001750 0001750 00000017714 11256465230 024132 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.*;
import org.antlr.misc.Utils;
import java.util.*;
/** An aspect of FA (finite automata) that knows how to dump them to serialized
* strings.
*/
public class FASerializer {
/** To prevent infinite recursion when walking state machines, record
* which states we've visited. Make a new set every time you start
* walking in case you reuse this object. Multiple threads will trash
* this shared variable. Use a different FASerializer per thread.
*/
protected Set markedStates;
/** Each state we walk will get a new state number for serialization
* purposes. This is the variable that tracks state numbers.
*/
protected int stateCounter = 0;
/** Rather than add a new instance variable to NFA and DFA just for
* serializing machines, map old state numbers to new state numbers
* by a State object -> Integer new state number HashMap.
*/
protected Map stateNumberTranslator;
protected Grammar grammar;
/** This aspect is associated with a grammar; used to get token names */
public FASerializer(Grammar grammar) {
this.grammar = grammar;
}
public String serialize(State s) {
if ( s==null ) {
return "<no automaton>";
}
return serialize(s, true);
}
/** Return a string representation of a state machine. Two identical
* NFAs or DFAs will have identical serialized representations. The
* state numbers inside the state are not used; instead, a new number
* is computed and because the serialization will walk the two
* machines using the same specific algorithm, then the state numbers
* will be identical. Accept states are distinguished from regular
* states.
*/
public String serialize(State s, boolean renumber) {
markedStates = new HashSet();
stateCounter = 0;
if ( renumber ) {
stateNumberTranslator = new HashMap();
walkFANormalizingStateNumbers(s);
}
List lines = new ArrayList();
if ( s.getNumberOfTransitions()>0 ) {
walkSerializingFA(lines, s);
}
else {
// special case: s0 is an accept
String s0 = getStateString(0, s);
lines.add(s0+"\n");
}
StringBuffer buf = new StringBuffer(0);
// sort lines to normalize; makes states come out ordered
// and then ordered by edge labels then by target state number :)
Collections.sort(lines);
for (int i = 0; i < lines.size(); i++) {
String line = (String) lines.get(i);
buf.append(line);
}
return buf.toString();
}
/** In stateNumberTranslator, get a map from State to new, normalized
* state number. Used by walkSerializingFA to make sure any two
* identical state machines will serialize the same way.
*/
protected void walkFANormalizingStateNumbers(State s) {
if ( s==null ) {
ErrorManager.internalError("null state s");
return;
}
if ( stateNumberTranslator.get(s)!=null ) {
return; // already did this state
}
// assign a new state number for this node if there isn't one
stateNumberTranslator.put(s, Utils.integer(stateCounter));
stateCounter++;
// visit nodes pointed to by each transition;
for (int i = 0; i < s.getNumberOfTransitions(); i++) {
Transition edge = (Transition) s.transition(i);
walkFANormalizingStateNumbers(edge.target); // keep walkin'
// if this transition is a rule reference, the node "following" this state
// will not be found and appear to be not in graph. Must explicitly jump
// to it, but don't "draw" an edge.
if ( edge instanceof RuleClosureTransition ) {
walkFANormalizingStateNumbers(((RuleClosureTransition) edge).followState);
}
}
}
protected void walkSerializingFA(List lines, State s) {
if ( markedStates.contains(s) ) {
return; // already visited this node
}
markedStates.add(s); // mark this node as completed.
int normalizedStateNumber = s.stateNumber;
if ( stateNumberTranslator!=null ) {
Integer normalizedStateNumberI = (Integer)stateNumberTranslator.get(s);
normalizedStateNumber = normalizedStateNumberI.intValue();
}
String stateStr = getStateString(normalizedStateNumber, s);
// depth first walk each transition, printing its edge first
for (int i = 0; i < s.getNumberOfTransitions(); i++) {
Transition edge = (Transition) s.transition(i);
StringBuffer buf = new StringBuffer();
buf.append(stateStr);
if ( edge.isAction() ) {
buf.append("-{}->");
}
else if ( edge.isEpsilon() ) {
buf.append("->");
}
else if ( edge.isSemanticPredicate() ) {
buf.append("-{"+edge.label.getSemanticContext()+"}?->");
}
else {
String predsStr = "";
if ( edge.target instanceof DFAState ) {
// look for gated predicates; don't add gated to simple sempred edges
SemanticContext preds =
((DFAState)edge.target).getGatedPredicatesInNFAConfigurations();
if ( preds!=null ) {
predsStr = "&&{"+
preds.genExpr(grammar.generator,
grammar.generator.getTemplates(), null).toString()
+"}?";
}
}
buf.append("-"+edge.label.toString(grammar)+predsStr+"->");
}
int normalizedTargetStateNumber = edge.target.stateNumber;
if ( stateNumberTranslator!=null ) {
Integer normalizedTargetStateNumberI =
(Integer)stateNumberTranslator.get(edge.target);
normalizedTargetStateNumber = normalizedTargetStateNumberI.intValue();
}
buf.append(getStateString(normalizedTargetStateNumber, edge.target));
buf.append("\n");
lines.add(buf.toString());
// walk this transition
walkSerializingFA(lines, edge.target);
// if this transition is a rule reference, the node "following" this state
// will not be found and appear to be not in graph. Must explicitly jump
// to it, but don't "draw" an edge.
if ( edge instanceof RuleClosureTransition ) {
walkSerializingFA(lines, ((RuleClosureTransition) edge).followState);
}
}
}
private String getStateString(int n, State s) {
String stateStr = ".s"+n;
if ( s.isAcceptState() ) {
if ( s instanceof DFAState ) {
stateStr = ":s"+n+"=>"+((DFAState)s).getUniquelyPredictedAlt();
}
else {
stateStr = ":s"+n;
}
}
return stateStr;
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarInsufficientPredicatesMessage.java 0000644 0001750 0001750 00000006526 11256465230 031057 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.analysis.*;
import antlr.Token;
import java.util.*;
public class GrammarInsufficientPredicatesMessage extends Message {
public DecisionProbe probe;
public Map<Integer, Set<Token>> altToLocations;
public DFAState problemState;
public GrammarInsufficientPredicatesMessage(DecisionProbe probe,
DFAState problemState,
Map<Integer, Set<Token>> altToLocations)
{
super(ErrorManager.MSG_INSUFFICIENT_PREDICATES);
this.probe = probe;
this.problemState = problemState;
this.altToLocations = altToLocations;
}
public String toString() {
GrammarAST decisionASTNode = probe.dfa.getDecisionASTNode();
line = decisionASTNode.getLine();
column = decisionASTNode.getColumn();
String fileName = probe.dfa.nfa.grammar.getFileName();
if ( fileName!=null ) {
file = fileName;
}
StringTemplate st = getMessageTemplate();
// convert to string key to avoid 3.1 ST bug
Map<String, Set<Token>> altToLocationsWithStringKey = new LinkedHashMap<String, Set<Token>>();
List<Integer> alts = new ArrayList<Integer>();
alts.addAll(altToLocations.keySet());
Collections.sort(alts);
for (Integer altI : alts) {
altToLocationsWithStringKey.put(altI.toString(), altToLocations.get(altI));
/*
List<String> tokens = new ArrayList<String>();
for (Token t : altToLocations.get(altI)) {
tokens.add(t.toString());
}
Collections.sort(tokens);
System.out.println("tokens=\n"+tokens);
*/
}
st.setAttribute("altToLocations", altToLocationsWithStringKey);
List<Label> sampleInputLabels = problemState.dfa.probe.getSampleNonDeterministicInputSequence(problemState);
String input = problemState.dfa.probe.getInputSequenceDisplay(sampleInputLabels);
st.setAttribute("upon", input);
st.setAttribute("hasPredicateBlockedByAction", problemState.dfa.hasPredicateBlockedByAction);
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/AttributeScope.java 0000644 0001750 0001750 00000013167 11256465230 024545 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import antlr.Token;
import org.antlr.codegen.CodeGenerator;
import java.util.*;
/** Track the attributes within a scope. A named scoped has just its list
* of attributes. Each rule has potentially 3 scopes: return values,
* parameters, and an implicitly-named scope (i.e., a scope defined in a rule).
* Implicitly-defined scopes are named after the rule; rules and scopes then
* must live in the same name space--no collisions allowed.
*/
public class AttributeScope {
/** All token scopes (token labels) share the same fixed scope of
* of predefined attributes. I keep this out of the runtime.Token
* object to avoid a runtime space burden.
*/
public static AttributeScope tokenScope = new AttributeScope("Token",null);
static {
tokenScope.addAttribute("text", null);
tokenScope.addAttribute("type", null);
tokenScope.addAttribute("line", null);
tokenScope.addAttribute("index", null);
tokenScope.addAttribute("pos", null);
tokenScope.addAttribute("channel", null);
tokenScope.addAttribute("tree", null);
tokenScope.addAttribute("int", null);
}
/** This scope is associated with which input token (for error handling)? */
public Token derivedFromToken;
public Grammar grammar;
/** The scope name */
private String name;
/** Not a rule scope, but visible to all rules "scope symbols { ...}" */
public boolean isDynamicGlobalScope;
/** Visible to all rules, but defined in rule "scope { int i; }" */
public boolean isDynamicRuleScope;
public boolean isParameterScope;
public boolean isReturnScope;
public boolean isPredefinedRuleScope;
public boolean isPredefinedLexerRuleScope;
/** The list of Attribute objects */
protected LinkedHashMap<String,Attribute> attributes = new LinkedHashMap();
public AttributeScope(String name, Token derivedFromToken) {
this(null,name,derivedFromToken);
}
public AttributeScope(Grammar grammar, String name, Token derivedFromToken) {
this.grammar = grammar;
this.name = name;
this.derivedFromToken = derivedFromToken;
}
public String getName() {
if ( isParameterScope ) {
return name+"_parameter";
}
else if ( isReturnScope ) {
return name+"_return";
}
return name;
}
/** From a chunk of text holding the definitions of the attributes,
* pull them apart and create an Attribute for each one. Add to
* the list of attributes for this scope. Pass in the character
* that terminates a definition such as ',' or ';'. For example,
*
* scope symbols {
* int n;
* List names;
* }
*
* would pass in definitions equal to the text in between {...} and
* separator=';'. It results in two Attribute objects.
*/
public void addAttributes(String definitions, int separator) {
List<String> attrs = new ArrayList<String>();
CodeGenerator.getListOfArgumentsFromAction(definitions,0,-1,separator,attrs);
for (String a : attrs) {
Attribute attr = new Attribute(a);
if ( !isReturnScope && attr.initValue!=null ) {
ErrorManager.grammarError(ErrorManager.MSG_ARG_INIT_VALUES_ILLEGAL,
grammar,
derivedFromToken,
attr.name);
attr.initValue=null; // wipe it out
}
attributes.put(attr.name, attr);
}
}
public void addAttribute(String name, String decl) {
attributes.put(name, new Attribute(name,decl));
}
public Attribute getAttribute(String name) {
return (Attribute)attributes.get(name);
}
/** Used by templates to get all attributes */
public List<Attribute> getAttributes() {
List<Attribute> a = new ArrayList<Attribute>();
a.addAll(attributes.values());
return a;
}
/** Return the set of keys that collide from
* this and other.
*/
public Set intersection(AttributeScope other) {
if ( other==null || other.size()==0 || size()==0 ) {
return null;
}
Set inter = new HashSet();
Set thisKeys = attributes.keySet();
for (Iterator it = thisKeys.iterator(); it.hasNext();) {
String key = (String) it.next();
if ( other.attributes.get(key)!=null ) {
inter.add(key);
}
}
if ( inter.size()==0 ) {
return null;
}
return inter;
}
public int size() {
return attributes==null?0:attributes.size();
}
public String toString() {
return (isDynamicGlobalScope?"global ":"")+getName()+":"+attributes;
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/Message.java 0000644 0001750 0001750 00000010054 11256465230 023164 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
/** The ANTLR code calls methods on ErrorManager to report errors etc...
* Rather than simply pass these arguments to the ANTLRErrorListener directly,
* create an object that encapsulates everything. In this way, the error
* listener interface does not have to change when I add a new kind of
* error message. I don't want to break a GUI for example every time
* I update the error system in ANTLR itself.
*
* To get a printable error/warning message, call toString().
*/
public abstract class Message {
// msgST is the actual text of the message
public StringTemplate msgST;
// these are for supporting different output formats
public StringTemplate locationST;
public StringTemplate reportST;
public StringTemplate messageFormatST;
public int msgID;
public Object arg;
public Object arg2;
public Throwable e;
// used for location template
public String file;
public int line = -1;
public int column = -1;
public Message() {
}
public Message(int msgID) {
this(msgID, null, null);
}
public Message(int msgID, Object arg, Object arg2) {
setMessageID(msgID);
this.arg = arg;
this.arg2 = arg2;
}
public void setLine(int line) {
this.line = line;
}
public void setColumn(int column) {
this.column = column;
}
public void setMessageID(int msgID) {
this.msgID = msgID;
msgST = ErrorManager.getMessage(msgID);
}
/** Return a new template instance every time someone tries to print
* a Message.
*/
public StringTemplate getMessageTemplate() {
return msgST.getInstanceOf();
}
/** Return a new template instance for the location part of a Message.
* TODO: Is this really necessary? -Kay
*/
public StringTemplate getLocationTemplate() {
return locationST.getInstanceOf();
}
public String toString(StringTemplate messageST) {
// setup the location
locationST = ErrorManager.getLocationFormat();
reportST = ErrorManager.getReportFormat();
messageFormatST = ErrorManager.getMessageFormat();
boolean locationValid = false;
if (line != -1) {
locationST.setAttribute("line", line);
locationValid = true;
}
if (column != -1) {
locationST.setAttribute("column", column);
locationValid = true;
}
if (file != null) {
locationST.setAttribute("file", file);
locationValid = true;
}
messageFormatST.setAttribute("id", msgID);
messageFormatST.setAttribute("text", messageST);
if (locationValid) {
reportST.setAttribute("location", locationST);
}
reportST.setAttribute("message", messageFormatST);
reportST.setAttribute("type", ErrorManager.getMessageType(msgID));
return reportST.toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/DOTGenerator.java 0000644 0001750 0001750 00000034343 11256465230 024104 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.Tool;
import org.antlr.analysis.*;
import org.antlr.misc.Utils;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import java.util.*;
import org.antlr.grammar.v2.ANTLRParser;
/** The DOT (part of graphviz) generation aspect. */
public class DOTGenerator {
public static final boolean STRIP_NONREDUCED_STATES = false;
protected String arrowhead="normal";
protected String rankdir="LR";
/** Library of output templates; use <attrname> format */
public static StringTemplateGroup stlib =
new StringTemplateGroup("toollib", AngleBracketTemplateLexer.class);
/** To prevent infinite recursion when walking state machines, record
* which states we've visited. Make a new set every time you start
* walking in case you reuse this object.
*/
protected Set markedStates = null;
protected Grammar grammar;
/** This aspect is associated with a grammar */
public DOTGenerator(Grammar grammar) {
this.grammar = grammar;
}
/** Return a String containing a DOT description that, when displayed,
* will show the incoming state machine visually. All nodes reachable
* from startState will be included.
*/
public String getDOT(State startState) {
if ( startState==null ) {
return null;
}
// The output DOT graph for visualization
StringTemplate dot = null;
markedStates = new HashSet();
if ( startState instanceof DFAState ) {
dot = stlib.getInstanceOf("org/antlr/tool/templates/dot/dfa");
dot.setAttribute("startState",
Utils.integer(startState.stateNumber));
dot.setAttribute("useBox",
Boolean.valueOf(Tool.internalOption_ShowNFAConfigsInDFA));
walkCreatingDFADOT(dot, (DFAState)startState);
}
else {
dot = stlib.getInstanceOf("org/antlr/tool/templates/dot/nfa");
dot.setAttribute("startState",
Utils.integer(startState.stateNumber));
walkRuleNFACreatingDOT(dot, startState);
}
dot.setAttribute("rankdir", rankdir);
return dot.toString();
}
/** Return a String containing a DOT description that, when displayed,
* will show the incoming state machine visually. All nodes reachable
* from startState will be included.
public String getRuleNFADOT(State startState) {
// The output DOT graph for visualization
StringTemplate dot = stlib.getInstanceOf("org/antlr/tool/templates/dot/nfa");
markedStates = new HashSet();
dot.setAttribute("startState",
Utils.integer(startState.stateNumber));
walkRuleNFACreatingDOT(dot, startState);
return dot.toString();
}
*/
/** Do a depth-first walk of the state machine graph and
* fill a DOT description template. Keep filling the
* states and edges attributes.
*/
protected void walkCreatingDFADOT(StringTemplate dot,
DFAState s)
{
if ( markedStates.contains(Utils.integer(s.stateNumber)) ) {
return; // already visited this node
}
markedStates.add(Utils.integer(s.stateNumber)); // mark this node as completed.
// first add this node
StringTemplate st;
if ( s.isAcceptState() ) {
st = stlib.getInstanceOf("org/antlr/tool/templates/dot/stopstate");
}
else {
st = stlib.getInstanceOf("org/antlr/tool/templates/dot/state");
}
st.setAttribute("name", getStateLabel(s));
dot.setAttribute("states", st);
// make a DOT edge for each transition
for (int i = 0; i < s.getNumberOfTransitions(); i++) {
Transition edge = (Transition) s.transition(i);
/*
System.out.println("dfa "+s.dfa.decisionNumber+
" edge from s"+s.stateNumber+" ["+i+"] of "+s.getNumberOfTransitions());
*/
if ( STRIP_NONREDUCED_STATES ) {
if ( edge.target instanceof DFAState &&
((DFAState)edge.target).getAcceptStateReachable()!=DFA.REACHABLE_YES )
{
continue; // don't generate nodes for terminal states
}
}
st = stlib.getInstanceOf("org/antlr/tool/templates/dot/edge");
st.setAttribute("label", getEdgeLabel(edge));
st.setAttribute("src", getStateLabel(s));
st.setAttribute("target", getStateLabel(edge.target));
st.setAttribute("arrowhead", arrowhead);
dot.setAttribute("edges", st);
walkCreatingDFADOT(dot, (DFAState)edge.target); // keep walkin'
}
}
/** Do a depth-first walk of the state machine graph and
* fill a DOT description template. Keep filling the
* states and edges attributes. We know this is an NFA
* for a rule so don't traverse edges to other rules and
* don't go past rule end state.
*/
protected void walkRuleNFACreatingDOT(StringTemplate dot,
State s)
{
if ( markedStates.contains(s) ) {
return; // already visited this node
}
markedStates.add(s); // mark this node as completed.
// first add this node
StringTemplate stateST;
if ( s.isAcceptState() ) {
stateST = stlib.getInstanceOf("org/antlr/tool/templates/dot/stopstate");
}
else {
stateST = stlib.getInstanceOf("org/antlr/tool/templates/dot/state");
}
stateST.setAttribute("name", getStateLabel(s));
dot.setAttribute("states", stateST);
if ( s.isAcceptState() ) {
return; // don't go past end of rule node to the follow states
}
// special case: if decision point, then line up the alt start states
// unless it's an end of block
if ( ((NFAState)s).isDecisionState() ) {
GrammarAST n = ((NFAState)s).associatedASTNode;
if ( n!=null && n.getType()!=ANTLRParser.EOB ) {
StringTemplate rankST = stlib.getInstanceOf("org/antlr/tool/templates/dot/decision-rank");
NFAState alt = (NFAState)s;
while ( alt!=null ) {
rankST.setAttribute("states", getStateLabel(alt));
if ( alt.transition[1] !=null ) {
alt = (NFAState)alt.transition[1].target;
}
else {
alt=null;
}
}
dot.setAttribute("decisionRanks", rankST);
}
}
// make a DOT edge for each transition
StringTemplate edgeST = null;
for (int i = 0; i < s.getNumberOfTransitions(); i++) {
Transition edge = (Transition) s.transition(i);
if ( edge instanceof RuleClosureTransition ) {
RuleClosureTransition rr = ((RuleClosureTransition)edge);
// don't jump to other rules, but display edge to follow node
edgeST = stlib.getInstanceOf("org/antlr/tool/templates/dot/edge");
if ( rr.rule.grammar != grammar ) {
edgeST.setAttribute("label", "<"+rr.rule.grammar.name+"."+rr.rule.name+">");
}
else {
edgeST.setAttribute("label", "<"+rr.rule.name+">");
}
edgeST.setAttribute("src", getStateLabel(s));
edgeST.setAttribute("target", getStateLabel(rr.followState));
edgeST.setAttribute("arrowhead", arrowhead);
dot.setAttribute("edges", edgeST);
walkRuleNFACreatingDOT(dot, rr.followState);
continue;
}
if ( edge.isAction() ) {
edgeST = stlib.getInstanceOf("org/antlr/tool/templates/dot/action-edge");
}
else if ( edge.isEpsilon() ) {
edgeST = stlib.getInstanceOf("org/antlr/tool/templates/dot/epsilon-edge");
}
else {
edgeST = stlib.getInstanceOf("org/antlr/tool/templates/dot/edge");
}
edgeST.setAttribute("label", getEdgeLabel(edge));
edgeST.setAttribute("src", getStateLabel(s));
edgeST.setAttribute("target", getStateLabel(edge.target));
edgeST.setAttribute("arrowhead", arrowhead);
dot.setAttribute("edges", edgeST);
walkRuleNFACreatingDOT(dot, edge.target); // keep walkin'
}
}
/*
public void writeDOTFilesForAllRuleNFAs() throws IOException {
Collection rules = grammar.getRules();
for (Iterator itr = rules.iterator(); itr.hasNext();) {
Grammar.Rule r = (Grammar.Rule) itr.next();
String ruleName = r.name;
writeDOTFile(
ruleName,
getRuleNFADOT(grammar.getRuleStartState(ruleName)));
}
}
*/
/*
public void writeDOTFilesForAllDecisionDFAs() throws IOException {
// for debugging, create a DOT file for each decision in
// a directory named for the grammar.
File grammarDir = new File(grammar.name+"_DFAs");
grammarDir.mkdirs();
List decisionList = grammar.getDecisionNFAStartStateList();
if ( decisionList==null ) {
return;
}
int i = 1;
Iterator iter = decisionList.iterator();
while (iter.hasNext()) {
NFAState decisionState = (NFAState)iter.next();
DFA dfa = decisionState.getDecisionASTNode().getLookaheadDFA();
if ( dfa!=null ) {
String dot = getDOT( dfa.startState );
writeDOTFile(grammarDir+"/dec-"+i, dot);
}
i++;
}
}
*/
/** Fix edge strings so they print out in DOT properly;
* generate any gated predicates on edge too.
*/
protected String getEdgeLabel(Transition edge) {
String label = edge.label.toString(grammar);
label = Utils.replace(label,"\\", "\\\\");
label = Utils.replace(label,"\"", "\\\"");
label = Utils.replace(label,"\n", "\\\\n");
label = Utils.replace(label,"\r", "");
if ( label.equals(Label.EPSILON_STR) ) {
label = "e";
}
State target = edge.target;
if ( !edge.isSemanticPredicate() && target instanceof DFAState ) {
// look for gated predicates; don't add gated to simple sempred edges
SemanticContext preds =
((DFAState)target).getGatedPredicatesInNFAConfigurations();
if ( preds!=null ) {
String predsStr = "";
predsStr = "&&{"+
preds.genExpr(grammar.generator,
grammar.generator.getTemplates(), null).toString()
+"}?";
label += predsStr;
}
}
return label;
}
protected String getStateLabel(State s) {
if ( s==null ) {
return "null";
}
String stateLabel = String.valueOf(s.stateNumber);
if ( s instanceof DFAState ) {
StringBuffer buf = new StringBuffer(250);
buf.append('s');
buf.append(s.stateNumber);
if ( Tool.internalOption_ShowNFAConfigsInDFA ) {
if ( s instanceof DFAState ) {
if ( ((DFAState)s).abortedDueToRecursionOverflow ) {
buf.append("\\n");
buf.append("abortedDueToRecursionOverflow");
}
}
Set alts = ((DFAState)s).getAltSet();
if ( alts!=null ) {
buf.append("\\n");
// separate alts
List altList = new ArrayList();
altList.addAll(alts);
Collections.sort(altList);
Set configurations = ((DFAState) s).nfaConfigurations;
for (int altIndex = 0; altIndex < altList.size(); altIndex++) {
Integer altI = (Integer) altList.get(altIndex);
int alt = altI.intValue();
if ( altIndex>0 ) {
buf.append("\\n");
}
buf.append("alt");
buf.append(alt);
buf.append(':');
// get a list of configs for just this alt
// it will help us print better later
List configsInAlt = new ArrayList();
for (Iterator it = configurations.iterator(); it.hasNext();) {
NFAConfiguration c = (NFAConfiguration) it.next();
if ( c.alt!=alt ) continue;
configsInAlt.add(c);
}
int n = 0;
for (int cIndex = 0; cIndex < configsInAlt.size(); cIndex++) {
NFAConfiguration c =
(NFAConfiguration)configsInAlt.get(cIndex);
n++;
buf.append(c.toString(false));
if ( (cIndex+1)<configsInAlt.size() ) {
buf.append(", ");
}
if ( n%5==0 && (configsInAlt.size()-cIndex)>3 ) {
buf.append("\\n");
}
}
}
}
}
stateLabel = buf.toString();
}
if ( (s instanceof NFAState) && ((NFAState)s).isDecisionState() ) {
stateLabel = stateLabel+",d="+
((NFAState)s).getDecisionNumber();
if ( ((NFAState)s).endOfBlockStateNumber!=State.INVALID_STATE_NUMBER ) {
stateLabel += ",eob="+((NFAState)s).endOfBlockStateNumber;
}
}
else if ( (s instanceof NFAState) &&
((NFAState)s).endOfBlockStateNumber!=State.INVALID_STATE_NUMBER)
{
NFAState n = ((NFAState)s);
stateLabel = stateLabel+",eob="+n.endOfBlockStateNumber;
}
else if ( s instanceof DFAState && ((DFAState)s).isAcceptState() ) {
stateLabel = stateLabel+
"=>"+((DFAState)s).getUniquelyPredictedAlt();
}
return '"'+stateLabel+'"';
}
public String getArrowheadType() {
return arrowhead;
}
public void setArrowheadType(String arrowhead) {
this.arrowhead = arrowhead;
}
public String getRankdir() {
return rankdir;
}
public void setRankdir(String rankdir) {
this.rankdir = rankdir;
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/Attribute.java 0000644 0001750 0001750 00000010261 11256465230 023543 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
/** Track the names of attributes define in arg lists, return values,
* scope blocks etc...
*/
public class Attribute {
/** The entire declaration such as "String foo;" */
public String decl;
/** The type; might be empty such as for Python which has no static typing */
public String type;
/** The name of the attribute "foo" */
public String name;
/** The optional attribute intialization expression */
public String initValue;
public Attribute(String decl) {
extractAttribute(decl);
}
public Attribute(String name, String decl) {
this.name = name;
this.decl = decl;
}
/** For decls like "String foo" or "char *foo32[3]" compute the ID
* and type declarations. Also handle "int x=3" and 'T t = new T("foo")'
* but if the separator is ',' you cannot use ',' in the initvalue.
* AttributeScope.addAttributes takes care of the separation so we are
* free here to use from '=' to end of string as the expression.
*
* Set name, type, initvalue, and full decl instance vars.
*/
protected void extractAttribute(String decl) {
if ( decl==null ) {
return;
}
boolean inID = false;
int start = -1;
int rightEdgeOfDeclarator = decl.length()-1;
int equalsIndex = decl.indexOf('=');
if ( equalsIndex>0 ) {
// everything after the '=' is the init value
this.initValue = decl.substring(equalsIndex+1,decl.length());
rightEdgeOfDeclarator = equalsIndex-1;
}
// walk backwards looking for start of an ID
for (int i=rightEdgeOfDeclarator; i>=0; i--) {
// if we haven't found the end yet, keep going
if ( !inID && Character.isLetterOrDigit(decl.charAt(i)) ) {
inID = true;
}
else if ( inID &&
!(Character.isLetterOrDigit(decl.charAt(i))||
decl.charAt(i)=='_') ) {
start = i+1;
break;
}
}
if ( start<0 && inID ) {
start = 0;
}
if ( start<0 ) {
ErrorManager.error(ErrorManager.MSG_CANNOT_FIND_ATTRIBUTE_NAME_IN_DECL,decl);
}
// walk forwards looking for end of an ID
int stop=-1;
for (int i=start; i<=rightEdgeOfDeclarator; i++) {
// if we haven't found the end yet, keep going
if ( !(Character.isLetterOrDigit(decl.charAt(i))||
decl.charAt(i)=='_') )
{
stop = i;
break;
}
if ( i==rightEdgeOfDeclarator ) {
stop = i+1;
}
}
// the name is the last ID
this.name = decl.substring(start,stop);
// the type is the decl minus the ID (could be empty)
this.type = decl.substring(0,start);
if ( stop<=rightEdgeOfDeclarator ) {
this.type += decl.substring(stop,rightEdgeOfDeclarator+1);
}
this.type = type.trim();
if ( this.type.length()==0 ) {
this.type = null;
}
this.decl = decl;
}
public String toString() {
if ( initValue!=null ) {
return type+" "+name+"="+initValue;
}
return type+" "+name;
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarSerializerFoo.java 0000644 0001750 0001750 00000012601 11256465230 025664 0 ustar twerner twerner package org.antlr.tool;
import org.antlr.runtime.SerializedGrammar;
import java.io.*;
import java.util.Stack;
/** Serialize a grammar into a highly compressed form with
* only the info needed to recognize sentences.
* FORMAT:
*
* file ::= $ANTLR<version:byte><grammartype:byte><name:string>;<numRules:short><rules>
* rule ::= R<rulename:string>;B<nalts:short><alts>.
* alt ::= A<elems>;
* elem ::= t<tokentype:short> | r<ruleIndex:short> | -<char:uchar><char:uchar> | ~<tokentype> | w
*/
public class GrammarSerializerFoo {
protected DataOutputStream out;
protected String filename;
protected Grammar g;
protected Stack streams = new Stack();
protected ByteArrayOutputStream altBuf;
protected int numElementsInAlt = 0;
public GrammarSerializerFoo(Grammar g) {
this.g = g;
}
public void open(String filename) throws IOException {
this.filename = filename;
FileOutputStream fos = new FileOutputStream(filename);
BufferedOutputStream bos = new BufferedOutputStream(fos);
out = new DataOutputStream(bos);
writeString(out, SerializedGrammar.COOKIE);
out.writeByte(SerializedGrammar.FORMAT_VERSION);
}
public void close() throws IOException {
if ( out!=null ) out.close();
out = null;
}
// WRITE
public void grammar(int grammarTokenType, String name) {
try {
/*
switch ( grammarTokenType ) {
case ANTLRParser.LEXER_GRAMMAR : out.writeByte('l'); break;
case ANTLRParser.PARSER_GRAMMAR : out.writeByte('p'); break;
case ANTLRParser.TREE_GRAMMAR: out.writeByte('t'); break;
case ANTLRParser.COMBINED_GRAMMAR : out.writeByte('c'); break;
}
writeString(out, name);
*/
out.writeShort(g.getRules().size());
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void rule(String name) {
try {
out.writeByte('R');
writeString(out, name);
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void endRule() {
try {
out.writeByte('.');
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void block(int nalts) {
try {
out.writeByte('B');
out.writeShort(nalts);
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void alt(GrammarAST alt) {
numElementsInAlt = 0;
try {
out.writeByte('A');
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
//streams.push(out);
//altBuf = new ByteArrayOutputStream();
//out = new DataOutputStream(altBuf);
}
public void endAlt() {
try {
//out.flush();
//out = (DataOutputStream)streams.pop(); // restore previous stream
out.writeByte(';');
//out.writeShort(numElementsInAlt);
//out.write(altBuf.toByteArray());
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void ruleRef(GrammarAST t) {
numElementsInAlt++;
try {
out.writeByte('r');
out.writeShort(g.getRuleIndex(t.getText()));
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void token(GrammarAST t) {
numElementsInAlt++;
try {
out.writeByte('t');
int ttype = g.getTokenType(t.getText());
out.writeShort(ttype);
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void charLiteral(GrammarAST t) {
numElementsInAlt++;
try {
if ( g.type!=Grammar.LEXER ) {
out.writeByte('t');
int ttype = g.getTokenType(t.getText());
out.writeShort(ttype);
}
// else lexer???
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void wildcard(GrammarAST t) {
numElementsInAlt++;
try {
out.writeByte('w');
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void range() { // must be char range
numElementsInAlt++;
try {
out.writeByte('-');
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void not() {
try {
out.writeByte('~');
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, filename);
}
}
public void writeString(DataOutputStream out, String s) throws IOException {
out.writeBytes(s);
out.writeByte(';');
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarDanglingStateMessage.java 0000644 0001750 0001750 00000005316 11256465230 027145 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.DFAState;
import org.antlr.analysis.DecisionProbe;
import org.antlr.stringtemplate.StringTemplate;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
/** Reports a potential parsing issue with a decision; the decision is
* nondeterministic in some way.
*/
public class GrammarDanglingStateMessage extends Message {
public DecisionProbe probe;
public DFAState problemState;
public GrammarDanglingStateMessage(DecisionProbe probe,
DFAState problemState)
{
super(ErrorManager.MSG_DANGLING_STATE);
this.probe = probe;
this.problemState = problemState;
}
public String toString() {
GrammarAST decisionASTNode = probe.dfa.getDecisionASTNode();
line = decisionASTNode.getLine();
column = decisionASTNode.getColumn();
String fileName = probe.dfa.nfa.grammar.getFileName();
if ( fileName!=null ) {
file = fileName;
}
List labels = probe.getSampleNonDeterministicInputSequence(problemState);
String input = probe.getInputSequenceDisplay(labels);
StringTemplate st = getMessageTemplate();
List alts = new ArrayList();
alts.addAll(problemState.getAltSet());
Collections.sort(alts);
st.setAttribute("danglingAlts", alts);
st.setAttribute("input", input);
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/RuleLabelScope.java 0000644 0001750 0001750 00000007174 11256465230 024452 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import antlr.Token;
public class RuleLabelScope extends AttributeScope {
/** Rules have a predefined set of attributes as well as
* the return values. 'text' needs to be computed though so.
*/
public static AttributeScope predefinedRulePropertiesScope =
new AttributeScope("RulePredefined",null) {{
addAttribute("text", null);
addAttribute("start", null);
addAttribute("stop", null);
addAttribute("tree", null);
addAttribute("st", null);
isPredefinedRuleScope = true;
}};
public static AttributeScope predefinedTreeRulePropertiesScope =
new AttributeScope("RulePredefined",null) {{
addAttribute("text", null);
addAttribute("start", null); // note: no stop; not meaningful
addAttribute("tree", null);
addAttribute("st", null);
isPredefinedRuleScope = true;
}};
public static AttributeScope predefinedLexerRulePropertiesScope =
new AttributeScope("LexerRulePredefined",null) {{
addAttribute("text", null);
addAttribute("type", null);
addAttribute("line", null);
addAttribute("index", null);
addAttribute("pos", null);
addAttribute("channel", null);
addAttribute("start", null);
addAttribute("stop", null);
addAttribute("int", null);
isPredefinedLexerRuleScope = true;
}};
public static AttributeScope[] grammarTypeToRulePropertiesScope =
{
null,
predefinedLexerRulePropertiesScope, // LEXER
predefinedRulePropertiesScope, // PARSER
predefinedTreeRulePropertiesScope, // TREE_PARSER
predefinedRulePropertiesScope, // COMBINED
};
public Rule referencedRule;
public RuleLabelScope(Rule referencedRule, Token actionToken) {
super("ref_"+referencedRule.name,actionToken);
this.referencedRule = referencedRule;
}
/** If you label a rule reference, you can access that rule's
* return values as well as any predefined attributes.
*/
public Attribute getAttribute(String name) {
AttributeScope rulePropertiesScope =
RuleLabelScope.grammarTypeToRulePropertiesScope[grammar.type];
if ( rulePropertiesScope.getAttribute(name)!=null ) {
return rulePropertiesScope.getAttribute(name);
}
if ( referencedRule.returnScope!=null ) {
return referencedRule.returnScope.getAttribute(name);
}
return null;
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/CompositeGrammarTree.java 0000644 0001750 0001750 00000011655 11256465230 025701 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import java.util.ArrayList;
import java.util.List;
/** A tree of grammars */
public class CompositeGrammarTree {
protected List<CompositeGrammarTree> children;
public Grammar grammar;
/** Who is the parent node of this node; if null, implies node is root */
public CompositeGrammarTree parent;
public CompositeGrammarTree(Grammar g) {
grammar = g;
}
public void addChild(CompositeGrammarTree t) {
//System.out.println("add "+t.toStringTree()+" as child to "+this.toStringTree());
if ( t==null ) {
return; // do nothing upon addChild(null)
}
if ( children==null ) {
children = new ArrayList<CompositeGrammarTree>();
}
children.add(t);
t.parent = this;
}
/** Find a rule by looking in current grammar then down towards the
* delegate grammars.
*/
public Rule getRule(String ruleName) {
Rule r = grammar.getLocallyDefinedRule(ruleName);
for (int i = 0; r==null && children!=null && i < children.size(); i++) {
CompositeGrammarTree child = children.get(i);
r = child.getRule(ruleName);
}
return r;
}
/** Find an option by looking up towards the root grammar rather than down */
public Object getOption(String key) {
Object o = grammar.getLocallyDefinedOption(key);
if ( o!=null ) {
return o;
}
if ( parent!=null ) {
return parent.getOption(key);
}
return null; // not found
}
public CompositeGrammarTree findNode(Grammar g) {
if ( g==null ) {
return null;
}
if ( this.grammar == g ) {
return this;
}
CompositeGrammarTree n = null;
for (int i = 0; n==null && children!=null && i < children.size(); i++) {
CompositeGrammarTree child = children.get(i);
n = child.findNode(g);
}
return n;
}
public CompositeGrammarTree findNode(String grammarName) {
if ( grammarName==null ) {
return null;
}
if ( grammarName.equals(this.grammar.name) ) {
return this;
}
CompositeGrammarTree n = null;
for (int i = 0; n==null && children!=null && i < children.size(); i++) {
CompositeGrammarTree child = children.get(i);
n = child.findNode(grammarName);
}
return n;
}
/** Return a postorder list of grammars; root is last in list */
public List<Grammar> getPostOrderedGrammarList() {
List<Grammar> grammars = new ArrayList<Grammar>();
_getPostOrderedGrammarList(grammars);
return grammars;
}
/** work for getPostOrderedGrammarList */
protected void _getPostOrderedGrammarList(List<Grammar> grammars) {
for (int i = 0; children!=null && i < children.size(); i++) {
CompositeGrammarTree child = children.get(i);
child._getPostOrderedGrammarList(grammars);
}
grammars.add(this.grammar);
}
/** Return a postorder list of grammars; root is last in list */
public List<Grammar> getPreOrderedGrammarList() {
List<Grammar> grammars = new ArrayList<Grammar>();
_getPreOrderedGrammarList(grammars);
return grammars;
}
protected void _getPreOrderedGrammarList(List<Grammar> grammars) {
grammars.add(this.grammar);
for (int i = 0; children!=null && i < children.size(); i++) {
CompositeGrammarTree child = children.get(i);
child._getPostOrderedGrammarList(grammars);
}
}
public void trimLexerImportsIntoCombined() {
CompositeGrammarTree p = this;
if ( p.grammar.type == Grammar.LEXER && p.parent!=null &&
p.parent.grammar.type == Grammar.COMBINED )
{
//System.out.println("wacking "+p.grammar.name+" from "+p.parent.grammar.name);
p.parent.children.remove(this);
}
for (int i = 0; children!=null && i < children.size(); i++) {
CompositeGrammarTree child = children.get(i);
child.trimLexerImportsIntoCombined();
}
}
} antlr-3.2/tool/src/main/java/org/antlr/tool/Grammar.java 0000644 0001750 0001750 00000304455 11256465230 023201 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import antlr.RecognitionException;
import antlr.Token;
import antlr.TokenStreamException;
import antlr.TokenStreamRewriteEngine;
import antlr.TokenWithIndex;
import org.antlr.grammar.v2.*;
import org.antlr.grammar.v3.*;
import org.antlr.misc.*;
import org.antlr.misc.Utils;
import antlr.collections.AST;
import org.antlr.Tool;
import org.antlr.analysis.*;
import org.antlr.codegen.CodeGenerator;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import java.io.*;
import java.util.*;
/** Represents a grammar in memory. */
public class Grammar {
public static final String SYNPRED_RULE_PREFIX = "synpred";
public static final String GRAMMAR_FILE_EXTENSION = ".g";
/** used for generating lexer temp files */
public static final String LEXER_GRAMMAR_FILE_EXTENSION = ".g";
public static final int INITIAL_DECISION_LIST_SIZE = 300;
public static final int INVALID_RULE_INDEX = -1;
// the various kinds of labels. t=type, id=ID, types+=type ids+=ID
public static final int RULE_LABEL = 1;
public static final int TOKEN_LABEL = 2;
public static final int RULE_LIST_LABEL = 3;
public static final int TOKEN_LIST_LABEL = 4;
public static final int CHAR_LABEL = 5; // used in lexer for x='a'
public static final int WILDCARD_TREE_LABEL = 6; // Used in tree grammar x=.
public static final int WILDCARD_TREE_LIST_LABEL = 7; // Used in tree grammar x+=.
public static String[] LabelTypeToString =
{"<invalid>", "rule", "token", "rule-list", "token-list", "wildcard-tree", "wildcard-tree-list"};
public static final String ARTIFICIAL_TOKENS_RULENAME = "Tokens";
public static final String FRAGMENT_RULE_MODIFIER = "fragment";
public static final String SYNPREDGATE_ACTION_NAME = "synpredgate";
/** When converting ANTLR char and string literals, here is the
* value set of escape chars.
*/
public static int ANTLRLiteralEscapedCharValue[] = new int[255];
/** Given a char, we need to be able to show as an ANTLR literal.
*/
public static String ANTLRLiteralCharValueEscape[] = new String[255];
static {
ANTLRLiteralEscapedCharValue['n'] = '\n';
ANTLRLiteralEscapedCharValue['r'] = '\r';
ANTLRLiteralEscapedCharValue['t'] = '\t';
ANTLRLiteralEscapedCharValue['b'] = '\b';
ANTLRLiteralEscapedCharValue['f'] = '\f';
ANTLRLiteralEscapedCharValue['\\'] = '\\';
ANTLRLiteralEscapedCharValue['\''] = '\'';
ANTLRLiteralEscapedCharValue['"'] = '"';
ANTLRLiteralCharValueEscape['\n'] = "\\n";
ANTLRLiteralCharValueEscape['\r'] = "\\r";
ANTLRLiteralCharValueEscape['\t'] = "\\t";
ANTLRLiteralCharValueEscape['\b'] = "\\b";
ANTLRLiteralCharValueEscape['\f'] = "\\f";
ANTLRLiteralCharValueEscape['\\'] = "\\\\";
ANTLRLiteralCharValueEscape['\''] = "\\'";
}
public static final int LEXER = 1;
public static final int PARSER = 2;
public static final int TREE_PARSER = 3;
public static final int COMBINED = 4;
public static final String[] grammarTypeToString = new String[] {
"<invalid>",
"lexer",
"parser",
"tree",
"combined"
};
public static final String[] grammarTypeToFileNameSuffix = new String[] {
"<invalid>",
"Lexer",
"Parser",
"", // no suffix for tree grammars
"Parser" // if combined grammar, gen Parser and Lexer will be done later
};
/** Set of valid imports. E.g., can only import a tree parser into
* another tree parser. Maps delegate to set of delegator grammar types.
* validDelegations.get(LEXER) gives list of the kinds of delegators
* that can import lexers.
*/
public static MultiMap<Integer,Integer> validDelegations =
new MultiMap<Integer,Integer>() {
{
map(LEXER, LEXER);
map(LEXER, PARSER);
map(LEXER, COMBINED);
map(PARSER, PARSER);
map(PARSER, COMBINED);
map(TREE_PARSER, TREE_PARSER);
// TODO: allow COMBINED
// map(COMBINED, COMBINED);
}
};
/** This is the buffer of *all* tokens found in the grammar file
* including whitespace tokens etc... I use this to extract
* lexer rules from combined grammars.
*/
protected TokenStreamRewriteEngine tokenBuffer;
public static final String IGNORE_STRING_IN_GRAMMAR_FILE_NAME = "__";
public static final String AUTO_GENERATED_TOKEN_NAME_PREFIX = "T__";
public static class Decision {
public int decision;
public NFAState startState;
public GrammarAST blockAST;
public DFA dfa;
}
public class LabelElementPair {
public antlr.Token label;
public GrammarAST elementRef;
public String referencedRuleName;
/** Has an action referenced the label? Set by ActionAnalysis.g
* Currently only set for rule labels.
*/
public boolean actionReferencesLabel;
public int type; // in {RULE_LABEL,TOKEN_LABEL,RULE_LIST_LABEL,TOKEN_LIST_LABEL}
public LabelElementPair(antlr.Token label, GrammarAST elementRef) {
this.label = label;
this.elementRef = elementRef;
this.referencedRuleName = elementRef.getText();
}
public Rule getReferencedRule() {
return getRule(referencedRuleName);
}
public String toString() {
return elementRef.toString();
}
}
/** What name did the user provide for this grammar? */
public String name;
/** What type of grammar is this: lexer, parser, tree walker */
public int type;
/** A list of options specified at the grammar level such as language=Java.
* The value can be an AST for complicated values such as character sets.
* There may be code generator specific options in here. I do no
* interpretation of the key/value pairs...they are simply available for
* who wants them.
*/
protected Map options;
public static final Set legalLexerOptions =
new HashSet() {
{
add("language"); add("tokenVocab");
add("TokenLabelType");
add("superClass");
add("filter");
add("k");
add("backtrack");
add("memoize");
}
};
public static final Set legalParserOptions =
new HashSet() {
{
add("language"); add("tokenVocab");
add("output"); add("rewrite"); add("ASTLabelType");
add("TokenLabelType");
add("superClass");
add("k");
add("backtrack");
add("memoize");
}
};
public static final Set legalTreeParserOptions =
new HashSet() {
{
add("language"); add("tokenVocab");
add("output"); add("rewrite"); add("ASTLabelType");
add("TokenLabelType");
add("superClass");
add("k");
add("backtrack");
add("memoize");
add("filter");
}
};
public static final Set doNotCopyOptionsToLexer =
new HashSet() {
{
add("output"); add("ASTLabelType"); add("superClass");
add("k"); add("backtrack"); add("memoize"); add("rewrite");
}
};
public static final Map defaultOptions =
new HashMap() {
{
put("language","Java");
}
};
public static final Set legalBlockOptions =
new HashSet() {{add("k"); add("greedy"); add("backtrack"); add("memoize");}};
/** What are the default options for a subrule? */
public static final Map defaultBlockOptions =
new HashMap() {{put("greedy","true");}};
public static final Map defaultLexerBlockOptions =
new HashMap() {{put("greedy","true");}};
// Token options are here to avoid contaminating Token object in runtime
/** Legal options for terminal refs like ID<node=MyVarNode> */
public static final Set legalTokenOptions =
new HashSet() {
{
add(defaultTokenOption);
add("associativity");
}
};
public static final String defaultTokenOption = "node";
/** Is there a global fixed lookahead set for this grammar?
* If 0, nothing specified. -1 implies we have not looked at
* the options table yet to set k.
*/
protected int global_k = -1;
/** Map a scope to a map of name:action pairs.
* Map<String, Map<String,GrammarAST>>
* The code generator will use this to fill holes in the output files.
* I track the AST node for the action in case I need the line number
* for errors.
*/
protected Map actions = new HashMap();
/** The NFA that represents the grammar with edges labelled with tokens
* or epsilon. It is more suitable to analysis than an AST representation.
*/
public NFA nfa;
protected NFAFactory factory;
/** If this grammar is part of a larger composite grammar via delegate
* statement, then this points at the composite. The composite holds
* a global list of rules, token types, decision numbers, etc...
*/
public CompositeGrammar composite;
/** A pointer back into grammar tree. Needed so we can add delegates. */
public CompositeGrammarTree compositeTreeNode;
/** If this is a delegate of another grammar, this is the label used
* as an instance var by that grammar to point at this grammar. null
* if no label was specified in the delegate statement.
*/
public String label;
/** TODO: hook this to the charVocabulary option */
protected IntSet charVocabulary = null;
/** For ANTLRWorks, we want to be able to map a line:col to a specific
* decision DFA so it can display DFA.
*/
Map lineColumnToLookaheadDFAMap = new HashMap();
public Tool tool;
/** The unique set of all rule references in any rule; set of tree node
* objects so two refs to same rule can exist but at different line/position.
*/
protected Set<GrammarAST> ruleRefs = new HashSet<GrammarAST>();
protected Set<GrammarAST> scopedRuleRefs = new HashSet();
/** The unique set of all token ID references in any rule */
protected Set<antlr.Token> tokenIDRefs = new HashSet<antlr.Token>();
/** Be able to assign a number to every decision in grammar;
* decisions in 1..n
*/
protected int decisionCount = 0;
/** A list of all rules that are in any left-recursive cycle. There
* could be multiple cycles, but this is a flat list of all problematic
* rules.
*/
protected Set<Rule> leftRecursiveRules;
/** An external tool requests that DFA analysis abort prematurely. Stops
* at DFA granularity, which are limited to a DFA size and time computation
* as failsafe.
*/
protected boolean externalAnalysisAbort;
/** When we read in a grammar, we track the list of syntactic predicates
* and build faux rules for them later. See my blog entry Dec 2, 2005:
* http://www.antlr.org/blog/antlr3/lookahead.tml
* This maps the name (we make up) for a pred to the AST grammar fragment.
*/
protected LinkedHashMap nameToSynpredASTMap;
/** At least one rule has memoize=true */
public boolean atLeastOneRuleMemoizes;
/** At least one backtrack=true in rule or decision or grammar. */
public boolean atLeastOneBacktrackOption;
/** Was this created from a COMBINED grammar? */
public boolean implicitLexer;
/** Map a rule to it's Rule object */
protected LinkedHashMap<String,Rule> nameToRuleMap = new LinkedHashMap<String,Rule>();
/** If this rule is a delegate, some rules might be overridden; don't
* want to gen code for them.
*/
public Set<String> overriddenRules = new HashSet<String>();
/** The list of all rules referenced in this grammar, not defined here,
* and defined in a delegate grammar. Not all of these will be generated
* in the recognizer for this file; only those that are affected by rule
* definitions in this grammar. I am not sure the Java target will need
* this but I'm leaving in case other targets need it.
* @see NameSpaceChecker.lookForReferencesToUndefinedSymbols()
*/
protected Set<Rule> delegatedRuleReferences = new HashSet();
/** The ANTLRParser tracks lexer rules when reading combined grammars
* so we can build the Tokens rule.
*/
public List<String> lexerRuleNamesInCombined = new ArrayList<String>();
/** Track the scopes defined outside of rules and the scopes associated
* with all rules (even if empty).
*/
protected Map scopes = new HashMap();
/** An AST that records entire input grammar with all rules. A simple
* grammar with one rule, "grammar t; a : A | B ;", looks like:
* ( grammar t ( rule a ( BLOCK ( ALT A ) ( ALT B ) ) <end-of-rule> ) )
*/
protected GrammarAST grammarTree = null;
/** Each subrule/rule is a decision point and we must track them so we
* can go back later and build DFA predictors for them. This includes
* all the rules, subrules, optional blocks, ()+, ()* etc...
*/
protected Vector<Decision> indexToDecision =
new Vector<Decision>(INITIAL_DECISION_LIST_SIZE);
/** If non-null, this is the code generator we will use to generate
* recognizers in the target language.
*/
protected CodeGenerator generator;
public NameSpaceChecker nameSpaceChecker = new NameSpaceChecker(this);
public LL1Analyzer ll1Analyzer = new LL1Analyzer(this);
/** For merged lexer/parsers, we must construct a separate lexer spec.
* This is the template for lexer; put the literals first then the
* regular rules. We don't need to specify a token vocab import as
* I make the new grammar import from the old all in memory; don't want
* to force it to read from the disk. Lexer grammar will have same
* name as original grammar but will be in different filename. Foo.g
* with combined grammar will have FooParser.java generated and
* Foo__.g with again Foo inside. It will however generate FooLexer.java
* as it's a lexer grammar. A bit odd, but autogenerated. Can tweak
* later if we want.
*/
protected StringTemplate lexerGrammarST =
new StringTemplate(
"lexer grammar <name>;\n" +
"<if(options)>" +
"options {\n" +
" <options:{<it.name>=<it.value>;<\\n>}>\n" +
"}<\\n>\n" +
"<endif>\n" +
"<if(imports)>import <imports; separator=\", \">;<endif>\n" +
"<actionNames,actions:{n,a|@<n> {<a>}\n}>\n" +
"<literals:{<it.ruleName> : <it.literal> ;\n}>\n" +
"<rules>",
AngleBracketTemplateLexer.class
);
/** What file name holds this grammar? */
protected String fileName;
/** How long in ms did it take to build DFAs for this grammar?
* If this grammar is a combined grammar, it only records time for
* the parser grammar component. This only records the time to
* do the LL(*) work; NFA->DFA conversion.
*/
public long DFACreationWallClockTimeInMS;
public int numberOfSemanticPredicates = 0;
public int numberOfManualLookaheadOptions = 0;
public Set<Integer> setOfNondeterministicDecisionNumbers = new HashSet<Integer>();
public Set<Integer> setOfNondeterministicDecisionNumbersResolvedWithPredicates =
new HashSet<Integer>();
public Set setOfDFAWhoseAnalysisTimedOut = new HashSet();
/** Track decisions with syn preds specified for reporting.
* This is the a set of BLOCK type AST nodes.
*/
public Set<GrammarAST> blocksWithSynPreds = new HashSet();
/** Track decisions that actually use the syn preds in the DFA.
* Computed during NFA to DFA conversion.
*/
public Set<DFA> decisionsWhoseDFAsUsesSynPreds = new HashSet<DFA>();
/** Track names of preds so we can avoid generating preds that aren't used
* Computed during NFA to DFA conversion. Just walk accept states
* and look for synpreds because that is the only state target whose
* incident edges can have synpreds. Same is try for
* decisionsWhoseDFAsUsesSynPreds.
*/
public Set<String> synPredNamesUsedInDFA = new HashSet();
/** Track decisions with syn preds specified for reporting.
* This is the a set of BLOCK type AST nodes.
*/
public Set<GrammarAST> blocksWithSemPreds = new HashSet();
/** Track decisions that actually use the syn preds in the DFA. */
public Set<DFA> decisionsWhoseDFAsUsesSemPreds = new HashSet();
protected boolean allDecisionDFACreated = false;
/** We need a way to detect when a lexer grammar is autogenerated from
* another grammar or we are just sending in a string representing a
* grammar. We don't want to generate a .tokens file, for example,
* in such cases.
*/
protected boolean builtFromString = false;
/** Factored out the sanity checking code; delegate to it. */
GrammarSanity sanity = new GrammarSanity(this);
/** Create a grammar from file name. */
public Grammar(Tool tool, String fileName, CompositeGrammar composite) {
this.composite = composite;
setTool(tool);
setFileName(fileName);
// ensure we have the composite set to something
if ( composite.delegateGrammarTreeRoot==null ) {
composite.setDelegationRoot(this);
}
}
/** Useful for when you are sure that you are not part of a composite
* already. Used in Interp/RandomPhrase and testing.
*/
public Grammar() {
builtFromString = true;
composite = new CompositeGrammar(this);
}
/** Used for testing; only useful on noncomposite grammars.*/
public Grammar(String grammarString)
throws antlr.RecognitionException, antlr.TokenStreamException
{
this(null, grammarString);
}
/** Used for testing and Interp/RandomPhrase. Only useful on
* noncomposite grammars.
*/
public Grammar(Tool tool, String grammarString)
throws antlr.RecognitionException
{
this();
setTool(tool);
setFileName("<string>");
StringReader r = new StringReader(grammarString);
parseAndBuildAST(r);
composite.assignTokenTypes();
defineGrammarSymbols();
checkNameSpaceAndActions();
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public String getFileName() {
return fileName;
}
public void setName(String name) {
if ( name==null ) {
return;
}
// don't error check autogenerated files (those with '__' in them)
String saneFile = fileName.replace('\\', '/');
int lastSlash = saneFile.lastIndexOf('/');
String onlyFileName = saneFile.substring(lastSlash+1, fileName.length());
if ( !builtFromString ) {
int lastDot = onlyFileName.lastIndexOf('.');
String onlyFileNameNoSuffix = null;
if ( lastDot < 0 ) {
ErrorManager.error(ErrorManager.MSG_FILENAME_EXTENSION_ERROR, fileName);
onlyFileNameNoSuffix = onlyFileName+GRAMMAR_FILE_EXTENSION;
}
else {
onlyFileNameNoSuffix = onlyFileName.substring(0,lastDot);
}
if ( !name.equals(onlyFileNameNoSuffix) ) {
ErrorManager.error(ErrorManager.MSG_FILE_AND_GRAMMAR_NAME_DIFFER,
name,
fileName);
}
}
this.name = name;
}
public void setGrammarContent(String grammarString) throws RecognitionException {
StringReader r = new StringReader(grammarString);
parseAndBuildAST(r);
composite.assignTokenTypes();
composite.defineGrammarSymbols();
}
public void parseAndBuildAST()
throws IOException
{
FileReader fr = null;
BufferedReader br = null;
try {
fr = new FileReader(fileName);
br = new BufferedReader(fr);
parseAndBuildAST(br);
br.close();
br = null;
}
finally {
if ( br!=null ) {
br.close();
}
}
}
public void parseAndBuildAST(Reader r) {
// BUILD AST FROM GRAMMAR
ANTLRLexer lexer = new ANTLRLexer(r);
lexer.setFilename(this.getFileName());
// use the rewrite engine because we want to buffer up all tokens
// in case they have a merged lexer/parser, send lexer rules to
// new grammar.
lexer.setTokenObjectClass("antlr.TokenWithIndex");
tokenBuffer = new TokenStreamRewriteEngine(lexer);
tokenBuffer.discard(ANTLRParser.WS);
tokenBuffer.discard(ANTLRParser.ML_COMMENT);
tokenBuffer.discard(ANTLRParser.COMMENT);
tokenBuffer.discard(ANTLRParser.SL_COMMENT);
ANTLRParser parser = new ANTLRParser(tokenBuffer);
parser.setFilename(this.getFileName());
try {
parser.grammar(this);
}
catch (TokenStreamException tse) {
ErrorManager.internalError("unexpected stream error from parsing "+fileName, tse);
}
catch (RecognitionException re) {
ErrorManager.internalError("unexpected parser recognition error from "+fileName, re);
}
dealWithTreeFilterMode(); // tree grammar and filter=true?
if ( lexer.hasASTOperator && !buildAST() ) {
Object value = getOption("output");
if ( value == null ) {
ErrorManager.grammarWarning(ErrorManager.MSG_REWRITE_OR_OP_WITH_NO_OUTPUT_OPTION,
this, null);
setOption("output", "AST", null);
}
else {
ErrorManager.grammarError(ErrorManager.MSG_AST_OP_WITH_NON_AST_OUTPUT_OPTION,
this, null, value);
}
}
grammarTree = (GrammarAST)parser.getAST();
setFileName(lexer.getFilename()); // the lexer #src might change name
if ( grammarTree==null || grammarTree.findFirstType(ANTLRParser.RULE)==null ) {
ErrorManager.error(ErrorManager.MSG_NO_RULES, getFileName());
return;
}
// Get syn pred rules and add to existing tree
List synpredRules =
getArtificialRulesForSyntacticPredicates(parser,
nameToSynpredASTMap);
for (int i = 0; i < synpredRules.size(); i++) {
GrammarAST rAST = (GrammarAST) synpredRules.get(i);
grammarTree.addChild(rAST);
}
}
protected void dealWithTreeFilterMode() {
Object filterMode = (String)getOption("filter");
if ( type==TREE_PARSER && filterMode!=null && filterMode.toString().equals("true") ) {
// check for conflicting options
// filter => backtrack=true
// filter&&output=AST => rewrite=true
// filter&&output!=AST => error
// any deviation from valid option set is an error
Object backtrack = (String)getOption("backtrack");
Object output = getOption("output");
Object rewrite = getOption("rewrite");
if ( backtrack!=null && !backtrack.toString().equals("true") ) {
ErrorManager.error(ErrorManager.MSG_CONFLICTING_OPTION_IN_TREE_FILTER,
"backtrack", backtrack);
}
if ( output!=null && !output.toString().equals("AST") ) {
ErrorManager.error(ErrorManager.MSG_CONFLICTING_OPTION_IN_TREE_FILTER,
"output", output);
setOption("output", "", null);
}
if ( rewrite!=null && !rewrite.toString().equals("true") ) {
ErrorManager.error(ErrorManager.MSG_CONFLICTING_OPTION_IN_TREE_FILTER,
"rewrite", rewrite);
}
// set options properly
setOption("backtrack", "true", null);
if ( output!=null && output.toString().equals("AST") ) {
setOption("rewrite", "true", null);
}
// @synpredgate set to state.backtracking==1 by code gen when filter=true
// superClass set in template target::treeParser
}
}
public void defineGrammarSymbols() {
if ( Tool.internalOption_PrintGrammarTree ) {
System.out.println(grammarTree.toStringList());
}
// DEFINE RULES
//System.out.println("### define "+name+" rules");
DefineGrammarItemsWalker defineItemsWalker = new DefineGrammarItemsWalker();
defineItemsWalker.setASTNodeClass("org.antlr.tool.GrammarAST");
try {
defineItemsWalker.grammar(grammarTree, this);
}
catch (RecognitionException re) {
ErrorManager.error(ErrorManager.MSG_BAD_AST_STRUCTURE,
re);
}
}
/** ANALYZE ACTIONS, LOOKING FOR LABEL AND ATTR REFS, sanity check */
public void checkNameSpaceAndActions() {
examineAllExecutableActions();
checkAllRulesForUselessLabels();
nameSpaceChecker.checkConflicts();
}
/** Many imports are illegal such as lexer into a tree grammar */
public boolean validImport(Grammar delegate) {
List<Integer> validDelegators = validDelegations.get(delegate.type);
return validDelegators!=null && validDelegators.contains(this.type);
}
/** If the grammar is a combined grammar, return the text of the implicit
* lexer grammar.
*/
public String getLexerGrammar() {
if ( lexerGrammarST.getAttribute("literals")==null &&
lexerGrammarST.getAttribute("rules")==null )
{
// if no rules, return nothing
return null;
}
lexerGrammarST.setAttribute("name", name);
// if there are any actions set for lexer, pass them in
if ( actions.get("lexer")!=null ) {
lexerGrammarST.setAttribute("actionNames",
((Map)actions.get("lexer")).keySet());
lexerGrammarST.setAttribute("actions",
((Map)actions.get("lexer")).values());
}
// make sure generated grammar has the same options
if ( options!=null ) {
Iterator optionNames = options.keySet().iterator();
while (optionNames.hasNext()) {
String optionName = (String) optionNames.next();
if ( !doNotCopyOptionsToLexer.contains(optionName) ) {
Object value = options.get(optionName);
lexerGrammarST.setAttribute("options.{name,value}", optionName, value);
}
}
}
return lexerGrammarST.toString();
}
public String getImplicitlyGeneratedLexerFileName() {
return name+
IGNORE_STRING_IN_GRAMMAR_FILE_NAME +
LEXER_GRAMMAR_FILE_EXTENSION;
}
/** Get the name of the generated recognizer; may or may not be same
* as grammar name.
* Recognizer is TParser and TLexer from T if combined, else
* just use T regardless of grammar type.
*/
public String getRecognizerName() {
String suffix = "";
List<Grammar> grammarsFromRootToMe = composite.getDelegators(this);
//System.out.println("grammarsFromRootToMe="+grammarsFromRootToMe);
String qualifiedName = name;
if ( grammarsFromRootToMe!=null ) {
StringBuffer buf = new StringBuffer();
for (Grammar g : grammarsFromRootToMe) {
buf.append(g.name);
buf.append('_');
}
buf.append(name);
qualifiedName = buf.toString();
}
if ( type==Grammar.COMBINED ||
(type==Grammar.LEXER && implicitLexer) )
{
suffix = Grammar.grammarTypeToFileNameSuffix[type];
}
return qualifiedName+suffix;
}
/** Parse a rule we add artificially that is a list of the other lexer
* rules like this: "Tokens : ID | INT | SEMI ;" nextToken() will invoke
* this to set the current token. Add char literals before
* the rule references.
*
* If in filter mode, we want every alt to backtrack and we need to
* do k=1 to force the "first token def wins" rule. Otherwise, the
* longest-match rule comes into play with LL(*).
*
* The ANTLRParser antlr.g file now invokes this when parsing a lexer
* grammar, which I think is proper even though it peeks at the info
* that later phases will (re)compute. It gets a list of lexer rules
* and builds a string representing the rule; then it creates a parser
* and adds the resulting tree to the grammar's tree.
*/
public GrammarAST addArtificialMatchTokensRule(GrammarAST grammarAST,
List<String> ruleNames,
List<String> delegateNames,
boolean filterMode) {
StringTemplate matchTokenRuleST = null;
if ( filterMode ) {
matchTokenRuleST = new StringTemplate(
ARTIFICIAL_TOKENS_RULENAME+
" options {k=1; backtrack=true;} : <rules; separator=\"|\">;",
AngleBracketTemplateLexer.class);
}
else {
matchTokenRuleST = new StringTemplate(
ARTIFICIAL_TOKENS_RULENAME+" : <rules; separator=\"|\">;",
AngleBracketTemplateLexer.class);
}
// Now add token rule references
for (int i = 0; i < ruleNames.size(); i++) {
String rname = (String) ruleNames.get(i);
matchTokenRuleST.setAttribute("rules", rname);
}
for (int i = 0; i < delegateNames.size(); i++) {
String dname = (String) delegateNames.get(i);
matchTokenRuleST.setAttribute("rules", dname+".Tokens");
}
//System.out.println("tokens rule: "+matchTokenRuleST.toString());
ANTLRLexer lexer = new ANTLRLexer(new StringReader(matchTokenRuleST.toString()));
lexer.setTokenObjectClass("antlr.TokenWithIndex");
TokenStreamRewriteEngine tokbuf =
new TokenStreamRewriteEngine(lexer);
tokbuf.discard(ANTLRParser.WS);
tokbuf.discard(ANTLRParser.ML_COMMENT);
tokbuf.discard(ANTLRParser.COMMENT);
tokbuf.discard(ANTLRParser.SL_COMMENT);
ANTLRParser parser = new ANTLRParser(tokbuf);
parser.setGrammar(this);
parser.setGtype(ANTLRParser.LEXER_GRAMMAR);
parser.setASTNodeClass("org.antlr.tool.GrammarAST");
try {
parser.rule();
if ( Tool.internalOption_PrintGrammarTree ) {
System.out.println("Tokens rule: "+parser.getAST().toStringTree());
}
GrammarAST p = grammarAST;
while ( p.getType()!=ANTLRParser.LEXER_GRAMMAR ) {
p = (GrammarAST)p.getNextSibling();
}
p.addChild(parser.getAST());
}
catch (Exception e) {
ErrorManager.error(ErrorManager.MSG_ERROR_CREATING_ARTIFICIAL_RULE,
e);
}
return (GrammarAST)parser.getAST();
}
/** for any syntactic predicates, we need to define rules for them; they will get
* defined automatically like any other rule. :)
*/
protected List getArtificialRulesForSyntacticPredicates(ANTLRParser parser,
LinkedHashMap nameToSynpredASTMap)
{
List rules = new ArrayList();
if ( nameToSynpredASTMap==null ) {
return rules;
}
Set predNames = nameToSynpredASTMap.keySet();
boolean isLexer = grammarTree.getType()==ANTLRParser.LEXER_GRAMMAR;
for (Iterator it = predNames.iterator(); it.hasNext();) {
String synpredName = (String)it.next();
GrammarAST fragmentAST =
(GrammarAST) nameToSynpredASTMap.get(synpredName);
GrammarAST ruleAST =
parser.createSimpleRuleAST(synpredName,
fragmentAST,
isLexer);
rules.add(ruleAST);
}
return rules;
}
/** Walk the list of options, altering this Grammar object according
* to any I recognize.
protected void processOptions() {
Iterator optionNames = options.keySet().iterator();
while (optionNames.hasNext()) {
String optionName = (String) optionNames.next();
Object value = options.get(optionName);
if ( optionName.equals("tokenVocab") ) {
}
}
}
*/
/** Define all the rule begin/end NFAStates to solve forward reference
* issues. Critical for composite grammars too.
* This is normally called on all root/delegates manually and then
* buildNFA() is called afterwards because the NFA construction needs
* to see rule start/stop states from potentially every grammar. Has
* to be have these created a priori. Testing routines will often
* just call buildNFA(), which forces a call to this method if not
* done already. Works ONLY for single noncomposite grammars.
*/
public void createRuleStartAndStopNFAStates() {
//System.out.println("### createRuleStartAndStopNFAStates "+getGrammarTypeString()+" grammar "+name+" NFAs");
if ( nfa!=null ) {
return;
}
nfa = new NFA(this);
factory = new NFAFactory(nfa);
Collection rules = getRules();
for (Iterator itr = rules.iterator(); itr.hasNext();) {
Rule r = (Rule) itr.next();
String ruleName = r.name;
NFAState ruleBeginState = factory.newState();
ruleBeginState.setDescription("rule "+ruleName+" start");
ruleBeginState.enclosingRule = r;
r.startState = ruleBeginState;
NFAState ruleEndState = factory.newState();
ruleEndState.setDescription("rule "+ruleName+" end");
ruleEndState.setAcceptState(true);
ruleEndState.enclosingRule = r;
r.stopState = ruleEndState;
}
}
public void buildNFA() {
if ( nfa==null ) {
createRuleStartAndStopNFAStates();
}
if ( nfa.complete ) {
// don't let it create more than once; has side-effects
return;
}
//System.out.println("### build "+getGrammarTypeString()+" grammar "+name+" NFAs");
if ( getRules().size()==0 ) {
return;
}
TreeToNFAConverter nfaBuilder = new TreeToNFAConverter(this, nfa, factory);
try {
nfaBuilder.grammar(grammarTree);
}
catch (RecognitionException re) {
ErrorManager.error(ErrorManager.MSG_BAD_AST_STRUCTURE,
name,
re);
}
nfa.complete = true;
}
/** For each decision in this grammar, compute a single DFA using the
* NFA states associated with the decision. The DFA construction
* determines whether or not the alternatives in the decision are
* separable using a regular lookahead language.
*
* Store the lookahead DFAs in the AST created from the user's grammar
* so the code generator or whoever can easily access it.
*
* This is a separate method because you might want to create a
* Grammar without doing the expensive analysis.
*/
public void createLookaheadDFAs() {
createLookaheadDFAs(true);
}
public void createLookaheadDFAs(boolean wackTempStructures) {
if ( nfa==null ) {
buildNFA();
}
// CHECK FOR LEFT RECURSION; Make sure we can actually do analysis
checkAllRulesForLeftRecursion();
/*
// was there a severe problem while sniffing the grammar?
if ( ErrorManager.doNotAttemptAnalysis() ) {
return;
}
*/
long start = System.currentTimeMillis();
//System.out.println("### create DFAs");
int numDecisions = getNumberOfDecisions();
if ( NFAToDFAConverter.SINGLE_THREADED_NFA_CONVERSION ) {
for (int decision=1; decision<=numDecisions; decision++) {
NFAState decisionStartState = getDecisionNFAStartState(decision);
if ( leftRecursiveRules.contains(decisionStartState.enclosingRule) ) {
// don't bother to process decisions within left recursive rules.
if ( composite.watchNFAConversion ) {
System.out.println("ignoring decision "+decision+
" within left-recursive rule "+decisionStartState.enclosingRule.name);
}
continue;
}
if ( !externalAnalysisAbort && decisionStartState.getNumberOfTransitions()>1 ) {
Rule r = decisionStartState.enclosingRule;
if ( r.isSynPred && !synPredNamesUsedInDFA.contains(r.name) ) {
continue;
}
DFA dfa = null;
// if k=* or k=1, try LL(1)
if ( getUserMaxLookahead(decision)==0 ||
getUserMaxLookahead(decision)==1 )
{
dfa = createLL_1_LookaheadDFA(decision);
}
if ( dfa==null ) {
if ( composite.watchNFAConversion ) {
System.out.println("decision "+decision+
" not suitable for LL(1)-optimized DFA analysis");
}
dfa = createLookaheadDFA(decision, wackTempStructures);
}
if ( dfa.startState==null ) {
// something went wrong; wipe out DFA
setLookaheadDFA(decision, null);
}
if ( Tool.internalOption_PrintDFA ) {
System.out.println("DFA d="+decision);
FASerializer serializer = new FASerializer(nfa.grammar);
String result = serializer.serialize(dfa.startState);
System.out.println(result);
}
}
}
}
else {
ErrorManager.info("two-threaded DFA conversion");
// create a barrier expecting n DFA and this main creation thread
Barrier barrier = new Barrier(3);
// assume 2 CPU for now
int midpoint = numDecisions/2;
NFAConversionThread t1 =
new NFAConversionThread(this, barrier, 1, midpoint);
new Thread(t1).start();
if ( midpoint == (numDecisions/2) ) {
midpoint++;
}
NFAConversionThread t2 =
new NFAConversionThread(this, barrier, midpoint, numDecisions);
new Thread(t2).start();
// wait for these two threads to finish
try {
barrier.waitForRelease();
}
catch(InterruptedException e) {
ErrorManager.internalError("what the hell? DFA interruptus", e);
}
}
long stop = System.currentTimeMillis();
DFACreationWallClockTimeInMS = stop - start;
// indicate that we've finished building DFA (even if #decisions==0)
allDecisionDFACreated = true;
}
public DFA createLL_1_LookaheadDFA(int decision) {
Decision d = getDecision(decision);
String enclosingRule = d.startState.enclosingRule.name;
Rule r = d.startState.enclosingRule;
NFAState decisionStartState = getDecisionNFAStartState(decision);
if ( composite.watchNFAConversion ) {
System.out.println("--------------------\nattempting LL(1) DFA (d="
+decisionStartState.getDecisionNumber()+") for "+
decisionStartState.getDescription());
}
if ( r.isSynPred && !synPredNamesUsedInDFA.contains(enclosingRule) ) {
return null;
}
// compute lookahead for each alt
int numAlts = getNumberOfAltsForDecisionNFA(decisionStartState);
LookaheadSet[] altLook = new LookaheadSet[numAlts+1];
for (int alt = 1; alt <= numAlts; alt++) {
int walkAlt =
decisionStartState.translateDisplayAltToWalkAlt(alt);
NFAState altLeftEdge = getNFAStateForAltOfDecision(decisionStartState, walkAlt);
NFAState altStartState = (NFAState)altLeftEdge.transition[0].target;
//System.out.println("alt "+alt+" start state = "+altStartState.stateNumber);
altLook[alt] = ll1Analyzer.LOOK(altStartState);
//System.out.println("alt "+alt+": "+altLook[alt].toString(this));
}
// compare alt i with alt j for disjointness
boolean decisionIsLL_1 = true;
outer:
for (int i = 1; i <= numAlts; i++) {
for (int j = i+1; j <= numAlts; j++) {
/*
System.out.println("compare "+i+", "+j+": "+
altLook[i].toString(this)+" with "+
altLook[j].toString(this));
*/
LookaheadSet collision = altLook[i].intersection(altLook[j]);
if ( !collision.isNil() ) {
//System.out.println("collision (non-LL(1)): "+collision.toString(this));
decisionIsLL_1 = false;
break outer;
}
}
}
boolean foundConfoundingPredicate =
ll1Analyzer.detectConfoundingPredicates(decisionStartState);
if ( decisionIsLL_1 && !foundConfoundingPredicate ) {
// build an LL(1) optimized DFA with edge for each altLook[i]
if ( NFAToDFAConverter.debug ) {
System.out.println("decision "+decision+" is simple LL(1)");
}
DFA lookaheadDFA = new LL1DFA(decision, decisionStartState, altLook);
setLookaheadDFA(decision, lookaheadDFA);
updateLineColumnToLookaheadDFAMap(lookaheadDFA);
return lookaheadDFA;
}
// not LL(1) but perhaps we can solve with simplified predicate search
// even if k=1 set manually, only resolve here if we have preds; i.e.,
// don't resolve etc...
/*
SemanticContext visiblePredicates =
ll1Analyzer.getPredicates(decisionStartState);
boolean foundConfoundingPredicate =
ll1Analyzer.detectConfoundingPredicates(decisionStartState);
*/
// exit if not forced k=1 or we found a predicate situation we
// can't handle: predicates in rules invoked from this decision.
if ( getUserMaxLookahead(decision)!=1 || // not manually set to k=1
!getAutoBacktrackMode(decision) ||
foundConfoundingPredicate )
{
//System.out.println("trying LL(*)");
return null;
}
List<IntervalSet> edges = new ArrayList<IntervalSet>();
for (int i = 1; i < altLook.length; i++) {
LookaheadSet s = altLook[i];
edges.add((IntervalSet)s.tokenTypeSet);
}
List<IntervalSet> disjoint = makeEdgeSetsDisjoint(edges);
//System.out.println("disjoint="+disjoint);
MultiMap<IntervalSet, Integer> edgeMap = new MultiMap<IntervalSet, Integer>();
for (int i = 0; i < disjoint.size(); i++) {
IntervalSet ds = (IntervalSet) disjoint.get(i);
for (int alt = 1; alt < altLook.length; alt++) {
LookaheadSet look = altLook[alt];
if ( !ds.and(look.tokenTypeSet).isNil() ) {
edgeMap.map(ds, alt);
}
}
}
//System.out.println("edge map: "+edgeMap);
// TODO: how do we know we covered stuff?
// build an LL(1) optimized DFA with edge for each altLook[i]
DFA lookaheadDFA = new LL1DFA(decision, decisionStartState, edgeMap);
setLookaheadDFA(decision, lookaheadDFA);
// create map from line:col to decision DFA (for ANTLRWorks)
updateLineColumnToLookaheadDFAMap(lookaheadDFA);
return lookaheadDFA;
}
private void updateLineColumnToLookaheadDFAMap(DFA lookaheadDFA) {
GrammarAST decisionAST = nfa.grammar.getDecisionBlockAST(lookaheadDFA.decisionNumber);
int line = decisionAST.getLine();
int col = decisionAST.getColumn();
lineColumnToLookaheadDFAMap.put(new StringBuffer().append(line + ":")
.append(col).toString(), lookaheadDFA);
}
protected List<IntervalSet> makeEdgeSetsDisjoint(List<IntervalSet> edges) {
OrderedHashSet<IntervalSet> disjointSets = new OrderedHashSet<IntervalSet>();
// walk each incoming edge label/set and add to disjoint set
int numEdges = edges.size();
for (int e = 0; e < numEdges; e++) {
IntervalSet t = (IntervalSet) edges.get(e);
if ( disjointSets.contains(t) ) { // exact set present
continue;
}
// compare t with set i for disjointness
IntervalSet remainder = t; // remainder starts out as whole set to add
int numDisjointElements = disjointSets.size();
for (int i = 0; i < numDisjointElements; i++) {
IntervalSet s_i = (IntervalSet)disjointSets.get(i);
if ( t.and(s_i).isNil() ) { // nothing in common
continue;
}
//System.out.println(label+" collides with "+rl);
// For any (s_i, t) with s_i&t!=nil replace with (s_i-t, s_i&t)
// (ignoring s_i-t if nil; don't put in list)
// Replace existing s_i with intersection since we
// know that will always be a non nil character class
IntervalSet intersection = (IntervalSet)s_i.and(t);
disjointSets.set(i, intersection);
// Compute s_i-t to see what is in current set and not in incoming
IntSet existingMinusNewElements = s_i.subtract(t);
//System.out.println(s_i+"-"+t+"="+existingMinusNewElements);
if ( !existingMinusNewElements.isNil() ) {
// found a new character class, add to the end (doesn't affect
// outer loop duration due to n computation a priori.
disjointSets.add(existingMinusNewElements);
}
// anything left to add to the reachableLabels?
remainder = (IntervalSet)t.subtract(s_i);
if ( remainder.isNil() ) {
break; // nothing left to add to set. done!
}
t = remainder;
}
if ( !remainder.isNil() ) {
disjointSets.add(remainder);
}
}
return disjointSets.elements();
}
public DFA createLookaheadDFA(int decision, boolean wackTempStructures) {
Decision d = getDecision(decision);
String enclosingRule = d.startState.enclosingRule.name;
Rule r = d.startState.enclosingRule;
//System.out.println("createLookaheadDFA(): "+enclosingRule+" dec "+decision+"; synprednames prev used "+synPredNamesUsedInDFA);
NFAState decisionStartState = getDecisionNFAStartState(decision);
long startDFA=0,stopDFA=0;
if ( composite.watchNFAConversion ) {
System.out.println("--------------------\nbuilding lookahead DFA (d="
+decisionStartState.getDecisionNumber()+") for "+
decisionStartState.getDescription());
startDFA = System.currentTimeMillis();
}
DFA lookaheadDFA = new DFA(decision, decisionStartState);
// Retry to create a simpler DFA if analysis failed (non-LL(*),
// recursion overflow, or time out).
boolean failed =
lookaheadDFA.analysisTimedOut() ||
lookaheadDFA.probe.isNonLLStarDecision() ||
lookaheadDFA.probe.analysisOverflowed();
if ( failed && lookaheadDFA.okToRetryDFAWithK1() ) {
// set k=1 option and try again.
// First, clean up tracking stuff
decisionsWhoseDFAsUsesSynPreds.remove(lookaheadDFA);
// TODO: clean up synPredNamesUsedInDFA also (harder)
d.blockAST.setBlockOption(this, "k", Utils.integer(1));
if ( composite.watchNFAConversion ) {
System.out.print("trying decision "+decision+
" again with k=1; reason: "+
lookaheadDFA.getReasonForFailure());
}
lookaheadDFA = null; // make sure other memory is "free" before redoing
lookaheadDFA = new DFA(decision, decisionStartState);
}
if ( lookaheadDFA.analysisTimedOut() ) { // did analysis bug out?
ErrorManager.internalError("could not even do k=1 for decision "+
decision+"; reason: "+
lookaheadDFA.getReasonForFailure());
}
setLookaheadDFA(decision, lookaheadDFA);
if ( wackTempStructures ) {
for (DFAState s : lookaheadDFA.getUniqueStates().values()) {
s.reset();
}
}
// create map from line:col to decision DFA (for ANTLRWorks)
updateLineColumnToLookaheadDFAMap(lookaheadDFA);
if ( composite.watchNFAConversion ) {
stopDFA = System.currentTimeMillis();
System.out.println("cost: "+lookaheadDFA.getNumberOfStates()+
" states, "+(int)(stopDFA-startDFA)+" ms");
}
//System.out.println("after create DFA; synPredNamesUsedInDFA="+synPredNamesUsedInDFA);
return lookaheadDFA;
}
/** Terminate DFA creation (grammar analysis).
*/
public void externallyAbortNFAToDFAConversion() {
externalAnalysisAbort = true;
}
public boolean NFAToDFAConversionExternallyAborted() {
return externalAnalysisAbort;
}
/** Return a new unique integer in the token type space */
public int getNewTokenType() {
composite.maxTokenType++;
return composite.maxTokenType;
}
/** Define a token at a particular token type value. Blast an
* old value with a new one. This is called normal grammar processsing
* and during import vocab operations to set tokens with specific values.
*/
public void defineToken(String text, int tokenType) {
//System.out.println("defineToken("+text+", "+tokenType+")");
if ( composite.tokenIDToTypeMap.get(text)!=null ) {
// already defined? Must be predefined one like EOF;
// do nothing
return;
}
// the index in the typeToTokenList table is actually shifted to
// hold faux labels as you cannot have negative indices.
if ( text.charAt(0)=='\'' ) {
composite.stringLiteralToTypeMap.put(text, Utils.integer(tokenType));
// track in reverse index too
if ( tokenType>=composite.typeToStringLiteralList.size() ) {
composite.typeToStringLiteralList.setSize(tokenType+1);
}
composite.typeToStringLiteralList.set(tokenType, text);
}
else { // must be a label like ID
composite.tokenIDToTypeMap.put(text, Utils.integer(tokenType));
}
int index = Label.NUM_FAUX_LABELS+tokenType-1;
//System.out.println("defining "+name+" token "+text+" at type="+tokenType+", index="+index);
composite.maxTokenType = Math.max(composite.maxTokenType, tokenType);
if ( index>=composite.typeToTokenList.size() ) {
composite.typeToTokenList.setSize(index+1);
}
String prevToken = (String)composite.typeToTokenList.get(index);
if ( prevToken==null || prevToken.charAt(0)=='\'' ) {
// only record if nothing there before or if thing before was a literal
composite.typeToTokenList.set(index, text);
}
}
/** Define a new rule. A new rule index is created by incrementing
* ruleIndex.
*/
public void defineRule(antlr.Token ruleToken,
String modifier,
Map options,
GrammarAST tree,
GrammarAST argActionAST,
int numAlts)
{
String ruleName = ruleToken.getText();
if ( getLocallyDefinedRule(ruleName)!=null ) {
ErrorManager.grammarError(ErrorManager.MSG_RULE_REDEFINITION,
this, ruleToken, ruleName);
return;
}
if ( (type==Grammar.PARSER||type==Grammar.TREE_PARSER) &&
Character.isUpperCase(ruleName.charAt(0)) )
{
ErrorManager.grammarError(ErrorManager.MSG_LEXER_RULES_NOT_ALLOWED,
this, ruleToken, ruleName);
return;
}
Rule r = new Rule(this, ruleName, composite.ruleIndex, numAlts);
/*
System.out.println("defineRule("+ruleName+",modifier="+modifier+
"): index="+r.index+", nalts="+numAlts);
*/
r.modifier = modifier;
nameToRuleMap.put(ruleName, r);
setRuleAST(ruleName, tree);
r.setOptions(options, ruleToken);
r.argActionAST = argActionAST;
composite.ruleIndexToRuleList.setSize(composite.ruleIndex+1);
composite.ruleIndexToRuleList.set(composite.ruleIndex, r);
composite.ruleIndex++;
if ( ruleName.startsWith(SYNPRED_RULE_PREFIX) ) {
r.isSynPred = true;
}
}
/** Define a new predicate and get back its name for use in building
* a semantic predicate reference to the syn pred.
*/
public String defineSyntacticPredicate(GrammarAST blockAST,
String currentRuleName)
{
if ( nameToSynpredASTMap==null ) {
nameToSynpredASTMap = new LinkedHashMap();
}
String predName =
SYNPRED_RULE_PREFIX+(nameToSynpredASTMap.size() + 1)+"_"+name;
blockAST.setTreeEnclosingRuleNameDeeply(predName);
nameToSynpredASTMap.put(predName, blockAST);
return predName;
}
public LinkedHashMap getSyntacticPredicates() {
return nameToSynpredASTMap;
}
public GrammarAST getSyntacticPredicate(String name) {
if ( nameToSynpredASTMap==null ) {
return null;
}
return (GrammarAST)nameToSynpredASTMap.get(name);
}
public void synPredUsedInDFA(DFA dfa, SemanticContext semCtx) {
decisionsWhoseDFAsUsesSynPreds.add(dfa);
semCtx.trackUseOfSyntacticPredicates(this); // walk ctx looking for preds
}
/*
public Set<Rule> getRuleNamesVisitedDuringLOOK() {
return rulesSensitiveToOtherRules;
}
*/
/** Given @scope::name {action} define it for this grammar. Later,
* the code generator will ask for the actions table. For composite
* grammars, make sure header action propogates down to all delegates.
*/
public void defineNamedAction(GrammarAST ampersandAST,
String scope,
GrammarAST nameAST,
GrammarAST actionAST)
{
if ( scope==null ) {
scope = getDefaultActionScope(type);
}
//System.out.println("@"+scope+"::"+nameAST.getText()+"{"+actionAST.getText()+"}");
String actionName = nameAST.getText();
Map scopeActions = (Map)actions.get(scope);
if ( scopeActions==null ) {
scopeActions = new HashMap();
actions.put(scope, scopeActions);
}
GrammarAST a = (GrammarAST)scopeActions.get(actionName);
if ( a!=null ) {
ErrorManager.grammarError(
ErrorManager.MSG_ACTION_REDEFINITION,this,
nameAST.getToken(),nameAST.getText());
}
else {
scopeActions.put(actionName,actionAST);
}
// propogate header (regardless of scope (lexer, parser, ...) ?
if ( this==composite.getRootGrammar() && actionName.equals("header") ) {
List<Grammar> allgrammars = composite.getRootGrammar().getDelegates();
for (Grammar g : allgrammars) {
g.defineNamedAction(ampersandAST, scope, nameAST, actionAST);
}
}
}
public void setSynPredGateIfNotAlready(StringTemplate gateST) {
String scope = getDefaultActionScope(type);
Map actionsForGrammarScope = (Map)actions.get(scope);
// if no synpredgate action set by user then set
if ( (actionsForGrammarScope==null ||
!actionsForGrammarScope.containsKey(Grammar.SYNPREDGATE_ACTION_NAME)) )
{
if ( actionsForGrammarScope==null ) {
actionsForGrammarScope=new HashMap();
actions.put(scope, actionsForGrammarScope);
}
actionsForGrammarScope.put(Grammar.SYNPREDGATE_ACTION_NAME,
gateST);
}
}
public Map getActions() {
return actions;
}
/** Given a grammar type, what should be the default action scope?
* If I say @members in a COMBINED grammar, for example, the
* default scope should be "parser".
*/
public String getDefaultActionScope(int grammarType) {
switch (grammarType) {
case Grammar.LEXER :
return "lexer";
case Grammar.PARSER :
case Grammar.COMBINED :
return "parser";
case Grammar.TREE_PARSER :
return "treeparser";
}
return null;
}
public void defineLexerRuleFoundInParser(antlr.Token ruleToken,
GrammarAST ruleAST)
{
//System.out.println("rule tree is:\n"+ruleAST.toStringTree());
/*
String ruleText = tokenBuffer.toOriginalString(ruleAST.ruleStartTokenIndex,
ruleAST.ruleStopTokenIndex);
*/
// first, create the text of the rule
StringBuffer buf = new StringBuffer();
buf.append("// $ANTLR src \"");
buf.append(getFileName());
buf.append("\" ");
buf.append(ruleAST.getLine());
buf.append("\n");
for (int i=ruleAST.ruleStartTokenIndex;
i<=ruleAST.ruleStopTokenIndex && i<tokenBuffer.size();
i++)
{
TokenWithIndex t = (TokenWithIndex)tokenBuffer.getToken(i);
// undo the text deletions done by the lexer (ugh)
if ( t.getType()==ANTLRParser.BLOCK ) {
buf.append("(");
}
else if ( t.getType()==ANTLRParser.ACTION ) {
buf.append("{");
buf.append(t.getText());
buf.append("}");
}
else if ( t.getType()==ANTLRParser.SEMPRED ||
t.getType()==ANTLRParser.SYN_SEMPRED ||
t.getType()==ANTLRParser.GATED_SEMPRED ||
t.getType()==ANTLRParser.BACKTRACK_SEMPRED )
{
buf.append("{");
buf.append(t.getText());
buf.append("}?");
}
else if ( t.getType()==ANTLRParser.ARG_ACTION ) {
buf.append("[");
buf.append(t.getText());
buf.append("]");
}
else {
buf.append(t.getText());
}
}
String ruleText = buf.toString();
//System.out.println("[["+ruleText+"]]");
// now put the rule into the lexer grammar template
if ( getGrammarIsRoot() ) { // don't build lexers for delegates
lexerGrammarST.setAttribute("rules", ruleText);
}
// track this lexer rule's name
composite.lexerRules.add(ruleToken.getText());
}
/** If someone does PLUS='+' in the parser, must make sure we get
* "PLUS : '+' ;" in lexer not "T73 : '+';"
*/
public void defineLexerRuleForAliasedStringLiteral(String tokenID,
String literal,
int tokenType)
{
if ( getGrammarIsRoot() ) { // don't build lexers for delegates
//System.out.println("defineLexerRuleForAliasedStringLiteral: "+literal+" "+tokenType);
lexerGrammarST.setAttribute("literals.{ruleName,type,literal}",
tokenID,
Utils.integer(tokenType),
literal);
}
// track this lexer rule's name
composite.lexerRules.add(tokenID);
}
public void defineLexerRuleForStringLiteral(String literal, int tokenType) {
//System.out.println("defineLexerRuleForStringLiteral: "+literal+" "+tokenType);
// compute new token name like T237 and define it as having tokenType
String tokenID = computeTokenNameFromLiteral(tokenType,literal);
defineToken(tokenID, tokenType);
// tell implicit lexer to define a rule to match the literal
if ( getGrammarIsRoot() ) { // don't build lexers for delegates
lexerGrammarST.setAttribute("literals.{ruleName,type,literal}",
tokenID,
Utils.integer(tokenType),
literal);
}
}
public Rule getLocallyDefinedRule(String ruleName) {
Rule r = nameToRuleMap.get(ruleName);
return r;
}
public Rule getRule(String ruleName) {
Rule r = composite.getRule(ruleName);
/*
if ( r!=null && r.grammar != this ) {
System.out.println(name+".getRule("+ruleName+")="+r);
}
*/
return r;
}
public Rule getRule(String scopeName, String ruleName) {
if ( scopeName!=null ) { // scope override
Grammar scope = composite.getGrammar(scopeName);
if ( scope==null ) {
return null;
}
return scope.getLocallyDefinedRule(ruleName);
}
return getRule(ruleName);
}
public int getRuleIndex(String scopeName, String ruleName) {
Rule r = getRule(scopeName, ruleName);
if ( r!=null ) {
return r.index;
}
return INVALID_RULE_INDEX;
}
public int getRuleIndex(String ruleName) {
return getRuleIndex(null, ruleName);
}
public String getRuleName(int ruleIndex) {
Rule r = composite.ruleIndexToRuleList.get(ruleIndex);
if ( r!=null ) {
return r.name;
}
return null;
}
/** Should codegen.g gen rule for ruleName?
* If synpred, only gen if used in a DFA.
* If regular rule, only gen if not overridden in delegator
* Always gen Tokens rule though.
*/
public boolean generateMethodForRule(String ruleName) {
if ( ruleName.equals(ARTIFICIAL_TOKENS_RULENAME) ) {
// always generate Tokens rule to satisfy lexer interface
// but it may have no alternatives.
return true;
}
if ( overriddenRules.contains(ruleName) ) {
// don't generate any overridden rules
return false;
}
// generate if non-synpred or synpred used in a DFA
Rule r = getLocallyDefinedRule(ruleName);
return !r.isSynPred ||
(r.isSynPred&&synPredNamesUsedInDFA.contains(ruleName));
}
public AttributeScope defineGlobalScope(String name, Token scopeAction) {
AttributeScope scope = new AttributeScope(this, name, scopeAction);
scopes.put(name,scope);
return scope;
}
public AttributeScope createReturnScope(String ruleName, Token retAction) {
AttributeScope scope = new AttributeScope(this, ruleName, retAction);
scope.isReturnScope = true;
return scope;
}
public AttributeScope createRuleScope(String ruleName, Token scopeAction) {
AttributeScope scope = new AttributeScope(this, ruleName, scopeAction);
scope.isDynamicRuleScope = true;
return scope;
}
public AttributeScope createParameterScope(String ruleName, Token argAction) {
AttributeScope scope = new AttributeScope(this, ruleName, argAction);
scope.isParameterScope = true;
return scope;
}
/** Get a global scope */
public AttributeScope getGlobalScope(String name) {
return (AttributeScope)scopes.get(name);
}
public Map getGlobalScopes() {
return scopes;
}
/** Define a label defined in a rule r; check the validity then ask the
* Rule object to actually define it.
*/
protected void defineLabel(Rule r, antlr.Token label, GrammarAST element, int type) {
boolean err = nameSpaceChecker.checkForLabelTypeMismatch(r, label, type);
if ( err ) {
return;
}
r.defineLabel(label, element, type);
}
public void defineTokenRefLabel(String ruleName,
antlr.Token label,
GrammarAST tokenRef)
{
Rule r = getLocallyDefinedRule(ruleName);
if ( r!=null ) {
if ( type==LEXER &&
(tokenRef.getType()==ANTLRParser.CHAR_LITERAL||
tokenRef.getType()==ANTLRParser.BLOCK||
tokenRef.getType()==ANTLRParser.NOT||
tokenRef.getType()==ANTLRParser.CHAR_RANGE||
tokenRef.getType()==ANTLRParser.WILDCARD))
{
defineLabel(r, label, tokenRef, CHAR_LABEL);
}
else {
defineLabel(r, label, tokenRef, TOKEN_LABEL);
}
}
}
public void defineWildcardTreeLabel(String ruleName,
antlr.Token label,
GrammarAST tokenRef)
{
Rule r = getLocallyDefinedRule(ruleName);
if ( r!=null ) {
defineLabel(r, label, tokenRef, WILDCARD_TREE_LABEL);
}
}
public void defineWildcardTreeListLabel(String ruleName,
antlr.Token label,
GrammarAST tokenRef)
{
Rule r = getLocallyDefinedRule(ruleName);
if ( r!=null ) {
defineLabel(r, label, tokenRef, WILDCARD_TREE_LIST_LABEL);
}
}
public void defineRuleRefLabel(String ruleName,
antlr.Token label,
GrammarAST ruleRef)
{
Rule r = getLocallyDefinedRule(ruleName);
if ( r!=null ) {
defineLabel(r, label, ruleRef, RULE_LABEL);
}
}
public void defineTokenListLabel(String ruleName,
antlr.Token label,
GrammarAST element)
{
Rule r = getLocallyDefinedRule(ruleName);
if ( r!=null ) {
defineLabel(r, label, element, TOKEN_LIST_LABEL);
}
}
public void defineRuleListLabel(String ruleName,
antlr.Token label,
GrammarAST element)
{
Rule r = getLocallyDefinedRule(ruleName);
if ( r!=null ) {
if ( !r.getHasMultipleReturnValues() ) {
ErrorManager.grammarError(
ErrorManager.MSG_LIST_LABEL_INVALID_UNLESS_RETVAL_STRUCT,this,
label,label.getText());
}
defineLabel(r, label, element, RULE_LIST_LABEL);
}
}
/** Given a set of all rewrite elements on right of ->, filter for
* label types such as Grammar.TOKEN_LABEL, Grammar.TOKEN_LIST_LABEL, ...
* Return a displayable token type name computed from the GrammarAST.
*/
public Set<String> getLabels(Set<GrammarAST> rewriteElements, int labelType) {
Set<String> labels = new HashSet<String>();
for (Iterator it = rewriteElements.iterator(); it.hasNext();) {
GrammarAST el = (GrammarAST) it.next();
if ( el.getType()==ANTLRParser.LABEL ) {
String labelName = el.getText();
Rule enclosingRule = getLocallyDefinedRule(el.enclosingRuleName);
LabelElementPair pair = enclosingRule.getLabel(labelName);
/*
// if tree grammar and we have a wildcard, only notice it
// when looking for rule labels not token label. x=. should
// look like a rule ref since could be subtree.
if ( type==TREE_PARSER && pair!=null &&
pair.elementRef.getType()==ANTLRParser.WILDCARD )
{
if ( labelType==WILDCARD_TREE_LABEL ) {
labels.add(labelName);
continue;
}
else continue;
}
*/
// if valid label and type is what we're looking for
// and not ref to old value val $rule, add to list
if ( pair!=null && pair.type==labelType &&
!labelName.equals(el.enclosingRuleName) )
{
labels.add(labelName);
}
}
}
return labels;
}
/** Before generating code, we examine all actions that can have
* $x.y and $y stuff in them because some code generation depends on
* Rule.referencedPredefinedRuleAttributes. I need to remove unused
* rule labels for example.
*/
protected void examineAllExecutableActions() {
Collection rules = getRules();
for (Iterator it = rules.iterator(); it.hasNext();) {
Rule r = (Rule) it.next();
// walk all actions within the rule elements, args, and exceptions
List<GrammarAST> actions = r.getInlineActions();
for (int i = 0; i < actions.size(); i++) {
GrammarAST actionAST = (GrammarAST) actions.get(i);
ActionAnalysis sniffer =
new ActionAnalysis(this, r.name, actionAST);
sniffer.analyze();
}
// walk any named actions like @init, @after
Collection<GrammarAST> namedActions = r.getActions().values();
for (Iterator it2 = namedActions.iterator(); it2.hasNext();) {
GrammarAST actionAST = (GrammarAST) it2.next();
ActionAnalysis sniffer =
new ActionAnalysis(this, r.name, actionAST);
sniffer.analyze();
}
}
}
/** Remove all labels on rule refs whose target rules have no return value.
* Do this for all rules in grammar.
*/
public void checkAllRulesForUselessLabels() {
if ( type==LEXER ) {
return;
}
Set rules = nameToRuleMap.keySet();
for (Iterator it = rules.iterator(); it.hasNext();) {
String ruleName = (String) it.next();
Rule r = getRule(ruleName);
removeUselessLabels(r.getRuleLabels());
removeUselessLabels(r.getRuleListLabels());
}
}
/** A label on a rule is useless if the rule has no return value, no
* tree or template output, and it is not referenced in an action.
*/
protected void removeUselessLabels(Map ruleToElementLabelPairMap) {
if ( ruleToElementLabelPairMap==null ) {
return;
}
Collection labels = ruleToElementLabelPairMap.values();
List kill = new ArrayList();
for (Iterator labelit = labels.iterator(); labelit.hasNext();) {
LabelElementPair pair = (LabelElementPair) labelit.next();
Rule refdRule = getRule(pair.elementRef.getText());
if ( refdRule!=null && !refdRule.getHasReturnValue() && !pair.actionReferencesLabel ) {
//System.out.println(pair.label.getText()+" is useless");
kill.add(pair.label.getText());
}
}
for (int i = 0; i < kill.size(); i++) {
String labelToKill = (String) kill.get(i);
// System.out.println("kill "+labelToKill);
ruleToElementLabelPairMap.remove(labelToKill);
}
}
/** Track a rule reference within an outermost alt of a rule. Used
* at the moment to decide if $ruleref refers to a unique rule ref in
* the alt. Rewrite rules force tracking of all rule AST results.
*
* This data is also used to verify that all rules have been defined.
*/
public void altReferencesRule(String enclosingRuleName,
GrammarAST refScopeAST,
GrammarAST refAST,
int outerAltNum)
{
/* Do nothing for now; not sure need; track S.x as x
String scope = null;
Grammar scopeG = null;
if ( refScopeAST!=null ) {
if ( !scopedRuleRefs.contains(refScopeAST) ) {
scopedRuleRefs.add(refScopeAST);
}
scope = refScopeAST.getText();
}
*/
Rule r = getRule(enclosingRuleName);
if ( r==null ) {
return; // no error here; see NameSpaceChecker
}
r.trackRuleReferenceInAlt(refAST, outerAltNum);
antlr.Token refToken = refAST.getToken();
if ( !ruleRefs.contains(refAST) ) {
ruleRefs.add(refAST);
}
}
/** Track a token reference within an outermost alt of a rule. Used
* to decide if $tokenref refers to a unique token ref in
* the alt. Does not track literals!
*
* Rewrite rules force tracking of all tokens.
*/
public void altReferencesTokenID(String ruleName, GrammarAST refAST, int outerAltNum) {
Rule r = getLocallyDefinedRule(ruleName);
if ( r==null ) {
return;
}
r.trackTokenReferenceInAlt(refAST, outerAltNum);
if ( !tokenIDRefs.contains(refAST.getToken()) ) {
tokenIDRefs.add(refAST.getToken());
}
}
/** To yield smaller, more readable code, track which rules have their
* predefined attributes accessed. If the rule has no user-defined
* return values, then don't generate the return value scope classes
* etc... Make the rule have void return value. Don't track for lexer
* rules.
*/
public void referenceRuleLabelPredefinedAttribute(String ruleName) {
Rule r = getRule(ruleName);
if ( r!=null && type!=LEXER ) {
// indicate that an action ref'd an attr unless it's in a lexer
// so that $ID.text refs don't force lexer rules to define
// return values...Token objects are created by the caller instead.
r.referencedPredefinedRuleAttributes = true;
}
}
public List checkAllRulesForLeftRecursion() {
return sanity.checkAllRulesForLeftRecursion();
}
/** Return a list of left-recursive rules; no analysis can be done
* successfully on these. Useful to skip these rules then and also
* for ANTLRWorks to highlight them.
*/
public Set<Rule> getLeftRecursiveRules() {
if ( nfa==null ) {
buildNFA();
}
if ( leftRecursiveRules!=null ) {
return leftRecursiveRules;
}
sanity.checkAllRulesForLeftRecursion();
return leftRecursiveRules;
}
public void checkRuleReference(GrammarAST scopeAST,
GrammarAST refAST,
GrammarAST argsAST,
String currentRuleName)
{
sanity.checkRuleReference(scopeAST, refAST, argsAST, currentRuleName);
}
/** Rules like "a : ;" and "a : {...} ;" should not generate
* try/catch blocks for RecognitionException. To detect this
* it's probably ok to just look for any reference to an atom
* that can match some input. W/o that, the rule is unlikey to have
* any else.
*/
public boolean isEmptyRule(GrammarAST block) {
GrammarAST aTokenRefNode =
block.findFirstType(ANTLRParser.TOKEN_REF);
GrammarAST aStringLiteralRefNode =
block.findFirstType(ANTLRParser.STRING_LITERAL);
GrammarAST aCharLiteralRefNode =
block.findFirstType(ANTLRParser.CHAR_LITERAL);
GrammarAST aWildcardRefNode =
block.findFirstType(ANTLRParser.WILDCARD);
GrammarAST aRuleRefNode =
block.findFirstType(ANTLRParser.RULE_REF);
if ( aTokenRefNode==null&&
aStringLiteralRefNode==null&&
aCharLiteralRefNode==null&&
aWildcardRefNode==null&&
aRuleRefNode==null )
{
return true;
}
return false;
}
public boolean isAtomTokenType(int ttype) {
return ttype == ANTLRParser.WILDCARD||
ttype == ANTLRParser.CHAR_LITERAL||
ttype == ANTLRParser.CHAR_RANGE||
ttype == ANTLRParser.STRING_LITERAL||
ttype == ANTLRParser.NOT||
(type != LEXER && ttype == ANTLRParser.TOKEN_REF);
}
public int getTokenType(String tokenName) {
Integer I = null;
if ( tokenName.charAt(0)=='\'') {
I = (Integer)composite.stringLiteralToTypeMap.get(tokenName);
}
else { // must be a label like ID
I = (Integer)composite.tokenIDToTypeMap.get(tokenName);
}
int i = (I!=null)?I.intValue():Label.INVALID;
//System.out.println("grammar type "+type+" "+tokenName+"->"+i);
return i;
}
/** Get the list of tokens that are IDs like BLOCK and LPAREN */
public Set getTokenIDs() {
return composite.tokenIDToTypeMap.keySet();
}
/** Return an ordered integer list of token types that have no
* corresponding token ID like INT or KEYWORD_BEGIN; for stuff
* like 'begin'.
*/
public Collection getTokenTypesWithoutID() {
List types = new ArrayList();
for (int t =Label.MIN_TOKEN_TYPE; t<=getMaxTokenType(); t++) {
String name = getTokenDisplayName(t);
if ( name.charAt(0)=='\'' ) {
types.add(Utils.integer(t));
}
}
return types;
}
/** Get a list of all token IDs and literals that have an associated
* token type.
*/
public Set<String> getTokenDisplayNames() {
Set<String> names = new HashSet<String>();
for (int t =Label.MIN_TOKEN_TYPE; t <=getMaxTokenType(); t++) {
names.add(getTokenDisplayName(t));
}
return names;
}
/** Given a literal like (the 3 char sequence with single quotes) 'a',
* return the int value of 'a'. Convert escape sequences here also.
* ANTLR's antlr.g parser does not convert escape sequences.
*
* 11/26/2005: I changed literals to always be '...' even for strings.
* This routine still works though.
*/
public static int getCharValueFromGrammarCharLiteral(String literal) {
switch ( literal.length() ) {
case 3 :
// 'x'
return literal.charAt(1); // no escape char
case 4 :
// '\x' (antlr lexer will catch invalid char)
if ( Character.isDigit(literal.charAt(2)) ) {
ErrorManager.error(ErrorManager.MSG_SYNTAX_ERROR,
"invalid char literal: "+literal);
return -1;
}
int escChar = literal.charAt(2);
int charVal = ANTLRLiteralEscapedCharValue[escChar];
if ( charVal==0 ) {
// Unnecessary escapes like '\{' should just yield {
return escChar;
}
return charVal;
case 8 :
// '\u1234'
String unicodeChars = literal.substring(3,literal.length()-1);
return Integer.parseInt(unicodeChars, 16);
default :
ErrorManager.error(ErrorManager.MSG_SYNTAX_ERROR,
"invalid char literal: "+literal);
return -1;
}
}
/** ANTLR does not convert escape sequences during the parse phase because
* it could not know how to print String/char literals back out when
* printing grammars etc... Someone in China might use the real unicode
* char in a literal as it will display on their screen; when printing
* back out, I could not know whether to display or use a unicode escape.
*
* This routine converts a string literal with possible escape sequences
* into a pure string of 16-bit char values. Escapes and unicode \u0000
* specs are converted to pure chars. return in a buffer; people may
* want to walk/manipulate further.
*
* The NFA construction routine must know the actual char values.
*/
public static StringBuffer getUnescapedStringFromGrammarStringLiteral(String literal) {
//System.out.println("escape: ["+literal+"]");
StringBuffer buf = new StringBuffer();
int last = literal.length()-1; // skip quotes on outside
for (int i=1; i<last; i++) {
char c = literal.charAt(i);
if ( c=='\\' ) {
i++;
c = literal.charAt(i);
if ( Character.toUpperCase(c)=='U' ) {
// \u0000
i++;
String unicodeChars = literal.substring(i,i+4);
// parse the unicode 16 bit hex value
int val = Integer.parseInt(unicodeChars, 16);
i+=4-1; // loop will inc by 1; only jump 3 then
buf.append((char)val);
}
else if ( Character.isDigit(c) ) {
ErrorManager.error(ErrorManager.MSG_SYNTAX_ERROR,
"invalid char literal: "+literal);
buf.append("\\"+(char)c);
}
else {
buf.append((char)ANTLRLiteralEscapedCharValue[c]); // normal \x escape
}
}
else {
buf.append(c); // simple char x
}
}
//System.out.println("string: ["+buf.toString()+"]");
return buf;
}
/** Pull your token definitions from an existing grammar in memory.
* You must use Grammar() ctor then this method then setGrammarContent()
* to make this work. This was useful primarily for testing and
* interpreting grammars until I added import grammar functionality.
* When you import a grammar you implicitly import its vocabulary as well
* and keep the same token type values.
*
* Returns the max token type found.
*/
public int importTokenVocabulary(Grammar importFromGr) {
Set importedTokenIDs = importFromGr.getTokenIDs();
for (Iterator it = importedTokenIDs.iterator(); it.hasNext();) {
String tokenID = (String) it.next();
int tokenType = importFromGr.getTokenType(tokenID);
composite.maxTokenType = Math.max(composite.maxTokenType,tokenType);
if ( tokenType>=Label.MIN_TOKEN_TYPE ) {
//System.out.println("import token from grammar "+tokenID+"="+tokenType);
defineToken(tokenID, tokenType);
}
}
return composite.maxTokenType; // return max found
}
/** Import the rules/tokens of a delegate grammar. All delegate grammars are
* read during the ctor of first Grammar created.
*
* Do not create NFA here because NFA construction needs to hook up with
* overridden rules in delegation root grammar.
*/
public void importGrammar(GrammarAST grammarNameAST, String label) {
String grammarName = grammarNameAST.getText();
//System.out.println("import "+gfile.getName());
String gname = grammarName + GRAMMAR_FILE_EXTENSION;
BufferedReader br = null;
try {
String fullName = tool.getLibraryFile(gname);
FileReader fr = new FileReader(fullName);
br = new BufferedReader(fr);
Grammar delegateGrammar = null;
delegateGrammar = new Grammar(tool, gname, composite);
delegateGrammar.label = label;
addDelegateGrammar(delegateGrammar);
delegateGrammar.parseAndBuildAST(br);
if ( !validImport(delegateGrammar) ) {
ErrorManager.grammarError(ErrorManager.MSG_INVALID_IMPORT,
this,
grammarNameAST.token,
this,
delegateGrammar);
return;
}
if ( this.type==COMBINED &&
(delegateGrammar.name.equals(this.name+grammarTypeToFileNameSuffix[LEXER])||
delegateGrammar.name.equals(this.name+grammarTypeToFileNameSuffix[PARSER])) )
{
ErrorManager.grammarError(ErrorManager.MSG_IMPORT_NAME_CLASH,
this,
grammarNameAST.token,
this,
delegateGrammar);
return;
}
if ( delegateGrammar.grammarTree!=null ) {
// we have a valid grammar
// deal with combined grammars
if ( delegateGrammar.type == LEXER && this.type == COMBINED ) {
// ooops, we wasted some effort; tell lexer to read it in
// later
lexerGrammarST.setAttribute("imports", grammarName);
// but, this parser grammar will need the vocab
// so add to composite anyway so we suck in the tokens later
}
}
//System.out.println("Got grammar:\n"+delegateGrammar);
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_OPEN_FILE,
gname,
ioe);
}
finally {
if ( br!=null ) {
try {
br.close();
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_CLOSE_FILE,
gname,
ioe);
}
}
}
}
/** add new delegate to composite tree */
protected void addDelegateGrammar(Grammar delegateGrammar) {
CompositeGrammarTree t = composite.delegateGrammarTreeRoot.findNode(this);
t.addChild(new CompositeGrammarTree(delegateGrammar));
// make sure new grammar shares this composite
delegateGrammar.composite = this.composite;
}
/** Load a vocab file <vocabName>.tokens and return max token type found. */
public int importTokenVocabulary(GrammarAST tokenVocabOptionAST,
String vocabName)
{
if ( !getGrammarIsRoot() ) {
ErrorManager.grammarWarning(ErrorManager.MSG_TOKEN_VOCAB_IN_DELEGATE,
this,
tokenVocabOptionAST.token,
name);
return composite.maxTokenType;
}
File fullFile = tool.getImportedVocabFile(vocabName);
try {
FileReader fr = new FileReader(fullFile);
BufferedReader br = new BufferedReader(fr);
StreamTokenizer tokenizer = new StreamTokenizer(br);
tokenizer.parseNumbers();
tokenizer.wordChars('_', '_');
tokenizer.eolIsSignificant(true);
tokenizer.slashSlashComments(true);
tokenizer.slashStarComments(true);
tokenizer.ordinaryChar('=');
tokenizer.quoteChar('\'');
tokenizer.whitespaceChars(' ',' ');
tokenizer.whitespaceChars('\t','\t');
int lineNum = 1;
int token = tokenizer.nextToken();
while (token != StreamTokenizer.TT_EOF) {
String tokenID;
if ( token == StreamTokenizer.TT_WORD ) {
tokenID = tokenizer.sval;
}
else if ( token == '\'' ) {
tokenID = "'"+tokenizer.sval+"'";
}
else {
ErrorManager.error(ErrorManager.MSG_TOKENS_FILE_SYNTAX_ERROR,
vocabName+CodeGenerator.VOCAB_FILE_EXTENSION,
Utils.integer(lineNum));
while ( tokenizer.nextToken() != StreamTokenizer.TT_EOL ) {;}
token = tokenizer.nextToken();
continue;
}
token = tokenizer.nextToken();
if ( token != '=' ) {
ErrorManager.error(ErrorManager.MSG_TOKENS_FILE_SYNTAX_ERROR,
vocabName+CodeGenerator.VOCAB_FILE_EXTENSION,
Utils.integer(lineNum));
while ( tokenizer.nextToken() != StreamTokenizer.TT_EOL ) {;}
token = tokenizer.nextToken();
continue;
}
token = tokenizer.nextToken(); // skip '='
if ( token != StreamTokenizer.TT_NUMBER ) {
ErrorManager.error(ErrorManager.MSG_TOKENS_FILE_SYNTAX_ERROR,
vocabName+CodeGenerator.VOCAB_FILE_EXTENSION,
Utils.integer(lineNum));
while ( tokenizer.nextToken() != StreamTokenizer.TT_EOL ) {;}
token = tokenizer.nextToken();
continue;
}
int tokenType = (int)tokenizer.nval;
token = tokenizer.nextToken();
//System.out.println("import "+tokenID+"="+tokenType);
composite.maxTokenType = Math.max(composite.maxTokenType,tokenType);
defineToken(tokenID, tokenType);
lineNum++;
if ( token != StreamTokenizer.TT_EOL ) {
ErrorManager.error(ErrorManager.MSG_TOKENS_FILE_SYNTAX_ERROR,
vocabName+CodeGenerator.VOCAB_FILE_EXTENSION,
Utils.integer(lineNum));
while ( tokenizer.nextToken() != StreamTokenizer.TT_EOL ) {;}
token = tokenizer.nextToken();
continue;
}
token = tokenizer.nextToken(); // skip newline
}
br.close();
}
catch (FileNotFoundException fnfe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_FIND_TOKENS_FILE,
fullFile);
}
catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_ERROR_READING_TOKENS_FILE,
fullFile,
ioe);
}
catch (Exception e) {
ErrorManager.error(ErrorManager.MSG_ERROR_READING_TOKENS_FILE,
fullFile,
e);
}
return composite.maxTokenType;
}
/** Given a token type, get a meaningful name for it such as the ID
* or string literal. If this is a lexer and the ttype is in the
* char vocabulary, compute an ANTLR-valid (possibly escaped) char literal.
*/
public String getTokenDisplayName(int ttype) {
String tokenName = null;
int index=0;
// inside any target's char range and is lexer grammar?
if ( this.type==LEXER &&
ttype >= Label.MIN_CHAR_VALUE && ttype <= Label.MAX_CHAR_VALUE )
{
return getANTLRCharLiteralForChar(ttype);
}
// faux label?
else if ( ttype<0 ) {
tokenName = (String)composite.typeToTokenList.get(Label.NUM_FAUX_LABELS+ttype);
}
else {
// compute index in typeToTokenList for ttype
index = ttype-1; // normalize to 0..n-1
index += Label.NUM_FAUX_LABELS; // jump over faux tokens
if ( index<composite.typeToTokenList.size() ) {
tokenName = (String)composite.typeToTokenList.get(index);
if ( tokenName!=null &&
tokenName.startsWith(AUTO_GENERATED_TOKEN_NAME_PREFIX) )
{
tokenName = composite.typeToStringLiteralList.get(ttype);
}
}
else {
tokenName = String.valueOf(ttype);
}
}
//System.out.println("getTokenDisplayName ttype="+ttype+", index="+index+", name="+tokenName);
return tokenName;
}
/** Get the list of ANTLR String literals */
public Set<String> getStringLiterals() {
return composite.stringLiteralToTypeMap.keySet();
}
public String getGrammarTypeString() {
return grammarTypeToString[type];
}
public int getGrammarMaxLookahead() {
if ( global_k>=0 ) {
return global_k;
}
Object k = getOption("k");
if ( k==null ) {
global_k = 0;
}
else if (k instanceof Integer) {
Integer kI = (Integer)k;
global_k = kI.intValue();
}
else {
// must be String "*"
if ( k.equals("*") ) { // this the default anyway
global_k = 0;
}
}
return global_k;
}
/** Save the option key/value pair and process it; return the key
* or null if invalid option.
*/
public String setOption(String key, Object value, antlr.Token optionsStartToken) {
if ( legalOption(key) ) {
ErrorManager.grammarError(ErrorManager.MSG_ILLEGAL_OPTION,
this,
optionsStartToken,
key);
return null;
}
if ( !optionIsValid(key, value) ) {
return null;
}
if ( key.equals("backtrack") && value.toString().equals("true") ) {
composite.getRootGrammar().atLeastOneBacktrackOption = true;
}
if ( options==null ) {
options = new HashMap();
}
options.put(key, value);
return key;
}
public boolean legalOption(String key) {
switch ( type ) {
case LEXER :
return !legalLexerOptions.contains(key);
case PARSER :
return !legalParserOptions.contains(key);
case TREE_PARSER :
return !legalTreeParserOptions.contains(key);
default :
return !legalParserOptions.contains(key);
}
}
public void setOptions(Map options, antlr.Token optionsStartToken) {
if ( options==null ) {
this.options = null;
return;
}
Set keys = options.keySet();
for (Iterator it = keys.iterator(); it.hasNext();) {
String optionName = (String) it.next();
Object optionValue = options.get(optionName);
String stored=setOption(optionName, optionValue, optionsStartToken);
if ( stored==null ) {
it.remove();
}
}
}
public Object getOption(String key) {
return composite.getOption(key);
}
public Object getLocallyDefinedOption(String key) {
Object value = null;
if ( options!=null ) {
value = options.get(key);
}
if ( value==null ) {
value = defaultOptions.get(key);
}
return value;
}
public Object getBlockOption(GrammarAST blockAST, String key) {
String v = (String)blockAST.getBlockOption(key);
if ( v!=null ) {
return v;
}
if ( type==Grammar.LEXER ) {
return defaultLexerBlockOptions.get(key);
}
return defaultBlockOptions.get(key);
}
public int getUserMaxLookahead(int decision) {
int user_k = 0;
GrammarAST blockAST = nfa.grammar.getDecisionBlockAST(decision);
Object k = blockAST.getBlockOption("k");
if ( k==null ) {
user_k = nfa.grammar.getGrammarMaxLookahead();
return user_k;
}
if (k instanceof Integer) {
Integer kI = (Integer)k;
user_k = kI.intValue();
}
else {
// must be String "*"
if ( k.equals("*") ) {
user_k = 0;
}
}
return user_k;
}
public boolean getAutoBacktrackMode(int decision) {
NFAState decisionNFAStartState = getDecisionNFAStartState(decision);
String autoBacktrack =
(String)getBlockOption(decisionNFAStartState.associatedASTNode, "backtrack");
if ( autoBacktrack==null ) {
autoBacktrack = (String)nfa.grammar.getOption("backtrack");
}
return autoBacktrack!=null&&autoBacktrack.equals("true");
}
public boolean optionIsValid(String key, Object value) {
return true;
}
public boolean buildAST() {
String outputType = (String)getOption("output");
if ( outputType!=null ) {
return outputType.toString().equals("AST");
}
return false;
}
public boolean rewriteMode() {
Object outputType = getOption("rewrite");
if ( outputType!=null ) {
return outputType.toString().equals("true");
}
return false;
}
public boolean isBuiltFromString() {
return builtFromString;
}
public boolean buildTemplate() {
String outputType = (String)getOption("output");
if ( outputType!=null ) {
return outputType.toString().equals("template");
}
return false;
}
public Collection<Rule> getRules() {
return nameToRuleMap.values();
}
/** Get the set of Rules that need to have manual delegations
* like "void rule() { importedGrammar.rule(); }"
*
* If this grammar is master, get list of all rule definitions from all
* delegate grammars. Only master has complete interface from combined
* grammars...we will generated delegates as helper objects.
*
* Composite grammars that are not the root/master do not have complete
* interfaces. It is not my intention that people use subcomposites.
* Only the outermost grammar should be used from outside code. The
* other grammar components are specifically generated to work only
* with the master/root.
*
* delegatedRules = imported - overridden
*/
public Set<Rule> getDelegatedRules() {
return composite.getDelegatedRules(this);
}
/** Get set of all rules imported from all delegate grammars even if
* indirectly delegated.
*/
public Set<Rule> getAllImportedRules() {
return composite.getAllImportedRules(this);
}
/** Get list of all delegates from all grammars directly or indirectly
* imported into this grammar.
*/
public List<Grammar> getDelegates() {
return composite.getDelegates(this);
}
public List<String> getDelegateNames() {
// compute delegates:{Grammar g | return g.name;}
List<String> names = new ArrayList<String>();
List<Grammar> delegates = composite.getDelegates(this);
if ( delegates!=null ) {
for (Grammar g : delegates) {
names.add(g.name);
}
}
return names;
}
public List<Grammar> getDirectDelegates() {
return composite.getDirectDelegates(this);
}
/** Get delegates below direct delegates */
public List<Grammar> getIndirectDelegates() {
return composite.getIndirectDelegates(this);
}
/** Get list of all delegators. This amounts to the grammars on the path
* to the root of the delegation tree.
*/
public List<Grammar> getDelegators() {
return composite.getDelegators(this);
}
/** Who's my direct parent grammar? */
public Grammar getDelegator() {
return composite.getDelegator(this);
}
public Set<Rule> getDelegatedRuleReferences() {
return delegatedRuleReferences;
}
public boolean getGrammarIsRoot() {
return composite.delegateGrammarTreeRoot.grammar == this;
}
public void setRuleAST(String ruleName, GrammarAST t) {
Rule r = getLocallyDefinedRule(ruleName);
if ( r!=null ) {
r.tree = t;
r.EORNode = t.getLastChild();
}
}
public NFAState getRuleStartState(String ruleName) {
return getRuleStartState(null, ruleName);
}
public NFAState getRuleStartState(String scopeName, String ruleName) {
Rule r = getRule(scopeName, ruleName);
if ( r!=null ) {
//System.out.println("getRuleStartState("+scopeName+", "+ruleName+")="+r.startState);
return r.startState;
}
//System.out.println("getRuleStartState("+scopeName+", "+ruleName+")=null");
return null;
}
public String getRuleModifier(String ruleName) {
Rule r = getRule(ruleName);
if ( r!=null ) {
return r.modifier;
}
return null;
}
public NFAState getRuleStopState(String ruleName) {
Rule r = getRule(ruleName);
if ( r!=null ) {
return r.stopState;
}
return null;
}
public int assignDecisionNumber(NFAState state) {
decisionCount++;
state.setDecisionNumber(decisionCount);
return decisionCount;
}
protected Decision getDecision(int decision) {
int index = decision-1;
if ( index >= indexToDecision.size() ) {
return null;
}
Decision d = (Decision)indexToDecision.get(index);
return d;
}
protected Decision createDecision(int decision) {
int index = decision-1;
if ( index < indexToDecision.size() ) {
return getDecision(decision); // don't recreate
}
Decision d = new Decision();
d.decision = decision;
indexToDecision.setSize(getNumberOfDecisions());
indexToDecision.set(index, d);
return d;
}
public List getDecisionNFAStartStateList() {
List states = new ArrayList(100);
for (int d = 0; d < indexToDecision.size(); d++) {
Decision dec = (Decision) indexToDecision.get(d);
states.add(dec.startState);
}
return states;
}
public NFAState getDecisionNFAStartState(int decision) {
Decision d = getDecision(decision);
if ( d==null ) {
return null;
}
return d.startState;
}
public DFA getLookaheadDFA(int decision) {
Decision d = getDecision(decision);
if ( d==null ) {
return null;
}
return d.dfa;
}
public GrammarAST getDecisionBlockAST(int decision) {
Decision d = getDecision(decision);
if ( d==null ) {
return null;
}
return d.blockAST;
}
/** returns a list of column numbers for all decisions
* on a particular line so ANTLRWorks choose the decision
* depending on the location of the cursor (otherwise,
* ANTLRWorks has to give the *exact* location which
* is not easy from the user point of view).
*
* This is not particularly fast as it walks entire line:col->DFA map
* looking for a prefix of "line:".
*/
public List getLookaheadDFAColumnsForLineInFile(int line) {
String prefix = line+":";
List columns = new ArrayList();
for(Iterator iter = lineColumnToLookaheadDFAMap.keySet().iterator();
iter.hasNext(); ) {
String key = (String)iter.next();
if(key.startsWith(prefix)) {
columns.add(Integer.valueOf(key.substring(prefix.length())));
}
}
return columns;
}
/** Useful for ANTLRWorks to map position in file to the DFA for display */
public DFA getLookaheadDFAFromPositionInFile(int line, int col) {
return (DFA)lineColumnToLookaheadDFAMap.get(
new StringBuffer().append(line + ":").append(col).toString());
}
public Map getLineColumnToLookaheadDFAMap() {
return lineColumnToLookaheadDFAMap;
}
/*
public void setDecisionOptions(int decision, Map options) {
Decision d = createDecision(decision);
d.options = options;
}
public void setDecisionOption(int decision, String name, Object value) {
Decision d = getDecision(decision);
if ( d!=null ) {
if ( d.options==null ) {
d.options = new HashMap();
}
d.options.put(name,value);
}
}
public Map getDecisionOptions(int decision) {
Decision d = getDecision(decision);
if ( d==null ) {
return null;
}
return d.options;
}
*/
public int getNumberOfDecisions() {
return decisionCount;
}
public int getNumberOfCyclicDecisions() {
int n = 0;
for (int i=1; i<=getNumberOfDecisions(); i++) {
Decision d = getDecision(i);
if ( d.dfa!=null && d.dfa.isCyclic() ) {
n++;
}
}
return n;
}
/** Set the lookahead DFA for a particular decision. This means
* that the appropriate AST node must updated to have the new lookahead
* DFA. This method could be used to properly set the DFAs without
* using the createLookaheadDFAs() method. You could do this
*
* Grammar g = new Grammar("...");
* g.setLookahead(1, dfa1);
* g.setLookahead(2, dfa2);
* ...
*/
public void setLookaheadDFA(int decision, DFA lookaheadDFA) {
Decision d = createDecision(decision);
d.dfa = lookaheadDFA;
GrammarAST ast = d.startState.associatedASTNode;
ast.setLookaheadDFA(lookaheadDFA);
}
public void setDecisionNFA(int decision, NFAState state) {
Decision d = createDecision(decision);
d.startState = state;
}
public void setDecisionBlockAST(int decision, GrammarAST blockAST) {
//System.out.println("setDecisionBlockAST("+decision+", "+blockAST.token);
Decision d = createDecision(decision);
d.blockAST = blockAST;
}
public boolean allDecisionDFAHaveBeenCreated() {
return allDecisionDFACreated;
}
/** How many token types have been allocated so far? */
public int getMaxTokenType() {
return composite.maxTokenType;
}
/** What is the max char value possible for this grammar's target? Use
* unicode max if no target defined.
*/
public int getMaxCharValue() {
if ( generator!=null ) {
return generator.target.getMaxCharValue(generator);
}
else {
return Label.MAX_CHAR_VALUE;
}
}
/** Return a set of all possible token or char types for this grammar */
public IntSet getTokenTypes() {
if ( type==LEXER ) {
return getAllCharValues();
}
return IntervalSet.of(Label.MIN_TOKEN_TYPE, getMaxTokenType());
}
/** If there is a char vocabulary, use it; else return min to max char
* as defined by the target. If no target, use max unicode char value.
*/
public IntSet getAllCharValues() {
if ( charVocabulary!=null ) {
return charVocabulary;
}
IntSet allChar = IntervalSet.of(Label.MIN_CHAR_VALUE, getMaxCharValue());
return allChar;
}
/** Return a string representing the escaped char for code c. E.g., If c
* has value 0x100, you will get "\u0100". ASCII gets the usual
* char (non-hex) representation. Control characters are spit out
* as unicode. While this is specially set up for returning Java strings,
* it can be used by any language target that has the same syntax. :)
*
* 11/26/2005: I changed this to use double quotes, consistent with antlr.g
* 12/09/2005: I changed so everything is single quotes
*/
public static String getANTLRCharLiteralForChar(int c) {
if ( c<Label.MIN_CHAR_VALUE ) {
ErrorManager.internalError("invalid char value "+c);
return "'<INVALID>'";
}
if ( c<ANTLRLiteralCharValueEscape.length && ANTLRLiteralCharValueEscape[c]!=null ) {
return '\''+ANTLRLiteralCharValueEscape[c]+'\'';
}
if ( Character.UnicodeBlock.of((char)c)==Character.UnicodeBlock.BASIC_LATIN &&
!Character.isISOControl((char)c) ) {
if ( c=='\\' ) {
return "'\\\\'";
}
if ( c=='\'') {
return "'\\''";
}
return '\''+Character.toString((char)c)+'\'';
}
// turn on the bit above max "\uFFFF" value so that we pad with zeros
// then only take last 4 digits
String hex = Integer.toHexString(c|0x10000).toUpperCase().substring(1,5);
String unicodeStr = "'\\u"+hex+"'";
return unicodeStr;
}
/** For lexer grammars, return everything in unicode not in set.
* For parser and tree grammars, return everything in token space
* from MIN_TOKEN_TYPE to last valid token type or char value.
*/
public IntSet complement(IntSet set) {
//System.out.println("complement "+set.toString(this));
//System.out.println("vocabulary "+getTokenTypes().toString(this));
IntSet c = set.complement(getTokenTypes());
//System.out.println("result="+c.toString(this));
return c;
}
public IntSet complement(int atom) {
return complement(IntervalSet.of(atom));
}
/** Given set tree like ( SET A B ), check that A and B
* are both valid sets themselves, else we must tree like a BLOCK
*/
public boolean isValidSet(TreeToNFAConverter nfabuilder, GrammarAST t) {
boolean valid = true;
try {
//System.out.println("parse BLOCK as set tree: "+t.toStringTree());
nfabuilder.testBlockAsSet(t);
}
catch (RecognitionException re) {
// The rule did not parse as a set, return null; ignore exception
valid = false;
}
//System.out.println("valid? "+valid);
return valid;
}
/** Get the set equivalent (if any) of the indicated rule from this
* grammar. Mostly used in the lexer to do ~T for some fragment rule
* T. If the rule AST has a SET use that. If the rule is a single char
* convert it to a set and return. If rule is not a simple set (w/o actions)
* then return null.
* Rules have AST form:
*
* ^( RULE ID modifier ARG RET SCOPE block EOR )
*/
public IntSet getSetFromRule(TreeToNFAConverter nfabuilder, String ruleName)
throws RecognitionException
{
Rule r = getRule(ruleName);
if ( r==null ) {
return null;
}
IntSet elements = null;
//System.out.println("parsed tree: "+r.tree.toStringTree());
elements = nfabuilder.setRule(r.tree);
//System.out.println("elements="+elements);
return elements;
}
/** Decisions are linked together with transition(1). Count how
* many there are. This is here rather than in NFAState because
* a grammar decides how NFAs are put together to form a decision.
*/
public int getNumberOfAltsForDecisionNFA(NFAState decisionState) {
if ( decisionState==null ) {
return 0;
}
int n = 1;
NFAState p = decisionState;
while ( p.transition[1] !=null ) {
n++;
p = (NFAState)p.transition[1].target;
}
return n;
}
/** Get the ith alternative (1..n) from a decision; return null when
* an invalid alt is requested. I must count in to find the right
* alternative number. For (A|B), you get NFA structure (roughly):
*
* o->o-A->o
* |
* o->o-B->o
*
* This routine returns the leftmost state for each alt. So alt=1, returns
* the upperleft most state in this structure.
*/
public NFAState getNFAStateForAltOfDecision(NFAState decisionState, int alt) {
if ( decisionState==null || alt<=0 ) {
return null;
}
int n = 1;
NFAState p = decisionState;
while ( p!=null ) {
if ( n==alt ) {
return p;
}
n++;
Transition next = p.transition[1];
p = null;
if ( next!=null ) {
p = (NFAState)next.target;
}
}
return null;
}
/*
public void computeRuleFOLLOWSets() {
if ( getNumberOfDecisions()==0 ) {
createNFAs();
}
for (Iterator it = getRules().iterator(); it.hasNext();) {
Rule r = (Rule)it.next();
if ( r.isSynPred ) {
continue;
}
LookaheadSet s = ll1Analyzer.FOLLOW(r);
System.out.println("FOLLOW("+r.name+")="+s);
}
}
*/
public LookaheadSet FIRST(NFAState s) {
return ll1Analyzer.FIRST(s);
}
public LookaheadSet LOOK(NFAState s) {
return ll1Analyzer.LOOK(s);
}
public void setCodeGenerator(CodeGenerator generator) {
this.generator = generator;
}
public CodeGenerator getCodeGenerator() {
return generator;
}
public GrammarAST getGrammarTree() {
return grammarTree;
}
public Tool getTool() {
return tool;
}
public void setTool(Tool tool) {
this.tool = tool;
}
/** given a token type and the text of the literal, come up with a
* decent token type label. For now it's just T<type>. Actually,
* if there is an aliased name from tokens like PLUS='+', use it.
*/
public String computeTokenNameFromLiteral(int tokenType, String literal) {
return AUTO_GENERATED_TOKEN_NAME_PREFIX +tokenType;
}
public String toString() {
return grammarTreeToString(grammarTree);
}
public String grammarTreeToString(GrammarAST t) {
return grammarTreeToString(t, true);
}
public String grammarTreeToString(GrammarAST t, boolean showActions) {
String s = null;
try {
s = t.getLine()+":"+t.getColumn()+": ";
s += new ANTLRTreePrinter().toString((AST)t, this, showActions);
}
catch (Exception e) {
s = "<invalid or missing tree structure>";
}
return s;
}
public void printGrammar(PrintStream output) {
ANTLRTreePrinter printer = new ANTLRTreePrinter();
printer.setASTNodeClass("org.antlr.tool.GrammarAST");
try {
String g = printer.toString(grammarTree, this, false);
output.println(g);
}
catch (RecognitionException re) {
ErrorManager.error(ErrorManager.MSG_SYNTAX_ERROR,re);
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/serialize.g 0000644 0001750 0001750 00000012727 11256465230 023105 0 ustar twerner twerner header {
/*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import java.util.*;
import org.antlr.analysis.*;
import org.antlr.misc.*;
import java.io.*;
}
class SerializerWalker extends TreeParser;
options {
importVocab = ANTLR;
ASTLabelType = "GrammarAST";
codeGenBitsetTestThreshold=999;
}
{
public void reportError(RecognitionException ex) {
Token token = null;
if ( ex instanceof MismatchedTokenException ) {
token = ((MismatchedTokenException)ex).token;
}
else if ( ex instanceof NoViableAltException ) {
token = ((NoViableAltException)ex).token;
}
ErrorManager.syntaxError(
ErrorManager.MSG_SYNTAX_ERROR,
grammar,
token,
"serialize: "+ex.toString(),
ex);
}
protected Grammar grammar;
protected String currentRuleName;
protected GrammarSerializer out;
}
grammar[GrammarSerializer out]
{
this.out = out;
}
: #( LEXER_GRAMMAR grammarSpec[#grammar.getType()] )
| #( PARSER_GRAMMAR grammarSpec[#grammar.getType()] )
| #( TREE_GRAMMAR grammarSpec[#grammar.getType()] )
| #( COMBINED_GRAMMAR grammarSpec[#grammar.getType()] )
;
grammarSpec[int gtokentype]
: id:ID {out.grammar(gtokentype, #id.getText());}
(cmt:DOC_COMMENT)?
(optionsSpec)?
(delegateGrammars)?
(tokensSpec)?
(attrScope)*
(AMPERSAND)* // skip actions
rules
;
attrScope
: #( "scope" ID ACTION )
;
optionsSpec
: #( OPTIONS (option)+ )
;
option
: #( ASSIGN ID optionValue )
;
optionValue
: id:ID
| s:STRING_LITERAL
| c:CHAR_LITERAL
| i:INT
;
charSet
: #( CHARSET charSetElement )
;
charSetElement
: c:CHAR_LITERAL
| #( OR c1:CHAR_LITERAL c2:CHAR_LITERAL )
| #( RANGE c3:CHAR_LITERAL c4:CHAR_LITERAL )
;
delegateGrammars
: #( "import"
( #(ASSIGN ID ID)
| ID
)+
)
;
tokensSpec
: #( TOKENS ( tokenSpec )+ )
;
tokenSpec
: t:TOKEN_REF
| #( ASSIGN
t2:TOKEN_REF
( s:STRING_LITERAL
| c:CHAR_LITERAL
)
)
;
rules
: ( rule )+
;
rule
: #( RULE id:ID {out.rule(#id.getText());}
(m:modifier)?
(ARG (ARG_ACTION)?)
(RET (ARG_ACTION)?)
(optionsSpec)?
(ruleScopeSpec)?
(AMPERSAND)*
b:block
(exceptionGroup)?
EOR {out.endRule();}
)
;
modifier
: "protected"
| "public"
| "private"
| "fragment"
;
ruleScopeSpec
: #( "scope" (ACTION)? ( ID )* )
;
block
: #( BLOCK {out.block(#BLOCK.getNumberOfChildrenWithType(ALT));}
(optionsSpec)?
( alternative rewrite )+
EOB
)
;
alternative
: #( ALT {out.alt(#alternative);} (element)+ EOA {out.endAlt();} )
;
exceptionGroup
: ( exceptionHandler )+ (finallyClause)?
| finallyClause
;
exceptionHandler
: #("catch" ARG_ACTION ACTION)
;
finallyClause
: #("finally" ACTION)
;
rewrite
: ( #( REWRITE (SEMPRED)? (ALT|TEMPLATE|ACTION|ETC) ) )*
;
element
: #(ROOT element)
| #(BANG element)
| atom
| #(NOT {out.not();} element)
| #(RANGE atom atom)
| #(CHAR_RANGE {out.range();} atom atom)
| #(ASSIGN ID element)
| #(PLUS_ASSIGN ID element)
| ebnf
| tree
| #( SYNPRED block )
| FORCED_ACTION
| ACTION
| SEMPRED
| SYN_SEMPRED
| BACKTRACK_SEMPRED
| GATED_SEMPRED
| EPSILON
;
ebnf: block
| #( OPTIONAL block )
| #( CLOSURE block )
| #( POSITIVE_CLOSURE block )
;
tree: #(TREE_BEGIN element (element)* )
;
atom
: #( rr:RULE_REF (rarg:ARG_ACTION)? ) {out.ruleRef(#rr);}
| #( t:TOKEN_REF (targ:ARG_ACTION )? ) {out.token(#t);}
| c:CHAR_LITERAL {out.charLiteral(#c);}
| s:STRING_LITERAL {out.charLiteral(#s);}
| WILDCARD {out.wildcard(#WILDCARD);}
| #(DOT ID atom) // scope override on rule
;
ast_suffix
: ROOT
| BANG
;
antlr-3.2/tool/src/main/java/org/antlr/tool/CompositeGrammar.java 0000644 0001750 0001750 00000044400 11256465230 025053 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/package org.antlr.tool;
import antlr.RecognitionException;
import org.antlr.analysis.Label;
import org.antlr.analysis.NFAState;
import org.antlr.misc.Utils;
import java.util.*;
import org.antlr.grammar.v2.AssignTokenTypesWalker;
/** A tree of component (delegate) grammars.
*
* Rules defined in delegates are "inherited" like multi-inheritance
* so you can override them. All token types must be consistent across
* rules from all delegate grammars, so they must be stored here in one
* central place.
*
* We have to start out assuming a composite grammar situation as we can't
* look into the grammar files a priori to see if there is a delegate
* statement. Because of this, and to avoid duplicating token type tracking
* in each grammar, even single noncomposite grammars use one of these objects
* to track token types.
*/
public class CompositeGrammar {
public static final int MIN_RULE_INDEX = 1;
public CompositeGrammarTree delegateGrammarTreeRoot;
/** Used during getRuleReferenceClosure to detect computation cycles */
protected Set<NFAState> refClosureBusy = new HashSet<NFAState>();
/** Used to assign state numbers; all grammars in composite share common
* NFA space. This NFA tracks state numbers number to state mapping.
*/
public int stateCounter = 0;
/** The NFA states in the NFA built from rules across grammars in composite.
* Maps state number to NFAState object.
* This is a Vector instead of a List because I need to be able to grow
* this properly. After talking to Josh Bloch, Collections guy at Sun,
* I decided this was easiest solution.
*/
protected Vector<NFAState> numberToStateList = new Vector<NFAState>(1000);
/** Token names and literal tokens like "void" are uniquely indexed.
* with -1 implying EOF. Characters are different; they go from
* -1 (EOF) to \uFFFE. For example, 0 could be a binary byte you
* want to lexer. Labels of DFA/NFA transitions can be both tokens
* and characters. I use negative numbers for bookkeeping labels
* like EPSILON. Char/String literals and token types overlap in the same
* space, however.
*/
protected int maxTokenType = Label.MIN_TOKEN_TYPE-1;
/** Map token like ID (but not literals like "while") to its token type */
public Map tokenIDToTypeMap = new HashMap();
/** Map token literals like "while" to its token type. It may be that
* WHILE="while"=35, in which case both tokenIDToTypeMap and this
* field will have entries both mapped to 35.
*/
public Map<String, Integer> stringLiteralToTypeMap = new HashMap<String, Integer>();
/** Reverse index for stringLiteralToTypeMap */
public Vector<String> typeToStringLiteralList = new Vector<String>();
/** Map a token type to its token name.
* Must subtract MIN_TOKEN_TYPE from index.
*/
public Vector<String> typeToTokenList = new Vector<String>();
/** If combined or lexer grammar, track the rules.
* Track lexer rules so we can warn about undefined tokens.
* This is combined set of lexer rules from all lexer grammars
* seen in all imports.
*/
protected Set<String> lexerRules = new HashSet<String>();
/** Rules are uniquely labeled from 1..n among all grammars */
protected int ruleIndex = MIN_RULE_INDEX;
/** Map a rule index to its name; use a Vector on purpose as new
* collections stuff won't let me setSize and make it grow. :(
* I need a specific guaranteed index, which the Collections stuff
* won't let me have.
*/
protected Vector<Rule> ruleIndexToRuleList = new Vector<Rule>();
public boolean watchNFAConversion = false;
protected void initTokenSymbolTables() {
// the faux token types take first NUM_FAUX_LABELS positions
// then we must have room for the predefined runtime token types
// like DOWN/UP used for tree parsing.
typeToTokenList.setSize(Label.NUM_FAUX_LABELS+Label.MIN_TOKEN_TYPE-1);
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.INVALID, "<INVALID>");
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.EOT, "<EOT>");
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.SEMPRED, "<SEMPRED>");
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.SET, "<SET>");
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.EPSILON, Label.EPSILON_STR);
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.EOF, "EOF");
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.EOR_TOKEN_TYPE-1, "<EOR>");
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.DOWN-1, "DOWN");
typeToTokenList.set(Label.NUM_FAUX_LABELS+Label.UP-1, "UP");
tokenIDToTypeMap.put("<INVALID>", Utils.integer(Label.INVALID));
tokenIDToTypeMap.put("<EOT>", Utils.integer(Label.EOT));
tokenIDToTypeMap.put("<SEMPRED>", Utils.integer(Label.SEMPRED));
tokenIDToTypeMap.put("<SET>", Utils.integer(Label.SET));
tokenIDToTypeMap.put("<EPSILON>", Utils.integer(Label.EPSILON));
tokenIDToTypeMap.put("EOF", Utils.integer(Label.EOF));
tokenIDToTypeMap.put("<EOR>", Utils.integer(Label.EOR_TOKEN_TYPE));
tokenIDToTypeMap.put("DOWN", Utils.integer(Label.DOWN));
tokenIDToTypeMap.put("UP", Utils.integer(Label.UP));
}
public CompositeGrammar() {
initTokenSymbolTables();
}
public CompositeGrammar(Grammar g) {
this();
setDelegationRoot(g);
}
public void setDelegationRoot(Grammar root) {
delegateGrammarTreeRoot = new CompositeGrammarTree(root);
root.compositeTreeNode = delegateGrammarTreeRoot;
}
public Rule getRule(String ruleName) {
return delegateGrammarTreeRoot.getRule(ruleName);
}
public Object getOption(String key) {
return delegateGrammarTreeRoot.getOption(key);
}
/** Add delegate grammar as child of delegator */
public void addGrammar(Grammar delegator, Grammar delegate) {
if ( delegator.compositeTreeNode==null ) {
delegator.compositeTreeNode = new CompositeGrammarTree(delegator);
}
delegator.compositeTreeNode.addChild(new CompositeGrammarTree(delegate));
/*// find delegator in tree so we can add a child to it
CompositeGrammarTree t = delegateGrammarTreeRoot.findNode(delegator);
t.addChild();
*/
// make sure new grammar shares this composite
delegate.composite = this;
}
/** Get parent of this grammar */
public Grammar getDelegator(Grammar g) {
CompositeGrammarTree me = delegateGrammarTreeRoot.findNode(g);
if ( me==null ) {
return null; // not found
}
if ( me.parent!=null ) {
return me.parent.grammar;
}
return null;
}
/** Get list of all delegates from all grammars in the delegate subtree of g.
* The grammars are in delegation tree preorder. Don't include g itself
* in list as it is not a delegate of itself.
*/
public List<Grammar> getDelegates(Grammar g) {
CompositeGrammarTree t = delegateGrammarTreeRoot.findNode(g);
if ( t==null ) {
return null; // no delegates
}
List<Grammar> grammars = t.getPostOrderedGrammarList();
grammars.remove(grammars.size()-1); // remove g (last one)
return grammars;
}
public List<Grammar> getDirectDelegates(Grammar g) {
CompositeGrammarTree t = delegateGrammarTreeRoot.findNode(g);
List<CompositeGrammarTree> children = t.children;
if ( children==null ) {
return null;
}
List<Grammar> grammars = new ArrayList();
for (int i = 0; children!=null && i < children.size(); i++) {
CompositeGrammarTree child = (CompositeGrammarTree) children.get(i);
grammars.add(child.grammar);
}
return grammars;
}
/** Get delegates below direct delegates of g */
public List<Grammar> getIndirectDelegates(Grammar g) {
List<Grammar> direct = getDirectDelegates(g);
List<Grammar> delegates = getDelegates(g);
delegates.removeAll(direct);
return delegates;
}
/** Return list of delegate grammars from root down to g.
* Order is root, ..., g.parent. (g not included).
*/
public List<Grammar> getDelegators(Grammar g) {
if ( g==delegateGrammarTreeRoot.grammar ) {
return null;
}
List<Grammar> grammars = new ArrayList();
CompositeGrammarTree t = delegateGrammarTreeRoot.findNode(g);
// walk backwards to root, collecting grammars
CompositeGrammarTree p = t.parent;
while ( p!=null ) {
grammars.add(0, p.grammar); // add to head so in order later
p = p.parent;
}
return grammars;
}
/** Get set of rules for grammar g that need to have manual delegation
* methods. This is the list of rules collected from all direct/indirect
* delegates minus rules overridden in grammar g.
*
* This returns null except for the delegate root because it is the only
* one that has to have a complete grammar rule interface. The delegates
* should not be instantiated directly for use as parsers (you can create
* them to pass to the root parser's ctor as arguments).
*/
public Set<Rule> getDelegatedRules(Grammar g) {
if ( g!=delegateGrammarTreeRoot.grammar ) {
return null;
}
Set<Rule> rules = getAllImportedRules(g);
for (Iterator it = rules.iterator(); it.hasNext();) {
Rule r = (Rule) it.next();
Rule localRule = g.getLocallyDefinedRule(r.name);
// if locally defined or it's not local but synpred, don't make
// a delegation method
if ( localRule!=null || r.isSynPred ) {
it.remove(); // kill overridden rules
}
}
return rules;
}
/** Get all rule definitions from all direct/indirect delegate grammars
* of g.
*/
public Set<Rule> getAllImportedRules(Grammar g) {
Set<String> ruleNames = new HashSet();
Set<Rule> rules = new HashSet();
CompositeGrammarTree subtreeRoot = delegateGrammarTreeRoot.findNode(g);
List<Grammar> grammars = subtreeRoot.getPostOrderedGrammarList();
// walk all grammars
for (int i = 0; i < grammars.size(); i++) {
Grammar delegate = (org.antlr.tool.Grammar) grammars.get(i);
// for each rule in delegate, add to rules if no rule with that
// name as been seen. (can't use removeAll; wrong hashcode/equals on Rule)
for (Iterator it = delegate.getRules().iterator(); it.hasNext();) {
Rule r = (Rule)it.next();
if ( !ruleNames.contains(r.name) ) {
ruleNames.add(r.name); // track that we've seen this
rules.add(r);
}
}
}
return rules;
}
public Grammar getRootGrammar() {
if ( delegateGrammarTreeRoot==null ) {
return null;
}
return delegateGrammarTreeRoot.grammar;
}
public Grammar getGrammar(String grammarName) {
CompositeGrammarTree t = delegateGrammarTreeRoot.findNode(grammarName);
if ( t!=null ) {
return t.grammar;
}
return null;
}
// NFA spans multiple grammars, must handle here
public int getNewNFAStateNumber() {
return stateCounter++;
}
public void addState(NFAState state) {
numberToStateList.setSize(state.stateNumber+1); // make sure we have room
numberToStateList.set(state.stateNumber, state);
}
public NFAState getState(int s) {
return (NFAState)numberToStateList.get(s);
}
public void assignTokenTypes() throws antlr.RecognitionException {
// ASSIGN TOKEN TYPES for all delegates (same walker)
//System.out.println("### assign types");
AssignTokenTypesWalker ttypesWalker = new AssignTokenTypesBehavior();
ttypesWalker.setASTNodeClass("org.antlr.tool.GrammarAST");
List<Grammar> grammars = delegateGrammarTreeRoot.getPostOrderedGrammarList();
for (int i = 0; grammars!=null && i < grammars.size(); i++) {
Grammar g = (Grammar)grammars.get(i);
try {
//System.out.println(" walking "+g.name);
ttypesWalker.grammar(g.getGrammarTree(), g);
}
catch (RecognitionException re) {
ErrorManager.error(ErrorManager.MSG_BAD_AST_STRUCTURE,
re);
}
}
// the walker has filled literals, tokens, and alias tables.
// now tell it to define them in the root grammar
ttypesWalker.defineTokens(delegateGrammarTreeRoot.grammar);
}
public void defineGrammarSymbols() {
delegateGrammarTreeRoot.trimLexerImportsIntoCombined();
List<Grammar> grammars = delegateGrammarTreeRoot.getPostOrderedGrammarList();
for (int i = 0; grammars!=null && i < grammars.size(); i++) {
Grammar g = (Grammar)grammars.get(i);
g.defineGrammarSymbols();
}
for (int i = 0; grammars!=null && i < grammars.size(); i++) {
Grammar g = (Grammar)grammars.get(i);
g.checkNameSpaceAndActions();
}
minimizeRuleSet();
}
public void createNFAs() {
if ( ErrorManager.doNotAttemptAnalysis() ) {
return;
}
List<Grammar> grammars = delegateGrammarTreeRoot.getPostOrderedGrammarList();
List<String> names = new ArrayList<String>();
for (int i = 0; i < grammars.size(); i++) {
Grammar g = (Grammar) grammars.get(i);
names.add(g.name);
}
//System.out.println("### createNFAs for composite; grammars: "+names);
for (int i = 0; grammars!=null && i < grammars.size(); i++) {
Grammar g = (Grammar)grammars.get(i);
g.createRuleStartAndStopNFAStates();
}
for (int i = 0; grammars!=null && i < grammars.size(); i++) {
Grammar g = (Grammar)grammars.get(i);
g.buildNFA();
}
}
public void minimizeRuleSet() {
Set<String> ruleDefs = new HashSet<String>();
_minimizeRuleSet(ruleDefs, delegateGrammarTreeRoot);
}
public void _minimizeRuleSet(Set<String> ruleDefs,
CompositeGrammarTree p) {
Set<String> localRuleDefs = new HashSet<String>();
Set<String> overrides = new HashSet<String>();
// compute set of non-overridden rules for this delegate
for (Rule r : p.grammar.getRules()) {
if ( !ruleDefs.contains(r.name) ) {
localRuleDefs.add(r.name);
}
else if ( !r.name.equals(Grammar.ARTIFICIAL_TOKENS_RULENAME) ) {
// record any overridden rule 'cept tokens rule
overrides.add(r.name);
}
}
//System.out.println("rule defs for "+p.grammar.name+": "+localRuleDefs);
//System.out.println("overridden rule for "+p.grammar.name+": "+overrides);
p.grammar.overriddenRules = overrides;
// make set of all rules defined thus far walking delegation tree.
// the same rule in two delegates resolves in favor of first found
// in tree therefore second must not be included
ruleDefs.addAll(localRuleDefs);
// pass larger set of defined rules to delegates
if ( p.children!=null ) {
for (CompositeGrammarTree delegate : p.children) {
_minimizeRuleSet(ruleDefs, delegate);
}
}
}
/*
public void minimizeRuleSet() {
Set<Rule> refs = _minimizeRuleSet(delegateGrammarTreeRoot);
System.out.println("all rule refs: "+refs);
}
public Set<Rule> _minimizeRuleSet(CompositeGrammarTree p) {
Set<Rule> refs = new HashSet<Rule>();
for (GrammarAST refAST : p.grammar.ruleRefs) {
System.out.println("ref "+refAST.getText()+": "+refAST.NFAStartState+
" enclosing rule: "+refAST.NFAStartState.enclosingRule+
" invoking rule: "+((NFAState)refAST.NFAStartState.transition[0].target).enclosingRule);
refs.add(((NFAState)refAST.NFAStartState.transition[0].target).enclosingRule);
}
if ( p.children!=null ) {
for (CompositeGrammarTree delegate : p.children) {
Set<Rule> delegateRuleRefs = _minimizeRuleSet(delegate);
refs.addAll(delegateRuleRefs);
}
}
return refs;
}
*/
/*
public void oldminimizeRuleSet() {
// first walk to remove all overridden rules
Set<String> ruleDefs = new HashSet<String>();
Set<String> ruleRefs = new HashSet<String>();
for (GrammarAST refAST : delegateGrammarTreeRoot.grammar.ruleRefs) {
String rname = refAST.getText();
ruleRefs.add(rname);
}
_minimizeRuleSet(ruleDefs,
ruleRefs,
delegateGrammarTreeRoot);
System.out.println("overall rule defs: "+ruleDefs);
}
public void _minimizeRuleSet(Set<String> ruleDefs,
Set<String> ruleRefs,
CompositeGrammarTree p) {
Set<String> localRuleDefs = new HashSet<String>();
for (Rule r : p.grammar.getRules()) {
if ( !ruleDefs.contains(r.name) ) {
localRuleDefs.add(r.name);
ruleDefs.add(r.name);
}
}
System.out.println("rule defs for "+p.grammar.name+": "+localRuleDefs);
// remove locally-defined rules not in ref set
// find intersection of local rules and references from delegator
// that is set of rules needed by delegator
Set<String> localRuleDefsSatisfyingRefsFromBelow = new HashSet<String>();
for (String r : ruleRefs) {
if ( localRuleDefs.contains(r) ) {
localRuleDefsSatisfyingRefsFromBelow.add(r);
}
}
// now get list of refs from localRuleDefsSatisfyingRefsFromBelow.
// Those rules are also allowed in this delegate
for (GrammarAST refAST : p.grammar.ruleRefs) {
if ( localRuleDefsSatisfyingRefsFromBelow.contains(refAST.enclosingRuleName) ) {
// found rule ref within needed rule
}
}
// remove rule refs not in the new rule def set
// walk all children, adding rules not already defined
if ( p.children!=null ) {
for (CompositeGrammarTree delegate : p.children) {
_minimizeRuleSet(ruleDefs, ruleRefs, delegate);
}
}
}
*/
/*
public void trackNFAStatesThatHaveLabeledEdge(Label label,
NFAState stateWithLabeledEdge)
{
Set<NFAState> states = typeToNFAStatesWithEdgeOfTypeMap.get(label);
if ( states==null ) {
states = new HashSet<NFAState>();
typeToNFAStatesWithEdgeOfTypeMap.put(label, states);
}
states.add(stateWithLabeledEdge);
}
public Map<Label, Set<NFAState>> getTypeToNFAStatesWithEdgeOfTypeMap() {
return typeToNFAStatesWithEdgeOfTypeMap;
}
public Set<NFAState> getStatesWithEdge(Label label) {
return typeToNFAStatesWithEdgeOfTypeMap.get(label);
}
*/
}
antlr-3.2/tool/src/main/java/org/antlr/tool/Rule.java 0000644 0001750 0001750 00000047031 11256465230 022514 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import antlr.CommonToken;
import org.antlr.analysis.NFAState;
import org.antlr.analysis.LookaheadSet;
import org.antlr.codegen.CodeGenerator;
import java.util.*;
import org.antlr.grammar.v2.ANTLRParser;
/** Combine the info associated with a rule. */
public class Rule {
public String name;
public int index;
public String modifier;
public NFAState startState;
public NFAState stopState;
/** This rule's options */
protected Map options;
public static final Set legalOptions =
new HashSet() {
{
add("k"); add("greedy"); add("memoize");
add("backtrack"); add("strategy");
}
};
/** The AST representing the whole rule */
public GrammarAST tree;
/** To which grammar does this belong? */
public Grammar grammar;
/** For convenience, track the argument def AST action node if any */
public GrammarAST argActionAST;
public GrammarAST EORNode;
/** The set of all tokens reachable from the start state w/o leaving
* via the accept state. If it reaches the accept state, FIRST
* includes EOR_TOKEN_TYPE.
*/
public LookaheadSet FIRST;
/** The return values of a rule and predefined rule attributes */
public AttributeScope returnScope;
public AttributeScope parameterScope;
/** the attributes defined with "scope {...}" inside a rule */
public AttributeScope ruleScope;
/** A list of scope names (String) used by this rule */
public List useScopes;
/** A list of all LabelElementPair attached to tokens like id=ID */
public LinkedHashMap tokenLabels;
/** A list of all LabelElementPair attached to tokens like x=. in tree grammar */
public LinkedHashMap wildcardTreeLabels;
/** A list of all LabelElementPair attached to tokens like x+=. in tree grammar */
public LinkedHashMap wildcardTreeListLabels;
/** A list of all LabelElementPair attached to single char literals like x='a' */
public LinkedHashMap charLabels;
/** A list of all LabelElementPair attached to rule references like f=field */
public LinkedHashMap ruleLabels;
/** A list of all Token list LabelElementPair like ids+=ID */
public LinkedHashMap tokenListLabels;
/** A list of all rule ref list LabelElementPair like ids+=expr */
public LinkedHashMap ruleListLabels;
/** All labels go in here (plus being split per the above lists) to
* catch dup label and label type mismatches.
*/
protected Map<String, Grammar.LabelElementPair> labelNameSpace =
new HashMap<String, Grammar.LabelElementPair>();
/** Map a name to an action for this rule. Currently init is only
* one we use, but we can add more in future.
* The code generator will use this to fill holes in the rule template.
* I track the AST node for the action in case I need the line number
* for errors. A better name is probably namedActions, but I don't
* want everyone to have to change their code gen templates now.
*/
protected Map<String, GrammarAST> actions =
new HashMap<String, GrammarAST>();
/** Track all executable actions other than named actions like @init.
* Also tracks exception handlers, predicates, and rewrite rewrites.
* We need to examine these actions before code generation so
* that we can detect refs to $rule.attr etc...
*/
protected List<GrammarAST> inlineActions = new ArrayList<GrammarAST>();
public int numberOfAlts;
/** Each alt has a Map<tokenRefName,List<tokenRefAST>>; range 1..numberOfAlts.
* So, if there are 3 ID refs in a rule's alt number 2, you'll have
* altToTokenRef[2].get("ID").size()==3. This is used to see if $ID is ok.
* There must be only one ID reference in the alt for $ID to be ok in
* an action--must be unique.
*
* This also tracks '+' and "int" literal token references
* (if not in LEXER).
*
* Rewrite rules force tracking of all tokens.
*/
protected Map<String, List<GrammarAST>>[] altToTokenRefMap;
/** Each alt has a Map<ruleRefName,List<ruleRefAST>>; range 1..numberOfAlts
* So, if there are 3 expr refs in a rule's alt number 2, you'll have
* altToRuleRef[2].get("expr").size()==3. This is used to see if $expr is ok.
* There must be only one expr reference in the alt for $expr to be ok in
* an action--must be unique.
*
* Rewrite rules force tracking of all rule result ASTs. 1..n
*/
protected Map<String, List<GrammarAST>>[] altToRuleRefMap;
/** Track which alts have rewrite rules associated with them. 1..n */
protected boolean[] altsWithRewrites;
/** Do not generate start, stop etc... in a return value struct unless
* somebody references $r.start somewhere.
*/
public boolean referencedPredefinedRuleAttributes = false;
public boolean isSynPred = false;
public boolean imported = false;
public Rule(Grammar grammar,
String ruleName,
int ruleIndex,
int numberOfAlts)
{
this.name = ruleName;
this.index = ruleIndex;
this.numberOfAlts = numberOfAlts;
this.grammar = grammar;
altToTokenRefMap = new Map[numberOfAlts+1];
altToRuleRefMap = new Map[numberOfAlts+1];
altsWithRewrites = new boolean[numberOfAlts+1];
for (int alt=1; alt<=numberOfAlts; alt++) {
altToTokenRefMap[alt] = new HashMap<String, List<GrammarAST>>();
altToRuleRefMap[alt] = new HashMap<String, List<GrammarAST>>();
}
}
public void defineLabel(antlr.Token label, GrammarAST elementRef, int type) {
Grammar.LabelElementPair pair = grammar.new LabelElementPair(label,elementRef);
pair.type = type;
labelNameSpace.put(label.getText(), pair);
switch ( type ) {
case Grammar.TOKEN_LABEL :
if ( tokenLabels==null ) tokenLabels = new LinkedHashMap();
tokenLabels.put(label.getText(), pair);
break;
case Grammar.WILDCARD_TREE_LABEL :
if ( wildcardTreeLabels==null ) wildcardTreeLabels = new LinkedHashMap();
wildcardTreeLabels.put(label.getText(), pair);
break;
case Grammar.WILDCARD_TREE_LIST_LABEL :
if ( wildcardTreeListLabels==null ) wildcardTreeListLabels = new LinkedHashMap();
wildcardTreeListLabels.put(label.getText(), pair);
break;
case Grammar.RULE_LABEL :
if ( ruleLabels==null ) ruleLabels = new LinkedHashMap();
ruleLabels.put(label.getText(), pair);
break;
case Grammar.TOKEN_LIST_LABEL :
if ( tokenListLabels==null ) tokenListLabels = new LinkedHashMap();
tokenListLabels.put(label.getText(), pair);
break;
case Grammar.RULE_LIST_LABEL :
if ( ruleListLabels==null ) ruleListLabels = new LinkedHashMap();
ruleListLabels.put(label.getText(), pair);
break;
case Grammar.CHAR_LABEL :
if ( charLabels==null ) charLabels = new LinkedHashMap();
charLabels.put(label.getText(), pair);
break;
}
}
public Grammar.LabelElementPair getLabel(String name) {
return (Grammar.LabelElementPair)labelNameSpace.get(name);
}
public Grammar.LabelElementPair getTokenLabel(String name) {
Grammar.LabelElementPair pair = null;
if ( tokenLabels!=null ) {
return (Grammar.LabelElementPair)tokenLabels.get(name);
}
return pair;
}
public Map getRuleLabels() {
return ruleLabels;
}
public Map getRuleListLabels() {
return ruleListLabels;
}
public Grammar.LabelElementPair getRuleLabel(String name) {
Grammar.LabelElementPair pair = null;
if ( ruleLabels!=null ) {
return (Grammar.LabelElementPair)ruleLabels.get(name);
}
return pair;
}
public Grammar.LabelElementPair getTokenListLabel(String name) {
Grammar.LabelElementPair pair = null;
if ( tokenListLabels!=null ) {
return (Grammar.LabelElementPair)tokenListLabels.get(name);
}
return pair;
}
public Grammar.LabelElementPair getRuleListLabel(String name) {
Grammar.LabelElementPair pair = null;
if ( ruleListLabels!=null ) {
return (Grammar.LabelElementPair)ruleListLabels.get(name);
}
return pair;
}
/** Track a token ID or literal like '+' and "void" as having been referenced
* somewhere within the alts (not rewrite sections) of a rule.
*
* This differs from Grammar.altReferencesTokenID(), which tracks all
* token IDs to check for token IDs without corresponding lexer rules.
*/
public void trackTokenReferenceInAlt(GrammarAST refAST, int outerAltNum) {
List refs = (List)altToTokenRefMap[outerAltNum].get(refAST.getText());
if ( refs==null ) {
refs = new ArrayList();
altToTokenRefMap[outerAltNum].put(refAST.getText(), refs);
}
refs.add(refAST);
}
public List getTokenRefsInAlt(String ref, int outerAltNum) {
if ( altToTokenRefMap[outerAltNum]!=null ) {
List tokenRefASTs = (List)altToTokenRefMap[outerAltNum].get(ref);
return tokenRefASTs;
}
return null;
}
public void trackRuleReferenceInAlt(GrammarAST refAST, int outerAltNum) {
List refs = (List)altToRuleRefMap[outerAltNum].get(refAST.getText());
if ( refs==null ) {
refs = new ArrayList();
altToRuleRefMap[outerAltNum].put(refAST.getText(), refs);
}
refs.add(refAST);
}
public List getRuleRefsInAlt(String ref, int outerAltNum) {
if ( altToRuleRefMap[outerAltNum]!=null ) {
List ruleRefASTs = (List)altToRuleRefMap[outerAltNum].get(ref);
return ruleRefASTs;
}
return null;
}
public Set getTokenRefsInAlt(int altNum) {
return altToTokenRefMap[altNum].keySet();
}
/** For use with rewrite rules, we must track all tokens matched on the
* left-hand-side; so we need Lists. This is a unique list of all
* token types for which the rule needs a list of tokens. This
* is called from the rule template not directly by the code generator.
*/
public Set getAllTokenRefsInAltsWithRewrites() {
String output = (String)grammar.getOption("output");
Set tokens = new HashSet();
if ( output==null || !output.equals("AST") ) {
// return nothing if not generating trees; i.e., don't do for templates
return tokens;
}
for (int i = 1; i <= numberOfAlts; i++) {
if ( altsWithRewrites[i] ) {
Map m = altToTokenRefMap[i];
Set s = m.keySet();
for (Iterator it = s.iterator(); it.hasNext();) {
// convert token name like ID to ID, "void" to 31
String tokenName = (String) it.next();
int ttype = grammar.getTokenType(tokenName);
String label = grammar.generator.getTokenTypeAsTargetLabel(ttype);
tokens.add(label);
}
}
}
return tokens;
}
public Set getRuleRefsInAlt(int outerAltNum) {
return altToRuleRefMap[outerAltNum].keySet();
}
/** For use with rewrite rules, we must track all rule AST results on the
* left-hand-side; so we need Lists. This is a unique list of all
* rule results for which the rule needs a list of results.
*/
public Set getAllRuleRefsInAltsWithRewrites() {
Set rules = new HashSet();
for (int i = 1; i <= numberOfAlts; i++) {
if ( altsWithRewrites[i] ) {
Map m = altToRuleRefMap[i];
rules.addAll(m.keySet());
}
}
return rules;
}
public List<GrammarAST> getInlineActions() {
return inlineActions;
}
public boolean hasRewrite(int i) {
if ( i >= altsWithRewrites.length ) {
ErrorManager.internalError("alt "+i+" exceeds number of "+name+
"'s alts ("+altsWithRewrites.length+")");
return false;
}
return altsWithRewrites[i];
}
/** Track which rules have rewrite rules. Pass in the ALT node
* for the alt so we can check for problems when output=template,
* rewrite=true, and grammar type is tree parser.
*/
public void trackAltsWithRewrites(GrammarAST altAST, int outerAltNum) {
if ( grammar.type==Grammar.TREE_PARSER &&
grammar.buildTemplate() &&
grammar.getOption("rewrite")!=null &&
grammar.getOption("rewrite").equals("true")
)
{
GrammarAST firstElementAST = (GrammarAST)altAST.getFirstChild();
grammar.sanity.ensureAltIsSimpleNodeOrTree(altAST,
firstElementAST,
outerAltNum);
}
altsWithRewrites[outerAltNum] = true;
}
/** Return the scope containing name */
public AttributeScope getAttributeScope(String name) {
AttributeScope scope = getLocalAttributeScope(name);
if ( scope!=null ) {
return scope;
}
if ( ruleScope!=null && ruleScope.getAttribute(name)!=null ) {
scope = ruleScope;
}
return scope;
}
/** Get the arg, return value, or predefined property for this rule */
public AttributeScope getLocalAttributeScope(String name) {
AttributeScope scope = null;
if ( returnScope!=null && returnScope.getAttribute(name)!=null ) {
scope = returnScope;
}
else if ( parameterScope!=null && parameterScope.getAttribute(name)!=null ) {
scope = parameterScope;
}
else {
AttributeScope rulePropertiesScope =
RuleLabelScope.grammarTypeToRulePropertiesScope[grammar.type];
if ( rulePropertiesScope.getAttribute(name)!=null ) {
scope = rulePropertiesScope;
}
}
return scope;
}
/** For references to tokens rather than by label such as $ID, we
* need to get the existing label for the ID ref or create a new
* one.
*/
public String getElementLabel(String refdSymbol,
int outerAltNum,
CodeGenerator generator)
{
GrammarAST uniqueRefAST;
if ( grammar.type != Grammar.LEXER &&
Character.isUpperCase(refdSymbol.charAt(0)) )
{
// symbol is a token
List tokenRefs = getTokenRefsInAlt(refdSymbol, outerAltNum);
uniqueRefAST = (GrammarAST)tokenRefs.get(0);
}
else {
// symbol is a rule
List ruleRefs = getRuleRefsInAlt(refdSymbol, outerAltNum);
uniqueRefAST = (GrammarAST)ruleRefs.get(0);
}
if ( uniqueRefAST.code==null ) {
// no code? must not have gen'd yet; forward ref
return null;
}
String labelName = null;
String existingLabelName =
(String)uniqueRefAST.code.getAttribute("label");
// reuse any label or list label if it exists
if ( existingLabelName!=null ) {
labelName = existingLabelName;
}
else {
// else create new label
labelName = generator.createUniqueLabel(refdSymbol);
CommonToken label = new CommonToken(ANTLRParser.ID, labelName);
if ( grammar.type != Grammar.LEXER &&
Character.isUpperCase(refdSymbol.charAt(0)) )
{
grammar.defineTokenRefLabel(name, label, uniqueRefAST);
}
else {
grammar.defineRuleRefLabel(name, label, uniqueRefAST);
}
uniqueRefAST.code.setAttribute("label", labelName);
}
return labelName;
}
/** If a rule has no user-defined return values and nobody references
* it's start/stop (predefined attributes), then there is no need to
* define a struct; otherwise for now we assume a struct. A rule also
* has multiple return values if you are building trees or templates.
*/
public boolean getHasMultipleReturnValues() {
return
referencedPredefinedRuleAttributes || grammar.buildAST() ||
grammar.buildTemplate() ||
(returnScope!=null && returnScope.attributes.size()>1);
}
public boolean getHasSingleReturnValue() {
return
!(referencedPredefinedRuleAttributes || grammar.buildAST() ||
grammar.buildTemplate()) &&
(returnScope!=null && returnScope.attributes.size()==1);
}
public boolean getHasReturnValue() {
return
referencedPredefinedRuleAttributes || grammar.buildAST() ||
grammar.buildTemplate() ||
(returnScope!=null && returnScope.attributes.size()>0);
}
public String getSingleValueReturnType() {
if ( returnScope!=null && returnScope.attributes.size()==1 ) {
Collection retvalAttrs = returnScope.attributes.values();
Object[] javaSucks = retvalAttrs.toArray();
return ((Attribute)javaSucks[0]).type;
}
return null;
}
public String getSingleValueReturnName() {
if ( returnScope!=null && returnScope.attributes.size()==1 ) {
Collection retvalAttrs = returnScope.attributes.values();
Object[] javaSucks = retvalAttrs.toArray();
return ((Attribute)javaSucks[0]).name;
}
return null;
}
/** Given @scope::name {action} define it for this grammar. Later,
* the code generator will ask for the actions table.
*/
public void defineNamedAction(GrammarAST ampersandAST,
GrammarAST nameAST,
GrammarAST actionAST)
{
//System.out.println("rule @"+nameAST.getText()+"{"+actionAST.getText()+"}");
String actionName = nameAST.getText();
GrammarAST a = (GrammarAST)actions.get(actionName);
if ( a!=null ) {
ErrorManager.grammarError(
ErrorManager.MSG_ACTION_REDEFINITION,grammar,
nameAST.getToken(),nameAST.getText());
}
else {
actions.put(actionName,actionAST);
}
}
public void trackInlineAction(GrammarAST actionAST) {
inlineActions.add(actionAST);
}
public Map<String, GrammarAST> getActions() {
return actions;
}
public void setActions(Map<String, GrammarAST> actions) {
this.actions = actions;
}
/** Save the option key/value pair and process it; return the key
* or null if invalid option.
*/
public String setOption(String key, Object value, antlr.Token optionsStartToken) {
if ( !legalOptions.contains(key) ) {
ErrorManager.grammarError(ErrorManager.MSG_ILLEGAL_OPTION,
grammar,
optionsStartToken,
key);
return null;
}
if ( options==null ) {
options = new HashMap();
}
if ( key.equals("memoize") && value.toString().equals("true") ) {
grammar.atLeastOneRuleMemoizes = true;
}
if ( key.equals("backtrack") && value.toString().equals("true") ) {
grammar.composite.getRootGrammar().atLeastOneBacktrackOption = true;
}
if ( key.equals("k") ) {
grammar.numberOfManualLookaheadOptions++;
}
options.put(key, value);
return key;
}
public void setOptions(Map options, antlr.Token optionsStartToken) {
if ( options==null ) {
this.options = null;
return;
}
Set keys = options.keySet();
for (Iterator it = keys.iterator(); it.hasNext();) {
String optionName = (String) it.next();
Object optionValue = options.get(optionName);
String stored=setOption(optionName, optionValue, optionsStartToken);
if ( stored==null ) {
it.remove();
}
}
}
/** Used during grammar imports to see if sets of rules intersect... This
* method and hashCode use the String name as the key for Rule objects.
public boolean equals(Object other) {
return this.name.equals(((Rule)other).name);
}
*/
/** Used during grammar imports to see if sets of rules intersect...
public int hashCode() {
return name.hashCode();
}
* */
public String toString() { // used for testing
return "["+grammar.name+"."+name+",index="+index+",line="+tree.getToken().getLine()+"]";
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/RecursionOverflowMessage.java 0000644 0001750 0001750 00000005657 11256465230 026617 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.analysis.*;
import antlr.Token;
import java.util.*;
/** Indicates recursion overflow. A DFA state tried add an NFA configuration
* with NFA state p that was mentioned in its stack context too many times.
*/
public class RecursionOverflowMessage extends Message {
public DecisionProbe probe;
public DFAState sampleBadState;
public int alt;
public Collection targetRules;
public Collection callSiteStates;
public RecursionOverflowMessage(DecisionProbe probe,
DFAState sampleBadState,
int alt,
Collection targetRules,
Collection callSiteStates)
{
super(ErrorManager.MSG_RECURSION_OVERLOW);
this.probe = probe;
this.sampleBadState = sampleBadState;
this.alt = alt;
this.targetRules = targetRules;
this.callSiteStates = callSiteStates;
}
public String toString() {
GrammarAST decisionASTNode = probe.dfa.getDecisionASTNode();
line = decisionASTNode.getLine();
column = decisionASTNode.getColumn();
String fileName = probe.dfa.nfa.grammar.getFileName();
if ( fileName!=null ) {
file = fileName;
}
StringTemplate st = getMessageTemplate();
st.setAttribute("targetRules", targetRules);
st.setAttribute("alt", alt);
st.setAttribute("callSiteStates", callSiteStates);
List labels =
probe.getSampleNonDeterministicInputSequence(sampleBadState);
String input = probe.getInputSequenceDisplay(labels);
st.setAttribute("input", input);
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/NFAFactory.java 0000644 0001750 0001750 00000067376 11256465230 023557 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.*;
import org.antlr.misc.IntSet;
import org.antlr.misc.IntervalSet;
import java.util.Iterator;
import java.util.List;
import java.util.ArrayList;
import antlr.Token;
/** Routines to construct StateClusters from EBNF grammar constructs.
* No optimization is done to remove unnecessary epsilon edges.
*
* TODO: add an optimization that reduces number of states and transitions
* will help with speed of conversion and make it easier to view NFA. For
* example, o-A->o-->o-B->o should be o-A->o-B->o
*/
public class NFAFactory {
/** This factory is attached to a specifc NFA that it is building.
* The NFA will be filled up with states and transitions.
*/
NFA nfa = null;
public Rule getCurrentRule() {
return currentRule;
}
public void setCurrentRule(Rule currentRule) {
this.currentRule = currentRule;
}
Rule currentRule = null;
public NFAFactory(NFA nfa) {
nfa.setFactory(this);
this.nfa = nfa;
}
public NFAState newState() {
NFAState n = new NFAState(nfa);
int state = nfa.getNewNFAStateNumber();
n.stateNumber = state;
nfa.addState(n);
n.enclosingRule = currentRule;
return n;
}
/** Optimize an alternative (list of grammar elements).
*
* Walk the chain of elements (which can be complicated loop blocks...)
* and throw away any epsilon transitions used to link up simple elements.
*
* This only removes 195 states from the java.g's NFA, but every little
* bit helps. Perhaps I can improve in the future.
*/
public void optimizeAlternative(StateCluster alt) {
NFAState s = alt.left;
while ( s!=alt.right ) {
// if it's a block element, jump over it and continue
if ( s.endOfBlockStateNumber!=State.INVALID_STATE_NUMBER ) {
s = nfa.getState(s.endOfBlockStateNumber);
continue;
}
Transition t = s.transition[0];
if ( t instanceof RuleClosureTransition ) {
s = ((RuleClosureTransition) t).followState;
continue;
}
if ( t.label.isEpsilon() && !t.label.isAction() && s.getNumberOfTransitions()==1 ) {
// bypass epsilon transition and point to what the epsilon's
// target points to unless that epsilon transition points to
// a block or loop etc.. Also don't collapse epsilons that
// point at the last node of the alt. Don't collapse action edges
NFAState epsilonTarget = (NFAState)t.target;
if ( epsilonTarget.endOfBlockStateNumber==State.INVALID_STATE_NUMBER &&
epsilonTarget.transition[0] !=null )
{
s.setTransition0(epsilonTarget.transition[0]);
/*
System.out.println("### opt "+s.stateNumber+"->"+
epsilonTarget.transition(0).target.stateNumber);
*/
}
}
s = (NFAState)t.target;
}
}
/** From label A build Graph o-A->o */
public StateCluster build_Atom(int label, GrammarAST associatedAST) {
NFAState left = newState();
NFAState right = newState();
left.associatedASTNode = associatedAST;
right.associatedASTNode = associatedAST;
transitionBetweenStates(left, right, label);
StateCluster g = new StateCluster(left, right);
return g;
}
public StateCluster build_Atom(GrammarAST atomAST) {
int tokenType = nfa.grammar.getTokenType(atomAST.getText());
return build_Atom(tokenType, atomAST);
}
/** From set build single edge graph o->o-set->o. To conform to
* what an alt block looks like, must have extra state on left.
*/
public StateCluster build_Set(IntSet set, GrammarAST associatedAST) {
NFAState left = newState();
NFAState right = newState();
left.associatedASTNode = associatedAST;
right.associatedASTNode = associatedAST;
Label label = new Label(set);
Transition e = new Transition(label,right);
left.addTransition(e);
StateCluster g = new StateCluster(left, right);
return g;
}
/** Can only complement block of simple alts; can complement build_Set()
* result, that is. Get set and complement, replace old with complement.
public StateCluster build_AlternativeBlockComplement(StateCluster blk) {
State s0 = blk.left;
IntSet set = getCollapsedBlockAsSet(s0);
if ( set!=null ) {
// if set is available, then structure known and blk is a set
set = nfa.grammar.complement(set);
Label label = s0.transition(0).target.transition(0).label;
label.setSet(set);
}
return blk;
}
*/
public StateCluster build_Range(int a, int b) {
NFAState left = newState();
NFAState right = newState();
Label label = new Label(IntervalSet.of(a, b));
Transition e = new Transition(label,right);
left.addTransition(e);
StateCluster g = new StateCluster(left, right);
return g;
}
/** From char 'c' build StateCluster o-intValue(c)->o
*/
public StateCluster build_CharLiteralAtom(GrammarAST charLiteralAST) {
int c = Grammar.getCharValueFromGrammarCharLiteral(charLiteralAST.getText());
return build_Atom(c, charLiteralAST);
}
/** From char 'c' build StateCluster o-intValue(c)->o
* can include unicode spec likes '\u0024' later. Accepts
* actual unicode 16-bit now, of course, by default.
* TODO not supplemental char clean!
*/
public StateCluster build_CharRange(String a, String b) {
int from = Grammar.getCharValueFromGrammarCharLiteral(a);
int to = Grammar.getCharValueFromGrammarCharLiteral(b);
return build_Range(from, to);
}
/** For a non-lexer, just build a simple token reference atom.
* For a lexer, a string is a sequence of char to match. That is,
* "fog" is treated as 'f' 'o' 'g' not as a single transition in
* the DFA. Machine== o-'f'->o-'o'->o-'g'->o and has n+1 states
* for n characters.
*/
public StateCluster build_StringLiteralAtom(GrammarAST stringLiteralAST) {
if ( nfa.grammar.type==Grammar.LEXER ) {
StringBuffer chars =
Grammar.getUnescapedStringFromGrammarStringLiteral(stringLiteralAST.getText());
NFAState first = newState();
NFAState last = null;
NFAState prev = first;
for (int i=0; i<chars.length(); i++) {
int c = chars.charAt(i);
NFAState next = newState();
transitionBetweenStates(prev, next, c);
prev = last = next;
}
return new StateCluster(first, last);
}
// a simple token reference in non-Lexers
int tokenType = nfa.grammar.getTokenType(stringLiteralAST.getText());
return build_Atom(tokenType, stringLiteralAST);
}
/** For reference to rule r, build
*
* o-e->(r) o
*
* where (r) is the start of rule r and the trailing o is not linked
* to from rule ref state directly (it's done thru the transition(0)
* RuleClosureTransition.
*
* If the rule r is just a list of tokens, it's block will be just
* a set on an edge o->o->o-set->o->o->o, could inline it rather than doing
* the rule reference, but i'm not doing this yet as I'm not sure
* it would help much in the NFA->DFA construction.
*
* TODO add to codegen: collapse alt blks that are sets into single matchSet
*/
public StateCluster build_RuleRef(Rule refDef, NFAState ruleStart) {
//System.out.println("building ref to rule "+nfa.grammar.name+"."+refDef.name);
NFAState left = newState();
// left.setDescription("ref to "+ruleStart.getDescription());
NFAState right = newState();
// right.setDescription("NFAState following ref to "+ruleStart.getDescription());
Transition e = new RuleClosureTransition(refDef,ruleStart,right);
left.addTransition(e);
StateCluster g = new StateCluster(left, right);
return g;
}
/** From an empty alternative build StateCluster o-e->o */
public StateCluster build_Epsilon() {
NFAState left = newState();
NFAState right = newState();
transitionBetweenStates(left, right, Label.EPSILON);
StateCluster g = new StateCluster(left, right);
return g;
}
/** Build what amounts to an epsilon transition with a semantic
* predicate action. The pred is a pointer into the AST of
* the SEMPRED token.
*/
public StateCluster build_SemanticPredicate(GrammarAST pred) {
// don't count syn preds
if ( !pred.getText().toUpperCase()
.startsWith(Grammar.SYNPRED_RULE_PREFIX.toUpperCase()) )
{
nfa.grammar.numberOfSemanticPredicates++;
}
NFAState left = newState();
NFAState right = newState();
Transition e = new Transition(new PredicateLabel(pred), right);
left.addTransition(e);
StateCluster g = new StateCluster(left, right);
return g;
}
/** Build what amounts to an epsilon transition with an action.
* The action goes into NFA though it is ignored during analysis.
* It slows things down a bit, but I must ignore predicates after
* having seen an action (5-5-2008).
*/
public StateCluster build_Action(GrammarAST action) {
NFAState left = newState();
NFAState right = newState();
Transition e = new Transition(new ActionLabel(action), right);
left.addTransition(e);
return new StateCluster(left, right);
}
/** add an EOF transition to any rule end NFAState that points to nothing
* (i.e., for all those rules not invoked by another rule). These
* are start symbols then.
*
* Return the number of grammar entry points; i.e., how many rules are
* not invoked by another rule (they can only be invoked from outside).
* These are the start rules.
*/
public int build_EOFStates(List rules) {
int numberUnInvokedRules = 0;
for (Iterator iterator = rules.iterator(); iterator.hasNext();) {
Rule r = (Rule) iterator.next();
NFAState endNFAState = r.stopState;
// Is this rule a start symbol? (no follow links)
if ( endNFAState.transition[0] ==null ) {
// if so, then don't let algorithm fall off the end of
// the rule, make it hit EOF/EOT.
build_EOFState(endNFAState);
// track how many rules have been invoked by another rule
numberUnInvokedRules++;
}
}
return numberUnInvokedRules;
}
/** set up an NFA NFAState that will yield eof tokens or,
* in the case of a lexer grammar, an EOT token when the conversion
* hits the end of a rule.
*/
private void build_EOFState(NFAState endNFAState) {
NFAState end = newState();
int label = Label.EOF;
if ( nfa.grammar.type==Grammar.LEXER ) {
label = Label.EOT;
end.setEOTTargetState(true);
}
/*
System.out.println("build "+nfa.grammar.getTokenDisplayName(label)+
" loop on end of state "+endNFAState.getDescription()+
" to state "+end.stateNumber);
*/
Transition toEnd = new Transition(label, end);
endNFAState.addTransition(toEnd);
}
/** From A B build A-e->B (that is, build an epsilon arc from right
* of A to left of B).
*
* As a convenience, return B if A is null or return A if B is null.
*/
public StateCluster build_AB(StateCluster A, StateCluster B) {
if ( A==null ) {
return B;
}
if ( B==null ) {
return A;
}
transitionBetweenStates(A.right, B.left, Label.EPSILON);
StateCluster g = new StateCluster(A.left, B.right);
return g;
}
/** From a set ('a'|'b') build
*
* o->o-'a'..'b'->o->o (last NFAState is blockEndNFAState pointed to by all alts)
*/
public StateCluster build_AlternativeBlockFromSet(StateCluster set) {
if ( set==null ) {
return null;
}
// single alt, no decision, just return only alt state cluster
NFAState startOfAlt = newState(); // must have this no matter what
transitionBetweenStates(startOfAlt, set.left, Label.EPSILON);
return new StateCluster(startOfAlt,set.right);
}
/** From A|B|..|Z alternative block build
*
* o->o-A->o->o (last NFAState is blockEndNFAState pointed to by all alts)
* | ^
* o->o-B->o--|
* | |
* ... |
* | |
* o->o-Z->o--|
*
* So every alternative gets begin NFAState connected by epsilon
* and every alt right side points at a block end NFAState. There is a
* new NFAState in the NFAState in the StateCluster for each alt plus one for the
* end NFAState.
*
* Special case: only one alternative: don't make a block with alt
* begin/end.
*
* Special case: if just a list of tokens/chars/sets, then collapse
* to a single edge'd o-set->o graph.
*
* Set alt number (1..n) in the left-Transition NFAState.
*/
public StateCluster build_AlternativeBlock(List alternativeStateClusters)
{
StateCluster result = null;
if ( alternativeStateClusters==null || alternativeStateClusters.size()==0 ) {
return null;
}
// single alt case
if ( alternativeStateClusters.size()==1 ) {
// single alt, no decision, just return only alt state cluster
StateCluster g = (StateCluster)alternativeStateClusters.get(0);
NFAState startOfAlt = newState(); // must have this no matter what
transitionBetweenStates(startOfAlt, g.left, Label.EPSILON);
//System.out.println("### opt saved start/stop end in (...)");
return new StateCluster(startOfAlt,g.right);
}
// even if we can collapse for lookahead purposes, we will still
// need to predict the alts of this subrule in case there are actions
// etc... This is the decision that is pointed to from the AST node
// (always)
NFAState prevAlternative = null; // tracks prev so we can link to next alt
NFAState firstAlt = null;
NFAState blockEndNFAState = newState();
blockEndNFAState.setDescription("end block");
int altNum = 1;
for (Iterator iter = alternativeStateClusters.iterator(); iter.hasNext();) {
StateCluster g = (StateCluster) iter.next();
// add begin NFAState for this alt connected by epsilon
NFAState left = newState();
left.setDescription("alt "+altNum+" of ()");
transitionBetweenStates(left, g.left, Label.EPSILON);
transitionBetweenStates(g.right, blockEndNFAState, Label.EPSILON);
// Are we the first alternative?
if ( firstAlt==null ) {
firstAlt = left; // track extreme left node of StateCluster
}
else {
// if not first alternative, must link to this alt from previous
transitionBetweenStates(prevAlternative, left, Label.EPSILON);
}
prevAlternative = left;
altNum++;
}
// return StateCluster pointing representing entire block
// Points to first alt NFAState on left, block end on right
result = new StateCluster(firstAlt, blockEndNFAState);
firstAlt.decisionStateType = NFAState.BLOCK_START;
// set EOB markers for Jean
firstAlt.endOfBlockStateNumber = blockEndNFAState.stateNumber;
return result;
}
/** From (A)? build either:
*
* o--A->o
* | ^
* o---->|
*
* or, if A is a block, just add an empty alt to the end of the block
*/
public StateCluster build_Aoptional(StateCluster A) {
StateCluster g = null;
int n = nfa.grammar.getNumberOfAltsForDecisionNFA(A.left);
if ( n==1 ) {
// no decision, just wrap in an optional path
//NFAState decisionState = newState();
NFAState decisionState = A.left; // resuse left edge
decisionState.setDescription("only alt of ()? block");
NFAState emptyAlt = newState();
emptyAlt.setDescription("epsilon path of ()? block");
NFAState blockEndNFAState = null;
blockEndNFAState = newState();
transitionBetweenStates(A.right, blockEndNFAState, Label.EPSILON);
blockEndNFAState.setDescription("end ()? block");
//transitionBetweenStates(decisionState, A.left, Label.EPSILON);
transitionBetweenStates(decisionState, emptyAlt, Label.EPSILON);
transitionBetweenStates(emptyAlt, blockEndNFAState, Label.EPSILON);
// set EOB markers for Jean
decisionState.endOfBlockStateNumber = blockEndNFAState.stateNumber;
blockEndNFAState.decisionStateType = NFAState.RIGHT_EDGE_OF_BLOCK;
g = new StateCluster(decisionState, blockEndNFAState);
}
else {
// a decision block, add an empty alt
NFAState lastRealAlt =
nfa.grammar.getNFAStateForAltOfDecision(A.left, n);
NFAState emptyAlt = newState();
emptyAlt.setDescription("epsilon path of ()? block");
transitionBetweenStates(lastRealAlt, emptyAlt, Label.EPSILON);
transitionBetweenStates(emptyAlt, A.right, Label.EPSILON);
// set EOB markers for Jean (I think this is redundant here)
A.left.endOfBlockStateNumber = A.right.stateNumber;
A.right.decisionStateType = NFAState.RIGHT_EDGE_OF_BLOCK;
g = A; // return same block, but now with optional last path
}
g.left.decisionStateType = NFAState.OPTIONAL_BLOCK_START;
return g;
}
/** From (A)+ build
*
* |---| (Transition 2 from A.right points at alt 1)
* v | (follow of loop is Transition 1)
* o->o-A-o->o
*
* Meaning that the last NFAState in A points back to A's left Transition NFAState
* and we add a new begin/end NFAState. A can be single alternative or
* multiple.
*
* During analysis we'll call the follow link (transition 1) alt n+1 for
* an n-alt A block.
*/
public StateCluster build_Aplus(StateCluster A) {
NFAState left = newState();
NFAState blockEndNFAState = newState();
blockEndNFAState.decisionStateType = NFAState.RIGHT_EDGE_OF_BLOCK;
// don't reuse A.right as loopback if it's right edge of another block
if ( A.right.decisionStateType == NFAState.RIGHT_EDGE_OF_BLOCK ) {
// nested A* so make another tail node to be the loop back
// instead of the usual A.right which is the EOB for inner loop
NFAState extraRightEdge = newState();
transitionBetweenStates(A.right, extraRightEdge, Label.EPSILON);
A.right = extraRightEdge;
}
transitionBetweenStates(A.right, blockEndNFAState, Label.EPSILON); // follow is Transition 1
// turn A's block end into a loopback (acts like alt 2)
transitionBetweenStates(A.right, A.left, Label.EPSILON); // loop back Transition 2
transitionBetweenStates(left, A.left, Label.EPSILON);
A.right.decisionStateType = NFAState.LOOPBACK;
A.left.decisionStateType = NFAState.BLOCK_START;
// set EOB markers for Jean
A.left.endOfBlockStateNumber = A.right.stateNumber;
StateCluster g = new StateCluster(left, blockEndNFAState);
return g;
}
/** From (A)* build
*
* |---|
* v |
* o->o-A-o--o (Transition 2 from block end points at alt 1; follow is Transition 1)
* | ^
* o---------| (optional branch is 2nd alt of optional block containing A+)
*
* Meaning that the last (end) NFAState in A points back to A's
* left side NFAState and we add 3 new NFAStates (the
* optional branch is built just like an optional subrule).
* See the Aplus() method for more on the loop back Transition.
* The new node on right edge is set to RIGHT_EDGE_OF_CLOSURE so we
* can detect nested (A*)* loops and insert an extra node. Previously,
* two blocks shared same EOB node.
*
* There are 2 or 3 decision points in a A*. If A is not a block (i.e.,
* it only has one alt), then there are two decisions: the optional bypass
* and then loopback. If A is a block of alts, then there are three
* decisions: bypass, loopback, and A's decision point.
*
* Note that the optional bypass must be outside the loop as (A|B)* is
* not the same thing as (A|B|)+.
*
* This is an accurate NFA representation of the meaning of (A)*, but
* for generating code, I don't need a DFA for the optional branch by
* virtue of how I generate code. The exit-loopback-branch decision
* is sufficient to let me make an appropriate enter, exit, loop
* determination. See codegen.g
*/
public StateCluster build_Astar(StateCluster A) {
NFAState bypassDecisionState = newState();
bypassDecisionState.setDescription("enter loop path of ()* block");
NFAState optionalAlt = newState();
optionalAlt.setDescription("epsilon path of ()* block");
NFAState blockEndNFAState = newState();
blockEndNFAState.decisionStateType = NFAState.RIGHT_EDGE_OF_BLOCK;
// don't reuse A.right as loopback if it's right edge of another block
if ( A.right.decisionStateType == NFAState.RIGHT_EDGE_OF_BLOCK ) {
// nested A* so make another tail node to be the loop back
// instead of the usual A.right which is the EOB for inner loop
NFAState extraRightEdge = newState();
transitionBetweenStates(A.right, extraRightEdge, Label.EPSILON);
A.right = extraRightEdge;
}
// convert A's end block to loopback
A.right.setDescription("()* loopback");
// Transition 1 to actual block of stuff
transitionBetweenStates(bypassDecisionState, A.left, Label.EPSILON);
// Transition 2 optional to bypass
transitionBetweenStates(bypassDecisionState, optionalAlt, Label.EPSILON);
transitionBetweenStates(optionalAlt, blockEndNFAState, Label.EPSILON);
// Transition 1 of end block exits
transitionBetweenStates(A.right, blockEndNFAState, Label.EPSILON);
// Transition 2 of end block loops
transitionBetweenStates(A.right, A.left, Label.EPSILON);
bypassDecisionState.decisionStateType = NFAState.BYPASS;
A.left.decisionStateType = NFAState.BLOCK_START;
A.right.decisionStateType = NFAState.LOOPBACK;
// set EOB markers for Jean
A.left.endOfBlockStateNumber = A.right.stateNumber;
bypassDecisionState.endOfBlockStateNumber = blockEndNFAState.stateNumber;
StateCluster g = new StateCluster(bypassDecisionState, blockEndNFAState);
return g;
}
/** Build an NFA predictor for special rule called Tokens manually that
* predicts which token will succeed. The refs to the rules are not
* RuleRefTransitions as I want DFA conversion to stop at the EOT
* transition on the end of each token, rather than return to Tokens rule.
* If I used normal build_alternativeBlock for this, the RuleRefTransitions
* would save return address when jumping away from Tokens rule.
*
* All I do here is build n new states for n rules with an epsilon
* edge to the rule start states and then to the next state in the
* list:
*
* o->(A) (a state links to start of A and to next in list)
* |
* o->(B)
* |
* ...
* |
* o->(Z)
*
* This is the NFA created for the artificial rule created in
* Grammar.addArtificialMatchTokensRule().
*
* 11/28/2005: removed so we can use normal rule construction for Tokens.
public NFAState build_ArtificialMatchTokensRuleNFA() {
int altNum = 1;
NFAState firstAlt = null; // the start state for the "rule"
NFAState prevAlternative = null;
Iterator iter = nfa.grammar.getRules().iterator();
// TODO: add a single decision node/state for good description
while (iter.hasNext()) {
Rule r = (Rule) iter.next();
String ruleName = r.name;
String modifier = nfa.grammar.getRuleModifier(ruleName);
if ( ruleName.equals(Grammar.ARTIFICIAL_TOKENS_RULENAME) ||
(modifier!=null &&
modifier.equals(Grammar.FRAGMENT_RULE_MODIFIER)) )
{
continue; // don't loop to yourself or do nontoken rules
}
NFAState ruleStartState = nfa.grammar.getRuleStartState(ruleName);
NFAState left = newState();
left.setDescription("alt "+altNum+" of artificial rule "+Grammar.ARTIFICIAL_TOKENS_RULENAME);
transitionBetweenStates(left, ruleStartState, Label.EPSILON);
// Are we the first alternative?
if ( firstAlt==null ) {
firstAlt = left; // track extreme top left node as rule start
}
else {
// if not first alternative, must link to this alt from previous
transitionBetweenStates(prevAlternative, left, Label.EPSILON);
}
prevAlternative = left;
altNum++;
}
firstAlt.decisionStateType = NFAState.BLOCK_START;
return firstAlt;
}
*/
/** Build an atom with all possible values in its label */
public StateCluster build_Wildcard(GrammarAST associatedAST) {
NFAState left = newState();
NFAState right = newState();
left.associatedASTNode = associatedAST;
right.associatedASTNode = associatedAST;
Label label = new Label(nfa.grammar.getTokenTypes()); // char or tokens
Transition e = new Transition(label,right);
left.addTransition(e);
StateCluster g = new StateCluster(left, right);
return g;
}
/** Build a subrule matching ^(. .*) (any tree or node). Let's use
* (^(. .+) | .) to be safe.
*/
public StateCluster build_WildcardTree(GrammarAST associatedAST) {
StateCluster wildRoot = build_Wildcard(associatedAST);
StateCluster down = build_Atom(Label.DOWN, associatedAST);
wildRoot = build_AB(wildRoot,down); // hook in; . DOWN
// make .+
StateCluster wildChildren = build_Wildcard(associatedAST);
wildChildren = build_Aplus(wildChildren);
wildRoot = build_AB(wildRoot,wildChildren); // hook in; . DOWN .+
StateCluster up = build_Atom(Label.UP, associatedAST);
wildRoot = build_AB(wildRoot,up); // hook in; . DOWN .+ UP
// make optional . alt
StateCluster optionalNodeAlt = build_Wildcard(associatedAST);
List alts = new ArrayList();
alts.add(wildRoot);
alts.add(optionalNodeAlt);
StateCluster blk = build_AlternativeBlock(alts);
return blk;
}
/** Given a collapsed block of alts (a set of atoms), pull out
* the set and return it.
*/
protected IntSet getCollapsedBlockAsSet(State blk) {
State s0 = blk;
if ( s0!=null && s0.transition(0)!=null ) {
State s1 = s0.transition(0).target;
if ( s1!=null && s1.transition(0)!=null ) {
Label label = s1.transition(0).label;
if ( label.isSet() ) {
return label.getSet();
}
}
}
return null;
}
private void transitionBetweenStates(NFAState a, NFAState b, int label) {
Transition e = new Transition(label,b);
a.addTransition(e);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarAST.java 0000644 0001750 0001750 00000036405 11256465230 023546 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import antlr.BaseAST;
import antlr.Token;
import antlr.TokenWithIndex;
import antlr.collections.AST;
import org.antlr.analysis.DFA;
import org.antlr.analysis.NFAState;
import org.antlr.misc.IntSet;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.grammar.v2.ANTLRParser;
import java.util.*;
/** Grammars are first converted to ASTs using this class and then are
* converted to NFAs via a tree walker.
*
* The reader may notice that I have made a very non-OO decision in this
* class to track variables for many different kinds of nodes. It wastes
* space for nodes that don't need the values and OO principles cry out
* for a new class type for each kind of node in my tree. I am doing this
* on purpose for a variety of reasons. I don't like using the type
* system for different node types; it yields too many damn class files
* which I hate. Perhaps if I put them all in one file. Most importantly
* though I hate all the type casting that would have to go on. I would
* have all sorts of extra work to do. Ick. Anyway, I'm doing all this
* on purpose, not out of ignorance. ;)
*/
public class GrammarAST extends BaseAST {
static int count = 0;
public int ID = ++count;
/** This AST node was created from what token? */
public Token token = null;
public String enclosingRuleName;
/** If this is a RULE node then track rule's start, stop tokens' index. */
public int ruleStartTokenIndex;
public int ruleStopTokenIndex;
/** If this is a decision node, what is the lookahead DFA? */
public DFA lookaheadDFA = null;
/** What NFA start state was built from this node? */
public NFAState NFAStartState = null;
/** This is used for TREE_BEGIN nodes to point into
* the NFA. TREE_BEGINs point at left edge of DOWN for LOOK computation
* purposes (Nullable tree child list needs special code gen when matching).
*/
public NFAState NFATreeDownState = null;
/** Rule ref nodes, token refs, set, and NOT set refs need to track their
* location in the generated NFA so that local FOLLOW sets can be
* computed during code gen for automatic error recovery.
*/
public NFAState followingNFAState = null;
/** If this is a SET node, what are the elements? */
protected IntSet setValue = null;
/** If this is a BLOCK node, track options here */
protected Map<String,Object> blockOptions;
/** If this is a BLOCK node for a rewrite rule, track referenced
* elements here. Don't track elements in nested subrules.
*/
public Set<GrammarAST> rewriteRefsShallow;
/* If REWRITE node, track EVERY element and label ref to right of ->
* for this rewrite rule. There could be multiple of these per
* rule:
*
* a : ( ... -> ... | ... -> ... ) -> ... ;
*
* We may need a list of all refs to do definitions for whole rewrite
* later.
*
* If BLOCK then tracks every element at that level and below.
*/
public Set<GrammarAST> rewriteRefsDeep;
public Map<String,Object> terminalOptions;
/** if this is an ACTION node, this is the outermost enclosing
* alt num in rule. For actions, define.g sets these (used to
* be codegen.g). We need these set so we can examine actions
* early, before code gen, for refs to rule predefined properties
* and rule labels. For most part define.g sets outerAltNum, but
* codegen.g does the ones for %foo(a={$ID.text}) type refs as
* the {$ID...} is not seen as an action until code gen pulls apart.
*/
public int outerAltNum;
/** if this is a TOKEN_REF or RULE_REF node, this is the code StringTemplate
* generated for this node. We need to update it later to add
* a label if someone does $tokenref or $ruleref in an action.
*/
public StringTemplate code;
/**
*
* @return
*/
public Map<String, Object> getBlockOptions() {
return blockOptions;
}
/**
*
* @param blockOptions
*/
public void setBlockOptions(Map<String, Object> blockOptions) {
this.blockOptions = blockOptions;
}
public GrammarAST() {;}
public GrammarAST(int t, String txt) {
initialize(t,txt);
}
public void initialize(int i, String s) {
token = new TokenWithIndex(i,s);
}
public void initialize(AST ast) {
GrammarAST t = ((GrammarAST)ast);
this.token = t.token;
this.enclosingRuleName = t.enclosingRuleName;
this.ruleStartTokenIndex = t.ruleStartTokenIndex;
this.ruleStopTokenIndex = t.ruleStopTokenIndex;
this.setValue = t.setValue;
this.blockOptions = t.blockOptions;
this.outerAltNum = t.outerAltNum;
}
public void initialize(Token token) {
this.token = token;
}
public DFA getLookaheadDFA() {
return lookaheadDFA;
}
public void setLookaheadDFA(DFA lookaheadDFA) {
this.lookaheadDFA = lookaheadDFA;
}
public Token getToken() {
return token;
}
public NFAState getNFAStartState() {
return NFAStartState;
}
public void setNFAStartState(NFAState nfaStartState) {
this.NFAStartState = nfaStartState;
}
/** Save the option key/value pair and process it; return the key
* or null if invalid option.
*/
public String setBlockOption(Grammar grammar, String key, Object value) {
if ( blockOptions == null ) {
blockOptions = new HashMap();
}
return setOption(blockOptions, Grammar.legalBlockOptions, grammar, key, value);
}
public String setTerminalOption(Grammar grammar, String key, Object value) {
if ( terminalOptions == null ) {
terminalOptions = new HashMap<String,Object>();
}
return setOption(terminalOptions, Grammar.legalTokenOptions, grammar, key, value);
}
public String setOption(Map options, Set legalOptions, Grammar grammar, String key, Object value) {
if ( !legalOptions.contains(key) ) {
ErrorManager.grammarError(ErrorManager.MSG_ILLEGAL_OPTION,
grammar,
token,
key);
return null;
}
if ( value instanceof String ) {
String vs = (String)value;
if ( vs.charAt(0)=='"' ) {
value = vs.substring(1,vs.length()-1); // strip quotes
}
}
if ( key.equals("k") ) {
grammar.numberOfManualLookaheadOptions++;
}
if ( key.equals("backtrack") && value.toString().equals("true") ) {
grammar.composite.getRootGrammar().atLeastOneBacktrackOption = true;
}
options.put(key, value);
return key;
}
public Object getBlockOption(String key) {
Object value = null;
if ( blockOptions != null ) {
value = blockOptions.get(key);
}
return value;
}
public void setOptions(Grammar grammar, Map options) {
if ( options==null ) {
this.blockOptions = null;
return;
}
Set keys = options.keySet();
for (Iterator it = keys.iterator(); it.hasNext();) {
String optionName = (String) it.next();
String stored= setBlockOption(grammar, optionName, options.get(optionName));
if ( stored==null ) {
it.remove();
}
}
}
public String getText() {
if ( token!=null ) {
return token.getText();
}
return "";
}
public void setType(int type) {
token.setType(type);
}
public void setText(String text) {
token.setText(text);
}
public int getType() {
if ( token!=null ) {
return token.getType();
}
return -1;
}
public int getLine() {
int line=0;
if ( token!=null ) {
line = token.getLine();
}
if ( line==0 ) {
AST child = getFirstChild();
if ( child!=null ) {
line = child.getLine();
}
}
return line;
}
public int getColumn() {
int col=0;
if ( token!=null ) {
col = token.getColumn();
}
if ( col==0 ) {
AST child = getFirstChild();
if ( child!=null ) {
col = child.getColumn();
}
}
return col;
}
public void setLine(int line) {
token.setLine(line);
}
public void setColumn(int col) {
token.setColumn(col);
}
public IntSet getSetValue() {
return setValue;
}
public void setSetValue(IntSet setValue) {
this.setValue = setValue;
}
public GrammarAST getLastChild() {
return ((GrammarAST)getFirstChild()).getLastSibling();
}
public GrammarAST getLastSibling() {
GrammarAST t = this;
GrammarAST last = null;
while ( t!=null ) {
last = t;
t = (GrammarAST)t.getNextSibling();
}
return last;
}
/** Get the ith child from 0 */
public GrammarAST getChild(int i) {
int n = 0;
AST t = getFirstChild();
while ( t!=null ) {
if ( n==i ) {
return (GrammarAST)t;
}
n++;
t = (GrammarAST)t.getNextSibling();
}
return null;
}
public GrammarAST getFirstChildWithType(int ttype) {
AST t = getFirstChild();
while ( t!=null ) {
if ( t.getType()==ttype ) {
return (GrammarAST)t;
}
t = (GrammarAST)t.getNextSibling();
}
return null;
}
public GrammarAST[] getChildrenAsArray() {
AST t = getFirstChild();
GrammarAST[] array = new GrammarAST[getNumberOfChildren()];
int i = 0;
while ( t!=null ) {
array[i] = (GrammarAST)t;
t = t.getNextSibling();
i++;
}
return array;
}
/** Return a reference to the first node (depth-first) that has
* token type ttype. Assume 'this' is a root node; don't visit siblings
* of root. Return null if no node found with ttype.
*/
public GrammarAST findFirstType(int ttype) {
// check this node (the root) first
if ( this.getType()==ttype ) {
return this;
}
// else check children
GrammarAST child = (GrammarAST)this.getFirstChild();
while ( child!=null ) {
GrammarAST result = child.findFirstType(ttype);
if ( result!=null ) {
return result;
}
child = (GrammarAST)child.getNextSibling();
}
return null;
}
public int getNumberOfChildrenWithType(int ttype) {
AST p = this.getFirstChild();
int n = 0;
while ( p!=null ) {
if ( p.getType()==ttype ) n++;
p = p.getNextSibling();
}
return n;
}
/** Make nodes unique based upon Token so we can add them to a Set; if
* not a GrammarAST, check type.
*/
public boolean equals(Object ast) {
if ( this == ast ) {
return true;
}
if ( !(ast instanceof GrammarAST) ) {
return this.getType() == ((AST)ast).getType();
}
GrammarAST t = (GrammarAST)ast;
return token.getLine() == t.getLine() &&
token.getColumn() == t.getColumn();
}
/** See if tree has exact token types and structure; no text */
public boolean hasSameTreeStructure(AST t) {
// check roots first.
if (this.getType() != t.getType()) return false;
// if roots match, do full list match test on children.
if (this.getFirstChild() != null) {
if (!(((GrammarAST)this.getFirstChild()).hasSameListStructure(t.getFirstChild()))) return false;
}
// sibling has no kids, make sure t doesn't either
else if (t.getFirstChild() != null) {
return false;
}
return true;
}
public boolean hasSameListStructure(AST t) {
AST sibling;
// the empty tree is not a match of any non-null tree.
if (t == null) {
return false;
}
// Otherwise, start walking sibling lists. First mismatch, return false.
for (sibling = this;
sibling != null && t != null;
sibling = sibling.getNextSibling(), t = t.getNextSibling())
{
// as a quick optimization, check roots first.
if (sibling.getType()!=t.getType()) {
return false;
}
// if roots match, do full list match test on children.
if (sibling.getFirstChild() != null) {
if (!((GrammarAST)sibling.getFirstChild()).hasSameListStructure(t.getFirstChild())) {
return false;
}
}
// sibling has no kids, make sure t doesn't either
else if (t.getFirstChild() != null) {
return false;
}
}
if (sibling == null && t == null) {
return true;
}
// one sibling list has more than the other
return false;
}
public static GrammarAST dup(AST t) {
if ( t==null ) {
return null;
}
GrammarAST dup_t = new GrammarAST();
dup_t.initialize(t);
return dup_t;
}
/** Duplicate tree including siblings of root. */
public static GrammarAST dupListNoActions(GrammarAST t, GrammarAST parent) {
GrammarAST result = dupTreeNoActions(t, parent); // if t == null, then result==null
GrammarAST nt = result;
while (t != null) { // for each sibling of the root
t = (GrammarAST)t.getNextSibling();
if ( t!=null && t.getType()==ANTLRParser.ACTION ) {
continue;
}
GrammarAST d = dupTreeNoActions(t, parent);
if ( d!=null ) {
if ( nt!=null ) {
nt.setNextSibling(d); // dup each subtree, building new tree
}
nt = d;
}
}
return result;
}
/**Duplicate a tree, assuming this is a root node of a tree--
* duplicate that node and what's below; ignore siblings of root node.
*/
public static GrammarAST dupTreeNoActions(GrammarAST t, GrammarAST parent) {
if ( t==null ) {
return null;
}
int ttype = t.getType();
if ( ttype==ANTLRParser.REWRITE ) {
return null;
}
if ( ttype==ANTLRParser.BANG || ttype==ANTLRParser.ROOT ) {
// return x from ^(ROOT x)
return (GrammarAST)dupListNoActions((GrammarAST)t.getFirstChild(), t);
}
/* DOH! Must allow labels for sem preds
if ( (ttype==ANTLRParser.ASSIGN||ttype==ANTLRParser.PLUS_ASSIGN) &&
(parent==null||parent.getType()!=ANTLRParser.OPTIONS) )
{
return dupTreeNoActions(t.getChild(1), t); // return x from ^(ASSIGN label x)
}
*/
GrammarAST result = dup(t); // make copy of root
// copy all children of root.
GrammarAST kids = dupListNoActions((GrammarAST)t.getFirstChild(), t);
result.setFirstChild(kids);
return result;
}
public void setTreeEnclosingRuleNameDeeply(String rname) {
GrammarAST t = this;
t.enclosingRuleName = rname;
t = t.getChild(0);
while (t != null) { // for each sibling of the root
t.setTreeEnclosingRuleNameDeeply(rname);
t = (GrammarAST)t.getNextSibling();
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/ErrorManager.java 0000644 0001750 0001750 00000102333 11256465230 024166 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import antlr.Token;
import org.antlr.Tool;
import org.antlr.misc.BitSet;
import org.antlr.analysis.DFAState;
import org.antlr.analysis.DecisionProbe;
import org.antlr.analysis.Label;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.StringTemplateErrorListener;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.language.AngleBracketTemplateLexer;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.util.*;
/** Defines all the errors ANTLR can generator for both the tool and for
* issues with a grammar.
*
* Here is a list of language names:
*
* http://ftp.ics.uci.edu/pub/ietf/http/related/iso639.txt
*
* Here is a list of country names:
*
* http://www.chemie.fu-berlin.de/diverse/doc/ISO_3166.html
*
* I use constants not strings to identify messages as the compiler will
* find any errors/mismatches rather than leaving a mistyped string in
* the code to be found randomly in the future. Further, Intellij can
* do field name expansion to save me some typing. I have to map
* int constants to template names, however, which could introduce a mismatch.
* Someone could provide a .stg file that had a template name wrong. When
* I load the group, then, I must verify that all messages are there.
*
* This is essentially the functionality of the resource bundle stuff Java
* has, but I don't want to load a property file--I want to load a template
* group file and this is so simple, why mess with their junk.
*
* I use the default Locale as defined by java to compute a group file name
* in the org/antlr/tool/templates/messages dir called en_US.stg and so on.
*
* Normally we want to use the default locale, but often a message file will
* not exist for it so we must fall back on the US local.
*
* During initialization of this class, all errors go straight to System.err.
* There is no way around this. If I have not set up the error system, how
* can I do errors properly? For example, if the string template group file
* full of messages has an error, how could I print to anything but System.err?
*
* TODO: how to map locale to a file encoding for the stringtemplate group file?
* StringTemplate knows how to pay attention to the default encoding so it
* should probably just work unless a GUI sets the local to some chinese
* variation but System.getProperty("file.encoding") is US. Hmm...
*
* TODO: get antlr.g etc.. parsing errors to come here.
*/
public class ErrorManager {
// TOOL ERRORS
// file errors
public static final int MSG_CANNOT_WRITE_FILE = 1;
public static final int MSG_CANNOT_CLOSE_FILE = 2;
public static final int MSG_CANNOT_FIND_TOKENS_FILE = 3;
public static final int MSG_ERROR_READING_TOKENS_FILE = 4;
public static final int MSG_DIR_NOT_FOUND = 5;
public static final int MSG_OUTPUT_DIR_IS_FILE = 6;
public static final int MSG_CANNOT_OPEN_FILE = 7;
public static final int MSG_FILE_AND_GRAMMAR_NAME_DIFFER = 8;
public static final int MSG_FILENAME_EXTENSION_ERROR = 9;
public static final int MSG_INTERNAL_ERROR = 10;
public static final int MSG_INTERNAL_WARNING = 11;
public static final int MSG_ERROR_CREATING_ARTIFICIAL_RULE = 12;
public static final int MSG_TOKENS_FILE_SYNTAX_ERROR = 13;
public static final int MSG_CANNOT_GEN_DOT_FILE = 14;
public static final int MSG_BAD_AST_STRUCTURE = 15;
public static final int MSG_BAD_ACTION_AST_STRUCTURE = 16;
// code gen errors
public static final int MSG_MISSING_CODE_GEN_TEMPLATES = 20;
public static final int MSG_MISSING_CYCLIC_DFA_CODE_GEN_TEMPLATES = 21;
public static final int MSG_CODE_GEN_TEMPLATES_INCOMPLETE = 22;
public static final int MSG_CANNOT_CREATE_TARGET_GENERATOR = 23;
//public static final int MSG_CANNOT_COMPUTE_SAMPLE_INPUT_SEQ = 24;
// GRAMMAR ERRORS
public static final int MSG_SYNTAX_ERROR = 100;
public static final int MSG_RULE_REDEFINITION = 101;
public static final int MSG_LEXER_RULES_NOT_ALLOWED = 102;
public static final int MSG_PARSER_RULES_NOT_ALLOWED = 103;
public static final int MSG_CANNOT_FIND_ATTRIBUTE_NAME_IN_DECL = 104;
public static final int MSG_NO_TOKEN_DEFINITION = 105;
public static final int MSG_UNDEFINED_RULE_REF = 106;
public static final int MSG_LITERAL_NOT_ASSOCIATED_WITH_LEXER_RULE = 107;
public static final int MSG_CANNOT_ALIAS_TOKENS_IN_LEXER = 108;
public static final int MSG_ATTRIBUTE_REF_NOT_IN_RULE = 111;
public static final int MSG_INVALID_RULE_SCOPE_ATTRIBUTE_REF = 112;
public static final int MSG_UNKNOWN_ATTRIBUTE_IN_SCOPE = 113;
public static final int MSG_UNKNOWN_SIMPLE_ATTRIBUTE = 114;
public static final int MSG_INVALID_RULE_PARAMETER_REF = 115;
public static final int MSG_UNKNOWN_RULE_ATTRIBUTE = 116;
public static final int MSG_ISOLATED_RULE_SCOPE = 117;
public static final int MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE = 118;
public static final int MSG_LABEL_CONFLICTS_WITH_RULE = 119;
public static final int MSG_LABEL_CONFLICTS_WITH_TOKEN = 120;
public static final int MSG_LABEL_CONFLICTS_WITH_RULE_SCOPE_ATTRIBUTE = 121;
public static final int MSG_LABEL_CONFLICTS_WITH_RULE_ARG_RETVAL = 122;
public static final int MSG_ATTRIBUTE_CONFLICTS_WITH_RULE = 123;
public static final int MSG_ATTRIBUTE_CONFLICTS_WITH_RULE_ARG_RETVAL = 124;
public static final int MSG_LABEL_TYPE_CONFLICT = 125;
public static final int MSG_ARG_RETVAL_CONFLICT = 126;
public static final int MSG_NONUNIQUE_REF = 127;
public static final int MSG_FORWARD_ELEMENT_REF = 128;
public static final int MSG_MISSING_RULE_ARGS = 129;
public static final int MSG_RULE_HAS_NO_ARGS = 130;
public static final int MSG_ARGS_ON_TOKEN_REF = 131;
public static final int MSG_RULE_REF_AMBIG_WITH_RULE_IN_ALT = 132;
public static final int MSG_ILLEGAL_OPTION = 133;
public static final int MSG_LIST_LABEL_INVALID_UNLESS_RETVAL_STRUCT = 134;
public static final int MSG_UNDEFINED_TOKEN_REF_IN_REWRITE = 135;
public static final int MSG_REWRITE_ELEMENT_NOT_PRESENT_ON_LHS = 136;
public static final int MSG_UNDEFINED_LABEL_REF_IN_REWRITE = 137;
public static final int MSG_NO_GRAMMAR_START_RULE = 138;
public static final int MSG_EMPTY_COMPLEMENT = 139;
public static final int MSG_UNKNOWN_DYNAMIC_SCOPE = 140;
public static final int MSG_UNKNOWN_DYNAMIC_SCOPE_ATTRIBUTE = 141;
public static final int MSG_ISOLATED_RULE_ATTRIBUTE = 142;
public static final int MSG_INVALID_ACTION_SCOPE = 143;
public static final int MSG_ACTION_REDEFINITION = 144;
public static final int MSG_DOUBLE_QUOTES_ILLEGAL = 145;
public static final int MSG_INVALID_TEMPLATE_ACTION = 146;
public static final int MSG_MISSING_ATTRIBUTE_NAME = 147;
public static final int MSG_ARG_INIT_VALUES_ILLEGAL = 148;
public static final int MSG_REWRITE_OR_OP_WITH_NO_OUTPUT_OPTION = 149;
public static final int MSG_NO_RULES = 150;
public static final int MSG_WRITE_TO_READONLY_ATTR = 151;
public static final int MSG_MISSING_AST_TYPE_IN_TREE_GRAMMAR = 152;
public static final int MSG_REWRITE_FOR_MULTI_ELEMENT_ALT = 153;
public static final int MSG_RULE_INVALID_SET = 154;
public static final int MSG_HETERO_ILLEGAL_IN_REWRITE_ALT = 155;
public static final int MSG_NO_SUCH_GRAMMAR_SCOPE = 156;
public static final int MSG_NO_SUCH_RULE_IN_SCOPE = 157;
public static final int MSG_TOKEN_ALIAS_CONFLICT = 158;
public static final int MSG_TOKEN_ALIAS_REASSIGNMENT = 159;
public static final int MSG_TOKEN_VOCAB_IN_DELEGATE = 160;
public static final int MSG_INVALID_IMPORT = 161;
public static final int MSG_IMPORTED_TOKENS_RULE_EMPTY = 162;
public static final int MSG_IMPORT_NAME_CLASH = 163;
public static final int MSG_AST_OP_WITH_NON_AST_OUTPUT_OPTION = 164;
public static final int MSG_AST_OP_IN_ALT_WITH_REWRITE = 165;
public static final int MSG_WILDCARD_AS_ROOT = 166;
public static final int MSG_CONFLICTING_OPTION_IN_TREE_FILTER = 167;
// GRAMMAR WARNINGS
public static final int MSG_GRAMMAR_NONDETERMINISM = 200; // A predicts alts 1,2
public static final int MSG_UNREACHABLE_ALTS = 201; // nothing predicts alt i
public static final int MSG_DANGLING_STATE = 202; // no edges out of state
public static final int MSG_INSUFFICIENT_PREDICATES = 203;
public static final int MSG_DUPLICATE_SET_ENTRY = 204; // (A|A)
public static final int MSG_ANALYSIS_ABORTED = 205;
public static final int MSG_RECURSION_OVERLOW = 206;
public static final int MSG_LEFT_RECURSION = 207;
public static final int MSG_UNREACHABLE_TOKENS = 208; // nothing predicts token
public static final int MSG_TOKEN_NONDETERMINISM = 209; // alts of Tokens rule
public static final int MSG_LEFT_RECURSION_CYCLES = 210;
public static final int MSG_NONREGULAR_DECISION = 211;
// Dependency sorting errors
//
public static final int MSG_CIRCULAR_DEPENDENCY = 212; // t1.g -> t2.g -> t3.g ->t1.g
public static final int MAX_MESSAGE_NUMBER = 212;
/** Do not do perform analysis if one of these happens */
public static final BitSet ERRORS_FORCING_NO_ANALYSIS = new BitSet() {
{
add(MSG_RULE_REDEFINITION);
add(MSG_UNDEFINED_RULE_REF);
add(MSG_LEFT_RECURSION_CYCLES);
add(MSG_REWRITE_OR_OP_WITH_NO_OUTPUT_OPTION);
add(MSG_NO_RULES);
add(MSG_NO_SUCH_GRAMMAR_SCOPE);
add(MSG_NO_SUCH_RULE_IN_SCOPE);
add(MSG_LEXER_RULES_NOT_ALLOWED);
add(MSG_WILDCARD_AS_ROOT);
add(MSG_CIRCULAR_DEPENDENCY);
// TODO: ...
}
};
/** Do not do code gen if one of these happens */
public static final BitSet ERRORS_FORCING_NO_CODEGEN = new BitSet() {
{
add(MSG_NONREGULAR_DECISION);
add(MSG_RECURSION_OVERLOW);
add(MSG_UNREACHABLE_ALTS);
add(MSG_FILE_AND_GRAMMAR_NAME_DIFFER);
add(MSG_INVALID_IMPORT);
add(MSG_AST_OP_WITH_NON_AST_OUTPUT_OPTION);
add(MSG_CIRCULAR_DEPENDENCY);
// TODO: ...
}
};
/** Only one error can be emitted for any entry in this table.
* Map<String,Set> where the key is a method name like danglingState.
* The set is whatever that method accepts or derives like a DFA.
*/
public static final Map emitSingleError = new HashMap() {
{
put("danglingState", new HashSet());
}
};
/** Messages should be sensitive to the locale. */
private static Locale locale;
private static String formatName;
/** Each thread might need it's own error listener; e.g., a GUI with
* multiple window frames holding multiple grammars.
*/
private static Map threadToListenerMap = new HashMap();
static class ErrorState {
public int errors;
public int warnings;
public int infos;
/** Track all msgIDs; we use to abort later if necessary
* also used in Message to find out what type of message it is via getMessageType()
*/
public BitSet errorMsgIDs = new BitSet();
public BitSet warningMsgIDs = new BitSet();
// TODO: figure out how to do info messages. these do not have IDs...kr
//public BitSet infoMsgIDs = new BitSet();
}
/** Track the number of errors regardless of the listener but track
* per thread.
*/
private static Map threadToErrorStateMap = new HashMap();
/** Each thread has its own ptr to a Tool object, which knows how
* to panic, for example. In a GUI, the thread might just throw an Error
* to exit rather than the suicide System.exit.
*/
private static Map threadToToolMap = new HashMap();
/** The group of templates that represent all possible ANTLR errors. */
private static StringTemplateGroup messages;
/** The group of templates that represent the current message format. */
private static StringTemplateGroup format;
/** From a msgID how can I get the name of the template that describes
* the error or warning?
*/
private static String[] idToMessageTemplateName = new String[MAX_MESSAGE_NUMBER+1];
static ANTLRErrorListener theDefaultErrorListener = new ANTLRErrorListener() {
public void info(String msg) {
if (formatWantsSingleLineMessage()) {
msg = msg.replaceAll("\n", " ");
}
System.err.println(msg);
}
public void error(Message msg) {
String outputMsg = msg.toString();
if (formatWantsSingleLineMessage()) {
outputMsg = outputMsg.replaceAll("\n", " ");
}
System.err.println(outputMsg);
}
public void warning(Message msg) {
String outputMsg = msg.toString();
if (formatWantsSingleLineMessage()) {
outputMsg = outputMsg.replaceAll("\n", " ");
}
System.err.println(outputMsg);
}
public void error(ToolMessage msg) {
String outputMsg = msg.toString();
if (formatWantsSingleLineMessage()) {
outputMsg = outputMsg.replaceAll("\n", " ");
}
System.err.println(outputMsg);
}
};
/** Handle all ST error listeners here (code gen, Grammar, and this class
* use templates.
*/
static StringTemplateErrorListener initSTListener =
new StringTemplateErrorListener() {
public void error(String s, Throwable e) {
System.err.println("ErrorManager init error: "+s);
if ( e!=null ) {
System.err.println("exception: "+e);
}
/*
if ( e!=null ) {
e.printStackTrace(System.err);
}
*/
}
public void warning(String s) {
System.err.println("ErrorManager init warning: "+s);
}
public void debug(String s) {}
};
/** During verification of the messages group file, don't gen errors.
* I'll handle them here. This is used only after file has loaded ok
* and only for the messages STG.
*/
static StringTemplateErrorListener blankSTListener =
new StringTemplateErrorListener() {
public void error(String s, Throwable e) {}
public void warning(String s) {}
public void debug(String s) {}
};
/** Errors during initialization related to ST must all go to System.err.
*/
static StringTemplateErrorListener theDefaultSTListener =
new StringTemplateErrorListener() {
public void error(String s, Throwable e) {
if ( e instanceof InvocationTargetException ) {
e = ((InvocationTargetException)e).getTargetException();
}
ErrorManager.error(ErrorManager.MSG_INTERNAL_ERROR, s, e);
}
public void warning(String s) {
ErrorManager.warning(ErrorManager.MSG_INTERNAL_WARNING, s);
}
public void debug(String s) {
}
};
// make sure that this class is ready to use after loading
static {
initIdToMessageNameMapping();
// it is inefficient to set the default locale here if another
// piece of code is going to set the locale, but that would
// require that a user call an init() function or something. I prefer
// that this class be ready to go when loaded as I'm absentminded ;)
setLocale(Locale.getDefault());
// try to load the message format group
// the user might have specified one on the command line
// if not, or if the user has given an illegal value, we will fall back to "antlr"
setFormat("antlr");
}
public static StringTemplateErrorListener getStringTemplateErrorListener() {
return theDefaultSTListener;
}
/** We really only need a single locale for entire running ANTLR code
* in a single VM. Only pay attention to the language, not the country
* so that French Canadians and French Frenchies all get the same
* template file, fr.stg. Just easier this way.
*/
public static void setLocale(Locale locale) {
ErrorManager.locale = locale;
String language = locale.getLanguage();
String fileName = "org/antlr/tool/templates/messages/languages/"+language+".stg";
ClassLoader cl = Thread.currentThread().getContextClassLoader();
InputStream is = cl.getResourceAsStream(fileName);
if ( is==null ) {
cl = ErrorManager.class.getClassLoader();
is = cl.getResourceAsStream(fileName);
}
if ( is==null && language.equals(Locale.US.getLanguage()) ) {
rawError("ANTLR installation corrupted; cannot find English messages file "+fileName);
panic();
}
else if ( is==null ) {
//rawError("no such locale file "+fileName+" retrying with English locale");
setLocale(Locale.US); // recurse on this rule, trying the US locale
return;
}
BufferedReader br = null;
try {
br = new BufferedReader(new InputStreamReader(is));
messages = new StringTemplateGroup(br,
AngleBracketTemplateLexer.class,
initSTListener);
br.close();
}
catch (IOException ioe) {
rawError("error reading message file "+fileName, ioe);
}
finally {
if ( br!=null ) {
try {
br.close();
}
catch (IOException ioe) {
rawError("cannot close message file "+fileName, ioe);
}
}
}
messages.setErrorListener(blankSTListener);
boolean messagesOK = verifyMessages();
if ( !messagesOK && language.equals(Locale.US.getLanguage()) ) {
rawError("ANTLR installation corrupted; English messages file "+language+".stg incomplete");
panic();
}
else if ( !messagesOK ) {
setLocale(Locale.US); // try US to see if that will work
}
}
/** The format gets reset either from the Tool if the user supplied a command line option to that effect
* Otherwise we just use the default "antlr".
*/
public static void setFormat(String formatName) {
ErrorManager.formatName = formatName;
String fileName = "org/antlr/tool/templates/messages/formats/"+formatName+".stg";
ClassLoader cl = Thread.currentThread().getContextClassLoader();
InputStream is = cl.getResourceAsStream(fileName);
if ( is==null ) {
cl = ErrorManager.class.getClassLoader();
is = cl.getResourceAsStream(fileName);
}
if ( is==null && formatName.equals("antlr") ) {
rawError("ANTLR installation corrupted; cannot find ANTLR messages format file "+fileName);
panic();
}
else if ( is==null ) {
rawError("no such message format file "+fileName+" retrying with default ANTLR format");
setFormat("antlr"); // recurse on this rule, trying the default message format
return;
}
BufferedReader br = null;
try {
br = new BufferedReader(new InputStreamReader(is));
format = new StringTemplateGroup(br,
AngleBracketTemplateLexer.class,
initSTListener);
}
finally {
try {
if ( br!=null ) {
br.close();
}
}
catch (IOException ioe) {
rawError("cannot close message format file "+fileName, ioe);
}
}
format.setErrorListener(blankSTListener);
boolean formatOK = verifyFormat();
if ( !formatOK && formatName.equals("antlr") ) {
rawError("ANTLR installation corrupted; ANTLR messages format file "+formatName+".stg incomplete");
panic();
}
else if ( !formatOK ) {
setFormat("antlr"); // recurse on this rule, trying the default message format
}
}
/** Encodes the error handling found in setLocale, but does not trigger
* panics, which would make GUI tools die if ANTLR's installation was
* a bit screwy. Duplicated code...ick.
public static Locale getLocaleForValidMessages(Locale locale) {
ErrorManager.locale = locale;
String language = locale.getLanguage();
String fileName = "org/antlr/tool/templates/messages/"+language+".stg";
ClassLoader cl = Thread.currentThread().getContextClassLoader();
InputStream is = cl.getResourceAsStream(fileName);
if ( is==null && language.equals(Locale.US.getLanguage()) ) {
return null;
}
else if ( is==null ) {
return getLocaleForValidMessages(Locale.US); // recurse on this rule, trying the US locale
}
boolean messagesOK = verifyMessages();
if ( !messagesOK && language.equals(Locale.US.getLanguage()) ) {
return null;
}
else if ( !messagesOK ) {
return getLocaleForValidMessages(Locale.US); // try US to see if that will work
}
return true;
}
*/
/** In general, you'll want all errors to go to a single spot.
* However, in a GUI, you might have two frames up with two
* different grammars. Two threads might launch to process the
* grammars--you would want errors to go to different objects
* depending on the thread. I store a single listener per
* thread.
*/
public static void setErrorListener(ANTLRErrorListener listener) {
threadToListenerMap.put(Thread.currentThread(), listener);
}
public static void removeErrorListener() {
threadToListenerMap.remove(Thread.currentThread());
}
public static void setTool(Tool tool) {
threadToToolMap.put(Thread.currentThread(), tool);
}
/** Given a message ID, return a StringTemplate that somebody can fill
* with data. We need to convert the int ID to the name of a template
* in the messages ST group.
*/
public static StringTemplate getMessage(int msgID) {
String msgName = idToMessageTemplateName[msgID];
return messages.getInstanceOf(msgName);
}
public static String getMessageType(int msgID) {
if (getErrorState().warningMsgIDs.member(msgID)) {
return messages.getInstanceOf("warning").toString();
}
else if (getErrorState().errorMsgIDs.member(msgID)) {
return messages.getInstanceOf("error").toString();
}
assertTrue(false, "Assertion failed! Message ID " + msgID + " created but is not present in errorMsgIDs or warningMsgIDs.");
return "";
}
/** Return a StringTemplate that refers to the current format used for
* emitting messages.
*/
public static StringTemplate getLocationFormat() {
return format.getInstanceOf("location");
}
public static StringTemplate getReportFormat() {
return format.getInstanceOf("report");
}
public static StringTemplate getMessageFormat() {
return format.getInstanceOf("message");
}
public static boolean formatWantsSingleLineMessage() {
return format.getInstanceOf("wantsSingleLineMessage").toString().equals("true");
}
public static ANTLRErrorListener getErrorListener() {
ANTLRErrorListener el =
(ANTLRErrorListener)threadToListenerMap.get(Thread.currentThread());
if ( el==null ) {
return theDefaultErrorListener;
}
return el;
}
public static ErrorState getErrorState() {
ErrorState ec =
(ErrorState)threadToErrorStateMap.get(Thread.currentThread());
if ( ec==null ) {
ec = new ErrorState();
threadToErrorStateMap.put(Thread.currentThread(), ec);
}
return ec;
}
public static int getNumErrors() {
return getErrorState().errors;
}
public static void resetErrorState() {
threadToListenerMap = new HashMap();
ErrorState ec = new ErrorState();
threadToErrorStateMap.put(Thread.currentThread(), ec);
}
public static void info(String msg) {
getErrorState().infos++;
getErrorListener().info(msg);
}
public static void error(int msgID) {
getErrorState().errors++;
getErrorState().errorMsgIDs.add(msgID);
getErrorListener().error(new ToolMessage(msgID));
}
public static void error(int msgID, Throwable e) {
getErrorState().errors++;
getErrorState().errorMsgIDs.add(msgID);
getErrorListener().error(new ToolMessage(msgID,e));
}
public static void error(int msgID, Object arg) {
getErrorState().errors++;
getErrorState().errorMsgIDs.add(msgID);
getErrorListener().error(new ToolMessage(msgID, arg));
}
public static void error(int msgID, Object arg, Object arg2) {
getErrorState().errors++;
getErrorState().errorMsgIDs.add(msgID);
getErrorListener().error(new ToolMessage(msgID, arg, arg2));
}
public static void error(int msgID, Object arg, Throwable e) {
getErrorState().errors++;
getErrorState().errorMsgIDs.add(msgID);
getErrorListener().error(new ToolMessage(msgID, arg, e));
}
public static void warning(int msgID, Object arg) {
getErrorState().warnings++;
getErrorState().warningMsgIDs.add(msgID);
getErrorListener().warning(new ToolMessage(msgID, arg));
}
public static void nondeterminism(DecisionProbe probe,
DFAState d)
{
getErrorState().warnings++;
Message msg = new GrammarNonDeterminismMessage(probe,d);
getErrorState().warningMsgIDs.add(msg.msgID);
getErrorListener().warning(msg);
}
public static void danglingState(DecisionProbe probe,
DFAState d)
{
getErrorState().errors++;
Message msg = new GrammarDanglingStateMessage(probe,d);
getErrorState().errorMsgIDs.add(msg.msgID);
Set seen = (Set)emitSingleError.get("danglingState");
if ( !seen.contains(d.dfa.decisionNumber+"|"+d.getAltSet()) ) {
getErrorListener().error(msg);
// we've seen this decision and this alt set; never again
seen.add(d.dfa.decisionNumber+"|"+d.getAltSet());
}
}
public static void analysisAborted(DecisionProbe probe)
{
getErrorState().warnings++;
Message msg = new GrammarAnalysisAbortedMessage(probe);
getErrorState().warningMsgIDs.add(msg.msgID);
getErrorListener().warning(msg);
}
public static void unreachableAlts(DecisionProbe probe,
List alts)
{
getErrorState().errors++;
Message msg = new GrammarUnreachableAltsMessage(probe,alts);
getErrorState().errorMsgIDs.add(msg.msgID);
getErrorListener().error(msg);
}
public static void insufficientPredicates(DecisionProbe probe,
DFAState d,
Map<Integer, Set<Token>> altToUncoveredLocations)
{
getErrorState().warnings++;
Message msg = new GrammarInsufficientPredicatesMessage(probe,d,altToUncoveredLocations);
getErrorState().warningMsgIDs.add(msg.msgID);
getErrorListener().warning(msg);
}
public static void nonLLStarDecision(DecisionProbe probe) {
getErrorState().errors++;
Message msg = new NonRegularDecisionMessage(probe, probe.getNonDeterministicAlts());
getErrorState().errorMsgIDs.add(msg.msgID);
getErrorListener().error(msg);
}
public static void recursionOverflow(DecisionProbe probe,
DFAState sampleBadState,
int alt,
Collection targetRules,
Collection callSiteStates)
{
getErrorState().errors++;
Message msg = new RecursionOverflowMessage(probe,sampleBadState, alt,
targetRules, callSiteStates);
getErrorState().errorMsgIDs.add(msg.msgID);
getErrorListener().error(msg);
}
/*
// TODO: we can remove I think. All detected now with cycles check.
public static void leftRecursion(DecisionProbe probe,
int alt,
Collection targetRules,
Collection callSiteStates)
{
getErrorState().warnings++;
Message msg = new LeftRecursionMessage(probe, alt, targetRules, callSiteStates);
getErrorState().warningMsgIDs.add(msg.msgID);
getErrorListener().warning(msg);
}
*/
public static void leftRecursionCycles(Collection cycles) {
getErrorState().errors++;
Message msg = new LeftRecursionCyclesMessage(cycles);
getErrorState().errorMsgIDs.add(msg.msgID);
getErrorListener().warning(msg);
}
public static void grammarError(int msgID,
Grammar g,
Token token,
Object arg,
Object arg2)
{
getErrorState().errors++;
Message msg = new GrammarSemanticsMessage(msgID,g,token,arg,arg2);
getErrorState().errorMsgIDs.add(msgID);
getErrorListener().error(msg);
}
public static void grammarError(int msgID,
Grammar g,
Token token,
Object arg)
{
grammarError(msgID,g,token,arg,null);
}
public static void grammarError(int msgID,
Grammar g,
Token token)
{
grammarError(msgID,g,token,null,null);
}
public static void grammarWarning(int msgID,
Grammar g,
Token token,
Object arg,
Object arg2)
{
getErrorState().warnings++;
Message msg = new GrammarSemanticsMessage(msgID,g,token,arg,arg2);
getErrorState().warningMsgIDs.add(msgID);
getErrorListener().warning(msg);
}
public static void grammarWarning(int msgID,
Grammar g,
Token token,
Object arg)
{
grammarWarning(msgID,g,token,arg,null);
}
public static void grammarWarning(int msgID,
Grammar g,
Token token)
{
grammarWarning(msgID,g,token,null,null);
}
public static void syntaxError(int msgID,
Grammar grammar,
Token token,
Object arg,
antlr.RecognitionException re)
{
getErrorState().errors++;
getErrorState().errorMsgIDs.add(msgID);
getErrorListener().error(
new GrammarSyntaxMessage(msgID,grammar,token,arg,re)
);
}
public static void internalError(Object error, Throwable e) {
StackTraceElement location = getLastNonErrorManagerCodeLocation(e);
String msg = "Exception "+e+"@"+location+": "+error;
error(MSG_INTERNAL_ERROR, msg);
}
public static void internalError(Object error) {
StackTraceElement location =
getLastNonErrorManagerCodeLocation(new Exception());
String msg = location+": "+error;
error(MSG_INTERNAL_ERROR, msg);
}
public static boolean doNotAttemptAnalysis() {
return !getErrorState().errorMsgIDs.and(ERRORS_FORCING_NO_ANALYSIS).isNil();
}
public static boolean doNotAttemptCodeGen() {
return doNotAttemptAnalysis() ||
!getErrorState().errorMsgIDs.and(ERRORS_FORCING_NO_CODEGEN).isNil();
}
/** Return first non ErrorManager code location for generating messages */
private static StackTraceElement getLastNonErrorManagerCodeLocation(Throwable e) {
StackTraceElement[] stack = e.getStackTrace();
int i = 0;
for (; i < stack.length; i++) {
StackTraceElement t = stack[i];
if ( t.toString().indexOf("ErrorManager")<0 ) {
break;
}
}
StackTraceElement location = stack[i];
return location;
}
// A S S E R T I O N C O D E
public static void assertTrue(boolean condition, String message) {
if ( !condition ) {
internalError(message);
}
}
// S U P P O R T C O D E
protected static boolean initIdToMessageNameMapping() {
// make sure a message exists, even if it's just to indicate a problem
for (int i = 0; i < idToMessageTemplateName.length; i++) {
idToMessageTemplateName[i] = "INVALID MESSAGE ID: "+i;
}
// get list of fields and use it to fill in idToMessageTemplateName mapping
Field[] fields = ErrorManager.class.getFields();
for (int i = 0; i < fields.length; i++) {
Field f = fields[i];
String fieldName = f.getName();
if ( !fieldName.startsWith("MSG_") ) {
continue;
}
String templateName =
fieldName.substring("MSG_".length(),fieldName.length());
int msgID = 0;
try {
// get the constant value from this class object
msgID = f.getInt(ErrorManager.class);
}
catch (IllegalAccessException iae) {
System.err.println("cannot get const value for "+f.getName());
continue;
}
if ( fieldName.startsWith("MSG_") ) {
idToMessageTemplateName[msgID] = templateName;
}
}
return true;
}
/** Use reflection to find list of MSG_ fields and then verify a
* template exists for each one from the locale's group.
*/
protected static boolean verifyMessages() {
boolean ok = true;
Field[] fields = ErrorManager.class.getFields();
for (int i = 0; i < fields.length; i++) {
Field f = fields[i];
String fieldName = f.getName();
String templateName =
fieldName.substring("MSG_".length(),fieldName.length());
if ( fieldName.startsWith("MSG_") ) {
if ( !messages.isDefined(templateName) ) {
System.err.println("Message "+templateName+" in locale "+
locale+" not found");
ok = false;
}
}
}
// check for special templates
if (!messages.isDefined("warning")) {
System.err.println("Message template 'warning' not found in locale "+ locale);
ok = false;
}
if (!messages.isDefined("error")) {
System.err.println("Message template 'error' not found in locale "+ locale);
ok = false;
}
return ok;
}
/** Verify the message format template group */
protected static boolean verifyFormat() {
boolean ok = true;
if (!format.isDefined("location")) {
System.err.println("Format template 'location' not found in " + formatName);
ok = false;
}
if (!format.isDefined("message")) {
System.err.println("Format template 'message' not found in " + formatName);
ok = false;
}
if (!format.isDefined("report")) {
System.err.println("Format template 'report' not found in " + formatName);
ok = false;
}
return ok;
}
/** If there are errors during ErrorManager init, we have no choice
* but to go to System.err.
*/
static void rawError(String msg) {
System.err.println(msg);
}
static void rawError(String msg, Throwable e) {
rawError(msg);
e.printStackTrace(System.err);
}
/** I *think* this will allow Tool subclasses to exit gracefully
* for GUIs etc...
*/
public static void panic() {
Tool tool = (Tool)threadToToolMap.get(Thread.currentThread());
if ( tool==null ) {
// no tool registered, exit
throw new Error("ANTLR ErrorManager panic");
}
else {
tool.panic();
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/NonRegularDecisionMessage.java 0000644 0001750 0001750 00000005120 11256465230 026635 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.DecisionProbe;
import org.antlr.stringtemplate.StringTemplate;
import java.util.*;
/** More a single alternative recurses so this decision is not regular. */
public class NonRegularDecisionMessage extends Message {
public DecisionProbe probe;
public Set<Integer> altsWithRecursion;
public NonRegularDecisionMessage(DecisionProbe probe, Set<Integer> altsWithRecursion) {
super(ErrorManager.MSG_NONREGULAR_DECISION);
this.probe = probe;
this.altsWithRecursion = altsWithRecursion;
}
public String toString() {
GrammarAST decisionASTNode = probe.dfa.getDecisionASTNode();
line = decisionASTNode.getLine();
column = decisionASTNode.getColumn();
String fileName = probe.dfa.nfa.grammar.getFileName();
if ( fileName!=null ) {
file = fileName;
}
StringTemplate st = getMessageTemplate();
String ruleName = probe.dfa.getNFADecisionStartState().enclosingRule.name;
st.setAttribute("ruleName", ruleName);
List sortedAlts = new ArrayList();
sortedAlts.addAll(altsWithRecursion);
Collections.sort(sortedAlts); // make sure it's 1, 2, ...
st.setAttribute("alts", sortedAlts);
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/RandomPhrase.java 0000644 0001750 0001750 00000017447 11256465230 024200 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.*;
import org.antlr.misc.Utils;
import org.antlr.misc.IntervalSet;
import org.antlr.Tool;
import java.util.*;
import java.io.FileReader;
import java.io.BufferedReader;
/** Generate a random phrase given a grammar.
* Usage:
* java org.antlr.tool.RandomPhrase grammarFile.g startRule [seed]
*
* For example:
* java org.antlr.tool.RandomPhrase simple.g program 342
*
* The seed acts like a unique identifier so you can get the same random
* phrase back during unit testing, for example.
*
* If you do not specify a seed then the current time in milliseconds is used
* guaranteeing that you'll never see that seed again.
*
* NOTE: this does not work well for large grammars...it tends to recurse
* too much and build really long strings. I need throttle control; later.
*/
public class RandomPhrase {
public static final boolean debug = false;
protected static Random random;
/** an experimental method to generate random phrases for a given
* grammar given a start rule. Return a list of token types.
*/
protected static void randomPhrase(Grammar g, List<Integer> tokenTypes, String startRule) {
NFAState state = g.getRuleStartState(startRule);
NFAState stopState = g.getRuleStopState(startRule);
Stack ruleInvocationStack = new Stack();
while ( true ) {
if ( state==stopState && ruleInvocationStack.size()==0 ) {
break;
}
if ( debug ) System.out.println("state "+state);
if ( state.getNumberOfTransitions()==0 ) {
if ( debug ) System.out.println("dangling state: "+state);
return;
}
// end of rule node
if ( state.isAcceptState() ) {
NFAState invokingState = (NFAState)ruleInvocationStack.pop();
if ( debug ) System.out.println("pop invoking state "+invokingState);
//System.out.println("leave "+state.enclosingRule.name);
RuleClosureTransition invokingTransition =
(RuleClosureTransition)invokingState.transition[0];
// move to node after state that invoked this rule
state = invokingTransition.followState;
continue;
}
if ( state.getNumberOfTransitions()==1 ) {
// no branching, just take this path
Transition t0 = state.transition[0];
if ( t0 instanceof RuleClosureTransition ) {
ruleInvocationStack.push(state);
if ( debug ) System.out.println("push state "+state);
//System.out.println("call "+((RuleClosureTransition)t0).rule.name);
//System.out.println("stack depth="+ruleInvocationStack.size());
}
else if ( t0.label.isSet() || t0.label.isAtom() ) {
tokenTypes.add( getTokenType(t0.label) );
}
state = (NFAState)t0.target;
continue;
}
int decisionNumber = state.getDecisionNumber();
if ( decisionNumber==0 ) {
System.out.println("weird: no decision number but a choice node");
continue;
}
// decision point, pick ith alternative randomly
int n = g.getNumberOfAltsForDecisionNFA(state);
int randomAlt = random.nextInt(n) + 1;
if ( debug ) System.out.println("randomAlt="+randomAlt);
NFAState altStartState =
g.getNFAStateForAltOfDecision(state, randomAlt);
Transition t = altStartState.transition[0];
state = (NFAState)t.target;
}
}
protected static Integer getTokenType(Label label) {
if ( label.isSet() ) {
// pick random element of set
IntervalSet typeSet = (IntervalSet)label.getSet();
int randomIndex = random.nextInt(typeSet.size());
return typeSet.get(randomIndex);
}
else {
return Utils.integer(label.getAtom());
}
//System.out.println(t0.label.toString(g));
}
/** Used to generate random strings */
public static void main(String[] args) {
if ( args.length < 2 ) {
System.err.println("usage: java org.antlr.tool.RandomPhrase grammarfile startrule");
return;
}
String grammarFileName = args[0];
String startRule = args[1];
long seed = System.currentTimeMillis(); // use random seed unless spec.
if ( args.length==3 ) {
String seedStr = args[2];
seed = Long.parseLong(seedStr);
}
try {
random = new Random(seed);
CompositeGrammar composite = new CompositeGrammar();
Grammar parser = new Grammar(new Tool(), grammarFileName, composite);
composite.setDelegationRoot(parser);
FileReader fr = new FileReader(grammarFileName);
BufferedReader br = new BufferedReader(fr);
parser.parseAndBuildAST(br);
br.close();
parser.composite.assignTokenTypes();
parser.composite.defineGrammarSymbols();
parser.composite.createNFAs();
List leftRecursiveRules = parser.checkAllRulesForLeftRecursion();
if ( leftRecursiveRules.size()>0 ) {
return;
}
if ( parser.getRule(startRule)==null ) {
System.out.println("undefined start rule "+startRule);
return;
}
String lexerGrammarText = parser.getLexerGrammar();
Grammar lexer = new Grammar();
lexer.importTokenVocabulary(parser);
lexer.fileName = grammarFileName;
if ( lexerGrammarText!=null ) {
lexer.setGrammarContent(lexerGrammarText);
}
else {
System.err.println("no lexer grammar found in "+grammarFileName);
}
lexer.buildNFA();
leftRecursiveRules = lexer.checkAllRulesForLeftRecursion();
if ( leftRecursiveRules.size()>0 ) {
return;
}
//System.out.println("lexer:\n"+lexer);
List<Integer> tokenTypes = new ArrayList<Integer>(100);
randomPhrase(parser, tokenTypes, startRule);
System.out.println("token types="+tokenTypes);
for (int i = 0; i < tokenTypes.size(); i++) {
Integer ttypeI = (Integer) tokenTypes.get(i);
int ttype = ttypeI.intValue();
String ttypeDisplayName = parser.getTokenDisplayName(ttype);
if ( Character.isUpperCase(ttypeDisplayName.charAt(0)) ) {
List<Integer> charsInToken = new ArrayList<Integer>(10);
randomPhrase(lexer, charsInToken, ttypeDisplayName);
System.out.print(" ");
for (int j = 0; j < charsInToken.size(); j++) {
java.lang.Integer cI = (java.lang.Integer) charsInToken.get(j);
System.out.print((char)cI.intValue());
}
}
else { // it's a literal
String literal =
ttypeDisplayName.substring(1,ttypeDisplayName.length()-1);
System.out.print(" "+literal);
}
}
System.out.println();
}
catch (Error er) {
System.err.println("Error walking "+grammarFileName+" rule "+startRule+" seed "+seed);
er.printStackTrace(System.err);
}
catch (Exception e) {
System.err.println("Exception walking "+grammarFileName+" rule "+startRule+" seed "+seed);
e.printStackTrace(System.err);
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/AssignTokenTypesBehavior.java 0000644 0001750 0001750 00000025264 11256465230 026543 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.Label;
import org.antlr.misc.Utils;
import java.util.*;
import org.antlr.grammar.v2.AssignTokenTypesWalker;
/** Move all of the functionality from assign.types.g grammar file. */
public class AssignTokenTypesBehavior extends AssignTokenTypesWalker {
protected static final Integer UNASSIGNED = Utils.integer(-1);
protected static final Integer UNASSIGNED_IN_PARSER_RULE = Utils.integer(-2);
protected Map<String,Integer> stringLiterals = new LinkedHashMap();
protected Map<String,Integer> tokens = new LinkedHashMap();
protected Map<String,String> aliases = new LinkedHashMap();
protected Map<String,String> aliasesReverseIndex = new HashMap<String,String>();
/** Track actual lexer rule defs so we don't get repeated token defs in
* generated lexer.
*/
protected Set<String> tokenRuleDefs = new HashSet();
@Override
protected void init(Grammar g) {
this.grammar = g;
currentRuleName = null;
if ( stringAlias==null ) {
// only init once; can't statically init since we need astFactory
initASTPatterns();
}
}
/** Track string literals (could be in tokens{} section) */
@Override
protected void trackString(GrammarAST t) {
// if lexer, don't allow aliasing in tokens section
if ( currentRuleName==null && grammar.type==Grammar.LEXER ) {
ErrorManager.grammarError(ErrorManager.MSG_CANNOT_ALIAS_TOKENS_IN_LEXER,
grammar,
t.token,
t.getText());
return;
}
// in a plain parser grammar rule, cannot reference literals
// (unless defined previously via tokenVocab option)
// don't warn until we hit root grammar as may be defined there.
if ( grammar.getGrammarIsRoot() &&
grammar.type==Grammar.PARSER &&
grammar.getTokenType(t.getText())== Label.INVALID )
{
ErrorManager.grammarError(ErrorManager.MSG_LITERAL_NOT_ASSOCIATED_WITH_LEXER_RULE,
grammar,
t.token,
t.getText());
}
// Don't record literals for lexers, they are things to match not tokens
if ( grammar.type==Grammar.LEXER ) {
return;
}
// otherwise add literal to token types if referenced from parser rule
// or in the tokens{} section
if ( (currentRuleName==null ||
Character.isLowerCase(currentRuleName.charAt(0))) &&
grammar.getTokenType(t.getText())==Label.INVALID )
{
stringLiterals.put(t.getText(), UNASSIGNED_IN_PARSER_RULE);
}
}
@Override
protected void trackToken(GrammarAST t) {
// imported token names might exist, only add if new
// Might have ';'=4 in vocab import and SEMI=';'. Avoid
// setting to UNASSIGNED if we have loaded ';'/SEMI
if ( grammar.getTokenType(t.getText())==Label.INVALID &&
tokens.get(t.getText())==null )
{
tokens.put(t.getText(), UNASSIGNED);
}
}
@Override
protected void trackTokenRule(GrammarAST t,
GrammarAST modifier,
GrammarAST block)
{
// imported token names might exist, only add if new
if ( grammar.type==Grammar.LEXER || grammar.type==Grammar.COMBINED ) {
if ( !Character.isUpperCase(t.getText().charAt(0)) ) {
return;
}
if ( t.getText().equals(Grammar.ARTIFICIAL_TOKENS_RULENAME) ) {
// don't add Tokens rule
return;
}
// track all lexer rules so we can look for token refs w/o
// associated lexer rules.
grammar.composite.lexerRules.add(t.getText());
int existing = grammar.getTokenType(t.getText());
if ( existing==Label.INVALID ) {
tokens.put(t.getText(), UNASSIGNED);
}
// look for "<TOKEN> : <literal> ;" pattern
// (can have optional action last)
if ( block.hasSameTreeStructure(charAlias) ||
block.hasSameTreeStructure(stringAlias) ||
block.hasSameTreeStructure(charAlias2) ||
block.hasSameTreeStructure(stringAlias2) )
{
tokenRuleDefs.add(t.getText());
/*
Grammar parent = grammar.composite.getDelegator(grammar);
boolean importedByParserOrCombined =
parent!=null &&
(parent.type==Grammar.LEXER||parent.type==Grammar.PARSER);
*/
if ( grammar.type==Grammar.COMBINED || grammar.type==Grammar.LEXER ) {
// only call this rule an alias if combined or lexer
alias(t, (GrammarAST)block.getFirstChild().getFirstChild());
}
}
}
// else error
}
@Override
protected void alias(GrammarAST t, GrammarAST s) {
String tokenID = t.getText();
String literal = s.getText();
String prevAliasLiteralID = aliasesReverseIndex.get(literal);
if ( prevAliasLiteralID!=null ) { // we've seen this literal before
if ( tokenID.equals(prevAliasLiteralID) ) {
// duplicate but identical alias; might be tokens {A='a'} and
// lexer rule A : 'a' ; Is ok, just return
return;
}
// give error unless both are rules (ok if one is in tokens section)
if ( !(tokenRuleDefs.contains(tokenID) && tokenRuleDefs.contains(prevAliasLiteralID)) )
{
// don't allow alias if A='a' in tokens section and B : 'a'; is rule.
// Allow if both are rules. Will get DFA nondeterminism error later.
ErrorManager.grammarError(ErrorManager.MSG_TOKEN_ALIAS_CONFLICT,
grammar,
t.token,
tokenID+"="+literal,
prevAliasLiteralID);
}
return; // don't do the alias
}
int existingLiteralType = grammar.getTokenType(literal);
if ( existingLiteralType !=Label.INVALID ) {
// we've seen this before from a tokenVocab most likely
// don't assign a new token type; use existingLiteralType.
tokens.put(tokenID, existingLiteralType);
}
String prevAliasTokenID = aliases.get(tokenID);
if ( prevAliasTokenID!=null ) {
ErrorManager.grammarError(ErrorManager.MSG_TOKEN_ALIAS_REASSIGNMENT,
grammar,
t.token,
tokenID+"="+literal,
prevAliasTokenID);
return; // don't do the alias
}
aliases.put(tokenID, literal);
aliasesReverseIndex.put(literal, tokenID);
}
@Override
public void defineTokens(Grammar root) {
/*
System.out.println("stringLiterals="+stringLiterals);
System.out.println("tokens="+tokens);
System.out.println("aliases="+aliases);
System.out.println("aliasesReverseIndex="+aliasesReverseIndex);
*/
assignTokenIDTypes(root);
aliasTokenIDsAndLiterals(root);
assignStringTypes(root);
/*
System.out.println("stringLiterals="+stringLiterals);
System.out.println("tokens="+tokens);
System.out.println("aliases="+aliases);
*/
defineTokenNamesAndLiteralsInGrammar(root);
}
/*
protected void defineStringLiteralsFromDelegates() {
if ( grammar.getGrammarIsMaster() && grammar.type==Grammar.COMBINED ) {
List<Grammar> delegates = grammar.getDelegates();
System.out.println("delegates in master combined: "+delegates);
for (int i = 0; i < delegates.size(); i++) {
Grammar d = (Grammar) delegates.get(i);
Set<String> literals = d.getStringLiterals();
for (Iterator it = literals.iterator(); it.hasNext();) {
String literal = (String) it.next();
System.out.println("literal "+literal);
int ttype = grammar.getTokenType(literal);
grammar.defineLexerRuleForStringLiteral(literal, ttype);
}
}
}
}
*/
@Override
protected void assignStringTypes(Grammar root) {
// walk string literals assigning types to unassigned ones
Set s = stringLiterals.keySet();
for (Iterator it = s.iterator(); it.hasNext();) {
String lit = (String) it.next();
Integer oldTypeI = (Integer)stringLiterals.get(lit);
int oldType = oldTypeI.intValue();
if ( oldType<Label.MIN_TOKEN_TYPE ) {
Integer typeI = Utils.integer(root.getNewTokenType());
stringLiterals.put(lit, typeI);
// if string referenced in combined grammar parser rule,
// automatically define in the generated lexer
root.defineLexerRuleForStringLiteral(lit, typeI.intValue());
}
}
}
@Override
protected void aliasTokenIDsAndLiterals(Grammar root) {
if ( root.type==Grammar.LEXER ) {
return; // strings/chars are never token types in LEXER
}
// walk aliases if any and assign types to aliased literals if literal
// was referenced
Set s = aliases.keySet();
for (Iterator it = s.iterator(); it.hasNext();) {
String tokenID = (String) it.next();
String literal = (String)aliases.get(tokenID);
if ( literal.charAt(0)=='\'' && stringLiterals.get(literal)!=null ) {
stringLiterals.put(literal, tokens.get(tokenID));
// an alias still means you need a lexer rule for it
Integer typeI = (Integer)tokens.get(tokenID);
if ( !tokenRuleDefs.contains(tokenID) ) {
root.defineLexerRuleForAliasedStringLiteral(tokenID, literal, typeI.intValue());
}
}
}
}
@Override
protected void assignTokenIDTypes(Grammar root) {
// walk token names, assigning values if unassigned
Set s = tokens.keySet();
for (Iterator it = s.iterator(); it.hasNext();) {
String tokenID = (String) it.next();
if ( tokens.get(tokenID)==UNASSIGNED ) {
tokens.put(tokenID, Utils.integer(root.getNewTokenType()));
}
}
}
@Override
protected void defineTokenNamesAndLiteralsInGrammar(Grammar root) {
Set s = tokens.keySet();
for (Iterator it = s.iterator(); it.hasNext();) {
String tokenID = (String) it.next();
int ttype = ((Integer)tokens.get(tokenID)).intValue();
root.defineToken(tokenID, ttype);
}
s = stringLiterals.keySet();
for (Iterator it = s.iterator(); it.hasNext();) {
String lit = (String) it.next();
int ttype = ((Integer)stringLiterals.get(lit)).intValue();
root.defineToken(lit, ttype);
}
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/Strip.java 0000644 0001750 0001750 00000022623 11256465230 022706 0 ustar twerner twerner package org.antlr.tool;
import org.antlr.grammar.v3.ANTLRv3Lexer;
import org.antlr.grammar.v3.ANTLRv3Parser;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.tree.TreeAdaptor;
import org.antlr.runtime.tree.TreeWizard;
import java.util.List;
/** A basic action stripper. */
public class Strip {
protected String filename;
protected TokenRewriteStream tokens;
protected boolean tree_option = false;
protected String args[];
public static void main(String args[]) throws Exception {
Strip s = new Strip(args);
s.parseAndRewrite();
System.out.println(s.tokens);
}
public Strip(String[] args) { this.args = args; }
public TokenRewriteStream getTokenStream() { return tokens; }
public void parseAndRewrite() throws Exception {
processArgs(args);
CharStream input = null;
if ( filename!=null ) input = new ANTLRFileStream(filename);
else input = new ANTLRInputStream(System.in);
// BUILD AST
ANTLRv3Lexer lex = new ANTLRv3Lexer(input);
tokens = new TokenRewriteStream(lex);
ANTLRv3Parser g = new ANTLRv3Parser(tokens);
ANTLRv3Parser.grammarDef_return r = g.grammarDef();
CommonTree t = (CommonTree)r.getTree();
if (tree_option) System.out.println(t.toStringTree());
rewrite(g.getTreeAdaptor(),t,g.getTokenNames());
}
public void rewrite(TreeAdaptor adaptor, CommonTree t, String[] tokenNames) throws Exception {
TreeWizard wiz = new TreeWizard(adaptor, tokenNames);
// ACTIONS STUFF
wiz.visit(t, ANTLRv3Parser.ACTION,
new TreeWizard.Visitor() {
public void visit(Object t) { ACTION(tokens, (CommonTree)t); }
});
wiz.visit(t, ANTLRv3Parser.AT, // ^('@' id ACTION) rule actions
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
CommonTree action = null;
if ( a.getChildCount()==2 ) action = (CommonTree)a.getChild(1);
else if ( a.getChildCount()==3 ) action = (CommonTree)a.getChild(2);
if ( action.getType()==ANTLRv3Parser.ACTION ) {
tokens.delete(a.getTokenStartIndex(),
a.getTokenStopIndex());
killTrailingNewline(tokens, action.getTokenStopIndex());
}
}
});
wiz.visit(t, ANTLRv3Parser.ARG, // wipe rule arguments
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
a = (CommonTree)a.getChild(0);
tokens.delete(a.token.getTokenIndex());
killTrailingNewline(tokens, a.token.getTokenIndex());
}
});
wiz.visit(t, ANTLRv3Parser.RET, // wipe rule return declarations
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
CommonTree ret = (CommonTree)a.getChild(0);
tokens.delete(a.token.getTokenIndex(),
ret.token.getTokenIndex());
}
});
wiz.visit(t, ANTLRv3Parser.SEMPRED, // comment out semantic predicates
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
tokens.replace(a.token.getTokenIndex(), "/*"+a.getText()+"*/");
}
});
wiz.visit(t, ANTLRv3Parser.GATED_SEMPRED, // comment out semantic predicates
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
String text = tokens.toString(a.getTokenStartIndex(),
a.getTokenStopIndex());
tokens.replace(a.getTokenStartIndex(),
a.getTokenStopIndex(),
"/*"+text+"*/");
}
});
wiz.visit(t, ANTLRv3Parser.SCOPE, // comment scope specs
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
tokens.delete(a.getTokenStartIndex(),
a.getTokenStopIndex());
killTrailingNewline(tokens, a.getTokenStopIndex());
}
});
wiz.visit(t, ANTLRv3Parser.ARG_ACTION, // args r[x,y] -> ^(r [x,y])
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
if ( a.getParent().getType()==ANTLRv3Parser.RULE_REF ) {
tokens.delete(a.getTokenStartIndex(),
a.getTokenStopIndex());
}
}
});
wiz.visit(t, ANTLRv3Parser.LABEL_ASSIGN, // ^('=' id ^(RULE_REF [arg])), ...
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
if ( !a.hasAncestor(ANTLRv3Parser.OPTIONS) ) { // avoid options
CommonTree child = (CommonTree)a.getChild(0);
tokens.delete(a.token.getTokenIndex()); // kill "id="
tokens.delete(child.token.getTokenIndex());
}
}
});
wiz.visit(t, ANTLRv3Parser.LIST_LABEL_ASSIGN, // ^('+=' id ^(RULE_REF [arg])), ...
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
CommonTree child = (CommonTree)a.getChild(0);
tokens.delete(a.token.getTokenIndex()); // kill "id+="
tokens.delete(child.token.getTokenIndex());
}
});
// AST STUFF
wiz.visit(t, ANTLRv3Parser.REWRITE,
new TreeWizard.Visitor() {
public void visit(Object t) {
CommonTree a = (CommonTree)t;
CommonTree child = (CommonTree)a.getChild(0);
int stop = child.getTokenStopIndex();
if ( child.getType()==ANTLRv3Parser.SEMPRED ) {
CommonTree rew = (CommonTree)a.getChild(1);
stop = rew.getTokenStopIndex();
}
tokens.delete(a.token.getTokenIndex(), stop);
killTrailingNewline(tokens, stop);
}
});
wiz.visit(t, ANTLRv3Parser.ROOT,
new TreeWizard.Visitor() {
public void visit(Object t) {
tokens.delete(((CommonTree)t).token.getTokenIndex());
}
});
wiz.visit(t, ANTLRv3Parser.BANG,
new TreeWizard.Visitor() {
public void visit(Object t) {
tokens.delete(((CommonTree)t).token.getTokenIndex());
}
});
}
public static void ACTION(TokenRewriteStream tokens, CommonTree t) {
CommonTree parent = (CommonTree)t.getParent();
int ptype = parent.getType();
if ( ptype==ANTLRv3Parser.SCOPE || // we have special rules for these
ptype==ANTLRv3Parser.AT )
{
return;
}
//System.out.println("ACTION: "+t.getText());
CommonTree root = (CommonTree)t.getAncestor(ANTLRv3Parser.RULE);
if ( root!=null ) {
CommonTree rule = (CommonTree)root.getChild(0);
//System.out.println("rule: "+rule);
if ( !Character.isUpperCase(rule.getText().charAt(0)) ) {
tokens.delete(t.getTokenStartIndex(),t.getTokenStopIndex());
killTrailingNewline(tokens, t.token.getTokenIndex());
}
}
}
private static void killTrailingNewline(TokenRewriteStream tokens, int index) {
List all = tokens.getTokens();
Token tok = (Token)all.get(index);
Token after = (Token)all.get(index+1);
String ws = after.getText();
if ( ws.startsWith("\n") ) {
//System.out.println("killing WS after action");
if ( ws.length()>1 ) {
int space = ws.indexOf(' ');
int tab = ws.indexOf('\t');
if ( ws.startsWith("\n") &&
space>=0 || tab>=0 )
{
return; // do nothing if \n + indent
}
// otherwise kill all \n
ws = ws.replaceAll("\n", "");
tokens.replace(after.getTokenIndex(), ws);
}
else {
tokens.delete(after.getTokenIndex());
}
}
}
public void processArgs(String[] args) {
if ( args==null || args.length==0 ) {
help();
return;
}
for (int i = 0; i < args.length; i++) {
if (args[i].equals("-tree")) tree_option = true;
else {
if (args[i].charAt(0) != '-') {
// Must be the grammar file
filename = args[i];
}
}
}
}
private static void help() {
System.err.println("usage: java org.antlr.tool.Strip [args] file.g");
System.err.println(" -tree print out ANTLR grammar AST");
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/NameSpaceChecker.java 0000644 0001750 0001750 00000022551 11256465230 024726 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import antlr.Token;
import org.antlr.analysis.Label;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
public class NameSpaceChecker {
protected Grammar grammar;
public NameSpaceChecker(Grammar grammar) {
this.grammar = grammar;
}
public void checkConflicts() {
for (int i = CompositeGrammar.MIN_RULE_INDEX; i < grammar.composite.ruleIndexToRuleList.size(); i++) {
Rule r = grammar.composite.ruleIndexToRuleList.elementAt(i);
if ( r==null ) {
continue;
}
// walk all labels for Rule r
if ( r.labelNameSpace!=null ) {
Iterator it = r.labelNameSpace.values().iterator();
while ( it.hasNext() ) {
Grammar.LabelElementPair pair = (Grammar.LabelElementPair) it.next();
checkForLabelConflict(r, pair.label);
}
}
// walk rule scope attributes for Rule r
if ( r.ruleScope!=null ) {
List attributes = r.ruleScope.getAttributes();
for (int j = 0; j < attributes.size(); j++) {
Attribute attribute = (Attribute) attributes.get(j);
checkForRuleScopeAttributeConflict(r, attribute);
}
}
checkForRuleDefinitionProblems(r);
checkForRuleArgumentAndReturnValueConflicts(r);
}
// check all global scopes against tokens
Iterator it = grammar.getGlobalScopes().values().iterator();
while (it.hasNext()) {
AttributeScope scope = (AttributeScope) it.next();
checkForGlobalScopeTokenConflict(scope);
}
// check for missing rule, tokens
lookForReferencesToUndefinedSymbols();
}
protected void checkForRuleArgumentAndReturnValueConflicts(Rule r) {
if ( r.returnScope!=null ) {
Set conflictingKeys = r.returnScope.intersection(r.parameterScope);
if (conflictingKeys!=null) {
for (Iterator it = conflictingKeys.iterator(); it.hasNext();) {
String key = (String) it.next();
ErrorManager.grammarError(
ErrorManager.MSG_ARG_RETVAL_CONFLICT,
grammar,
r.tree.getToken(),
key,
r.name);
}
}
}
}
protected void checkForRuleDefinitionProblems(Rule r) {
String ruleName = r.name;
antlr.Token ruleToken = r.tree.getToken();
int msgID = 0;
if ( (grammar.type==Grammar.PARSER||grammar.type==Grammar.TREE_PARSER) &&
Character.isUpperCase(ruleName.charAt(0)) )
{
msgID = ErrorManager.MSG_LEXER_RULES_NOT_ALLOWED;
}
else if ( grammar.type==Grammar.LEXER &&
Character.isLowerCase(ruleName.charAt(0)) &&
!r.isSynPred )
{
msgID = ErrorManager.MSG_PARSER_RULES_NOT_ALLOWED;
}
else if ( grammar.getGlobalScope(ruleName)!=null ) {
msgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
}
if ( msgID!=0 ) {
ErrorManager.grammarError(msgID, grammar, ruleToken, ruleName);
}
}
/** If ref to undefined rule, give error at first occurrence.
*
* Give error if you cannot find the scope override on a rule reference.
*
* If you ref ID in a combined grammar and don't define ID as a lexer rule
* it is an error.
*/
protected void lookForReferencesToUndefinedSymbols() {
// for each rule ref, ask if there is a rule definition
for (Iterator iter = grammar.ruleRefs.iterator(); iter.hasNext();) {
GrammarAST refAST = (GrammarAST)iter.next();
Token tok = refAST.token;
String ruleName = tok.getText();
Rule localRule = grammar.getLocallyDefinedRule(ruleName);
Rule rule = grammar.getRule(ruleName);
if ( localRule==null && rule!=null ) { // imported rule?
grammar.delegatedRuleReferences.add(rule);
rule.imported = true;
}
if ( rule==null && grammar.getTokenType(ruleName)!=Label.EOF ) {
ErrorManager.grammarError(ErrorManager.MSG_UNDEFINED_RULE_REF,
grammar,
tok,
ruleName);
}
}
if ( grammar.type==Grammar.COMBINED ) {
// if we're a combined grammar, we know which token IDs have no
// associated lexer rule.
for (Iterator iter = grammar.tokenIDRefs.iterator(); iter.hasNext();) {
Token tok = (Token) iter.next();
String tokenID = tok.getText();
if ( !grammar.composite.lexerRules.contains(tokenID) &&
grammar.getTokenType(tokenID)!=Label.EOF )
{
ErrorManager.grammarWarning(ErrorManager.MSG_NO_TOKEN_DEFINITION,
grammar,
tok,
tokenID);
}
}
}
// check scopes and scoped rule refs
for (Iterator it = grammar.scopedRuleRefs.iterator(); it.hasNext();) {
GrammarAST scopeAST = (GrammarAST)it.next(); // ^(DOT ID atom)
Grammar scopeG = grammar.composite.getGrammar(scopeAST.getText());
GrammarAST refAST = scopeAST.getChild(1);
String ruleName = refAST.getText();
if ( scopeG==null ) {
ErrorManager.grammarError(ErrorManager.MSG_NO_SUCH_GRAMMAR_SCOPE,
grammar,
scopeAST.getToken(),
scopeAST.getText(),
ruleName);
}
else {
Rule rule = grammar.getRule(scopeG.name, ruleName);
if ( rule==null ) {
ErrorManager.grammarError(ErrorManager.MSG_NO_SUCH_RULE_IN_SCOPE,
grammar,
scopeAST.getToken(),
scopeAST.getText(),
ruleName);
}
}
}
}
protected void checkForGlobalScopeTokenConflict(AttributeScope scope) {
if ( grammar.getTokenType(scope.getName())!=Label.INVALID ) {
ErrorManager.grammarError(ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE,
grammar, null, scope.getName());
}
}
/** Check for collision of a rule-scope dynamic attribute with:
* arg, return value, rule name itself. Labels are checked elsewhere.
*/
public void checkForRuleScopeAttributeConflict(Rule r, Attribute attribute) {
int msgID = 0;
Object arg2 = null;
String attrName = attribute.name;
if ( r.name.equals(attrName) ) {
msgID = ErrorManager.MSG_ATTRIBUTE_CONFLICTS_WITH_RULE;
arg2 = r.name;
}
else if ( (r.returnScope!=null&&r.returnScope.getAttribute(attrName)!=null) ||
(r.parameterScope!=null&&r.parameterScope.getAttribute(attrName)!=null) )
{
msgID = ErrorManager.MSG_ATTRIBUTE_CONFLICTS_WITH_RULE_ARG_RETVAL;
arg2 = r.name;
}
if ( msgID!=0 ) {
ErrorManager.grammarError(msgID,grammar,r.tree.getToken(),attrName,arg2);
}
}
/** Make sure a label doesn't conflict with another symbol.
* Labels must not conflict with: rules, tokens, scope names,
* return values, parameters, and rule-scope dynamic attributes
* defined in surrounding rule.
*/
protected void checkForLabelConflict(Rule r, antlr.Token label) {
int msgID = 0;
Object arg2 = null;
if ( grammar.getGlobalScope(label.getText())!=null ) {
msgID = ErrorManager.MSG_SYMBOL_CONFLICTS_WITH_GLOBAL_SCOPE;
}
else if ( grammar.getRule(label.getText())!=null ) {
msgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_RULE;
}
else if ( grammar.getTokenType(label.getText())!=Label.INVALID ) {
msgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_TOKEN;
}
else if ( r.ruleScope!=null && r.ruleScope.getAttribute(label.getText())!=null ) {
msgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_RULE_SCOPE_ATTRIBUTE;
arg2 = r.name;
}
else if ( (r.returnScope!=null&&r.returnScope.getAttribute(label.getText())!=null) ||
(r.parameterScope!=null&&r.parameterScope.getAttribute(label.getText())!=null) )
{
msgID = ErrorManager.MSG_LABEL_CONFLICTS_WITH_RULE_ARG_RETVAL;
arg2 = r.name;
}
if ( msgID!=0 ) {
ErrorManager.grammarError(msgID,grammar,label,label.getText(),arg2);
}
}
/** If type of previous label differs from new label's type, that's an error.
*/
public boolean checkForLabelTypeMismatch(Rule r, antlr.Token label, int type) {
Grammar.LabelElementPair prevLabelPair =
(Grammar.LabelElementPair)r.labelNameSpace.get(label.getText());
if ( prevLabelPair!=null ) {
// label already defined; if same type, no problem
if ( prevLabelPair.type != type ) {
String typeMismatchExpr =
Grammar.LabelTypeToString[type]+"!="+
Grammar.LabelTypeToString[prevLabelPair.type];
ErrorManager.grammarError(
ErrorManager.MSG_LABEL_TYPE_CONFLICT,
grammar,
label,
label.getText(),
typeMismatchExpr);
return true;
}
}
return false;
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/LeftRecursionCyclesMessage.java 0000644 0001750 0001750 00000004177 11256465230 027045 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.analysis.*;
import antlr.Token;
import java.util.*;
/** Similar to LeftRecursionMessage except this is used for announcing
* cycles found by walking rules without decisions; the other msg is
* invoked when a decision DFA construction finds a problem in closure.
*/
public class LeftRecursionCyclesMessage extends Message {
public Collection cycles;
public LeftRecursionCyclesMessage(Collection cycles) {
super(ErrorManager.MSG_LEFT_RECURSION_CYCLES);
this.cycles = cycles;
}
public String toString() {
StringTemplate st = getMessageTemplate();
st.setAttribute("listOfCycles", cycles);
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarReport.java 0000644 0001750 0001750 00000030412 11256465230 024362 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.DFA;
import org.antlr.runtime.misc.Stats;
import org.antlr.misc.Utils;
import java.util.*;
public class GrammarReport {
/** Because I may change the stats, I need to track that for later
* computations to be consistent.
*/
public static final String Version = "4";
public static final String GRAMMAR_STATS_FILENAME = "grammar.stats";
public static final int NUM_GRAMMAR_STATS = 41;
public static final String newline = System.getProperty("line.separator");
public Grammar grammar;
public GrammarReport(Grammar grammar) {
this.grammar = grammar;
}
/** Create a single-line stats report about this grammar suitable to
* send to the notify page at antlr.org
*/
public String toNotifyString() {
StringBuffer buf = new StringBuffer();
buf.append(Version);
buf.append('\t');
buf.append(grammar.name);
buf.append('\t');
buf.append(grammar.getGrammarTypeString());
buf.append('\t');
buf.append(grammar.getOption("language"));
int totalNonSynPredProductions = 0;
int totalNonSynPredRules = 0;
Collection rules = grammar.getRules();
for (Iterator it = rules.iterator(); it.hasNext();) {
Rule r = (Rule) it.next();
if ( !r.name.toUpperCase()
.startsWith(Grammar.SYNPRED_RULE_PREFIX.toUpperCase()) )
{
totalNonSynPredProductions += r.numberOfAlts;
totalNonSynPredRules++;
}
}
buf.append('\t');
buf.append(totalNonSynPredRules);
buf.append('\t');
buf.append(totalNonSynPredProductions);
int numACyclicDecisions =
grammar.getNumberOfDecisions()-grammar.getNumberOfCyclicDecisions();
int[] depths = new int[numACyclicDecisions];
int[] acyclicDFAStates = new int[numACyclicDecisions];
int[] cyclicDFAStates = new int[grammar.getNumberOfCyclicDecisions()];
int acyclicIndex = 0;
int cyclicIndex = 0;
int numLL1 = 0;
int numDec = 0;
for (int i=1; i<=grammar.getNumberOfDecisions(); i++) {
Grammar.Decision d = grammar.getDecision(i);
if( d.dfa==null ) {
continue;
}
numDec++;
if ( !d.dfa.isCyclic() ) {
int maxk = d.dfa.getMaxLookaheadDepth();
if ( maxk==1 ) {
numLL1++;
}
depths[acyclicIndex] = maxk;
acyclicDFAStates[acyclicIndex] = d.dfa.getNumberOfStates();
acyclicIndex++;
}
else {
cyclicDFAStates[cyclicIndex] = d.dfa.getNumberOfStates();
cyclicIndex++;
}
}
buf.append('\t');
buf.append(numDec);
buf.append('\t');
buf.append(grammar.getNumberOfCyclicDecisions());
buf.append('\t');
buf.append(numLL1);
buf.append('\t');
buf.append(Stats.min(depths));
buf.append('\t');
buf.append(Stats.max(depths));
buf.append('\t');
buf.append(Stats.avg(depths));
buf.append('\t');
buf.append(Stats.stddev(depths));
buf.append('\t');
buf.append(Stats.min(acyclicDFAStates));
buf.append('\t');
buf.append(Stats.max(acyclicDFAStates));
buf.append('\t');
buf.append(Stats.avg(acyclicDFAStates));
buf.append('\t');
buf.append(Stats.stddev(acyclicDFAStates));
buf.append('\t');
buf.append(Stats.sum(acyclicDFAStates));
buf.append('\t');
buf.append(Stats.min(cyclicDFAStates));
buf.append('\t');
buf.append(Stats.max(cyclicDFAStates));
buf.append('\t');
buf.append(Stats.avg(cyclicDFAStates));
buf.append('\t');
buf.append(Stats.stddev(cyclicDFAStates));
buf.append('\t');
buf.append(Stats.sum(cyclicDFAStates));
buf.append('\t');
buf.append(grammar.getTokenTypes().size());
buf.append('\t');
buf.append(grammar.DFACreationWallClockTimeInMS);
buf.append('\t');
buf.append(grammar.numberOfSemanticPredicates);
buf.append('\t');
buf.append(grammar.numberOfManualLookaheadOptions);
buf.append('\t');
buf.append(grammar.setOfNondeterministicDecisionNumbers.size());
buf.append('\t');
buf.append(grammar.setOfNondeterministicDecisionNumbersResolvedWithPredicates.size());
buf.append('\t');
buf.append(grammar.setOfDFAWhoseAnalysisTimedOut.size());
buf.append('\t');
buf.append(ErrorManager.getErrorState().errors);
buf.append('\t');
buf.append(ErrorManager.getErrorState().warnings);
buf.append('\t');
buf.append(ErrorManager.getErrorState().infos);
buf.append('\t');
Map synpreds = grammar.getSyntacticPredicates();
int num_synpreds = synpreds!=null ? synpreds.size() : 0;
buf.append(num_synpreds);
buf.append('\t');
buf.append(grammar.blocksWithSynPreds.size());
buf.append('\t');
buf.append(grammar.decisionsWhoseDFAsUsesSynPreds.size());
buf.append('\t');
buf.append(grammar.blocksWithSemPreds.size());
buf.append('\t');
buf.append(grammar.decisionsWhoseDFAsUsesSemPreds.size());
buf.append('\t');
String output = (String)grammar.getOption("output");
if ( output==null ) {
output = "none";
}
buf.append(output);
buf.append('\t');
Object k = grammar.getOption("k");
if ( k==null ) {
k = "none";
}
buf.append(k);
buf.append('\t');
String backtrack = (String)grammar.getOption("backtrack");
if ( backtrack==null ) {
backtrack = "false";
}
buf.append(backtrack);
return buf.toString();
}
public String getBacktrackingReport() {
StringBuffer buf = new StringBuffer();
buf.append("Backtracking report:");
buf.append(newline);
buf.append("Number of decisions that backtrack: ");
buf.append(grammar.decisionsWhoseDFAsUsesSynPreds.size());
buf.append(newline);
buf.append(getDFALocations(grammar.decisionsWhoseDFAsUsesSynPreds));
return buf.toString();
}
public String getAnalysisTimeoutReport() {
StringBuffer buf = new StringBuffer();
buf.append("NFA conversion early termination report:");
buf.append(newline);
buf.append("Number of NFA conversions that terminated early: ");
buf.append(grammar.setOfDFAWhoseAnalysisTimedOut.size());
buf.append(newline);
buf.append(getDFALocations(grammar.setOfDFAWhoseAnalysisTimedOut));
return buf.toString();
}
protected String getDFALocations(Set dfas) {
Set decisions = new HashSet();
StringBuffer buf = new StringBuffer();
Iterator it = dfas.iterator();
while ( it.hasNext() ) {
DFA dfa = (DFA) it.next();
// if we aborted a DFA and redid with k=1, the backtrackin
if ( decisions.contains(Utils.integer(dfa.decisionNumber)) ) {
continue;
}
decisions.add(Utils.integer(dfa.decisionNumber));
buf.append("Rule ");
buf.append(dfa.decisionNFAStartState.enclosingRule.name);
buf.append(" decision ");
buf.append(dfa.decisionNumber);
buf.append(" location ");
GrammarAST decisionAST =
dfa.decisionNFAStartState.associatedASTNode;
buf.append(decisionAST.getLine());
buf.append(":");
buf.append(decisionAST.getColumn());
buf.append(newline);
}
return buf.toString();
}
/** Given a stats line suitable for sending to the antlr.org site,
* return a human-readable version. Return null if there is a
* problem with the data.
*/
public String toString() {
return toString(toNotifyString());
}
protected static String[] decodeReportData(String data) {
String[] fields = new String[NUM_GRAMMAR_STATS];
StringTokenizer st = new StringTokenizer(data, "\t");
int i = 0;
while ( st.hasMoreTokens() ) {
fields[i] = st.nextToken();
i++;
}
if ( i!=NUM_GRAMMAR_STATS ) {
return null;
}
return fields;
}
public static String toString(String notifyDataLine) {
String[] fields = decodeReportData(notifyDataLine);
if ( fields==null ) {
return null;
}
StringBuffer buf = new StringBuffer();
buf.append("ANTLR Grammar Report; Stats Version ");
buf.append(fields[0]);
buf.append('\n');
buf.append("Grammar: ");
buf.append(fields[1]);
buf.append('\n');
buf.append("Type: ");
buf.append(fields[2]);
buf.append('\n');
buf.append("Target language: ");
buf.append(fields[3]);
buf.append('\n');
buf.append("Output: ");
buf.append(fields[38]);
buf.append('\n');
buf.append("Grammar option k: ");
buf.append(fields[39]);
buf.append('\n');
buf.append("Grammar option backtrack: ");
buf.append(fields[40]);
buf.append('\n');
buf.append("Rules: ");
buf.append(fields[4]);
buf.append('\n');
buf.append("Productions: ");
buf.append(fields[5]);
buf.append('\n');
buf.append("Decisions: ");
buf.append(fields[6]);
buf.append('\n');
buf.append("Cyclic DFA decisions: ");
buf.append(fields[7]);
buf.append('\n');
buf.append("LL(1) decisions: "); buf.append(fields[8]);
buf.append('\n');
buf.append("Min fixed k: "); buf.append(fields[9]);
buf.append('\n');
buf.append("Max fixed k: "); buf.append(fields[10]);
buf.append('\n');
buf.append("Average fixed k: "); buf.append(fields[11]);
buf.append('\n');
buf.append("Standard deviation of fixed k: "); buf.append(fields[12]);
buf.append('\n');
buf.append("Min acyclic DFA states: "); buf.append(fields[13]);
buf.append('\n');
buf.append("Max acyclic DFA states: "); buf.append(fields[14]);
buf.append('\n');
buf.append("Average acyclic DFA states: "); buf.append(fields[15]);
buf.append('\n');
buf.append("Standard deviation of acyclic DFA states: "); buf.append(fields[16]);
buf.append('\n');
buf.append("Total acyclic DFA states: "); buf.append(fields[17]);
buf.append('\n');
buf.append("Min cyclic DFA states: "); buf.append(fields[18]);
buf.append('\n');
buf.append("Max cyclic DFA states: "); buf.append(fields[19]);
buf.append('\n');
buf.append("Average cyclic DFA states: "); buf.append(fields[20]);
buf.append('\n');
buf.append("Standard deviation of cyclic DFA states: "); buf.append(fields[21]);
buf.append('\n');
buf.append("Total cyclic DFA states: "); buf.append(fields[22]);
buf.append('\n');
buf.append("Vocabulary size: ");
buf.append(fields[23]);
buf.append('\n');
buf.append("DFA creation time in ms: ");
buf.append(fields[24]);
buf.append('\n');
buf.append("Number of semantic predicates found: ");
buf.append(fields[25]);
buf.append('\n');
buf.append("Number of manual fixed lookahead k=value options: ");
buf.append(fields[26]);
buf.append('\n');
buf.append("Number of nondeterministic decisions: ");
buf.append(fields[27]);
buf.append('\n');
buf.append("Number of nondeterministic decisions resolved with predicates: ");
buf.append(fields[28]);
buf.append('\n');
buf.append("Number of DFA conversions terminated early: ");
buf.append(fields[29]);
buf.append('\n');
buf.append("Number of errors: ");
buf.append(fields[30]);
buf.append('\n');
buf.append("Number of warnings: ");
buf.append(fields[31]);
buf.append('\n');
buf.append("Number of infos: ");
buf.append(fields[32]);
buf.append('\n');
buf.append("Number of syntactic predicates found: ");
buf.append(fields[33]);
buf.append('\n');
buf.append("Decisions with syntactic predicates: ");
buf.append(fields[34]);
buf.append('\n');
buf.append("Decision DFAs using syntactic predicates: ");
buf.append(fields[35]);
buf.append('\n');
buf.append("Decisions with semantic predicates: ");
buf.append(fields[36]);
buf.append('\n');
buf.append("Decision DFAs using semantic predicates: ");
buf.append(fields[37]);
buf.append('\n');
return buf.toString();
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarAnalysisAbortedMessage.java 0000644 0001750 0001750 00000004422 11256465230 027502 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.analysis.DecisionProbe;
/** Reports the condition that ANTLR's LL(*) analysis engine terminated
* early.
*/
public class GrammarAnalysisAbortedMessage extends Message {
public DecisionProbe probe;
public GrammarAnalysisAbortedMessage(DecisionProbe probe) {
super(ErrorManager.MSG_ANALYSIS_ABORTED);
this.probe = probe;
}
public String toString() {
GrammarAST decisionASTNode = probe.dfa.getDecisionASTNode();
line = decisionASTNode.getLine();
column = decisionASTNode.getColumn();
String fileName = probe.dfa.nfa.grammar.getFileName();
if ( fileName!=null ) {
file = fileName;
}
StringTemplate st = getMessageTemplate();
st.setAttribute("enclosingRule",
probe.dfa.getNFADecisionStartState().enclosingRule.name);
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarSyntaxMessage.java 0000644 0001750 0001750 00000005550 11256465230 025707 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
import antlr.Token;
/** A problem with the syntax of your antlr grammar such as
* "The '{' came as a complete surprise to me at this point in your program"
*/
public class GrammarSyntaxMessage extends Message {
public Grammar g;
/** Most of the time, we'll have a token and so this will be set. */
public Token offendingToken;
public antlr.RecognitionException exception;
public GrammarSyntaxMessage(int msgID,
Grammar grammar,
Token offendingToken,
antlr.RecognitionException exception)
{
this(msgID,grammar,offendingToken,null,exception);
}
public GrammarSyntaxMessage(int msgID,
Grammar grammar,
Token offendingToken,
Object arg,
antlr.RecognitionException exception)
{
super(msgID, arg, null);
this.offendingToken = offendingToken;
this.exception = exception;
this.g = grammar;
}
public String toString() {
line = 0;
column = 0;
if ( offendingToken!=null ) {
line = offendingToken.getLine();
column = offendingToken.getColumn();
}
// TODO: actually set the right Grammar instance to get the filename
// TODO: have to update all v2 grammar files for this. or use errormanager and tool to get the current grammar
if (g != null) {
file = g.getFileName();
}
StringTemplate st = getMessageTemplate();
if ( arg!=null ) {
st.setAttribute("arg", arg);
}
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/Interpreter.java 0000644 0001750 0001750 00000034543 11256465230 024114 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.DFA;
import org.antlr.analysis.*;
import org.antlr.runtime.*;
import org.antlr.runtime.debug.DebugEventListener;
import org.antlr.runtime.debug.BlankDebugEventListener;
import org.antlr.runtime.tree.ParseTree;
import org.antlr.runtime.debug.ParseTreeBuilder;
import org.antlr.misc.IntervalSet;
import java.util.List;
import java.util.Stack;
/** The recognition interpreter/engine for grammars. Separated
* out of Grammar as it's related, but technically not a Grammar function.
* You create an interpreter for a grammar and an input stream. This object
* can act as a TokenSource so that you can hook up two grammars (via
* a CommonTokenStream) to lex/parse. Being a token source only makes sense
* for a lexer grammar of course.
*/
public class Interpreter implements TokenSource {
protected Grammar grammar;
protected IntStream input;
/** A lexer listener that just creates token objects as they
* are matched. scan() use this listener to get a single object.
* To get a stream of tokens, you must call scan() multiple times,
* recording the token object result after each call.
*/
class LexerActionGetTokenType extends BlankDebugEventListener {
public CommonToken token;
Grammar g;
public LexerActionGetTokenType(Grammar g) {
this.g = g;
}
public void exitRule(String grammarFileName, String ruleName) {
if ( !ruleName.equals(Grammar.ARTIFICIAL_TOKENS_RULENAME) ){
int type = g.getTokenType(ruleName);
int channel = Token.DEFAULT_CHANNEL;
token = new CommonToken((CharStream)input,type,channel,0,0);
}
}
}
public Interpreter(Grammar grammar, IntStream input) {
this.grammar = grammar;
this.input = input;
}
public Token nextToken() {
if ( grammar.type!=Grammar.LEXER ) {
return null;
}
if ( input.LA(1)==CharStream.EOF ) {
return Token.EOF_TOKEN;
}
int start = input.index();
int charPos = ((CharStream)input).getCharPositionInLine();
CommonToken token = null;
loop:
while (input.LA(1)!=CharStream.EOF) {
try {
token = scan(Grammar.ARTIFICIAL_TOKENS_RULENAME, null);
break;
}
catch (RecognitionException re) {
// report a problem and try for another
reportScanError(re);
continue loop;
}
}
// the scan can only set type
// we must set the line, and other junk here to make it a complete token
int stop = input.index()-1;
if ( token==null ) {
return Token.EOF_TOKEN;
}
token.setLine(((CharStream)input).getLine());
token.setStartIndex(start);
token.setStopIndex(stop);
token.setCharPositionInLine(charPos);
return token;
}
/** For a given input char stream, try to match against the NFA
* starting at startRule. This is a deterministic parse even though
* it is using an NFA because it uses DFAs at each decision point to
* predict which alternative will succeed. This is exactly what the
* generated parser will do.
*
* This only does lexer grammars.
*
* Return the token type associated with the final rule end state.
*/
public void scan(String startRule,
DebugEventListener actions,
List visitedStates)
throws RecognitionException
{
if ( grammar.type!=Grammar.LEXER ) {
return;
}
CharStream in = (CharStream)this.input;
//System.out.println("scan("+startRule+",'"+in.substring(in.index(),in.size()-1)+"')");
// Build NFAs/DFAs from the grammar AST if NFAs haven't been built yet
if ( grammar.getRuleStartState(startRule)==null ) {
grammar.buildNFA();
}
if ( !grammar.allDecisionDFAHaveBeenCreated() ) {
// Create the DFA predictors for each decision
grammar.createLookaheadDFAs();
}
// do the parse
Stack ruleInvocationStack = new Stack();
NFAState start = grammar.getRuleStartState(startRule);
NFAState stop = grammar.getRuleStopState(startRule);
parseEngine(startRule, start, stop, in, ruleInvocationStack,
actions, visitedStates);
}
public CommonToken scan(String startRule)
throws RecognitionException
{
return scan(startRule, null);
}
public CommonToken scan(String startRule,
List visitedStates)
throws RecognitionException
{
LexerActionGetTokenType actions = new LexerActionGetTokenType(grammar);
scan(startRule, actions, visitedStates);
return actions.token;
}
public void parse(String startRule,
DebugEventListener actions,
List visitedStates)
throws RecognitionException
{
//System.out.println("parse("+startRule+")");
// Build NFAs/DFAs from the grammar AST if NFAs haven't been built yet
if ( grammar.getRuleStartState(startRule)==null ) {
grammar.buildNFA();
}
if ( !grammar.allDecisionDFAHaveBeenCreated() ) {
// Create the DFA predictors for each decision
grammar.createLookaheadDFAs();
}
// do the parse
Stack ruleInvocationStack = new Stack();
NFAState start = grammar.getRuleStartState(startRule);
NFAState stop = grammar.getRuleStopState(startRule);
parseEngine(startRule, start, stop, input, ruleInvocationStack,
actions, visitedStates);
}
public ParseTree parse(String startRule)
throws RecognitionException
{
return parse(startRule, null);
}
public ParseTree parse(String startRule, List visitedStates)
throws RecognitionException
{
ParseTreeBuilder actions = new ParseTreeBuilder(grammar.name);
try {
parse(startRule, actions, visitedStates);
}
catch (RecognitionException re) {
// Errors are tracked via the ANTLRDebugInterface
// Exceptions are used just to blast out of the parse engine
// The error will be in the parse tree.
}
return actions.getTree();
}
/** Fill a list of all NFA states visited during the parse */
protected void parseEngine(String startRule,
NFAState start,
NFAState stop,
IntStream input,
Stack ruleInvocationStack,
DebugEventListener actions,
List visitedStates)
throws RecognitionException
{
NFAState s = start;
if ( actions!=null ) {
actions.enterRule(s.nfa.grammar.getFileName(), start.enclosingRule.name);
}
int t = input.LA(1);
while ( s!=stop ) {
if ( visitedStates!=null ) {
visitedStates.add(s);
}
/*
System.out.println("parse state "+s.stateNumber+" input="+
s.nfa.grammar.getTokenDisplayName(t));
*/
// CASE 1: decision state
if ( s.getDecisionNumber()>0 && s.nfa.grammar.getNumberOfAltsForDecisionNFA(s)>1 ) {
// decision point, must predict and jump to alt
DFA dfa = s.nfa.grammar.getLookaheadDFA(s.getDecisionNumber());
/*
if ( s.nfa.grammar.type!=Grammar.LEXER ) {
System.out.println("decision: "+
dfa.getNFADecisionStartState().getDescription()+
" input="+s.nfa.grammar.getTokenDisplayName(t));
}
*/
int m = input.mark();
int predictedAlt = predict(dfa);
if ( predictedAlt == NFA.INVALID_ALT_NUMBER ) {
String description = dfa.getNFADecisionStartState().getDescription();
NoViableAltException nvae =
new NoViableAltException(description,
dfa.getDecisionNumber(),
s.stateNumber,
input);
if ( actions!=null ) {
actions.recognitionException(nvae);
}
input.consume(); // recover
throw nvae;
}
input.rewind(m);
int parseAlt =
s.translateDisplayAltToWalkAlt(predictedAlt);
/*
if ( s.nfa.grammar.type!=Grammar.LEXER ) {
System.out.println("predicted alt "+predictedAlt+", parseAlt "+
parseAlt);
}
*/
NFAState alt;
if ( parseAlt > s.nfa.grammar.getNumberOfAltsForDecisionNFA(s) ) {
// implied branch of loop etc...
alt = s.nfa.grammar.nfa.getState( s.endOfBlockStateNumber );
}
else {
alt = s.nfa.grammar.getNFAStateForAltOfDecision(s, parseAlt);
}
s = (NFAState)alt.transition[0].target;
continue;
}
// CASE 2: finished matching a rule
if ( s.isAcceptState() ) { // end of rule node
if ( actions!=null ) {
actions.exitRule(s.nfa.grammar.getFileName(), s.enclosingRule.name);
}
if ( ruleInvocationStack.empty() ) {
// done parsing. Hit the start state.
//System.out.println("stack empty in stop state for "+s.getEnclosingRule());
break;
}
// pop invoking state off the stack to know where to return to
NFAState invokingState = (NFAState)ruleInvocationStack.pop();
RuleClosureTransition invokingTransition =
(RuleClosureTransition)invokingState.transition[0];
// move to node after state that invoked this rule
s = invokingTransition.followState;
continue;
}
Transition trans = s.transition[0];
Label label = trans.label;
if ( label.isSemanticPredicate() ) {
FailedPredicateException fpe =
new FailedPredicateException(input,
s.enclosingRule.name,
"can't deal with predicates yet");
if ( actions!=null ) {
actions.recognitionException(fpe);
}
}
// CASE 3: epsilon transition
if ( label.isEpsilon() ) {
// CASE 3a: rule invocation state
if ( trans instanceof RuleClosureTransition ) {
ruleInvocationStack.push(s);
s = (NFAState)trans.target;
//System.out.println("call "+s.enclosingRule.name+" from "+s.nfa.grammar.getFileName());
if ( actions!=null ) {
actions.enterRule(s.nfa.grammar.getFileName(), s.enclosingRule.name);
}
// could be jumping to new grammar, make sure DFA created
if ( !s.nfa.grammar.allDecisionDFAHaveBeenCreated() ) {
s.nfa.grammar.createLookaheadDFAs();
}
}
// CASE 3b: plain old epsilon transition, just move
else {
s = (NFAState)trans.target;
}
}
// CASE 4: match label on transition
else if ( label.matches(t) ) {
if ( actions!=null ) {
if ( s.nfa.grammar.type == Grammar.PARSER ||
s.nfa.grammar.type == Grammar.COMBINED )
{
actions.consumeToken(((TokenStream)input).LT(1));
}
}
s = (NFAState)s.transition[0].target;
input.consume();
t = input.LA(1);
}
// CASE 5: error condition; label is inconsistent with input
else {
if ( label.isAtom() ) {
MismatchedTokenException mte =
new MismatchedTokenException(label.getAtom(), input);
if ( actions!=null ) {
actions.recognitionException(mte);
}
input.consume(); // recover
throw mte;
}
else if ( label.isSet() ) {
MismatchedSetException mse =
new MismatchedSetException(((IntervalSet)label.getSet()).toRuntimeBitSet(),
input);
if ( actions!=null ) {
actions.recognitionException(mse);
}
input.consume(); // recover
throw mse;
}
else if ( label.isSemanticPredicate() ) {
FailedPredicateException fpe =
new FailedPredicateException(input,
s.enclosingRule.name,
label.getSemanticContext().toString());
if ( actions!=null ) {
actions.recognitionException(fpe);
}
input.consume(); // recover
throw fpe;
}
else {
throw new RecognitionException(input); // unknown error
}
}
}
//System.out.println("hit stop state for "+stop.getEnclosingRule());
if ( actions!=null ) {
actions.exitRule(s.nfa.grammar.getFileName(), stop.enclosingRule.name);
}
}
/** Given an input stream, return the unique alternative predicted by
* matching the input. Upon error, return NFA.INVALID_ALT_NUMBER
* The first symbol of lookahead is presumed to be primed; that is,
* input.lookahead(1) must point at the input symbol you want to start
* predicting with.
*/
public int predict(DFA dfa) {
DFAState s = dfa.startState;
int c = input.LA(1);
Transition eotTransition = null;
dfaLoop:
while ( !s.isAcceptState() ) {
/*
System.out.println("DFA.predict("+s.getStateNumber()+", "+
dfa.getNFA().getGrammar().getTokenName(c)+")");
*/
// for each edge of s, look for intersection with current char
for (int i=0; i<s.getNumberOfTransitions(); i++) {
Transition t = s.transition(i);
// special case: EOT matches any char
if ( t.label.matches(c) ) {
// take transition i
s = (DFAState)t.target;
input.consume();
c = input.LA(1);
continue dfaLoop;
}
if ( t.label.getAtom()==Label.EOT ) {
eotTransition = t;
}
}
if ( eotTransition!=null ) {
s = (DFAState)eotTransition.target;
continue dfaLoop;
}
/*
ErrorManager.error(ErrorManager.MSG_NO_VIABLE_DFA_ALT,
s,
dfa.nfa.grammar.getTokenName(c));
*/
return NFA.INVALID_ALT_NUMBER;
}
// woohoo! We know which alt to predict
// nothing emanates from a stop state; must terminate anyway
/*
System.out.println("DFA stop state "+s.getStateNumber()+" predicts "+
s.getUniquelyPredictedAlt());
*/
return s.getUniquelyPredictedAlt();
}
public void reportScanError(RecognitionException re) {
CharStream cs = (CharStream)input;
// print as good of a message as we can, given that we do not have
// a Lexer object and, hence, cannot call the routine to get a
// decent error message.
System.err.println("problem matching token at "+
cs.getLine()+":"+cs.getCharPositionInLine()+" "+re);
}
public String getSourceName() {
return input.getSourceName();
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/Interp.java 0000644 0001750 0001750 00000010122 11256465230 023035 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.runtime.ANTLRFileStream;
import org.antlr.runtime.CharStream;
import org.antlr.runtime.CommonTokenStream;
import org.antlr.runtime.tree.ParseTree;
import org.antlr.Tool;
import java.util.StringTokenizer;
import java.util.List;
import java.io.FileReader;
import java.io.BufferedReader;
/** Interpret any ANTLR grammar:
*
* java Interp file.g tokens-to-ignore start-rule input-file
*
* java Interp C.g 'WS COMMENT' program t.c
*
* where the WS and COMMENT are the names of tokens you want to have
* the parser ignore.
*/
public class Interp {
// pass me a java file to parse
public static void main(String[] args) throws Exception {
if ( args.length!=4 ) {
System.err.println("java Interp file.g tokens-to-ignore start-rule input-file");
return;
}
String grammarFileName = args[0];
String ignoreTokens = args[1];
String startRule = args[2];
String inputFileName = args[3];
// TODO: using wrong constructor now
Tool tool = new Tool();
CompositeGrammar composite = new CompositeGrammar();
Grammar parser = new Grammar(tool, grammarFileName, composite);
composite.setDelegationRoot(parser);
FileReader fr = new FileReader(grammarFileName);
BufferedReader br = new BufferedReader(fr);
parser.parseAndBuildAST(br);
br.close();
parser.composite.assignTokenTypes();
parser.composite.defineGrammarSymbols();
parser.composite.createNFAs();
List leftRecursiveRules = parser.checkAllRulesForLeftRecursion();
if ( leftRecursiveRules.size()>0 ) {
return;
}
if ( parser.getRule(startRule)==null ) {
System.out.println("undefined start rule "+startRule);
return;
}
String lexerGrammarText = parser.getLexerGrammar();
Grammar lexer = new Grammar();
lexer.importTokenVocabulary(parser);
lexer.fileName = grammarFileName;
lexer.setTool(tool);
if ( lexerGrammarText!=null ) {
lexer.setGrammarContent(lexerGrammarText);
}
else {
System.err.println("no lexer grammar found in "+grammarFileName);
}
lexer.composite.createNFAs();
CharStream input =
new ANTLRFileStream(inputFileName);
Interpreter lexEngine = new Interpreter(lexer, input);
CommonTokenStream tokens = new CommonTokenStream(lexEngine);
StringTokenizer tk = new StringTokenizer(ignoreTokens, " ");
while ( tk.hasMoreTokens() ) {
String tokenName = tk.nextToken();
tokens.setTokenTypeChannel(lexer.getTokenType(tokenName), 99);
}
if ( parser.getRule(startRule)==null ) {
System.err.println("Rule "+startRule+" does not exist in "+grammarFileName);
return;
}
Interpreter parseEngine = new Interpreter(parser, tokens);
ParseTree t = parseEngine.parse(startRule);
System.out.println(t.toStringTree());
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarUnreachableAltsMessage.java 0000644 0001750 0001750 00000006413 11256465230 027455 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
import org.antlr.analysis.DecisionProbe;
import org.antlr.analysis.DFAState;
import org.antlr.analysis.NFAState;
import org.antlr.analysis.SemanticContext;
import antlr.Token;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
/** Reports a potential parsing issue with a decision; the decision is
* nondeterministic in some way.
*/
public class GrammarUnreachableAltsMessage extends Message {
public DecisionProbe probe;
public List alts;
public GrammarUnreachableAltsMessage(DecisionProbe probe,
List alts)
{
super(ErrorManager.MSG_UNREACHABLE_ALTS);
this.probe = probe;
this.alts = alts;
// flip msg ID if alts are actually token refs in Tokens rule
if ( probe.dfa.isTokensRuleDecision() ) {
setMessageID(ErrorManager.MSG_UNREACHABLE_TOKENS);
}
}
public String toString() {
GrammarAST decisionASTNode = probe.dfa.getDecisionASTNode();
line = decisionASTNode.getLine();
column = decisionASTNode.getColumn();
String fileName = probe.dfa.nfa.grammar.getFileName();
if ( fileName!=null ) {
file = fileName;
}
StringTemplate st = getMessageTemplate();
if ( probe.dfa.isTokensRuleDecision() ) {
// alts are token rules, convert to the names instead of numbers
for (int i = 0; i < alts.size(); i++) {
Integer altI = (Integer) alts.get(i);
String tokenName =
probe.getTokenNameForTokensRuleAlt(altI.intValue());
// reset the line/col to the token definition
NFAState ruleStart =
probe.dfa.nfa.grammar.getRuleStartState(tokenName);
line = ruleStart.associatedASTNode.getLine();
column = ruleStart.associatedASTNode.getColumn();
st.setAttribute("tokens", tokenName);
}
}
else {
// regular alt numbers, show the alts
st.setAttribute("alts", alts);
}
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarSemanticsMessage.java 0000644 0001750 0001750 00000005330 11256465230 026343 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
import antlr.Token;
/** A problem with the symbols and/or meaning of a grammar such as rule
* redefinition.
*/
public class GrammarSemanticsMessage extends Message {
public Grammar g;
/** Most of the time, we'll have a token such as an undefined rule ref
* and so this will be set.
*/
public Token offendingToken;
public GrammarSemanticsMessage(int msgID,
Grammar g,
Token offendingToken)
{
this(msgID,g,offendingToken,null,null);
}
public GrammarSemanticsMessage(int msgID,
Grammar g,
Token offendingToken,
Object arg)
{
this(msgID,g,offendingToken,arg,null);
}
public GrammarSemanticsMessage(int msgID,
Grammar g,
Token offendingToken,
Object arg,
Object arg2)
{
super(msgID,arg,arg2);
this.g = g;
this.offendingToken = offendingToken;
}
public String toString() {
line = 0;
column = 0;
if ( offendingToken!=null ) {
line = offendingToken.getLine();
column = offendingToken.getColumn();
}
if ( g!=null ) {
file = g.getFileName();
}
StringTemplate st = getMessageTemplate();
if ( arg!=null ) {
st.setAttribute("arg", arg);
}
if ( arg2!=null ) {
st.setAttribute("arg2", arg2);
}
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarSanity.java 0000644 0001750 0001750 00000025600 11256465230 024361 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.NFAState;
import org.antlr.analysis.Transition;
import org.antlr.analysis.RuleClosureTransition;
import java.util.List;
import java.util.HashSet;
import java.util.ArrayList;
import java.util.Set;
import org.antlr.grammar.v2.ANTLRParser;
/** Factor out routines that check sanity of rules, alts, grammars, etc.. */
public class GrammarSanity {
/** The checkForLeftRecursion method needs to track what rules it has
* visited to track infinite recursion.
*/
protected Set<Rule> visitedDuringRecursionCheck = null;
protected Grammar grammar;
public GrammarSanity(Grammar grammar) {
this.grammar = grammar;
}
/** Check all rules for infinite left recursion before analysis. Return list
* of troublesome rule cycles. This method has two side-effects: it notifies
* the error manager that we have problems and it sets the list of
* recursive rules that we should ignore during analysis.
*/
public List<Set<Rule>> checkAllRulesForLeftRecursion() {
grammar.buildNFA(); // make sure we have NFAs
grammar.leftRecursiveRules = new HashSet();
List<Set<Rule>> listOfRecursiveCycles = new ArrayList();
for (int i = 0; i < grammar.composite.ruleIndexToRuleList.size(); i++) {
Rule r = grammar.composite.ruleIndexToRuleList.elementAt(i);
if ( r!=null ) {
visitedDuringRecursionCheck = new HashSet();
visitedDuringRecursionCheck.add(r);
Set visitedStates = new HashSet();
traceStatesLookingForLeftRecursion(r.startState,
visitedStates,
listOfRecursiveCycles);
}
}
if ( listOfRecursiveCycles.size()>0 ) {
ErrorManager.leftRecursionCycles(listOfRecursiveCycles);
}
return listOfRecursiveCycles;
}
/** From state s, look for any transition to a rule that is currently
* being traced. When tracing r, visitedDuringRecursionCheck has r
* initially. If you reach an accept state, return but notify the
* invoking rule that it is nullable, which implies that invoking
* rule must look at follow transition for that invoking state.
* The visitedStates tracks visited states within a single rule so
* we can avoid epsilon-loop-induced infinite recursion here. Keep
* filling the cycles in listOfRecursiveCycles and also, as a
* side-effect, set leftRecursiveRules.
*/
protected boolean traceStatesLookingForLeftRecursion(NFAState s,
Set visitedStates,
List<Set<Rule>> listOfRecursiveCycles)
{
if ( s.isAcceptState() ) {
// this rule must be nullable!
// At least one epsilon edge reached accept state
return true;
}
if ( visitedStates.contains(s) ) {
// within same rule, we've hit same state; quit looping
return false;
}
visitedStates.add(s);
boolean stateReachesAcceptState = false;
Transition t0 = s.transition[0];
if ( t0 instanceof RuleClosureTransition ) {
RuleClosureTransition refTrans = (RuleClosureTransition)t0;
Rule refRuleDef = refTrans.rule;
//String targetRuleName = ((NFAState)t0.target).getEnclosingRule();
if ( visitedDuringRecursionCheck.contains(refRuleDef) ) {
// record left-recursive rule, but don't go back in
grammar.leftRecursiveRules.add(refRuleDef);
/*
System.out.println("already visited "+refRuleDef+", calling from "+
s.enclosingRule);
*/
addRulesToCycle(refRuleDef,
s.enclosingRule,
listOfRecursiveCycles);
}
else {
// must visit if not already visited; send new visitedStates set
visitedDuringRecursionCheck.add(refRuleDef);
boolean callReachedAcceptState =
traceStatesLookingForLeftRecursion((NFAState)t0.target,
new HashSet(),
listOfRecursiveCycles);
// we're back from visiting that rule
visitedDuringRecursionCheck.remove(refRuleDef);
// must keep going in this rule then
if ( callReachedAcceptState ) {
NFAState followingState =
((RuleClosureTransition) t0).followState;
stateReachesAcceptState |=
traceStatesLookingForLeftRecursion(followingState,
visitedStates,
listOfRecursiveCycles);
}
}
}
else if ( t0.label.isEpsilon() || t0.label.isSemanticPredicate() ) {
stateReachesAcceptState |=
traceStatesLookingForLeftRecursion((NFAState)t0.target, visitedStates, listOfRecursiveCycles);
}
// else it has a labeled edge
// now do the other transition if it exists
Transition t1 = s.transition[1];
if ( t1!=null ) {
stateReachesAcceptState |=
traceStatesLookingForLeftRecursion((NFAState)t1.target,
visitedStates,
listOfRecursiveCycles);
}
return stateReachesAcceptState;
}
/** enclosingRuleName calls targetRuleName, find the cycle containing
* the target and add the caller. Find the cycle containing the caller
* and add the target. If no cycles contain either, then create a new
* cycle. listOfRecursiveCycles is List<Set<String>> that holds a list
* of cycles (sets of rule names).
*/
protected void addRulesToCycle(Rule targetRule,
Rule enclosingRule,
List<Set<Rule>> listOfRecursiveCycles)
{
boolean foundCycle = false;
for (int i = 0; i < listOfRecursiveCycles.size(); i++) {
Set<Rule> rulesInCycle = listOfRecursiveCycles.get(i);
// ensure both rules are in same cycle
if ( rulesInCycle.contains(targetRule) ) {
rulesInCycle.add(enclosingRule);
foundCycle = true;
}
if ( rulesInCycle.contains(enclosingRule) ) {
rulesInCycle.add(targetRule);
foundCycle = true;
}
}
if ( !foundCycle ) {
Set cycle = new HashSet();
cycle.add(targetRule);
cycle.add(enclosingRule);
listOfRecursiveCycles.add(cycle);
}
}
public void checkRuleReference(GrammarAST scopeAST,
GrammarAST refAST,
GrammarAST argsAST,
String currentRuleName)
{
Rule r = grammar.getRule(refAST.getText());
if ( refAST.getType()==ANTLRParser.RULE_REF ) {
if ( argsAST!=null ) {
// rule[args]; ref has args
if ( r!=null && r.argActionAST==null ) {
// but rule def has no args
ErrorManager.grammarError(
ErrorManager.MSG_RULE_HAS_NO_ARGS,
grammar,
argsAST.getToken(),
r.name);
}
}
else {
// rule ref has no args
if ( r!=null && r.argActionAST!=null ) {
// but rule def has args
ErrorManager.grammarError(
ErrorManager.MSG_MISSING_RULE_ARGS,
grammar,
refAST.getToken(),
r.name);
}
}
}
else if ( refAST.getType()==ANTLRParser.TOKEN_REF ) {
if ( grammar.type!=Grammar.LEXER ) {
if ( argsAST!=null ) {
// args on a token ref not in a lexer rule
ErrorManager.grammarError(
ErrorManager.MSG_ARGS_ON_TOKEN_REF,
grammar,
refAST.getToken(),
refAST.getText());
}
return; // ignore token refs in nonlexers
}
if ( argsAST!=null ) {
// tokenRef[args]; ref has args
if ( r!=null && r.argActionAST==null ) {
// but token rule def has no args
ErrorManager.grammarError(
ErrorManager.MSG_RULE_HAS_NO_ARGS,
grammar,
argsAST.getToken(),
r.name);
}
}
else {
// token ref has no args
if ( r!=null && r.argActionAST!=null ) {
// but token rule def has args
ErrorManager.grammarError(
ErrorManager.MSG_MISSING_RULE_ARGS,
grammar,
refAST.getToken(),
r.name);
}
}
}
}
/** Rules in tree grammar that use -> rewrites and are spitting out
* templates via output=template and then use rewrite=true must only
* use -> on alts that are simple nodes or trees or single rule refs
* that match either nodes or trees. The altAST is the ALT node
* for an ALT. Verify that its first child is simple. Must be either
* ( ALT ^( A B ) <end-of-alt> ) or ( ALT A <end-of-alt> ) or
* other element.
*
* Ignore predicates in front and labels.
*/
public void ensureAltIsSimpleNodeOrTree(GrammarAST altAST,
GrammarAST elementAST,
int outerAltNum)
{
if ( isValidSimpleElementNode(elementAST) ) {
GrammarAST next = (GrammarAST)elementAST.getNextSibling();
if ( !isNextNonActionElementEOA(next)) {
ErrorManager.grammarWarning(ErrorManager.MSG_REWRITE_FOR_MULTI_ELEMENT_ALT,
grammar,
next.token,
new Integer(outerAltNum));
}
return;
}
switch ( elementAST.getType() ) {
case ANTLRParser.ASSIGN : // labels ok on non-rule refs
case ANTLRParser.PLUS_ASSIGN :
if ( isValidSimpleElementNode(elementAST.getChild(1)) ) {
return;
}
break;
case ANTLRParser.ACTION : // skip past actions
case ANTLRParser.SEMPRED :
case ANTLRParser.SYN_SEMPRED :
case ANTLRParser.BACKTRACK_SEMPRED :
case ANTLRParser.GATED_SEMPRED :
ensureAltIsSimpleNodeOrTree(altAST,
(GrammarAST)elementAST.getNextSibling(),
outerAltNum);
return;
}
ErrorManager.grammarWarning(ErrorManager.MSG_REWRITE_FOR_MULTI_ELEMENT_ALT,
grammar,
elementAST.token,
new Integer(outerAltNum));
}
protected boolean isValidSimpleElementNode(GrammarAST t) {
switch ( t.getType() ) {
case ANTLRParser.TREE_BEGIN :
case ANTLRParser.TOKEN_REF :
case ANTLRParser.CHAR_LITERAL :
case ANTLRParser.STRING_LITERAL :
case ANTLRParser.WILDCARD :
return true;
default :
return false;
}
}
protected boolean isNextNonActionElementEOA(GrammarAST t) {
while ( t.getType()==ANTLRParser.ACTION ||
t.getType()==ANTLRParser.SEMPRED )
{
t = (GrammarAST)t.getNextSibling();
}
if ( t.getType()==ANTLRParser.EOA ) {
return true;
}
return false;
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/ToolMessage.java 0000644 0001750 0001750 00000005362 11256465230 024030 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.stringtemplate.StringTemplate;
/** A generic message from the tool such as "file not found" type errors; there
* is no reason to create a special object for each error unlike the grammar
* errors, which may be rather complex.
*
* Sometimes you need to pass in a filename or something to say it is "bad".
* Allow a generic object to be passed in and the string template can deal
* with just printing it or pulling a property out of it.
*
* TODO what to do with exceptions? Want stack trace for internal errors?
*/
public class ToolMessage extends Message {
public ToolMessage(int msgID) {
super(msgID, null, null);
}
public ToolMessage(int msgID, Object arg) {
super(msgID, arg, null);
}
public ToolMessage(int msgID, Throwable e) {
super(msgID);
this.e = e;
}
public ToolMessage(int msgID, Object arg, Object arg2) {
super(msgID, arg, arg2);
}
public ToolMessage(int msgID, Object arg, Throwable e) {
super(msgID,arg,null);
this.e = e;
}
public String toString() {
StringTemplate st = getMessageTemplate();
if ( arg!=null ) {
st.setAttribute("arg", arg);
}
if ( arg2!=null ) {
st.setAttribute("arg2", arg2);
}
if ( e!=null ) {
st.setAttribute("exception", e);
st.setAttribute("stackTrace", e.getStackTrace());
}
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/tool/ANTLRErrorListener.java 0000644 0001750 0001750 00000004025 11256465230 025201 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
/** Defines behavior of object able to handle error messages from ANTLR including
* both tool errors like "can't write file" and grammar ambiguity warnings.
* To avoid having to change tools that use ANTLR (like GUIs), I am
* wrapping error data in Message objects and passing them to the listener.
* In this way, users of this interface are less sensitive to changes in
* the info I need for error messages.
*/
public interface ANTLRErrorListener {
public void info(String msg);
public void error(Message msg);
public void warning(Message msg);
public void error(ToolMessage msg);
}
antlr-3.2/tool/src/main/java/org/antlr/tool/GrammarNonDeterminismMessage.java 0000644 0001750 0001750 00000011265 11256465230 027354 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr.tool;
import org.antlr.analysis.DFAState;
import org.antlr.analysis.DecisionProbe;
import org.antlr.analysis.NFAState;
import org.antlr.stringtemplate.StringTemplate;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
/** Reports a potential parsing issue with a decision; the decision is
* nondeterministic in some way.
*/
public class GrammarNonDeterminismMessage extends Message {
public DecisionProbe probe;
public DFAState problemState;
public GrammarNonDeterminismMessage(DecisionProbe probe,
DFAState problemState)
{
super(ErrorManager.MSG_GRAMMAR_NONDETERMINISM);
this.probe = probe;
this.problemState = problemState;
// flip msg ID if alts are actually token refs in Tokens rule
if ( probe.dfa.isTokensRuleDecision() ) {
setMessageID(ErrorManager.MSG_TOKEN_NONDETERMINISM);
}
}
public String toString() {
GrammarAST decisionASTNode = probe.dfa.getDecisionASTNode();
line = decisionASTNode.getLine();
column = decisionASTNode.getColumn();
String fileName = probe.dfa.nfa.grammar.getFileName();
if ( fileName!=null ) {
file = fileName;
}
StringTemplate st = getMessageTemplate();
// Now fill template with information about problemState
List labels = probe.getSampleNonDeterministicInputSequence(problemState);
String input = probe.getInputSequenceDisplay(labels);
st.setAttribute("input", input);
if ( probe.dfa.isTokensRuleDecision() ) {
Set disabledAlts = probe.getDisabledAlternatives(problemState);
for (Iterator it = disabledAlts.iterator(); it.hasNext();) {
Integer altI = (Integer) it.next();
String tokenName =
probe.getTokenNameForTokensRuleAlt(altI.intValue());
// reset the line/col to the token definition (pick last one)
NFAState ruleStart =
probe.dfa.nfa.grammar.getRuleStartState(tokenName);
line = ruleStart.associatedASTNode.getLine();
column = ruleStart.associatedASTNode.getColumn();
st.setAttribute("disabled", tokenName);
}
}
else {
st.setAttribute("disabled", probe.getDisabledAlternatives(problemState));
}
List nondetAlts = probe.getNonDeterministicAltsForState(problemState);
NFAState nfaStart = probe.dfa.getNFADecisionStartState();
// all state paths have to begin with same NFA state
int firstAlt = 0;
if ( nondetAlts!=null ) {
for (Iterator iter = nondetAlts.iterator(); iter.hasNext();) {
Integer displayAltI = (Integer) iter.next();
if ( DecisionProbe.verbose ) {
int tracePathAlt =
nfaStart.translateDisplayAltToWalkAlt(displayAltI.intValue());
if ( firstAlt == 0 ) {
firstAlt = tracePathAlt;
}
List path =
probe.getNFAPathStatesForAlt(firstAlt,
tracePathAlt,
labels);
st.setAttribute("paths.{alt,states}",
displayAltI, path);
}
else {
if ( probe.dfa.isTokensRuleDecision() ) {
// alts are token rules, convert to the names instead of numbers
String tokenName =
probe.getTokenNameForTokensRuleAlt(displayAltI.intValue());
st.setAttribute("conflictingTokens", tokenName);
}
else {
st.setAttribute("conflictingAlts", displayAltI);
}
}
}
}
st.setAttribute("hasPredicateBlockedByAction", problemState.dfa.hasPredicateBlockedByAction);
return super.toString(st);
}
}
antlr-3.2/tool/src/main/java/org/antlr/Tool.java 0000644 0001750 0001750 00000154354 11256465226 021561 0 ustar twerner twerner /*
[The "BSD licence"]
Copyright (c) 2005-2008 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
package org.antlr;
import antlr.TokenStreamException;
import antlr.RecognitionException;
import antlr.ANTLRException;
import org.antlr.analysis.*;
import org.antlr.codegen.CodeGenerator;
import org.antlr.runtime.misc.Stats;
import org.antlr.tool.*;
import org.antlr.misc.Graph;
import java.io.*;
import java.util.*;
/** The main ANTLR entry point. Read a grammar and generate a parser. */
public class Tool {
public final Properties antlrSettings = new Properties();
public String VERSION = "!Unknown version!";
//public static final String VERSION = "${project.version}";
public static final String UNINITIALIZED_DIR = "<unset-dir>";
private List<String> grammarFileNames = new ArrayList<String>();
private boolean generate_NFA_dot = false;
private boolean generate_DFA_dot = false;
private String outputDirectory = ".";
private boolean haveOutputDir = false;
private String inputDirectory = null;
private String parentGrammarDirectory;
private String grammarOutputDirectory;
private boolean haveInputDir = false;
private String libDirectory = ".";
private boolean debug = false;
private boolean trace = false;
private boolean profile = false;
private boolean report = false;
private boolean printGrammar = false;
private boolean depend = false;
private boolean forceAllFilesToOutputDir = false;
private boolean forceRelativeOutput = false;
protected boolean deleteTempLexer = true;
private boolean verbose = false;
/** Don't process grammar file if generated files are newer than grammar */
private boolean make = false;
private boolean showBanner = true;
private static boolean exitNow = false;
// The internal options are for my use on the command line during dev
//
public static boolean internalOption_PrintGrammarTree = false;
public static boolean internalOption_PrintDFA = false;
public static boolean internalOption_ShowNFAConfigsInDFA = false;
public static boolean internalOption_watchNFAConversion = false;
/**
* A list of dependency generators that are accumulated aaaas (and if) the
* tool is required to sort the provided grammars into build dependency order.
protected Map<String, BuildDependencyGenerator> buildDependencyGenerators;
*/
public static void main(String[] args) {
Tool antlr = new Tool(args);
if (!exitNow) {
antlr.process();
if (ErrorManager.getNumErrors() > 0) {
System.exit(1);
}
System.exit(0);
}
}
/**
* Load the properties file org/antlr/antlr.properties and populate any
* variables that must be initialized from it, such as the version of ANTLR.
*/
private void loadResources() {
InputStream in = null;
in = this.getClass().getResourceAsStream("antlr.properties");
// If we found the resource, then load it, otherwise revert to the
// defaults.
//
if (in != null) {
try {
// Load the resources into the map
//
antlrSettings.load(in);
// Set any variables that we need to populate from the resources
//
VERSION = antlrSettings.getProperty("antlr.version");
} catch (Exception e) {
// Do nothing, just leave the defaults in place
}
}
}
public Tool() {
loadResources();
}
public Tool(String[] args) {
loadResources();
// Set all the options and pick up all the named grammar files
//
processArgs(args);
}
public void processArgs(String[] args) {
if (isVerbose()) {
ErrorManager.info("ANTLR Parser Generator Version " + VERSION);
showBanner = false;
}
if (args == null || args.length == 0) {
help();
return;
}
for (int i = 0; i < args.length; i++) {
if (args[i].equals("-o") || args[i].equals("-fo")) {
if (i + 1 >= args.length) {
System.err.println("missing output directory with -fo/-o option; ignoring");
}
else {
if (args[i].equals("-fo")) { // force output into dir
setForceAllFilesToOutputDir(true);
}
i++;
outputDirectory = args[i];
if (outputDirectory.endsWith("/") ||
outputDirectory.endsWith("\\")) {
outputDirectory =
outputDirectory.substring(0, getOutputDirectory().length() - 1);
}
File outDir = new File(outputDirectory);
haveOutputDir = true;
if (outDir.exists() && !outDir.isDirectory()) {
ErrorManager.error(ErrorManager.MSG_OUTPUT_DIR_IS_FILE, outputDirectory);
setLibDirectory(".");
}
}
}
else if (args[i].equals("-lib")) {
if (i + 1 >= args.length) {
System.err.println("missing library directory with -lib option; ignoring");
}
else {
i++;
setLibDirectory(args[i]);
if (getLibraryDirectory().endsWith("/") ||
getLibraryDirectory().endsWith("\\")) {
setLibDirectory(getLibraryDirectory().substring(0, getLibraryDirectory().length() - 1));
}
File outDir = new File(getLibraryDirectory());
if (!outDir.exists()) {
ErrorManager.error(ErrorManager.MSG_DIR_NOT_FOUND, getLibraryDirectory());
setLibDirectory(".");
}
}
}
else if (args[i].equals("-nfa")) {
setGenerate_NFA_dot(true);
}
else if (args[i].equals("-dfa")) {
setGenerate_DFA_dot(true);
}
else if (args[i].equals("-debug")) {
setDebug(true);
}
else if (args[i].equals("-trace")) {
setTrace(true);
}
else if (args[i].equals("-report")) {
setReport(true);
}
else if (args[i].equals("-profile")) {
setProfile(true);
}
else if (args[i].equals("-print")) {
setPrintGrammar(true);
}
else if (args[i].equals("-depend")) {
setDepend(true);
}
else if (args[i].equals("-verbose")) {
setVerbose(true);
}
else if (args[i].equals("-version")) {
version();
exitNow = true;
}
else if (args[i].equals("-make")) {
setMake(true);
}
else if (args[i].equals("-message-format")) {
if (i + 1 >= args.length) {
System.err.println("missing output format with -message-format option; using default");
}
else {
i++;
ErrorManager.setFormat(args[i]);
}
}
else if (args[i].equals("-Xgrtree")) {
internalOption_PrintGrammarTree = true; // print grammar tree
}
else if (args[i].equals("-Xdfa")) {
internalOption_PrintDFA = true;
}
else if (args[i].equals("-Xnoprune")) {
DFAOptimizer.PRUNE_EBNF_EXIT_BRANCHES = false;
}
else if (args[i].equals("-Xnocollapse")) {
DFAOptimizer.COLLAPSE_ALL_PARALLEL_EDGES = false;
}
else if (args[i].equals("-Xdbgconversion")) {
NFAToDFAConverter.debug = true;
}
else if (args[i].equals("-Xmultithreaded")) {
NFAToDFAConverter.SINGLE_THREADED_NFA_CONVERSION = false;
}
else if (args[i].equals("-Xnomergestopstates")) {
DFAOptimizer.MERGE_STOP_STATES = false;
}
else if (args[i].equals("-Xdfaverbose")) {
internalOption_ShowNFAConfigsInDFA = true;
}
else if (args[i].equals("-Xwatchconversion")) {
internalOption_watchNFAConversion = true;
}
else if (args[i].equals("-XdbgST")) {
CodeGenerator.EMIT_TEMPLATE_DELIMITERS = true;
}
else if (args[i].equals("-Xmaxinlinedfastates")) {
if (i + 1 >= args.length) {
System.err.println("missing max inline dfa states -Xmaxinlinedfastates option; ignoring");
}
else {
i++;
CodeGenerator.MAX_ACYCLIC_DFA_STATES_INLINE = Integer.parseInt(args[i]);
}
}
else if (args[i].equals("-Xmaxswitchcaselabels")) {
if (i + 1 >= args.length) {
System.err.println("missing max switch case labels -Xmaxswitchcaselabels option; ignoring");
}
else {
i++;
CodeGenerator.MAX_SWITCH_CASE_LABELS = Integer.parseInt(args[i]);
}
}
else if (args[i].equals("-Xminswitchalts")) {
if (i + 1 >= args.length) {
System.err.println("missing min switch alternatives -Xminswitchalts option; ignoring");
}
else {
i++;
CodeGenerator.MIN_SWITCH_ALTS = Integer.parseInt(args[i]);
}
}
else if (args[i].equals("-Xm")) {
if (i + 1 >= args.length) {
System.err.println("missing max recursion with -Xm option; ignoring");
}
else {
i++;
NFAContext.MAX_SAME_RULE_INVOCATIONS_PER_NFA_CONFIG_STACK = Integer.parseInt(args[i]);
}
}
else if (args[i].equals("-Xmaxdfaedges")) {
if (i + 1 >= args.length) {
System.err.println("missing max number of edges with -Xmaxdfaedges option; ignoring");
}
else {
i++;
DFA.MAX_STATE_TRANSITIONS_FOR_TABLE = Integer.parseInt(args[i]);
}
}
else if (args[i].equals("-Xconversiontimeout")) {
if (i + 1 >= args.length) {
System.err.println("missing max time in ms -Xconversiontimeout option; ignoring");
}
else {
i++;
DFA.MAX_TIME_PER_DFA_CREATION = Integer.parseInt(args[i]);
}
}
else if (args[i].equals("-Xnfastates")) {
DecisionProbe.verbose = true;
}
else if (args[i].equals("-X")) {
Xhelp();
}
else {
if (args[i].charAt(0) != '-') {
// Must be the grammar file
addGrammarFile(args[i]);
}
}
}
}
/*
protected void checkForInvalidArguments(String[] args, BitSet cmdLineArgValid) {
// check for invalid command line args
for (int a = 0; a < args.length; a++) {
if (!cmdLineArgValid.member(a)) {
System.err.println("invalid command-line argument: " + args[a] + "; ignored");
}
}
}
*/
/**
* Checks to see if the list of outputFiles all exist, and have
* last-modified timestamps which are later than the last-modified
* timestamp of all the grammar files involved in build the output
* (imports must be checked). If these conditions hold, the method
* returns false, otherwise, it returns true.
*
* @param grammarFileName The grammar file we are checking
*/
public boolean buildRequired(String grammarFileName)
throws IOException, ANTLRException
{
BuildDependencyGenerator bd =
new BuildDependencyGenerator(this, grammarFileName);
List<File> outputFiles = bd.getGeneratedFileList();
List<File> inputFiles = bd.getDependenciesFileList();
// Note that input directory must be set to use buildRequired
File grammarFile;
if (haveInputDir) {
grammarFile = new File(inputDirectory, grammarFileName);
}
else {
grammarFile = new File(grammarFileName);
}
long grammarLastModified = grammarFile.lastModified();
for (File outputFile : outputFiles) {
if (!outputFile.exists() || grammarLastModified > outputFile.lastModified()) {
// One of the output files does not exist or is out of date, so we must build it
return true;
}
// Check all of the imported grammars and see if any of these are younger
// than any of the output files.
if (inputFiles != null) {
for (File inputFile : inputFiles) {
if (inputFile.lastModified() > outputFile.lastModified()) {
// One of the imported grammar files has been updated so we must build
return true;
}
}
}
}
if (isVerbose()) {
System.out.println("Grammar " + grammarFile + " is up to date - build skipped");
}
return false;
}
public void process() {
boolean exceptionWhenWritingLexerFile = false;
String lexerGrammarFileName = null; // necessary at this scope to have access in the catch below
// Have to be tricky here when Maven or build tools call in and must new Tool()
// before setting options. The banner won't display that way!
if (isVerbose() && showBanner) {
ErrorManager.info("ANTLR Parser Generator Version " + VERSION);
showBanner = false;
}
try {
sortGrammarFiles(); // update grammarFileNames
}
catch (Exception e) {
ErrorManager.error(ErrorManager.MSG_INTERNAL_ERROR,e);
}
catch (Error e) {
ErrorManager.error(ErrorManager.MSG_INTERNAL_ERROR, e);
}
for (String grammarFileName : grammarFileNames) {
// If we are in make mode (to support build tools like Maven) and the
// file is already up to date, then we do not build it (and in verbose mode
// we will say so).
if (make) {
try {
if ( !buildRequired(grammarFileName) ) continue;
}
catch (Exception e) {
ErrorManager.error(ErrorManager.MSG_INTERNAL_ERROR,e);
}
}
if (isVerbose() && !isDepend()) {
System.out.println(grammarFileName);
}
try {
if (isDepend()) {
BuildDependencyGenerator dep =
new BuildDependencyGenerator(this, grammarFileName);
/*
List outputFiles = dep.getGeneratedFileList();
List dependents = dep.getDependenciesFileList();
System.out.println("output: "+outputFiles);
System.out.println("dependents: "+dependents);
*/
System.out.println(dep.getDependencies());
continue;
}
Grammar grammar = getRootGrammar(grammarFileName);
// we now have all grammars read in as ASTs
// (i.e., root and all delegates)
grammar.composite.assignTokenTypes();
grammar.composite.defineGrammarSymbols();
grammar.composite.createNFAs();
generateRecognizer(grammar);
if (isPrintGrammar()) {
grammar.printGrammar(System.out);
}
if (isReport()) {
GrammarReport greport = new GrammarReport(grammar);
System.out.println(greport.toString());
// print out a backtracking report too (that is not encoded into log)
System.out.println(greport.getBacktrackingReport());
// same for aborted NFA->DFA conversions
System.out.println(greport.getAnalysisTimeoutReport());
}
if (isProfile()) {
GrammarReport greport = new GrammarReport(grammar);
Stats.writeReport(GrammarReport.GRAMMAR_STATS_FILENAME,
greport.toNotifyString());
}
// now handle the lexer if one was created for a merged spec
String lexerGrammarStr = grammar.getLexerGrammar();
//System.out.println("lexer grammar:\n"+lexerGrammarStr);
if (grammar.type == Grammar.COMBINED && lexerGrammarStr != null) {
lexerGrammarFileName = grammar.getImplicitlyGeneratedLexerFileName();
try {
Writer w = getOutputFile(grammar, lexerGrammarFileName);
w.write(lexerGrammarStr);
w.close();
}
catch (IOException e) {
// emit different error message when creating the implicit lexer fails
// due to write permission error
exceptionWhenWritingLexerFile = true;
throw e;
}
try {
StringReader sr = new StringReader(lexerGrammarStr);
Grammar lexerGrammar = new Grammar();
lexerGrammar.composite.watchNFAConversion = internalOption_watchNFAConversion;
lexerGrammar.implicitLexer = true;
lexerGrammar.setTool(this);
File lexerGrammarFullFile =
new File(getFileDirectory(lexerGrammarFileName), lexerGrammarFileName);
lexerGrammar.setFileName(lexerGrammarFullFile.toString());
lexerGrammar.importTokenVocabulary(grammar);
lexerGrammar.parseAndBuildAST(sr);
sr.close();
lexerGrammar.composite.assignTokenTypes();
lexerGrammar.composite.defineGrammarSymbols();
lexerGrammar.composite.createNFAs();
generateRecognizer(lexerGrammar);
}
finally {
// make sure we clean up
if (deleteTempLexer) {
File outputDir = getOutputDirectory(lexerGrammarFileName);
File outputFile = new File(outputDir, lexerGrammarFileName);
outputFile.delete();
}
}
}
}
catch (IOException e) {
if (exceptionWhenWritingLexerFile) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE,
lexerGrammarFileName, e);
}
else {
ErrorManager.error(ErrorManager.MSG_CANNOT_OPEN_FILE,
grammarFileName);
}
}
catch (Exception e) {
ErrorManager.error(ErrorManager.MSG_INTERNAL_ERROR, grammarFileName, e);
}
/*
finally {
System.out.println("creates="+ Interval.creates);
System.out.println("hits="+ Interval.hits);
System.out.println("misses="+ Interval.misses);
System.out.println("outOfRange="+ Interval.outOfRange);
}
*/
}
}
public void sortGrammarFiles() throws IOException {
//System.out.println("Grammar names "+getGrammarFileNames());
Graph g = new Graph();
List<String> missingFiles = new ArrayList<String>();
for (String gfile : grammarFileNames) {
try {
GrammarSpelunker grammar = new GrammarSpelunker(inputDirectory, gfile);
grammar.parse();
String vocabName = grammar.getTokenVocab();
String grammarName = grammar.getGrammarName();
// Make all grammars depend on any tokenVocab options
if ( vocabName!=null ) g.addEdge(gfile, vocabName+CodeGenerator.VOCAB_FILE_EXTENSION);
// Make all generated tokens files depend on their grammars
g.addEdge(grammarName+CodeGenerator.VOCAB_FILE_EXTENSION, gfile);
}
catch (FileNotFoundException fnfe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_OPEN_FILE, gfile);
missingFiles.add(gfile);
}
}
List<Object> sorted = g.sort();
//System.out.println("sorted="+sorted);
grammarFileNames.clear(); // wipe so we can give new ordered list
for (int i = 0; i < sorted.size(); i++) {
String f = (String)sorted.get(i);
if ( missingFiles.contains(f) ) continue;
if ( !(f.endsWith(".g") || f.endsWith(".g3")) ) continue;
grammarFileNames.add(f);
}
//System.out.println("new grammars="+grammarFileNames);
}
/** Get a grammar mentioned on the command-line and any delegates */
public Grammar getRootGrammar(String grammarFileName)
throws IOException
{
//StringTemplate.setLintMode(true);
// grammars mentioned on command line are either roots or single grammars.
// create the necessary composite in case it's got delegates; even
// single grammar needs it to get token types.
CompositeGrammar composite = new CompositeGrammar();
Grammar grammar = new Grammar(this, grammarFileName, composite);
composite.setDelegationRoot(grammar);
FileReader fr = null;
File f = null;
if (haveInputDir) {
f = new File(inputDirectory, grammarFileName);
}
else {
f = new File(grammarFileName);
}
// Store the location of this grammar as if we import files, we can then
// search for imports in the same location as the original grammar as well as in
// the lib directory.
//
parentGrammarDirectory = f.getParent();
if (grammarFileName.lastIndexOf(File.separatorChar) == -1) {
grammarOutputDirectory = ".";
}
else {
grammarOutputDirectory = grammarFileName.substring(0, grammarFileName.lastIndexOf(File.separatorChar));
}
fr = new FileReader(f);
BufferedReader br = new BufferedReader(fr);
grammar.parseAndBuildAST(br);
composite.watchNFAConversion = internalOption_watchNFAConversion;
br.close();
fr.close();
return grammar;
}
/** Create NFA, DFA and generate code for grammar.
* Create NFA for any delegates first. Once all NFA are created,
* it's ok to create DFA, which must check for left-recursion. That check
* is done by walking the full NFA, which therefore must be complete.
* After all NFA, comes DFA conversion for root grammar then code gen for
* root grammar. DFA and code gen for delegates comes next.
*/
protected void generateRecognizer(Grammar grammar) {
String language = (String) grammar.getOption("language");
if (language != null) {
CodeGenerator generator = new CodeGenerator(this, grammar, language);
grammar.setCodeGenerator(generator);
generator.setDebug(isDebug());
generator.setProfile(isProfile());
generator.setTrace(isTrace());
// generate NFA early in case of crash later (for debugging)
if (isGenerate_NFA_dot()) {
generateNFAs(grammar);
}
// GENERATE CODE
generator.genRecognizer();
if (isGenerate_DFA_dot()) {
generateDFAs(grammar);
}
List<Grammar> delegates = grammar.getDirectDelegates();
for (int i = 0; delegates != null && i < delegates.size(); i++) {
Grammar delegate = (Grammar) delegates.get(i);
if (delegate != grammar) { // already processing this one
generateRecognizer(delegate);
}
}
}
}
public void generateDFAs(Grammar g) {
for (int d = 1; d <= g.getNumberOfDecisions(); d++) {
DFA dfa = g.getLookaheadDFA(d);
if (dfa == null) {
continue; // not there for some reason, ignore
}
DOTGenerator dotGenerator = new DOTGenerator(g);
String dot = dotGenerator.getDOT(dfa.startState);
String dotFileName = g.name + "." + "dec-" + d;
if (g.implicitLexer) {
dotFileName = g.name + Grammar.grammarTypeToFileNameSuffix[g.type] + "." + "dec-" + d;
}
try {
writeDOTFile(g, dotFileName, dot);
} catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_GEN_DOT_FILE,
dotFileName,
ioe);
}
}
}
protected void generateNFAs(Grammar g) {
DOTGenerator dotGenerator = new DOTGenerator(g);
Collection rules = g.getAllImportedRules();
rules.addAll(g.getRules());
for (Iterator itr = rules.iterator(); itr.hasNext();) {
Rule r = (Rule) itr.next();
try {
String dot = dotGenerator.getDOT(r.startState);
if (dot != null) {
writeDOTFile(g, r, dot);
}
} catch (IOException ioe) {
ErrorManager.error(ErrorManager.MSG_CANNOT_WRITE_FILE, ioe);
}
}
}
protected void writeDOTFile(Grammar g, Rule r, String dot) throws IOException {
writeDOTFile(g, r.grammar.name + "." + r.name, dot);
}
protected void writeDOTFile(Grammar g, String name, String dot) throws IOException {
Writer fw = getOutputFile(g, name + ".dot");
fw.write(dot);
fw.close();
}
private static void version() {
ErrorManager.info("ANTLR Parser Generator Version " + new Tool().VERSION);
}
private static void help() {
ErrorManager.info("ANTLR Parser Generator Version " + new Tool().VERSION);
System.err.println("usage: java org.antlr.Tool [args] file.g [file2.g file3.g ...]");
System.err.println(" -o outputDir specify output directory where all output is generated");
System.err.println(" -fo outputDir same as -o but force even files with relative paths to dir");
System.err.println(" -lib dir specify location of token files");
System.err.println(" -depend generate file dependencies");
System.err.println(" -report print out a report about the grammar(s) processed");
System.err.println(" -print print out the grammar without actions");
System.err.println(" -debug generate a parser that emits debugging events");
System.err.println(" -profile generate a parser that computes profiling information");
System.err.println(" -nfa generate an NFA for each rule");
System.err.println(" -dfa generate a DFA for each decision point");
System.err.println(" -message-format name specify output style for messages");
System.err.println(" -verbose generate ANTLR version and other information");
System.err.println(" -make only build if generated files older than grammar");
System.err.println(" -version print the version of ANTLR and exit.");
System.err.println(" -X display extended argument list");
}
private static void Xhelp() {
ErrorManager.info("ANTLR Parser Generator Version " + new Tool().VERSION);
System.err.println(" -Xgrtree print the grammar AST");
System.err.println(" -Xdfa print DFA as text ");
System.err.println(" -Xnoprune test lookahead against EBNF block exit branches");
System.err.println(" -Xnocollapse collapse incident edges into DFA states");
System.err.println(" -Xdbgconversion dump lots of info during NFA conversion");
System.err.println(" -Xmultithreaded run the analysis in 2 threads");
System.err.println(" -Xnomergestopstates do not merge stop states");
System.err.println(" -Xdfaverbose generate DFA states in DOT with NFA configs");
System.err.println(" -Xwatchconversion print a message for each NFA before converting");
System.err.println(" -XdbgST put tags at start/stop of all templates in output");
System.err.println(" -Xnfastates for nondeterminisms, list NFA states for each path");
System.err.println(" -Xm m max number of rule invocations during conversion [" + NFAContext.MAX_SAME_RULE_INVOCATIONS_PER_NFA_CONFIG_STACK + "]");
System.err.println(" -Xmaxdfaedges m max \"comfortable\" number of edges for single DFA state [" + DFA.MAX_STATE_TRANSITIONS_FOR_TABLE + "]");
System.err.println(" -Xconversiontimeout t set NFA conversion timeout (ms) for each decision [" + DFA.MAX_TIME_PER_DFA_CREATION + "]");
System.err.println(" -Xmaxinlinedfastates m max DFA states before table used rather than inlining [" + CodeGenerator.MADSI_DEFAULT +"]");
System.err.println(" -Xmaxswitchcaselabels m don't generate switch() statements for dfas bigger than m [" + CodeGenerator.MSCL_DEFAULT +"]");
System.err.println(" -Xminswitchalts m don't generate switch() statements for dfas smaller than m [" + CodeGenerator.MSA_DEFAULT + "]");
}
/**
* Set the threshold of case labels beyond which ANTLR will not instruct the target template
* to generate switch() { case xxx: ...
*
* @param maxSwitchCaseLabels Maximum number of case lables that ANTLR should allow the target code
*/
public void setMaxSwitchCaseLabels(int maxSwitchCaseLabels) {
CodeGenerator.MAX_SWITCH_CASE_LABELS = maxSwitchCaseLabels;
}
/**
* Set the threshold of the number alts, below which ANTLR will not instruct the target
* template to use a switch statement.
*
* @param minSwitchAlts the minimum number of alts required to use a switch staement
*/
public void setMinSwitchAlts(int minSwitchAlts) {
CodeGenerator.MIN_SWITCH_ALTS = minSwitchAlts;
}
/**
* Set the location (base directory) where output files should be produced
* by the ANTLR tool.
* @param outputDirectory
*/
public void setOutputDirectory(String outputDirectory) {
haveOutputDir = true;
this.outputDirectory = outputDirectory;
}
/**
* Used by build tools to force the output files to always be
* relative to the base output directory, even though the tool
* had to set the output directory to an absolute path as it
* cannot rely on the workign directory like command line invocation
* can.
*
* @param forceRelativeOutput true if output files hould always be relative to base output directory
*/
public void setForceRelativeOutput(boolean forceRelativeOutput) {
this.forceRelativeOutput = forceRelativeOutput;
}
/**
* Set the base location of input files. Normally (when the tool is
* invoked from the command line), the inputDirectory is not set, but
* for build tools such as Maven, we need to be able to locate the input
* files relative to the base, as the working directory could be anywhere and
* changing workig directories is not a valid concept for JVMs because of threading and
* so on. Setting the directory just means that the getFileDirectory() method will
* try to open files relative to this input directory.
*
* @param inputDirectory Input source base directory
*/
public void setInputDirectory(String inputDirectory) {
this.inputDirectory = inputDirectory;
haveInputDir = true;
}
/** This method is used by all code generators to create new output
* files. If the outputDir set by -o is not present it will be created.
* The final filename is sensitive to the output directory and
* the directory where the grammar file was found. If -o is /tmp
* and the original grammar file was foo/t.g then output files
* go in /tmp/foo.
*
* The output dir -o spec takes precedence if it's absolute.
* E.g., if the grammar file dir is absolute the output dir is given
* precendence. "-o /tmp /usr/lib/t.g" results in "/tmp/T.java" as
* output (assuming t.g holds T.java).
*
* If no -o is specified, then just write to the directory where the
* grammar file was found.
*
* If outputDirectory==null then write a String.
*/
public Writer getOutputFile(Grammar g, String fileName) throws IOException {
if (getOutputDirectory() == null) {
return new StringWriter();
}
// output directory is a function of where the grammar file lives
// for subdir/T.g, you get subdir here. Well, depends on -o etc...
// But, if this is a .tokens file, then we force the output to
// be the base output directory (or current directory if there is not a -o)
//
File outputDir;
if (fileName.endsWith(CodeGenerator.VOCAB_FILE_EXTENSION)) {
if (haveOutputDir) {
outputDir = new File(getOutputDirectory());
}
else {
outputDir = new File(".");
}
}
else {
outputDir = getOutputDirectory(g.getFileName());
}
File outputFile = new File(outputDir, fileName);
if (!outputDir.exists()) {
outputDir.mkdirs();
}
FileWriter fw = new FileWriter(outputFile);
return new BufferedWriter(fw);
}
/**
* Return the location where ANTLR will generate output files for a given file. This is a
* base directory and output files will be relative to here in some cases
* such as when -o option is used and input files are given relative
* to the input directory.
*
* @param fileNameWithPath path to input source
* @return
*/
public File getOutputDirectory(String fileNameWithPath) {
File outputDir = new File(getOutputDirectory());
String fileDirectory;
// Some files are given to us without a PATH but should should
// still be written to the output directory in the relative path of
// the output directory. The file directory is either the set of sub directories
// or just or the relative path recorded for the parent grammar. This means
// that when we write the tokens files, or the .java files for imported grammars
// taht we will write them in the correct place.
//
if (fileNameWithPath.lastIndexOf(File.separatorChar) == -1) {
// No path is included in the file name, so make the file
// directory the same as the parent grammar (which might sitll be just ""
// but when it is not, we will write the file in the correct place.
//
fileDirectory = grammarOutputDirectory;
}
else {
fileDirectory = fileNameWithPath.substring(0, fileNameWithPath.lastIndexOf(File.separatorChar));
}
if (haveOutputDir) {
// -o /tmp /var/lib/t.g => /tmp/T.java
// -o subdir/output /usr/lib/t.g => subdir/output/T.java
// -o . /usr/lib/t.g => ./T.java
if ((fileDirectory != null && !forceRelativeOutput) &&
(new File(fileDirectory).isAbsolute() ||
fileDirectory.startsWith("~")) || // isAbsolute doesn't count this :(
isForceAllFilesToOutputDir()) {
// somebody set the dir, it takes precendence; write new file there
outputDir = new File(getOutputDirectory());
}
else {
// -o /tmp subdir/t.g => /tmp/subdir/t.g
if (fileDirectory != null) {
outputDir = new File(getOutputDirectory(), fileDirectory);
}
else {
outputDir = new File(getOutputDirectory());
}
}
}
else {
// they didn't specify a -o dir so just write to location
// where grammar is, absolute or relative, this will only happen
// with command line invocation as build tools will always
// supply an output directory.
//
outputDir = new File(fileDirectory);
}
return outputDir;
}
/**
* Name a file from the -lib dir. Imported grammars and .tokens files
*
* If we do not locate the file in the library directory, then we try
* the location of the originating grammar.
*
* @param fileName input name we are looking for
* @return Path to file that we think shuold be the import file
*
* @throws java.io.IOException
*/
public String getLibraryFile(String fileName) throws IOException {
// First, see if we can find the file in the library directory
//
File f = new File(getLibraryDirectory() + File.separator + fileName);
if (f.exists()) {
// Found in the library directory
//
return f.getAbsolutePath();
}
// Need to assume it is in the same location as the input file. Note that
// this is only relevant for external build tools and when the input grammar
// was specified relative to the source directory (working directory if using
// the command line.
//
return parentGrammarDirectory + File.separator + fileName;
}
/** Return the directory containing the grammar file for this grammar.
* normally this is a relative path from current directory. People will
* often do "java org.antlr.Tool grammars/*.g3" So the file will be
* "grammars/foo.g3" etc... This method returns "grammars".
*
* If we have been given a specific input directory as a base, then
* we must find the directory relative to this directory, unless the
* file name is given to us in absolute terms.
*/
public String getFileDirectory(String fileName) {
File f;
if (haveInputDir && !fileName.startsWith(File.separator)) {
f = new File(inputDirectory, fileName);
}
else {
f = new File(fileName);
}
// And ask Java what the base directory of this location is
//
return f.getParent();
}
/** Return a File descriptor for vocab file. Look in library or
* in -o output path. antlr -o foo T.g U.g where U needs T.tokens
* won't work unless we look in foo too. If we do not find the
* file in the lib directory then must assume that the .tokens file
* is going to be generated as part of this build and we have defined
* .tokens files so that they ALWAYS are generated in the base output
* directory, which means the current directory for the command line tool if there
* was no output directory specified.
*/
public File getImportedVocabFile(String vocabName) {
File f = new File(getLibraryDirectory(),
File.separator +
vocabName +
CodeGenerator.VOCAB_FILE_EXTENSION);
if (f.exists()) {
return f;
}
// We did not find the vocab file in the lib directory, so we need
// to look for it in the output directory which is where .tokens
// files are generated (in the base, not relative to the input
// location.)
//
if (haveOutputDir) {
f = new File(getOutputDirectory(), vocabName + CodeGenerator.VOCAB_FILE_EXTENSION);
}
else {
f = new File(vocabName + CodeGenerator.VOCAB_FILE_EXTENSION);
}
return f;
}
/** If the tool needs to panic/exit, how do we do that?
*/
public void panic() {
throw new Error("ANTLR panic");
}
/** Return a time stamp string accurate to sec: yyyy-mm-dd hh:mm:ss
*/
public static String getCurrentTimeStamp() {
GregorianCalendar calendar = new java.util.GregorianCalendar();
int y = calendar.get(Calendar.YEAR);
int m = calendar.get(Calendar.MONTH) + 1; // zero-based for months
int d = calendar.get(Calendar.DAY_OF_MONTH);
int h = calendar.get(Calendar.HOUR_OF_DAY);
int min = calendar.get(Calendar.MINUTE);
int sec = calendar.get(Calendar.SECOND);
String sy = String.valueOf(y);
String sm = m < 10 ? "0" + m : String.valueOf(m);
String sd = d < 10 ? "0" + d : String.valueOf(d);
String sh = h < 10 ? "0" + h : String.valueOf(h);
String smin = min < 10 ? "0" + min : String.valueOf(min);
String ssec = sec < 10 ? "0" + sec : String.valueOf(sec);
return new StringBuffer().append(sy).append("-").append(sm).append("-").append(sd).append(" ").append(sh).append(":").append(smin).append(":").append(ssec).toString();
}
/**
* Provide the List of all grammar file names that the ANTLR tool will
* process or has processed.
*
* @return the grammarFileNames
*/
public List<String> getGrammarFileNames() {
return grammarFileNames;
}
/**
* Indicates whether ANTLR has gnerated or will generate a description of
* all the NFAs in <a href="http://www.graphviz.org">Dot format</a>
*
* @return the generate_NFA_dot
*/
public boolean isGenerate_NFA_dot() {
return generate_NFA_dot;
}
/**
* Indicates whether ANTLR has generated or will generate a description of
* all the NFAs in <a href="http://www.graphviz.org">Dot format</a>
*
* @return the generate_DFA_dot
*/
public boolean isGenerate_DFA_dot() {
return generate_DFA_dot;
}
/**
* Return the Path to the base output directory, where ANTLR
* will generate all the output files for the current language target as
* well as any ancillary files such as .tokens vocab files.
*
* @return the output Directory
*/
public String getOutputDirectory() {
return outputDirectory;
}
/**
* Return the Path to the directory in which ANTLR will search for ancillary
* files such as .tokens vocab files and imported grammar files.
*
* @return the lib Directory
*/
public String getLibraryDirectory() {
return libDirectory;
}
/**
* Indicate if ANTLR has generated, or will generate a debug version of the
* recognizer. Debug versions of a parser communicate with a debugger such
* as that contained in ANTLRWorks and at start up will 'hang' waiting for
* a connection on an IP port (49100 by default).
*
* @return the debug flag
*/
public boolean isDebug() {
return debug;
}
/**
* Indicate whether ANTLR has generated, or will generate a version of the
* recognizer that prints trace messages on entry and exit of each rule.
*
* @return the trace flag
*/
public boolean isTrace() {
return trace;
}
/**
* Indicates whether ANTLR has generated or will generate a version of the
* recognizer that gathers statistics about its execution, which it prints when
* it terminates.
*
* @return the profile
*/
public boolean isProfile() {
return profile;
}
/**
* Indicates whether ANTLR has generated or will generate a report of various
* elements of the grammar analysis, once it it has finished analyzing a grammar
* file.
*
* @return the report flag
*/
public boolean isReport() {
return report;
}
/**
* Indicates whether ANTLR has printed, or will print, a version of the input grammar
* file(s) that is stripped of any action code embedded within.
*
* @return the printGrammar flag
*/
public boolean isPrintGrammar() {
return printGrammar;
}
/**
* Indicates whether ANTLR has supplied, or will supply, a list of all the things
* that the input grammar depends upon and all the things that will be generated
* when that grammar is successfully analyzed.
*
* @return the depend flag
*/
public boolean isDepend() {
return depend;
}
/**
* Indicates whether ANTLR will force all files to the output directory, even
* if the input files have relative paths from the input directory.
*
* @return the forceAllFilesToOutputDir flag
*/
public boolean isForceAllFilesToOutputDir() {
return forceAllFilesToOutputDir;
}
/**
* Indicates whether ANTLR will be verbose when analyzing grammar files, such as
* displaying the names of the files it is generating and similar information.
*
* @return the verbose flag
*/
public boolean isVerbose() {
return verbose;
}
/**
* Provide the current setting of the conversion timeout on DFA creation.
*
* @return DFA creation timeout value in milliseconds
*/
public int getConversionTimeout() {
return DFA.MAX_TIME_PER_DFA_CREATION;
}
/**
* Returns the current setting of the message format descriptor
* @return Current message format
*/
public String getMessageFormat() {
return ErrorManager.getMessageFormat().toString();
}
/**
* Returns the number of errors that the analysis/processing threw up.
* @return Error count
*/
public int getNumErrors() {
return ErrorManager.getNumErrors();
}
/**
* Indicate whether the tool will analyze the dependencies of the provided grammar
* file list and ensure that grammars with dependencies are built
* after any of the other gramamrs in the list that they are dependent on. Setting
* this option also has the side effect that any grammars that are includes for other
* grammars in the list are excluded from individual analysis, which allows the caller
* to invoke the tool via org.antlr.tool -make *.g and not worry about the inclusion
* of grammars that are just includes for other grammars or what order the grammars
* appear on the command line.
*
* This option was coded to make life easier for tool integration (such as Maven) but
* may also be useful at the command line.
*
* @return true if the tool is currently configured to analyze and sort grammar files.
*/
public boolean getMake() {
return make;
}
/**
* Set the message format to one of ANTLR, gnu, vs2005
*
* @param format
*/
public void setMessageFormat(String format) {
ErrorManager.setFormat(format);
}
/**
* Set the timeout value (in milliseconds) after which DFA creation stops
*
* @param timeout value in milliseconds
*/
public void setConversionTimeout(int timeout) {
DFA.MAX_TIME_PER_DFA_CREATION = timeout;
}
/** Provide the List of all grammar file names that the ANTLR tool should process.
*
* @param grammarFileNames The list of grammar files to process
*/
public void setGrammarFileNames(List<String> grammarFileNames) {
this.grammarFileNames = grammarFileNames;
}
public void addGrammarFile(String grammarFileName) {
if (!grammarFileNames.contains(grammarFileName)) {
grammarFileNames.add(grammarFileName);
}
}
/**
* Indicate whether ANTLR should generate a description of
* all the NFAs in <a href="http://www.graphviz.org">Dot format</a>
*
* @param generate_NFA_dot True to generate dot descriptions
*/
public void setGenerate_NFA_dot(boolean generate_NFA_dot) {
this.generate_NFA_dot = generate_NFA_dot;
}
/**
* Indicates whether ANTLR should generate a description of
* all the NFAs in <a href="http://www.graphviz.org">Dot format</a>
*
* @param generate_DFA_dot True to generate dot descriptions
*/
public void setGenerate_DFA_dot(boolean generate_DFA_dot) {
this.generate_DFA_dot = generate_DFA_dot;
}
/**
* Set the Path to the directory in which ANTLR will search for ancillary
* files such as .tokens vocab files and imported grammar files.
*
* @param libDirectory the libDirectory to set
*/
public void setLibDirectory(String libDirectory) {
this.libDirectory = libDirectory;
}
/**
* Indicate whether ANTLR should generate a debug version of the
* recognizer. Debug versions of a parser communicate with a debugger such
* as that contained in ANTLRWorks and at start up will 'hang' waiting for
* a connection on an IP port (49100 by default).
*
* @param debug true to generate a debug mode parser
*/
public void setDebug(boolean debug) {
this.debug = debug;
}
/**
* Indicate whether ANTLR should generate a version of the
* recognizer that prints trace messages on entry and exit of each rule
*
* @param trace true to generate a tracing parser
*/
public void setTrace(boolean trace) {
this.trace = trace;
}
/**
* Indicate whether ANTLR should generate a version of the
* recognizer that gathers statistics about its execution, which it prints when
* it terminates.
*
* @param profile true to generate a profiling parser
*/
public void setProfile(boolean profile) {
this.profile = profile;
}
/**
* Indicate whether ANTLR should generate a report of various
* elements of the grammar analysis, once it it has finished analyzing a grammar
* file.
*
* @param report true to generate the analysis report
*/
public void setReport(boolean report) {
this.report = report;
}
/**
* Indicate whether ANTLR should print a version of the input grammar
* file(s) that is stripped of any action code embedded within.
*
* @param printGrammar true to generate a stripped file
*/
public void setPrintGrammar(boolean printGrammar) {
this.printGrammar = printGrammar;
}
/**
* Indicate whether ANTLR should supply a list of all the things
* that the input grammar depends upon and all the things that will be generated
* when that gramamr is successfully analyzed.
*
* @param depend true to get depends set rather than process the grammar
*/
public void setDepend(boolean depend) {
this.depend = depend;
}
/**
* Indicates whether ANTLR will force all files to the output directory, even
* if the input files have relative paths from the input directory.
*
* @param forceAllFilesToOutputDir true to force files to output directory
*/
public void setForceAllFilesToOutputDir(boolean forceAllFilesToOutputDir) {
this.forceAllFilesToOutputDir = forceAllFilesToOutputDir;
}
/**
* Indicate whether ANTLR should be verbose when analyzing grammar files, such as
* displaying the names of the files it is generating and similar information.
*
* @param verbose true to be verbose
*/
public void setVerbose(boolean verbose) {
this.verbose = verbose;
}
/**
* Indicate whether the tool should analyze the dependencies of the provided grammar
* file list and ensure that the grammars with dependencies are built
* after any of the other gramamrs in the list that they are dependent on. Setting
* this option also has the side effect that any grammars that are includes for other
* grammars in the list are excluded from individual analysis, which allows the caller
* to invoke the tool via org.antlr.tool -make *.g and not worry about the inclusion
* of grammars that are just includes for other grammars or what order the grammars
* appear on the command line.
*
* This option was coded to make life easier for tool integration (such as Maven) but
* may also be useful at the command line.
*
* @param make
*/
public void setMake(boolean make) {
this.make = make;
}
}
antlr-3.2/tool/CHANGES.txt 0000644 0001750 0001750 00000333117 11256465226 015243 0 ustar twerner twerner ANTLR 3.2 Release
Sep 21, 2009
Terence Parr, parrt at cs usfca edu
ANTLR project lead and supreme dictator for life
University of San Francisco
CHANGES
September 22, 2009 -- ANTLR v3.2
September 21, 2009 [Jim Idle]
* Added new options for tool invocation to control the points at which the code
generator tells the target code to use its equivalent of switch() instead of
inline ifs.
-Xmaxswitchcaselabels m don't generate switch() statements for dfas
bigger than m [300]
-Xminswitchalts m don't generate switch() statements for dfas smaller
than m [3]
* Upgraded -X help output to include new optins and provide the default
settings, as well as provide units for those settings that need them.
* Change the C Target to overide the deafults for the new settings to
generate the most optimizable C code from the modern C compiler point of
view. This is essentially to always use swtich statements unless there
is absolutely no other option. C defaults are to use 1 for minimum and
3000 for maximum number of alts that trigger switch(). This results in
object code that is 30% smaller and up to 20% faster.
April 23, 2009
* Added reset to TreeNodeStream interface.
April 22, 2009
* Fixed ANTLR-374. Was caused by moved of grammars. %foo() stuff didn't work
April 9, 2009
* .g and .g3 file extensions work again.
* introduced bug in 3.1.3: gives exception not error msg upon
missing .g file
March 26, 2009
* Made ctor in TreeRewriter and TreeFilter call this not super.
March 21, 2009
* Added ctor to RecognizerSharedState to allow cloning it.
March 17, 2009 -- ANTLR v3.1.3
* improved ANTLRv3.g to handle <...> element options
March 15, 2009
* Fixed ANTLR-389. Strip didn't ignore options in subrules; also seemed
to demand stdin.
March 15, 2009
* ANTLR always sorts incoming grammar list by dependency. For example,
If W.g depends on tokens from P.g then P.g is done first even if
W.g mentioned first on command line. It does not ignore any files you
specify the commandline. If you do *.g and that includes some
imported grammars, it will run antlr on them.
* -make option prevents ANTLR from running on P.g if P older than
generated files.
* Added org.antlr.tool.GrammarSpelunker to build a faster dependency
checker (what grammars depend on etc...). Totally independent of any
ANTLR code; easy to pull into other tools.
* Added org.antlr.misc.Graph, a general graph with nodes
containing an Object payload. It knows how to do a topological sort
on the nodes.
March 10, 2009
* Added associativity token option to support upcoming special expression
parsing. Added rule option strategy=precedence also
March 1, 2009
* Changed ANTLRWorks debug port from 49153 to 49100. Apparently we change the port in
ANTLRWorks to 49100 in 1.2 but forgot to do so in the ANTLR targets.
START CHANGES FOR TREE FILTER MODE (pulled from dev branch)
This feature will be announced in 3.2, but I am integrating from my development branch now into the mainline so target developers have a chance to implement. We might release 3.1.3 bug fix release before 3.2.
* CommonTreeNodeStream -> BufferedTreeNodeStream. Now,
CommonTreeNodeStream is completely unbuffered unless you are
backtracking. No longer making a list of all nodes before tree parsing.
* Added tree grammar filter=true mode.
Altered templates:
Java.stg: added filterMode to genericParser and treeParser.
This required a change to ANTLRCore.sti
Defined a default parameter in treeParser to set the superclass
to TreeFilter for tree grammar with filter=true. It sets
superclass to TreeRewriter if filter=true and output=AST.
Other them that, I only had to change ANTLR itself a little bit.
Made filter mode valid for tree grammars and have it automatically set
the necessary elements: @synpredgate, backtrack=true, rewrite=true
(if output=AST). Added error message for detecting conflicting
options.
* Added misc.FastQueue and TestFastQueue:
A queue that can dequeue and get(i) in O(1) and grow arbitrarily large.
A linked list is fast at dequeue but slow at get(i). An array is
the reverse. This is O(1) for both operations.
* Added tree.TreeIterator, a generator that walks a doubly linked tree.
The nodes must know what index they are. It's an Iterator but
remove() is not supported. Returns navigation nodes always:
UP, DOWN, EOF.
* Added misc.LookaheadStream: A lookahead queue that knows how
to mark/release locations in the buffer for backtracking purposes.
I hope to use for both tree nodes and tokens. Just implement
nextElement() to say how to get next node or token.
END CHANGES FOR TREE FILTER MODE
February 23, 2009 -- ANTLR v3.1.2
February 18, 2009
* Added org.antlr.tool.Strip (reads from file arg or stdin, emits to stdout)
to strip actions from a grammar.
February 4, 2009
* Added CommonTree.setUnknownTokenBoundaries(). Sometimes we build trees
in a grammar and some of the token boundaries are not set properly.
This only matters if you want to print out the original text associated
with a subtree. Check this out rule:
postfixExpression
: primary ('.'^ ID)*
;
For a.b.c, we get a '.' that does not have the token boundaries set.
ANTLR only sets token boundaries for subtrees returned from a rule.
SO, the overall '.' operator has the token boundaries set from 'a'
to 'c' tokens, but the lower '.' subtree does not get the boundaries
set (they are -1,-1). Calling setUnknownTokenBoundaries() on
the returned tree sets the boundaries appropriately according to the
children's token boundaries.
January 22, 2009
* fixed to be listeners.add(listener); in addListener() of DebugEventHub.java
January 20, 2009
* Removed runtime method: mismatch in BaseRecognizer and TreeParser. Seems
to be unused. Had to override method recoverFromMismatchedToken() in
TreeParser to get rid of single token insertion and deletion for
tree parsing because it makes no sense with all of the up-and-down nodes.
* Changed JIRA port number from 8888 to no port spec (aka port 80) and all
refs to it in this file.
* Changed BaseTree to Tree typecase in getChild and toStringTree() and
deleteChild() to make more generic.
December 16, 2008
* Added -verbose cmd-line option and turned off standard header
and list of read files. Silent now without -verbose.
November 24, 2008
* null-ptr protected getParent and a few others.
* Added new ctor to CommonTreeNodeStream for walking subtrees. Avoids
having to make new serialized stream as it can reuse overall node stream
buffer.
November 20, 2008
* Updated BaseTest to isolate tests better.
November 17, 2008
* BaseTreeAdaptor.getType() was hosed; always gave 0. Thanks to Sam Harwell.
November 8, 2008
* Added methods to BaseRecognizer:
public void setBacktrackingLevel(int n) { state.backtracking = n; }
/** Return whether or not a backtracking attempt failed. */
public boolean failed() { return state.failed; }
November 5, 2008
* Tweaked traceIn/Out to say "fail/succeeded"
* Bug in code gen for tree grammar wildcard list label x+=.
* Use of backtrack=true anywhere in grammar causes backtracking sensitive
code to be generated. Actions are gated etc... Previously, that only
happened when a syntactic predicate appeared in a DFA. But, we need
to gate actions when backtracking option is set even if no decision
is generated to support filtering of trees.
October 25, 2008
* Fixed debug event socket protocol to allow spaces in filenames.
* Added TreeVisitor and TreeVisitorAction to org.antlr.runtime.tree.
October 22, 2008
* Added inContext() to TreeParser. Very useful for predicating
tree grammar productions according to context (their parent list).
Added new TestTreeContext unit tests (15).
/** Check if current node in input has a context. Context means sequence
* of nodes towards root of tree. For example, you might say context
* is "MULT" which means my parent must be MULT. "CLASS VARDEF" says
* current node must be child of a VARDEF and whose parent is a CLASS node.
* You can use "..." to mean zero-or-more nodes. "METHOD ... VARDEF"
* means my parent is VARDEF and somewhere above that is a METHOD node.
* The first node in the context is not necessarily the root. The context
* matcher stops matching and returns true when it runs out of context.
* There is no way to force the first node to be the root.
*/
public boolean inContext(String context) {...}
* Added 3 methods to Tree interface [BREAKS BACKWARD COMPATIBILITY]
/** Is there is a node above with token type ttype? */
public boolean hasAncestor(int ttype);
/** Walk upwards and get first ancestor with this token type. */
public Tree getAncestor(int ttype);
/** Return a list of all ancestors of this node. The first node of
* list is the root and the last is the parent of this node.
*/
public List getAncestors();
October 21, 2008
* Updated unit tests to be correct for \uFFFE->\uFFFF change
* Made . in tree grammar look like ^(. .*) to analysis, though ^(. foo)
is illegal (can't have . at root). Wildcard is subtree or node.
Fixed bugs:
http://www.antlr.org/browse/ANTLR-248
http://www.antlr.org/browse/ANTLR-344
October 1, 2008 -- ANTLR v3.1.1
September 8, 2008
* Labels on tokens, rules carry into synpreds now so semantic predicates work.
This didn't work since labels were stripped in the synpred and they weren't
defined in the generated method.
a : x=A z=a {$x.text.equals($z.text)}? A
| y=A a A A
;
September 3, 2008
* Made a REV static variable in Tool so that we can change the rev for
daily builds.
* Made \uFFFF a valid character. Token types are 32-bit clean using -1
not 0x0000FFFF as -1 so it should be okay. Label.java:
public static final int MIN_CHAR_VALUE = '\u0000';
public static final int MAX_CHAR_VALUE = '\uFFFF';
August 30, 2008
* Changed messages in en.stg so that TOKEN_NONDETERMINISM correctly
indicates when actions hid semantic predicates.
August 15, 2008
* Tweaked build properties and build.xml
August 13, 2008
* Fixed ANTLR-314; 3.1 introduced a problem with list labels +=
August 12, 2008 -- ANTLR v3.1
* Added JavaScript target
August 7, 2008
* an NFA target of EOF predicate transition in DFA cause an exception in
getPredicatesPerNonDeterministicAlt().
* Kay Roepke found a nasty bug when debugging AST-constructing
composite recognizers. If the input state was null to the constructor,
super class constructor created a new parser state object.
Later, though we passed the argument state not this.state
to the delegate constructors, forcing them to share a different
state objects! Changed state to this.state in Dbg.stg constructors.
* Ack. messed up debug/AST. Have to set proxy's tree adaptor; it's
a circular ref. Just an ASTDbg.stg change.
August 4, 2008
* superClass works now for lexers
* Made Grammar.defineNamedAction propogate header actions down to all
delegates if root grammar; regardless of lexer/parser scope.
* Rejiggered AST templates to propogate changes to tree adaptor
for delegate grammars. Fixes ANTLR-302
August 4, 2008
* FOLLOW set computations altered constant FOLLOW bit sets.
* Added (...) are all predicate evaluations.
* Extra init code for tree parser nonrewrite mode removed.
* Added empty child list check in becomeRoot
August 3, 2008
* Was using RuleReturnScope not Rulename_return for list labels in tree
parser.
* Didn't set _last in tree parser for rule ref track stuff (rewrite=true)
August 2, 2008
* Benjamin found another rewrite engine bug.
July 30, 2008
* CommonTreeNodeStream / CommonTokenStream did not reset properly.
July 29, 2008
* Fixed another bug in TokenRewriteStream; didn't like inserts after end.
July 28, 2008
* Fixed bug in TokenRewriteStream.toString(start,stop); it ignored
parameters. ;)
July 17, 2008
* allow qualified type names in hetero <...> options like T<a.b.c.Node>
July 5, 2008
* treeLevel not set for setBlock alts; added unit test
July 3, 2008
* Fixed ANTLR-267. parse tree added nodes during backtracking and
cyclic DFAs. tracks hidden tokens too now. Added toInputString() to
get text back including hidden tokens. Shows <epsilon> for rules
that match nothing.
June 26, 2008
* Added gParent ptr that points to immediate parent grammar. E.g.,
// delegators
public MParser gM;
public M_S gS;
public M_S gParent = gS; // NEW
* Grammar imports didn't set all the delegate pointers...lots of imported
grammars would cause a null ptr exception. Fixes ANTLR-292.
June 25, 2008
* List labels in tree construction didn't always track the tree; sometimes
had a rule result structure.
June 4, 2008
* Improved unit testing so that each test suite executes and builds grammars
in a separate temporary directory. This means they can execute concurrently.
Also seem to be a problem with my class path during execution. Moved
tmpdir for ahead of standard CLASSPATH.
* By virtue of an improvement to StringTemplate, output newlines
in generated files should be normalized to whatever your host uses.
June 3, 2008
* Restrict legality of grammar options; for example you cannot use output option
in lexer anymore.
June 2, 2008
* Throw illegal arg exception upon invalid TokenRewriteStream ops. Rewrote
core of engine. Slightly different operation. Added many more unit tests.
3.1b1 - May 20, 2008
May 11, 2008
* rewrite=true, output=AST for tree grammar was not working. Altered trees were not
propagated back up the rule reference chain. Required a number of mods to
ASTTreeParser.stg. Added unit tests.
May 10, 2008
* [BACKWARD INCOMPATIBLE if you override match()]
I had turned off single token insertion and deletion because I could not figure
out how to work with trees and actions. Figure that out and so I turned it back on.
match() returns Object matched now (parser, tree parser) so we can set labels
on token refs properly after single token ins/del error recovery. Allows actions
and tree construction to proceed normally even though we recover in the middle of
an alternative. Added methods for conjuring up missing symbols: getMissingSymbol().
* refactored BaseRecognizer error handling routines
* Single token error recovery was not properly taking into consideration EOF.
* ANTLR no longer tries to recover in tree parsers inline using single node deletion or insertion; throw exception. Trees should be well formed as they are not created by users.
* Added empty constructors to the exception classes that did not have them so that ANTLRWorks can create the exceptions.
* Made debug tree adaptor deal with tokens conjured up during error recovery.
* Removed extra location() debug element that was emitted.
May 8, 2008
* ANTLR didn't update line/col to DFA map for AW.
May 6-7, 2008
* Insufficiently covered (with semantic predicates) alt warnings are now emitted before
nondeterminisms so it's clear the nondeterminism is a result of insufficient preds.
* Improved insufficiently covered alt warnings from:
warning(203): T.g:2:3: The following alternatives are insufficiently covered with predicates: 1
to:
warning(203): T.g:2:3: Input B is insufficiently covered with predicates at loca
tions: alt 1: line 3:15, alt 2: line 2:9
* Improved nondeterminism warning to have:
Semantic predicates were present but were hidden by actions.
parser grammar U;
a : (A B)? ;
b : X a {p1}? A B | Y a {a1} {p2}? A B | Z a ;
To create the prediction DFA for the optional sub rule in 'a', ANTLR must find all references to 'a' to determine what can follow. A B can follow 'a' in the first two alts rule 'b'. To resolve the conflict between matching A B immediately in the sub rule and exiting rule 'a' to match it in 'b', ANTLR looks for predicates. In this case, there are two predicates that indicate the semantic context in which the surrounding alternatives are valid. The problem is that one of the predicates is hidden by an action. It took me 1.5 days, but I've finally have gotten ANTLR to properly track the insufficiently covered alternatives. Further, I have gotten it to tell you precisely where the uncovered predicates are even if they are simply hidden by actions. I have also updated all of the nondeterminism warnings so that it tells you if there was a predicate but one hidden by an action (this could be a separate condition from insufficiently covered predicates). here are your messages from ANTLR:
ANTLR Parser Generator Version 3.1b1 (??) 1989-2007
warning(203): U.g:2:5: Input such as "A B" is insufficiently covered with predicates at locations: alt 2: line 3:38 at B
Semantic predicates were present but were hidden by actions.
warning(200): U.g:2:5: Decision can match input such as "A B" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
Semantic predicates were present but were hidden by actions.
* Fixed issue where
r41
: (INT -> INT) ( ('+' i=INT) -> ^($i $r41) )* ';'
;
still warned about $r41 being ambig.
* actions are now added to the NFA.
* Fixed ANTLR-222. ANTLR now ignores preds after actions.
May 5, 2008
* Fixed ANTLR-235 by backing out a change from 12-31-07.
* Fixed ANTLR-249; I include semantic context again in closure busy signal.
May 3, 2008
* Fixed ANTLR-208. Looks in library or in -o output path. antlr -o foo T.g U.g where U needs T.tokens won't work unless we look in foo too. fixed.
* Refactored assign.types.g to move methods to a class called AssignTokenTypesBehavior.
* Fixed ANTLR-207. Lexers importing vocabs didn't see ';'=4 type aliases in .tokens.
* Fixed ANTLR-228. Couldn't use wildcard in alts with AST rewrites.
May 2, 2008
* Fixed ANTLR-230; can use \' now in action.
* Scope attributes no longer have a stack depth check on front. If you ref $r::a when r has not invoked you, then you get an exception not a default value. Back to the way 3.0.1 worked.
* $channel was a global variable in 3.0.1 unlike $type which did not affect an invoking lexer rule. Now it's local too. Only $type and $channel are ever set with regularity. Setting those should not affect an invoking lexer rule as in the following should work:
X : ID WS? '=' ID ; // result is X on normal channel
WS : ' '+ {$channel = HIDDEN; } ;
STRING : '"' (ESC|.)* '"' ; // result is STRING not ESC
FLOAT : INT '.' INT? ; // should be FLOAT
INT : Digit+ ;
fragment
Digit : '0'..'9' ;
* Fixed bug in interpreter regarding (...)* loops
May 1, 2008
* Fixed ANTLR-202. These now give warnings about ambig ref to $a.
a : ID a -> $a | INT ;
and
a : A a {$a.text} | B ;
April 30, 2008
* Fixed ANTLR-237. updated -depend to know about imported grammars.
$ java org.antlr.Tool -depend -lib foo T.g
ANTLR Parser Generator Version 3.1b1 (??) 1989-2007
T.g: foo/Java.g
TParser.java : T.g
T.tokens : T.g
TLexer.java : T.g
T_Java : T.g
April 29, 2008
* Fixed ANTLR-217; scope A,B,C; didn't work
* Fixed ANTLR-224; ! or ^ on item in alt with rewrite gave exception
* Added token options to terminals: ID<node=V; foo="Big bob"> etc...
node is default so you can do ID<V> for hetero tree types. most common.
April 17, 2008
* Use default msg if unknown recog type in getErrorMessage():
String msg = e.getMessage();
April 14, 2008
* %x.y = foo; was not working in @members section
March 29, 2008
* Import couldn't handle A imports B imports C.
March 27, 2008
* Added get/setInputStream to Token interface and affected classes.
February 26, 2008
* made fillBuffer public in CommonTreeNodeStream so we can add trees
to stream for interactive interpreters.
February 14, 2008
* Fixed a bug in the code generation where tree level 0 was used
no matter what to rewrite trees in tree grammars. added unit test
* Fixed ANTLR-221. exceptions were generated when using
AST construction operators and no output=AST option.
February 13, 2008
* Improved error msgs for unreachable alts and tokens.
February 11-12, 2008
* Fixed ANTLR-219.
It looks like the AST construction code for sets was totally messed up.
This was for not only the new tree parser AST construction, but also
the regular tree construction for parsers. I had to introduce templates
in the ASTTreeParser.stg file to deal with this. added unit tests:
TestTreeGrammarRewriteAST.testSetMatchNoRewrite(),
testSetMatchNoRewriteLevel2(), testSetMatchNoRewriteLevel2Root().
Had to add template matchRuleBlockSet()
to differentiate between a regular set in one that is an entire rule.
If it is an entire rule, it has to set the return value, retval.tree.
* Fixed ANTLR-220.
Made TreeAdaptor dupNode and dupTree events emit debugging events
so AW could see tree node duplications.
February 4, 2008
* BACKWARD INCOMPATIBILITY
Added getSourceName to IntStream and TokenSource interfaces and also the
BaseRecognizer. Have to know where char come from for error messages.
Widespread change, but a trivial one.
January 17, 2008
* Interpreter throws FailedPredicateException now when it sees a predicate;
before it was silently failing. I'll make it work one of these days. ;)
January 12, 2008
* Copy ctor not copying start and stop for common token. Fixes ANTLR-212
* Removed single token insertion and deletion for tokens, sets.
Required a change to the code generation for matchSet() template
and a tweak inside the BaseRecognizer. To engage this again is easy,
just override mismatch() to call mismatchRecover(). I changed it to simply
throw an exception.
* Added syntaxError recognizer state var so you can easily tell if
a recognizer failed. Added getNumberOfSyntaxErrors() to recognizers.
* Added doc for the error node stuff:
http://www.antlr.org/wiki/display/ANTLR3/Tree+construction
* Fixed ANTLR-193
* Added recognizer methods to answer questions about current mismatched
token error. Useful now since i don't automatically recover inline
to such errors (I throw exception):
mismatchIsUnwantedToken(IntStream input, int ttype)
mismatchIsMissingToken(IntStream input, BitSet follow)
* Added UnwantedTokenException and MissingTokenException to make
match() problems more precise in case you want to catch differently.
Updated getErrorMessage() to be more precise. Says:
line 2:9 missing EQ at '0'
now instead of
line 2:9 mismatched input '0' expecting EQ
Input "x=9 9;" gives
line 3:8 extraneous input '9' expecting ';'
When very confused, "x=9 for;", you still get old mismatched message:
line 3:8 extraneous input 'for' expecting ';'
line 3:11 mismatched input ';' expecting '('
* Added unit tests to TestAutoAST and copied to TestRewriteAST with
suitable rewrites to ensure basic error node insertion works.
January 11, 2008
* Adding errorNode to TreeAdaptor and various debug
events/listeners. Had to add new class runtime.tree.CommonErrorNode
to hold all the goodies: input stream, start/stop objects.
* Tweaked CommonTree.getType() to return INVALID_TOKEN_TYPE
instead of literal 0 (same thing at moment though).
* Updated ANTLRWorks to show error nodes in tree as much as I could; Jean
will get to rest of it.
January 9-10, 2008
* Continued work on debugging/profiling composite grammars.
* Updated debug protocol for debugging composite grammars. enter/exit
rule needs grammar to know when to flip display in AW.
* Fixed ANTLR-209. ANTLR consumed 2 not 1 char to recover in lexer.
* Added two faqs instead of making changes to antlr runtime about
lexer error handling:
http://www.antlr.org/wiki/pages/viewpage.action?pageId=5341230
http://www.antlr.org/wiki/pages/viewpage.action?pageId=5341217
January 1-8, 2008
* Making debugging/profiling work with composite grammars.
* Updated ANTLRWorks so it works still for noncomposite grammars.
* two new examples: import and composite-java (the java example grammar
broken up into multiple pieces using import).
* Worked on composite grammars. Had to refactor a lot of code to make
ANTLR deal with one meta grammar made up of multiple grammars. I
thought I had it sort of working back in August. Yes, but barely. Lots
of work to do it seemed. Lots of clean up work. Many new unit tests
in TestCompositeGrammars. Had to add new error messages warning about
conflicting tokens inherited from multiple grammars etc...
TOKEN_ALIAS_CONFLICT(arg,arg2) ::=
"cannot alias <arg>; string already assigned to <arg2>"
TOKEN_ALIAS_REASSIGNMENT(arg,arg2) ::=
"cannot alias <arg>; token name already assigned to <arg2>"
TOKEN_VOCAB_IN_DELEGATE(arg,arg2) ::=
"tokenVocab option ignored in imported grammar <arg>"
INVALID_IMPORT(arg,arg2) ::=
"<arg.grammarTypeString> grammar <arg.name> cannot import <arg2.grammarTypeString> grammar <arg2.name>"
IMPORTED_TOKENS_RULE_EMPTY(arg,arg2) ::=
"no lexer rules contributed to <arg> from imported grammar <arg2>"
IMPORT_NAME_CLASH(arg,arg2) ::=
"combined grammar <arg.name> and imported <arg2.grammarTypeString> grammar <arg2.name> both generate <arg2.recognizerName>; import ignored"
This stuff got really really complicated. Syntactic predicate names even
had to be scoped per grammar so they don't conflict.
* When using subrules like (atom->atom) to set result tree, it was not
properly setting result (early enough). Future code got null for
$rule.tree.
December 31, 2007
* Added the start of a semantic predicate computation for LL(1) to
solve a problem with slow grammar analysis even with k=1 due to
predicates. Then I realized the problem with that grammar was
elsewhere. Semantic context really shouldn't be used when
preventing closure recomputation (May 2008 I discovered I was
wrong--you do need it). The predicates became huge even though the
reduced value would be no different. The analyzer seems faster now
that I am not testing predicate values all the time. Further it may
terminate sooner just due to reduced closure recursion.
* Moved FIRST/FOLLOW computations to a separate class LL1Analyzer to
tidy up.
* ANTLR lexer allowed octal escapes, but they didn't work. ;) Rather than
fix, I'm removing. Use '\uxxxx' to get even 8 bit char values: \u00xx.
December 29, 2007
* Fixed ANTLR-206. I wasn't avoiding analyzing decisions in
left-recursive rules.
* Had to add hetero arg to all tokenRef*() templates. Added _last
local var to track last child so we can do replaceChildren() during
AST rewrite mode for tree grammars. Should be useful later for .text
property. Ack, hetero arg is on lots of templates. :( Moved
ruleCleanUp() template into ASTTreeParser and ASTParser groups.
* added noRewrite() template (to Java.stg) so we can insert code during
rewrite mode to return original tree if no rewrite. Might be useful
for token rewrites later. For templates too?
* Had to add if !rewriteMode around tree construction in tree parser
templates.
* Harald Muller pointed out that we need to use <initValue(attr.type)>
in our tests for null token/rule property references. For int types
we need 0 not null. (p!=null?p.line:0). Changed scopeAttributeRef,
ruleLabelRef. Also changed the known typed attributes like
lexerRuleLabelPropertyRef_line to yield 0 upon null rule ref to
be consistent with case when we don't know the type. Fixes ANTLR-195.
Added testTypeOfGuardedAttributeRefIsCorrect test and reset expected
output for 13 tests that now "fail".
December 28, 2007
* added polydiff example (Java target)
* added "int" property for token and lexer rule refs. super convenient. E.g.,
a : b=INT {int x = $b.int;} ;
December 27, 2007
* Changed -Xnoinlinedfa to -Xmaxinlinedfastates m where m is
maximum number of states a DFA can have before ANTLR avoids
inlining it. Instead, you get a table-based DFA. This
affectively avoids some acyclic DFA that still have many states
with multiple incident edges. The combinatorial explosion smacks
of infinite loop. Fixes ANTLR-130.
* [...] are allowed in args now but ] must be escaped as \]. E.g.,
a[String[\] ick, int i] : ... ;
And calling a rule: foo[x[i\], 34]
Fixes ANTLR-140.
* Fixed ANTLR-105. Target.getTargetStringLiteralFromANTLRStringLiteral()
escaped " that were already escaped.
* target's can now specify how to encode int as char escape. Moved
DFA.encodeIntAsCharEscape to Target.
* Bug in runtime.DFA. If a special state (one with predicate) failed, it
tried to continue (causing out of range exception due to state = -1)
instead of reporting error.
* If -dfa with combined grammar T.g, builds T.dec-*.dot and TLexer.dec-*.dot
* Fix ANTLR-165.
Generate TParser.java and TLexer.java from T.g if combined, else
use T.java as output regardless of type.
BACKWARD INCOMPATIBILITY since file names change.
I changed the examples-v3/java to be consistent. Required XML.g ->
XMLLexer.java and fuzzy/Main.java change.
* Fix ANTLR-169. Deletes tmp lexer grammar file.
December 25, 2007
* Fixed ANTLR-111. More unit tests in TestAttributes.
December 25, 2007
* Dangling states ("decision cannot distinguish between alternatives
for at least one input sequence") is now an error not a warning.
* Added sample input sequence that leads to dangling DFA state, one
that cannot reach an accept state. ANTLR ran into a case where
the same input sequence reaches multiple locations in the NFA
(and so not nondeterministic), but analysis ran out of further
NFA states to look for more input. Commonly at EOF target states.
Now says:
error(202): CS.g:248:95: the decision cannot distinguish between alternative(s) 1,2 for input such as "DOT IDENTIFIER EOF"
Also fixed bug where dangling states did not resolve to stop states.
* Fixed ANTLR-123
December 17-21, 2007
* k=1 doesn't prevent backtracking anymore as in
(options {k=1;}:'else' statement)?
if backtrack=true for overall grammar. Set to false in subrule.
* Optimized the analysis engine for LL(1). Doesn't attempt LL(*) unless
LL(1) fails. If not LL(1) but autobacktracking but no other kind of
predicate, it also avoids LL(*). This is only important for really
big 4000 line grammars etc...
* Lots of code clean up
December 16, 2007
* Yet more Kay pair programming. Saved yet more RAM; 15% by
wacking NFA configurations etc in each DFA state after DFA construction.
* Overall we drop from 2m49s to 1m11s for a huge 4000 line TSQL grammar
with k=*. Only needs -Xconversiontimeout 2000 now not
-Xconversiontimeout 5000 too. With k=1, it's 1m30s down to 40s.
December 15, 2007
* Working with Kay Roepke, we got about 15% speed improvement in
overall ANTLR exec time. Memory footprint seems to be about 50%
smaller.
December 13-14, 2007
* I abort entire DFA construction now when I see recursion in > 1 alt.
Decision is non-LL(*) even if some pieces are LL(*). Safer to bail
out and try with fixed k. If user set fixed k then it continues because
analysis will eventually terminate for sure. If a pred is encountered
and k=* and it's non-LL(*), it aborts and retries at k=1 but does NOT
emit an error.
* Decided that recursion overflow while computing a lookahead DFA is
serious enough that I should bail out of entire DFA computation.
Previously analysis tried to keep going and made the rules about
how analysis worked more complicated. Better to simply abort when
decision can't be computed with current max stack (-Xm option).
User can adjust or add predicate etc... This is now an error
not a warning.
* Recursion overflow and unreachable alt is now a fatal error; no code gen.
The decision will literally not work.
* Cleaned up how DFA construction/analysis aborts due to non-LL(*) and
overflow etc... Throws exceptions now, which cleans up a bunch of IF
checks etc... Very nice now. Exceptions:
analysis/AnalysisRecursionOverflowException.java
analysis/AnalysisTimeoutException.java
analysis/NonLLStarDecisionException.java
* ErrorManager.grammarWarning() counted them as errors not warnings.
* Unreachable alt warnings are now errors.
* The upshot of these changes is that I fixed ANTLR-178 and did
lots of refactoring of code handling analysis failure.
December 11, 2007
* Could not deal with spaces, oddly enough in arg lists:
grammar Bad;
a : A b["foo", $A.text] ;
b[String x, String y] : C ;
October 28, 2007
* Made ANTLR emit a better error message when it cannot write the
implicit lexer file from a combined grammar. Used to say "cannot open
file", now says "cannot write file" and gives backtrace.
September 15, 2007
add getCharStream to Lexer.
September 10, 2007
* Added {{...}} forced action executed even during backtracking.
September 9, 2007
* r='string' in lexer got a duplicate label definition.
August 21, 2007
* $scope::variable refs now check for empty stack so that expr == null if
$scope has an empty stack. Works for $scope[...]::variable too. Nice!
August 20, 2007
* Added reset() to CommonTreeNodeStream, token stream too
* Made refs to rule/token properties use ?: to avoid null ptr exception.
$label.st now is label!=null?label.st:null. Updated TestAttributes.
This is useful not only for optional rule/token refs, but also during
error recovery. If ID is not matched, $ID.text won't cause a null ptr.
August 20, 2007
* Fixed ANTLR-177: hashCode/equals not consistent for label
Fixed bug where Rule was compared to string; introduced from dev branch
August 15, 2007 -- Got rough draft of the grammar import working.
Submit to dev and then integrate into mainline.
All file changes/additions:
README.txt # edit
CHANGES.txt # add
Factored out the changes from the readme.
runtime/Java/src/org/antlr/runtime/BaseRecognizer.java # edit
runtime/Java/src/org/antlr/runtime/DFA.java # edit
runtime/Java/src/org/antlr/runtime/Lexer.java # edit
runtime/Java/src/org/antlr/runtime/Parser.java # edit
runtime/Java/src/org/antlr/runtime/debug/DebugParser.java # edit
runtime/Java/src/org/antlr/runtime/tree/TreeParser.java # edit
Factored state fields into RecognizerSharedState
object. You will see a lot of things like
state.errorRecovery = false;
runtime/Java/src/org/antlr/runtime/RecognizerSharedState.java # add
Shares all recognizer state variables including lexer even though
these are superfluous to parsers and tree parsers. There
was a casting issue that I could not resolve.
src/org/antlr/Tool.java # edit
Broke a part Grammar.setGrammarContent() into
parseAndBuildAST() and analyzeGrammar() to make the grammar
import work. I needed to be able to look at the trees for
imported grammars before analyzing them and building DFA. Added
use of the CompositeGrammar object and handling of multiple
delegate grammars. Changed decision DFA DOT file names to
include the grammar name.
src/org/antlr/analysis/DFA.java # edit
Just tweaked to use generics, updated a comment.
src/org/antlr/analysis/DecisionProbe.java # edit
Just tweaked to use generics.
src/org/antlr/analysis/NFA.java # edit
NFA now span multiple grammars and so I moved the NFAs state
tracking to the composite grammar object.
src/org/antlr/analysis/NFAState.java # edit
Added some null checking and made a field public.
src/org/antlr/analysis/NFAToDFAConverter.java # edit
Changed a method call to directly access a field.
src/org/antlr/analysis/RuleClosureTransition.java # edit
Instead of using a rule index, which does not span multiple
grammars, the transition object now attracts a pointer to
the actual Rule definition object.
src/org/antlr/analysis/SemanticContext.java # edit
Tweaked to use a field instead of a method
src/org/antlr/codegen/ActionTranslator.g # edit
src/org/antlr/codegen/ActionTranslatorLexer.java # edit
Tweaked to use new runtime and they changed method name.
src/org/antlr/codegen/CodeGenerator.java # edit
Tweaked comments.
src/org/antlr/codegen/codegen.g # edit
Added import grammar syntax and altered rule atom to pass a
scope around so that grammar.rule works. Caution this
feature is used internally by ANTLR and is not meant to be
used by users at this point.
src/org/antlr/codegen/templates/ANTLRCore.sti # edit
Added scope to all ruleref template interfaces.
src/org/antlr/codegen/templates/Java/Java.stg # edit
Grammars can now import other grammars, which I implemented
using a delegation pointer to the other grammar(s). So if
grammar A imports grammars B and C, then the generated
recognizer for A must have delegation pointers to BParser
and CParser objects. These are now fields:
// delegates
<grammar.delegates:
{g|public <g.name>Lexer <g:delegateName()>;}; separator="\n">
Also, B and C must have back pointers to the delegator so
that they can refer to rules that have been overridden.
This is a mechanism akin to static inheritance:
// delegators
<grammar.delegators:
{g|public <g.name>Lexer <g:delegateName()>;}; separator="\n">
This file also has a lot of changes so that state variables
now are state.backtracking instead of the implied
this.backtracking.
The file also refers to grammar.delegatedRules attribute
which is the list of Rule objects for which you must
generate manual delegation. This amounts to a stub whereby
rule foo's method foo() simply calls X.foo() if foo is not
defined inside the delegator.
You will notice that the ruleref templates now take a scope
so that I can have implicit rule Tokens referred to
delegate.Tokens rule in a delegate grammar. This is the way
I do lexer grammar imports.
I added a template called delegateName which uses the
grammar name to compute a delegate name if the user does not
specify a label in the import statement such as:
import x=X;
Oh, note that rule reference templates all receive a Rule
object now instead of the simple rule name as the 'rule'
attribute. You will see me doing <rule.name> instead of
<name> now.
src/org/antlr/codegen/templates/Java/Dbg.stg # edit
Changes mirroring the constructor and field stuff from
Java.stg. Part of this is a cut and paste because of a bug
in ST.
src/org/antlr/codegen/templates/Java/AST.stg # edit
src/org/antlr/codegen/templates/Java/ASTParser.stg # edit
src/org/antlr/codegen/templates/Java/ASTTreeParser.stg # edit
Just added the scope attribute.
src/org/antlr/test/BaseTest.java # edit
Added functionality to support testing composite grammars.
execLexer()
src/org/antlr/test/TestAttributes.java # edit
Tweak to deal with shared recognizer state.
src/org/antlr/test/TestCompositeGrammars.java # add
Start of my unit tests.
src/org/antlr/tool/CompositeGrammar.java # add
src/org/antlr/tool/CompositeGrammarTree.java # add
Tracks main grammar and all delegate grammars. Tracks unique
NFA state numbers and unique token types. This keeps a tree
of grammars computed from the import/delegation chain. When
you want to look up a rule, it starts at the root of the
tree and does a pre-order search to find the rule.
src/org/antlr/tool/ActionAnalysis.g # edit
src/org/antlr/tool/ActionAnalysisLexer.java # edit
src/org/antlr/tool/AttributeScope.java # edit
Updated to use generics in one place.
src/org/antlr/tool/DOTGenerator.java # edit
Updated to indicate when nonlocal rules are referenced.
src/org/antlr/tool/ErrorManager.java # edit
Added some error messages for import grammars; I need more.
src/org/antlr/tool/FASerializer.java # edit
Tweaked to use a field not method.
src/org/antlr/tool/Grammar.java # edit
This is where most of the meat is for the grammar import
stuff as you can imagine. I factored out the token type
tracking into the CompositeGrammar object. I added code to
the addArtificialMatchTokensRule method so that it includes
references to all delegate lexer Tokens rules. Altered the
rule lookup stuff so that it knows about delegate grammars.
src/org/antlr/tool/GrammarAST.java # edit
src/org/antlr/tool/GrammarAnalysisAbortedMessage.java # edit
src/org/antlr/tool/GrammarReport.java # edit
src/org/antlr/tool/NonRegularDecisionMessage.java # edit
Made enclosing rule visible as field.
src/org/antlr/tool/GrammarSanity.java # edit
General cleanup and addition of generics.
src/org/antlr/tool/Interpreter.java # edit
Reference fields instead of methods.
src/org/antlr/tool/NFAFactory.java # edit
General cleanup and use of Rule object instead of rule
index.
src/org/antlr/tool/NameSpaceChecker.java # edit
A little bit of cleanup and changes to use either the local
or globally visible rule. Added code to check that scopes
are valid on scoped rule references. again this is an
internal feature, not to be used by users.
src/org/antlr/tool/RandomPhrase.java # edit
Tweaked.
src/org/antlr/tool/Rule.java # edit
Added field imported. Removed some unused methods by
commenting them out. Made toString() more expressive.
src/org/antlr/tool/antlr.g # edit
src/org/antlr/tool/antlr.print.g # edit
src/org/antlr/tool/assign.types.g # edit
src/org/antlr/tool/buildnfa.g # edit
src/org/antlr/tool/define.g # edit
Added syntax for import statement. assign.types.g is the
grammar that invokes Grammar.importGrammar().
src/org/antlr/tool/templates/messages/languages/en.stg # edit
Added error messages.
Added
CHANGES.txt
runtime/Java/src/org/antlr/runtime/RecognizerSharedState.java
src/org/antlr/test/TestCompositeGrammars.java
src/org/antlr/tool/CompositeGrammar.java
src/org/antlr/tool/CompositeGrammarTree.java
3.0.1 - August 13, 2007
[See target pages on the wiki for more information on the non-Java targets]
August 7, 2007
* added escaping of double quotes in DOTTreeGenerator
July 22, 2007
* fixed dynamic scope implementation in lexers. They were not creating new scope
entries on the stack. Unsupported feature!
July 30, 2007
* float return values were initalized to 0.0 not 0.0f in java.
July 28, 2007
* Sam Ellis points out an init var bug in ANTLRReaderStream.
July 27, 2007 (done in dev branch)
* Moved token type index stuff from CommonTreeNodeStream to TreeWizard
* Added getChildren to BaseTree.
* Added heterogeneous tree functionality; rewrite for parser/tree parser
and auto AST constr. for parser.
org/antlr/runtime/tree/RewriteRuleElementStream.java
org/antlr/runtime/tree/RewriteRuleNodeStream.java
org/antlr/runtime/tree/RewriteRuleTokenStream.java
Renamed method next() and re-factor things to have more
specific methods: nextToken, nextNode, nextTree.
codegen/codegen.g
Updated to include new <NodeType> AST structure for
token references. Pushed hereto attribute into
all tokenRef* templates.
codegen/templates/Java/AST.stg
Factored out a few templates:
createImaginaryNode(tokenType,hetero,args)
createRewriteNodeFromElement(token,hetero,args)
Converted a lot of stream next() calls to more specific
nextToken, nextNode, nextTree per above.
codegen/templates/Java/ASTParser.stg
Added createNodeFromToken template and re-factored creation
sites to use that template. Added hetero attribute.
codegen/templates/Java/ASTTreeParser.stg
Added createRewriteNodeFromElement template and re-factored.
test/TestHeteroAST.java
New file. Unit tests to test new hetero tree construction.
test/TestRewriteAST.java
Fixed test. Nil single-node trees no longer return nil;
They return null.
tool/ErrorManager.java
tool/templates/messages/languages/en.stg
Added error message:
HETERO_ILLEGAL_IN_REWRITE_ALT(arg) ::=
"alts with rewrites can't use heterogeneous types left of ->"
tool/antlr.g
tool/antlr.print.g
tool/assign.types.g
tool/buildnfa.g
tool/define.g
Added syntax for <NodeType> to token references.
Altered AST structure rippled through different phases.
July 24, 2007
* Deleted DoubleLinkTree.java; CommonTree does that now.
July 23, 2007
* template group outputFile; changed rewrite arg to rewriteMode.
* added rewrite mode for tree parser build AST.
July 22, 2007
* Kay fixed dynamic scope implementation in lexers. They were not
creating new scope entries on the stack. This is an UNSUPPORTED feature.
* added getParent and getChildIndex to TreeAdaptor. Added
implementation to CommonTree. It's just too useful having those
parent and child indexes available for rewriting etc... I tried 2x
to make an implementation of tree rewriting w/o this and the
constraints just made it too expensive and complicated. Have to
update adaptors to set parent, child index values. Updated Tree
interface and BaseTree also. Should only affect target developers
not users. Well, unless they impl Tree.
* dupNode (via ctor) of CommonTree didn't copy start/stop token indexes.
TARGET DEVELOPERS WARNING -- AST.stg split with some functionality
going into ASTParser.stg then I added
ASTTreeParser.stg. CodeGenerator
assumes new subgroups exist.
July 20, 2007
* Added AST construction for tree parsers including -> rewrite rules.
Rewrite mode (rewrite=true) alters the tree in place rather than
constructing a whole new tree. Implementation notes:
org/antlr/runtime/tree/Tree.java
Add methods for parent and child index functionality.
Also added freshenParentAndChildIndexes() which you can use
to ensure that all double linking is set up right after you
manipulate the tree manually. The setChild preteens etc. do
the proper thing so you shouldn't need this.
Added replaceChildren() to support tree rewrite mode in tree parsers
org/antlr/runtime/tree/BaseTree.java
Updated to set parent and child index stuff. Added replaceChildren
method etc... It still only has a list of children as sole field
but calls methods that subclasses can choose to implement such as
CommonTree.
org/antlr/runtime/tree/CommonTree.java
Added parent and childIndex fields to doubly link.
org/antlr/runtime/tree/TreeAdaptor.java
Added methods for new parent and child index functionality.
Also added method for rewrite mode in tree parsers:
replaceChildren(Object parent, int startChildIndex,
int stopChildIndex, Object t);
Added setChild and deleteChild methods
org/antlr/runtime/tree/BaseTreeAdaptor.java
Moved dupTree here from BaseTree.
Updated rulePostProcessing to deal with parent and child index.
Added setChild and deleteChild implementations
org/antlr/runtime/tree/CommonTreeAdaptor.java
Added methods to deal with the parent and child index for a node.
org/antlr/runtime/tree/CommonTreeNodeStream.java
Removed token type index and method fillReverseIndex etc...
Probably will move into the tree wizard in the future.
Changed call/seek stack implementation to use IntArray
Added replaceChildren interface.
org/antlr/runtime/tree/TreeNodeStream.java
Added replaceChildren.
org/antlr/runtime/tree/UnBufferedTreeNodeStream.java
Added replaceChildren method but no implementation
codegen/templates/ANTLRCore.sti
Changed rewrite to a better name: rewriteMode
Added tree level argument to alt, tree so that auto AST
construction can occur while recognizing in tree parsers.
codegen/templates/Java/AST.stg
Split template group: added two subclasses to handle different
functionality for normal parsing and tree parsing + AST
construction. Tree parsers default behavior is to dup tree
not construct another. Added ASTParser.stg and
ASTTreeParser.stg to handle auto AST construction during
recognition for the two different parser types. I just copied
the token, rule, set, wildcard templates to the subclasses.
The rewrite templates are still in AST.stg. I factored out the
node creation so that the same rewrite templates can be used
for both parsing and tree parsing.
codegen/templates/Java/ASTParser.stg
The templates needed to build trees with auto construction
during parsing.
codegen/templates/Java/ASTTreeParser.stg
The templates needed to build trees with auto construction
during tree parsing.
codegen/templates/Java/Java.stg
genericParser now has rewriteElementType (Note or Token) so
that the rewrite streams know what kind of elements are inside
during rewrite rule tree construction.
codegen/templates/Java/ST.stg
rewrite attribute name change to rewriteMode
org/antlr/runtime/debug/DebugTreeAdaptor.java
org/antlr/runtime/debug/DebugTreeNodeStream.java
Updated to handle new interfaces
test/BaseTest.java
Added test rig update to handle AST construction by tree parsers.
All tree construction runs automatically test sanity of parent
and child indexes.
test/TestTreeGrammarRewriteAST.java
test/TestTreeNodeStream.java
test/TestTrees.java
new file; tests the new parent and child index stuff in trees.
July 19, 2007
* implemented new unique ID; GC was causing non unique hash codes. Debugging
tree grammars was messing up.
* got tree rewrites working in tree grammars. It builds a completely new
tree from old tree; i.e., you get two trees in memory. W/o a rewrite
rule, the input for that rule is duplicated and returned. -> w/o elements
to the right means don't return anything; i.e., delete. Ooops...way
harder than I thought. Real implementation notes above.
INCOMPATIBILITY WARNING -- templates have changed; must regen output from
grammars. Runtime libraries have also changed.
Debug event listener interface has changed also.
July 17, 2007
* Added line/charposition to node socket events and event dump so
we have more info during tree parsing. Only works if your
tree adaptor returns a value Token object from getToken(treenode)
with line/col set. Refactored consumeNode/LN to use deserializeNode().
* Fixed mismatched tree node exceptions; for imaginary nodes, it said
"missing null". Now prints the token type we found.
* Cleaned up exception stuff. MismatchedTreeNodeException was setting
line/col, but only RecognitionException should do that.
* If imaginary token gets a mismatch, there is no line info. Search
backwards in stream if input node stream supports to find last
node with good line/col info. E.g.,
ANTLRv3Tree.g: node from after line 156:72 mismatched tree node: EOA expecting <UP>
which used to be:
ANTLRv3Tree.g: node from line 0:0 mismatched tree node: null expecting <UP>
* mismatched tree node exceptions were not sent to the debug event stream.
Due to a type being slightly different on recoverFromMismatchedToken()
in DebugTreeParser. Was calling BaseRecognizer version not subclass.
Now we get:
9459: Recognition exception MismatchedTreeNodeException(0!=0)
* List labels were not allowed as root nodes in tree rewrites like
^($listlabel ...). Had to add a template to AST.stg:
/** Gen ^($label ...) where label+=... */
rewriteTokenListLabelRefRoot ::= rewriteTokenLabelRefRoot
July 16, 2007
* fixed nextNode in RewriteRuleSubtreeStream was dup'ing too much,
screwing up debug event stream. Also there was a bug in how
the rewrite tree stream stuff decided to dup nodes.
* fixed bug in LT for tree parsing; text was not transmitted properly;
only single words worked.
* made decision for rule put line/col on colon not first token of first alt.
* remote ProxyToken now emits token index for easier debugging when looking
at AW's event stream. For example, the @5 here is the token index:
31 Consume hidden [ /<64>,channel=99,30:7, @5]
* same is true for consume nodes now:
25586 Consume node [')'/, <44>, 4712040,@1749] 25
When debugging tree parsers, it helps to track errors when you know
what corresponding input symbol created this tree node.
* Changed debug events associated with trees quite a bit. Passes nodes around
now rather than text, type, unique IDs etc... Mostly affects internal stuff.
Target developers will have some work in their runtime to do to match
this change. :( BUT, there is only a slight tweak in the Dbg.stg
and ASTDbg.stg templates.
Interface just didn't make sense as is. If you turn on debugging, and
want to track a node creation, you want the node pointer not its ID,
text, etc...
Added ProxyTree for passing across socket. Has line/charpos and tokenIndex
July 15, 2007
* added null ptr protection in CommonTreeAdaptor.
July 14, 2007
* null child in TreeAdaptor does nothing now. Changed interface and
implementation. Changed DebugTreeAdaptor to not fire events on null add
as well.
July 12, 2007
* added get method for the line/col to DFA map in Grammar.java
July 7, 2007
* fixed wrong order of test for exceptions in Lexer.getErrorMessage()
June 28, 2007
* Added ability to set the port number in the constructor for the debug parser.
June 5, 2007
* Changed (hidden) option -verbose to -Xnfastates; this just prints out the NFA states along each nondeterministic path for nondeterminism warnings.
May 18, 2007
* there were some dependencies with org.antlr.* that I removed from
org.antlr.runtime.*
3.0 final - May 17, 2007
May 14, 2007
* Auto backtracking didn't work with ! and ^ suffixes on first element
of an alt.
* Auto backtracking didn't work with an action as first element.
May 10, 2007
* turn off the warning about no local messages:
no such locale file org/antlr/tool/templates/messages/languages/ru.stg retrying with English locale
May 5, 2007
* moving org.antlr.runtime to runtime/Java/src/org/... Other target
source / libs are under runtime/targetname.
May 4, 2007
* You could not use arguments on a token reference that was a route in a
tree rewrite rule like -> ^(ID[args] ...).
May 3, 2007
* Fixed ANTLR-82. Actions after the root were considered part of
an optional child. They were not always executed. Required a change
to the ANTLRCore.sti interface for tree() template.
May 2, 2007
* Fixed ANTLR-117. Wasn't building decisions properly for subrules in
syntactic predicates.
April 22, 2007
* Made build.xml ref all jars in antlr lib. Thanks to Miguel Ping.
* Fixed ANTLR-11
* Now labels on ranges and such in lexer work properly.
* ActionAnalysisLexer was in wrong package.
April 21, 2007
* Pushing a huge update that fixes:
http://www.antlr.org/browse/ANTLR-112
http://www.antlr.org/browse/ANTLR-110
http://www.antlr.org/browse/ANTLR-109
http://www.antlr.org/browse/ANTLR-103
http://www.antlr.org/browse/ANTLR-97
http://www.antlr.org/browse/ANTLR-113
http://www.antlr.org/browse/ANTLR-66
http://www.antlr.org/browse/ANTLR-98
http://www.antlr.org/browse/ANTLR-24
http://www.antlr.org/browse/ANTLR-114
http://www.antlr.org/browse/ANTLR-5
http://www.antlr.org/browse/ANTLR-6
Basically, I gutted the way AST rewrites work. MUCH better.
* Fixed lots of little label issues in the lexer. Couldn't do x+=ID
in lexer, for example. Fixed ANTLR-114, ANTLR-112
* Isolated EOT transition in lexer generated dangling else clause.
Fixed ANTLR-113.
April 17, 2007
* Fixed a major problem with gated semantic predicates. Added more
unit tests.
* Fixed bug in cyclic DFA with syntactic predicates. Wasn't rewinding
properly. Further, mark() in token stream did not fill buffer so
when you rewound back to last marker index was -1 not 0. At same time
I fixed ANTLR-103. Syn preds evaluated only once now.
* Altered code gen file writing so it writes directly to a file
instead of building a big string and then writing that out. Should
be faster and much less memory intensive.
* Fixed so antlr writes files to correct location again. See:
http://www.antlr.org/wiki/pages/viewpage.action?pageId=1862
3.0b7 - April 12, 2007
April 10, 2007
* Allows -> {...} actions now when building ASTs. Fixed ANTLR-14.
* Allows ! on sets and wildcard now during output=AST option. Fixed ANTLR-17.
* Fixed ANTLR-92 bug. Couldn't use sets with -> tree construction.
* No lexer rule for a token type is now a warning.
* Fixed set labels in lexer; ANTLR-60 bug
* Fixed problem with duplicate state variable definitions in switch-case
April 9, 2007
* Gated predicates didn't work properly in cyclic DFA.
April 7, 2007
* Couldn't have more than one set per rule it seems. Fixed.
April 3, 2007
* Fix a problem in my unused label optimization. Added new
pass over actions to examine them.
* RuleReturnScope has method back:
/** Has a value potentially if output=template; Don't use StringTemplate
* type as it then causes a dependency with ST lib.
*/
public Object getTemplate() { return null; }
March 30, 2007
* Fixed ANTLR-8. Labels to rules w/o return values caused compile errors.
* Fixed ANTLR-89; semantic predicates in lexer sometimes
caused exception in code gen.
* Fixed ANTLR-36; remove runtime dependency with ST
March 29, 2007
* Over last few days, I've gutted how ANTLR handles sets of chars or
tokens. I cleaned up a lot of stuff in the grammars and added lots
of unit tests.
March 26, 2007
* CommonTreeNodeStream didn't push correctly; couldn't handle very
deeply nested trees.
* Fixed bug that E : 'a' 'b' ; made E be seen as an alias of 'a'.
March 22, 2007
* Working with Egor Ushakov from Sun Optimization / NetBeans team I
made all the Java lexer transition tables static w/o screwing up
ability to reference semantic predicates etc... Only changed Java.stg
* cached text string in CommonToken.getText(); saves on repeated calls;
Java mode.
* made all generated methods final; saves a few percent speed according to
Egor Ushakov (Java only).
* removed most assignments from each lexer rule and even the Lexer.emit()
call! All done in nextToken now. Saves on code gen size and a wee bit of
execution speed probably. Variables became fields: type, channel, line,
etc... Now emit() needs no args even. Again, Egor helped on this.
March 17, 2007
* Jonathan DeKlotz updated C# templates to be 3.0b6 current
March 14, 2007
* Manually-specified (...)=> force backtracking eval of that predicate.
backtracking=true mode does not however. Added unit test.
March 14, 2007
* Fixed bug in lexer where ~T didn't compute the set from rule T.
* Added -Xnoinlinedfa make all DFA with tables; no inline prediction with IFs
* Fixed http://www.antlr.org/browse/ANTLR-80.
Sem pred states didn't define lookahead vars.
* Fixed http://www.antlr.org/browse/ANTLR-91.
When forcing some acyclic DFA to be state tables, they broke.
Forcing all DFA to be state tables should give same results.
March 12, 2007
* setTokenSource in CommonTokenStream didn't clear tokens list.
setCharStream calls reset in Lexer.
* Altered -depend. No longer printing grammar files for multiple input
files with -depend. Doesn't show T__.g temp file anymore. Added
TLexer.tokens. Added .h files if defined.
February 11, 2007
* Added -depend command-line option that, instead of processing files,
it shows you what files the input grammar(s) depend on and what files
they generate. For combined grammar T.g:
$ java org.antlr.Tool -depend T.g
You get:
TParser.java : T.g
T.tokens : T.g
T__.g : T.g
Now, assuming U.g is a tree grammar ref'd T's tokens:
$ java org.antlr.Tool -depend T.g U.g
TParser.java : T.g
T.tokens : T.g
T__.g : T.g
U.g: T.tokens
U.java : U.g
U.tokens : U.g
Handles spaces by escaping them. Pays attention to -o, -fo and -lib.
Dir 'x y' is a valid dir in current dir.
$ java org.antlr.Tool -depend -lib /usr/local/lib -o 'x y' T.g U.g
x\ y/TParser.java : T.g
x\ y/T.tokens : T.g
x\ y/T__.g : T.g
U.g: /usr/local/lib/T.tokens
x\ y/U.java : U.g
x\ y/U.tokens : U.g
You have API access via org.antlr.tool.BuildDependencyGenerator class:
getGeneratedFileList(), getDependenciesFileList(). You can also access
the output template: getDependencies(). The file
org/antlr/tool/templates/depend.stg contains the template. You can
modify as you want. File objects go in so you can play with path etc...
February 10, 2007
* no more .gl files generated. All .g all the time.
* changed @finally to be @after and added a finally clause to the
exception stuff. I also removed the superfluous "exception"
keyword. Here's what the new syntax looks like:
a
@after { System.out.println("ick"); }
: 'a'
;
catch[RecognitionException e] { System.out.println("foo"); }
catch[IOException e] { System.out.println("io"); }
finally { System.out.println("foobar"); }
@after executes after bookkeeping to set $rule.stop, $rule.tree but
before scopes pop and any memoization happens. Dynamic scopes and
memoization are still in generated finally block because they must
exec even if error in rule. The @after action and tree setting
stuff can technically be skipped upon syntax error in rule. [Later
we might add something to finally to stick an ERROR token in the
tree and set the return value.] Sequence goes: set $stop, $tree (if
any), @after (if any), pop scopes (if any), memoize (if needed),
grammar finally clause. Last 3 are in generated code's finally
clause.
3.0b6 - January 31, 2007
January 30, 2007
* Fixed bug in IntervalSet.and: it returned the same empty set all the time
rather than new empty set. Code altered the same empty set.
* Made analysis terminate faster upon a decision that takes too long;
it seemed to keep doing work for a while. Refactored some names
and updated comments. Also made it terminate when it realizes it's
non-LL(*) due to recursion. just added terminate conditions to loop
in convert().
* Sometimes fatal non-LL(*) messages didn't appear; instead you got
"antlr couldn't analyze", which is actually untrue. I had the
order of some prints wrong in the DecisionProbe.
* The code generator incorrectly detected when it could use a fixed,
acyclic inline DFA (i.e., using an IF). Upon non-LL(*) decisions
with predicates, analysis made cyclic DFA. But this stops
the computation detecting whether they are cyclic. I just added
a protection in front of the acyclic DFA generator to avoid if
non-LL(*). Updated comments.
January 23, 2007
* Made tree node streams use adaptor to create navigation nodes.
Thanks to Emond Papegaaij.
January 22, 2007
* Added lexer rule properties: start, stop
January 1, 2007
* analysis failsafe is back on; if a decision takes too long, it bails out
and uses k=1
January 1, 2007
* += labels for rules only work for output option; previously elements
of list were the return value structs, but are now either the tree or
StringTemplate return value. You can label different rules now
x+=a x+=b.
December 30, 2006
* Allow \" to work correctly in "..." template.
December 28, 2006
* errors that are now warnings: missing AST label type in trees.
Also "no start rule detected" is warning.
* tree grammars also can do rewrite=true for output=template.
Only works for alts with single node or tree as alt elements.
If you are going to use $text in a tree grammar or do rewrite=true
for templates, you must use in your main:
nodes.setTokenStream(tokens);
* You get a warning for tree grammars that do rewrite=true and
output=template and have -> for alts that are not simple nodes
or simple trees. new unit tests in TestRewriteTemplates at end.
December 27, 2006
* Error message appears when you use -> in tree grammar with
output=template and rewrite=true for alt that is not simple
node or tree ref.
* no more $stop attribute for tree parsers; meaningless/useless.
Removed from TreeRuleReturnScope also.
* rule text attribute in tree parser must pull from token buffer.
Makes no sense otherwise. added getTokenStream to TreeNodeStream
so rule $text attr works. CommonTreeNodeStream etc... now let
you set the token stream so you can access later from tree parser.
$text is not well-defined for rules like
slist : stat+ ;
because stat is not a single node nor rooted with a single node.
$slist.text will get only first stat. I need to add a warning about
this...
* Fixed http://www.antlr.org/browse/ANTLR-76 for Java.
Enhanced TokenRewriteStream so it accepts any object; converts
to string at last second. Allows you to rewrite with StringTemplate
templates now :)
* added rewrite option that makes -> template rewrites do replace ops for
TokenRewriteStream input stream. In output=template and rewrite=true mode
same as before 'cept that the parser does
((TokenRewriteStream)input).replace(
((Token)retval.start).getTokenIndex(),
input.LT(-1).getTokenIndex(),
retval.st);
after each rewrite so that the input stream is altered. Later refs to
$text will have rewrites. Here's a sample test program for grammar Rew.
FileReader groupFileR = new FileReader("Rew.stg");
StringTemplateGroup templates = new StringTemplateGroup(groupFileR);
ANTLRInputStream input = new ANTLRInputStream(System.in);
RewLexer lexer = new RewLexer(input);
TokenRewriteStream tokens = new TokenRewriteStream(lexer);
RewParser parser = new RewParser(tokens);
parser.setTemplateLib(templates);
parser.program();
System.out.println(tokens.toString());
groupFileR.close();
December 26, 2006
* BaseTree.dupTree didn't dup recursively.
December 24, 2006
* Cleaned up some comments and removed field treeNode
from MismatchedTreeNodeException class. It is "node" in
RecognitionException.
* Changed type from Object to BitSet for expecting fields in
MismatchedSetException and MismatchedNotSetException
* Cleaned up error printing in lexers and the messages that it creates.
* Added this to TreeAdaptor:
/** Return the token object from which this node was created.
* Currently used only for printing an error message.
* The error display routine in BaseRecognizer needs to
* display where the input the error occurred. If your
* tree of limitation does not store information that can
* lead you to the token, you can create a token filled with
* the appropriate information and pass that back. See
* BaseRecognizer.getErrorMessage().
*/
public Token getToken(Object t);
December 23, 2006
* made BaseRecognizer.displayRecognitionError nonstatic so people can
override it. Not sure why it was static before.
* Removed state/decision message that comes out of no
viable alternative exceptions, as that was too much.
removed the decision number from the early exit exception
also. During development, you can simply override
displayRecognitionError from BaseRecognizer to add the stuff
back in if you want.
* made output go to an output method you can override: emitErrorMessage()
* general cleanup of the error emitting code in BaseRecognizer. Lots
more stuff you can override: getErrorHeader, getTokenErrorDisplay,
emitErrorMessage, getErrorMessage.
December 22, 2006
* Altered Tree.Parser.matchAny() so that it skips entire trees if
node has children otherwise skips one node. Now this works to
skip entire body of function if single-rooted subtree:
^(FUNC name=ID arg=ID .)
* Added "reverse index" from node to stream index. Override
fillReverseIndex() in CommonTreeNodeStream if you want to change.
Use getNodeIndex(node) to find stream index for a specific tree node.
See getNodeIndex(), reverseIndex(Set tokenTypes),
reverseIndex(int tokenType), fillReverseIndex(). The indexing
costs time and memory to fill, but pulling stuff out will be lots
faster as it can jump from a node ptr straight to a stream index.
* Added TreeNodeStream.get(index) to make it easier for interpreters to
jump around in tree node stream.
* New CommonTreeNodeStream buffers all nodes in stream for fast jumping
around. It now has push/pop methods to invoke other locations in
the stream for building interpreters.
* Moved CommonTreeNodeStream to UnBufferedTreeNodeStream and removed
Iterator implementation. moved toNodesOnlyString() to TestTreeNodeStream
* [BREAKS ANY TREE IMPLEMENTATION]
made CommonTreeNodeStream work with any tree node type. TreeAdaptor
now implements isNil so must add; trivial, but does break back
compatibility.
December 17, 2006
* Added traceIn/Out methods to recognizers so that you can override them;
previously they were in-line print statements. The message has also
been slightly improved.
* Factored BuildParseTree into debug package; cleaned stuff up. Fixed
unit tests.
December 15, 2006
* [BREAKS ANY TREE IMPLEMENTATION]
org.antlr.runtime.tree.Tree; needed to add get/set for token start/stop
index so CommonTreeAdaptor can assume Tree interface not CommonTree
implementation. Otherwise, no way to create your own nodes that satisfy
Tree because CommonTreeAdaptor was doing
public int getTokenStartIndex(Object t) {
return ((CommonTree)t).startIndex;
}
Added to Tree:
/** What is the smallest token index (indexing from 0) for this node
* and its children?
*/
int getTokenStartIndex();
void setTokenStartIndex(int index);
/** What is the largest token index (indexing from 0) for this node
* and its children?
*/
int getTokenStopIndex();
void setTokenStopIndex(int index);
December 13, 2006
* Added org.antlr.runtime.tree.DOTTreeGenerator so you can generate DOT
diagrams easily from trees.
CharStream input = new ANTLRInputStream(System.in);
TLexer lex = new TLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lex);
TParser parser = new TParser(tokens);
TParser.e_return r = parser.e();
Tree t = (Tree)r.tree;
System.out.println(t.toStringTree());
DOTTreeGenerator gen = new DOTTreeGenerator();
StringTemplate st = gen.toDOT(t);
System.out.println(st);
* Changed the way mark()/rewind() work in CommonTreeNode stream to mirror
more flexible solution in ANTLRStringStream. Forgot to set lastMarker
anyway. Now you can rewind to non-most-recent marker.
December 12, 2006
* Temp lexer now end in .gl (T__.gl, for example)
* TreeParser suffix no longer generated for tree grammars
* Defined reset for lexer, parser, tree parser; rewinds the input stream also
December 10, 2006
* Made Grammar.abortNFAToDFAConversion() abort in middle of a DFA.
December 9, 2006
* fixed bug in OrderedHashSet.add(). It didn't track elements correctly.
December 6, 2006
* updated build.xml for future Ant compatibility, thanks to Matt Benson.
* various tests in TestRewriteTemplate and TestSyntacticPredicateEvaluation
were using the old 'channel' vs. new '$channel' notation.
TestInterpretedParsing didn't pick up an earlier change to CommonToken.
Reported by Matt Benson.
* fixed platform dependent test failures in TestTemplates, supplied by Matt
Benson.
November 29, 2006
* optimized semantic predicate evaluation so that p||!p yields true.
November 22, 2006
* fixed bug that prevented var = $rule.some_retval from working in anything
but the first alternative of a rule or subrule.
* attribute names containing digits were not allowed, this is now fixed,
allowing attributes like 'name1' but not '1name1'.
November 19, 2006
* Removed LeftRecursionMessage and apparatus because it seems that I check
for left recursion upfront before analysis and everything gets specified as
recursion cycles at this point.
November 16, 2006
* TokenRewriteStream.replace was not passing programName to next method.
November 15, 2006
* updated DOT files for DFA generation to make smaller circles.
* made epsilon edges italics in the NFA diagrams.
3.0b5 - November 15, 2006
The biggest thing is that your grammar file names must match the grammar name
inside (your generated class names will also be different) and we use
$channel=HIDDEN now instead of channel=99 inside lexer actions.
Should be compatible other than that. Please look at complete list of
changes.
November 14, 2006
* Force token index to be -1 for CommonIndex in case not set.
November 11, 2006
* getUniqueID for TreeAdaptor now uses identityHashCode instead of hashCode.
November 10, 2006
* No grammar nondeterminism warning now when wildcard '.' is final alt.
Examples:
a : A | B | . ;
A : 'a'
| .
;
SL_COMMENT
: '//' (options {greedy=false;} : .)* '\r'? '\n'
;
SL_COMMENT2
: '//' (options {greedy=false;} : 'x'|.)* '\r'? '\n'
;
November 8, 2006
* Syntactic predicates did not get hoisting properly upon non-LL(*) decision. Other hoisting issues fixed. Cleaned up code.
* Removed failsafe that check to see if I'm spending too much time on a single DFA; I don't think we need it anymore.
November 3, 2006
* $text, $line, etc... were not working in assignments. Fixed and added
test case.
* $label.text translated to label.getText in lexer even if label was on a char
November 2, 2006
* Added error if you don't specify what the AST type is; actions in tree
grammar won't work without it.
$ cat x.g
tree grammar x;
a : ID {String s = $ID.text;} ;
ANTLR Parser Generator Early Access Version 3.0b5 (??, 2006) 1989-2006
error: x.g:0:0: (152) tree grammar x has no ASTLabelType option
November 1, 2006
* $text, $line, etc... were not working properly within lexer rule.
October 32, 2006
* Finally actions now execute before dynamic scopes are popped it in the
rule. Previously was not possible to access the rules scoped variables
in a finally action.
October 29, 2006
* Altered ActionTranslator to emit errors on setting read-only attributes
such as $start, $stop, $text in a rule. Also forbid setting any attributes
in rules/tokens referenced by a label or name.
Setting dynamic scopes's attributes and your own parameter attributes
is legal.
October 27, 2006
* Altered how ANTLR figures out what decision is associated with which
block of grammar. Makes ANTLRWorks correctly find DFA for a block.
October 26, 2006
* Fixed bug where EOT transitions led to no NFA configs in a DFA state,
yielding an error in DFA table generation.
* renamed action.g to ActionTranslator.g
the ActionTranslator class is now called ActionTranslatorLexer, as ANTLR
generates this classname now. Fixed rest of codebase accordingly.
* added rules recognizing setting of scopes' attributes to ActionTranslator.g
the Objective C target needed access to the right-hand side of the assignment
in order to generate correct code
* changed ANTLRCore.sti to reflect the new mandatory templates to support the above
namely: scopeSetAttributeRef, returnSetAttributeRef and the ruleSetPropertyRef_*
templates, with the exception of ruleSetPropertyRef_text. we cannot set this attribute
October 19, 2006
* Fixed 2 bugs in DFA conversion that caused exceptions.
altered functionality of getMinElement so it ignores elements<0.
October 18, 2006
* moved resetStateNumbersToBeContiguous() to after issuing of warnings;
an internal error in that routine should make more sense as issues
with decision will appear first.
* fixed cut/paste bug I introduced when fixed EOF in min/max
bug. Prevented C grammar from working briefly.
October 17, 2006
* Removed a failsafe that seems to be unnecessary that ensure DFA didn't
get too big. It was resulting in some failures in code generation that
led me on quite a strange debugging trip.
October 16, 2006
* Use channel=HIDDEN not channel=99 to put tokens on hidden channel.
October 12, 2006
* ANTLR now has a customizable message format for errors and warnings,
to make it easier to fulfill requirements by IDEs and such.
The format to be used can be specified via the '-message-format name'
command line switch. The default for name is 'antlr', also available
at the moment is 'gnu'. This is done via StringTemplate, for details
on the requirements look in org/antlr/tool/templates/messages/formats/
* line numbers for lexers in combined grammars are now reported correctly.
September 29, 2006
* ANTLRReaderStream improperly checked for end of input.
September 28, 2006
* For ANTLRStringStream, LA(-1) was off by one...gave you LA(-2).
3.0b4 - August 24, 2006
* error when no rules in grammar. doesn't crash now.
* Token is now an interface.
* remove dependence on non runtime classes in runtime package.
* filename and grammar name must be same Foo in Foo.g. Generates FooParser,
FooLexer, ... Combined grammar Foo generates Foo$Lexer.g which generates
FooLexer.java. tree grammars generate FooTreeParser.java
August 24, 2006
* added C# target to lib, codegen, templates
August 11, 2006
* added tree arg to navigation methods in treeadaptor
August 07, 2006
* fixed bug related to (a|)+ on end of lexer rules. crashed instead
of warning.
* added warning that interpreter doesn't do synpreds yet
* allow different source of classloader:
ClassLoader cl = Thread.currentThread().getContextClassLoader();
if ( cl==null ) {
cl = this.getClass().getClassLoader();
}
July 26, 2006
* compressed DFA edge tables significantly. All edge tables are
unique. The transition table can reuse arrays. Look like this now:
public static readonly DFA30_transition0 =
new short[] { 46, 46, -1, 46, 46, -1, -1, -1, -1, -1, -1, -1,...};
public static readonly DFA30_transition1 =
new short[] { 21 };
public static readonly short[][] DFA30_transition = {
DFA30_transition0,
DFA30_transition0,
DFA30_transition1,
...
};
* If you defined both a label like EQ and '=', sometimes the '=' was
used instead of the EQ label.
* made headerFile template have same arg list as outputFile for consistency
* outputFile, lexer, genericParser, parser, treeParser templates
reference cyclicDFAs attribute which was no longer used after I
started the new table-based DFA. I made cyclicDFADescriptors
argument to outputFile and headerFile (only). I think this is
correct as only OO languages will want the DFA in the recognizer.
At the top level, C and friends can use it. Changed name to use
cyclicDFAs again as it's a better name probably. Removed parameter
from the lexer, ... For example, my parser template says this now:
<cyclicDFAs:cyclicDFA()> <! dump tables for all DFA !>
* made all token ref token types go thru code gen's
getTokenTypeAsTargetLabel()
* no more computing DFA transition tables for acyclic DFA.
July 25, 2006
* fixed a place where I was adding syn predicates into rewrite stuff.
* turned off invalid token index warning in AW support; had a problem.
* bad location event generated with -debug for synpreds in autobacktrack mode.
July 24, 2006
* changed runtime.DFA so that it treats all chars and token types as
char (unsigned 16 bit int). -1 becomes '\uFFFF' then or 65535.
* changed MAX_STATE_TRANSITIONS_FOR_TABLE to be 65534 by default
now. This means that all states can use a table to do transitions.
* was not making synpreds on (C)* type loops with backtrack=true
* was copying tree stuff and actions into synpreds with backtrack=true
* was making synpreds on even single alt rules / blocks with backtrack=true
3.0b3 - July 21, 2006
* ANTLR fails to analyze complex decisions much less frequently. It
turns out that the set of decisions for which ANTLR fails (times
out) is the same set (so far) of non-LL(*) decisions. Morever, I'm
able to detect this situation quickly and report rather than timing
out. Errors look like:
java.g:468:23: [fatal] rule concreteDimensions has non-LL(*)
decision due to recursive rule invocations in alts 1,2. Resolve
by left-factoring or using syntactic predicates with fixed k
lookahead or use backtrack=true option.
This message only appears when k=*.
* Shortened no viable alt messages to not include decision
description:
[compilationUnit, declaration]: line 8:8 decision=<<67:1: declaration
: ( ( fieldDeclaration )=> fieldDeclaration | ( methodDeclaration )=>
methodDeclaration | ( constructorDeclaration )=>
constructorDeclaration | ( classDeclaration )=> classDeclaration | (
interfaceDeclaration )=> interfaceDeclaration | ( blockDeclaration )=>
blockDeclaration | emptyDeclaration );>> state 3 (decision=14) no
viable alt; token=[@1,184:187='java',<122>,8:8]
too long and hard to read.
July 19, 2006
* Code gen bug: states with no emanating edges were ignored by ST.
Now an empty list is used.
* Added grammar parameter to recognizer templates so they can access
properties like getName(), ...
July 10, 2006
* Fixed the gated pred merged state bug. Added unit test.
* added new method to Target: getTokenTypeAsTargetLabel()
July 7, 2006
* I was doing an AND instead of OR in the gated predicate stuff.
Thanks to Stephen Kou!
* Reduce op for combining predicates was insanely slow sometimes and
didn't actually work well. Now it's fast and works.
* There is a bug in merging of DFA stop states related to gated
preds...turned it off for now.
3.0b2 - July 5, 2006
July 5, 2006
* token emission not properly protected in lexer filter mode.
* EOT, EOT DFA state transition tables should be init'd to -1 (only
was doing this for compressed tables). Fixed.
* in trace mode, exit method not shown for memoized rules
* added -Xmaxdfaedges to allow you to increase number of edges allowed
for a single DFA state before it becomes "special" and can't fit in
a simple table.
* Bug in tables. Short are signed so min/max tables for DFA are now
char[]. Bizarre.
July 3, 2006
* Added a method to reset the tool error state for current thread.
See ErrorManager.java
* [Got this working properly today] backtrack mode that let's you type
in any old crap and ANTLR will backtrack if it can't figure out what
you meant. No errors are reported by antlr during analysis. It
implicitly adds a syn pred in front of every production, using them
only if static grammar LL(*) analysis fails. Syn pred code is not
generated if the pred is not used in a decision.
This is essentially a rapid prototyping mode.
* Added backtracking report to the -report option
* Added NFA->DFA conversion early termination report to the -report option
* Added grammar level k and backtrack options to -report
* Added a dozen unit tests to test autobacktrack NFA construction.
* If you are using filter mode, you must manually use option
memoize=true now.
July 2, 2006
* Added k=* option so you can set k=2, for example, on whole grammar,
but an individual decision can be LL(*).
* memoize option for grammars, rules, blocks. Remove -nomemo cmd-line option
* but in DOT generator for DFA; fixed.
* runtime.DFA reported errors even when backtracking
July 1, 2006
* Added -X option list to help
* Syn preds were being hoisted into other rules, causing lots of extra
backtracking.
June 29, 2006
* unnecessary files removed during build.
* Matt Benson updated build.xml
* Detecting use of synpreds in analysis now instead of codegen. In
this way, I can avoid analyzing decisions in synpreds for synpreds
not used in a DFA for a real rule. This is used to optimize things
for backtrack option.
* Code gen must add _fragment or whatever to end of pred name in
template synpredRule to avoid having ANTLR know anything about
method names.
* Added -IdbgST option to emit ST delimiters at start/stop of all
templates spit out.
June 28, 2006
* Tweaked message when ANTLR cannot handle analysis.
3.0b1 - June 27, 2006
June 24, 2006
* syn preds no longer generate little static classes; they also don't
generate a whole bunch of extra crap in the rules built to test syn
preds. Removed GrammarFragmentPointer class from runtime.
June 23-24, 2006
* added output option to -report output.
* added profiling info:
Number of rule invocations in "guessing" mode
number of rule memoization cache hits
number of rule memoization cache misses
* made DFA DOT diagrams go left to right not top to bottom
* I try to recursive overflow states now by resolving these states
with semantic/syntactic predicates if they exist. The DFA is then
deterministic rather than simply resolving by choosing first
nondeterministic alt. I used to generated errors:
~/tmp $ java org.antlr.Tool -dfa t.g
ANTLR Parser Generator Early Access Version 3.0b2 (July 5, 2006) 1989-2006
t.g:2:5: Alternative 1: after matching input such as A A A A A decision cannot predict what comes next due to recursion overflow to b from b
t.g:2:5: Alternative 2: after matching input such as A A A A A decision cannot predict what comes next due to recursion overflow to b from b
Now, I uses predicates if available and emits no warnings.
* made sem preds share accept states. Previously, multiple preds in a
decision forked new accepts each time for each nondet state.
June 19, 2006
* Need parens around the prediction expressions in templates.
* Referencing $ID.text in an action forced bad code gen in lexer rule ID.
* Fixed a bug in how predicates are collected. The definition of
"last predicated alternative" was incorrect in the analysis. Further,
gated predicates incorrectly missed a case where an edge should become
true (a tautology).
* Removed an unnecessary input.consume() reference in the runtime/DFA class.
June 14, 2006
* -> ($rulelabel)? didn't generate proper code for ASTs.
* bug in code gen (did not compile)
a : ID -> ID
| ID -> ID
;
Problem is repeated ref to ID from left side. Juergen pointed this out.
* use of tokenVocab with missing file yielded exception
* (A|B)=> foo yielded an exception as (A|B) is a set not a block. Fixed.
* Didn't set ID1= and INT1= for this alt:
| ^(ID INT+ {System.out.print(\"^(\"+$ID+\" \"+$INT+\")\");})
* Fixed so repeated dangling state errors only occur once like:
t.g:4:17: the decision cannot distinguish between alternative(s) 2,1 for at least one input sequence
* tracking of rule elements was on (making list defs at start of
method) with templates instead of just with ASTs. Turned off.
* Doesn't crash when you give it a missing file now.
* -report: add output info: how many LL(1) decisions.
June 13, 2006
* ^(ROOT ID?) Didn't work; nor did any other nullable child list such as
^(ROOT ID* INT?). Now, I check to see if child list is nullable using
Grammar.LOOK() and, if so, I generate an "IF lookahead is DOWN" gate
around the child list so the whole thing is optional.
* Fixed a bug in LOOK that made it not look through nullable rules.
* Using AST suffixes or -> rewrite syntax now gives an error w/o a grammar
output option. Used to crash ;)
* References to EOF ended up with improper -1 refs instead of EOF in output.
* didn't warn of ambig ref to $expr in rewrite; fixed.
list
: '[' expr 'for' type ID 'in' expr ']'
-> comprehension(expr={$expr.st},type={},list={},i={})
;
June 12, 2006
* EOF works in the parser as a token name.
* Rule b:(A B?)*; didn't display properly in AW due to the way ANTLR
generated NFA.
* "scope x;" in a rule for unknown x gives no error. Fixed. Added unit test.
* Label type for refs to start/stop in tree parser and other parsers were
not used. Lots of casting. Ick. Fixed.
* couldn't refer to $tokenlabel in isolation; but need so we can test if
something was matched. Fixed.
* Lots of little bugs fixed in $x.y, %... translation due to new
action translator.
* Improperly tracking block nesting level; result was that you couldn't
see $ID in action of rule "a : A+ | ID {Token t = $ID;} | C ;"
* a : ID ID {$ID.text;} ; did not get a warning about ambiguous $ID ref.
* No error was found on $COMMENT.text:
COMMENT
: '/*' (options {greedy=false;} : . )* '*/'
{System.out.println("found method "+$COMMENT.text);}
;
$enclosinglexerrule scope does not exist. Use text or setText() here.
June 11, 2006
* Single return values are initialized now to default or to your spec.
* cleaned up input stream stuff. Added ANTLRReaderStream, ANTLRInputStream
and refactored. You can specify encodings now on ANTLRFileStream (and
ANTLRInputStream) now.
* You can set text local var now in a lexer rule and token gets that text.
start/stop indexes are still set for the token.
* Changed lexer slightly. Calling a nonfragment rule from a
nonfragment rule does not set the overall token.
June 10, 2006
* Fixed bug where unnecessary escapes yield char==0 like '\{'.
* Fixed analysis bug. This grammar didn't report a recursion warning:
x : y X
| y Y
;
y : L y R
| B
;
The DFAState.equals() method was messed up.
* Added @synpredgate {...} action so you can tell ANTLR how to gate actions
in/out during syntactic predicate evaluation.
* Fuzzy parsing should be more efficient. It should backtrack over a rule
and then rewind and do it again "with feeling" to exec actions. It was
actually doing it 3x not 2x.
June 9, 2006
* Gutted and rebuilt the action translator for $x.y, $x::y, ...
Uses ANTLR v3 now for the first time inside v3 source. :)
ActionTranslator.java
* Fixed a bug where referencing a return value on a rule didn't work
because later a ref to that rule's predefined properties didn't
properly force a return value struct to be built. Added unit test.
June 6, 2006
* New DFA mechanisms. Cyclic DFA are implemented as state tables,
encoded via strings as java cannot handle large static arrays :(
States with edges emanating that have predicates are specially
treated. A method is generated to do these states. The DFA
simulation routine uses the "special" array to figure out if the
state is special. See March 25, 2006 entry for description:
http://www.antlr.org/blog/antlr3/codegen.tml. analysis.DFA now has
all the state tables generated for code gen. CyclicCodeGenerator.java
disappeared as it's unneeded code. :)
* Internal general clean up of the DFA.states vs uniqueStates thing.
Fixed lookahead decisions no longer fill uniqueStates. Waste of
time. Also noted that when adding sem pred edges, I didn't check
for state reuse. Fixed.
June 4, 2006
* When resolving ambig DFA states predicates, I did not add the new states
to the list of unique DFA states. No observable effect on output except
that DFA state numbers were not always contiguous for predicated decisions.
I needed this fix for new DFA tables.
3.0ea10 - June 2, 2006
June 2, 2006
* Improved grammar stats and added syntactic pred tracking.
June 1, 2006
* Due to a type mismatch, the DebugParser.recoverFromMismatchedToken()
method was not called. Debug events for mismatched token error
notification were not sent to ANTLRWorks probably
* Added getBacktrackingLevel() for any recognizer; needed for profiler.
* Only writes profiling data for antlr grammar analysis with -profile set
* Major update and bug fix to (runtime) Profiler.
May 27, 2006
* Added Lexer.skip() to force lexer to ignore current token and look for
another; no token is created for current rule and is not passed on to
parser (or other consumer of the lexer).
* Parsers are much faster now. I removed use of java.util.Stack for pushing
follow sets and use a hardcoded array stack instead. Dropped from
5900ms to 3900ms for parse+lex time parsing entire java 1.4.2 source. Lex
time alone was about 1500ms. Just looking at parse time, we get about 2x
speed improvement. :)
May 26, 2006
* Fixed NFA construction so it generates NFA for (A*)* such that ANTLRWorks
can display it properly.
May 25, 2006
* added abort method to Grammar so AW can terminate the conversion if it's
taking too long.
May 24, 2006
* added method to get left recursive rules from grammar without doing full
grammar analysis.
* analysis, code gen not attempted if serious error (like
left-recursion or missing rule definition) occurred while reading
the grammar in and defining symbols.
* added amazing optimization; reduces analysis time by 90% for java
grammar; simple IF statement addition!
3.0ea9 - May 20, 2006
* added global k value for grammar to limit lookahead for all decisions unless
overridden in a particular decision.
* added failsafe so that any decision taking longer than 2 seconds to create
the DFA will fall back on k=1. Use -ImaxtimeforDFA n (in ms) to set the time.
* added an option (turned off for now) to use multiple threads to
perform grammar analysis. Not much help on a 2-CPU computer as
garbage collection seems to peg the 2nd CPU already. :( Gotta wait for
a 4 CPU box ;)
* switched from #src to // $ANTLR src directive.
* CommonTokenStream.getTokens() looked past end of buffer sometimes. fixed.
* unicode literals didn't really work in DOT output and generated code. fixed.
* fixed the unit test rig so it compiles nicely with Java 1.5
* Added ant build.xml file (reads build.properties file)
* predicates sometimes failed to compile/eval properly due to missing (...)
in IF expressions. Forced (..)
* (...)? with only one alt were not optimized. Was:
// t.g:4:7: ( B )?
int alt1=2;
int LA1_0 = input.LA(1);
if ( LA1_0==B ) {
alt1=1;
}
else if ( LA1_0==-1 ) {
alt1=2;
}
else {
NoViableAltException nvae =
new NoViableAltException("4:7: ( B )?", 1, 0, input);
throw nvae;
}
is now:
// t.g:4:7: ( B )?
int alt1=2;
int LA1_0 = input.LA(1);
if ( LA1_0==B ) {
alt1=1;
}
Smaller, faster and more readable.
* Allow manual init of return values now:
functionHeader returns [int x=3*4, char (*f)()=null] : ... ;
* Added optimization for DFAs that fixed a codegen bug with rules in lexer:
EQ : '=' ;
ASSIGNOP : '=' | '+=' ;
EQ is a subset of other rule. It did not given an error which is
correct, but generated bad code.
* ANTLR was sending column not char position to ANTLRWorks.
* Bug fix: location 0, 0 emitted for synpreds and empty alts.
* debugging event handshake how sends grammar file name. Added getGrammarFileName() to recognizers. Java.stg generates it:
public String getGrammarFileName() { return "<fileName>"; }
* tree parsers can do arbitrary lookahead now including backtracking. I
updated CommonTreeNodeStream.
* added events for debugging tree parsers:
/** Input for a tree parser is an AST, but we know nothing for sure
* about a node except its type and text (obtained from the adaptor).
* This is the analog of the consumeToken method. Again, the ID is
* the hashCode usually of the node so it only works if hashCode is
* not implemented.
*/
public void consumeNode(int ID, String text, int type);
/** The tree parser looked ahead */
public void LT(int i, int ID, String text, int type);
/** The tree parser has popped back up from the child list to the
* root node.
*/
public void goUp();
/** The tree parser has descended to the first child of a the current
* root node.
*/
public void goDown();
* Added DebugTreeNodeStream and DebugTreeParser classes
* Added ctor because the debug tree node stream will need to ask quesitons about nodes and since nodes are just Object, it needs an adaptor to decode the nodes and get text/type info for the debugger.
public CommonTreeNodeStream(TreeAdaptor adaptor, Tree tree);
* added getter to TreeNodeStream:
public TreeAdaptor getTreeAdaptor();
* Implemented getText/getType in CommonTreeAdaptor.
* Added TraceDebugEventListener that can dump all events to stdout.
* I broke down and make Tree implement getText
* tree rewrites now gen location debug events.
* added AST debug events to listener; added blank listener for convenience
* updated debug events to send begin/end backtrack events for debugging
* with a : (b->b) ('+' b -> ^(PLUS $a b))* ; you get b[0] each time as
there is no loop in rewrite rule itself. Need to know context that
the -> is inside the rule and hence b means last value of b not all
values.
* Bug in TokenRewriteStream; ops at indexes < start index blocked proper op.
* Actions in ST rewrites "-> ({$op})()" were not translated
* Added new action name:
@rulecatch {
catch (RecognitionException re) {
reportError(re);
recover(input,re);
}
catch (Throwable t) {
System.err.println(t);
}
}
Overrides rule catch stuff.
* Isolated $ refs caused exception
3.0ea8 - March 11, 2006
* added @finally {...} action like @init for rules. Executes in
finally block (java target) after all other stuff like rule memoization.
No code changes needs; ST just refs a new action:
<ruleDescriptor.actions.finally>
* hideous bug fixed: PLUS='+' didn't result in '+' rule in lexer
* TokenRewriteStream didn't do toString() right when no rewrites had been done.
* lexer errors in interpreter were not printed properly
* bitsets are dumped in hex not decimal now for FOLLOW sets
* /* epsilon */ is not printed now when printing out grammars with empty alts
* Fixed another bug in tree rewrite stuff where it was checking that elements
had at least one element. Strange...commented out for now to see if I can remember what's up.
* Tree rewrites had problems when you didn't have x+=FOO variables. Rules
like this work now:
a : (x=ID)? y=ID -> ($x $y)?;
* filter=true for lexers turns on k=1 and backtracking for every token
alternative. Put the rules in priority order.
* added getLine() etc... to Tree to support better error reporting for
trees. Added MismatchedTreeNodeException.
* $templates::foo() is gone. added % as special template symbol.
%foo(a={},b={},...) ctor (even shorter than $templates::foo(...))
%({name-expr})(a={},...) indirect template ctor reference
The above are parsed by antlr.g and translated by codegen.g
The following are parsed manually here:
%{string-expr} anonymous template from string expr
%{expr}.y = z; template attribute y of StringTemplate-typed expr to z
%x.y = z; set template attribute y of x (always set never get attr)
to z [languages like python without ';' must still use the
';' which the code generator is free to remove during code gen]
* -> ({expr})(a={},...) notation for indirect template rewrite.
expr is the name of the template.
* $x[i]::y and $x[-i]::y notation for accesssing absolute scope stack
indexes and relative negative scopes. $x[-1]::y is the y attribute
of the previous scope (stack top - 1).
* filter=true mode for lexers; can do this now...upon mismatch, just
consumes a char and tries again:
lexer grammar FuzzyJava;
options {filter=true;}
FIELD
: TYPE WS? name=ID WS? (';'|'=')
{System.out.println("found var "+$name.text);}
;
* refactored char streams so ANTLRFileStream is now a subclass of
ANTLRStringStream.
* char streams for lexer now allowed nested backtracking in lexer.
* added TokenLabelType for lexer/parser for all token labels
* line numbers for error messages were not updated properly in antlr.g
for strings, char literals and <<...>>
* init action in lexer rules was before the type,start,line,... decls.
* Tree grammars can now specify output; I've only tested output=templat
though.
* You can reference EOF now in the parser and lexer. It's just token type
or char value -1.
* Bug fix: $ID refs in the *lexer* were all messed up. Cleaned up the
set of properties available...
* Bug fix: .st not found in rule ref when rule has scope:
field
scope {
StringTemplate funcDef;
}
: ...
{$field::funcDef = $field.st;}
;
it gets field_stack.st instead
* return in backtracking must return retval or null if return value.
* $property within a rule now works like $text, $st, ...
* AST/Template Rewrites were not gated by backtracking==0 so they
executed even when guessing. Auto AST construction is now gated also.
* CommonTokenStream was somehow returning tokens not text in toString()
* added useful methods to runtime.BitSet and also to CommonToken so you can
update the text. Added nice Token stream method:
/** Given a start and stop index, return a List of all tokens in
* the token type BitSet. Return null if no tokens were found. This
* method looks at both on and off channel tokens.
*/
public List getTokens(int start, int stop, BitSet types);
* literals are now passed in the .tokens files so you can ref them in
tree parses, for example.
* added basic exception handling; no labels, just general catches:
a : {;}A | B ;
exception
catch[RecognitionException re] {
System.out.println("recog error");
}
catch[Exception e] {
System.out.println("error");
}
* Added method to TokenStream:
public String toString(Token start, Token stop);
* antlr generates #src lines in lexer grammars generated from combined grammars
so error messages refer to original file.
* lexers generated from combined grammars now use originally formatting.
* predicates have $x.y stuff translated now. Warning: predicates might be
hoisted out of context.
* return values in return val structs are now public.
* output=template with return values on rules was broken. I assume return values with ASTs was broken too. Fixed.
3.0ea7 - December 14, 2005
* Added -print option to print out grammar w/o actions
* Renamed BaseParser to be BaseRecognizer and even made Lexer derive from
this; nice as it now shares backtracking support code.
* Added syntactic predicates (...)=>. See December 4, 2005 entry:
http://www.antlr.org/blog/antlr3/lookahead.tml
Note that we have a new option for turning off rule memoization during
backtracking:
-nomemo when backtracking don't generate memoization code
* Predicates are now tested in order that you specify the alts. If you
leave the last alt "naked" (w/o pred), it will assume a true pred rather
than union of other preds.
* Added gated predicates "{p}?=>" that literally turn off a production whereas
disambiguating predicates are only hoisted into the predictor when syntax alone
is not sufficient to uniquely predict alternatives.
A : {p}? => "a" ;
B : {!p}? => ("a"|"b")+ ;
* bug fixed related to predicates in predictor
lexer grammar w;
A : {p}? "a" ;
B : {!p}? ("a"|"b")+ ;
DFA is correct. A state splits for input "a" on the pred.
Generated code though was hosed. No pred tests in prediction code!
I added testLexerPreds() and others in TestSemanticPredicateEvaluation.java
* added execAction template in case we want to do something in front of
each action execution or something.
* left-recursive cycles from rules w/o decisions were not detected.
* undefined lexer rules were not announced! fixed.
* unreachable messages for Tokens rule now indicate rule name not alt. E.g.,
Ruby.lexer.g:24:1: The following token definitions are unreachable: IVAR
* nondeterminism warnings improved for Tokens rule:
Ruby.lexer.g:10:1: Multiple token rules can match input such as ""0".."9"": INT, FLOAT
As a result, tokens(s) FLOAT were disabled for that input
* DOT diagrams didn't show escaped char properly.
* Char/string literals are now all 'abc' not "abc".
* action syntax changed "@scope::actionname {action}" where scope defaults
to "parser" if parser grammar or combined grammar, "lexer" if lexer grammar,
and "treeparser" if tree grammar. The code generation targets decide
what scopes are available. Each "scope" yields a hashtable for use in
the output templates. The scopes full of actions are sent to all output
file templates (currently headerFile and outputFile) as attribute actions.
Then you can reference <actions.scope> to get the map of actions associated
with scope and <actions.parser.header> to get the parser's header action
for example. This should be very flexible. The target should only have
to define which scopes are valid, but the action names should be variable
so we don't have to recompile ANTLR to add actions to code gen templates.
grammar T;
options {language=Java;}
@header { package foo; }
@parser::stuff { int i; } // names within scope not checked; target dependent
@members { int i; }
@lexer::header {head}
@lexer::members { int j; }
@headerfile::blort {...} // error: this target doesn't have headerfile
@treeparser::members {...} // error: this is not a tree parser
a
@init {int i;}
: ID
;
ID : 'a'..'z';
For now, the Java target uses members and header as a valid name. Within a
rule, the init action name is valid.
* changed $dynamicscope.value to $dynamicscope::value even if value is defined
in same rule such as $function::name where rule function defines name.
* $dynamicscope gets you the stack
* rule scopes go like this now:
rule
scope {...}
scope slist,Symbols;
: ...
;
* Created RuleReturnScope as a generic rule return value. Makes it easier
to do this:
RuleReturnScope r = parser.program();
System.out.println(r.getTemplate().toString());
* $template, $tree, $start, etc...
* $r.x in current rule. $r is ignored as fully-qualified name. $r.start works too
* added warning about $r referring to both return value of rule and dynamic scope of rule
* integrated StringTemplate in a very simple manner
Syntax:
-> template(arglist) "..."
-> template(arglist) <<...>>
-> namedTemplate(arglist)
-> {free expression}
-> // empty
Predicate syntax:
a : A B -> {p1}? foo(a={$A.text})
-> {p2}? foo(a={$B.text})
-> // return nothing
An arg list is just a list of template attribute assignments to actions in curlies.
There is a setTemplateLib() method for you to use with named template rewrites.
Use a new option:
grammar t;
options {output=template;}
...
This all should work for tree grammars too, but I'm still testing.
* fixed bugs where strings were improperly escaped in exceptions, comments, etc.. For example, newlines came out as newlines not the escaped version
3.0ea6 - November 13, 2005
* turned off -debug/-profile, which was on by default
* completely refactored the output templates; added some missing templates.
* dramatically improved infinite recursion error messages (actually
left-recursion never even was printed out before).
* wasn't printing dangling state messages when it reanalyzes with k=1.
* fixed a nasty bug in the analysis engine dealing with infinite recursion.
Spent all day thinking about it and cleaned up the code dramatically.
Bug fixed and software is more powerful and I understand it better! :)
* improved verbose DFA nodes; organized by alt
* got much better random phrase generation. For example:
$ java org.antlr.tool.RandomPhrase simple.g program
int Ktcdn ';' method wh '(' ')' '{' return 5 ';' '}'
* empty rules like "a : ;" generated code that didn't compile due to
try/catch for RecognitionException. Generated code couldn't possibly
throw that exception.
* when printing out a grammar, such as in comments in generated code,
ANTLR didn't print ast suffix stuff back out for literals.
* This never exited loop:
DATA : (options {greedy=false;}: .* '\n' )* '\n' '.' ;
and now it works due to new default nongreedy .* Also this works:
DATA : (options {greedy=false;}: .* '\n' )* '.' ;
* Dot star ".*" syntax didn't work; in lexer it is nongreedy by
default. In parser it is on greedy but also k=1 by default. Added
unit tests. Added blog entry to describe.
* ~T where T is the only token yielded an empty set but no error
* Used to generate unreachable message here:
parser grammar t;
a : ID a
| ID
;
z.g:3:11: The following alternatives are unreachable: 2
In fact it should really be an error; now it generates:
no start rule in grammar t (no rule can obviously be followed by EOF)
Per next change item, ANTLR cannot know that EOF follows rule 'a'.
* added error message indicating that ANTLR can't figure out what your
start rule is. Required to properly generate code in some cases.
* validating semantic predicates now work (if they are false, they
throw a new FailedPredicateException
* two hideous bug fixes in the IntervalSet, which made analysis go wrong
in a few cases. Thanks to Oliver Zeigermann for finding lots of bugs
and making suggested fixes (including the next two items)!
* cyclic DFAs are now nonstatic and hence can access instance variables
* labels are now allowed on lexical elements (in the lexer)
* added some internal debugging options
* ~'a'* and ~('a')* were not working properly; refactored antlr.g grammar
3.0ea5 - July 5, 2005
* Using '\n' in a parser grammar resulted in a nonescaped version of '\n' in the token names table making compilation fail. I fixed this by reorganizing/cleaning up portion of ANTLR that deals with literals. See comment org.antlr.codegen.Target.
* Target.getMaxCharValue() did not use the appropriate max value constant.
* ALLCHAR was a constant when it should use the Target max value def. set complement for wildcard also didn't use the Target def. Generally cleaned up the max char value stuff.
* Code gen didn't deal with ASTLabelType properly...I think even the 3.0ea7 example tree parser was broken! :(
* Added a few more unit tests dealing with escaped literals
3.0ea4 - June 29, 2005
* tree parsers work; added CommonTreeNodeStream. See simplecTreeParser
example in examples-v3 tarball.
* added superClass and ASTLabelType options
* refactored Parser to have a BaseParser and added TreeParser
* bug fix: actions being dumped in description strings; compile errors
resulted
3.0ea3 - June 23, 2005
Enhancements
* Automatic tree construction operators are in: ! ^ ^^
* Tree construction rewrite rules are in
-> {pred1}? rewrite1
-> {pred2}? rewrite2
...
-> rewriteN
The rewrite rules may be elements like ID, expr, $label, {node expr}
and trees ^( <root> <children> ). You have have (...)?, (...)*, (...)+
subrules as well.
You may have rewrites in subrules not just at outer level of rule, but
any -> rewrite forces auto AST construction off for that alternative
of that rule.
To avoid cycles, copy semantics are used:
r : INT -> INT INT ;
means make two new nodes from the same INT token.
Repeated references to a rule element implies a copy for at least one
tree:
a : atom -> ^(atom atom) ; // NOT CYCLE! (dup atom tree)
* $ruleLabel.tree refers to tree created by matching the labeled element.
* A description of the blocks/alts is generated as a comment in output code
* A timestamp / signature is put at top of each generated code file
3.0ea2 - June 12, 2005
Bug fixes
* Some error messages were missing the stackTrace parameter
* Removed the file locking mechanism as it's not cross platform
* Some absolute vs relative path name problems with writing output
files. Rules are now more concrete. -o option takes precedence
// -o /tmp /var/lib/t.g => /tmp/T.java
// -o subdir/output /usr/lib/t.g => subdir/output/T.java
// -o . /usr/lib/t.g => ./T.java
// -o /tmp subdir/t.g => /tmp/subdir/t.g
// If they didn't specify a -o dir so just write to location
// where grammar is, absolute or relative
* does error checking on unknown option names now
* Using just language code not locale name for error message file. I.e.,
the default (and for any English speaking locale) is en.stg not en_US.stg
anymore.
* The error manager now asks the Tool to panic rather than simply doing
a System.exit().
* Lots of refactoring concerning grammar, rule, subrule options. Now
detects invalid options.
3.0ea1 - June 1, 2005
Initial early access release
antlr-3.2/tool/pom.xml 0000644 0001750 0001750 00000006555 11256466764 014762 0 ustar twerner twerner <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.antlr</groupId>
<artifactId>antlr</artifactId>
<packaging>jar</packaging>
<name>ANTLR Grammar Tool</name>
<url>http://antlr.org</url>
<!--
Inherit from the ANTLR master pom, which tells us what
version we are and allows us to inherit dependencies
and so on.
-->
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr-master</artifactId>
<version>3.2</version>
</parent>
<dependencies>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr-runtime</artifactId>
<version>${project.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.5</version>
<scope>test</scope>
</dependency>
</dependencies>
<!--
Tell Maven which other artifacts we need in order to
build, run and test the ANTLR Tool. The ANTLR Tool uses earlier versions
of ANTLR at runtime (for the moment), uses the current
released version of ANTLR String template, but obviously is
reliant on the latest snapshot of the runtime, which will either be
taken from the antlr-snapshot repository, or your local .m2
repository if you built and installed that locally.
-->
<build>
<defaultGoal>install</defaultGoal>
<plugins>
<plugin>
<groupId>org.antlr</groupId>
<artifactId>antlr3-maven-plugin</artifactId>
<version>3.1.3-1</version>
<configuration>
<libDirectory>target/generated-sources/antlr/org/antlr/grammar/v3</libDirectory>
</configuration>
<executions>
<execution>
<goals>
<goal>antlr</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>antlr-maven-plugin</artifactId>
<configuration>
<sourceDirectory>src/main/antlr2/org/antlr/grammar/v2</sourceDirectory>
<grammars>antlr.g codegen.g, antlr.print.g, assign.types.g, buildnfa.g, define.g</grammars>
</configuration>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
<sourceDirectory>src</sourceDirectory>
</configuration>
</plugin>
</plugins>
</build>
</project>
antlr-3.2/gunit-maven-plugin/ 0000755 0001750 0001750 00000000000 11410174110 016150 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/ 0000755 0001750 0001750 00000000000 11410174110 016737 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/main/ 0000755 0001750 0001750 00000000000 11410174110 017663 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/main/java/ 0000755 0001750 0001750 00000000000 11410174110 020604 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/main/java/org/ 0000755 0001750 0001750 00000000000 11410174110 021373 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/main/java/org/antlr/ 0000755 0001750 0001750 00000000000 11410174110 022513 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/main/java/org/antlr/mojo/ 0000755 0001750 0001750 00000000000 11410174110 023457 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/main/java/org/antlr/mojo/antlr3/ 0000755 0001750 0001750 00000000000 11410174110 024662 5 ustar twerner twerner antlr-3.2/gunit-maven-plugin/src/main/java/org/antlr/mojo/antlr3/GUnitExecuteMojo.java 0000644 0001750 0001750 00000032347 11256465172 030757 0 ustar twerner twerner package org.antlr.mojo.antlr3;
import java.util.List;
import java.util.Set;
import java.util.HashSet;
import java.util.ArrayList;
import java.util.Collections;
import java.io.File;
import java.io.IOException;
import java.io.Writer;
import java.io.FileWriter;
import java.io.BufferedWriter;
import java.net.URL;
import java.net.MalformedURLException;
import java.net.URLClassLoader;
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugin.MojoExecutionException;
import org.apache.maven.plugin.MojoFailureException;
import org.apache.maven.project.MavenProject;
import org.apache.maven.artifact.Artifact;
import org.apache.maven.artifact.DependencyResolutionRequiredException;
import org.apache.maven.artifact.versioning.ArtifactVersion;
import org.apache.maven.artifact.versioning.DefaultArtifactVersion;
import org.apache.maven.artifact.versioning.OverConstrainedVersionException;
import org.codehaus.plexus.util.StringUtils;
import org.codehaus.plexus.util.FileUtils;
import org.codehaus.plexus.compiler.util.scan.mapping.SourceMapping;
import org.codehaus.plexus.compiler.util.scan.mapping.SuffixMapping;
import org.codehaus.plexus.compiler.util.scan.SourceInclusionScanner;
import org.codehaus.plexus.compiler.util.scan.SimpleSourceInclusionScanner;
import org.codehaus.plexus.compiler.util.scan.InclusionScanException;
import org.antlr.runtime.ANTLRFileStream;
import org.antlr.runtime.RecognitionException;
import org.antlr.gunit.GrammarInfo;
import org.antlr.gunit.gUnitExecutor;
import org.antlr.gunit.AbstractTest;
import org.antlr.gunit.Interp;
/**
* Takes gUnit scripts and directly performs testing.
*
* @goal gunit
*
* @phase test
* @requiresDependencyResolution test
* @requiresProject true
*
* @author Steve Ebersole
*/
public class GUnitExecuteMojo extends AbstractMojo {
public static final String ANTLR_GROUP_ID = "org.antlr";
public static final String ANTLR_ARTIFACT_NAME = "antlr";
public static final String ANTLR_RUNTIME_ARTIFACT_NAME = "antlr-runtime";
/**
* INTERNAL : The Maven Project to which we are attached
*
* @parameter expression="${project}"
* @required
*/
private MavenProject project;
/**
* INTERNAL : The artifacts associated to the dependencies defined as part
* of our configuration within the project to which we are being attached.
*
* @parameter expression="${plugin.artifacts}"
* @required
* @readonly
*/
private List<Artifact> pluginArtifacts;
/**
* Specifies the directory containing the gUnit testing files.
*
* @parameter expression="${basedir}/src/test/gunit"
* @required
*/
private File sourceDirectory;
/**
* A set of patterns for matching files from the sourceDirectory that
* should be included as gUnit source files.
*
* @parameter
*/
private Set includes;
/**
* A set of exclude patterns.
*
* @parameter
*/
private Set excludes;
/**
* Specifies directory to which gUnit reports should get written.
*
* @parameter expression="${basedir}/target/gunit-report"
* @required
*/
private File reportDirectory;
/**
* Should gUnit functionality be completely by-passed?
* <p/>
* By default we skip gUnit tests if the user requested that all testing be skipped using 'maven.test.skip'
*
* @parameter expression="${maven.test.skip}"
*/
private boolean skip;
public Set getIncludePatterns() {
return includes == null || includes.isEmpty()
? Collections.singleton( "**/*.testsuite" )
: includes;
}
public Set getExcludePatterns() {
return excludes == null
? Collections.emptySet()
: excludes;
}
public final void execute() throws MojoExecutionException, MojoFailureException {
if ( skip ) {
getLog().info( "Skipping gUnit processing" );
return;
}
Artifact pluginAntlrArtifact = determinePluginAntlrArtifact();
validateProjectsAntlrVersion( determineArtifactVersion( pluginAntlrArtifact ) );
performExecution( determineProjectCompileScopeClassLoader( pluginAntlrArtifact ) );
}
private Artifact determinePluginAntlrArtifact() throws MojoExecutionException {
for ( Artifact artifact : pluginArtifacts ) {
boolean match = ANTLR_GROUP_ID.equals( artifact.getGroupId() )
&& ANTLR_ARTIFACT_NAME.equals( artifact.getArtifactId() );
if ( match ) {
return artifact;
}
}
throw new MojoExecutionException(
"Unexpected state : could not locate " + ANTLR_GROUP_ID + ':' + ANTLR_ARTIFACT_NAME +
" in plugin dependencies"
);
}
private ArtifactVersion determineArtifactVersion(Artifact artifact) throws MojoExecutionException {
try {
return artifact.getVersion() != null
? new DefaultArtifactVersion( artifact.getVersion() )
: artifact.getSelectedVersion();
}
catch ( OverConstrainedVersionException e ) {
throw new MojoExecutionException( "artifact [" + artifact.getId() + "] defined an overly constrained version range" );
}
}
private void validateProjectsAntlrVersion(ArtifactVersion pluginAntlrVersion) throws MojoExecutionException {
Artifact antlrArtifact = null;
Artifact antlrRuntimeArtifact = null;
if ( project.getCompileArtifacts() != null ) {
for ( Object o : project.getCompileArtifacts() ) {
final Artifact artifact = ( Artifact ) o;
if ( ANTLR_GROUP_ID.equals( artifact.getGroupId() ) ) {
if ( ANTLR_ARTIFACT_NAME.equals( artifact.getArtifactId() ) ) {
antlrArtifact = artifact;
break;
}
if ( ANTLR_RUNTIME_ARTIFACT_NAME.equals( artifact.getArtifactId() ) ) {
antlrRuntimeArtifact = artifact;
}
}
}
}
validateBuildTimeArtifact( antlrArtifact, pluginAntlrVersion );
validateRunTimeArtifact( antlrRuntimeArtifact, pluginAntlrVersion );
}
@SuppressWarnings(value = "unchecked")
protected void validateBuildTimeArtifact(Artifact antlrArtifact, ArtifactVersion pluginAntlrVersion)
throws MojoExecutionException {
if ( antlrArtifact == null ) {
validateMissingBuildtimeArtifact();
return;
}
// otherwise, lets make sure they match...
ArtifactVersion projectAntlrVersion = determineArtifactVersion( antlrArtifact );
if ( pluginAntlrVersion.compareTo( projectAntlrVersion ) != 0 ) {
getLog().warn(
"Encountered " + ANTLR_GROUP_ID + ':' + ANTLR_ARTIFACT_NAME + ':' + projectAntlrVersion.toString() +
" which did not match Antlr version used by plugin [" + pluginAntlrVersion.toString() + "]"
);
}
}
protected void validateMissingBuildtimeArtifact() {
// generally speaking, its ok for the project to not define a dep on the build-time artifact...
}
@SuppressWarnings(value = "unchecked")
protected void validateRunTimeArtifact(Artifact antlrRuntimeArtifact, ArtifactVersion pluginAntlrVersion)
throws MojoExecutionException {
if ( antlrRuntimeArtifact == null ) {
// its possible, if the project instead depends on the build-time (or full) artifact.
return;
}
ArtifactVersion projectAntlrVersion = determineArtifactVersion( antlrRuntimeArtifact );
if ( pluginAntlrVersion.compareTo( projectAntlrVersion ) != 0 ) {
getLog().warn(
"Encountered " + ANTLR_GROUP_ID + ':' + ANTLR_RUNTIME_ARTIFACT_NAME + ':' + projectAntlrVersion.toString() +
" which did not match Antlr version used by plugin [" + pluginAntlrVersion.toString() + "]"
);
}
}
/**
* Builds the classloader to pass to gUnit.
*
* @param antlrArtifact The plugin's (our) Antlr dependency artifact.
*
* @return The classloader for gUnit to use
*
* @throws MojoExecutionException Problem resolving artifacts to {@link java.net.URL urls}.
*/
private ClassLoader determineProjectCompileScopeClassLoader(Artifact antlrArtifact)
throws MojoExecutionException {
ArrayList<URL> classPathUrls = new ArrayList<URL>();
getLog().info( "Adding Antlr artifact : " + antlrArtifact.getId() );
classPathUrls.add( resolveLocalURL( antlrArtifact ) );
for ( String path : classpathElements() ) {
try {
getLog().info( "Adding project compile classpath element : " + path );
classPathUrls.add( new File( path ).toURI().toURL() );
}
catch ( MalformedURLException e ) {
throw new MojoExecutionException( "Unable to build path URL [" + path + "]" );
}
}
return new URLClassLoader( classPathUrls.toArray( new URL[classPathUrls.size()] ), getClass().getClassLoader() );
}
protected static URL resolveLocalURL(Artifact artifact) throws MojoExecutionException {
try {
return artifact.getFile().toURI().toURL();
}
catch ( MalformedURLException e ) {
throw new MojoExecutionException( "Unable to resolve artifact url : " + artifact.getId(), e );
}
}
@SuppressWarnings( "unchecked" )
private List<String> classpathElements() throws MojoExecutionException {
try {
// todo : should we combine both compile and test scoped elements?
return ( List<String> ) project.getTestClasspathElements();
}
catch ( DependencyResolutionRequiredException e ) {
throw new MojoExecutionException( "Call to Project#getCompileClasspathElements required dependency resolution" );
}
}
private void performExecution(ClassLoader projectCompileScopeClassLoader) throws MojoExecutionException {
getLog().info( "gUnit report directory : " + reportDirectory.getAbsolutePath() );
if ( !reportDirectory.exists() ) {
boolean directoryCreated = reportDirectory.mkdirs();
if ( !directoryCreated ) {
getLog().warn( "mkdirs() reported problem creating report directory" );
}
}
Result runningResults = new Result();
ArrayList<String> failureNames = new ArrayList<String>();
System.out.println();
System.out.println( "-----------------------------------------------------------" );
System.out.println( " G U N I T R E S U L T S" );
System.out.println( "-----------------------------------------------------------" );
for ( File script : collectIncludedSourceGrammars() ) {
final String scriptPath = script.getAbsolutePath();
System.out.println( "Executing script " + scriptPath );
try {
String scriptBaseName = StringUtils.chompLast( FileUtils.basename( script.getName() ), "." );
ANTLRFileStream antlrStream = new ANTLRFileStream( scriptPath );
GrammarInfo grammarInfo = Interp.parse( antlrStream );
gUnitExecutor executor = new gUnitExecutor(
grammarInfo,
projectCompileScopeClassLoader,
script.getParentFile().getAbsolutePath()
);
String report = executor.execTest();
writeReportFile( new File( reportDirectory, scriptBaseName + ".txt" ), report );
Result testResult = new Result();
testResult.tests = executor.numOfTest;
testResult.failures = executor.numOfFailure;
testResult.invalids = executor.numOfInvalidInput;
System.out.println( testResult.render() );
runningResults.add( testResult );
for ( AbstractTest test : executor.failures ) {
failureNames.add( scriptBaseName + "#" + test.getHeader() );
}
}
catch ( IOException e ) {
throw new MojoExecutionException( "Could not open specified script file", e );
}
catch ( RecognitionException e ) {
throw new MojoExecutionException( "Could not parse gUnit script", e );
}
}
System.out.println();
System.out.println( "Summary :" );
if ( ! failureNames.isEmpty() ) {
System.out.println( " Found " + failureNames.size() + " failures" );
for ( String name : failureNames ) {
System.out.println( " - " + name );
}
}
System.out.println( runningResults.render() );
System.out.println();
if ( runningResults.failures > 0 ) {
throw new MojoExecutionException( "Found gUnit test failures" );
}
if ( runningResults.invalids > 0 ) {
throw new MojoExecutionException( "Found invalid gUnit tests" );
}
}
private Set<File> collectIncludedSourceGrammars() throws MojoExecutionException {
SourceMapping mapping = new SuffixMapping( "g", Collections.EMPTY_SET );
SourceInclusionScanner scan = new SimpleSourceInclusionScanner( getIncludePatterns(), getExcludePatterns() );
scan.addSourceMapping( mapping );
try {
Set scanResults = scan.getIncludedSources( sourceDirectory, null );
Set<File> results = new HashSet<File>();
for ( Object result : scanResults ) {
if ( result instanceof File ) {
results.add( ( File ) result );
}
else if ( result instanceof String ) {
results.add( new File( ( String ) result ) );
}
else {
throw new MojoExecutionException( "Unexpected result type from scanning [" + result.getClass().getName() + "]" );
}
}
return results;
}
catch ( InclusionScanException e ) {
throw new MojoExecutionException( "Error determining gUnit sources", e );
}
}
private void writeReportFile(File reportFile, String results) {
try {
Writer writer = new FileWriter( reportFile );
writer = new BufferedWriter( writer );
try {
writer.write( results );
writer.flush();
}
finally {
try {
writer.close();
}
catch ( IOException ignore ) {
}
}
}
catch ( IOException e ) {
getLog().warn( "Error writing gUnit report file", e );
}
}
private static class Result {
private int tests = 0;
private int failures = 0;
private int invalids = 0;
public String render() {
return String.format( "Tests run: %d, Failures: %d, Invalid: %d", tests, failures, invalids );
}
public void add(Result result) {
this.tests += result.tests;
this.failures += result.failures;
this.invalids += result.invalids;
}
}
}
antlr-3.2/gunit-maven-plugin/pom.xml 0000644 0001750 0001750 00000020060 11256467024 017504 0 ustar twerner twerner <!--
[The "BSD licence"]
ANTLR - Copyright (c) 2005-2008 Terence Parr
Maven Plugin - Copyright (c) 2009 Jim Idle
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<prerequisites>
<maven>2.0</maven>
</prerequisites>
<groupId>org.antlr</groupId>
<artifactId>maven-gunit-plugin</artifactId>
<packaging>maven-plugin</packaging>
<version>3.2</version>
<name>Maven plugin for gUnit ANTLR V3 </name>
<description>A Maven plugin for incorporating gUnit testing of grammars</description>
<url>http://antlr.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<!-- Where do we track bugs for this project?
-->
<issueManagement>
<system>JIRA</system>
<url>http://antlr.org/jira/browse/ANTLR</url>
</issueManagement>
<!-- Location of the license description for this project
-->
<licenses>
<license>
<distribution>repo</distribution>
<name>The BSD License</name>
<url>http://www.antlr.org/LICENSE.txt </url>
</license>
</licenses>
<distributionManagement>
<!-- The main release repo for Antlr artifacts -->
<repository>
<id>antlr-repo</id>
<name>ANTLR Testing repository</name>
<url>scpexe://antlr.org/home/mavensync/antlr-repo</url>
</repository>
<!-- The snapshot repo for Antlr artifacts -->
<snapshotRepository>
<id>antlr-snapshot</id>
<name>ANTLR Testing Snapshot Repository</name>
<url>scpexe://antlr.org/home/mavensync/antlr-snapshot</url>
</snapshotRepository>
<!-- The place where site deployment should go -->
<site>
<id>antlr-repo</id>
<name>ANTLR gUnit Maven Plugin Web Site</name>
<url>scpexe://antlr.org/home/mavensync/antlr-maven-webs/maven-gunit-plugin</url>
</site>
</distributionManagement>
<!--
Inform Maven of the ANTLR snapshot repository, which it will
need to consult to get the latest snapshot build of the runtime and tool
if it was not built and installed locally.
-->
<repositories>
<!--
This is the ANTLR repository.
-->
<repository>
<id>antlr-snapshot</id>
<name>ANTLR Testing Snapshot Repository</name>
<url>http://antlr.org/antlr-snapshot</url>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
<!-- Ancilliary information for completeness
-->
<inceptionYear>2009</inceptionYear>
<mailingLists>
<mailingList>
<archive>http://antlr.markmail.org/</archive>
<otherArchives>
<otherArchive>http://www.antlr.org/pipermail/antlr-interest/</otherArchive>
</otherArchives>
<name>ANTLR Users</name>
<subscribe>http://www.antlr.org/mailman/listinfo/antlr-interest/</subscribe>
<unsubscribe>http://www.antlr.org/mailman/options/antlr-interest/</unsubscribe>
<post>antlr-interest@antlr.org</post>
</mailingList>
</mailingLists>
<organization>
<name>ANTLR.org</name>
<url>http://www.antlr.org</url>
</organization>
<!-- ============================================================================= -->
<!--
What are we depedent on for the Mojos to execute? We need the
plugin API itself and of course we need the ANTLR Tool and runtime
and any of their dependencies, which we inherit. The Tool itself provides
us with all the dependencies, so we need only name it here.
-->
<dependencies>
<!--
The things we need to build the target language recognizer
-->
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-api</artifactId>
<version>2.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-project</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-compiler-api</artifactId>
<version>1.5.3</version>
</dependency>
<!--
The version of ANTLR tool that this version of the plugin controls.
We have decided that this should be in lockstep with ANTLR itself, other
than -1 -2 -3 etc patch releases.
-->
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr</artifactId>
<version>3.2</version>
</dependency>
<!--
Dependency on the gUnit artifact.
-->
<dependency>
<groupId>${groupId}</groupId>
<artifactId>gunit</artifactId>
<version>3.2</version>
</dependency>
<!--
Testing requirements...
-->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.maven.shared</groupId>
<artifactId>maven-plugin-testing-harness</artifactId>
<version>1.0</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<defaultGoal>install</defaultGoal>
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ssh-external</artifactId>
<version>1.0-beta-2</version>
</extension>
</extensions>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-project-info-reports-plugin</artifactId>
<configuration>
<dependencyLocationsEnabled>false</dependencyLocationsEnabled>
</configuration>
</plugin>
</plugins>
</build>
</project>
antlr-3.2/pom.xml 0000644 0001750 0001750 00000020554 11256466372 013773 0 ustar twerner twerner <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.antlr</groupId>
<artifactId>antlr-master</artifactId>
<packaging>pom</packaging>
<version>3.2</version>
<name>ANTLR Master build control POM</name>
<url>http://maven.apache.org</url>
<!--
What version of ANTLR are we building? This sets the
the version number for all other things that are built
as part of an ANTLR release, unless they override or
ignore it. We do this via a properites file for this
pom.
-->
<!--
This is the master pom for building the ANTLR
toolset and runtime (Java) at the specific level
defined above. Hence we specify here the modules that
this pom will build when we build this pom
-->
<modules>
<module>runtime/Java</module>
<module>tool</module>
<module>antlr3-maven-plugin</module>
<module>gunit</module>
<module>gunit-maven-plugin</module>
</modules>
<!--
Define where the ANTLR releated jars are deployed both for
the main ANTLR repository, which syncs with the maven main
repository, and the snapshot repository, which can be
used by developers that need the latest development version of
something, but is used here to show maven where to deploy
snapshots and releases.
-->
<distributionManagement>
<repository>
<id>antlr-repo</id>
<name>ANTLR Testing repository</name>
<url>scpexe://antlr.org/home/mavensync/antlr-repo</url>
</repository>
<snapshotRepository>
<id>antlr-snapshot</id>
<name>ANTLR Testing Snapshot Repository</name>
<url>scpexe://antlr.org/home/mavensync/antlr-snapshot</url>
</snapshotRepository>
</distributionManagement>
<!--
Inform Maven of the ANTLR snapshot repository, which it will
need to consult to get the latest snapshot build of the runtime
if it was not built and installed locally.
-->
<repositories>
<!--
This is the ANTLR repository.
-->
<repository>
<id>antlr-snapshot</id>
<name>ANTLR Testing Snapshot Repository</name>
<url>http://antlr.org/antlr-snapshot</url>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
<!--
Tell Maven which other artifacts we need in order to
build, run and test the ANTLR jars.
This is the master pom, and so it only contains those
dependencies that are common to all the modules below
or are just included for test
-->
<dependencyManagement>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>antlr</groupId>
<artifactId>antlr</artifactId>
<version>2.7.7</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>stringtemplate</artifactId>
<version>3.2.1</version>
<scope>compile</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<defaultGoal>install</defaultGoal>
<!--
The following filter definition means that both the master
project and the sub projects will read in a file in the same
directory as the pom.xml is located and set any properties
that are defined there in the standard x=y format. These
properties can then be referenced via ${x} in any resource
file specified in any pom. So, there is a master antlr.config
file in the same location as this pom.xml file and here you can
define anything that is relevant to all the modules that we
build here. However each module also has an antlr.config file
where you can override property values from the master file or
define things that are only relevant to that module.
-->
<filters>
<filter>antlr.config</filter>
</filters>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>buildnumber-maven-plugin</artifactId>
<configuration>
<format>{0,date,MMM dd, yyyy} {0,time,kk:mm:ss}</format>
<items>
<item>timestamp</item>
</items>
</configuration>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>create</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>jsr14</target>
<sourceDirectory>src</sourceDirectory>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<configuration>
<findbugsXmlOutput>true</findbugsXmlOutput>
<findbugsXmlWithMessages>true</findbugsXmlWithMessages>
<xmlOutput>true</xmlOutput>
</configuration>
</plugin>
<plugin>
<!--
Build an uber-jar for the ANTLR Tool that is packaged with all the other dependencies,
such as the antlr-runtime and stringtemplate etc. This will be useful
for developers, who then do not need to download anything else or
remember that they need stringtemplate.jar in their CLASSPATH and so
on.
This does not preclude any of the module generated jars from
being used on their own of course.
Here, we also build a master source jar as I was unable to pursuade
this plugin to use multiple configurations and not have the thing
screw up because of multiple modules :-(
-->
<artifactId>maven-assembly-plugin</artifactId>
<!--
Do not make the child modules build an assembly
-->
<inherited>false</inherited>
<configuration>
<descriptors>
<descriptor>antlrjar.xml</descriptor>
<descriptor>antlrsources.xml</descriptor>
</descriptors>
<!--
Specify that we want the resulting jar to be executable
via java -jar, which we do by modifying the manifest
of course.
-->
<archive>
<manifest>
<mainClass>org.antlr.Tool</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ssh-external</artifactId>
<version>1.0-beta-2</version>
</extension>
</extensions>
</build>
</project>