flex-2.5.4/ 40755 75424 12 0 6366534462 10751 5ustar kwzhstaffflex-2.5.4/README100444 75424 12 3772 5736052642 11730 0ustar kwzhstaffThis is release 2.5 of flex. See "version.h" for the exact patch-level. See the file "NEWS" to find out what is new in this Flex release. Read the file "INSTALL" for general installation directives. Peek near the beginning of the file "Makefile.in" for special DEFS values. On most systems, you can just run the "configure" script and type "make" to build flex; then "make check" to test whether it built correctly; and if it did, then "make install" to install it. If you're feeling adventurous, you can also issue "make bigcheck" (be prepared to wait a while). Note that flex is distributed under a copyright very similar to that of BSD Unix, and not under the GNU General Public License (GPL), except for the "configure" script, which is covered by the GPL. Many thanks to the 2.5 beta-testers for finding bugs and helping test and increase portability: Stan Adermann, Scott David Daniels, Charles Elliott, Joe Gayda, Chris Meier, James Nordby, Terrence O'Kane, Karsten Pahnke, Francois Pinard, Pat Rankin, Andreas Scherer, Marc Wiese, Nathan Zelle. Please send bug reports and feedback to: Vern Paxson (vern@ee.lbl.gov). The flex distribution consists of the following files: README This message NEWS Differences between the various releases INSTALL General installation information COPYING flex's copyright conf.in, configure.in, configure, Makefile.in, install.sh, mkinstalldirs elements of the "autoconf" auto-configuration process flexdef.h, parse.y, scan.l, ccl.c, dfa.c, ecs.c, gen.c, main.c, misc.c, nfa.c, sym.c, tblcmp.c, yylex.c source files version.h version of this flex release flex.skl flex scanner skeleton mkskel.sh script for converting flex.skl to C source file skel.c skel.c pre-converted C version of flex.skl libmain.c flex library (-lfl) sources libyywrap.c initscan.c pre-flex'd version of scan.l FlexLexer.h header file for C++ lexer class flex.1 user documentation MISC/ a directory containing miscellaneous contributions. See MISC/README for details. flex-2.5.4/NEWS100444 75424 12 132412 6215401553 11571 0ustar kwzhstaffChanges between release 2.5.4 (11Sep96) and release 2.5.3: - Fixed a bug introduced in 2.5.3 that blew it when a call to input() occurred at the end of an input file. - Fixed scanner skeleton so the example in the man page of scanning strings using exclusive start conditions works. - Minor Makefile tweaks. Changes between release 2.5.3 (29May96) and release 2.5.2: - Some serious bugs in yymore() have been fixed. In particular, when using AT&T-lex-compatibility or %array, you can intermix calls to input(), unput(), and yymore(). (This still doesn't work for %pointer, and isn't likely to in the future.) - A bug in handling NUL's in the input stream of scanners using REJECT has been fixed. - The default main() in libfl.a now repeatedly calls yylex() until it returns 0, rather than just calling it once. - Minor tweak for Windows NT Makefile, MISC/NT/Makefile. Changes between release 2.5.2 (25Apr95) and release 2.5.1: - The --prefix configuration option now works. - A bug that completely broke the "-Cf" table compression option has been fixed. - A major headache involving "const" declarators and Solaris systems has been fixed. - An octal escape sequence in a flex regular expression must now contain only the digits 0-7. - You can now use "--" on the flex command line to mark the end of flex options. - You can now specify the filename '-' as a synonym for stdin. - By default, the scanners generated by flex no longer statically initialize yyin and yyout to stdin and stdout. This change is necessary because in some ANSI environments, stdin and stdout are not compile-time constant. You can force the initialization using "%option stdinit" in the first section of your flex input. - "%option nounput" now correctly omits the unput() routine from the output. - "make clean" now removes config.log, config.cache, and the flex binary. The fact that it removes the flex binary means you should take care if making changes to scan.l, to make sure you don't wind up in a bootstrap problem. - In general, the Makefile has been reworked somewhat (thanks to Francois Pinard) for added flexibility - more changes will follow in subsequent releases. - The .texi and .info files in MISC/texinfo/ have been updated, thanks also to Francois Pinard. - The FlexLexer::yylex(istream* new_in, ostream* new_out) method now does not have a default for the first argument, to disambiguate it from FlexLexer::yylex(). - A bug in destructing a FlexLexer object before doing any scanning with it has been fixed. - A problem with including FlexLexer.h multiple times has been fixed. - The alloca() chud necessary to accommodate bison has grown even uglier, but hopefully more correct. - A portability tweak has been added to accommodate compilers that use char* generic pointers. - EBCDIC contact information in the file MISC/EBCDIC has been updated. - An OS/2 Makefile and config.h for flex 2.5 is now available in MISC/OS2/, contributed by Kai Uwe Rommel. - The descrip.mms file for building flex under VMS has been updated, thanks to Pat Rankin. - The notes on building flex for the Amiga have been updated for flex 2.5, contributed by Andreas Scherer. Changes between release 2.5.1 (28Mar95) and release 2.4.7: - A new concept of "start condition" scope has been introduced. A start condition scope is begun with: { where SCs is a list of one or more start conditions. Inside the start condition scope, every rule automatically has the prefix applied to it, until a '}' which matches the initial '{'. So, for example: { "\\n" return '\n'; "\\r" return '\r'; "\\f" return '\f'; "\\0" return '\0'; } is equivalent to: "\\n" return '\n'; "\\r" return '\r'; "\\f" return '\f'; "\\0" return '\0'; As indicated in this example, rules inside start condition scopes (and any rule, actually, other than the first) can be indented, to better show the extent of the scope. Start condition scopes may be nested. - The new %option directive can be used in the first section of a flex scanner to control scanner-generation options. Most options are given simply as names, optionally preceded by the word "no" (with no intervening whitespace) to negate their meaning. Some are equivalent to flex flags, so putting them in your scanner source is equivalent to always specifying the flag (%option's take precedence over flags): 7bit -7 option 8bit -8 option align -Ca option backup -b option batch -B option c++ -+ option caseful opposite of -i option (caseful is the default); case-sensitive same as above caseless -i option; case-insensitive same as above debug -d option default opposite of -s option ecs -Ce option fast -F option full -f option interactive -I option lex-compat -l option meta-ecs -Cm option perf-report -p option read -Cr option stdout -t option verbose -v option warn opposite of -w option (so use "%option nowarn" for -w) array equivalent to "%array" pointer equivalent to "%pointer" (default) Some provide new features: always-interactive generate a scanner which always considers its input "interactive" (no call to isatty() will be made when the scanner runs) main supply a main program for the scanner, which simply calls yylex(). Implies %option noyywrap. never-interactive generate a scanner which never considers its input "interactive" (no call to isatty() will be made when the scanner runs) stack if set, enable start condition stacks (see below) stdinit if unset ("%option nostdinit"), initialize yyin and yyout statically to nil FILE* pointers, instead of stdin and stdout yylineno if set, keep track of the current line number in global yylineno (this option is expensive in terms of performance). The line number is available to C++ scanning objects via the new member function lineno(). yywrap if unset ("%option noyywrap"), scanner does not call yywrap() upon EOF but simply assumes there are no more files to scan Flex scans your rule actions to determine whether you use the REJECT or yymore features (this is not new). Two %options can be used to override its decision, either by setting them to indicate the feature is indeed used, or unsetting them to indicate it actually is not used: reject yymore Three %option's take string-delimited values, offset with '=': outfile="" equivalent to -o prefix="" equivalent to -P yyclass="" set the name of the C++ scanning class (see below) A number of %option's are available for lint purists who want to suppress the appearance of unneeded routines in the generated scanner. Each of the following, if unset, results in the corresponding routine not appearing in the generated scanner: input, unput yy_push_state, yy_pop_state, yy_top_state yy_scan_buffer, yy_scan_bytes, yy_scan_string You can specify multiple options with a single %option directive, and multiple directives in the first section of your flex input file. - The new function: YY_BUFFER_STATE yy_scan_string( const char *str ) returns a YY_BUFFER_STATE (which also becomes the current input buffer) for scanning the given string, which occurs starting with the next call to yylex(). The string must be NUL-terminated. A related function: YY_BUFFER_STATE yy_scan_bytes( const char *bytes, int len ) creates a buffer for scanning "len" bytes (including possibly NUL's) starting at location "bytes". Note that both of these functions create and scan a *copy* of the string/bytes. (This may be desirable, since yylex() modifies the contents of the buffer it is scanning.) You can avoid the copy by using: YY_BUFFER_STATE yy_scan_buffer( char *base, yy_size_t size ) which scans in place the buffer starting at "base", consisting of "size" bytes, the last two bytes of which *must* be YY_END_OF_BUFFER_CHAR (these bytes are not scanned; thus, scanning consists of base[0] through base[size-2], inclusive). If you fail to set up "base" in this manner, yy_scan_buffer returns a nil pointer instead of creating a new input buffer. The type yy_size_t is an integral type to which you can cast an integer expression reflecting the size of the buffer. - Three new routines are available for manipulating stacks of start conditions: void yy_push_state( int new_state ) pushes the current start condition onto the top of the stack and BEGIN's "new_state" (recall that start condition names are also integers). void yy_pop_state() pops the top of the stack and BEGIN's to it, and int yy_top_state() returns the top of the stack without altering the stack's contents. The start condition stack grows dynamically and so has no built-in size limitation. If memory is exhausted, program execution is aborted. To use start condition stacks, your scanner must include a "%option stack" directive. - flex now supports POSIX character class expressions. These are expressions enclosed inside "[:" and ":]" delimiters (which themselves must appear between the '[' and ']' of a character class; other elements may occur inside the character class, too). The expressions flex recognizes are: [:alnum:] [:alpha:] [:blank:] [:cntrl:] [:digit:] [:graph:] [:lower:] [:print:] [:punct:] [:space:] [:upper:] [:xdigit:] These expressions all designate a set of characters equivalent to the corresponding isXXX function (for example, [:alnum:] designates those characters for which isalnum() returns true - i.e., any alphabetic or numeric). Some systems don't provide isblank(), so flex defines [:blank:] as a blank or a tab. For example, the following character classes are all equivalent: [[:alnum:]] [[:alpha:][:digit:] [[:alpha:]0-9] [a-zA-Z0-9] If your scanner is case-insensitive (-i flag), then [:upper:] and [:lower:] are equivalent to [:alpha:]. - The promised rewrite of the C++ FlexLexer class has not yet been done. Support for FlexLexer is limited at the moment to fixing show-stopper bugs, so, for example, the new functions yy_scan_string() & friends are not available to FlexLexer objects. - The new macro yy_set_interactive(is_interactive) can be used to control whether the current buffer is considered "interactive". An interactive buffer is processed more slowly, but must be used when the scanner's input source is indeed interactive to avoid problems due to waiting to fill buffers (see the discussion of the -I flag in flex.1). A non-zero value in the macro invocation marks the buffer as interactive, a zero value as non-interactive. Note that use of this macro overrides "%option always-interactive" or "%option never-interactive". yy_set_interactive() must be invoked prior to beginning to scan the buffer. - The new macro yy_set_bol(at_bol) can be used to control whether the current buffer's scanning context for the next token match is done as though at the beginning of a line (non-zero macro argument; makes '^' anchored rules active) or not at the beginning of a line (zero argument, '^' rules inactive). - Related to this change, the mechanism for determining when a scan is starting at the beginning of a line has changed. It used to be that '^' was active iff the character prior to that at which the scan started was a newline. The mechanism now is that '^' is active iff the last token ended in a newline (or the last call to input() returned a newline). For most users, the difference in mechanisms is negligible. Where it will make a difference, however, is if unput() or yyless() is used to alter the input stream. When in doubt, use yy_set_bol(). - The new beginning-of-line mechanism involved changing some fairly twisted code, so it may have introduced bugs - beware ... - The macro YY_AT_BOL() returns true if the next token scanned from the current buffer will have '^' rules active, false otherwise. - The new function void yy_flush_buffer( struct yy_buffer_state* b ) flushes the contents of the current buffer (i.e., next time the scanner attempts to match a token using b as the current buffer, it will begin by invoking YY_INPUT to fill the buffer). This routine is also available to C++ scanners (unlike some of the other new routines). The related macro YY_FLUSH_BUFFER flushes the contents of the current buffer. - A new "-ooutput" option writes the generated scanner to "output". If used with -t, the scanner is still written to stdout, but its internal #line directives (see previous item) use "output". - Flex now generates #line directives relating the code it produces to the output file; this means that error messages in the flex-generated code should be correctly pinpointed. - When generating #line directives, filenames with embedded '\'s have those characters escaped (i.e., turned into '\\'). This feature helps with reporting filenames for some MS-DOS and OS/2 systems. - The FlexLexer class includes two new public member functions: virtual void switch_streams( istream* new_in = 0, ostream* new_out = 0 ) reassigns yyin to new_in (if non-nil) and yyout to new_out (ditto), deleting the previous input buffer if yyin is reassigned. It is used by: int yylex( istream* new_in = 0, ostream* new_out = 0 ) which first calls switch_streams() and then returns the value of calling yylex(). - C++ scanners now have yy_flex_debug as a member variable of FlexLexer rather than a global, and member functions for testing and setting it. - When generating a C++ scanning class, you can now use %option yyclass="foo" to inform flex that you have derived "foo" as a subclass of yyFlexLexer, so flex will place your actions in the member function foo::yylex() instead of yyFlexLexer::yylex(). It also generates a yyFlexLexer::yylex() member function that generates a run-time error if called (by invoking yyFlexLexer::LexerError()). This feature is necessary if your subclass "foo" introduces some additional member functions or variables that you need to access from yylex(). - Current texinfo files in MISC/texinfo, contributed by Francois Pinard. - You can now change the name "flex" to something else (e.g., "lex") by redefining $(FLEX) in the Makefile. - Two bugs (one serious) that could cause "bigcheck" to fail have been fixed. - A number of portability/configuration changes have been made for easier portability. - You can use "YYSTATE" in your scanner as an alias for YY_START (for AT&T lex compatibility). - input() now maintains yylineno. - input() no longer trashes yytext. - interactive scanners now read characters in YY_INPUT up to a newline, a large performance gain. - C++ scanner objects now work with the -P option. You include once per scanner - see comments in (or flex.1) for details. - C++ FlexLexer objects now use the "cerr" stream to report -d output instead of stdio. - The -c flag now has its full glorious POSIX interpretation (do nothing), rather than being interpreted as an old-style -C flag. - Scanners generated by flex now include two #define's giving the major and minor version numbers (YY_FLEX_MAJOR_VERSION, YY_FLEX_MINOR_VERSION). These can then be tested to see whether certain flex features are available. - Scanners generated using -l lex compatibility now have the symbol YY_FLEX_LEX_COMPAT #define'd. - When initializing (i.e., yy_init is non-zero on entry to yylex()), generated scanners now set yy_init to zero before executing YY_USER_INIT. This means that you can set yy_init back to a non-zero value in YY_USER_INIT if you need the scanner to be reinitialized on the next call. - You can now use "#line" directives in the first section of your scanner specification. - When generating full-table scanners (-Cf), flex now puts braces around each row of the 2-d array initialization, to silence warnings on over-zealous compilers. - Improved support for MS-DOS. The flex sources have been successfully built, unmodified, for Borland 4.02 (all that's required is a Borland Makefile and config.h file, which are supplied in MISC/Borland - contributed by Terrence O Kane). - Improved support for Macintosh using Think C - the sources should build for this platform "out of the box". Contributed by Scott Hofmann. - Improved support for VMS, in MISC/VMS/, contributed by Pat Rankin. - Support for the Amiga, in MISC/Amiga/, contributed by Andreas Scherer. Note that the contributed files were developed for flex 2.4 and have not been tested with flex 2.5. - Some notes on support for the NeXT, in MISC/NeXT, contributed by Raf Schietekat. - The MISC/ directory now includes a preformatted version of flex.1 in flex.man, and pre-yacc'd versions of parse.y in parse.{c,h}. - The flex.1 and flexdoc.1 manual pages have been merged. There is now just one document, flex.1, which includes an overview at the beginning to help you find the section you need. - Documentation now clarifies that start conditions persist across switches to new input files or different input buffers. If you want to e.g., return to INITIAL, you must explicitly do so. - The "Performance Considerations" section of the manual has been updated. - Documented the "yy_act" variable, which when YY_USER_ACTION is invoked holds the number of the matched rule, and added an example of using yy_act to profile how often each rule is matched. - Added YY_NUM_RULES, a definition that gives the total number of rules in the file, including the default rule (even if you use -s). - Documentation now clarifies that you can pass a nil FILE* pointer to yy_create_buffer() or yyrestart() if you've arrange YY_INPUT to not need yyin. - Documentation now clarifies that YY_BUFFER_STATE is a pointer to an opaque "struct yy_buffer_state". - Documentation now stresses that you gain the benefits of removing backing-up states only if you remove *all* of them. - Documentation now points out that traditional lex allows you to put the action on a separate line from the rule pattern if the pattern has trailing whitespace (ugh!), but flex doesn't support this. - A broken example in documentation of the difference between inclusive and exclusive start conditions is now fixed. - Usage (-h) report now goes to stdout. - Version (-V) info now goes to stdout. - More #ifdef chud has been added to the parser in attempt to deal with bison's use of alloca(). - "make clean" no longer deletes emacs backup files (*~). - Some memory leaks have been fixed. - A bug was fixed in which dynamically-expanded buffers were reallocated a couple of bytes too small. - A bug was fixed which could cause flex to read and write beyond the end of the input buffer. - -S will not be going away. Changes between release 2.4.7 (03Aug94) and release 2.4.6: - Fixed serious bug in reading multiple files. - Fixed bug in scanning NUL's. - Fixed bug in input() returning 8-bit characters. - Fixed bug in matching text with embedded NUL's when using %array or lex compatibility. - Fixed multiple invocations of YY_USER_ACTION when using '|' continuation action. - Minor prototyping fixes. Changes between release 2.4.6 (04Jan94) and release 2.4.5: - Linking with -lfl no longer required if your program includes its own yywrap() and main() functions. (This change will cause problems if you have a non-ANSI compiler on a system for which sizeof(int) != sizeof(void*) or sizeof(int) != sizeof(size_t).) - The use of 'extern "C++"' in FlexLexer.h has been modified to get around an incompatibility with g++'s header files. Changes between release 2.4.5 (11Dec93) and release 2.4.4: - Fixed bug breaking C++ scanners that use REJECT or variable trailing context. - Fixed serious input problem for interactive scanners on systems for which char is unsigned. - Fixed bug in incorrectly treating '$' operator as variable trailing context. - Fixed bug in -CF table representation that could lead to corrupt tables. - Fixed fairly benign memory leak. - Added `extern "C++"' wrapper to FlexLexer.h header. This should overcome the g++ 2.5.X problems mentioned in the NEWS for release 2.4.3. - Changed #include of FlexLexer.h to use <> instead of "". - Added feature to control whether the scanner attempts to refill the input buffer once it's exhausted. This feature will be documented in the 2.5 release. Changes between release 2.4.4 (07Dec93) and release 2.4.3: - Fixed two serious bugs in scanning 8-bit characters. - Fixed bug in YY_USER_ACTION that caused it to be executed inappropriately (on the scanner's own internal actions, and with incorrect yytext/yyleng values). - Fixed bug in pointing yyin at a new file and resuming scanning. - Portability fix regarding min/max/abs macros conflicting with function definitions in standard header files. - Added a virtual LexerError() method to the C++ yyFlexLexer class for reporting error messages instead of always using cerr. - Added warning in flexdoc that the C++ scanning class is presently experimental and subject to considerable change between major releases. Changes between release 2.4.3 (03Dec93) and release 2.4.2: - Fixed bug causing fatal scanner messages to fail to print. - Fixed things so FlexLexer.h can be included in other C++ sources. One side-effect of this change is that -+ and -CF are now incompatible. - libfl.a now supplies private versions of the the / string routines needed by flex and the scanners it generates, to enhance portability to some BSD systems. - More robust solution to 2.4.2's flexfatal() bug fix. - Added ranlib of installed libfl.a. - Some lint tweaks. - NOTE: problems have been encountered attempting to build flex C++ scanners using g++ version 2.5.X. The problem is due to an unfortunate heuristic in g++ 2.5.X that attempts to discern between C and C++ headers. Because FlexLexer.h is installed (by default) in /usr/local/include and not /usr/local/lib/g++-include, g++ 2.5.X decides that it's a C header :-(. So if you have problems, install the header in /usr/local/lib/g++-include instead. Changes between release 2.4.2 (01Dec93) and release 2.4.1: - Fixed bug in libfl.a referring to non-existent "flexfatal" function. - Modified to produce both compress'd and gzip'd tar files for distributions (you probably don't care about this change!). Changes between release 2.4.1 (30Nov93) and release 2.3.8: - The new '-+' flag instructs flex to generate a C++ scanner class (thanks to Kent Williams). flex writes an implementation of the class defined in FlexLexer.h to lex.yy.cc. You may include multiple scanner classes in your program using the -P flag. Note that the scanner class also provides a mechanism for creating reentrant scanners. The scanner class uses C++ streams for I/O instead of FILE*'s (thanks to Tom Epperly). If the flex executable's name ends in '+' then the '-+' flag is automatically on, so creating a symlink or copy of "flex" to "flex++" results in a version of flex that can be used exclusively for C++ scanners. Note that without the '-+' flag, flex-generated scanners can still be compiled using C++ compilers, though they use FILE*'s for I/O instead of streams. See the "GENERATING C++ SCANNERS" section of flexdoc for details. - The new '-l' flag turns on maximum AT&T lex compatibility. In particular, -l includes support for "yylineno" and makes yytext be an array instead of a pointer. It does not, however, do away with all incompatibilities. See the "INCOMPATIBILITIES WITH LEX AND POSIX" section of flexdoc for details. - The new '-P' option specifies a prefix to use other than "yy" for the scanner's globally-visible variables, and for the "lex.yy.c" filename. Using -P you can link together multiple flex scanners in the same executable. - The distribution includes a "texinfo" version of flexdoc.1, contributed by Roland Pesch (thanks also to Marq Kole, who contributed another version). It has not been brought up to date, but reflects version 2.3. See MISC/flex.texinfo. The flex distribution will soon include G.T. Nicol's flex manual; he is presently bringing it up-to-date for version 2.4. - yywrap() is now a function, and you now *must* link flex scanners with libfl.a. - Site-configuration is now done via an autoconf-generated "configure" script contributed by Francois Pinard. - Scanners now use fread() (or getc(), if interactive) and not read() for input. A new "table compression" option, -Cr, overrides this change and causes the scanner to use read() (because read() is a bit faster than fread()). -f and -F are now equivalent to -Cfr and -CFr; i.e., they imply the -Cr option. - In the blessed name of POSIX compliance, flex supports "%array" and "%pointer" directives in the definitions (first) section of the scanner specification. The former specifies that yytext should be an array (of size YYLMAX), the latter, that it should be a pointer. The array version of yytext is universally slower than the pointer version, but has the advantage that its contents remain unmodified across calls to input() and unput() (the pointer version of yytext is, still, trashed by such calls). "%array" cannot be used with the '-+' C++ scanner class option. - The new '-Ca' option directs flex to trade off memory for natural alignment when generating a scanner's tables. In particular, table entries that would otherwise be "short" become "long". - The new '-h' option produces a summary of the flex flags. - The new '-V' option reports the flex version number and exits. - The new scanner macro YY_START returns an integer value corresponding to the current start condition. You can return to that start condition by passing the value to a subsequent "BEGIN" action. You also can implement "start condition stacks" by storing the values in an integer stack. - You can now redefine macros such as YY_INPUT by just #define'ing them to some other value in the first section of the flex input; no need to first #undef them. - flex now generates warnings for rules that can't be matched. These warnings can be turned off using the new '-w' flag. If your scanner uses REJECT then you will not get these warnings. - If you specify the '-s' flag but the default rule can be matched, flex now generates a warning. - "yyleng" is now a global, and may be modified by the user (though doing so and then using yymore() will yield weird results). - Name definitions in the first section of a scanner specification can now include a leading '^' or trailing '$' operator. In this case, the definition is *not* pushed back inside of parentheses. - Scanners with compressed tables are now "interactive" (-I option) by default. You can suppress this attribute (which makes them run slightly slower) using the new '-B' flag. - Flex now generates 8-bit scanners by default, unless you use the -Cf or -CF compression options (-Cfe and -CFe result in 8-bit scanners). You can force it to generate a 7-bit scanner using the new '-7' flag. You can build flex to generate 8-bit scanners for -Cf and -CF, too, by adding -DDEFAULT_CSIZE=256 to CFLAGS in the Makefile. - You no longer need to call the scanner routine yyrestart() to inform the scanner that you have switched to a new file after having seen an EOF on the current input file. Instead, just point yyin at the new file and continue scanning. - You no longer need to invoke YY_NEW_FILE in an <> action to indicate you wish to continue scanning. Simply point yyin at a new file. - A leading '#' no longer introduces a comment in a flex input. - flex no longer considers formfeed ('\f') a whitespace character. - %t, I'm happy to report, has been nuked. - The '-p' option may be given twice ('-pp') to instruct flex to report minor performance problems as well as major ones. - The '-v' verbose output no longer includes start/finish time information. - Newlines in flex inputs can optionally include leading or trailing carriage-returns ('\r'), in support of several PC/Mac run-time libraries that automatically include these. - A start condition of the form "<*>" makes the following rule active in every start condition, whether exclusive or inclusive. - The following items have been corrected in the flex documentation: - '-C' table compression options *are* cumulative. - You may modify yytext but not lengthen it by appending characters to the end. Modifying its final character will affect '^' anchoring for the next rule matched if the character is changed to or from a newline. - The term "backtracking" has been renamed "backing up", since it is a one-time repositioning and not a repeated search. What used to be the "lex.backtrack" file is now "lex.backup". - Unindented "/* ... */" comments are allowed in the first flex input section, but not in the second. - yyless() can only be used in the flex input source, not externally. - You can use "yyrestart(yyin)" to throw away the current contents of the input buffer. - To write high-speed scanners, attempt to match as much text as possible with each rule. See MISC/fastwc/README for more information. - Using the beginning-of-line operator ('^') is fairly cheap. Using unput() is expensive. Using yyless() is cheap. - An example of scanning strings with embedded escape sequences has been added. - The example of backing-up in flexdoc was erroneous; it has been corrected. - A flex scanner's internal buffer now dynamically grows if needed to match large tokens. Note that growing the buffer presently requires rescanning the (large) token, so consuming a lot of text this way is a slow process. Also note that presently the buffer does *not* grow if you unput() more text than can fit into the buffer. - The MISC/ directory has been reorganized; see MISC/README for details. - yyless() can now be used in the third (user action) section of a scanner specification, thanks to Ceriel Jacobs. yyless() remains a macro and cannot be used outside of the scanner source. - The skeleton file is no longer opened at run-time, but instead compiled into a large string array (thanks to John Gilmore and friends at Cygnus). You can still use the -S flag to point flex at a different skeleton file. - flex no longer uses a temporary file to store the scanner's actions. - A number of changes have been made to decrease porting headaches. In particular, flex no longer uses memset() or ctime(), and provides a single simple mechanism for dealing with C compilers that still define malloc() as returning char* instead of void*. - Flex now detects if the scanner specification requires the -8 flag but the flag was not given or on by default. - A number of table-expansion fencepost bugs have been fixed, making flex more robust for generating large scanners. - flex more consistently identifies the location of errors in its input. - YY_USER_ACTION is now invoked only for "real" actions, not for internal actions used by the scanner for things like filling the buffer or handling EOF. - The rule "[^]]" now matches any character other than a ']'; formerly it matched any character at all followed by a ']'. This change was made for compatibility with AT&T lex. - A large number of miscellaneous bugs have been found and fixed thanks to Gerhard Wilhelms. - The source code has been heavily reformatted, making patches relative to previous flex releases no longer accurate. Changes between 2.3 Patch #8 (21Feb93) and 2.3 Patch #7: - Fixed bugs in dynamic memory allocation leading to grievous fencepost problems when generating large scanners. - Fixed bug causing infinite loops on character classes with 8-bit characters in them. - Fixed bug in matching repetitions with a lower bound of 0. - Fixed bug in scanning NUL characters using an "interactive" scanner. - Fixed bug in using yymore() at the end of a file. - Fixed bug in misrecognizing rules with variable trailing context. - Fixed bug compiling flex on Suns using gcc 2. - Fixed bug in not recognizing that input files with the character ASCII 128 in them require the -8 flag. - Fixed bug that could cause an infinite loop writing out error messages. - Fixed bug in not recognizing old-style lex % declarations if followed by a tab instead of a space. - Fixed potential crash when flex terminated early (usually due to a bad flag) and the -v flag had been given. - Added some missing declarations of void functions. - Changed to only use '\a' for __STDC__ compilers. - Updated mailing addresses. Changes between 2.3 Patch #7 (28Mar91) and 2.3 Patch #6: - Fixed out-of-bounds array access that caused bad tables to be produced on machines where the bad reference happened to yield a 1. This caused problems installing or running flex on some Suns, in particular. Changes between 2.3 Patch #6 (29Aug90) and 2.3 Patch #5: - Fixed a serious bug in yymore() which basically made it completely broken. Thanks goes to Jean Christophe of the Nethack development team for finding the problem and passing along the fix. Changes between 2.3 Patch #5 (16Aug90) and 2.3 Patch #4: - An up-to-date version of initscan.c so "make test" will work after applying the previous patches Changes between 2.3 Patch #4 (14Aug90) and 2.3 Patch #3: - Fixed bug in hexadecimal escapes which allowed only digits, not letters, in escapes - Fixed bug in previous "Changes" file! Changes between 2.3 Patch #3 (03Aug90) and 2.3 Patch #2: - Correction to patch #2 for gcc compilation; thanks goes to Paul Eggert for catching this. Changes between 2.3 Patch #2 (02Aug90) and original 2.3 release: - Fixed (hopefully) headaches involving declaring malloc() and free() for gcc, which defines __STDC__ but (often) doesn't come with the standard include files such as . Reordered #ifdef maze in the scanner skeleton in the hope of getting the declarations right for cfront and g++, too. - Note that this patch supercedes patch #1 for release 2.3, which was never announced but was available briefly for anonymous ftp. Changes between 2.3 (full) release of 28Jun90 and 2.2 (alpha) release: User-visible: - A lone <> rule (that is, one which is not qualified with a list of start conditions) now specifies the EOF action for *all* start conditions which haven't already had <> actions given. To specify an end-of-file action for just the initial state, use <>. - -d debug output is now contigent on the global yy_flex_debug being set to a non-zero value, which it is by default. - A new macro, YY_USER_INIT, is provided for the user to specify initialization action to be taken on the first call to the scanner. This action is done before the scanner does its own initialization. - yy_new_buffer() has been added as an alias for yy_create_buffer() - Comments beginning with '#' and extending to the end of the line now work, but have been deprecated (in anticipation of making flex recognize #line directives). - The funky restrictions on when semi-colons could follow the YY_NEW_FILE and yyless macros have been removed. They now behave identically to functions. - A bug in the sample redefinition of YY_INPUT in the documentation has been corrected. - A bug in the sample simple tokener in the documentation has been corrected. - The documentation on the incompatibilities between flex and lex has been reordered so that the discussion of yylineno and input() come first, as it's anticipated that these will be the most common source of headaches. Things which didn't used to be documented but now are: - flex interprets "^foo|bar" differently from lex. flex interprets it as "match either a 'foo' or a 'bar', providing it comes at the beginning of a line", whereas lex interprets it as "match either a 'foo' at the beginning of a line, or a 'bar' anywhere". - flex initializes the global "yyin" on the first call to the scanner, while lex initializes it at compile-time. - yy_switch_to_buffer() can be used in the yywrap() macro/routine. - flex scanners do not use stdio for their input, and hence when writing an interactive scanner one must explictly call fflush() after writing out a prompt. - flex scanner can be made reentrant (after a fashion) by using "yyrestart( yyin );". This is useful for interactive scanners which have interrupt handlers that long-jump out of the scanner. - a defense of why yylineno is not supported is included, along with a suggestion on how to convert scanners which rely on it. Other changes: - Prototypes and proper declarations of void routines have been added to the flex source code, courtesy of Kevin B. Kenny. - Routines dealing with memory allocation now use void* pointers instead of char* - see Makefile for porting implications. - Error-checking is now done when flex closes a file. - Various lint tweaks were added to reduce the number of gripes. - Makefile has been further parameterized to aid in porting. - Support for SCO Unix added. - Flex now sports the latest & greatest UC copyright notice (which is only slightly different from the previous one). - A note has been added to flexdoc.1 mentioning work in progress on modifying flex to generate straight C code rather than a table-driven automaton, with an email address of whom to contact if you are working along similar lines. Changes between 2.2 Patch #3 (30Mar90) and 2.2 Patch #2: - fixed bug which caused -I scanners to bomb Changes between 2.2 Patch #2 (27Mar90) and 2.2 Patch #1: - fixed bug writing past end of input buffer in yyunput() - fixed bug detecting NUL's at the end of a buffer Changes between 2.2 Patch #1 (23Mar90) and 2.2 (alpha) release: - Makefile fixes: definition of MAKE variable for systems which don't have it; installation of flexdoc.1 along with flex.1; fixed two bugs which could cause "bigtest" to fail. - flex.skel fix for compiling with g++. - README and flexdoc.1 no longer list an out-of-date BITNET address for contacting me. - minor typos and formatting changes to flex.1 and flexdoc.1. Changes between 2.2 (alpha) release of March '90 and previous release: User-visible: - Full user documentation now available. - Support for 8-bit scanners. - Scanners now accept NUL's. - A facility has been added for dealing with multiple input buffers. - Two manual entries now. One which fully describes flex (rather than just its differences from lex), and the other for quick(er) reference. - A number of changes to bring flex closer into compliance with the latest POSIX lex draft: %t support flex now accepts multiple input files and concatenates them together to form its input previous -c (compress) flag renamed -C do-nothing -c and -n flags added Any indented code or code within %{}'s in section 2 is now copied to the output - yyleng is now a bona fide global integer. - -d debug information now gives the line number of the matched rule instead of which number rule it was from the beginning of the file. - -v output now includes a summary of the flags used to generate the scanner. - unput() and yyrestart() are now globally callable. - yyrestart() no longer closes the previous value of yyin. - C++ support; generated scanners can be compiled with C++ compiler. - Primitive -lfl library added, containing default main() which calls yylex(). A number of routines currently living in the scanner skeleton will probably migrate to here in the future (in particular, yywrap() will probably cease to be a macro and instead be a function in the -lfl library). - Hexadecimal (\x) escape sequences added. - Support for MS-DOS, VMS, and Turbo-C integrated. - The %used/%unused operators have been deprecated. They may go away soon. Other changes: - Makefile enhanced for easier testing and installation. - The parser has been tweaked to detect some erroneous constructions which previously were missed. - Scanner input buffer overflow is now detected. - Bugs with missing "const" declarations fixed. - Out-of-date Minix/Atari patches provided. - Scanners no longer require printf() unless FLEX_DEBUG is being used. - A subtle input() bug has been fixed. - Line numbers for "continued action" rules (those following the special '|' action) are now correct. - unput() bug fixed; had been causing problems porting flex to VMS. - yymore() handling rewritten to fix bug with interaction between yymore() and trailing context. - EOF in actions now generates an error message. - Bug involving -CFe and generating equivalence classes fixed. - Bug which made -CF be treated as -Cf fixed. - Support for SysV tmpnam() added. - Unused #define's for scanner no longer generated. - Error messages which are associated with a particular input line are now all identified with their input line in standard format. - % directives which are valid to lex but not to flex are now ignored instead of generating warnings. - -DSYS_V flag can now also be specified -DUSG for System V compilation. Changes between 2.1 beta-test release of June '89 and previous release: User-visible: - -p flag generates a performance report to stderr. The report consists of comments regarding features of the scanner rules which result in slower scanners. - -b flag generates backtracking information to lex.backtrack. This is a list of scanner states which require backtracking and the characters on which they do so. By adding rules one can remove backtracking states. If all backtracking states are eliminated, the generated scanner will run faster. Backtracking is not yet documented in the manual entry. - Variable trailing context now works, i.e., one can have rules like "(foo)*/[ \t]*bletch". Some trailing context patterns still cannot be properly matched and generate error messages. These are patterns where the ending of the first part of the rule matches the beginning of the second part, such as "zx*/xy*", where the 'x*' matches the 'x' at the beginning of the trailing context. Lex won't get these patterns right either. - Faster scanners. - End-of-file rules. The special rule "<>" indicates actions which are to be taken when an end-of-file is encountered and yywrap() returns non-zero (i.e., indicates no further files to process). See manual entry for example. - The -r (reject used) flag is gone. flex now scans the input for occurrences of the string "REJECT" to determine if the action is needed. It tries to be intelligent about this but can be fooled. One can force the presence or absence of REJECT by adding a line in the first section of the form "%used REJECT" or "%unused REJECT". - yymore() has been implemented. Similarly to REJECT, flex detects the use of yymore(), which can be overridden using "%used" or "%unused". - Patterns like "x{0,3}" now work (i.e., with lower-limit == 0). - Removed '\^x' for ctrl-x misfeature. - Added '\a' and '\v' escape sequences. - \ now works for octal escape sequences; previously \0 was required. - Better error reporting; line numbers are associated with rules. - yyleng is a macro; it cannot be accessed outside of the scanner source file. - yytext and yyleng should not be modified within a flex action. - Generated scanners #define the name FLEX_SCANNER. - Rules are internally separated by YY_BREAK in lex.yy.c rather than break, to allow redefinition. - The macro YY_USER_ACTION can be redefined to provide an action which is always executed prior to the matched rule's action. - yyrestart() is a new action which can be used to restart the scanner after it has seen an end-of-file (a "real" one, that is, one for which yywrap() returned non-zero). It takes a FILE* argument indicating a new file to scan and sets things up so that a subsequent call to yylex() will start scanning that file. - Internal scanner names all preceded by "yy_" - lex.yy.c is deleted if errors are encountered during processing. - Comments may be put in the first section of the input by preceding them with '#'. Other changes: - Some portability-related bugs fixed, in particular for machines with unsigned characters or sizeof( int* ) != sizeof( int ). Also, tweaks for VMS and Microsoft C (MS-DOS), and identifiers all trimmed to be 31 or fewer characters. Shortened file names for dinosaur OS's. Checks for allocating > 64K memory on 16 bit'ers. Amiga tweaks. Compiles using gcc on a Sun-3. - Compressed and fast scanner skeletons merged. - Skeleton header files done away with. - Generated scanner uses prototypes and "const" for __STDC__. - -DSV flag is now -DSYS_V for System V compilation. - Removed all references to FTL language. - Software now covered by BSD Copyright. - flex will replace lex in subsequent BSD releases. flex-2.5.4/COPYING100444 75424 12 3445 6366532357 12106 0ustar kwzhstaffFlex carries the copyright used for BSD software, slightly modified because it originated at the Lawrence Berkeley (not Livermore!) Laboratory, which operates under a contract with the Department of Energy: Copyright (c) 1990 The Regents of the University of California. All rights reserved. This code is derived from software contributed to Berkeley by Vern Paxson. The United States Government has rights in this work pursuant to contract no. DE-AC03-76SF00098 between the United States Department of Energy and the University of California. Redistribution and use in source and binary forms with or without modification are permitted provided that: (1) source distributions retain this entire copyright notice and comment, and (2) distributions including binaries display the following acknowledgement: ``This product includes software developed by the University of California, Berkeley and its contributors'' in the documentation or other materials provided with the distribution and in all advertising materials mentioning features or use of this software. Neither the name of the University nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. This basically says "do whatever you please with this software except remove this notice or take advantage of the University's (or the flex authors') name". Note that the "flex.skl" scanner skeleton carries no copyright notice. You are free to do whatever you please with scanners generated using flex; for them, you are not even bound by the above copyright. flex-2.5.4/INSTALL100444 75424 12 13133 5475521271 12110 0ustar kwzhstaffThis is a generic INSTALL file for utilities distributions. If this package does not come with, e.g., installable documentation or data files, please ignore the references to them below. To compile this package: 1. Configure the package for your system. In the directory that this file is in, type `./configure'. If you're using `csh' on an old version of System V, you might need to type `sh configure' instead to prevent `csh' from trying to execute `configure' itself. The `configure' shell script attempts to guess correct values for various system-dependent variables used during compilation, and creates the Makefile(s) (one in each subdirectory of the source directory). In some packages it creates a C header file containing system-dependent definitions. It also creates a file `config.status' that you can run in the future to recreate the current configuration. Running `configure' takes a minute or two. While it is running, it prints some messages that tell what it is doing. If you don't want to see the messages, run `configure' with its standard output redirected to `/dev/null'; for example, `./configure >/dev/null'. To compile the package in a different directory from the one containing the source code, you must use a version of `make' that supports the VPATH variable, such as GNU `make'. `cd' to the directory where you want the object files and executables to go and run `configure'. `configure' automatically checks for the source code in the directory that `configure' is in and in `..'. If for some reason `configure' is not in the source code directory that you are configuring, then it will report that it can't find the source code. In that case, run `configure' with the option `--srcdir=DIR', where DIR is the directory that contains the source code. By default, `make install' will install the package's files in /usr/local/bin, /usr/local/lib, /usr/local/man, etc. You can specify an installation prefix other than /usr/local by giving `configure' the option `--prefix=PATH'. Alternately, you can do so by giving a value for the `prefix' variable when you run `make', e.g., make prefix=/usr/gnu You can specify separate installation prefixes for architecture-specific files and architecture-independent files. If you give `configure' the option `--exec-prefix=PATH' or set the `make' variable `exec_prefix' to PATH, the package will use PATH as the prefix for installing programs and libraries. Data files and documentation will still use the regular prefix. Normally, all files are installed using the regular prefix. Another `configure' option is useful mainly in `Makefile' rules for updating `config.status' and `Makefile'. The `--no-create' option figures out the configuration for your system and records it in `config.status', without actually configuring the package (creating `Makefile's and perhaps a configuration header file). Later, you can run `./config.status' to actually configure the package. You can also give `config.status' the `--recheck' option, which makes it re-run `configure' with the same arguments you used before. This option is useful if you change `configure'. Some packages pay attention to `--with-PACKAGE' options to `configure', where PACKAGE is something like `gnu-libc' or `x' (for X windows). The README should mention any --with- options that the package recognizes. `configure' ignores any other arguments that you give it. If your system requires unusual options for compilation or linking that `configure' doesn't know about, you can give `configure' initial values for some variables by setting them in the environment. In Bourne-compatible shells, you can do that on the command line like this: CC='gcc -traditional' DEFS=-D_POSIX_SOURCE ./configure The `make' variables that you might want to override with environment variables when running `configure' are: (For these variables, any value given in the environment overrides the value that `configure' would choose:) CC C compiler program. Default is `cc', or `gcc' if `gcc' is in your PATH. INSTALL Program to use to install files. Default is `install' if you have it, `cp' otherwise. (For these variables, any value given in the environment is added to the value that `configure' chooses:) DEFS Configuration options, in the form `-Dfoo -Dbar ...' Do not use this variable in packages that create a configuration header file. LIBS Libraries to link with, in the form `-lfoo -lbar ...' If you need to do unusual things to compile the package, we encourage you to figure out how `configure' could check whether to do them, and mail diffs or instructions to the address given in the README so we can include them in the next release. 2. Type `make' to compile the package. If you want, you can override the `make' variables CFLAGS and LDFLAGS like this: make CFLAGS=-O2 LDFLAGS=-s 3. If the package comes with self-tests and you want to run them, type `make check'. If you're not sure whether there are any, try it; if `make' responds with something like make: *** No way to make target `check'. Stop. then the package does not come with self-tests. 4. Type `make install' to install programs, data files, and documentation. 5. You can remove the program binaries and object files from the source directory by typing `make clean'. To also remove the Makefile(s), the header file containing system-dependent definitions (if the package uses one), and `config.status' (all the files that `configure' created), type `make distclean'. The file `configure.in' is used as a template to create `configure' by a program called `autoconf'. You will only need it if you want to regenerate `configure' using a newer version of `autoconf'. flex-2.5.4/FlexLexer.h100444 75424 12 13165 6366532542 13136 0ustar kwzhstaff// $Header: /home/daffy/u0/vern/flex/RCS/FlexLexer.h,v 1.19 96/05/25 20:43:02 vern Exp $ // FlexLexer.h -- define interfaces for lexical analyzer classes generated // by flex // Copyright (c) 1993 The Regents of the University of California. // All rights reserved. // // This code is derived from software contributed to Berkeley by // Kent Williams and Tom Epperly. // // Redistribution and use in source and binary forms with or without // modification are permitted provided that: (1) source distributions retain // this entire copyright notice and comment, and (2) distributions including // binaries display the following acknowledgement: ``This product includes // software developed by the University of California, Berkeley and its // contributors'' in the documentation or other materials provided with the // distribution and in all advertising materials mentioning features or use // of this software. Neither the name of the University nor the names of // its contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED // WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF // MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. // This file defines FlexLexer, an abstract class which specifies the // external interface provided to flex C++ lexer objects, and yyFlexLexer, // which defines a particular lexer class. // // If you want to create multiple lexer classes, you use the -P flag // to rename each yyFlexLexer to some other xxFlexLexer. You then // include in your other sources once per lexer class: // // #undef yyFlexLexer // #define yyFlexLexer xxFlexLexer // #include // // #undef yyFlexLexer // #define yyFlexLexer zzFlexLexer // #include // ... #ifndef __FLEX_LEXER_H // Never included before - need to define base class. #define __FLEX_LEXER_H #include extern "C++" { struct yy_buffer_state; typedef int yy_state_type; class FlexLexer { public: virtual ~FlexLexer() { } const char* YYText() { return yytext; } int YYLeng() { return yyleng; } virtual void yy_switch_to_buffer( struct yy_buffer_state* new_buffer ) = 0; virtual struct yy_buffer_state* yy_create_buffer( istream* s, int size ) = 0; virtual void yy_delete_buffer( struct yy_buffer_state* b ) = 0; virtual void yyrestart( istream* s ) = 0; virtual int yylex() = 0; // Call yylex with new input/output sources. int yylex( istream* new_in, ostream* new_out = 0 ) { switch_streams( new_in, new_out ); return yylex(); } // Switch to new input/output streams. A nil stream pointer // indicates "keep the current one". virtual void switch_streams( istream* new_in = 0, ostream* new_out = 0 ) = 0; int lineno() const { return yylineno; } int debug() const { return yy_flex_debug; } void set_debug( int flag ) { yy_flex_debug = flag; } protected: char* yytext; int yyleng; int yylineno; // only maintained if you use %option yylineno int yy_flex_debug; // only has effect with -d or "%option debug" }; } #endif #if defined(yyFlexLexer) || ! defined(yyFlexLexerOnce) // Either this is the first time through (yyFlexLexerOnce not defined), // or this is a repeated include to define a different flavor of // yyFlexLexer, as discussed in the flex man page. #define yyFlexLexerOnce class yyFlexLexer : public FlexLexer { public: // arg_yyin and arg_yyout default to the cin and cout, but we // only make that assignment when initializing in yylex(). yyFlexLexer( istream* arg_yyin = 0, ostream* arg_yyout = 0 ); virtual ~yyFlexLexer(); void yy_switch_to_buffer( struct yy_buffer_state* new_buffer ); struct yy_buffer_state* yy_create_buffer( istream* s, int size ); void yy_delete_buffer( struct yy_buffer_state* b ); void yyrestart( istream* s ); virtual int yylex(); virtual void switch_streams( istream* new_in, ostream* new_out ); protected: virtual int LexerInput( char* buf, int max_size ); virtual void LexerOutput( const char* buf, int size ); virtual void LexerError( const char* msg ); void yyunput( int c, char* buf_ptr ); int yyinput(); void yy_load_buffer_state(); void yy_init_buffer( struct yy_buffer_state* b, istream* s ); void yy_flush_buffer( struct yy_buffer_state* b ); int yy_start_stack_ptr; int yy_start_stack_depth; int* yy_start_stack; void yy_push_state( int new_state ); void yy_pop_state(); int yy_top_state(); yy_state_type yy_get_previous_state(); yy_state_type yy_try_NUL_trans( yy_state_type current_state ); int yy_get_next_buffer(); istream* yyin; // input source for default LexerInput ostream* yyout; // output sink for default LexerOutput struct yy_buffer_state* yy_current_buffer; // yy_hold_char holds the character lost when yytext is formed. char yy_hold_char; // Number of characters read into yy_ch_buf. int yy_n_chars; // Points to current character in buffer. char* yy_c_buf_p; int yy_init; // whether we need to initialize int yy_start; // start state number // Flag which is used to allow yywrap()'s to do buffer switches // instead of setting up a fresh yyin. A bit of a hack ... int yy_did_buffer_switch_on_eof; // The following are not always needed, but may be depending // on use of certain flex features (like REJECT or yymore()). yy_state_type yy_last_accepting_state; char* yy_last_accepting_cpos; yy_state_type* yy_state_buf; yy_state_type* yy_state_ptr; char* yy_full_match; int* yy_full_state; int yy_full_lp; int yy_lp; int yy_looking_for_trail_begin; int yy_more_flag; int yy_more_len; int yy_more_offset; int yy_prev_more_offset; }; #endif flex-2.5.4/configure.in100444 75424 12 643 5745537537 13346 0ustar kwzhstaffdnl Process this file with autoconf to produce a configure script. dnl AC_INIT(initscan.c) AC_CONFIG_HEADER(config.h:conf.in) AC_LN_S AC_PROG_YACC AC_PROG_CC AC_PROG_RANLIB AC_PROG_INSTALL AC_PROG_MAKE_SET AC_CONST AC_TYPE_SIZE_T AC_HEADER_STDC AC_HAVE_HEADERS(string.h malloc.h sys/types.h) case "$YACC" in *bison*) AC_ALLOCA ;; esac AC_OUTPUT(Makefile, [test -z "$CONFIG_HEADERS" || echo timestamp > stamp-h]) flex-2.5.4/conf.in100444 75424 12 1331 5704314213 12300 0ustar kwzhstaff/* $Header: /home/daffy/u0/vern/flex/RCS/conf.in,v 1.2 95/01/09 12:11:51 vern Exp $ */ /* Define to empty if the keyword does not work. */ #undef const /* Define to `unsigned' if doesn't define. */ #undef size_t /* Define if you have the ANSI C header files. */ #undef STDC_HEADERS /* Define if you have the header file. */ #undef HAVE_MALLOC_H /* Define if you have the header file. */ #undef HAVE_STRING_H /* Define if you have the header file. */ #undef HAVE_SYS_TYPES_H /* Define if you have and it should be used (not on Ultrix). */ #undef HAVE_ALLOCA_H /* Define if platform-specific command line handling is necessary. */ #undef NEED_ARGV_FIXUP flex-2.5.4/Makefile.in100644 75424 12 16740 6215400317 13123 0ustar kwzhstaff# @(#) $Header: /home/daffy/u0/vern/flex/RCS/Makefile.in,v 1.45 96/09/10 16:59:04 vern Exp $ (LBL) @SET_MAKE@ # Possible values for DEFS: # # By default, flex generates 8-bit scanners when using table compression, # and 7-bit scanners when using uncompressed tables (-f or -F options). # For flex to always generate 8-bit scanners, add "-DDEFAULT_CSIZE=256" # to DEFS. # # For Vax/VMS, add "-DVMS" to DEFS. # # For MS-DOS, add "-DMS_DOS" to DEFS. See the directory MISC/MSDOS for # additional info. CFLAGS = @CFLAGS@ CPPFLAGS = @CPPFLAGS@ DEFS = @DEFS@ LDFLAGS = LIBS = @LIBS@ # Installation targeting. Files will be installed under the tree # rooted at prefix. flex will be installed in bindir, libfl.a in # libdir, FlexLexer.h will be installed in includedir, and the manual # pages will be installed in mandir with extension manext. # # Raw, unformatted troff source will be installed if INSTALLMAN=man, # nroff preformatted versions will be installed if INSTALLMAN=cat. prefix = @prefix@ exec_prefix = @exec_prefix@ bindir = $(exec_prefix)/bin libdir = $(exec_prefix)/lib includedir = $(prefix)/include manext = 1 mandir = $(prefix)/man/man$(manext) # You can define these to be "lex" and "libl.a" if you want to replace # lex at your site. FLEX = flex FLEXLIB = libfl.a INSTALLMAN = man SHELL = /bin/sh srcdir = @srcdir@ VPATH = @srcdir@ LN_S = @LN_S@ YACC = @YACC@ CC = @CC@ AR = ar RANLIB = @RANLIB@ INSTALL = @INSTALL@ INSTALL_DATA = @INSTALL_DATA@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ # You normally do not need to modify anything below this point. # ------------------------------------------------------------ CPPFLAGS = -I. -I$(srcdir) .c.o: $(CC) -c $(CPPFLAGS) $(CFLAGS) $< HEADERS = flexdef.h version.h SOURCES = ccl.c dfa.c ecs.c gen.c main.c misc.c nfa.c parse.y \ scan.l skel.c sym.c tblcmp.c yylex.c OBJECTS = ccl.o dfa.o ecs.o gen.o main.o misc.o nfa.o parse.o \ scan.o skel.o sym.o tblcmp.o yylex.o @ALLOCA@ LIBSRCS = libmain.c libyywrap.c LIBOBJS = libmain.o libyywrap.o LINTSRCS = ccl.c dfa.c ecs.c gen.c main.c misc.c nfa.c parse.c \ scan.c skel.c sym.c tblcmp.c yylex.c DISTFILES = README NEWS COPYING INSTALL FlexLexer.h \ configure.in conf.in Makefile.in mkskel.sh flex.skl \ $(HEADERS) $(SOURCES) $(LIBSRCS) MISC \ flex.1 scan.c install.sh mkinstalldirs configure DIST_NAME = flex # which "flex" to use to generate scan.c from scan.l FLEX_EXEC = ./$(FLEX) FLEX_FLAGS = -t $(PERF_REPORT) COMPRESSION = PERF_REPORT = -p all: $(FLEX) $(FLEX): .bootstrap $(OBJECTS) $(FLEXLIB) $(CC) $(CFLAGS) -o $(FLEX) $(LDFLAGS) $(OBJECTS) $(FLEXLIB) $(LIBS) .bootstrap: initscan.c @rm -f scan.c cp $(srcdir)/initscan.c scan.c touch .bootstrap parse.c: parse.y $(YACC) -d $(srcdir)/parse.y @sed '/extern char.*malloc/d' parse.tmp @mv parse.tmp parse.c @mv y.tab.h parse.h @rm -f y.tab.c parse.h: parse.c scan.c: scan.l $(FLEX_EXEC) $(FLEX_FLAGS) $(COMPRESSION) $(srcdir)/scan.l >scan.c @sed s,\"$(srcdir)/scan.l\",\"scan.l\", scan.tmp @mv scan.tmp scan.c scan.o: scan.c parse.h flexdef.h config.h yylex.o: yylex.c parse.h flexdef.h config.h skel.c: flex.skl mkskel.sh $(SHELL) $(srcdir)/mkskel.sh $(srcdir)/flex.skl >skel.c main.o: main.c flexdef.h config.h version.h ccl.o: ccl.c flexdef.h config.h dfa.o: dfa.c flexdef.h config.h ecs.o: ecs.c flexdef.h config.h gen.o: gen.c flexdef.h config.h misc.o: misc.c flexdef.h config.h nfa.o: nfa.c flexdef.h config.h parse.o: parse.c flexdef.h config.h skel.o: skel.c flexdef.h config.h sym.o: sym.c flexdef.h config.h tblcmp.o: tblcmp.c flexdef.h config.h alloca.o: alloca.c $(CC) $(CPPFLAGS) $(CFLAGS) -c -Dxmalloc=yy_flex_xmalloc alloca.c alloca.c: $(srcdir)/MISC/alloca.c @rm -f alloca.c cp $(srcdir)/MISC/alloca.c . test: check check: $(FLEX) $(FLEX_EXEC) $(FLEX_FLAGS) $(COMPRESSION) $(srcdir)/scan.l \ | sed s,\"$(srcdir)/scan.l\",\"scan.l\", \ | diff scan.c - @echo "Check successful, using COMPRESSION=\"$(COMPRESSION)\"" bigcheck: rm -f scan.c ; $(MAKE) COMPRESSION="-C" check rm -f scan.c ; $(MAKE) COMPRESSION="-Ce" check rm -f scan.c ; $(MAKE) COMPRESSION="-Cm" check rm -f scan.c ; $(MAKE) COMPRESSION="-f" check rm -f scan.c ; $(MAKE) COMPRESSION="-Cfea" check rm -f scan.c ; $(MAKE) COMPRESSION="-CFer" check rm -f scan.c ; $(MAKE) COMPRESSION="-l" PERF_REPORT="" check rm -f scan.c ; $(MAKE) @echo "All checks successful" $(FLEXLIB): $(LIBOBJS) $(AR) cru $(FLEXLIB) $(LIBOBJS) -$(RANLIB) $(FLEXLIB) $(FLEX).man: flex.1 cd $(srcdir) && nroff -man flex.1 >$(FLEX).man install: $(FLEX) $(FLEXLIB) installdirs install.$(INSTALLMAN) $(INSTALL_PROGRAM) $(FLEX) $(bindir)/$(FLEX) @rm -f $(bindir)/$(FLEX)++ cd $(bindir) && $(LN_S) $(FLEX) $(FLEX)++ $(INSTALL_DATA) $(FLEXLIB) $(libdir)/$(FLEXLIB) -cd $(libdir) && $(RANLIB) $(FLEXLIB) $(INSTALL_DATA) $(srcdir)/FlexLexer.h $(includedir)/FlexLexer.h # Note, the following rules delete any vestigial flexdoc installed # for a prior flex release. install.man: flex.1 rm -f $(mandir)/$(FLEX)doc.$(manext) $(INSTALL_DATA) $(srcdir)/flex.1 $(mandir)/$(FLEX).$(manext) install.cat: $(FLEX).man rm -f $(mandir)/$(FLEX)doc.$(manext) $(INSTALL_DATA) $(srcdir)/$(FLEX).man $(mandir)/$(FLEX).$(manext) installdirs: $(SHELL) $(srcdir)/mkinstalldirs \ $(bindir) $(libdir) $(includedir) $(mandir) uninstall: rm -f $(bindir)/$(FLEX) $(bindir)/$(FLEX)++ rm -f $(libdir)/$(FLEXLIB) rm -f $(includedir)/FlexLexer.h rm -f $(mandir)/$(FLEX).$(manext) $(mandir)/$(FLEX)doc.$(manext) tags: $(SOURCES) ctags $(SOURCES) TAGS: $(SOURCES) etags $(SOURCES) lint: $(LINTSRCS) lint -Dconst= $(LINTSRCS) > flex.lint gcc-lint: $(LINTSRCS) gcc -Dlint -Wall $(LINTSRCS) >flex.gcc-lint 2>&1 mostlyclean: rm -f a.out *.bak core errs scan.tmp clean: mostlyclean rm -f flex parse.c parse.h *.o alloca.c *.lint lex.yy.c lex.yy.cc \ $(FLEXLIB) config.log config.cache distclean: clean rm -f .bootstrap $(FLEX) scan.c tags TAGS Makefile config.status \ config.h stamp-h config.log config.cache maintainer-clean: distclean @echo "This command is intended for maintainers to use;" @echo "it deletes files that may require special tools to rebuild." rm -f $(FLEX).man skel.c flex*.tar.gz flex*.tar.Z dist: $(FLEX) $(DISTFILES) parse.c parse.h $(srcdir)/$(FLEX).man $(MAKE) DIST_NAME=flex-`sed $(DIST_NAME)/MISC/flex.man tar chf $(DIST_NAME).tar $(DIST_NAME) compress <$(DIST_NAME).tar >$(DIST_NAME).tar.Z gzip <$(DIST_NAME).tar >$(DIST_NAME).tar.gz @rm $(DIST_NAME).tar # For an explanation of the following Makefile rules, see node # `Automatic Remaking' in GNU Autoconf documentation. Makefile: $(srcdir)/Makefile.in config.status CONFIG_FILES=$@ CONFIG_HEADERS= ./config.status config.status: configure ./config.status --recheck configure: configure.in cd $(srcdir) && autoconf config.h: stamp-h stamp-h: conf.in config.status CONFIG_FILES= CONFIG_HEADERS=config.h:conf.in ./config.status echo timestamp >stamp-h # conf.in: stamp-h.in # stamp-h.in: configure.in acconfig.h # cd $(srcdir) && autoheader # config.h.in conf.in # cho timestamp > $(srcdir)/stamp-h.in # Tell versions [3.59,3.63) of GNU make not to export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: flex-2.5.4/mkskel.sh100555 75424 12 300 5704314163 12627 0ustar kwzhstaff#! /bin/sh cat < %* /* cfront 1.2 defines "c_plusplus" instead of "__cplusplus" */ #ifdef c_plusplus #ifndef __cplusplus #define __cplusplus #endif #endif #ifdef __cplusplus #include %+ class istream; %* #include /* Use prototypes in function declarations. */ #define YY_USE_PROTOS /* The "const" storage-class-modifier is valid. */ #define YY_USE_CONST #else /* ! __cplusplus */ #if __STDC__ #define YY_USE_PROTOS #define YY_USE_CONST #endif /* __STDC__ */ #endif /* ! __cplusplus */ #ifdef __TURBOC__ #pragma warn -rch #pragma warn -use #include #include #define YY_USE_CONST #define YY_USE_PROTOS #endif #ifdef YY_USE_CONST #define yyconst const #else #define yyconst #endif #ifdef YY_USE_PROTOS #define YY_PROTO(proto) proto #else #define YY_PROTO(proto) () #endif /* Returned upon end-of-file. */ #define YY_NULL 0 /* Promotes a possibly negative, possibly signed char to an unsigned * integer for use as an array index. If the signed char is negative, * we want to instead treat it as an 8-bit unsigned char, hence the * double cast. */ #define YY_SC_TO_UI(c) ((unsigned int) (unsigned char) c) /* Enter a start condition. This macro really ought to take a parameter, * but we do it the disgusting crufty way forced on us by the ()-less * definition of BEGIN. */ #define BEGIN yy_start = 1 + 2 * /* Translate the current start state into a value that can be later handed * to BEGIN to return to the state. The YYSTATE alias is for lex * compatibility. */ #define YY_START ((yy_start - 1) / 2) #define YYSTATE YY_START /* Action number for EOF rule of a given start state. */ #define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1) /* Special action meaning "start processing a new file". */ #define YY_NEW_FILE yyrestart( yyin ) #define YY_END_OF_BUFFER_CHAR 0 /* Size of default input buffer. */ #define YY_BUF_SIZE 16384 typedef struct yy_buffer_state *YY_BUFFER_STATE; extern int yyleng; %- extern FILE *yyin, *yyout; %* #define EOB_ACT_CONTINUE_SCAN 0 #define EOB_ACT_END_OF_FILE 1 #define EOB_ACT_LAST_MATCH 2 /* The funky do-while in the following #define is used to turn the definition * int a single C statement (which needs a semi-colon terminator). This * avoids problems with code like: * * if ( condition_holds ) * yyless( 5 ); * else * do_something_else(); * * Prior to using the do-while the compiler would get upset at the * "else" because it interpreted the "if" statement as being all * done when it reached the ';' after the yyless() call. */ /* Return all but the first 'n' matched characters back to the input stream. */ #define yyless(n) \ do \ { \ /* Undo effects of setting up yytext. */ \ *yy_cp = yy_hold_char; \ YY_RESTORE_YY_MORE_OFFSET \ yy_c_buf_p = yy_cp = yy_bp + n - YY_MORE_ADJ; \ YY_DO_BEFORE_ACTION; /* set up yytext again */ \ } \ while ( 0 ) #define unput(c) yyunput( c, yytext_ptr ) /* The following is because we cannot portably get our hands on size_t * (without autoconf's help, which isn't available because we want * flex-generated scanners to compile on their own). */ typedef unsigned int yy_size_t; struct yy_buffer_state { %- FILE *yy_input_file; %+ istream* yy_input_file; %* char *yy_ch_buf; /* input buffer */ char *yy_buf_pos; /* current position in input buffer */ /* Size of input buffer in bytes, not including room for EOB * characters. */ yy_size_t yy_buf_size; /* Number of characters read into yy_ch_buf, not including EOB * characters. */ int yy_n_chars; /* Whether we "own" the buffer - i.e., we know we created it, * and can realloc() it to grow it, and should free() it to * delete it. */ int yy_is_our_buffer; /* Whether this is an "interactive" input source; if so, and * if we're using stdio for input, then we want to use getc() * instead of fread(), to make sure we stop fetching input after * each newline. */ int yy_is_interactive; /* Whether we're considered to be at the beginning of a line. * If so, '^' rules will be active on the next match, otherwise * not. */ int yy_at_bol; /* Whether to try to fill the input buffer when we reach the * end of it. */ int yy_fill_buffer; int yy_buffer_status; #define YY_BUFFER_NEW 0 #define YY_BUFFER_NORMAL 1 /* When an EOF's been seen but there's still some text to process * then we mark the buffer as YY_EOF_PENDING, to indicate that we * shouldn't try reading from the input source any more. We might * still have a bunch of tokens to match, though, because of * possible backing-up. * * When we actually see the EOF, we change the status to "new" * (via yyrestart()), so that the user can continue scanning by * just pointing yyin at a new input file. */ #define YY_BUFFER_EOF_PENDING 2 }; %- Standard (non-C++) definition static YY_BUFFER_STATE yy_current_buffer = 0; %* /* We provide macros for accessing buffer states in case in the * future we want to put the buffer states in a more general * "scanner state". */ #define YY_CURRENT_BUFFER yy_current_buffer %- Standard (non-C++) definition /* yy_hold_char holds the character lost when yytext is formed. */ static char yy_hold_char; static int yy_n_chars; /* number of characters read into yy_ch_buf */ int yyleng; /* Points to current character in buffer. */ static char *yy_c_buf_p = (char *) 0; static int yy_init = 1; /* whether we need to initialize */ static int yy_start = 0; /* start state number */ /* Flag which is used to allow yywrap()'s to do buffer switches * instead of setting up a fresh yyin. A bit of a hack ... */ static int yy_did_buffer_switch_on_eof; void yyrestart YY_PROTO(( FILE *input_file )); void yy_switch_to_buffer YY_PROTO(( YY_BUFFER_STATE new_buffer )); void yy_load_buffer_state YY_PROTO(( void )); YY_BUFFER_STATE yy_create_buffer YY_PROTO(( FILE *file, int size )); void yy_delete_buffer YY_PROTO(( YY_BUFFER_STATE b )); void yy_init_buffer YY_PROTO(( YY_BUFFER_STATE b, FILE *file )); void yy_flush_buffer YY_PROTO(( YY_BUFFER_STATE b )); #define YY_FLUSH_BUFFER yy_flush_buffer( yy_current_buffer ) YY_BUFFER_STATE yy_scan_buffer YY_PROTO(( char *base, yy_size_t size )); YY_BUFFER_STATE yy_scan_string YY_PROTO(( yyconst char *yy_str )); YY_BUFFER_STATE yy_scan_bytes YY_PROTO(( yyconst char *bytes, int len )); %* static void *yy_flex_alloc YY_PROTO(( yy_size_t )); static void *yy_flex_realloc YY_PROTO(( void *, yy_size_t )); static void yy_flex_free YY_PROTO(( void * )); #define yy_new_buffer yy_create_buffer #define yy_set_interactive(is_interactive) \ { \ if ( ! yy_current_buffer ) \ yy_current_buffer = yy_create_buffer( yyin, YY_BUF_SIZE ); \ yy_current_buffer->yy_is_interactive = is_interactive; \ } #define yy_set_bol(at_bol) \ { \ if ( ! yy_current_buffer ) \ yy_current_buffer = yy_create_buffer( yyin, YY_BUF_SIZE ); \ yy_current_buffer->yy_at_bol = at_bol; \ } #define YY_AT_BOL() (yy_current_buffer->yy_at_bol) %% yytext/yyin/yyout/yy_state_type/yylineno etc. def's & init go here %- Standard (non-C++) definition static yy_state_type yy_get_previous_state YY_PROTO(( void )); static yy_state_type yy_try_NUL_trans YY_PROTO(( yy_state_type current_state )); static int yy_get_next_buffer YY_PROTO(( void )); static void yy_fatal_error YY_PROTO(( yyconst char msg[] )); %* /* Done after the current pattern has been matched and before the * corresponding action - sets up yytext. */ #define YY_DO_BEFORE_ACTION \ yytext_ptr = yy_bp; \ %% code to fiddle yytext and yyleng for yymore() goes here yy_hold_char = *yy_cp; \ *yy_cp = '\0'; \ %% code to copy yytext_ptr to yytext[] goes here, if %array yy_c_buf_p = yy_cp; %% data tables for the DFA and the user's section 1 definitions go here /* Macros after this point can all be overridden by user definitions in * section 1. */ #ifndef YY_SKIP_YYWRAP #ifdef __cplusplus extern "C" int yywrap YY_PROTO(( void )); #else extern int yywrap YY_PROTO(( void )); #endif #endif %- #ifndef YY_NO_UNPUT static void yyunput YY_PROTO(( int c, char *buf_ptr )); #endif %* #ifndef yytext_ptr static void yy_flex_strncpy YY_PROTO(( char *, yyconst char *, int )); #endif #ifdef YY_NEED_STRLEN static int yy_flex_strlen YY_PROTO(( yyconst char * )); #endif #ifndef YY_NO_INPUT %- Standard (non-C++) definition #ifdef __cplusplus static int yyinput YY_PROTO(( void )); #else static int input YY_PROTO(( void )); #endif %* #endif #if YY_STACK_USED static int yy_start_stack_ptr = 0; static int yy_start_stack_depth = 0; static int *yy_start_stack = 0; #ifndef YY_NO_PUSH_STATE static void yy_push_state YY_PROTO(( int new_state )); #endif #ifndef YY_NO_POP_STATE static void yy_pop_state YY_PROTO(( void )); #endif #ifndef YY_NO_TOP_STATE static int yy_top_state YY_PROTO(( void )); #endif #else #define YY_NO_PUSH_STATE 1 #define YY_NO_POP_STATE 1 #define YY_NO_TOP_STATE 1 #endif #ifdef YY_MALLOC_DECL YY_MALLOC_DECL #else #if __STDC__ #ifndef __cplusplus #include #endif #else /* Just try to get by without declaring the routines. This will fail * miserably on non-ANSI systems for which sizeof(size_t) != sizeof(int) * or sizeof(void*) != sizeof(int). */ #endif #endif /* Amount of stuff to slurp up with each read. */ #ifndef YY_READ_BUF_SIZE #define YY_READ_BUF_SIZE 8192 #endif /* Copy whatever the last rule matched to the standard output. */ #ifndef ECHO %- Standard (non-C++) definition /* This used to be an fputs(), but since the string might contain NUL's, * we now use fwrite(). */ #define ECHO (void) fwrite( yytext, yyleng, 1, yyout ) %+ C++ definition #define ECHO LexerOutput( yytext, yyleng ) %* #endif /* Gets input and stuffs it into "buf". number of characters read, or YY_NULL, * is returned in "result". */ #ifndef YY_INPUT #define YY_INPUT(buf,result,max_size) \ %% fread()/read() definition of YY_INPUT goes here unless we're doing C++ %+ C++ definition if ( (result = LexerInput( (char *) buf, max_size )) < 0 ) \ YY_FATAL_ERROR( "input in flex scanner failed" ); %* #endif /* No semi-colon after return; correct usage is to write "yyterminate();" - * we don't want an extra ';' after the "return" because that will cause * some compilers to complain about unreachable statements. */ #ifndef yyterminate #define yyterminate() return YY_NULL #endif /* Number of entries by which start-condition stack grows. */ #ifndef YY_START_STACK_INCR #define YY_START_STACK_INCR 25 #endif /* Report a fatal error. */ #ifndef YY_FATAL_ERROR %- #define YY_FATAL_ERROR(msg) yy_fatal_error( msg ) %+ #define YY_FATAL_ERROR(msg) LexerError( msg ) %* #endif /* Default declaration of generated scanner - a define so the user can * easily add parameters. */ #ifndef YY_DECL %- Standard (non-C++) definition #define YY_DECL int yylex YY_PROTO(( void )) %+ C++ definition #define YY_DECL int yyFlexLexer::yylex() %* #endif /* Code executed at the beginning of each rule, after yytext and yyleng * have been set up. */ #ifndef YY_USER_ACTION #define YY_USER_ACTION #endif /* Code executed at the end of each rule. */ #ifndef YY_BREAK #define YY_BREAK break; #endif %% YY_RULE_SETUP definition goes here YY_DECL { register yy_state_type yy_current_state; register char *yy_cp, *yy_bp; register int yy_act; %% user's declarations go here if ( yy_init ) { yy_init = 0; #ifdef YY_USER_INIT YY_USER_INIT; #endif if ( ! yy_start ) yy_start = 1; /* first start state */ if ( ! yyin ) %- yyin = stdin; %+ yyin = &cin; %* if ( ! yyout ) %- yyout = stdout; %+ yyout = &cout; %* if ( ! yy_current_buffer ) yy_current_buffer = yy_create_buffer( yyin, YY_BUF_SIZE ); yy_load_buffer_state(); } while ( 1 ) /* loops until end-of-file is reached */ { %% yymore()-related code goes here yy_cp = yy_c_buf_p; /* Support of yytext. */ *yy_cp = yy_hold_char; /* yy_bp points to the position in yy_ch_buf of the start of * the current run. */ yy_bp = yy_cp; %% code to set up and find next match goes here yy_find_action: %% code to find the action number goes here YY_DO_BEFORE_ACTION; %% code for yylineno update goes here do_action: /* This label is used only to access EOF actions. */ %% debug code goes here switch ( yy_act ) { /* beginning of action switch */ %% actions go here case YY_END_OF_BUFFER: { /* Amount of text matched not including the EOB char. */ int yy_amount_of_matched_text = (int) (yy_cp - yytext_ptr) - 1; /* Undo the effects of YY_DO_BEFORE_ACTION. */ *yy_cp = yy_hold_char; YY_RESTORE_YY_MORE_OFFSET if ( yy_current_buffer->yy_buffer_status == YY_BUFFER_NEW ) { /* We're scanning a new file or input source. It's * possible that this happened because the user * just pointed yyin at a new source and called * yylex(). If so, then we have to assure * consistency between yy_current_buffer and our * globals. Here is the right place to do so, because * this is the first action (other than possibly a * back-up) that will match for the new input source. */ yy_n_chars = yy_current_buffer->yy_n_chars; yy_current_buffer->yy_input_file = yyin; yy_current_buffer->yy_buffer_status = YY_BUFFER_NORMAL; } /* Note that here we test for yy_c_buf_p "<=" to the position * of the first EOB in the buffer, since yy_c_buf_p will * already have been incremented past the NUL character * (since all states make transitions on EOB to the * end-of-buffer state). Contrast this with the test * in input(). */ if ( yy_c_buf_p <= &yy_current_buffer->yy_ch_buf[yy_n_chars] ) { /* This was really a NUL. */ yy_state_type yy_next_state; yy_c_buf_p = yytext_ptr + yy_amount_of_matched_text; yy_current_state = yy_get_previous_state(); /* Okay, we're now positioned to make the NUL * transition. We couldn't have * yy_get_previous_state() go ahead and do it * for us because it doesn't know how to deal * with the possibility of jamming (and we don't * want to build jamming into it because then it * will run more slowly). */ yy_next_state = yy_try_NUL_trans( yy_current_state ); yy_bp = yytext_ptr + YY_MORE_ADJ; if ( yy_next_state ) { /* Consume the NUL. */ yy_cp = ++yy_c_buf_p; yy_current_state = yy_next_state; goto yy_match; } else { %% code to do back-up for compressed tables and set up yy_cp goes here goto yy_find_action; } } else switch ( yy_get_next_buffer() ) { case EOB_ACT_END_OF_FILE: { yy_did_buffer_switch_on_eof = 0; if ( yywrap() ) { /* Note: because we've taken care in * yy_get_next_buffer() to have set up * yytext, we can now set up * yy_c_buf_p so that if some total * hoser (like flex itself) wants to * call the scanner after we return the * YY_NULL, it'll still work - another * YY_NULL will get returned. */ yy_c_buf_p = yytext_ptr + YY_MORE_ADJ; yy_act = YY_STATE_EOF(YY_START); goto do_action; } else { if ( ! yy_did_buffer_switch_on_eof ) YY_NEW_FILE; } break; } case EOB_ACT_CONTINUE_SCAN: yy_c_buf_p = yytext_ptr + yy_amount_of_matched_text; yy_current_state = yy_get_previous_state(); yy_cp = yy_c_buf_p; yy_bp = yytext_ptr + YY_MORE_ADJ; goto yy_match; case EOB_ACT_LAST_MATCH: yy_c_buf_p = &yy_current_buffer->yy_ch_buf[yy_n_chars]; yy_current_state = yy_get_previous_state(); yy_cp = yy_c_buf_p; yy_bp = yytext_ptr + YY_MORE_ADJ; goto yy_find_action; } break; } default: YY_FATAL_ERROR( "fatal flex scanner internal error--no action found" ); } /* end of action switch */ } /* end of scanning one token */ } /* end of yylex */ %+ yyFlexLexer::yyFlexLexer( istream* arg_yyin, ostream* arg_yyout ) { yyin = arg_yyin; yyout = arg_yyout; yy_c_buf_p = 0; yy_init = 1; yy_start = 0; yy_flex_debug = 0; yylineno = 1; // this will only get updated if %option yylineno yy_did_buffer_switch_on_eof = 0; yy_looking_for_trail_begin = 0; yy_more_flag = 0; yy_more_len = 0; yy_more_offset = yy_prev_more_offset = 0; yy_start_stack_ptr = yy_start_stack_depth = 0; yy_start_stack = 0; yy_current_buffer = 0; #ifdef YY_USES_REJECT yy_state_buf = new yy_state_type[YY_BUF_SIZE + 2]; #else yy_state_buf = 0; #endif } yyFlexLexer::~yyFlexLexer() { delete yy_state_buf; yy_delete_buffer( yy_current_buffer ); } void yyFlexLexer::switch_streams( istream* new_in, ostream* new_out ) { if ( new_in ) { yy_delete_buffer( yy_current_buffer ); yy_switch_to_buffer( yy_create_buffer( new_in, YY_BUF_SIZE ) ); } if ( new_out ) yyout = new_out; } #ifdef YY_INTERACTIVE int yyFlexLexer::LexerInput( char* buf, int /* max_size */ ) #else int yyFlexLexer::LexerInput( char* buf, int max_size ) #endif { if ( yyin->eof() || yyin->fail() ) return 0; #ifdef YY_INTERACTIVE yyin->get( buf[0] ); if ( yyin->eof() ) return 0; if ( yyin->bad() ) return -1; return 1; #else (void) yyin->read( buf, max_size ); if ( yyin->bad() ) return -1; else return yyin->gcount(); #endif } void yyFlexLexer::LexerOutput( const char* buf, int size ) { (void) yyout->write( buf, size ); } %* /* yy_get_next_buffer - try to read in a new buffer * * Returns a code representing an action: * EOB_ACT_LAST_MATCH - * EOB_ACT_CONTINUE_SCAN - continue scanning from current position * EOB_ACT_END_OF_FILE - end of file */ %- static int yy_get_next_buffer() %+ int yyFlexLexer::yy_get_next_buffer() %* { register char *dest = yy_current_buffer->yy_ch_buf; register char *source = yytext_ptr; register int number_to_move, i; int ret_val; if ( yy_c_buf_p > &yy_current_buffer->yy_ch_buf[yy_n_chars + 1] ) YY_FATAL_ERROR( "fatal flex scanner internal error--end of buffer missed" ); if ( yy_current_buffer->yy_fill_buffer == 0 ) { /* Don't try to fill the buffer, so this is an EOF. */ if ( yy_c_buf_p - yytext_ptr - YY_MORE_ADJ == 1 ) { /* We matched a single character, the EOB, so * treat this as a final EOF. */ return EOB_ACT_END_OF_FILE; } else { /* We matched some text prior to the EOB, first * process it. */ return EOB_ACT_LAST_MATCH; } } /* Try to read more data. */ /* First move last chars to start of buffer. */ number_to_move = (int) (yy_c_buf_p - yytext_ptr) - 1; for ( i = 0; i < number_to_move; ++i ) *(dest++) = *(source++); if ( yy_current_buffer->yy_buffer_status == YY_BUFFER_EOF_PENDING ) /* don't do the read, it's not guaranteed to return an EOF, * just force an EOF */ yy_current_buffer->yy_n_chars = yy_n_chars = 0; else { int num_to_read = yy_current_buffer->yy_buf_size - number_to_move - 1; while ( num_to_read <= 0 ) { /* Not enough room in the buffer - grow it. */ #ifdef YY_USES_REJECT YY_FATAL_ERROR( "input buffer overflow, can't enlarge buffer because scanner uses REJECT" ); #else /* just a shorter name for the current buffer */ YY_BUFFER_STATE b = yy_current_buffer; int yy_c_buf_p_offset = (int) (yy_c_buf_p - b->yy_ch_buf); if ( b->yy_is_our_buffer ) { int new_size = b->yy_buf_size * 2; if ( new_size <= 0 ) b->yy_buf_size += b->yy_buf_size / 8; else b->yy_buf_size *= 2; b->yy_ch_buf = (char *) /* Include room in for 2 EOB chars. */ yy_flex_realloc( (void *) b->yy_ch_buf, b->yy_buf_size + 2 ); } else /* Can't grow it, we don't own it. */ b->yy_ch_buf = 0; if ( ! b->yy_ch_buf ) YY_FATAL_ERROR( "fatal error - scanner input buffer overflow" ); yy_c_buf_p = &b->yy_ch_buf[yy_c_buf_p_offset]; num_to_read = yy_current_buffer->yy_buf_size - number_to_move - 1; #endif } if ( num_to_read > YY_READ_BUF_SIZE ) num_to_read = YY_READ_BUF_SIZE; /* Read in more data. */ YY_INPUT( (&yy_current_buffer->yy_ch_buf[number_to_move]), yy_n_chars, num_to_read ); yy_current_buffer->yy_n_chars = yy_n_chars; } if ( yy_n_chars == 0 ) { if ( number_to_move == YY_MORE_ADJ ) { ret_val = EOB_ACT_END_OF_FILE; yyrestart( yyin ); } else { ret_val = EOB_ACT_LAST_MATCH; yy_current_buffer->yy_buffer_status = YY_BUFFER_EOF_PENDING; } } else ret_val = EOB_ACT_CONTINUE_SCAN; yy_n_chars += number_to_move; yy_current_buffer->yy_ch_buf[yy_n_chars] = YY_END_OF_BUFFER_CHAR; yy_current_buffer->yy_ch_buf[yy_n_chars + 1] = YY_END_OF_BUFFER_CHAR; yytext_ptr = &yy_current_buffer->yy_ch_buf[0]; return ret_val; } /* yy_get_previous_state - get the state just before the EOB char was reached */ %- static yy_state_type yy_get_previous_state() %+ yy_state_type yyFlexLexer::yy_get_previous_state() %* { register yy_state_type yy_current_state; register char *yy_cp; %% code to get the start state into yy_current_state goes here for ( yy_cp = yytext_ptr + YY_MORE_ADJ; yy_cp < yy_c_buf_p; ++yy_cp ) { %% code to find the next state goes here } return yy_current_state; } /* yy_try_NUL_trans - try to make a transition on the NUL character * * synopsis * next_state = yy_try_NUL_trans( current_state ); */ %- #ifdef YY_USE_PROTOS static yy_state_type yy_try_NUL_trans( yy_state_type yy_current_state ) #else static yy_state_type yy_try_NUL_trans( yy_current_state ) yy_state_type yy_current_state; #endif %+ yy_state_type yyFlexLexer::yy_try_NUL_trans( yy_state_type yy_current_state ) %* { register int yy_is_jam; %% code to find the next state, and perhaps do backing up, goes here return yy_is_jam ? 0 : yy_current_state; } %- #ifndef YY_NO_UNPUT #ifdef YY_USE_PROTOS static void yyunput( int c, register char *yy_bp ) #else static void yyunput( c, yy_bp ) int c; register char *yy_bp; #endif %+ void yyFlexLexer::yyunput( int c, register char* yy_bp ) %* { register char *yy_cp = yy_c_buf_p; /* undo effects of setting up yytext */ *yy_cp = yy_hold_char; if ( yy_cp < yy_current_buffer->yy_ch_buf + 2 ) { /* need to shift things up to make room */ /* +2 for EOB chars. */ register int number_to_move = yy_n_chars + 2; register char *dest = &yy_current_buffer->yy_ch_buf[ yy_current_buffer->yy_buf_size + 2]; register char *source = &yy_current_buffer->yy_ch_buf[number_to_move]; while ( source > yy_current_buffer->yy_ch_buf ) *--dest = *--source; yy_cp += (int) (dest - source); yy_bp += (int) (dest - source); yy_current_buffer->yy_n_chars = yy_n_chars = yy_current_buffer->yy_buf_size; if ( yy_cp < yy_current_buffer->yy_ch_buf + 2 ) YY_FATAL_ERROR( "flex scanner push-back overflow" ); } *--yy_cp = (char) c; %% update yylineno here yytext_ptr = yy_bp; yy_hold_char = *yy_cp; yy_c_buf_p = yy_cp; } %- #endif /* ifndef YY_NO_UNPUT */ %* %- #ifdef __cplusplus static int yyinput() #else static int input() #endif %+ int yyFlexLexer::yyinput() %* { int c; *yy_c_buf_p = yy_hold_char; if ( *yy_c_buf_p == YY_END_OF_BUFFER_CHAR ) { /* yy_c_buf_p now points to the character we want to return. * If this occurs *before* the EOB characters, then it's a * valid NUL; if not, then we've hit the end of the buffer. */ if ( yy_c_buf_p < &yy_current_buffer->yy_ch_buf[yy_n_chars] ) /* This was really a NUL. */ *yy_c_buf_p = '\0'; else { /* need more input */ int offset = yy_c_buf_p - yytext_ptr; ++yy_c_buf_p; switch ( yy_get_next_buffer() ) { case EOB_ACT_LAST_MATCH: /* This happens because yy_g_n_b() * sees that we've accumulated a * token and flags that we need to * try matching the token before * proceeding. But for input(), * there's no matching to consider. * So convert the EOB_ACT_LAST_MATCH * to EOB_ACT_END_OF_FILE. */ /* Reset buffer status. */ yyrestart( yyin ); /* fall through */ case EOB_ACT_END_OF_FILE: { if ( yywrap() ) return EOF; if ( ! yy_did_buffer_switch_on_eof ) YY_NEW_FILE; #ifdef __cplusplus return yyinput(); #else return input(); #endif } case EOB_ACT_CONTINUE_SCAN: yy_c_buf_p = yytext_ptr + offset; break; } } } c = *(unsigned char *) yy_c_buf_p; /* cast for 8-bit char's */ *yy_c_buf_p = '\0'; /* preserve yytext */ yy_hold_char = *++yy_c_buf_p; %% update BOL and yylineno return c; } %- #ifdef YY_USE_PROTOS void yyrestart( FILE *input_file ) #else void yyrestart( input_file ) FILE *input_file; #endif %+ void yyFlexLexer::yyrestart( istream* input_file ) %* { if ( ! yy_current_buffer ) yy_current_buffer = yy_create_buffer( yyin, YY_BUF_SIZE ); yy_init_buffer( yy_current_buffer, input_file ); yy_load_buffer_state(); } %- #ifdef YY_USE_PROTOS void yy_switch_to_buffer( YY_BUFFER_STATE new_buffer ) #else void yy_switch_to_buffer( new_buffer ) YY_BUFFER_STATE new_buffer; #endif %+ void yyFlexLexer::yy_switch_to_buffer( YY_BUFFER_STATE new_buffer ) %* { if ( yy_current_buffer == new_buffer ) return; if ( yy_current_buffer ) { /* Flush out information for old buffer. */ *yy_c_buf_p = yy_hold_char; yy_current_buffer->yy_buf_pos = yy_c_buf_p; yy_current_buffer->yy_n_chars = yy_n_chars; } yy_current_buffer = new_buffer; yy_load_buffer_state(); /* We don't actually know whether we did this switch during * EOF (yywrap()) processing, but the only time this flag * is looked at is after yywrap() is called, so it's safe * to go ahead and always set it. */ yy_did_buffer_switch_on_eof = 1; } %- #ifdef YY_USE_PROTOS void yy_load_buffer_state( void ) #else void yy_load_buffer_state() #endif %+ void yyFlexLexer::yy_load_buffer_state() %* { yy_n_chars = yy_current_buffer->yy_n_chars; yytext_ptr = yy_c_buf_p = yy_current_buffer->yy_buf_pos; yyin = yy_current_buffer->yy_input_file; yy_hold_char = *yy_c_buf_p; } %- #ifdef YY_USE_PROTOS YY_BUFFER_STATE yy_create_buffer( FILE *file, int size ) #else YY_BUFFER_STATE yy_create_buffer( file, size ) FILE *file; int size; #endif %+ YY_BUFFER_STATE yyFlexLexer::yy_create_buffer( istream* file, int size ) %* { YY_BUFFER_STATE b; b = (YY_BUFFER_STATE) yy_flex_alloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); b->yy_buf_size = size; /* yy_ch_buf has to be 2 characters longer than the size given because * we need to put in 2 end-of-buffer characters. */ b->yy_ch_buf = (char *) yy_flex_alloc( b->yy_buf_size + 2 ); if ( ! b->yy_ch_buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); b->yy_is_our_buffer = 1; yy_init_buffer( b, file ); return b; } %- #ifdef YY_USE_PROTOS void yy_delete_buffer( YY_BUFFER_STATE b ) #else void yy_delete_buffer( b ) YY_BUFFER_STATE b; #endif %+ void yyFlexLexer::yy_delete_buffer( YY_BUFFER_STATE b ) %* { if ( ! b ) return; if ( b == yy_current_buffer ) yy_current_buffer = (YY_BUFFER_STATE) 0; if ( b->yy_is_our_buffer ) yy_flex_free( (void *) b->yy_ch_buf ); yy_flex_free( (void *) b ); } %- #ifndef YY_ALWAYS_INTERACTIVE #ifndef YY_NEVER_INTERACTIVE extern int isatty YY_PROTO(( int )); #endif #endif #ifdef YY_USE_PROTOS void yy_init_buffer( YY_BUFFER_STATE b, FILE *file ) #else void yy_init_buffer( b, file ) YY_BUFFER_STATE b; FILE *file; #endif %+ extern "C" int isatty YY_PROTO(( int )); void yyFlexLexer::yy_init_buffer( YY_BUFFER_STATE b, istream* file ) %* { yy_flush_buffer( b ); b->yy_input_file = file; b->yy_fill_buffer = 1; %- #if YY_ALWAYS_INTERACTIVE b->yy_is_interactive = 1; #else #if YY_NEVER_INTERACTIVE b->yy_is_interactive = 0; #else b->yy_is_interactive = file ? (isatty( fileno(file) ) > 0) : 0; #endif #endif %+ b->yy_is_interactive = 0; %* } %- #ifdef YY_USE_PROTOS void yy_flush_buffer( YY_BUFFER_STATE b ) #else void yy_flush_buffer( b ) YY_BUFFER_STATE b; #endif %+ void yyFlexLexer::yy_flush_buffer( YY_BUFFER_STATE b ) %* { if ( ! b ) return; b->yy_n_chars = 0; /* We always need two end-of-buffer characters. The first causes * a transition to the end-of-buffer state. The second causes * a jam in that state. */ b->yy_ch_buf[0] = YY_END_OF_BUFFER_CHAR; b->yy_ch_buf[1] = YY_END_OF_BUFFER_CHAR; b->yy_buf_pos = &b->yy_ch_buf[0]; b->yy_at_bol = 1; b->yy_buffer_status = YY_BUFFER_NEW; if ( b == yy_current_buffer ) yy_load_buffer_state(); } %* #ifndef YY_NO_SCAN_BUFFER %- #ifdef YY_USE_PROTOS YY_BUFFER_STATE yy_scan_buffer( char *base, yy_size_t size ) #else YY_BUFFER_STATE yy_scan_buffer( base, size ) char *base; yy_size_t size; #endif { YY_BUFFER_STATE b; if ( size < 2 || base[size-2] != YY_END_OF_BUFFER_CHAR || base[size-1] != YY_END_OF_BUFFER_CHAR ) /* They forgot to leave room for the EOB's. */ return 0; b = (YY_BUFFER_STATE) yy_flex_alloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) YY_FATAL_ERROR( "out of dynamic memory in yy_scan_buffer()" ); b->yy_buf_size = size - 2; /* "- 2" to take care of EOB's */ b->yy_buf_pos = b->yy_ch_buf = base; b->yy_is_our_buffer = 0; b->yy_input_file = 0; b->yy_n_chars = b->yy_buf_size; b->yy_is_interactive = 0; b->yy_at_bol = 1; b->yy_fill_buffer = 0; b->yy_buffer_status = YY_BUFFER_NEW; yy_switch_to_buffer( b ); return b; } %* #endif #ifndef YY_NO_SCAN_STRING %- #ifdef YY_USE_PROTOS YY_BUFFER_STATE yy_scan_string( yyconst char *yy_str ) #else YY_BUFFER_STATE yy_scan_string( yy_str ) yyconst char *yy_str; #endif { int len; for ( len = 0; yy_str[len]; ++len ) ; return yy_scan_bytes( yy_str, len ); } %* #endif #ifndef YY_NO_SCAN_BYTES %- #ifdef YY_USE_PROTOS YY_BUFFER_STATE yy_scan_bytes( yyconst char *bytes, int len ) #else YY_BUFFER_STATE yy_scan_bytes( bytes, len ) yyconst char *bytes; int len; #endif { YY_BUFFER_STATE b; char *buf; yy_size_t n; int i; /* Get memory for full buffer, including space for trailing EOB's. */ n = len + 2; buf = (char *) yy_flex_alloc( n ); if ( ! buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_scan_bytes()" ); for ( i = 0; i < len; ++i ) buf[i] = bytes[i]; buf[len] = buf[len+1] = YY_END_OF_BUFFER_CHAR; b = yy_scan_buffer( buf, n ); if ( ! b ) YY_FATAL_ERROR( "bad buffer in yy_scan_bytes()" ); /* It's okay to grow etc. this buffer, and we should throw it * away when we're done. */ b->yy_is_our_buffer = 1; return b; } %* #endif #ifndef YY_NO_PUSH_STATE %- #ifdef YY_USE_PROTOS static void yy_push_state( int new_state ) #else static void yy_push_state( new_state ) int new_state; #endif %+ void yyFlexLexer::yy_push_state( int new_state ) %* { if ( yy_start_stack_ptr >= yy_start_stack_depth ) { yy_size_t new_size; yy_start_stack_depth += YY_START_STACK_INCR; new_size = yy_start_stack_depth * sizeof( int ); if ( ! yy_start_stack ) yy_start_stack = (int *) yy_flex_alloc( new_size ); else yy_start_stack = (int *) yy_flex_realloc( (void *) yy_start_stack, new_size ); if ( ! yy_start_stack ) YY_FATAL_ERROR( "out of memory expanding start-condition stack" ); } yy_start_stack[yy_start_stack_ptr++] = YY_START; BEGIN(new_state); } #endif #ifndef YY_NO_POP_STATE %- static void yy_pop_state() %+ void yyFlexLexer::yy_pop_state() %* { if ( --yy_start_stack_ptr < 0 ) YY_FATAL_ERROR( "start-condition stack underflow" ); BEGIN(yy_start_stack[yy_start_stack_ptr]); } #endif #ifndef YY_NO_TOP_STATE %- static int yy_top_state() %+ int yyFlexLexer::yy_top_state() %* { return yy_start_stack[yy_start_stack_ptr - 1]; } #endif #ifndef YY_EXIT_FAILURE #define YY_EXIT_FAILURE 2 #endif %- #ifdef YY_USE_PROTOS static void yy_fatal_error( yyconst char msg[] ) #else static void yy_fatal_error( msg ) char msg[]; #endif { (void) fprintf( stderr, "%s\n", msg ); exit( YY_EXIT_FAILURE ); } %+ void yyFlexLexer::LexerError( yyconst char msg[] ) { cerr << msg << '\n'; exit( YY_EXIT_FAILURE ); } %* /* Redefine yyless() so it works in section 3 code. */ #undef yyless #define yyless(n) \ do \ { \ /* Undo effects of setting up yytext. */ \ yytext[yyleng] = yy_hold_char; \ yy_c_buf_p = yytext + n; \ yy_hold_char = *yy_c_buf_p; \ *yy_c_buf_p = '\0'; \ yyleng = n; \ } \ while ( 0 ) /* Internal utility routines. */ #ifndef yytext_ptr #ifdef YY_USE_PROTOS static void yy_flex_strncpy( char *s1, yyconst char *s2, int n ) #else static void yy_flex_strncpy( s1, s2, n ) char *s1; yyconst char *s2; int n; #endif { register int i; for ( i = 0; i < n; ++i ) s1[i] = s2[i]; } #endif #ifdef YY_NEED_STRLEN #ifdef YY_USE_PROTOS static int yy_flex_strlen( yyconst char *s ) #else static int yy_flex_strlen( s ) yyconst char *s; #endif { register int n; for ( n = 0; s[n]; ++n ) ; return n; } #endif #ifdef YY_USE_PROTOS static void *yy_flex_alloc( yy_size_t size ) #else static void *yy_flex_alloc( size ) yy_size_t size; #endif { return (void *) malloc( size ); } #ifdef YY_USE_PROTOS static void *yy_flex_realloc( void *ptr, yy_size_t size ) #else static void *yy_flex_realloc( ptr, size ) void *ptr; yy_size_t size; #endif { /* The cast to (char *) in the following accommodates both * implementations that use char* generic pointers, and those * that use void* generic pointers. It works with the latter * because both ANSI C and C++ allow castless assignment from * any pointer type to void*, and deal with argument conversions * as though doing an assignment. */ return (void *) realloc( (char *) ptr, size ); } #ifdef YY_USE_PROTOS static void yy_flex_free( void *ptr ) #else static void yy_flex_free( ptr ) void *ptr; #endif { free( ptr ); } #if YY_MAIN int main() { yylex(); return 0; } #endif flex-2.5.4/flexdef.h100444 75424 12 106502 6366532652 12675 0ustar kwzhstaff/* flexdef - definitions file for flex */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* @(#) $Header: /home/daffy/u0/vern/flex/RCS/flexdef.h,v 2.53 95/04/20 11:17:36 vern Exp $ (LBL) */ #include #include #include "config.h" #ifdef __TURBOC__ #define HAVE_STRING_H 1 #define MS_DOS 1 #ifndef __STDC__ #define __STDC__ 1 #endif #pragma warn -pro #pragma warn -rch #pragma warn -use #pragma warn -aus #pragma warn -par #pragma warn -pia #endif #ifdef HAVE_STRING_H #include #else #include #endif #ifdef HAVE_SYS_TYPES_H #include #endif #ifdef HAVE_MALLOC_H #include #endif #ifdef STDC_HEADERS #include #endif /* As an aid for the internationalization patch to flex, which * is maintained outside this distribution for copyright reasons. */ #define _(String) (String) /* Always be prepared to generate an 8-bit scanner. */ #define CSIZE 256 #define Char unsigned char /* Size of input alphabet - should be size of ASCII set. */ #ifndef DEFAULT_CSIZE #define DEFAULT_CSIZE 128 #endif #ifndef PROTO #if __STDC__ #define PROTO(proto) proto #else #define PROTO(proto) () #endif #endif #ifdef VMS #ifndef __VMS_POSIX #define unlink remove #define SHORT_FILE_NAMES #endif #endif #ifdef MS_DOS #define SHORT_FILE_NAMES #endif /* Maximum line length we'll have to deal with. */ #define MAXLINE 2048 #ifndef MIN #define MIN(x,y) ((x) < (y) ? (x) : (y)) #endif #ifndef MAX #define MAX(x,y) ((x) > (y) ? (x) : (y)) #endif #ifndef ABS #define ABS(x) ((x) < 0 ? -(x) : (x)) #endif /* ANSI C does not guarantee that isascii() is defined */ #ifndef isascii #define isascii(c) ((c) <= 0177) #endif #define true 1 #define false 0 #define unspecified -1 /* Special chk[] values marking the slots taking by end-of-buffer and action * numbers. */ #define EOB_POSITION -1 #define ACTION_POSITION -2 /* Number of data items per line for -f output. */ #define NUMDATAITEMS 10 /* Number of lines of data in -f output before inserting a blank line for * readability. */ #define NUMDATALINES 10 /* transition_struct_out() definitions. */ #define TRANS_STRUCT_PRINT_LENGTH 14 /* Returns true if an nfa state has an epsilon out-transition slot * that can be used. This definition is currently not used. */ #define FREE_EPSILON(state) \ (transchar[state] == SYM_EPSILON && \ trans2[state] == NO_TRANSITION && \ finalst[state] != state) /* Returns true if an nfa state has an epsilon out-transition character * and both slots are free */ #define SUPER_FREE_EPSILON(state) \ (transchar[state] == SYM_EPSILON && \ trans1[state] == NO_TRANSITION) \ /* Maximum number of NFA states that can comprise a DFA state. It's real * big because if there's a lot of rules, the initial state will have a * huge epsilon closure. */ #define INITIAL_MAX_DFA_SIZE 750 #define MAX_DFA_SIZE_INCREMENT 750 /* A note on the following masks. They are used to mark accepting numbers * as being special. As such, they implicitly limit the number of accepting * numbers (i.e., rules) because if there are too many rules the rule numbers * will overload the mask bits. Fortunately, this limit is \large/ (0x2000 == * 8192) so unlikely to actually cause any problems. A check is made in * new_rule() to ensure that this limit is not reached. */ /* Mask to mark a trailing context accepting number. */ #define YY_TRAILING_MASK 0x2000 /* Mask to mark the accepting number of the "head" of a trailing context * rule. */ #define YY_TRAILING_HEAD_MASK 0x4000 /* Maximum number of rules, as outlined in the above note. */ #define MAX_RULE (YY_TRAILING_MASK - 1) /* NIL must be 0. If not, its special meaning when making equivalence classes * (it marks the representative of a given e.c.) will be unidentifiable. */ #define NIL 0 #define JAM -1 /* to mark a missing DFA transition */ #define NO_TRANSITION NIL #define UNIQUE -1 /* marks a symbol as an e.c. representative */ #define INFINITY -1 /* for x{5,} constructions */ #define INITIAL_MAX_CCLS 100 /* max number of unique character classes */ #define MAX_CCLS_INCREMENT 100 /* Size of table holding members of character classes. */ #define INITIAL_MAX_CCL_TBL_SIZE 500 #define MAX_CCL_TBL_SIZE_INCREMENT 250 #define INITIAL_MAX_RULES 100 /* default maximum number of rules */ #define MAX_RULES_INCREMENT 100 #define INITIAL_MNS 2000 /* default maximum number of nfa states */ #define MNS_INCREMENT 1000 /* amount to bump above by if it's not enough */ #define INITIAL_MAX_DFAS 1000 /* default maximum number of dfa states */ #define MAX_DFAS_INCREMENT 1000 #define JAMSTATE -32766 /* marks a reference to the state that always jams */ /* Maximum number of NFA states. */ #define MAXIMUM_MNS 31999 /* Enough so that if it's subtracted from an NFA state number, the result * is guaranteed to be negative. */ #define MARKER_DIFFERENCE (MAXIMUM_MNS+2) /* Maximum number of nxt/chk pairs for non-templates. */ #define INITIAL_MAX_XPAIRS 2000 #define MAX_XPAIRS_INCREMENT 2000 /* Maximum number of nxt/chk pairs needed for templates. */ #define INITIAL_MAX_TEMPLATE_XPAIRS 2500 #define MAX_TEMPLATE_XPAIRS_INCREMENT 2500 #define SYM_EPSILON (CSIZE + 1) /* to mark transitions on the symbol epsilon */ #define INITIAL_MAX_SCS 40 /* maximum number of start conditions */ #define MAX_SCS_INCREMENT 40 /* amount to bump by if it's not enough */ #define ONE_STACK_SIZE 500 /* stack of states with only one out-transition */ #define SAME_TRANS -1 /* transition is the same as "default" entry for state */ /* The following percentages are used to tune table compression: * The percentage the number of out-transitions a state must be of the * number of equivalence classes in order to be considered for table * compaction by using protos. */ #define PROTO_SIZE_PERCENTAGE 15 /* The percentage the number of homogeneous out-transitions of a state * must be of the number of total out-transitions of the state in order * that the state's transition table is first compared with a potential * template of the most common out-transition instead of with the first * proto in the proto queue. */ #define CHECK_COM_PERCENTAGE 50 /* The percentage the number of differences between a state's transition * table and the proto it was first compared with must be of the total * number of out-transitions of the state in order to keep the first * proto as a good match and not search any further. */ #define FIRST_MATCH_DIFF_PERCENTAGE 10 /* The percentage the number of differences between a state's transition * table and the most similar proto must be of the state's total number * of out-transitions to use the proto as an acceptable close match. */ #define ACCEPTABLE_DIFF_PERCENTAGE 50 /* The percentage the number of homogeneous out-transitions of a state * must be of the number of total out-transitions of the state in order * to consider making a template from the state. */ #define TEMPLATE_SAME_PERCENTAGE 60 /* The percentage the number of differences between a state's transition * table and the most similar proto must be of the state's total number * of out-transitions to create a new proto from the state. */ #define NEW_PROTO_DIFF_PERCENTAGE 20 /* The percentage the total number of out-transitions of a state must be * of the number of equivalence classes in order to consider trying to * fit the transition table into "holes" inside the nxt/chk table. */ #define INTERIOR_FIT_PERCENTAGE 15 /* Size of region set aside to cache the complete transition table of * protos on the proto queue to enable quick comparisons. */ #define PROT_SAVE_SIZE 2000 #define MSP 50 /* maximum number of saved protos (protos on the proto queue) */ /* Maximum number of out-transitions a state can have that we'll rummage * around through the interior of the internal fast table looking for a * spot for it. */ #define MAX_XTIONS_FULL_INTERIOR_FIT 4 /* Maximum number of rules which will be reported as being associated * with a DFA state. */ #define MAX_ASSOC_RULES 100 /* Number that, if used to subscript an array, has a good chance of producing * an error; should be small enough to fit into a short. */ #define BAD_SUBSCRIPT -32767 /* Absolute value of largest number that can be stored in a short, with a * bit of slop thrown in for general paranoia. */ #define MAX_SHORT 32700 /* Declarations for global variables. */ /* Variables for symbol tables: * sctbl - start-condition symbol table * ndtbl - name-definition symbol table * ccltab - character class text symbol table */ struct hash_entry { struct hash_entry *prev, *next; char *name; char *str_val; int int_val; } ; typedef struct hash_entry **hash_table; #define NAME_TABLE_HASH_SIZE 101 #define START_COND_HASH_SIZE 101 #define CCL_HASH_SIZE 101 extern struct hash_entry *ndtbl[NAME_TABLE_HASH_SIZE]; extern struct hash_entry *sctbl[START_COND_HASH_SIZE]; extern struct hash_entry *ccltab[CCL_HASH_SIZE]; /* Variables for flags: * printstats - if true (-v), dump statistics * syntaxerror - true if a syntax error has been found * eofseen - true if we've seen an eof in the input file * ddebug - if true (-d), make a "debug" scanner * trace - if true (-T), trace processing * nowarn - if true (-w), do not generate warnings * spprdflt - if true (-s), suppress the default rule * interactive - if true (-I), generate an interactive scanner * caseins - if true (-i), generate a case-insensitive scanner * lex_compat - if true (-l), maximize compatibility with AT&T lex * do_yylineno - if true, generate code to maintain yylineno * useecs - if true (-Ce flag), use equivalence classes * fulltbl - if true (-Cf flag), don't compress the DFA state table * usemecs - if true (-Cm flag), use meta-equivalence classes * fullspd - if true (-F flag), use Jacobson method of table representation * gen_line_dirs - if true (i.e., no -L flag), generate #line directives * performance_report - if > 0 (i.e., -p flag), generate a report relating * to scanner performance; if > 1 (-p -p), report on minor performance * problems, too * backing_up_report - if true (i.e., -b flag), generate "lex.backup" file * listing backing-up states * C_plus_plus - if true (i.e., -+ flag), generate a C++ scanner class; * otherwise, a standard C scanner * long_align - if true (-Ca flag), favor long-word alignment. * use_read - if true (-f, -F, or -Cr) then use read() for scanner input; * otherwise, use fread(). * yytext_is_array - if true (i.e., %array directive), then declare * yytext as a array instead of a character pointer. Nice and inefficient. * do_yywrap - do yywrap() processing on EOF. If false, EOF treated as * "no more files". * csize - size of character set for the scanner we're generating; * 128 for 7-bit chars and 256 for 8-bit * yymore_used - if true, yymore() is used in input rules * reject - if true, generate back-up tables for REJECT macro * real_reject - if true, scanner really uses REJECT (as opposed to just * having "reject" set for variable trailing context) * continued_action - true if this rule's action is to "fall through" to * the next rule's action (i.e., the '|' action) * in_rule - true if we're inside an individual rule, false if not. * yymore_really_used - whether to treat yymore() as really used, regardless * of what we think based on references to it in the user's actions. * reject_really_used - same for REJECT */ extern int printstats, syntaxerror, eofseen, ddebug, trace, nowarn, spprdflt; extern int interactive, caseins, lex_compat, do_yylineno; extern int useecs, fulltbl, usemecs, fullspd; extern int gen_line_dirs, performance_report, backing_up_report; extern int C_plus_plus, long_align, use_read, yytext_is_array, do_yywrap; extern int csize; extern int yymore_used, reject, real_reject, continued_action, in_rule; extern int yymore_really_used, reject_really_used; /* Variables used in the flex input routines: * datapos - characters on current output line * dataline - number of contiguous lines of data in current data * statement. Used to generate readable -f output * linenum - current input line number * out_linenum - current output line number * skelfile - the skeleton file * skel - compiled-in skeleton array * skel_ind - index into "skel" array, if skelfile is nil * yyin - input file * backing_up_file - file to summarize backing-up states to * infilename - name of input file * outfilename - name of output file * did_outfilename - whether outfilename was explicitly set * prefix - the prefix used for externally visible names ("yy" by default) * yyclass - yyFlexLexer subclass to use for YY_DECL * do_stdinit - whether to initialize yyin/yyout to stdin/stdout * use_stdout - the -t flag * input_files - array holding names of input files * num_input_files - size of input_files array * program_name - name with which program was invoked * * action_array - array to hold the rule actions * action_size - size of action_array * defs1_offset - index where the user's section 1 definitions start * in action_array * prolog_offset - index where the prolog starts in action_array * action_offset - index where the non-prolog starts in action_array * action_index - index where the next action should go, with respect * to "action_array" */ extern int datapos, dataline, linenum, out_linenum; extern FILE *skelfile, *yyin, *backing_up_file; extern const char *skel[]; extern int skel_ind; extern char *infilename, *outfilename; extern int did_outfilename; extern char *prefix, *yyclass; extern int do_stdinit, use_stdout; extern char **input_files; extern int num_input_files; extern char *program_name; extern char *action_array; extern int action_size; extern int defs1_offset, prolog_offset, action_offset, action_index; /* Variables for stack of states having only one out-transition: * onestate - state number * onesym - transition symbol * onenext - target state * onedef - default base entry * onesp - stack pointer */ extern int onestate[ONE_STACK_SIZE], onesym[ONE_STACK_SIZE]; extern int onenext[ONE_STACK_SIZE], onedef[ONE_STACK_SIZE], onesp; /* Variables for nfa machine data: * current_mns - current maximum on number of NFA states * num_rules - number of the last accepting state; also is number of * rules created so far * num_eof_rules - number of <> rules * default_rule - number of the default rule * current_max_rules - current maximum number of rules * lastnfa - last nfa state number created * firstst - physically the first state of a fragment * lastst - last physical state of fragment * finalst - last logical state of fragment * transchar - transition character * trans1 - transition state * trans2 - 2nd transition state for epsilons * accptnum - accepting number * assoc_rule - rule associated with this NFA state (or 0 if none) * state_type - a STATE_xxx type identifying whether the state is part * of a normal rule, the leading state in a trailing context * rule (i.e., the state which marks the transition from * recognizing the text-to-be-matched to the beginning of * the trailing context), or a subsequent state in a trailing * context rule * rule_type - a RULE_xxx type identifying whether this a ho-hum * normal rule or one which has variable head & trailing * context * rule_linenum - line number associated with rule * rule_useful - true if we've determined that the rule can be matched */ extern int current_mns, current_max_rules; extern int num_rules, num_eof_rules, default_rule, lastnfa; extern int *firstst, *lastst, *finalst, *transchar, *trans1, *trans2; extern int *accptnum, *assoc_rule, *state_type; extern int *rule_type, *rule_linenum, *rule_useful; /* Different types of states; values are useful as masks, as well, for * routines like check_trailing_context(). */ #define STATE_NORMAL 0x1 #define STATE_TRAILING_CONTEXT 0x2 /* Global holding current type of state we're making. */ extern int current_state_type; /* Different types of rules. */ #define RULE_NORMAL 0 #define RULE_VARIABLE 1 /* True if the input rules include a rule with both variable-length head * and trailing context, false otherwise. */ extern int variable_trailing_context_rules; /* Variables for protos: * numtemps - number of templates created * numprots - number of protos created * protprev - backlink to a more-recently used proto * protnext - forward link to a less-recently used proto * prottbl - base/def table entry for proto * protcomst - common state of proto * firstprot - number of the most recently used proto * lastprot - number of the least recently used proto * protsave contains the entire state array for protos */ extern int numtemps, numprots, protprev[MSP], protnext[MSP], prottbl[MSP]; extern int protcomst[MSP], firstprot, lastprot, protsave[PROT_SAVE_SIZE]; /* Variables for managing equivalence classes: * numecs - number of equivalence classes * nextecm - forward link of Equivalence Class members * ecgroup - class number or backward link of EC members * nummecs - number of meta-equivalence classes (used to compress * templates) * tecfwd - forward link of meta-equivalence classes members * tecbck - backward link of MEC's */ /* Reserve enough room in the equivalence class arrays so that we * can use the CSIZE'th element to hold equivalence class information * for the NUL character. Later we'll move this information into * the 0th element. */ extern int numecs, nextecm[CSIZE + 1], ecgroup[CSIZE + 1], nummecs; /* Meta-equivalence classes are indexed starting at 1, so it's possible * that they will require positions from 1 .. CSIZE, i.e., CSIZE + 1 * slots total (since the arrays are 0-based). nextecm[] and ecgroup[] * don't require the extra position since they're indexed from 1 .. CSIZE - 1. */ extern int tecfwd[CSIZE + 1], tecbck[CSIZE + 1]; /* Variables for start conditions: * lastsc - last start condition created * current_max_scs - current limit on number of start conditions * scset - set of rules active in start condition * scbol - set of rules active only at the beginning of line in a s.c. * scxclu - true if start condition is exclusive * sceof - true if start condition has EOF rule * scname - start condition name */ extern int lastsc, *scset, *scbol, *scxclu, *sceof; extern int current_max_scs; extern char **scname; /* Variables for dfa machine data: * current_max_dfa_size - current maximum number of NFA states in DFA * current_max_xpairs - current maximum number of non-template xtion pairs * current_max_template_xpairs - current maximum number of template pairs * current_max_dfas - current maximum number DFA states * lastdfa - last dfa state number created * nxt - state to enter upon reading character * chk - check value to see if "nxt" applies * tnxt - internal nxt table for templates * base - offset into "nxt" for given state * def - where to go if "chk" disallows "nxt" entry * nultrans - NUL transition for each state * NUL_ec - equivalence class of the NUL character * tblend - last "nxt/chk" table entry being used * firstfree - first empty entry in "nxt/chk" table * dss - nfa state set for each dfa * dfasiz - size of nfa state set for each dfa * dfaacc - accepting set for each dfa state (if using REJECT), or accepting * number, if not * accsiz - size of accepting set for each dfa state * dhash - dfa state hash value * numas - number of DFA accepting states created; note that this * is not necessarily the same value as num_rules, which is the analogous * value for the NFA * numsnpairs - number of state/nextstate transition pairs * jambase - position in base/def where the default jam table starts * jamstate - state number corresponding to "jam" state * end_of_buffer_state - end-of-buffer dfa state number */ extern int current_max_dfa_size, current_max_xpairs; extern int current_max_template_xpairs, current_max_dfas; extern int lastdfa, *nxt, *chk, *tnxt; extern int *base, *def, *nultrans, NUL_ec, tblend, firstfree, **dss, *dfasiz; extern union dfaacc_union { int *dfaacc_set; int dfaacc_state; } *dfaacc; extern int *accsiz, *dhash, numas; extern int numsnpairs, jambase, jamstate; extern int end_of_buffer_state; /* Variables for ccl information: * lastccl - ccl index of the last created ccl * current_maxccls - current limit on the maximum number of unique ccl's * cclmap - maps a ccl index to its set pointer * ccllen - gives the length of a ccl * cclng - true for a given ccl if the ccl is negated * cclreuse - counts how many times a ccl is re-used * current_max_ccl_tbl_size - current limit on number of characters needed * to represent the unique ccl's * ccltbl - holds the characters in each ccl - indexed by cclmap */ extern int lastccl, *cclmap, *ccllen, *cclng, cclreuse; extern int current_maxccls, current_max_ccl_tbl_size; extern Char *ccltbl; /* Variables for miscellaneous information: * nmstr - last NAME scanned by the scanner * sectnum - section number currently being parsed * nummt - number of empty nxt/chk table entries * hshcol - number of hash collisions detected by snstods * dfaeql - number of times a newly created dfa was equal to an old one * numeps - number of epsilon NFA states created * eps2 - number of epsilon states which have 2 out-transitions * num_reallocs - number of times it was necessary to realloc() a group * of arrays * tmpuses - number of DFA states that chain to templates * totnst - total number of NFA states used to make DFA states * peakpairs - peak number of transition pairs we had to store internally * numuniq - number of unique transitions * numdup - number of duplicate transitions * hshsave - number of hash collisions saved by checking number of states * num_backing_up - number of DFA states requiring backing up * bol_needed - whether scanner needs beginning-of-line recognition */ extern char nmstr[MAXLINE]; extern int sectnum, nummt, hshcol, dfaeql, numeps, eps2, num_reallocs; extern int tmpuses, totnst, peakpairs, numuniq, numdup, hshsave; extern int num_backing_up, bol_needed; void *allocate_array PROTO((int, size_t)); void *reallocate_array PROTO((void*, int, size_t)); void *flex_alloc PROTO((size_t)); void *flex_realloc PROTO((void*, size_t)); void flex_free PROTO((void*)); #define allocate_integer_array(size) \ (int *) allocate_array( size, sizeof( int ) ) #define reallocate_integer_array(array,size) \ (int *) reallocate_array( (void *) array, size, sizeof( int ) ) #define allocate_int_ptr_array(size) \ (int **) allocate_array( size, sizeof( int * ) ) #define allocate_char_ptr_array(size) \ (char **) allocate_array( size, sizeof( char * ) ) #define allocate_dfaacc_union(size) \ (union dfaacc_union *) \ allocate_array( size, sizeof( union dfaacc_union ) ) #define reallocate_int_ptr_array(array,size) \ (int **) reallocate_array( (void *) array, size, sizeof( int * ) ) #define reallocate_char_ptr_array(array,size) \ (char **) reallocate_array( (void *) array, size, sizeof( char * ) ) #define reallocate_dfaacc_union(array, size) \ (union dfaacc_union *) \ reallocate_array( (void *) array, size, sizeof( union dfaacc_union ) ) #define allocate_character_array(size) \ (char *) allocate_array( size, sizeof( char ) ) #define reallocate_character_array(array,size) \ (char *) reallocate_array( (void *) array, size, sizeof( char ) ) #define allocate_Character_array(size) \ (Char *) allocate_array( size, sizeof( Char ) ) #define reallocate_Character_array(array,size) \ (Char *) reallocate_array( (void *) array, size, sizeof( Char ) ) /* Used to communicate between scanner and parser. The type should really * be YYSTYPE, but we can't easily get our hands on it. */ extern int yylval; /* External functions that are cross-referenced among the flex source files. */ /* from file ccl.c */ extern void ccladd PROTO((int, int)); /* add a single character to a ccl */ extern int cclinit PROTO((void)); /* make an empty ccl */ extern void cclnegate PROTO((int)); /* negate a ccl */ /* List the members of a set of characters in CCL form. */ extern void list_character_set PROTO((FILE*, int[])); /* from file dfa.c */ /* Check a DFA state for backing up. */ extern void check_for_backing_up PROTO((int, int[])); /* Check to see if NFA state set constitutes "dangerous" trailing context. */ extern void check_trailing_context PROTO((int*, int, int*, int)); /* Construct the epsilon closure of a set of ndfa states. */ extern int *epsclosure PROTO((int*, int*, int[], int*, int*)); /* Increase the maximum number of dfas. */ extern void increase_max_dfas PROTO((void)); extern void ntod PROTO((void)); /* convert a ndfa to a dfa */ /* Converts a set of ndfa states into a dfa state. */ extern int snstods PROTO((int[], int, int[], int, int, int*)); /* from file ecs.c */ /* Convert character classes to set of equivalence classes. */ extern void ccl2ecl PROTO((void)); /* Associate equivalence class numbers with class members. */ extern int cre8ecs PROTO((int[], int[], int)); /* Update equivalence classes based on character class transitions. */ extern void mkeccl PROTO((Char[], int, int[], int[], int, int)); /* Create equivalence class for single character. */ extern void mkechar PROTO((int, int[], int[])); /* from file gen.c */ extern void do_indent PROTO((void)); /* indent to the current level */ /* Generate the code to keep backing-up information. */ extern void gen_backing_up PROTO((void)); /* Generate the code to perform the backing up. */ extern void gen_bu_action PROTO((void)); /* Generate full speed compressed transition table. */ extern void genctbl PROTO((void)); /* Generate the code to find the action number. */ extern void gen_find_action PROTO((void)); extern void genftbl PROTO((void)); /* generate full transition table */ /* Generate the code to find the next compressed-table state. */ extern void gen_next_compressed_state PROTO((char*)); /* Generate the code to find the next match. */ extern void gen_next_match PROTO((void)); /* Generate the code to find the next state. */ extern void gen_next_state PROTO((int)); /* Generate the code to make a NUL transition. */ extern void gen_NUL_trans PROTO((void)); /* Generate the code to find the start state. */ extern void gen_start_state PROTO((void)); /* Generate data statements for the transition tables. */ extern void gentabs PROTO((void)); /* Write out a formatted string at the current indentation level. */ extern void indent_put2s PROTO((char[], char[])); /* Write out a string + newline at the current indentation level. */ extern void indent_puts PROTO((char[])); extern void make_tables PROTO((void)); /* generate transition tables */ /* from file main.c */ extern void check_options PROTO((void)); extern void flexend PROTO((int)); extern void usage PROTO((void)); /* from file misc.c */ /* Add a #define to the action file. */ extern void action_define PROTO(( char *defname, int value )); /* Add the given text to the stored actions. */ extern void add_action PROTO(( char *new_text )); /* True if a string is all lower case. */ extern int all_lower PROTO((register char *)); /* True if a string is all upper case. */ extern int all_upper PROTO((register char *)); /* Bubble sort an integer array. */ extern void bubble PROTO((int [], int)); /* Check a character to make sure it's in the expected range. */ extern void check_char PROTO((int c)); /* Replace upper-case letter to lower-case. */ extern Char clower PROTO((int)); /* Returns a dynamically allocated copy of a string. */ extern char *copy_string PROTO((register const char *)); /* Returns a dynamically allocated copy of a (potentially) unsigned string. */ extern Char *copy_unsigned_string PROTO((register Char *)); /* Shell sort a character array. */ extern void cshell PROTO((Char [], int, int)); /* Finish up a block of data declarations. */ extern void dataend PROTO((void)); /* Flush generated data statements. */ extern void dataflush PROTO((void)); /* Report an error message and terminate. */ extern void flexerror PROTO((const char[])); /* Report a fatal error message and terminate. */ extern void flexfatal PROTO((const char[])); /* Convert a hexadecimal digit string to an integer value. */ extern int htoi PROTO((Char[])); /* Report an error message formatted with one integer argument. */ extern void lerrif PROTO((const char[], int)); /* Report an error message formatted with one string argument. */ extern void lerrsf PROTO((const char[], const char[])); /* Spit out a "#line" statement. */ extern void line_directive_out PROTO((FILE*, int)); /* Mark the current position in the action array as the end of the section 1 * user defs. */ extern void mark_defs1 PROTO((void)); /* Mark the current position in the action array as the end of the prolog. */ extern void mark_prolog PROTO((void)); /* Generate a data statment for a two-dimensional array. */ extern void mk2data PROTO((int)); extern void mkdata PROTO((int)); /* generate a data statement */ /* Return the integer represented by a string of digits. */ extern int myctoi PROTO((char [])); /* Return character corresponding to escape sequence. */ extern Char myesc PROTO((Char[])); /* Convert an octal digit string to an integer value. */ extern int otoi PROTO((Char [] )); /* Output a (possibly-formatted) string to the generated scanner. */ extern void out PROTO((const char [])); extern void out_dec PROTO((const char [], int)); extern void out_dec2 PROTO((const char [], int, int)); extern void out_hex PROTO((const char [], unsigned int)); extern void out_line_count PROTO((const char [])); extern void out_str PROTO((const char [], const char [])); extern void out_str3 PROTO((const char [], const char [], const char [], const char [])); extern void out_str_dec PROTO((const char [], const char [], int)); extern void outc PROTO((int)); extern void outn PROTO((const char [])); /* Return a printable version of the given character, which might be * 8-bit. */ extern char *readable_form PROTO((int)); /* Write out one section of the skeleton file. */ extern void skelout PROTO((void)); /* Output a yy_trans_info structure. */ extern void transition_struct_out PROTO((int, int)); /* Only needed when using certain broken versions of bison to build parse.c. */ extern void *yy_flex_xmalloc PROTO(( int )); /* Set a region of memory to 0. */ extern void zero_out PROTO((char *, size_t)); /* from file nfa.c */ /* Add an accepting state to a machine. */ extern void add_accept PROTO((int, int)); /* Make a given number of copies of a singleton machine. */ extern int copysingl PROTO((int, int)); /* Debugging routine to write out an nfa. */ extern void dumpnfa PROTO((int)); /* Finish up the processing for a rule. */ extern void finish_rule PROTO((int, int, int, int)); /* Connect two machines together. */ extern int link_machines PROTO((int, int)); /* Mark each "beginning" state in a machine as being a "normal" (i.e., * not trailing context associated) state. */ extern void mark_beginning_as_normal PROTO((register int)); /* Make a machine that branches to two machines. */ extern int mkbranch PROTO((int, int)); extern int mkclos PROTO((int)); /* convert a machine into a closure */ extern int mkopt PROTO((int)); /* make a machine optional */ /* Make a machine that matches either one of two machines. */ extern int mkor PROTO((int, int)); /* Convert a machine into a positive closure. */ extern int mkposcl PROTO((int)); extern int mkrep PROTO((int, int, int)); /* make a replicated machine */ /* Create a state with a transition on a given symbol. */ extern int mkstate PROTO((int)); extern void new_rule PROTO((void)); /* initialize for a new rule */ /* from file parse.y */ /* Build the "<>" action for the active start conditions. */ extern void build_eof_action PROTO((void)); /* Write out a message formatted with one string, pinpointing its location. */ extern void format_pinpoint_message PROTO((char[], char[])); /* Write out a message, pinpointing its location. */ extern void pinpoint_message PROTO((char[])); /* Write out a warning, pinpointing it at the given line. */ extern void line_warning PROTO(( char[], int )); /* Write out a message, pinpointing it at the given line. */ extern void line_pinpoint PROTO(( char[], int )); /* Report a formatted syntax error. */ extern void format_synerr PROTO((char [], char[])); extern void synerr PROTO((char [])); /* report a syntax error */ extern void format_warn PROTO((char [], char[])); extern void warn PROTO((char [])); /* report a warning */ extern void yyerror PROTO((char [])); /* report a parse error */ extern int yyparse PROTO((void)); /* the YACC parser */ /* from file scan.l */ /* The Flex-generated scanner for flex. */ extern int flexscan PROTO((void)); /* Open the given file (if NULL, stdin) for scanning. */ extern void set_input_file PROTO((char*)); /* Wrapup a file in the lexical analyzer. */ extern int yywrap PROTO((void)); /* from file sym.c */ /* Add symbol and definitions to symbol table. */ extern int addsym PROTO((register char[], char*, int, hash_table, int)); /* Save the text of a character class. */ extern void cclinstal PROTO ((Char [], int)); /* Lookup the number associated with character class. */ extern int ccllookup PROTO((Char [])); /* Find symbol in symbol table. */ extern struct hash_entry *findsym PROTO((register char[], hash_table, int )); extern void ndinstal PROTO((char[], Char[])); /* install a name definition */ extern Char *ndlookup PROTO((char[])); /* lookup a name definition */ /* Increase maximum number of SC's. */ extern void scextend PROTO((void)); extern void scinstal PROTO((char[], int)); /* make a start condition */ /* Lookup the number associated with a start condition. */ extern int sclookup PROTO((char[])); /* from file tblcmp.c */ /* Build table entries for dfa state. */ extern void bldtbl PROTO((int[], int, int, int, int)); extern void cmptmps PROTO((void)); /* compress template table entries */ extern void expand_nxt_chk PROTO((void)); /* increase nxt/chk arrays */ /* Finds a space in the table for a state to be placed. */ extern int find_table_space PROTO((int*, int)); extern void inittbl PROTO((void)); /* initialize transition tables */ /* Make the default, "jam" table entries. */ extern void mkdeftbl PROTO((void)); /* Create table entries for a state (or state fragment) which has * only one out-transition. */ extern void mk1tbl PROTO((int, int, int, int)); /* Place a state into full speed transition table. */ extern void place_state PROTO((int*, int, int)); /* Save states with only one out-transition to be processed later. */ extern void stack1 PROTO((int, int, int, int)); /* from file yylex.c */ extern int yylex PROTO((void)); flex-2.5.4/version.h100444 75424 12 35 6215400266 12623 0ustar kwzhstaff#define FLEX_VERSION "2.5.4" flex-2.5.4/ccl.c100444 75424 12 7226 6366531461 11754 0ustar kwzhstaff/* ccl - routines for character classes */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/ccl.c,v 2.9 93/09/16 20:32:14 vern Exp $ */ #include "flexdef.h" /* ccladd - add a single character to a ccl */ void ccladd( cclp, ch ) int cclp; int ch; { int ind, len, newpos, i; check_char( ch ); len = ccllen[cclp]; ind = cclmap[cclp]; /* check to see if the character is already in the ccl */ for ( i = 0; i < len; ++i ) if ( ccltbl[ind + i] == ch ) return; newpos = ind + len; if ( newpos >= current_max_ccl_tbl_size ) { current_max_ccl_tbl_size += MAX_CCL_TBL_SIZE_INCREMENT; ++num_reallocs; ccltbl = reallocate_Character_array( ccltbl, current_max_ccl_tbl_size ); } ccllen[cclp] = len + 1; ccltbl[newpos] = ch; } /* cclinit - return an empty ccl */ int cclinit() { if ( ++lastccl >= current_maxccls ) { current_maxccls += MAX_CCLS_INCREMENT; ++num_reallocs; cclmap = reallocate_integer_array( cclmap, current_maxccls ); ccllen = reallocate_integer_array( ccllen, current_maxccls ); cclng = reallocate_integer_array( cclng, current_maxccls ); } if ( lastccl == 1 ) /* we're making the first ccl */ cclmap[lastccl] = 0; else /* The new pointer is just past the end of the last ccl. * Since the cclmap points to the \first/ character of a * ccl, adding the length of the ccl to the cclmap pointer * will produce a cursor to the first free space. */ cclmap[lastccl] = cclmap[lastccl - 1] + ccllen[lastccl - 1]; ccllen[lastccl] = 0; cclng[lastccl] = 0; /* ccl's start out life un-negated */ return lastccl; } /* cclnegate - negate the given ccl */ void cclnegate( cclp ) int cclp; { cclng[cclp] = 1; } /* list_character_set - list the members of a set of characters in CCL form * * Writes to the given file a character-class representation of those * characters present in the given CCL. A character is present if it * has a non-zero value in the cset array. */ void list_character_set( file, cset ) FILE *file; int cset[]; { register int i; putc( '[', file ); for ( i = 0; i < csize; ++i ) { if ( cset[i] ) { register int start_char = i; putc( ' ', file ); fputs( readable_form( i ), file ); while ( ++i < csize && cset[i] ) ; if ( i - 1 > start_char ) /* this was a run */ fprintf( file, "-%s", readable_form( i - 1 ) ); putc( ' ', file ); } } putc( ']', file ); } flex-2.5.4/dfa.c100444 75424 12 62551 6366531462 11770 0ustar kwzhstaff/* dfa - DFA construction routines */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/dfa.c,v 2.26 95/04/20 13:53:14 vern Exp $ */ #include "flexdef.h" /* declare functions that have forward references */ void dump_associated_rules PROTO((FILE*, int)); void dump_transitions PROTO((FILE*, int[])); void sympartition PROTO((int[], int, int[], int[])); int symfollowset PROTO((int[], int, int, int[])); /* check_for_backing_up - check a DFA state for backing up * * synopsis * void check_for_backing_up( int ds, int state[numecs] ); * * ds is the number of the state to check and state[] is its out-transitions, * indexed by equivalence class. */ void check_for_backing_up( ds, state ) int ds; int state[]; { if ( (reject && ! dfaacc[ds].dfaacc_set) || (! reject && ! dfaacc[ds].dfaacc_state) ) { /* state is non-accepting */ ++num_backing_up; if ( backing_up_report ) { fprintf( backing_up_file, _( "State #%d is non-accepting -\n" ), ds ); /* identify the state */ dump_associated_rules( backing_up_file, ds ); /* Now identify it further using the out- and * jam-transitions. */ dump_transitions( backing_up_file, state ); putc( '\n', backing_up_file ); } } } /* check_trailing_context - check to see if NFA state set constitutes * "dangerous" trailing context * * synopsis * void check_trailing_context( int nfa_states[num_states+1], int num_states, * int accset[nacc+1], int nacc ); * * NOTES * Trailing context is "dangerous" if both the head and the trailing * part are of variable size \and/ there's a DFA state which contains * both an accepting state for the head part of the rule and NFA states * which occur after the beginning of the trailing context. * * When such a rule is matched, it's impossible to tell if having been * in the DFA state indicates the beginning of the trailing context or * further-along scanning of the pattern. In these cases, a warning * message is issued. * * nfa_states[1 .. num_states] is the list of NFA states in the DFA. * accset[1 .. nacc] is the list of accepting numbers for the DFA state. */ void check_trailing_context( nfa_states, num_states, accset, nacc ) int *nfa_states, num_states; int *accset; int nacc; { register int i, j; for ( i = 1; i <= num_states; ++i ) { int ns = nfa_states[i]; register int type = state_type[ns]; register int ar = assoc_rule[ns]; if ( type == STATE_NORMAL || rule_type[ar] != RULE_VARIABLE ) { /* do nothing */ } else if ( type == STATE_TRAILING_CONTEXT ) { /* Potential trouble. Scan set of accepting numbers * for the one marking the end of the "head". We * assume that this looping will be fairly cheap * since it's rare that an accepting number set * is large. */ for ( j = 1; j <= nacc; ++j ) if ( accset[j] & YY_TRAILING_HEAD_MASK ) { line_warning( _( "dangerous trailing context" ), rule_linenum[ar] ); return; } } } } /* dump_associated_rules - list the rules associated with a DFA state * * Goes through the set of NFA states associated with the DFA and * extracts the first MAX_ASSOC_RULES unique rules, sorts them, * and writes a report to the given file. */ void dump_associated_rules( file, ds ) FILE *file; int ds; { register int i, j; register int num_associated_rules = 0; int rule_set[MAX_ASSOC_RULES + 1]; int *dset = dss[ds]; int size = dfasiz[ds]; for ( i = 1; i <= size; ++i ) { register int rule_num = rule_linenum[assoc_rule[dset[i]]]; for ( j = 1; j <= num_associated_rules; ++j ) if ( rule_num == rule_set[j] ) break; if ( j > num_associated_rules ) { /* new rule */ if ( num_associated_rules < MAX_ASSOC_RULES ) rule_set[++num_associated_rules] = rule_num; } } bubble( rule_set, num_associated_rules ); fprintf( file, _( " associated rule line numbers:" ) ); for ( i = 1; i <= num_associated_rules; ++i ) { if ( i % 8 == 1 ) putc( '\n', file ); fprintf( file, "\t%d", rule_set[i] ); } putc( '\n', file ); } /* dump_transitions - list the transitions associated with a DFA state * * synopsis * dump_transitions( FILE *file, int state[numecs] ); * * Goes through the set of out-transitions and lists them in human-readable * form (i.e., not as equivalence classes); also lists jam transitions * (i.e., all those which are not out-transitions, plus EOF). The dump * is done to the given file. */ void dump_transitions( file, state ) FILE *file; int state[]; { register int i, ec; int out_char_set[CSIZE]; for ( i = 0; i < csize; ++i ) { ec = ABS( ecgroup[i] ); out_char_set[i] = state[ec]; } fprintf( file, _( " out-transitions: " ) ); list_character_set( file, out_char_set ); /* now invert the members of the set to get the jam transitions */ for ( i = 0; i < csize; ++i ) out_char_set[i] = ! out_char_set[i]; fprintf( file, _( "\n jam-transitions: EOF " ) ); list_character_set( file, out_char_set ); putc( '\n', file ); } /* epsclosure - construct the epsilon closure of a set of ndfa states * * synopsis * int *epsclosure( int t[num_states], int *numstates_addr, * int accset[num_rules+1], int *nacc_addr, * int *hashval_addr ); * * NOTES * The epsilon closure is the set of all states reachable by an arbitrary * number of epsilon transitions, which themselves do not have epsilon * transitions going out, unioned with the set of states which have non-null * accepting numbers. t is an array of size numstates of nfa state numbers. * Upon return, t holds the epsilon closure and *numstates_addr is updated. * accset holds a list of the accepting numbers, and the size of accset is * given by *nacc_addr. t may be subjected to reallocation if it is not * large enough to hold the epsilon closure. * * hashval is the hash value for the dfa corresponding to the state set. */ int *epsclosure( t, ns_addr, accset, nacc_addr, hv_addr ) int *t, *ns_addr, accset[], *nacc_addr, *hv_addr; { register int stkpos, ns, tsp; int numstates = *ns_addr, nacc, hashval, transsym, nfaccnum; int stkend, nstate; static int did_stk_init = false, *stk; #define MARK_STATE(state) \ trans1[state] = trans1[state] - MARKER_DIFFERENCE; #define IS_MARKED(state) (trans1[state] < 0) #define UNMARK_STATE(state) \ trans1[state] = trans1[state] + MARKER_DIFFERENCE; #define CHECK_ACCEPT(state) \ { \ nfaccnum = accptnum[state]; \ if ( nfaccnum != NIL ) \ accset[++nacc] = nfaccnum; \ } #define DO_REALLOCATION \ { \ current_max_dfa_size += MAX_DFA_SIZE_INCREMENT; \ ++num_reallocs; \ t = reallocate_integer_array( t, current_max_dfa_size ); \ stk = reallocate_integer_array( stk, current_max_dfa_size ); \ } \ #define PUT_ON_STACK(state) \ { \ if ( ++stkend >= current_max_dfa_size ) \ DO_REALLOCATION \ stk[stkend] = state; \ MARK_STATE(state) \ } #define ADD_STATE(state) \ { \ if ( ++numstates >= current_max_dfa_size ) \ DO_REALLOCATION \ t[numstates] = state; \ hashval += state; \ } #define STACK_STATE(state) \ { \ PUT_ON_STACK(state) \ CHECK_ACCEPT(state) \ if ( nfaccnum != NIL || transchar[state] != SYM_EPSILON ) \ ADD_STATE(state) \ } if ( ! did_stk_init ) { stk = allocate_integer_array( current_max_dfa_size ); did_stk_init = true; } nacc = stkend = hashval = 0; for ( nstate = 1; nstate <= numstates; ++nstate ) { ns = t[nstate]; /* The state could be marked if we've already pushed it onto * the stack. */ if ( ! IS_MARKED(ns) ) { PUT_ON_STACK(ns) CHECK_ACCEPT(ns) hashval += ns; } } for ( stkpos = 1; stkpos <= stkend; ++stkpos ) { ns = stk[stkpos]; transsym = transchar[ns]; if ( transsym == SYM_EPSILON ) { tsp = trans1[ns] + MARKER_DIFFERENCE; if ( tsp != NO_TRANSITION ) { if ( ! IS_MARKED(tsp) ) STACK_STATE(tsp) tsp = trans2[ns]; if ( tsp != NO_TRANSITION && ! IS_MARKED(tsp) ) STACK_STATE(tsp) } } } /* Clear out "visit" markers. */ for ( stkpos = 1; stkpos <= stkend; ++stkpos ) { if ( IS_MARKED(stk[stkpos]) ) UNMARK_STATE(stk[stkpos]) else flexfatal( _( "consistency check failed in epsclosure()" ) ); } *ns_addr = numstates; *hv_addr = hashval; *nacc_addr = nacc; return t; } /* increase_max_dfas - increase the maximum number of DFAs */ void increase_max_dfas() { current_max_dfas += MAX_DFAS_INCREMENT; ++num_reallocs; base = reallocate_integer_array( base, current_max_dfas ); def = reallocate_integer_array( def, current_max_dfas ); dfasiz = reallocate_integer_array( dfasiz, current_max_dfas ); accsiz = reallocate_integer_array( accsiz, current_max_dfas ); dhash = reallocate_integer_array( dhash, current_max_dfas ); dss = reallocate_int_ptr_array( dss, current_max_dfas ); dfaacc = reallocate_dfaacc_union( dfaacc, current_max_dfas ); if ( nultrans ) nultrans = reallocate_integer_array( nultrans, current_max_dfas ); } /* ntod - convert an ndfa to a dfa * * Creates the dfa corresponding to the ndfa we've constructed. The * dfa starts out in state #1. */ void ntod() { int *accset, ds, nacc, newds; int sym, hashval, numstates, dsize; int num_full_table_rows; /* used only for -f */ int *nset, *dset; int targptr, totaltrans, i, comstate, comfreq, targ; int symlist[CSIZE + 1]; int num_start_states; int todo_head, todo_next; /* Note that the following are indexed by *equivalence classes* * and not by characters. Since equivalence classes are indexed * beginning with 1, even if the scanner accepts NUL's, this * means that (since every character is potentially in its own * equivalence class) these arrays must have room for indices * from 1 to CSIZE, so their size must be CSIZE + 1. */ int duplist[CSIZE + 1], state[CSIZE + 1]; int targfreq[CSIZE + 1], targstate[CSIZE + 1]; accset = allocate_integer_array( num_rules + 1 ); nset = allocate_integer_array( current_max_dfa_size ); /* The "todo" queue is represented by the head, which is the DFA * state currently being processed, and the "next", which is the * next DFA state number available (not in use). We depend on the * fact that snstods() returns DFA's \in increasing order/, and thus * need only know the bounds of the dfas to be processed. */ todo_head = todo_next = 0; for ( i = 0; i <= csize; ++i ) { duplist[i] = NIL; symlist[i] = false; } for ( i = 0; i <= num_rules; ++i ) accset[i] = NIL; if ( trace ) { dumpnfa( scset[1] ); fputs( _( "\n\nDFA Dump:\n\n" ), stderr ); } inittbl(); /* Check to see whether we should build a separate table for * transitions on NUL characters. We don't do this for full-speed * (-F) scanners, since for them we don't have a simple state * number lying around with which to index the table. We also * don't bother doing it for scanners unless (1) NUL is in its own * equivalence class (indicated by a positive value of * ecgroup[NUL]), (2) NUL's equivalence class is the last * equivalence class, and (3) the number of equivalence classes is * the same as the number of characters. This latter case comes * about when useecs is false or when it's true but every character * still manages to land in its own class (unlikely, but it's * cheap to check for). If all these things are true then the * character code needed to represent NUL's equivalence class for * indexing the tables is going to take one more bit than the * number of characters, and therefore we won't be assured of * being able to fit it into a YY_CHAR variable. This rules out * storing the transitions in a compressed table, since the code * for interpreting them uses a YY_CHAR variable (perhaps it * should just use an integer, though; this is worth pondering ... * ###). * * Finally, for full tables, we want the number of entries in the * table to be a power of two so the array references go fast (it * will just take a shift to compute the major index). If * encoding NUL's transitions in the table will spoil this, we * give it its own table (note that this will be the case if we're * not using equivalence classes). */ /* Note that the test for ecgroup[0] == numecs below accomplishes * both (1) and (2) above */ if ( ! fullspd && ecgroup[0] == numecs ) { /* NUL is alone in its equivalence class, which is the * last one. */ int use_NUL_table = (numecs == csize); if ( fulltbl && ! use_NUL_table ) { /* We still may want to use the table if numecs * is a power of 2. */ int power_of_two; for ( power_of_two = 1; power_of_two <= csize; power_of_two *= 2 ) if ( numecs == power_of_two ) { use_NUL_table = true; break; } } if ( use_NUL_table ) nultrans = allocate_integer_array( current_max_dfas ); /* From now on, nultrans != nil indicates that we're * saving null transitions for later, separate encoding. */ } if ( fullspd ) { for ( i = 0; i <= numecs; ++i ) state[i] = 0; place_state( state, 0, 0 ); dfaacc[0].dfaacc_state = 0; } else if ( fulltbl ) { if ( nultrans ) /* We won't be including NUL's transitions in the * table, so build it for entries from 0 .. numecs - 1. */ num_full_table_rows = numecs; else /* Take into account the fact that we'll be including * the NUL entries in the transition table. Build it * from 0 .. numecs. */ num_full_table_rows = numecs + 1; /* Unless -Ca, declare it "short" because it's a real * long-shot that that won't be large enough. */ out_str_dec( "static yyconst %s yy_nxt[][%d] =\n {\n", /* '}' so vi doesn't get too confused */ long_align ? "long" : "short", num_full_table_rows ); outn( " {" ); /* Generate 0 entries for state #0. */ for ( i = 0; i < num_full_table_rows; ++i ) mk2data( 0 ); dataflush(); outn( " },\n" ); } /* Create the first states. */ num_start_states = lastsc * 2; for ( i = 1; i <= num_start_states; ++i ) { numstates = 1; /* For each start condition, make one state for the case when * we're at the beginning of the line (the '^' operator) and * one for the case when we're not. */ if ( i % 2 == 1 ) nset[numstates] = scset[(i / 2) + 1]; else nset[numstates] = mkbranch( scbol[i / 2], scset[i / 2] ); nset = epsclosure( nset, &numstates, accset, &nacc, &hashval ); if ( snstods( nset, numstates, accset, nacc, hashval, &ds ) ) { numas += nacc; totnst += numstates; ++todo_next; if ( variable_trailing_context_rules && nacc > 0 ) check_trailing_context( nset, numstates, accset, nacc ); } } if ( ! fullspd ) { if ( ! snstods( nset, 0, accset, 0, 0, &end_of_buffer_state ) ) flexfatal( _( "could not create unique end-of-buffer state" ) ); ++numas; ++num_start_states; ++todo_next; } while ( todo_head < todo_next ) { targptr = 0; totaltrans = 0; for ( i = 1; i <= numecs; ++i ) state[i] = 0; ds = ++todo_head; dset = dss[ds]; dsize = dfasiz[ds]; if ( trace ) fprintf( stderr, _( "state # %d:\n" ), ds ); sympartition( dset, dsize, symlist, duplist ); for ( sym = 1; sym <= numecs; ++sym ) { if ( symlist[sym] ) { symlist[sym] = 0; if ( duplist[sym] == NIL ) { /* Symbol has unique out-transitions. */ numstates = symfollowset( dset, dsize, sym, nset ); nset = epsclosure( nset, &numstates, accset, &nacc, &hashval ); if ( snstods( nset, numstates, accset, nacc, hashval, &newds ) ) { totnst = totnst + numstates; ++todo_next; numas += nacc; if ( variable_trailing_context_rules && nacc > 0 ) check_trailing_context( nset, numstates, accset, nacc ); } state[sym] = newds; if ( trace ) fprintf( stderr, "\t%d\t%d\n", sym, newds ); targfreq[++targptr] = 1; targstate[targptr] = newds; ++numuniq; } else { /* sym's equivalence class has the same * transitions as duplist(sym)'s * equivalence class. */ targ = state[duplist[sym]]; state[sym] = targ; if ( trace ) fprintf( stderr, "\t%d\t%d\n", sym, targ ); /* Update frequency count for * destination state. */ i = 0; while ( targstate[++i] != targ ) ; ++targfreq[i]; ++numdup; } ++totaltrans; duplist[sym] = NIL; } } if ( caseins && ! useecs ) { register int j; for ( i = 'A', j = 'a'; i <= 'Z'; ++i, ++j ) { if ( state[i] == 0 && state[j] != 0 ) /* We're adding a transition. */ ++totaltrans; else if ( state[i] != 0 && state[j] == 0 ) /* We're taking away a transition. */ --totaltrans; state[i] = state[j]; } } numsnpairs += totaltrans; if ( ds > num_start_states ) check_for_backing_up( ds, state ); if ( nultrans ) { nultrans[ds] = state[NUL_ec]; state[NUL_ec] = 0; /* remove transition */ } if ( fulltbl ) { outn( " {" ); /* Supply array's 0-element. */ if ( ds == end_of_buffer_state ) mk2data( -end_of_buffer_state ); else mk2data( end_of_buffer_state ); for ( i = 1; i < num_full_table_rows; ++i ) /* Jams are marked by negative of state * number. */ mk2data( state[i] ? state[i] : -ds ); dataflush(); outn( " },\n" ); } else if ( fullspd ) place_state( state, ds, totaltrans ); else if ( ds == end_of_buffer_state ) /* Special case this state to make sure it does what * it's supposed to, i.e., jam on end-of-buffer. */ stack1( ds, 0, 0, JAMSTATE ); else /* normal, compressed state */ { /* Determine which destination state is the most * common, and how many transitions to it there are. */ comfreq = 0; comstate = 0; for ( i = 1; i <= targptr; ++i ) if ( targfreq[i] > comfreq ) { comfreq = targfreq[i]; comstate = targstate[i]; } bldtbl( state, ds, totaltrans, comstate, comfreq ); } } if ( fulltbl ) dataend(); else if ( ! fullspd ) { cmptmps(); /* create compressed template entries */ /* Create tables for all the states with only one * out-transition. */ while ( onesp > 0 ) { mk1tbl( onestate[onesp], onesym[onesp], onenext[onesp], onedef[onesp] ); --onesp; } mkdeftbl(); } flex_free( (void *) accset ); flex_free( (void *) nset ); } /* snstods - converts a set of ndfa states into a dfa state * * synopsis * is_new_state = snstods( int sns[numstates], int numstates, * int accset[num_rules+1], int nacc, * int hashval, int *newds_addr ); * * On return, the dfa state number is in newds. */ int snstods( sns, numstates, accset, nacc, hashval, newds_addr ) int sns[], numstates, accset[], nacc, hashval, *newds_addr; { int didsort = 0; register int i, j; int newds, *oldsns; for ( i = 1; i <= lastdfa; ++i ) if ( hashval == dhash[i] ) { if ( numstates == dfasiz[i] ) { oldsns = dss[i]; if ( ! didsort ) { /* We sort the states in sns so we * can compare it to oldsns quickly. * We use bubble because there probably * aren't very many states. */ bubble( sns, numstates ); didsort = 1; } for ( j = 1; j <= numstates; ++j ) if ( sns[j] != oldsns[j] ) break; if ( j > numstates ) { ++dfaeql; *newds_addr = i; return 0; } ++hshcol; } else ++hshsave; } /* Make a new dfa. */ if ( ++lastdfa >= current_max_dfas ) increase_max_dfas(); newds = lastdfa; dss[newds] = allocate_integer_array( numstates + 1 ); /* If we haven't already sorted the states in sns, we do so now, * so that future comparisons with it can be made quickly. */ if ( ! didsort ) bubble( sns, numstates ); for ( i = 1; i <= numstates; ++i ) dss[newds][i] = sns[i]; dfasiz[newds] = numstates; dhash[newds] = hashval; if ( nacc == 0 ) { if ( reject ) dfaacc[newds].dfaacc_set = (int *) 0; else dfaacc[newds].dfaacc_state = 0; accsiz[newds] = 0; } else if ( reject ) { /* We sort the accepting set in increasing order so the * disambiguating rule that the first rule listed is considered * match in the event of ties will work. We use a bubble * sort since the list is probably quite small. */ bubble( accset, nacc ); dfaacc[newds].dfaacc_set = allocate_integer_array( nacc + 1 ); /* Save the accepting set for later */ for ( i = 1; i <= nacc; ++i ) { dfaacc[newds].dfaacc_set[i] = accset[i]; if ( accset[i] <= num_rules ) /* Who knows, perhaps a REJECT can yield * this rule. */ rule_useful[accset[i]] = true; } accsiz[newds] = nacc; } else { /* Find lowest numbered rule so the disambiguating rule * will work. */ j = num_rules + 1; for ( i = 1; i <= nacc; ++i ) if ( accset[i] < j ) j = accset[i]; dfaacc[newds].dfaacc_state = j; if ( j <= num_rules ) rule_useful[j] = true; } *newds_addr = newds; return 1; } /* symfollowset - follow the symbol transitions one step * * synopsis * numstates = symfollowset( int ds[current_max_dfa_size], int dsize, * int transsym, int nset[current_max_dfa_size] ); */ int symfollowset( ds, dsize, transsym, nset ) int ds[], dsize, transsym, nset[]; { int ns, tsp, sym, i, j, lenccl, ch, numstates, ccllist; numstates = 0; for ( i = 1; i <= dsize; ++i ) { /* for each nfa state ns in the state set of ds */ ns = ds[i]; sym = transchar[ns]; tsp = trans1[ns]; if ( sym < 0 ) { /* it's a character class */ sym = -sym; ccllist = cclmap[sym]; lenccl = ccllen[sym]; if ( cclng[sym] ) { for ( j = 0; j < lenccl; ++j ) { /* Loop through negated character * class. */ ch = ccltbl[ccllist + j]; if ( ch == 0 ) ch = NUL_ec; if ( ch > transsym ) /* Transsym isn't in negated * ccl. */ break; else if ( ch == transsym ) /* next 2 */ goto bottom; } /* Didn't find transsym in ccl. */ nset[++numstates] = tsp; } else for ( j = 0; j < lenccl; ++j ) { ch = ccltbl[ccllist + j]; if ( ch == 0 ) ch = NUL_ec; if ( ch > transsym ) break; else if ( ch == transsym ) { nset[++numstates] = tsp; break; } } } else if ( sym >= 'A' && sym <= 'Z' && caseins ) flexfatal( _( "consistency check failed in symfollowset" ) ); else if ( sym == SYM_EPSILON ) { /* do nothing */ } else if ( ABS( ecgroup[sym] ) == transsym ) nset[++numstates] = tsp; bottom: ; } return numstates; } /* sympartition - partition characters with same out-transitions * * synopsis * sympartition( int ds[current_max_dfa_size], int numstates, * int symlist[numecs], int duplist[numecs] ); */ void sympartition( ds, numstates, symlist, duplist ) int ds[], numstates; int symlist[], duplist[]; { int tch, i, j, k, ns, dupfwd[CSIZE + 1], lenccl, cclp, ich; /* Partitioning is done by creating equivalence classes for those * characters which have out-transitions from the given state. Thus * we are really creating equivalence classes of equivalence classes. */ for ( i = 1; i <= numecs; ++i ) { /* initialize equivalence class list */ duplist[i] = i - 1; dupfwd[i] = i + 1; } duplist[1] = NIL; dupfwd[numecs] = NIL; for ( i = 1; i <= numstates; ++i ) { ns = ds[i]; tch = transchar[ns]; if ( tch != SYM_EPSILON ) { if ( tch < -lastccl || tch >= csize ) { flexfatal( _( "bad transition character detected in sympartition()" ) ); } if ( tch >= 0 ) { /* character transition */ int ec = ecgroup[tch]; mkechar( ec, dupfwd, duplist ); symlist[ec] = 1; } else { /* character class */ tch = -tch; lenccl = ccllen[tch]; cclp = cclmap[tch]; mkeccl( ccltbl + cclp, lenccl, dupfwd, duplist, numecs, NUL_ec ); if ( cclng[tch] ) { j = 0; for ( k = 0; k < lenccl; ++k ) { ich = ccltbl[cclp + k]; if ( ich == 0 ) ich = NUL_ec; for ( ++j; j < ich; ++j ) symlist[j] = 1; } for ( ++j; j <= numecs; ++j ) symlist[j] = 1; } else for ( k = 0; k < lenccl; ++k ) { ich = ccltbl[cclp + k]; if ( ich == 0 ) ich = NUL_ec; symlist[ich] = 1; } } } } } flex-2.5.4/ecs.c100444 75424 12 13057 6366531462 12005 0ustar kwzhstaff/* ecs - equivalence class routines */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/ecs.c,v 2.9 93/12/07 10:18:20 vern Exp $ */ #include "flexdef.h" /* ccl2ecl - convert character classes to set of equivalence classes */ void ccl2ecl() { int i, ich, newlen, cclp, ccls, cclmec; for ( i = 1; i <= lastccl; ++i ) { /* We loop through each character class, and for each character * in the class, add the character's equivalence class to the * new "character" class we are creating. Thus when we are all * done, character classes will really consist of collections * of equivalence classes */ newlen = 0; cclp = cclmap[i]; for ( ccls = 0; ccls < ccllen[i]; ++ccls ) { ich = ccltbl[cclp + ccls]; cclmec = ecgroup[ich]; if ( cclmec > 0 ) { ccltbl[cclp + newlen] = cclmec; ++newlen; } } ccllen[i] = newlen; } } /* cre8ecs - associate equivalence class numbers with class members * * fwd is the forward linked-list of equivalence class members. bck * is the backward linked-list, and num is the number of class members. * * Returned is the number of classes. */ int cre8ecs( fwd, bck, num ) int fwd[], bck[], num; { int i, j, numcl; numcl = 0; /* Create equivalence class numbers. From now on, ABS( bck(x) ) * is the equivalence class number for object x. If bck(x) * is positive, then x is the representative of its equivalence * class. */ for ( i = 1; i <= num; ++i ) if ( bck[i] == NIL ) { bck[i] = ++numcl; for ( j = fwd[i]; j != NIL; j = fwd[j] ) bck[j] = -numcl; } return numcl; } /* mkeccl - update equivalence classes based on character class xtions * * synopsis * Char ccls[]; * int lenccl, fwd[llsiz], bck[llsiz], llsiz, NUL_mapping; * void mkeccl( Char ccls[], int lenccl, int fwd[llsiz], int bck[llsiz], * int llsiz, int NUL_mapping ); * * ccls contains the elements of the character class, lenccl is the * number of elements in the ccl, fwd is the forward link-list of equivalent * characters, bck is the backward link-list, and llsiz size of the link-list. * * NUL_mapping is the value which NUL (0) should be mapped to. */ void mkeccl( ccls, lenccl, fwd, bck, llsiz, NUL_mapping ) Char ccls[]; int lenccl, fwd[], bck[], llsiz, NUL_mapping; { int cclp, oldec, newec; int cclm, i, j; static unsigned char cclflags[CSIZE]; /* initialized to all '\0' */ /* Note that it doesn't matter whether or not the character class is * negated. The same results will be obtained in either case. */ cclp = 0; while ( cclp < lenccl ) { cclm = ccls[cclp]; if ( NUL_mapping && cclm == 0 ) cclm = NUL_mapping; oldec = bck[cclm]; newec = cclm; j = cclp + 1; for ( i = fwd[cclm]; i != NIL && i <= llsiz; i = fwd[i] ) { /* look for the symbol in the character class */ for ( ; j < lenccl; ++j ) { register int ccl_char; if ( NUL_mapping && ccls[j] == 0 ) ccl_char = NUL_mapping; else ccl_char = ccls[j]; if ( ccl_char > i ) break; if ( ccl_char == i && ! cclflags[j] ) { /* We found an old companion of cclm * in the ccl. Link it into the new * equivalence class and flag it as * having been processed. */ bck[i] = newec; fwd[newec] = i; newec = i; /* Set flag so we don't reprocess. */ cclflags[j] = 1; /* Get next equivalence class member. */ /* continue 2 */ goto next_pt; } } /* Symbol isn't in character class. Put it in the old * equivalence class. */ bck[i] = oldec; if ( oldec != NIL ) fwd[oldec] = i; oldec = i; next_pt: ; } if ( bck[cclm] != NIL || oldec != bck[cclm] ) { bck[cclm] = NIL; fwd[oldec] = NIL; } fwd[newec] = NIL; /* Find next ccl member to process. */ for ( ++cclp; cclflags[cclp] && cclp < lenccl; ++cclp ) { /* Reset "doesn't need processing" flag. */ cclflags[cclp] = 0; } } } /* mkechar - create equivalence class for single character */ void mkechar( tch, fwd, bck ) int tch, fwd[], bck[]; { /* If until now the character has been a proper subset of * an equivalence class, break it away to create a new ec */ if ( fwd[tch] != NIL ) bck[fwd[tch]] = bck[tch]; if ( bck[tch] != NIL ) fwd[bck[tch]] = fwd[tch]; fwd[tch] = NIL; bck[tch] = NIL; } flex-2.5.4/gen.c100444 75424 12 110445 6366531462 12023 0ustar kwzhstaff/* gen - actual generation (writing) of flex scanners */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/gen.c,v 2.56 96/05/25 20:43:38 vern Exp $ */ #include "flexdef.h" /* declare functions that have forward references */ void gen_next_state PROTO((int)); void genecs PROTO((void)); void indent_put2s PROTO((char [], char [])); void indent_puts PROTO((char [])); static int indent_level = 0; /* each level is 8 spaces */ #define indent_up() (++indent_level) #define indent_down() (--indent_level) #define set_indent(indent_val) indent_level = indent_val /* Almost everything is done in terms of arrays starting at 1, so provide * a null entry for the zero element of all C arrays. (The exception * to this is that the fast table representation generally uses the * 0 elements of its arrays, too.) */ static char C_int_decl[] = "static yyconst int %s[%d] =\n { 0,\n"; static char C_short_decl[] = "static yyconst short int %s[%d] =\n { 0,\n"; static char C_long_decl[] = "static yyconst long int %s[%d] =\n { 0,\n"; static char C_state_decl[] = "static yyconst yy_state_type %s[%d] =\n { 0,\n"; /* Indent to the current level. */ void do_indent() { register int i = indent_level * 8; while ( i >= 8 ) { outc( '\t' ); i -= 8; } while ( i > 0 ) { outc( ' ' ); --i; } } /* Generate the code to keep backing-up information. */ void gen_backing_up() { if ( reject || num_backing_up == 0 ) return; if ( fullspd ) indent_puts( "if ( yy_current_state[-1].yy_nxt )" ); else indent_puts( "if ( yy_accept[yy_current_state] )" ); indent_up(); indent_puts( "{" ); indent_puts( "yy_last_accepting_state = yy_current_state;" ); indent_puts( "yy_last_accepting_cpos = yy_cp;" ); indent_puts( "}" ); indent_down(); } /* Generate the code to perform the backing up. */ void gen_bu_action() { if ( reject || num_backing_up == 0 ) return; set_indent( 3 ); indent_puts( "case 0: /* must back up */" ); indent_puts( "/* undo the effects of YY_DO_BEFORE_ACTION */" ); indent_puts( "*yy_cp = yy_hold_char;" ); if ( fullspd || fulltbl ) indent_puts( "yy_cp = yy_last_accepting_cpos + 1;" ); else /* Backing-up info for compressed tables is taken \after/ * yy_cp has been incremented for the next state. */ indent_puts( "yy_cp = yy_last_accepting_cpos;" ); indent_puts( "yy_current_state = yy_last_accepting_state;" ); indent_puts( "goto yy_find_action;" ); outc( '\n' ); set_indent( 0 ); } /* genctbl - generates full speed compressed transition table */ void genctbl() { register int i; int end_of_buffer_action = num_rules + 1; /* Table of verify for transition and offset to next state. */ out_dec( "static yyconst struct yy_trans_info yy_transition[%d] =\n", tblend + numecs + 1 ); outn( " {" ); /* We want the transition to be represented as the offset to the * next state, not the actual state number, which is what it currently * is. The offset is base[nxt[i]] - (base of current state)]. That's * just the difference between the starting points of the two involved * states (to - from). * * First, though, we need to find some way to put in our end-of-buffer * flags and states. We do this by making a state with absolutely no * transitions. We put it at the end of the table. */ /* We need to have room in nxt/chk for two more slots: One for the * action and one for the end-of-buffer transition. We now *assume* * that we're guaranteed the only character we'll try to index this * nxt/chk pair with is EOB, i.e., 0, so we don't have to make sure * there's room for jam entries for other characters. */ while ( tblend + 2 >= current_max_xpairs ) expand_nxt_chk(); while ( lastdfa + 1 >= current_max_dfas ) increase_max_dfas(); base[lastdfa + 1] = tblend + 2; nxt[tblend + 1] = end_of_buffer_action; chk[tblend + 1] = numecs + 1; chk[tblend + 2] = 1; /* anything but EOB */ /* So that "make test" won't show arb. differences. */ nxt[tblend + 2] = 0; /* Make sure every state has an end-of-buffer transition and an * action #. */ for ( i = 0; i <= lastdfa; ++i ) { int anum = dfaacc[i].dfaacc_state; int offset = base[i]; chk[offset] = EOB_POSITION; chk[offset - 1] = ACTION_POSITION; nxt[offset - 1] = anum; /* action number */ } for ( i = 0; i <= tblend; ++i ) { if ( chk[i] == EOB_POSITION ) transition_struct_out( 0, base[lastdfa + 1] - i ); else if ( chk[i] == ACTION_POSITION ) transition_struct_out( 0, nxt[i] ); else if ( chk[i] > numecs || chk[i] == 0 ) transition_struct_out( 0, 0 ); /* unused slot */ else /* verify, transition */ transition_struct_out( chk[i], base[nxt[i]] - (i - chk[i]) ); } /* Here's the final, end-of-buffer state. */ transition_struct_out( chk[tblend + 1], nxt[tblend + 1] ); transition_struct_out( chk[tblend + 2], nxt[tblend + 2] ); outn( " };\n" ); /* Table of pointers to start states. */ out_dec( "static yyconst struct yy_trans_info *yy_start_state_list[%d] =\n", lastsc * 2 + 1 ); outn( " {" ); /* } so vi doesn't get confused */ for ( i = 0; i <= lastsc * 2; ++i ) out_dec( " &yy_transition[%d],\n", base[i] ); dataend(); if ( useecs ) genecs(); } /* Generate equivalence-class tables. */ void genecs() { register int i, j; int numrows; out_str_dec( C_int_decl, "yy_ec", csize ); for ( i = 1; i < csize; ++i ) { if ( caseins && (i >= 'A') && (i <= 'Z') ) ecgroup[i] = ecgroup[clower( i )]; ecgroup[i] = ABS( ecgroup[i] ); mkdata( ecgroup[i] ); } dataend(); if ( trace ) { fputs( _( "\n\nEquivalence Classes:\n\n" ), stderr ); numrows = csize / 8; for ( j = 0; j < numrows; ++j ) { for ( i = j; i < csize; i = i + numrows ) { fprintf( stderr, "%4s = %-2d", readable_form( i ), ecgroup[i] ); putc( ' ', stderr ); } putc( '\n', stderr ); } } } /* Generate the code to find the action number. */ void gen_find_action() { if ( fullspd ) indent_puts( "yy_act = yy_current_state[-1].yy_nxt;" ); else if ( fulltbl ) indent_puts( "yy_act = yy_accept[yy_current_state];" ); else if ( reject ) { indent_puts( "yy_current_state = *--yy_state_ptr;" ); indent_puts( "yy_lp = yy_accept[yy_current_state];" ); outn( "find_rule: /* we branch to this label when backing up */" ); indent_puts( "for ( ; ; ) /* until we find what rule we matched */" ); indent_up(); indent_puts( "{" ); indent_puts( "if ( yy_lp && yy_lp < yy_accept[yy_current_state + 1] )" ); indent_up(); indent_puts( "{" ); indent_puts( "yy_act = yy_acclist[yy_lp];" ); if ( variable_trailing_context_rules ) { indent_puts( "if ( yy_act & YY_TRAILING_HEAD_MASK ||" ); indent_puts( " yy_looking_for_trail_begin )" ); indent_up(); indent_puts( "{" ); indent_puts( "if ( yy_act == yy_looking_for_trail_begin )" ); indent_up(); indent_puts( "{" ); indent_puts( "yy_looking_for_trail_begin = 0;" ); indent_puts( "yy_act &= ~YY_TRAILING_HEAD_MASK;" ); indent_puts( "break;" ); indent_puts( "}" ); indent_down(); indent_puts( "}" ); indent_down(); indent_puts( "else if ( yy_act & YY_TRAILING_MASK )" ); indent_up(); indent_puts( "{" ); indent_puts( "yy_looking_for_trail_begin = yy_act & ~YY_TRAILING_MASK;" ); indent_puts( "yy_looking_for_trail_begin |= YY_TRAILING_HEAD_MASK;" ); if ( real_reject ) { /* Remember matched text in case we back up * due to REJECT. */ indent_puts( "yy_full_match = yy_cp;" ); indent_puts( "yy_full_state = yy_state_ptr;" ); indent_puts( "yy_full_lp = yy_lp;" ); } indent_puts( "}" ); indent_down(); indent_puts( "else" ); indent_up(); indent_puts( "{" ); indent_puts( "yy_full_match = yy_cp;" ); indent_puts( "yy_full_state = yy_state_ptr;" ); indent_puts( "yy_full_lp = yy_lp;" ); indent_puts( "break;" ); indent_puts( "}" ); indent_down(); indent_puts( "++yy_lp;" ); indent_puts( "goto find_rule;" ); } else { /* Remember matched text in case we back up due to * trailing context plus REJECT. */ indent_up(); indent_puts( "{" ); indent_puts( "yy_full_match = yy_cp;" ); indent_puts( "break;" ); indent_puts( "}" ); indent_down(); } indent_puts( "}" ); indent_down(); indent_puts( "--yy_cp;" ); /* We could consolidate the following two lines with those at * the beginning, but at the cost of complaints that we're * branching inside a loop. */ indent_puts( "yy_current_state = *--yy_state_ptr;" ); indent_puts( "yy_lp = yy_accept[yy_current_state];" ); indent_puts( "}" ); indent_down(); } else { /* compressed */ indent_puts( "yy_act = yy_accept[yy_current_state];" ); if ( interactive && ! reject ) { /* Do the guaranteed-needed backing up to figure out * the match. */ indent_puts( "if ( yy_act == 0 )" ); indent_up(); indent_puts( "{ /* have to back up */" ); indent_puts( "yy_cp = yy_last_accepting_cpos;" ); indent_puts( "yy_current_state = yy_last_accepting_state;" ); indent_puts( "yy_act = yy_accept[yy_current_state];" ); indent_puts( "}" ); indent_down(); } } } /* genftbl - generate full transition table */ void genftbl() { register int i; int end_of_buffer_action = num_rules + 1; out_str_dec( long_align ? C_long_decl : C_short_decl, "yy_accept", lastdfa + 1 ); dfaacc[end_of_buffer_state].dfaacc_state = end_of_buffer_action; for ( i = 1; i <= lastdfa; ++i ) { register int anum = dfaacc[i].dfaacc_state; mkdata( anum ); if ( trace && anum ) fprintf( stderr, _( "state # %d accepts: [%d]\n" ), i, anum ); } dataend(); if ( useecs ) genecs(); /* Don't have to dump the actual full table entries - they were * created on-the-fly. */ } /* Generate the code to find the next compressed-table state. */ void gen_next_compressed_state( char_map ) char *char_map; { indent_put2s( "register YY_CHAR yy_c = %s;", char_map ); /* Save the backing-up info \before/ computing the next state * because we always compute one more state than needed - we * always proceed until we reach a jam state */ gen_backing_up(); indent_puts( "while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state )" ); indent_up(); indent_puts( "{" ); indent_puts( "yy_current_state = (int) yy_def[yy_current_state];" ); if ( usemecs ) { /* We've arrange it so that templates are never chained * to one another. This means we can afford to make a * very simple test to see if we need to convert to * yy_c's meta-equivalence class without worrying * about erroneously looking up the meta-equivalence * class twice */ do_indent(); /* lastdfa + 2 is the beginning of the templates */ out_dec( "if ( yy_current_state >= %d )\n", lastdfa + 2 ); indent_up(); indent_puts( "yy_c = yy_meta[(unsigned int) yy_c];" ); indent_down(); } indent_puts( "}" ); indent_down(); indent_puts( "yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c];" ); } /* Generate the code to find the next match. */ void gen_next_match() { /* NOTE - changes in here should be reflected in gen_next_state() and * gen_NUL_trans(). */ char *char_map = useecs ? "yy_ec[YY_SC_TO_UI(*yy_cp)]" : "YY_SC_TO_UI(*yy_cp)"; char *char_map_2 = useecs ? "yy_ec[YY_SC_TO_UI(*++yy_cp)]" : "YY_SC_TO_UI(*++yy_cp)"; if ( fulltbl ) { indent_put2s( "while ( (yy_current_state = yy_nxt[yy_current_state][%s]) > 0 )", char_map ); indent_up(); if ( num_backing_up > 0 ) { indent_puts( "{" ); /* } for vi */ gen_backing_up(); outc( '\n' ); } indent_puts( "++yy_cp;" ); if ( num_backing_up > 0 ) /* { for vi */ indent_puts( "}" ); indent_down(); outc( '\n' ); indent_puts( "yy_current_state = -yy_current_state;" ); } else if ( fullspd ) { indent_puts( "{" ); /* } for vi */ indent_puts( "register yyconst struct yy_trans_info *yy_trans_info;\n" ); indent_puts( "register YY_CHAR yy_c;\n" ); indent_put2s( "for ( yy_c = %s;", char_map ); indent_puts( " (yy_trans_info = &yy_current_state[(unsigned int) yy_c])->" ); indent_puts( "yy_verify == yy_c;" ); indent_put2s( " yy_c = %s )", char_map_2 ); indent_up(); if ( num_backing_up > 0 ) indent_puts( "{" ); /* } for vi */ indent_puts( "yy_current_state += yy_trans_info->yy_nxt;" ); if ( num_backing_up > 0 ) { outc( '\n' ); gen_backing_up(); /* { for vi */ indent_puts( "}" ); } indent_down(); /* { for vi */ indent_puts( "}" ); } else { /* compressed */ indent_puts( "do" ); indent_up(); indent_puts( "{" ); /* } for vi */ gen_next_state( false ); indent_puts( "++yy_cp;" ); /* { for vi */ indent_puts( "}" ); indent_down(); do_indent(); if ( interactive ) out_dec( "while ( yy_base[yy_current_state] != %d );\n", jambase ); else out_dec( "while ( yy_current_state != %d );\n", jamstate ); if ( ! reject && ! interactive ) { /* Do the guaranteed-needed backing up to figure out * the match. */ indent_puts( "yy_cp = yy_last_accepting_cpos;" ); indent_puts( "yy_current_state = yy_last_accepting_state;" ); } } } /* Generate the code to find the next state. */ void gen_next_state( worry_about_NULs ) int worry_about_NULs; { /* NOTE - changes in here should be reflected in gen_next_match() */ char char_map[256]; if ( worry_about_NULs && ! nultrans ) { if ( useecs ) (void) sprintf( char_map, "(*yy_cp ? yy_ec[YY_SC_TO_UI(*yy_cp)] : %d)", NUL_ec ); else (void) sprintf( char_map, "(*yy_cp ? YY_SC_TO_UI(*yy_cp) : %d)", NUL_ec ); } else strcpy( char_map, useecs ? "yy_ec[YY_SC_TO_UI(*yy_cp)]" : "YY_SC_TO_UI(*yy_cp)" ); if ( worry_about_NULs && nultrans ) { if ( ! fulltbl && ! fullspd ) /* Compressed tables back up *before* they match. */ gen_backing_up(); indent_puts( "if ( *yy_cp )" ); indent_up(); indent_puts( "{" ); /* } for vi */ } if ( fulltbl ) indent_put2s( "yy_current_state = yy_nxt[yy_current_state][%s];", char_map ); else if ( fullspd ) indent_put2s( "yy_current_state += yy_current_state[%s].yy_nxt;", char_map ); else gen_next_compressed_state( char_map ); if ( worry_about_NULs && nultrans ) { /* { for vi */ indent_puts( "}" ); indent_down(); indent_puts( "else" ); indent_up(); indent_puts( "yy_current_state = yy_NUL_trans[yy_current_state];" ); indent_down(); } if ( fullspd || fulltbl ) gen_backing_up(); if ( reject ) indent_puts( "*yy_state_ptr++ = yy_current_state;" ); } /* Generate the code to make a NUL transition. */ void gen_NUL_trans() { /* NOTE - changes in here should be reflected in gen_next_match() */ /* Only generate a definition for "yy_cp" if we'll generate code * that uses it. Otherwise lint and the like complain. */ int need_backing_up = (num_backing_up > 0 && ! reject); if ( need_backing_up && (! nultrans || fullspd || fulltbl) ) /* We're going to need yy_cp lying around for the call * below to gen_backing_up(). */ indent_puts( "register char *yy_cp = yy_c_buf_p;" ); outc( '\n' ); if ( nultrans ) { indent_puts( "yy_current_state = yy_NUL_trans[yy_current_state];" ); indent_puts( "yy_is_jam = (yy_current_state == 0);" ); } else if ( fulltbl ) { do_indent(); out_dec( "yy_current_state = yy_nxt[yy_current_state][%d];\n", NUL_ec ); indent_puts( "yy_is_jam = (yy_current_state <= 0);" ); } else if ( fullspd ) { do_indent(); out_dec( "register int yy_c = %d;\n", NUL_ec ); indent_puts( "register yyconst struct yy_trans_info *yy_trans_info;\n" ); indent_puts( "yy_trans_info = &yy_current_state[(unsigned int) yy_c];" ); indent_puts( "yy_current_state += yy_trans_info->yy_nxt;" ); indent_puts( "yy_is_jam = (yy_trans_info->yy_verify != yy_c);" ); } else { char NUL_ec_str[20]; (void) sprintf( NUL_ec_str, "%d", NUL_ec ); gen_next_compressed_state( NUL_ec_str ); do_indent(); out_dec( "yy_is_jam = (yy_current_state == %d);\n", jamstate ); if ( reject ) { /* Only stack this state if it's a transition we * actually make. If we stack it on a jam, then * the state stack and yy_c_buf_p get out of sync. */ indent_puts( "if ( ! yy_is_jam )" ); indent_up(); indent_puts( "*yy_state_ptr++ = yy_current_state;" ); indent_down(); } } /* If we've entered an accepting state, back up; note that * compressed tables have *already* done such backing up, so * we needn't bother with it again. */ if ( need_backing_up && (fullspd || fulltbl) ) { outc( '\n' ); indent_puts( "if ( ! yy_is_jam )" ); indent_up(); indent_puts( "{" ); gen_backing_up(); indent_puts( "}" ); indent_down(); } } /* Generate the code to find the start state. */ void gen_start_state() { if ( fullspd ) { if ( bol_needed ) { indent_puts( "yy_current_state = yy_start_state_list[yy_start + YY_AT_BOL()];" ); } else indent_puts( "yy_current_state = yy_start_state_list[yy_start];" ); } else { indent_puts( "yy_current_state = yy_start;" ); if ( bol_needed ) indent_puts( "yy_current_state += YY_AT_BOL();" ); if ( reject ) { /* Set up for storing up states. */ indent_puts( "yy_state_ptr = yy_state_buf;" ); indent_puts( "*yy_state_ptr++ = yy_current_state;" ); } } } /* gentabs - generate data statements for the transition tables */ void gentabs() { int i, j, k, *accset, nacc, *acc_array, total_states; int end_of_buffer_action = num_rules + 1; acc_array = allocate_integer_array( current_max_dfas ); nummt = 0; /* The compressed table format jams by entering the "jam state", * losing information about the previous state in the process. * In order to recover the previous state, we effectively need * to keep backing-up information. */ ++num_backing_up; if ( reject ) { /* Write out accepting list and pointer list. * * First we generate the "yy_acclist" array. In the process, * we compute the indices that will go into the "yy_accept" * array, and save the indices in the dfaacc array. */ int EOB_accepting_list[2]; /* Set up accepting structures for the End Of Buffer state. */ EOB_accepting_list[0] = 0; EOB_accepting_list[1] = end_of_buffer_action; accsiz[end_of_buffer_state] = 1; dfaacc[end_of_buffer_state].dfaacc_set = EOB_accepting_list; out_str_dec( long_align ? C_long_decl : C_short_decl, "yy_acclist", MAX( numas, 1 ) + 1 ); j = 1; /* index into "yy_acclist" array */ for ( i = 1; i <= lastdfa; ++i ) { acc_array[i] = j; if ( accsiz[i] != 0 ) { accset = dfaacc[i].dfaacc_set; nacc = accsiz[i]; if ( trace ) fprintf( stderr, _( "state # %d accepts: " ), i ); for ( k = 1; k <= nacc; ++k ) { int accnum = accset[k]; ++j; if ( variable_trailing_context_rules && ! (accnum & YY_TRAILING_HEAD_MASK) && accnum > 0 && accnum <= num_rules && rule_type[accnum] == RULE_VARIABLE ) { /* Special hack to flag * accepting number as part * of trailing context rule. */ accnum |= YY_TRAILING_MASK; } mkdata( accnum ); if ( trace ) { fprintf( stderr, "[%d]", accset[k] ); if ( k < nacc ) fputs( ", ", stderr ); else putc( '\n', stderr ); } } } } /* add accepting number for the "jam" state */ acc_array[i] = j; dataend(); } else { dfaacc[end_of_buffer_state].dfaacc_state = end_of_buffer_action; for ( i = 1; i <= lastdfa; ++i ) acc_array[i] = dfaacc[i].dfaacc_state; /* add accepting number for jam state */ acc_array[i] = 0; } /* Spit out "yy_accept" array. If we're doing "reject", it'll be * pointers into the "yy_acclist" array. Otherwise it's actual * accepting numbers. In either case, we just dump the numbers. */ /* "lastdfa + 2" is the size of "yy_accept"; includes room for C arrays * beginning at 0 and for "jam" state. */ k = lastdfa + 2; if ( reject ) /* We put a "cap" on the table associating lists of accepting * numbers with state numbers. This is needed because we tell * where the end of an accepting list is by looking at where * the list for the next state starts. */ ++k; out_str_dec( long_align ? C_long_decl : C_short_decl, "yy_accept", k ); for ( i = 1; i <= lastdfa; ++i ) { mkdata( acc_array[i] ); if ( ! reject && trace && acc_array[i] ) fprintf( stderr, _( "state # %d accepts: [%d]\n" ), i, acc_array[i] ); } /* Add entry for "jam" state. */ mkdata( acc_array[i] ); if ( reject ) /* Add "cap" for the list. */ mkdata( acc_array[i] ); dataend(); if ( useecs ) genecs(); if ( usemecs ) { /* Write out meta-equivalence classes (used to index * templates with). */ if ( trace ) fputs( _( "\n\nMeta-Equivalence Classes:\n" ), stderr ); out_str_dec( C_int_decl, "yy_meta", numecs + 1 ); for ( i = 1; i <= numecs; ++i ) { if ( trace ) fprintf( stderr, "%d = %d\n", i, ABS( tecbck[i] ) ); mkdata( ABS( tecbck[i] ) ); } dataend(); } total_states = lastdfa + numtemps; out_str_dec( (tblend >= MAX_SHORT || long_align) ? C_long_decl : C_short_decl, "yy_base", total_states + 1 ); for ( i = 1; i <= lastdfa; ++i ) { register int d = def[i]; if ( base[i] == JAMSTATE ) base[i] = jambase; if ( d == JAMSTATE ) def[i] = jamstate; else if ( d < 0 ) { /* Template reference. */ ++tmpuses; def[i] = lastdfa - d + 1; } mkdata( base[i] ); } /* Generate jam state's base index. */ mkdata( base[i] ); for ( ++i /* skip jam state */; i <= total_states; ++i ) { mkdata( base[i] ); def[i] = jamstate; } dataend(); out_str_dec( (total_states >= MAX_SHORT || long_align) ? C_long_decl : C_short_decl, "yy_def", total_states + 1 ); for ( i = 1; i <= total_states; ++i ) mkdata( def[i] ); dataend(); out_str_dec( (total_states >= MAX_SHORT || long_align) ? C_long_decl : C_short_decl, "yy_nxt", tblend + 1 ); for ( i = 1; i <= tblend; ++i ) { /* Note, the order of the following test is important. * If chk[i] is 0, then nxt[i] is undefined. */ if ( chk[i] == 0 || nxt[i] == 0 ) nxt[i] = jamstate; /* new state is the JAM state */ mkdata( nxt[i] ); } dataend(); out_str_dec( (total_states >= MAX_SHORT || long_align) ? C_long_decl : C_short_decl, "yy_chk", tblend + 1 ); for ( i = 1; i <= tblend; ++i ) { if ( chk[i] == 0 ) ++nummt; mkdata( chk[i] ); } dataend(); } /* Write out a formatted string (with a secondary string argument) at the * current indentation level, adding a final newline. */ void indent_put2s( fmt, arg ) char fmt[], arg[]; { do_indent(); out_str( fmt, arg ); outn( "" ); } /* Write out a string at the current indentation level, adding a final * newline. */ void indent_puts( str ) char str[]; { do_indent(); outn( str ); } /* make_tables - generate transition tables and finishes generating output file */ void make_tables() { register int i; int did_eof_rule = false; skelout(); /* First, take care of YY_DO_BEFORE_ACTION depending on yymore * being used. */ set_indent( 1 ); if ( yymore_used && ! yytext_is_array ) { indent_puts( "yytext_ptr -= yy_more_len; \\" ); indent_puts( "yyleng = (int) (yy_cp - yytext_ptr); \\" ); } else indent_puts( "yyleng = (int) (yy_cp - yy_bp); \\" ); /* Now also deal with copying yytext_ptr to yytext if needed. */ skelout(); if ( yytext_is_array ) { if ( yymore_used ) indent_puts( "if ( yyleng + yy_more_offset >= YYLMAX ) \\" ); else indent_puts( "if ( yyleng >= YYLMAX ) \\" ); indent_up(); indent_puts( "YY_FATAL_ERROR( \"token too large, exceeds YYLMAX\" ); \\" ); indent_down(); if ( yymore_used ) { indent_puts( "yy_flex_strncpy( &yytext[yy_more_offset], yytext_ptr, yyleng + 1 ); \\" ); indent_puts( "yyleng += yy_more_offset; \\" ); indent_puts( "yy_prev_more_offset = yy_more_offset; \\" ); indent_puts( "yy_more_offset = 0; \\" ); } else { indent_puts( "yy_flex_strncpy( yytext, yytext_ptr, yyleng + 1 ); \\" ); } } set_indent( 0 ); skelout(); out_dec( "#define YY_NUM_RULES %d\n", num_rules ); out_dec( "#define YY_END_OF_BUFFER %d\n", num_rules + 1 ); if ( fullspd ) { /* Need to define the transet type as a size large * enough to hold the biggest offset. */ int total_table_size = tblend + numecs + 1; char *trans_offset_type = (total_table_size >= MAX_SHORT || long_align) ? "long" : "short"; set_indent( 0 ); indent_puts( "struct yy_trans_info" ); indent_up(); indent_puts( "{" ); /* } for vi */ if ( long_align ) indent_puts( "long yy_verify;" ); else indent_puts( "short yy_verify;" ); /* In cases where its sister yy_verify *is* a "yes, there is * a transition", yy_nxt is the offset (in records) to the * next state. In most cases where there is no transition, * the value of yy_nxt is irrelevant. If yy_nxt is the -1th * record of a state, though, then yy_nxt is the action number * for that state. */ indent_put2s( "%s yy_nxt;", trans_offset_type ); indent_puts( "};" ); indent_down(); } if ( fullspd ) genctbl(); else if ( fulltbl ) genftbl(); else gentabs(); /* Definitions for backing up. We don't need them if REJECT * is being used because then we use an alternative backin-up * technique instead. */ if ( num_backing_up > 0 && ! reject ) { if ( ! C_plus_plus ) { indent_puts( "static yy_state_type yy_last_accepting_state;" ); indent_puts( "static char *yy_last_accepting_cpos;\n" ); } } if ( nultrans ) { out_str_dec( C_state_decl, "yy_NUL_trans", lastdfa + 1 ); for ( i = 1; i <= lastdfa; ++i ) { if ( fullspd ) out_dec( " &yy_transition[%d],\n", base[i] ); else mkdata( nultrans[i] ); } dataend(); } if ( ddebug ) { /* Spit out table mapping rules to line numbers. */ if ( ! C_plus_plus ) { indent_puts( "extern int yy_flex_debug;" ); indent_puts( "int yy_flex_debug = 1;\n" ); } out_str_dec( long_align ? C_long_decl : C_short_decl, "yy_rule_linenum", num_rules ); for ( i = 1; i < num_rules; ++i ) mkdata( rule_linenum[i] ); dataend(); } if ( reject ) { /* Declare state buffer variables. */ if ( ! C_plus_plus ) { outn( "static yy_state_type yy_state_buf[YY_BUF_SIZE + 2], *yy_state_ptr;" ); outn( "static char *yy_full_match;" ); outn( "static int yy_lp;" ); } if ( variable_trailing_context_rules ) { if ( ! C_plus_plus ) { outn( "static int yy_looking_for_trail_begin = 0;" ); outn( "static int yy_full_lp;" ); outn( "static int *yy_full_state;" ); } out_hex( "#define YY_TRAILING_MASK 0x%x\n", (unsigned int) YY_TRAILING_MASK ); out_hex( "#define YY_TRAILING_HEAD_MASK 0x%x\n", (unsigned int) YY_TRAILING_HEAD_MASK ); } outn( "#define REJECT \\" ); outn( "{ \\" ); /* } for vi */ outn( "*yy_cp = yy_hold_char; /* undo effects of setting up yytext */ \\" ); outn( "yy_cp = yy_full_match; /* restore poss. backed-over text */ \\" ); if ( variable_trailing_context_rules ) { outn( "yy_lp = yy_full_lp; /* restore orig. accepting pos. */ \\" ); outn( "yy_state_ptr = yy_full_state; /* restore orig. state */ \\" ); outn( "yy_current_state = *yy_state_ptr; /* restore curr. state */ \\" ); } outn( "++yy_lp; \\" ); outn( "goto find_rule; \\" ); /* { for vi */ outn( "}" ); } else { outn( "/* The intent behind this definition is that it'll catch" ); outn( " * any uses of REJECT which flex missed." ); outn( " */" ); outn( "#define REJECT reject_used_but_not_detected" ); } if ( yymore_used ) { if ( ! C_plus_plus ) { if ( yytext_is_array ) { indent_puts( "static int yy_more_offset = 0;" ); indent_puts( "static int yy_prev_more_offset = 0;" ); } else { indent_puts( "static int yy_more_flag = 0;" ); indent_puts( "static int yy_more_len = 0;" ); } } if ( yytext_is_array ) { indent_puts( "#define yymore() (yy_more_offset = yy_flex_strlen( yytext ))" ); indent_puts( "#define YY_NEED_STRLEN" ); indent_puts( "#define YY_MORE_ADJ 0" ); indent_puts( "#define YY_RESTORE_YY_MORE_OFFSET \\" ); indent_up(); indent_puts( "{ \\" ); indent_puts( "yy_more_offset = yy_prev_more_offset; \\" ); indent_puts( "yyleng -= yy_more_offset; \\" ); indent_puts( "}" ); indent_down(); } else { indent_puts( "#define yymore() (yy_more_flag = 1)" ); indent_puts( "#define YY_MORE_ADJ yy_more_len" ); indent_puts( "#define YY_RESTORE_YY_MORE_OFFSET" ); } } else { indent_puts( "#define yymore() yymore_used_but_not_detected" ); indent_puts( "#define YY_MORE_ADJ 0" ); indent_puts( "#define YY_RESTORE_YY_MORE_OFFSET" ); } if ( ! C_plus_plus ) { if ( yytext_is_array ) { outn( "#ifndef YYLMAX" ); outn( "#define YYLMAX 8192" ); outn( "#endif\n" ); outn( "char yytext[YYLMAX];" ); outn( "char *yytext_ptr;" ); } else outn( "char *yytext;" ); } out( &action_array[defs1_offset] ); line_directive_out( stdout, 0 ); skelout(); if ( ! C_plus_plus ) { if ( use_read ) { outn( "\tif ( (result = read( fileno(yyin), (char *) buf, max_size )) < 0 ) \\" ); outn( "\t\tYY_FATAL_ERROR( \"input in flex scanner failed\" );" ); } else { outn( "\tif ( yy_current_buffer->yy_is_interactive ) \\" ); outn( "\t\t{ \\" ); outn( "\t\tint c = '*', n; \\" ); outn( "\t\tfor ( n = 0; n < max_size && \\" ); outn( "\t\t\t (c = getc( yyin )) != EOF && c != '\\n'; ++n ) \\" ); outn( "\t\t\tbuf[n] = (char) c; \\" ); outn( "\t\tif ( c == '\\n' ) \\" ); outn( "\t\t\tbuf[n++] = (char) c; \\" ); outn( "\t\tif ( c == EOF && ferror( yyin ) ) \\" ); outn( "\t\t\tYY_FATAL_ERROR( \"input in flex scanner failed\" ); \\" ); outn( "\t\tresult = n; \\" ); outn( "\t\t} \\" ); outn( "\telse if ( ((result = fread( buf, 1, max_size, yyin )) == 0) \\" ); outn( "\t\t && ferror( yyin ) ) \\" ); outn( "\t\tYY_FATAL_ERROR( \"input in flex scanner failed\" );" ); } } skelout(); indent_puts( "#define YY_RULE_SETUP \\" ); indent_up(); if ( bol_needed ) { indent_puts( "if ( yyleng > 0 ) \\" ); indent_up(); indent_puts( "yy_current_buffer->yy_at_bol = \\" ); indent_puts( "\t\t(yytext[yyleng - 1] == '\\n'); \\" ); indent_down(); } indent_puts( "YY_USER_ACTION" ); indent_down(); skelout(); /* Copy prolog to output file. */ out( &action_array[prolog_offset] ); line_directive_out( stdout, 0 ); skelout(); set_indent( 2 ); if ( yymore_used && ! yytext_is_array ) { indent_puts( "yy_more_len = 0;" ); indent_puts( "if ( yy_more_flag )" ); indent_up(); indent_puts( "{" ); indent_puts( "yy_more_len = yy_c_buf_p - yytext_ptr;" ); indent_puts( "yy_more_flag = 0;" ); indent_puts( "}" ); indent_down(); } skelout(); gen_start_state(); /* Note, don't use any indentation. */ outn( "yy_match:" ); gen_next_match(); skelout(); set_indent( 2 ); gen_find_action(); skelout(); if ( do_yylineno ) { indent_puts( "if ( yy_act != YY_END_OF_BUFFER )" ); indent_up(); indent_puts( "{" ); indent_puts( "int yyl;" ); indent_puts( "for ( yyl = 0; yyl < yyleng; ++yyl )" ); indent_up(); indent_puts( "if ( yytext[yyl] == '\\n' )" ); indent_up(); indent_puts( "++yylineno;" ); indent_down(); indent_down(); indent_puts( "}" ); indent_down(); } skelout(); if ( ddebug ) { indent_puts( "if ( yy_flex_debug )" ); indent_up(); indent_puts( "{" ); indent_puts( "if ( yy_act == 0 )" ); indent_up(); indent_puts( C_plus_plus ? "cerr << \"--scanner backing up\\n\";" : "fprintf( stderr, \"--scanner backing up\\n\" );" ); indent_down(); do_indent(); out_dec( "else if ( yy_act < %d )\n", num_rules ); indent_up(); if ( C_plus_plus ) { indent_puts( "cerr << \"--accepting rule at line \" << yy_rule_linenum[yy_act] <<" ); indent_puts( " \"(\\\"\" << yytext << \"\\\")\\n\";" ); } else { indent_puts( "fprintf( stderr, \"--accepting rule at line %d (\\\"%s\\\")\\n\"," ); indent_puts( " yy_rule_linenum[yy_act], yytext );" ); } indent_down(); do_indent(); out_dec( "else if ( yy_act == %d )\n", num_rules ); indent_up(); if ( C_plus_plus ) { indent_puts( "cerr << \"--accepting default rule (\\\"\" << yytext << \"\\\")\\n\";" ); } else { indent_puts( "fprintf( stderr, \"--accepting default rule (\\\"%s\\\")\\n\"," ); indent_puts( " yytext );" ); } indent_down(); do_indent(); out_dec( "else if ( yy_act == %d )\n", num_rules + 1 ); indent_up(); indent_puts( C_plus_plus ? "cerr << \"--(end of buffer or a NUL)\\n\";" : "fprintf( stderr, \"--(end of buffer or a NUL)\\n\" );" ); indent_down(); do_indent(); outn( "else" ); indent_up(); if ( C_plus_plus ) { indent_puts( "cerr << \"--EOF (start condition \" << YY_START << \")\\n\";" ); } else { indent_puts( "fprintf( stderr, \"--EOF (start condition %d)\\n\", YY_START );" ); } indent_down(); indent_puts( "}" ); indent_down(); } /* Copy actions to output file. */ skelout(); indent_up(); gen_bu_action(); out( &action_array[action_offset] ); line_directive_out( stdout, 0 ); /* generate cases for any missing EOF rules */ for ( i = 1; i <= lastsc; ++i ) if ( ! sceof[i] ) { do_indent(); out_str( "case YY_STATE_EOF(%s):\n", scname[i] ); did_eof_rule = true; } if ( did_eof_rule ) { indent_up(); indent_puts( "yyterminate();" ); indent_down(); } /* Generate code for handling NUL's, if needed. */ /* First, deal with backing up and setting up yy_cp if the scanner * finds that it should JAM on the NUL. */ skelout(); set_indent( 4 ); if ( fullspd || fulltbl ) indent_puts( "yy_cp = yy_c_buf_p;" ); else { /* compressed table */ if ( ! reject && ! interactive ) { /* Do the guaranteed-needed backing up to figure * out the match. */ indent_puts( "yy_cp = yy_last_accepting_cpos;" ); indent_puts( "yy_current_state = yy_last_accepting_state;" ); } else /* Still need to initialize yy_cp, though * yy_current_state was set up by * yy_get_previous_state(). */ indent_puts( "yy_cp = yy_c_buf_p;" ); } /* Generate code for yy_get_previous_state(). */ set_indent( 1 ); skelout(); gen_start_state(); set_indent( 2 ); skelout(); gen_next_state( true ); set_indent( 1 ); skelout(); gen_NUL_trans(); skelout(); if ( do_yylineno ) { /* update yylineno inside of unput() */ indent_puts( "if ( c == '\\n' )" ); indent_up(); indent_puts( "--yylineno;" ); indent_down(); } skelout(); /* Update BOL and yylineno inside of input(). */ if ( bol_needed ) { indent_puts( "yy_current_buffer->yy_at_bol = (c == '\\n');" ); if ( do_yylineno ) { indent_puts( "if ( yy_current_buffer->yy_at_bol )" ); indent_up(); indent_puts( "++yylineno;" ); indent_down(); } } else if ( do_yylineno ) { indent_puts( "if ( c == '\\n' )" ); indent_up(); indent_puts( "++yylineno;" ); indent_down(); } skelout(); /* Copy remainder of input to output. */ line_directive_out( stdout, 1 ); if ( sectnum == 3 ) (void) flexscan(); /* copy remainder of input to output */ } flex-2.5.4/main.c100444 75424 12 70161 6366531463 12157 0ustar kwzhstaff/* flex - tool to generate fast lexical analyzers */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ #ifndef lint char copyright[] = "@(#) Copyright (c) 1990 The Regents of the University of California.\n\ All rights reserved.\n"; #endif /* not lint */ /* $Header: /home/daffy/u0/vern/flex/RCS/main.c,v 2.64 96/05/25 20:42:42 vern Exp $ */ #include "flexdef.h" #include "version.h" static char flex_version[] = FLEX_VERSION; /* declare functions that have forward references */ void flexinit PROTO((int, char**)); void readin PROTO((void)); void set_up_initial_allocations PROTO((void)); #ifdef NEED_ARGV_FIXUP extern void argv_fixup PROTO((int *, char ***)); #endif /* these globals are all defined and commented in flexdef.h */ int printstats, syntaxerror, eofseen, ddebug, trace, nowarn, spprdflt; int interactive, caseins, lex_compat, do_yylineno, useecs, fulltbl, usemecs; int fullspd, gen_line_dirs, performance_report, backing_up_report; int C_plus_plus, long_align, use_read, yytext_is_array, do_yywrap, csize; int yymore_used, reject, real_reject, continued_action, in_rule; int yymore_really_used, reject_really_used; int datapos, dataline, linenum, out_linenum; FILE *skelfile = NULL; int skel_ind = 0; char *action_array; int action_size, defs1_offset, prolog_offset, action_offset, action_index; char *infilename = NULL, *outfilename = NULL; int did_outfilename; char *prefix, *yyclass; int do_stdinit, use_stdout; int onestate[ONE_STACK_SIZE], onesym[ONE_STACK_SIZE]; int onenext[ONE_STACK_SIZE], onedef[ONE_STACK_SIZE], onesp; int current_mns, current_max_rules; int num_rules, num_eof_rules, default_rule, lastnfa; int *firstst, *lastst, *finalst, *transchar, *trans1, *trans2; int *accptnum, *assoc_rule, *state_type; int *rule_type, *rule_linenum, *rule_useful; int current_state_type; int variable_trailing_context_rules; int numtemps, numprots, protprev[MSP], protnext[MSP], prottbl[MSP]; int protcomst[MSP], firstprot, lastprot, protsave[PROT_SAVE_SIZE]; int numecs, nextecm[CSIZE + 1], ecgroup[CSIZE + 1], nummecs, tecfwd[CSIZE + 1]; int tecbck[CSIZE + 1]; int lastsc, *scset, *scbol, *scxclu, *sceof; int current_max_scs; char **scname; int current_max_dfa_size, current_max_xpairs; int current_max_template_xpairs, current_max_dfas; int lastdfa, *nxt, *chk, *tnxt; int *base, *def, *nultrans, NUL_ec, tblend, firstfree, **dss, *dfasiz; union dfaacc_union *dfaacc; int *accsiz, *dhash, numas; int numsnpairs, jambase, jamstate; int lastccl, *cclmap, *ccllen, *cclng, cclreuse; int current_maxccls, current_max_ccl_tbl_size; Char *ccltbl; char nmstr[MAXLINE]; int sectnum, nummt, hshcol, dfaeql, numeps, eps2, num_reallocs; int tmpuses, totnst, peakpairs, numuniq, numdup, hshsave; int num_backing_up, bol_needed; FILE *backing_up_file; int end_of_buffer_state; char **input_files; int num_input_files; /* Make sure program_name is initialized so we don't crash if writing * out an error message before getting the program name from argv[0]. */ char *program_name = "flex"; #ifndef SHORT_FILE_NAMES static char *outfile_template = "lex.%s.%s"; static char *backing_name = "lex.backup"; #else static char *outfile_template = "lex%s.%s"; static char *backing_name = "lex.bck"; #endif #ifdef THINK_C #include #endif #ifdef MS_DOS extern unsigned _stklen = 16384; #endif static char outfile_path[MAXLINE]; static int outfile_created = 0; static char *skelname = NULL; int main( argc, argv ) int argc; char **argv; { int i; #ifdef THINK_C argc = ccommand( &argv ); #endif #ifdef NEED_ARGV_FIXUP argv_fixup( &argc, &argv ); #endif flexinit( argc, argv ); readin(); ntod(); for ( i = 1; i <= num_rules; ++i ) if ( ! rule_useful[i] && i != default_rule ) line_warning( _( "rule cannot be matched" ), rule_linenum[i] ); if ( spprdflt && ! reject && rule_useful[default_rule] ) line_warning( _( "-s option given but default rule can be matched" ), rule_linenum[default_rule] ); /* Generate the C state transition tables from the DFA. */ make_tables(); /* Note, flexend does not return. It exits with its argument * as status. */ flexend( 0 ); return 0; /* keep compilers/lint happy */ } /* check_options - check user-specified options */ void check_options() { int i; if ( lex_compat ) { if ( C_plus_plus ) flexerror( _( "Can't use -+ with -l option" ) ); if ( fulltbl || fullspd ) flexerror( _( "Can't use -f or -F with -l option" ) ); /* Don't rely on detecting use of yymore() and REJECT, * just assume they'll be used. */ yymore_really_used = reject_really_used = true; yytext_is_array = true; do_yylineno = true; use_read = false; } if ( do_yylineno ) /* This should really be "maintain_backup_tables = true" */ reject_really_used = true; if ( csize == unspecified ) { if ( (fulltbl || fullspd) && ! useecs ) csize = DEFAULT_CSIZE; else csize = CSIZE; } if ( interactive == unspecified ) { if ( fulltbl || fullspd ) interactive = false; else interactive = true; } if ( fulltbl || fullspd ) { if ( usemecs ) flexerror( _( "-Cf/-CF and -Cm don't make sense together" ) ); if ( interactive ) flexerror( _( "-Cf/-CF and -I are incompatible" ) ); if ( lex_compat ) flexerror( _( "-Cf/-CF are incompatible with lex-compatibility mode" ) ); if ( do_yylineno ) flexerror( _( "-Cf/-CF and %option yylineno are incompatible" ) ); if ( fulltbl && fullspd ) flexerror( _( "-Cf and -CF are mutually exclusive" ) ); } if ( C_plus_plus && fullspd ) flexerror( _( "Can't use -+ with -CF option" ) ); if ( C_plus_plus && yytext_is_array ) { warn( _( "%array incompatible with -+ option" ) ); yytext_is_array = false; } if ( useecs ) { /* Set up doubly-linked equivalence classes. */ /* We loop all the way up to csize, since ecgroup[csize] is * the position used for NUL characters. */ ecgroup[1] = NIL; for ( i = 2; i <= csize; ++i ) { ecgroup[i] = i - 1; nextecm[i - 1] = i; } nextecm[csize] = NIL; } else { /* Put everything in its own equivalence class. */ for ( i = 1; i <= csize; ++i ) { ecgroup[i] = i; nextecm[i] = BAD_SUBSCRIPT; /* to catch errors */ } } if ( ! use_stdout ) { FILE *prev_stdout; if ( ! did_outfilename ) { char *suffix; if ( C_plus_plus ) suffix = "cc"; else suffix = "c"; sprintf( outfile_path, outfile_template, prefix, suffix ); outfilename = outfile_path; } prev_stdout = freopen( outfilename, "w", stdout ); if ( prev_stdout == NULL ) lerrsf( _( "could not create %s" ), outfilename ); outfile_created = 1; } if ( skelname && (skelfile = fopen( skelname, "r" )) == NULL ) lerrsf( _( "can't open skeleton file %s" ), skelname ); if ( strcmp( prefix, "yy" ) ) { #define GEN_PREFIX(name) out_str3( "#define yy%s %s%s\n", name, prefix, name ) if ( C_plus_plus ) GEN_PREFIX( "FlexLexer" ); else { GEN_PREFIX( "_create_buffer" ); GEN_PREFIX( "_delete_buffer" ); GEN_PREFIX( "_scan_buffer" ); GEN_PREFIX( "_scan_string" ); GEN_PREFIX( "_scan_bytes" ); GEN_PREFIX( "_flex_debug" ); GEN_PREFIX( "_init_buffer" ); GEN_PREFIX( "_flush_buffer" ); GEN_PREFIX( "_load_buffer_state" ); GEN_PREFIX( "_switch_to_buffer" ); GEN_PREFIX( "in" ); GEN_PREFIX( "leng" ); GEN_PREFIX( "lex" ); GEN_PREFIX( "out" ); GEN_PREFIX( "restart" ); GEN_PREFIX( "text" ); if ( do_yylineno ) GEN_PREFIX( "lineno" ); } if ( do_yywrap ) GEN_PREFIX( "wrap" ); outn( "" ); } if ( did_outfilename ) line_directive_out( stdout, 0 ); skelout(); } /* flexend - terminate flex * * note * This routine does not return. */ void flexend( exit_status ) int exit_status; { int tblsiz; int unlink(); if ( skelfile != NULL ) { if ( ferror( skelfile ) ) lerrsf( _( "input error reading skeleton file %s" ), skelname ); else if ( fclose( skelfile ) ) lerrsf( _( "error closing skeleton file %s" ), skelname ); } if ( exit_status != 0 && outfile_created ) { if ( ferror( stdout ) ) lerrsf( _( "error writing output file %s" ), outfilename ); else if ( fclose( stdout ) ) lerrsf( _( "error closing output file %s" ), outfilename ); else if ( unlink( outfilename ) ) lerrsf( _( "error deleting output file %s" ), outfilename ); } if ( backing_up_report && backing_up_file ) { if ( num_backing_up == 0 ) fprintf( backing_up_file, _( "No backing up.\n" ) ); else if ( fullspd || fulltbl ) fprintf( backing_up_file, _( "%d backing up (non-accepting) states.\n" ), num_backing_up ); else fprintf( backing_up_file, _( "Compressed tables always back up.\n" ) ); if ( ferror( backing_up_file ) ) lerrsf( _( "error writing backup file %s" ), backing_name ); else if ( fclose( backing_up_file ) ) lerrsf( _( "error closing backup file %s" ), backing_name ); } if ( printstats ) { fprintf( stderr, _( "%s version %s usage statistics:\n" ), program_name, flex_version ); fprintf( stderr, _( " scanner options: -" ) ); if ( C_plus_plus ) putc( '+', stderr ); if ( backing_up_report ) putc( 'b', stderr ); if ( ddebug ) putc( 'd', stderr ); if ( caseins ) putc( 'i', stderr ); if ( lex_compat ) putc( 'l', stderr ); if ( performance_report > 0 ) putc( 'p', stderr ); if ( performance_report > 1 ) putc( 'p', stderr ); if ( spprdflt ) putc( 's', stderr ); if ( use_stdout ) putc( 't', stderr ); if ( printstats ) putc( 'v', stderr ); /* always true! */ if ( nowarn ) putc( 'w', stderr ); if ( interactive == false ) putc( 'B', stderr ); if ( interactive == true ) putc( 'I', stderr ); if ( ! gen_line_dirs ) putc( 'L', stderr ); if ( trace ) putc( 'T', stderr ); if ( csize == unspecified ) /* We encountered an error fairly early on, so csize * never got specified. Define it now, to prevent * bogus table sizes being written out below. */ csize = 256; if ( csize == 128 ) putc( '7', stderr ); else putc( '8', stderr ); fprintf( stderr, " -C" ); if ( long_align ) putc( 'a', stderr ); if ( fulltbl ) putc( 'f', stderr ); if ( fullspd ) putc( 'F', stderr ); if ( useecs ) putc( 'e', stderr ); if ( usemecs ) putc( 'm', stderr ); if ( use_read ) putc( 'r', stderr ); if ( did_outfilename ) fprintf( stderr, " -o%s", outfilename ); if ( skelname ) fprintf( stderr, " -S%s", skelname ); if ( strcmp( prefix, "yy" ) ) fprintf( stderr, " -P%s", prefix ); putc( '\n', stderr ); fprintf( stderr, _( " %d/%d NFA states\n" ), lastnfa, current_mns ); fprintf( stderr, _( " %d/%d DFA states (%d words)\n" ), lastdfa, current_max_dfas, totnst ); fprintf( stderr, _( " %d rules\n" ), num_rules + num_eof_rules - 1 /* - 1 for def. rule */ ); if ( num_backing_up == 0 ) fprintf( stderr, _( " No backing up\n" ) ); else if ( fullspd || fulltbl ) fprintf( stderr, _( " %d backing-up (non-accepting) states\n" ), num_backing_up ); else fprintf( stderr, _( " Compressed tables always back-up\n" ) ); if ( bol_needed ) fprintf( stderr, _( " Beginning-of-line patterns used\n" ) ); fprintf( stderr, _( " %d/%d start conditions\n" ), lastsc, current_max_scs ); fprintf( stderr, _( " %d epsilon states, %d double epsilon states\n" ), numeps, eps2 ); if ( lastccl == 0 ) fprintf( stderr, _( " no character classes\n" ) ); else fprintf( stderr, _( " %d/%d character classes needed %d/%d words of storage, %d reused\n" ), lastccl, current_maxccls, cclmap[lastccl] + ccllen[lastccl], current_max_ccl_tbl_size, cclreuse ); fprintf( stderr, _( " %d state/nextstate pairs created\n" ), numsnpairs ); fprintf( stderr, _( " %d/%d unique/duplicate transitions\n" ), numuniq, numdup ); if ( fulltbl ) { tblsiz = lastdfa * numecs; fprintf( stderr, _( " %d table entries\n" ), tblsiz ); } else { tblsiz = 2 * (lastdfa + numtemps) + 2 * tblend; fprintf( stderr, _( " %d/%d base-def entries created\n" ), lastdfa + numtemps, current_max_dfas ); fprintf( stderr, _( " %d/%d (peak %d) nxt-chk entries created\n" ), tblend, current_max_xpairs, peakpairs ); fprintf( stderr, _( " %d/%d (peak %d) template nxt-chk entries created\n" ), numtemps * nummecs, current_max_template_xpairs, numtemps * numecs ); fprintf( stderr, _( " %d empty table entries\n" ), nummt ); fprintf( stderr, _( " %d protos created\n" ), numprots ); fprintf( stderr, _( " %d templates created, %d uses\n" ), numtemps, tmpuses ); } if ( useecs ) { tblsiz = tblsiz + csize; fprintf( stderr, _( " %d/%d equivalence classes created\n" ), numecs, csize ); } if ( usemecs ) { tblsiz = tblsiz + numecs; fprintf( stderr, _( " %d/%d meta-equivalence classes created\n" ), nummecs, csize ); } fprintf( stderr, _( " %d (%d saved) hash collisions, %d DFAs equal\n" ), hshcol, hshsave, dfaeql ); fprintf( stderr, _( " %d sets of reallocations needed\n" ), num_reallocs ); fprintf( stderr, _( " %d total table entries needed\n" ), tblsiz ); } exit( exit_status ); } /* flexinit - initialize flex */ void flexinit( argc, argv ) int argc; char **argv; { int i, sawcmpflag; char *arg; printstats = syntaxerror = trace = spprdflt = caseins = false; lex_compat = C_plus_plus = backing_up_report = ddebug = fulltbl = false; fullspd = long_align = nowarn = yymore_used = continued_action = false; do_yylineno = yytext_is_array = in_rule = reject = do_stdinit = false; yymore_really_used = reject_really_used = unspecified; interactive = csize = unspecified; do_yywrap = gen_line_dirs = usemecs = useecs = true; performance_report = 0; did_outfilename = 0; prefix = "yy"; yyclass = 0; use_read = use_stdout = false; sawcmpflag = false; /* Initialize dynamic array for holding the rule actions. */ action_size = 2048; /* default size of action array in bytes */ action_array = allocate_character_array( action_size ); defs1_offset = prolog_offset = action_offset = action_index = 0; action_array[0] = '\0'; program_name = argv[0]; if ( program_name[0] != '\0' && program_name[strlen( program_name ) - 1] == '+' ) C_plus_plus = true; /* read flags */ for ( --argc, ++argv; argc ; --argc, ++argv ) { arg = argv[0]; if ( arg[0] != '-' || arg[1] == '\0' ) break; if ( arg[1] == '-' ) { /* --option */ if ( ! strcmp( arg, "--help" ) ) arg = "-h"; else if ( ! strcmp( arg, "--version" ) ) arg = "-V"; else if ( ! strcmp( arg, "--" ) ) { /* end of options */ --argc; ++argv; break; } } for ( i = 1; arg[i] != '\0'; ++i ) switch ( arg[i] ) { case '+': C_plus_plus = true; break; case 'B': interactive = false; break; case 'b': backing_up_report = true; break; case 'c': break; case 'C': if ( i != 1 ) flexerror( _( "-C flag must be given separately" ) ); if ( ! sawcmpflag ) { useecs = false; usemecs = false; fulltbl = false; sawcmpflag = true; } for ( ++i; arg[i] != '\0'; ++i ) switch ( arg[i] ) { case 'a': long_align = true; break; case 'e': useecs = true; break; case 'F': fullspd = true; break; case 'f': fulltbl = true; break; case 'm': usemecs = true; break; case 'r': use_read = true; break; default: lerrif( _( "unknown -C option '%c'" ), (int) arg[i] ); break; } goto get_next_arg; case 'd': ddebug = true; break; case 'f': useecs = usemecs = false; use_read = fulltbl = true; break; case 'F': useecs = usemecs = false; use_read = fullspd = true; break; case '?': case 'h': usage(); exit( 0 ); case 'I': interactive = true; break; case 'i': caseins = true; break; case 'l': lex_compat = true; break; case 'L': gen_line_dirs = false; break; case 'n': /* Stupid do-nothing deprecated * option. */ break; case 'o': if ( i != 1 ) flexerror( _( "-o flag must be given separately" ) ); outfilename = arg + i + 1; did_outfilename = 1; goto get_next_arg; case 'P': if ( i != 1 ) flexerror( _( "-P flag must be given separately" ) ); prefix = arg + i + 1; goto get_next_arg; case 'p': ++performance_report; break; case 'S': if ( i != 1 ) flexerror( _( "-S flag must be given separately" ) ); skelname = arg + i + 1; goto get_next_arg; case 's': spprdflt = true; break; case 't': use_stdout = true; break; case 'T': trace = true; break; case 'v': printstats = true; break; case 'V': printf( _( "%s version %s\n" ), program_name, flex_version ); exit( 0 ); case 'w': nowarn = true; break; case '7': csize = 128; break; case '8': csize = CSIZE; break; default: fprintf( stderr, _( "%s: unknown flag '%c'. For usage, try\n\t%s --help\n" ), program_name, (int) arg[i], program_name ); exit( 1 ); } /* Used by -C, -S, -o, and -P flags in lieu of a "continue 2" * control. */ get_next_arg: ; } num_input_files = argc; input_files = argv; set_input_file( num_input_files > 0 ? input_files[0] : NULL ); lastccl = lastsc = lastdfa = lastnfa = 0; num_rules = num_eof_rules = default_rule = 0; numas = numsnpairs = tmpuses = 0; numecs = numeps = eps2 = num_reallocs = hshcol = dfaeql = totnst = 0; numuniq = numdup = hshsave = eofseen = datapos = dataline = 0; num_backing_up = onesp = numprots = 0; variable_trailing_context_rules = bol_needed = false; out_linenum = linenum = sectnum = 1; firstprot = NIL; /* Used in mkprot() so that the first proto goes in slot 1 * of the proto queue. */ lastprot = 1; set_up_initial_allocations(); } /* readin - read in the rules section of the input file(s) */ void readin() { static char yy_stdinit[] = "FILE *yyin = stdin, *yyout = stdout;"; static char yy_nostdinit[] = "FILE *yyin = (FILE *) 0, *yyout = (FILE *) 0;"; line_directive_out( (FILE *) 0, 1 ); if ( yyparse() ) { pinpoint_message( _( "fatal parse error" ) ); flexend( 1 ); } if ( syntaxerror ) flexend( 1 ); if ( backing_up_report ) { backing_up_file = fopen( backing_name, "w" ); if ( backing_up_file == NULL ) lerrsf( _( "could not create backing-up info file %s" ), backing_name ); } else backing_up_file = NULL; if ( yymore_really_used == true ) yymore_used = true; else if ( yymore_really_used == false ) yymore_used = false; if ( reject_really_used == true ) reject = true; else if ( reject_really_used == false ) reject = false; if ( performance_report > 0 ) { if ( lex_compat ) { fprintf( stderr, _( "-l AT&T lex compatibility option entails a large performance penalty\n" ) ); fprintf( stderr, _( " and may be the actual source of other reported performance penalties\n" ) ); } else if ( do_yylineno ) { fprintf( stderr, _( "%%option yylineno entails a large performance penalty\n" ) ); } if ( performance_report > 1 ) { if ( interactive ) fprintf( stderr, _( "-I (interactive) entails a minor performance penalty\n" ) ); if ( yymore_used ) fprintf( stderr, _( "yymore() entails a minor performance penalty\n" ) ); } if ( reject ) fprintf( stderr, _( "REJECT entails a large performance penalty\n" ) ); if ( variable_trailing_context_rules ) fprintf( stderr, _( "Variable trailing context rules entail a large performance penalty\n" ) ); } if ( reject ) real_reject = true; if ( variable_trailing_context_rules ) reject = true; if ( (fulltbl || fullspd) && reject ) { if ( real_reject ) flexerror( _( "REJECT cannot be used with -f or -F" ) ); else if ( do_yylineno ) flexerror( _( "%option yylineno cannot be used with -f or -F" ) ); else flexerror( _( "variable trailing context rules cannot be used with -f or -F" ) ); } if ( reject ) outn( "\n#define YY_USES_REJECT" ); if ( ! do_yywrap ) { outn( "\n#define yywrap() 1" ); outn( "#define YY_SKIP_YYWRAP" ); } if ( ddebug ) outn( "\n#define FLEX_DEBUG" ); if ( csize == 256 ) outn( "typedef unsigned char YY_CHAR;" ); else outn( "typedef char YY_CHAR;" ); if ( C_plus_plus ) { outn( "#define yytext_ptr yytext" ); if ( interactive ) outn( "#define YY_INTERACTIVE" ); } else { if ( do_stdinit ) { outn( "#ifdef VMS" ); outn( "#ifndef __VMS_POSIX" ); outn( yy_nostdinit ); outn( "#else" ); outn( yy_stdinit ); outn( "#endif" ); outn( "#else" ); outn( yy_stdinit ); outn( "#endif" ); } else outn( yy_nostdinit ); } if ( fullspd ) outn( "typedef yyconst struct yy_trans_info *yy_state_type;" ); else if ( ! C_plus_plus ) outn( "typedef int yy_state_type;" ); if ( ddebug ) outn( "\n#define FLEX_DEBUG" ); if ( lex_compat ) outn( "#define YY_FLEX_LEX_COMPAT" ); if ( do_yylineno && ! C_plus_plus ) { outn( "extern int yylineno;" ); outn( "int yylineno = 1;" ); } if ( C_plus_plus ) { outn( "\n#include " ); if ( yyclass ) { outn( "int yyFlexLexer::yylex()" ); outn( "\t{" ); outn( "\tLexerError( \"yyFlexLexer::yylex invoked but %option yyclass used\" );" ); outn( "\treturn 0;" ); outn( "\t}" ); out_str( "\n#define YY_DECL int %s::yylex()\n", yyclass ); } } else { if ( yytext_is_array ) outn( "extern char yytext[];\n" ); else { outn( "extern char *yytext;" ); outn( "#define yytext_ptr yytext" ); } if ( yyclass ) flexerror( _( "%option yyclass only meaningful for C++ scanners" ) ); } if ( useecs ) numecs = cre8ecs( nextecm, ecgroup, csize ); else numecs = csize; /* Now map the equivalence class for NUL to its expected place. */ ecgroup[0] = ecgroup[csize]; NUL_ec = ABS( ecgroup[0] ); if ( useecs ) ccl2ecl(); } /* set_up_initial_allocations - allocate memory for internal tables */ void set_up_initial_allocations() { current_mns = INITIAL_MNS; firstst = allocate_integer_array( current_mns ); lastst = allocate_integer_array( current_mns ); finalst = allocate_integer_array( current_mns ); transchar = allocate_integer_array( current_mns ); trans1 = allocate_integer_array( current_mns ); trans2 = allocate_integer_array( current_mns ); accptnum = allocate_integer_array( current_mns ); assoc_rule = allocate_integer_array( current_mns ); state_type = allocate_integer_array( current_mns ); current_max_rules = INITIAL_MAX_RULES; rule_type = allocate_integer_array( current_max_rules ); rule_linenum = allocate_integer_array( current_max_rules ); rule_useful = allocate_integer_array( current_max_rules ); current_max_scs = INITIAL_MAX_SCS; scset = allocate_integer_array( current_max_scs ); scbol = allocate_integer_array( current_max_scs ); scxclu = allocate_integer_array( current_max_scs ); sceof = allocate_integer_array( current_max_scs ); scname = allocate_char_ptr_array( current_max_scs ); current_maxccls = INITIAL_MAX_CCLS; cclmap = allocate_integer_array( current_maxccls ); ccllen = allocate_integer_array( current_maxccls ); cclng = allocate_integer_array( current_maxccls ); current_max_ccl_tbl_size = INITIAL_MAX_CCL_TBL_SIZE; ccltbl = allocate_Character_array( current_max_ccl_tbl_size ); current_max_dfa_size = INITIAL_MAX_DFA_SIZE; current_max_xpairs = INITIAL_MAX_XPAIRS; nxt = allocate_integer_array( current_max_xpairs ); chk = allocate_integer_array( current_max_xpairs ); current_max_template_xpairs = INITIAL_MAX_TEMPLATE_XPAIRS; tnxt = allocate_integer_array( current_max_template_xpairs ); current_max_dfas = INITIAL_MAX_DFAS; base = allocate_integer_array( current_max_dfas ); def = allocate_integer_array( current_max_dfas ); dfasiz = allocate_integer_array( current_max_dfas ); accsiz = allocate_integer_array( current_max_dfas ); dhash = allocate_integer_array( current_max_dfas ); dss = allocate_int_ptr_array( current_max_dfas ); dfaacc = allocate_dfaacc_union( current_max_dfas ); nultrans = (int *) 0; } void usage() { FILE *f = stdout; fprintf( f, _( "%s [-bcdfhilnpstvwBFILTV78+? -C[aefFmr] -ooutput -Pprefix -Sskeleton]\n" ), program_name ); fprintf( f, _( "\t[--help --version] [file ...]\n" ) ); fprintf( f, _( "\t-b generate backing-up information to %s\n" ), backing_name ); fprintf( f, _( "\t-c do-nothing POSIX option\n" ) ); fprintf( f, _( "\t-d turn on debug mode in generated scanner\n" ) ); fprintf( f, _( "\t-f generate fast, large scanner\n" ) ); fprintf( f, _( "\t-h produce this help message\n" ) ); fprintf( f, _( "\t-i generate case-insensitive scanner\n" ) ); fprintf( f, _( "\t-l maximal compatibility with original lex\n" ) ); fprintf( f, _( "\t-n do-nothing POSIX option\n" ) ); fprintf( f, _( "\t-p generate performance report to stderr\n" ) ); fprintf( f, _( "\t-s suppress default rule to ECHO unmatched text\n" ) ); if ( ! did_outfilename ) { sprintf( outfile_path, outfile_template, prefix, C_plus_plus ? "cc" : "c" ); outfilename = outfile_path; } fprintf( f, _( "\t-t write generated scanner on stdout instead of %s\n" ), outfilename ); fprintf( f, _( "\t-v write summary of scanner statistics to f\n" ) ); fprintf( f, _( "\t-w do not generate warnings\n" ) ); fprintf( f, _( "\t-B generate batch scanner (opposite of -I)\n" ) ); fprintf( f, _( "\t-F use alternative fast scanner representation\n" ) ); fprintf( f, _( "\t-I generate interactive scanner (opposite of -B)\n" ) ); fprintf( f, _( "\t-L suppress #line directives in scanner\n" ) ); fprintf( f, _( "\t-T %s should run in trace mode\n" ), program_name ); fprintf( f, _( "\t-V report %s version\n" ), program_name ); fprintf( f, _( "\t-7 generate 7-bit scanner\n" ) ); fprintf( f, _( "\t-8 generate 8-bit scanner\n" ) ); fprintf( f, _( "\t-+ generate C++ scanner class\n" ) ); fprintf( f, _( "\t-? produce this help message\n" ) ); fprintf( f, _( "\t-C specify degree of table compression (default is -Cem):\n" ) ); fprintf( f, _( "\t\t-Ca trade off larger tables for better memory alignment\n" ) ); fprintf( f, _( "\t\t-Ce construct equivalence classes\n" ) ); fprintf( f, _( "\t\t-Cf do not compress scanner tables; use -f representation\n" ) ); fprintf( f, _( "\t\t-CF do not compress scanner tables; use -F representation\n" ) ); fprintf( f, _( "\t\t-Cm construct meta-equivalence classes\n" ) ); fprintf( f, _( "\t\t-Cr use read() instead of stdio for scanner input\n" ) ); fprintf( f, _( "\t-o specify output filename\n" ) ); fprintf( f, _( "\t-P specify scanner prefix other than \"yy\"\n" ) ); fprintf( f, _( "\t-S specify skeleton file\n" ) ); fprintf( f, _( "\t--help produce this help message\n" ) ); fprintf( f, _( "\t--version report %s version\n" ), program_name ); } flex-2.5.4/misc.c100444 75424 12 37664 6366531463 12201 0ustar kwzhstaff/* misc - miscellaneous flex routines */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/misc.c,v 2.47 95/04/28 11:39:39 vern Exp $ */ #include "flexdef.h" void action_define( defname, value ) char *defname; int value; { char buf[MAXLINE]; if ( (int) strlen( defname ) > MAXLINE / 2 ) { format_pinpoint_message( _( "name \"%s\" ridiculously long" ), defname ); return; } sprintf( buf, "#define %s %d\n", defname, value ); add_action( buf ); } void add_action( new_text ) char *new_text; { int len = strlen( new_text ); while ( len + action_index >= action_size - 10 /* slop */ ) { int new_size = action_size * 2; if ( new_size <= 0 ) /* Increase just a little, to try to avoid overflow * on 16-bit machines. */ action_size += action_size / 8; else action_size = new_size; action_array = reallocate_character_array( action_array, action_size ); } strcpy( &action_array[action_index], new_text ); action_index += len; } /* allocate_array - allocate memory for an integer array of the given size */ void *allocate_array( size, element_size ) int size; size_t element_size; { register void *mem; size_t num_bytes = element_size * size; mem = flex_alloc( num_bytes ); if ( ! mem ) flexfatal( _( "memory allocation failed in allocate_array()" ) ); return mem; } /* all_lower - true if a string is all lower-case */ int all_lower( str ) register char *str; { while ( *str ) { if ( ! isascii( (Char) *str ) || ! islower( *str ) ) return 0; ++str; } return 1; } /* all_upper - true if a string is all upper-case */ int all_upper( str ) register char *str; { while ( *str ) { if ( ! isascii( (Char) *str ) || ! isupper( *str ) ) return 0; ++str; } return 1; } /* bubble - bubble sort an integer array in increasing order * * synopsis * int v[n], n; * void bubble( v, n ); * * description * sorts the first n elements of array v and replaces them in * increasing order. * * passed * v - the array to be sorted * n - the number of elements of 'v' to be sorted */ void bubble( v, n ) int v[], n; { register int i, j, k; for ( i = n; i > 1; --i ) for ( j = 1; j < i; ++j ) if ( v[j] > v[j + 1] ) /* compare */ { k = v[j]; /* exchange */ v[j] = v[j + 1]; v[j + 1] = k; } } /* check_char - checks a character to make sure it's within the range * we're expecting. If not, generates fatal error message * and exits. */ void check_char( c ) int c; { if ( c >= CSIZE ) lerrsf( _( "bad character '%s' detected in check_char()" ), readable_form( c ) ); if ( c >= csize ) lerrsf( _( "scanner requires -8 flag to use the character %s" ), readable_form( c ) ); } /* clower - replace upper-case letter to lower-case */ Char clower( c ) register int c; { return (Char) ((isascii( c ) && isupper( c )) ? tolower( c ) : c); } /* copy_string - returns a dynamically allocated copy of a string */ char *copy_string( str ) register const char *str; { register const char *c1; register char *c2; char *copy; unsigned int size; /* find length */ for ( c1 = str; *c1; ++c1 ) ; size = (c1 - str + 1) * sizeof( char ); copy = (char *) flex_alloc( size ); if ( copy == NULL ) flexfatal( _( "dynamic memory failure in copy_string()" ) ); for ( c2 = copy; (*c2++ = *str++) != 0; ) ; return copy; } /* copy_unsigned_string - * returns a dynamically allocated copy of a (potentially) unsigned string */ Char *copy_unsigned_string( str ) register Char *str; { register Char *c; Char *copy; /* find length */ for ( c = str; *c; ++c ) ; copy = allocate_Character_array( c - str + 1 ); for ( c = copy; (*c++ = *str++) != 0; ) ; return copy; } /* cshell - shell sort a character array in increasing order * * synopsis * * Char v[n]; * int n, special_case_0; * cshell( v, n, special_case_0 ); * * description * Does a shell sort of the first n elements of array v. * If special_case_0 is true, then any element equal to 0 * is instead assumed to have infinite weight. * * passed * v - array to be sorted * n - number of elements of v to be sorted */ void cshell( v, n, special_case_0 ) Char v[]; int n, special_case_0; { int gap, i, j, jg; Char k; for ( gap = n / 2; gap > 0; gap = gap / 2 ) for ( i = gap; i < n; ++i ) for ( j = i - gap; j >= 0; j = j - gap ) { jg = j + gap; if ( special_case_0 ) { if ( v[jg] == 0 ) break; else if ( v[j] != 0 && v[j] <= v[jg] ) break; } else if ( v[j] <= v[jg] ) break; k = v[j]; v[j] = v[jg]; v[jg] = k; } } /* dataend - finish up a block of data declarations */ void dataend() { if ( datapos > 0 ) dataflush(); /* add terminator for initialization; { for vi */ outn( " } ;\n" ); dataline = 0; datapos = 0; } /* dataflush - flush generated data statements */ void dataflush() { outc( '\n' ); if ( ++dataline >= NUMDATALINES ) { /* Put out a blank line so that the table is grouped into * large blocks that enable the user to find elements easily. */ outc( '\n' ); dataline = 0; } /* Reset the number of characters written on the current line. */ datapos = 0; } /* flexerror - report an error message and terminate */ void flexerror( msg ) const char msg[]; { fprintf( stderr, "%s: %s\n", program_name, msg ); flexend( 1 ); } /* flexfatal - report a fatal error message and terminate */ void flexfatal( msg ) const char msg[]; { fprintf( stderr, _( "%s: fatal internal error, %s\n" ), program_name, msg ); exit( 1 ); } /* htoi - convert a hexadecimal digit string to an integer value */ int htoi( str ) Char str[]; { unsigned int result; (void) sscanf( (char *) str, "%x", &result ); return result; } /* lerrif - report an error message formatted with one integer argument */ void lerrif( msg, arg ) const char msg[]; int arg; { char errmsg[MAXLINE]; (void) sprintf( errmsg, msg, arg ); flexerror( errmsg ); } /* lerrsf - report an error message formatted with one string argument */ void lerrsf( msg, arg ) const char msg[], arg[]; { char errmsg[MAXLINE]; (void) sprintf( errmsg, msg, arg ); flexerror( errmsg ); } /* line_directive_out - spit out a "#line" statement */ void line_directive_out( output_file, do_infile ) FILE *output_file; int do_infile; { char directive[MAXLINE], filename[MAXLINE]; char *s1, *s2, *s3; static char line_fmt[] = "#line %d \"%s\"\n"; if ( ! gen_line_dirs ) return; if ( (do_infile && ! infilename) || (! do_infile && ! outfilename) ) /* don't know the filename to use, skip */ return; s1 = do_infile ? infilename : outfilename; s2 = filename; s3 = &filename[sizeof( filename ) - 2]; while ( s2 < s3 && *s1 ) { if ( *s1 == '\\' ) /* Escape the '\' */ *s2++ = '\\'; *s2++ = *s1++; } *s2 = '\0'; if ( do_infile ) sprintf( directive, line_fmt, linenum, filename ); else { if ( output_file == stdout ) /* Account for the line directive itself. */ ++out_linenum; sprintf( directive, line_fmt, out_linenum, filename ); } /* If output_file is nil then we should put the directive in * the accumulated actions. */ if ( output_file ) { fputs( directive, output_file ); } else add_action( directive ); } /* mark_defs1 - mark the current position in the action array as * representing where the user's section 1 definitions end * and the prolog begins */ void mark_defs1() { defs1_offset = 0; action_array[action_index++] = '\0'; action_offset = prolog_offset = action_index; action_array[action_index] = '\0'; } /* mark_prolog - mark the current position in the action array as * representing the end of the action prolog */ void mark_prolog() { action_array[action_index++] = '\0'; action_offset = action_index; action_array[action_index] = '\0'; } /* mk2data - generate a data statement for a two-dimensional array * * Generates a data statement initializing the current 2-D array to "value". */ void mk2data( value ) int value; { if ( datapos >= NUMDATAITEMS ) { outc( ',' ); dataflush(); } if ( datapos == 0 ) /* Indent. */ out( " " ); else outc( ',' ); ++datapos; out_dec( "%5d", value ); } /* mkdata - generate a data statement * * Generates a data statement initializing the current array element to * "value". */ void mkdata( value ) int value; { if ( datapos >= NUMDATAITEMS ) { outc( ',' ); dataflush(); } if ( datapos == 0 ) /* Indent. */ out( " " ); else outc( ',' ); ++datapos; out_dec( "%5d", value ); } /* myctoi - return the integer represented by a string of digits */ int myctoi( array ) char array[]; { int val = 0; (void) sscanf( array, "%d", &val ); return val; } /* myesc - return character corresponding to escape sequence */ Char myesc( array ) Char array[]; { Char c, esc_char; switch ( array[1] ) { case 'b': return '\b'; case 'f': return '\f'; case 'n': return '\n'; case 'r': return '\r'; case 't': return '\t'; #if __STDC__ case 'a': return '\a'; case 'v': return '\v'; #else case 'a': return '\007'; case 'v': return '\013'; #endif case '0': case '1': case '2': case '3': case '4': case '5': case '6': case '7': { /* \ */ int sptr = 1; while ( isascii( array[sptr] ) && isdigit( array[sptr] ) ) /* Don't increment inside loop control * because if isdigit() is a macro it might * expand into multiple increments ... */ ++sptr; c = array[sptr]; array[sptr] = '\0'; esc_char = otoi( array + 1 ); array[sptr] = c; return esc_char; } case 'x': { /* \x */ int sptr = 2; while ( isascii( array[sptr] ) && isxdigit( (char) array[sptr] ) ) /* Don't increment inside loop control * because if isdigit() is a macro it might * expand into multiple increments ... */ ++sptr; c = array[sptr]; array[sptr] = '\0'; esc_char = htoi( array + 2 ); array[sptr] = c; return esc_char; } default: return array[1]; } } /* otoi - convert an octal digit string to an integer value */ int otoi( str ) Char str[]; { unsigned int result; (void) sscanf( (char *) str, "%o", &result ); return result; } /* out - various flavors of outputing a (possibly formatted) string for the * generated scanner, keeping track of the line count. */ void out( str ) const char str[]; { fputs( str, stdout ); out_line_count( str ); } void out_dec( fmt, n ) const char fmt[]; int n; { printf( fmt, n ); out_line_count( fmt ); } void out_dec2( fmt, n1, n2 ) const char fmt[]; int n1, n2; { printf( fmt, n1, n2 ); out_line_count( fmt ); } void out_hex( fmt, x ) const char fmt[]; unsigned int x; { printf( fmt, x ); out_line_count( fmt ); } void out_line_count( str ) const char str[]; { register int i; for ( i = 0; str[i]; ++i ) if ( str[i] == '\n' ) ++out_linenum; } void out_str( fmt, str ) const char fmt[], str[]; { printf( fmt, str ); out_line_count( fmt ); out_line_count( str ); } void out_str3( fmt, s1, s2, s3 ) const char fmt[], s1[], s2[], s3[]; { printf( fmt, s1, s2, s3 ); out_line_count( fmt ); out_line_count( s1 ); out_line_count( s2 ); out_line_count( s3 ); } void out_str_dec( fmt, str, n ) const char fmt[], str[]; int n; { printf( fmt, str, n ); out_line_count( fmt ); out_line_count( str ); } void outc( c ) int c; { putc( c, stdout ); if ( c == '\n' ) ++out_linenum; } void outn( str ) const char str[]; { puts( str ); out_line_count( str ); ++out_linenum; } /* readable_form - return the the human-readable form of a character * * The returned string is in static storage. */ char *readable_form( c ) register int c; { static char rform[10]; if ( (c >= 0 && c < 32) || c >= 127 ) { switch ( c ) { case '\b': return "\\b"; case '\f': return "\\f"; case '\n': return "\\n"; case '\r': return "\\r"; case '\t': return "\\t"; #if __STDC__ case '\a': return "\\a"; case '\v': return "\\v"; #endif default: (void) sprintf( rform, "\\%.3o", (unsigned int) c ); return rform; } } else if ( c == ' ' ) return "' '"; else { rform[0] = c; rform[1] = '\0'; return rform; } } /* reallocate_array - increase the size of a dynamic array */ void *reallocate_array( array, size, element_size ) void *array; int size; size_t element_size; { register void *new_array; size_t num_bytes = element_size * size; new_array = flex_realloc( array, num_bytes ); if ( ! new_array ) flexfatal( _( "attempt to increase array size failed" ) ); return new_array; } /* skelout - write out one section of the skeleton file * * Description * Copies skelfile or skel array to stdout until a line beginning with * "%%" or EOF is found. */ void skelout() { char buf_storage[MAXLINE]; char *buf = buf_storage; int do_copy = 1; /* Loop pulling lines either from the skelfile, if we're using * one, or from the skel[] array. */ while ( skelfile ? (fgets( buf, MAXLINE, skelfile ) != NULL) : ((buf = (char *) skel[skel_ind++]) != 0) ) { /* copy from skel array */ if ( buf[0] == '%' ) { /* control line */ switch ( buf[1] ) { case '%': return; case '+': do_copy = C_plus_plus; break; case '-': do_copy = ! C_plus_plus; break; case '*': do_copy = 1; break; default: flexfatal( _( "bad line in skeleton file" ) ); } } else if ( do_copy ) { if ( skelfile ) /* Skeleton file reads include final * newline, skel[] array does not. */ out( buf ); else outn( buf ); } } } /* transition_struct_out - output a yy_trans_info structure * * outputs the yy_trans_info structure with the two elements, element_v and * element_n. Formats the output with spaces and carriage returns. */ void transition_struct_out( element_v, element_n ) int element_v, element_n; { out_dec2( " {%4d,%4d },", element_v, element_n ); datapos += TRANS_STRUCT_PRINT_LENGTH; if ( datapos >= 79 - TRANS_STRUCT_PRINT_LENGTH ) { outc( '\n' ); if ( ++dataline % 10 == 0 ) outc( '\n' ); datapos = 0; } } /* The following is only needed when building flex's parser using certain * broken versions of bison. */ void *yy_flex_xmalloc( size ) int size; { void *result = flex_alloc( (size_t) size ); if ( ! result ) flexfatal( _( "memory allocation failed in yy_flex_xmalloc()" ) ); return result; } /* zero_out - set a region of memory to 0 * * Sets region_ptr[0] through region_ptr[size_in_bytes - 1] to zero. */ void zero_out( region_ptr, size_in_bytes ) char *region_ptr; size_t size_in_bytes; { register char *rp, *rp_end; rp = region_ptr; rp_end = region_ptr + size_in_bytes; while ( rp < rp_end ) *rp++ = 0; } flex-2.5.4/nfa.c100444 75424 12 41641 6366531463 12000 0ustar kwzhstaff/* nfa - NFA construction routines */ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/nfa.c,v 2.17 95/03/04 16:11:42 vern Exp $ */ #include "flexdef.h" /* declare functions that have forward references */ int dupmachine PROTO((int)); void mkxtion PROTO((int, int)); /* add_accept - add an accepting state to a machine * * accepting_number becomes mach's accepting number. */ void add_accept( mach, accepting_number ) int mach, accepting_number; { /* Hang the accepting number off an epsilon state. if it is associated * with a state that has a non-epsilon out-transition, then the state * will accept BEFORE it makes that transition, i.e., one character * too soon. */ if ( transchar[finalst[mach]] == SYM_EPSILON ) accptnum[finalst[mach]] = accepting_number; else { int astate = mkstate( SYM_EPSILON ); accptnum[astate] = accepting_number; (void) link_machines( mach, astate ); } } /* copysingl - make a given number of copies of a singleton machine * * synopsis * * newsng = copysingl( singl, num ); * * newsng - a new singleton composed of num copies of singl * singl - a singleton machine * num - the number of copies of singl to be present in newsng */ int copysingl( singl, num ) int singl, num; { int copy, i; copy = mkstate( SYM_EPSILON ); for ( i = 1; i <= num; ++i ) copy = link_machines( copy, dupmachine( singl ) ); return copy; } /* dumpnfa - debugging routine to write out an nfa */ void dumpnfa( state1 ) int state1; { int sym, tsp1, tsp2, anum, ns; fprintf( stderr, _( "\n\n********** beginning dump of nfa with start state %d\n" ), state1 ); /* We probably should loop starting at firstst[state1] and going to * lastst[state1], but they're not maintained properly when we "or" * all of the rules together. So we use our knowledge that the machine * starts at state 1 and ends at lastnfa. */ /* for ( ns = firstst[state1]; ns <= lastst[state1]; ++ns ) */ for ( ns = 1; ns <= lastnfa; ++ns ) { fprintf( stderr, _( "state # %4d\t" ), ns ); sym = transchar[ns]; tsp1 = trans1[ns]; tsp2 = trans2[ns]; anum = accptnum[ns]; fprintf( stderr, "%3d: %4d, %4d", sym, tsp1, tsp2 ); if ( anum != NIL ) fprintf( stderr, " [%d]", anum ); fprintf( stderr, "\n" ); } fprintf( stderr, _( "********** end of dump\n" ) ); } /* dupmachine - make a duplicate of a given machine * * synopsis * * copy = dupmachine( mach ); * * copy - holds duplicate of mach * mach - machine to be duplicated * * note that the copy of mach is NOT an exact duplicate; rather, all the * transition states values are adjusted so that the copy is self-contained, * as the original should have been. * * also note that the original MUST be contiguous, with its low and high * states accessible by the arrays firstst and lastst */ int dupmachine( mach ) int mach; { int i, init, state_offset; int state = 0; int last = lastst[mach]; for ( i = firstst[mach]; i <= last; ++i ) { state = mkstate( transchar[i] ); if ( trans1[i] != NO_TRANSITION ) { mkxtion( finalst[state], trans1[i] + state - i ); if ( transchar[i] == SYM_EPSILON && trans2[i] != NO_TRANSITION ) mkxtion( finalst[state], trans2[i] + state - i ); } accptnum[state] = accptnum[i]; } if ( state == 0 ) flexfatal( _( "empty machine in dupmachine()" ) ); state_offset = state - i + 1; init = mach + state_offset; firstst[init] = firstst[mach] + state_offset; finalst[init] = finalst[mach] + state_offset; lastst[init] = lastst[mach] + state_offset; return init; } /* finish_rule - finish up the processing for a rule * * An accepting number is added to the given machine. If variable_trail_rule * is true then the rule has trailing context and both the head and trail * are variable size. Otherwise if headcnt or trailcnt is non-zero then * the machine recognizes a pattern with trailing context and headcnt is * the number of characters in the matched part of the pattern, or zero * if the matched part has variable length. trailcnt is the number of * trailing context characters in the pattern, or zero if the trailing * context has variable length. */ void finish_rule( mach, variable_trail_rule, headcnt, trailcnt ) int mach, variable_trail_rule, headcnt, trailcnt; { char action_text[MAXLINE]; add_accept( mach, num_rules ); /* We did this in new_rule(), but it often gets the wrong * number because we do it before we start parsing the current rule. */ rule_linenum[num_rules] = linenum; /* If this is a continued action, then the line-number has already * been updated, giving us the wrong number. */ if ( continued_action ) --rule_linenum[num_rules]; sprintf( action_text, "case %d:\n", num_rules ); add_action( action_text ); if ( variable_trail_rule ) { rule_type[num_rules] = RULE_VARIABLE; if ( performance_report > 0 ) fprintf( stderr, _( "Variable trailing context rule at line %d\n" ), rule_linenum[num_rules] ); variable_trailing_context_rules = true; } else { rule_type[num_rules] = RULE_NORMAL; if ( headcnt > 0 || trailcnt > 0 ) { /* Do trailing context magic to not match the trailing * characters. */ char *scanner_cp = "yy_c_buf_p = yy_cp"; char *scanner_bp = "yy_bp"; add_action( "*yy_cp = yy_hold_char; /* undo effects of setting up yytext */\n" ); if ( headcnt > 0 ) { sprintf( action_text, "%s = %s + %d;\n", scanner_cp, scanner_bp, headcnt ); add_action( action_text ); } else { sprintf( action_text, "%s -= %d;\n", scanner_cp, trailcnt ); add_action( action_text ); } add_action( "YY_DO_BEFORE_ACTION; /* set up yytext again */\n" ); } } /* Okay, in the action code at this point yytext and yyleng have * their proper final values for this rule, so here's the point * to do any user action. But don't do it for continued actions, * as that'll result in multiple YY_RULE_SETUP's. */ if ( ! continued_action ) add_action( "YY_RULE_SETUP\n" ); line_directive_out( (FILE *) 0, 1 ); } /* link_machines - connect two machines together * * synopsis * * new = link_machines( first, last ); * * new - a machine constructed by connecting first to last * first - the machine whose successor is to be last * last - the machine whose predecessor is to be first * * note: this routine concatenates the machine first with the machine * last to produce a machine new which will pattern-match first first * and then last, and will fail if either of the sub-patterns fails. * FIRST is set to new by the operation. last is unmolested. */ int link_machines( first, last ) int first, last; { if ( first == NIL ) return last; else if ( last == NIL ) return first; else { mkxtion( finalst[first], last ); finalst[first] = finalst[last]; lastst[first] = MAX( lastst[first], lastst[last] ); firstst[first] = MIN( firstst[first], firstst[last] ); return first; } } /* mark_beginning_as_normal - mark each "beginning" state in a machine * as being a "normal" (i.e., not trailing context- * associated) states * * The "beginning" states are the epsilon closure of the first state */ void mark_beginning_as_normal( mach ) register int mach; { switch ( state_type[mach] ) { case STATE_NORMAL: /* Oh, we've already visited here. */ return; case STATE_TRAILING_CONTEXT: state_type[mach] = STATE_NORMAL; if ( transchar[mach] == SYM_EPSILON ) { if ( trans1[mach] != NO_TRANSITION ) mark_beginning_as_normal( trans1[mach] ); if ( trans2[mach] != NO_TRANSITION ) mark_beginning_as_normal( trans2[mach] ); } break; default: flexerror( _( "bad state type in mark_beginning_as_normal()" ) ); break; } } /* mkbranch - make a machine that branches to two machines * * synopsis * * branch = mkbranch( first, second ); * * branch - a machine which matches either first's pattern or second's * first, second - machines whose patterns are to be or'ed (the | operator) * * Note that first and second are NEITHER destroyed by the operation. Also, * the resulting machine CANNOT be used with any other "mk" operation except * more mkbranch's. Compare with mkor() */ int mkbranch( first, second ) int first, second; { int eps; if ( first == NO_TRANSITION ) return second; else if ( second == NO_TRANSITION ) return first; eps = mkstate( SYM_EPSILON ); mkxtion( eps, first ); mkxtion( eps, second ); return eps; } /* mkclos - convert a machine into a closure * * synopsis * new = mkclos( state ); * * new - a new state which matches the closure of "state" */ int mkclos( state ) int state; { return mkopt( mkposcl( state ) ); } /* mkopt - make a machine optional * * synopsis * * new = mkopt( mach ); * * new - a machine which optionally matches whatever mach matched * mach - the machine to make optional * * notes: * 1. mach must be the last machine created * 2. mach is destroyed by the call */ int mkopt( mach ) int mach; { int eps; if ( ! SUPER_FREE_EPSILON(finalst[mach]) ) { eps = mkstate( SYM_EPSILON ); mach = link_machines( mach, eps ); } /* Can't skimp on the following if FREE_EPSILON(mach) is true because * some state interior to "mach" might point back to the beginning * for a closure. */ eps = mkstate( SYM_EPSILON ); mach = link_machines( eps, mach ); mkxtion( mach, finalst[mach] ); return mach; } /* mkor - make a machine that matches either one of two machines * * synopsis * * new = mkor( first, second ); * * new - a machine which matches either first's pattern or second's * first, second - machines whose patterns are to be or'ed (the | operator) * * note that first and second are both destroyed by the operation * the code is rather convoluted because an attempt is made to minimize * the number of epsilon states needed */ int mkor( first, second ) int first, second; { int eps, orend; if ( first == NIL ) return second; else if ( second == NIL ) return first; else { /* See comment in mkopt() about why we can't use the first * state of "first" or "second" if they satisfy "FREE_EPSILON". */ eps = mkstate( SYM_EPSILON ); first = link_machines( eps, first ); mkxtion( first, second ); if ( SUPER_FREE_EPSILON(finalst[first]) && accptnum[finalst[first]] == NIL ) { orend = finalst[first]; mkxtion( finalst[second], orend ); } else if ( SUPER_FREE_EPSILON(finalst[second]) && accptnum[finalst[second]] == NIL ) { orend = finalst[second]; mkxtion( finalst[first], orend ); } else { eps = mkstate( SYM_EPSILON ); first = link_machines( first, eps ); orend = finalst[first]; mkxtion( finalst[second], orend ); } } finalst[first] = orend; return first; } /* mkposcl - convert a machine into a positive closure * * synopsis * new = mkposcl( state ); * * new - a machine matching the positive closure of "state" */ int mkposcl( state ) int state; { int eps; if ( SUPER_FREE_EPSILON(finalst[state]) ) { mkxtion( finalst[state], state ); return state; } else { eps = mkstate( SYM_EPSILON ); mkxtion( eps, state ); return link_machines( state, eps ); } } /* mkrep - make a replicated machine * * synopsis * new = mkrep( mach, lb, ub ); * * new - a machine that matches whatever "mach" matched from "lb" * number of times to "ub" number of times * * note * if "ub" is INFINITY then "new" matches "lb" or more occurrences of "mach" */ int mkrep( mach, lb, ub ) int mach, lb, ub; { int base_mach, tail, copy, i; base_mach = copysingl( mach, lb - 1 ); if ( ub == INFINITY ) { copy = dupmachine( mach ); mach = link_machines( mach, link_machines( base_mach, mkclos( copy ) ) ); } else { tail = mkstate( SYM_EPSILON ); for ( i = lb; i < ub; ++i ) { copy = dupmachine( mach ); tail = mkopt( link_machines( copy, tail ) ); } mach = link_machines( mach, link_machines( base_mach, tail ) ); } return mach; } /* mkstate - create a state with a transition on a given symbol * * synopsis * * state = mkstate( sym ); * * state - a new state matching sym * sym - the symbol the new state is to have an out-transition on * * note that this routine makes new states in ascending order through the * state array (and increments LASTNFA accordingly). The routine DUPMACHINE * relies on machines being made in ascending order and that they are * CONTIGUOUS. Change it and you will have to rewrite DUPMACHINE (kludge * that it admittedly is) */ int mkstate( sym ) int sym; { if ( ++lastnfa >= current_mns ) { if ( (current_mns += MNS_INCREMENT) >= MAXIMUM_MNS ) lerrif( _( "input rules are too complicated (>= %d NFA states)" ), current_mns ); ++num_reallocs; firstst = reallocate_integer_array( firstst, current_mns ); lastst = reallocate_integer_array( lastst, current_mns ); finalst = reallocate_integer_array( finalst, current_mns ); transchar = reallocate_integer_array( transchar, current_mns ); trans1 = reallocate_integer_array( trans1, current_mns ); trans2 = reallocate_integer_array( trans2, current_mns ); accptnum = reallocate_integer_array( accptnum, current_mns ); assoc_rule = reallocate_integer_array( assoc_rule, current_mns ); state_type = reallocate_integer_array( state_type, current_mns ); } firstst[lastnfa] = lastnfa; finalst[lastnfa] = lastnfa; lastst[lastnfa] = lastnfa; transchar[lastnfa] = sym; trans1[lastnfa] = NO_TRANSITION; trans2[lastnfa] = NO_TRANSITION; accptnum[lastnfa] = NIL; assoc_rule[lastnfa] = num_rules; state_type[lastnfa] = current_state_type; /* Fix up equivalence classes base on this transition. Note that any * character which has its own transition gets its own equivalence * class. Thus only characters which are only in character classes * have a chance at being in the same equivalence class. E.g. "a|b" * puts 'a' and 'b' into two different equivalence classes. "[ab]" * puts them in the same equivalence class (barring other differences * elsewhere in the input). */ if ( sym < 0 ) { /* We don't have to update the equivalence classes since * that was already done when the ccl was created for the * first time. */ } else if ( sym == SYM_EPSILON ) ++numeps; else { check_char( sym ); if ( useecs ) /* Map NUL's to csize. */ mkechar( sym ? sym : csize, nextecm, ecgroup ); } return lastnfa; } /* mkxtion - make a transition from one state to another * * synopsis * * mkxtion( statefrom, stateto ); * * statefrom - the state from which the transition is to be made * stateto - the state to which the transition is to be made */ void mkxtion( statefrom, stateto ) int statefrom, stateto; { if ( trans1[statefrom] == NO_TRANSITION ) trans1[statefrom] = stateto; else if ( (transchar[statefrom] != SYM_EPSILON) || (trans2[statefrom] != NO_TRANSITION) ) flexfatal( _( "found too many transitions in mkxtion()" ) ); else { /* second out-transition for an epsilon state */ ++eps2; trans2[statefrom] = stateto; } } /* new_rule - initialize for a new rule */ void new_rule() { if ( ++num_rules >= current_max_rules ) { ++num_reallocs; current_max_rules += MAX_RULES_INCREMENT; rule_type = reallocate_integer_array( rule_type, current_max_rules ); rule_linenum = reallocate_integer_array( rule_linenum, current_max_rules ); rule_useful = reallocate_integer_array( rule_useful, current_max_rules ); } if ( num_rules > MAX_RULE ) lerrif( _( "too many rules (> %d)!" ), MAX_RULE ); rule_linenum[num_rules] = linenum; rule_useful[num_rules] = false; } flex-2.5.4/parse.y100444 75424 12 42620 6366532746 12377 0ustar kwzhstaff/* parse.y - parser for flex input */ %token CHAR NUMBER SECTEND SCDECL XSCDECL NAME PREVCCL EOF_OP %token OPTION_OP OPT_OUTFILE OPT_PREFIX OPT_YYCLASS %token CCE_ALNUM CCE_ALPHA CCE_BLANK CCE_CNTRL CCE_DIGIT CCE_GRAPH %token CCE_LOWER CCE_PRINT CCE_PUNCT CCE_SPACE CCE_UPPER CCE_XDIGIT %{ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/parse.y,v 2.28 95/04/21 11:51:51 vern Exp $ */ /* Some versions of bison are broken in that they use alloca() but don't * declare it properly. The following is the patented (just kidding!) * #ifdef chud to fix the problem, courtesy of Francois Pinard. */ #ifdef YYBISON /* AIX requires this to be the first thing in the file. What a piece. */ # ifdef _AIX #pragma alloca # endif #endif #include "flexdef.h" /* The remainder of the alloca() cruft has to come after including flexdef.h, * so HAVE_ALLOCA_H is (possibly) defined. */ #ifdef YYBISON # ifdef __GNUC__ # ifndef alloca # define alloca __builtin_alloca # endif # else # if HAVE_ALLOCA_H # include # else # ifdef __hpux void *alloca (); # else # ifdef __TURBOC__ # include # else char *alloca (); # endif # endif # endif # endif #endif /* Bletch, ^^^^ that was ugly! */ int pat, scnum, eps, headcnt, trailcnt, anyccl, lastchar, i, rulelen; int trlcontxt, xcluflg, currccl, cclsorted, varlength, variable_trail_rule; int *scon_stk; int scon_stk_ptr; static int madeany = false; /* whether we've made the '.' character class */ int previous_continued_action; /* whether the previous rule's action was '|' */ /* Expand a POSIX character class expression. */ #define CCL_EXPR(func) \ { \ int c; \ for ( c = 0; c < csize; ++c ) \ if ( isascii(c) && func(c) ) \ ccladd( currccl, c ); \ } /* While POSIX defines isblank(), it's not ANSI C. */ #define IS_BLANK(c) ((c) == ' ' || (c) == '\t') /* On some over-ambitious machines, such as DEC Alpha's, the default * token type is "long" instead of "int"; this leads to problems with * declaring yylval in flexdef.h. But so far, all the yacc's I've seen * wrap their definitions of YYSTYPE with "#ifndef YYSTYPE"'s, so the * following should ensure that the default token type is "int". */ #define YYSTYPE int %} %% goal : initlex sect1 sect1end sect2 initforrule { /* add default rule */ int def_rule; pat = cclinit(); cclnegate( pat ); def_rule = mkstate( -pat ); /* Remember the number of the default rule so we * don't generate "can't match" warnings for it. */ default_rule = num_rules; finish_rule( def_rule, false, 0, 0 ); for ( i = 1; i <= lastsc; ++i ) scset[i] = mkbranch( scset[i], def_rule ); if ( spprdflt ) add_action( "YY_FATAL_ERROR( \"flex scanner jammed\" )" ); else add_action( "ECHO" ); add_action( ";\n\tYY_BREAK\n" ); } ; initlex : { /* initialize for processing rules */ /* Create default DFA start condition. */ scinstal( "INITIAL", false ); } ; sect1 : sect1 startconddecl namelist1 | sect1 options | | error { synerr( "unknown error processing section 1" ); } ; sect1end : SECTEND { check_options(); scon_stk = allocate_integer_array( lastsc + 1 ); scon_stk_ptr = 0; } ; startconddecl : SCDECL { xcluflg = false; } | XSCDECL { xcluflg = true; } ; namelist1 : namelist1 NAME { scinstal( nmstr, xcluflg ); } | NAME { scinstal( nmstr, xcluflg ); } | error { synerr( "bad start condition list" ); } ; options : OPTION_OP optionlist ; optionlist : optionlist option | ; option : OPT_OUTFILE '=' NAME { outfilename = copy_string( nmstr ); did_outfilename = 1; } | OPT_PREFIX '=' NAME { prefix = copy_string( nmstr ); } | OPT_YYCLASS '=' NAME { yyclass = copy_string( nmstr ); } ; sect2 : sect2 scon initforrule flexrule '\n' { scon_stk_ptr = $2; } | sect2 scon '{' sect2 '}' { scon_stk_ptr = $2; } | ; initforrule : { /* Initialize for a parse of one rule. */ trlcontxt = variable_trail_rule = varlength = false; trailcnt = headcnt = rulelen = 0; current_state_type = STATE_NORMAL; previous_continued_action = continued_action; in_rule = true; new_rule(); } ; flexrule : '^' rule { pat = $2; finish_rule( pat, variable_trail_rule, headcnt, trailcnt ); if ( scon_stk_ptr > 0 ) { for ( i = 1; i <= scon_stk_ptr; ++i ) scbol[scon_stk[i]] = mkbranch( scbol[scon_stk[i]], pat ); } else { /* Add to all non-exclusive start conditions, * including the default (0) start condition. */ for ( i = 1; i <= lastsc; ++i ) if ( ! scxclu[i] ) scbol[i] = mkbranch( scbol[i], pat ); } if ( ! bol_needed ) { bol_needed = true; if ( performance_report > 1 ) pinpoint_message( "'^' operator results in sub-optimal performance" ); } } | rule { pat = $1; finish_rule( pat, variable_trail_rule, headcnt, trailcnt ); if ( scon_stk_ptr > 0 ) { for ( i = 1; i <= scon_stk_ptr; ++i ) scset[scon_stk[i]] = mkbranch( scset[scon_stk[i]], pat ); } else { for ( i = 1; i <= lastsc; ++i ) if ( ! scxclu[i] ) scset[i] = mkbranch( scset[i], pat ); } } | EOF_OP { if ( scon_stk_ptr > 0 ) build_eof_action(); else { /* This EOF applies to all start conditions * which don't already have EOF actions. */ for ( i = 1; i <= lastsc; ++i ) if ( ! sceof[i] ) scon_stk[++scon_stk_ptr] = i; if ( scon_stk_ptr == 0 ) warn( "all start conditions already have <> rules" ); else build_eof_action(); } } | error { synerr( "unrecognized rule" ); } ; scon_stk_ptr : { $$ = scon_stk_ptr; } ; scon : '<' scon_stk_ptr namelist2 '>' { $$ = $2; } | '<' '*' '>' { $$ = scon_stk_ptr; for ( i = 1; i <= lastsc; ++i ) { int j; for ( j = 1; j <= scon_stk_ptr; ++j ) if ( scon_stk[j] == i ) break; if ( j > scon_stk_ptr ) scon_stk[++scon_stk_ptr] = i; } } | { $$ = scon_stk_ptr; } ; namelist2 : namelist2 ',' sconname | sconname | error { synerr( "bad start condition list" ); } ; sconname : NAME { if ( (scnum = sclookup( nmstr )) == 0 ) format_pinpoint_message( "undeclared start condition %s", nmstr ); else { for ( i = 1; i <= scon_stk_ptr; ++i ) if ( scon_stk[i] == scnum ) { format_warn( "<%s> specified twice", scname[scnum] ); break; } if ( i > scon_stk_ptr ) scon_stk[++scon_stk_ptr] = scnum; } } ; rule : re2 re { if ( transchar[lastst[$2]] != SYM_EPSILON ) /* Provide final transition \now/ so it * will be marked as a trailing context * state. */ $2 = link_machines( $2, mkstate( SYM_EPSILON ) ); mark_beginning_as_normal( $2 ); current_state_type = STATE_NORMAL; if ( previous_continued_action ) { /* We need to treat this as variable trailing * context so that the backup does not happen * in the action but before the action switch * statement. If the backup happens in the * action, then the rules "falling into" this * one's action will *also* do the backup, * erroneously. */ if ( ! varlength || headcnt != 0 ) warn( "trailing context made variable due to preceding '|' action" ); /* Mark as variable. */ varlength = true; headcnt = 0; } if ( lex_compat || (varlength && headcnt == 0) ) { /* variable trailing context rule */ /* Mark the first part of the rule as the * accepting "head" part of a trailing * context rule. * * By the way, we didn't do this at the * beginning of this production because back * then current_state_type was set up for a * trail rule, and add_accept() can create * a new state ... */ add_accept( $1, num_rules | YY_TRAILING_HEAD_MASK ); variable_trail_rule = true; } else trailcnt = rulelen; $$ = link_machines( $1, $2 ); } | re2 re '$' { synerr( "trailing context used twice" ); } | re '$' { headcnt = 0; trailcnt = 1; rulelen = 1; varlength = false; current_state_type = STATE_TRAILING_CONTEXT; if ( trlcontxt ) { synerr( "trailing context used twice" ); $$ = mkstate( SYM_EPSILON ); } else if ( previous_continued_action ) { /* See the comment in the rule for "re2 re" * above. */ warn( "trailing context made variable due to preceding '|' action" ); varlength = true; } if ( lex_compat || varlength ) { /* Again, see the comment in the rule for * "re2 re" above. */ add_accept( $1, num_rules | YY_TRAILING_HEAD_MASK ); variable_trail_rule = true; } trlcontxt = true; eps = mkstate( SYM_EPSILON ); $$ = link_machines( $1, link_machines( eps, mkstate( '\n' ) ) ); } | re { $$ = $1; if ( trlcontxt ) { if ( lex_compat || (varlength && headcnt == 0) ) /* Both head and trail are * variable-length. */ variable_trail_rule = true; else trailcnt = rulelen; } } ; re : re '|' series { varlength = true; $$ = mkor( $1, $3 ); } | series { $$ = $1; } ; re2 : re '/' { /* This rule is written separately so the * reduction will occur before the trailing * series is parsed. */ if ( trlcontxt ) synerr( "trailing context used twice" ); else trlcontxt = true; if ( varlength ) /* We hope the trailing context is * fixed-length. */ varlength = false; else headcnt = rulelen; rulelen = 0; current_state_type = STATE_TRAILING_CONTEXT; $$ = $1; } ; series : series singleton { /* This is where concatenation of adjacent patterns * gets done. */ $$ = link_machines( $1, $2 ); } | singleton { $$ = $1; } ; singleton : singleton '*' { varlength = true; $$ = mkclos( $1 ); } | singleton '+' { varlength = true; $$ = mkposcl( $1 ); } | singleton '?' { varlength = true; $$ = mkopt( $1 ); } | singleton '{' NUMBER ',' NUMBER '}' { varlength = true; if ( $3 > $5 || $3 < 0 ) { synerr( "bad iteration values" ); $$ = $1; } else { if ( $3 == 0 ) { if ( $5 <= 0 ) { synerr( "bad iteration values" ); $$ = $1; } else $$ = mkopt( mkrep( $1, 1, $5 ) ); } else $$ = mkrep( $1, $3, $5 ); } } | singleton '{' NUMBER ',' '}' { varlength = true; if ( $3 <= 0 ) { synerr( "iteration value must be positive" ); $$ = $1; } else $$ = mkrep( $1, $3, INFINITY ); } | singleton '{' NUMBER '}' { /* The singleton could be something like "(foo)", * in which case we have no idea what its length * is, so we punt here. */ varlength = true; if ( $3 <= 0 ) { synerr( "iteration value must be positive" ); $$ = $1; } else $$ = link_machines( $1, copysingl( $1, $3 - 1 ) ); } | '.' { if ( ! madeany ) { /* Create the '.' character class. */ anyccl = cclinit(); ccladd( anyccl, '\n' ); cclnegate( anyccl ); if ( useecs ) mkeccl( ccltbl + cclmap[anyccl], ccllen[anyccl], nextecm, ecgroup, csize, csize ); madeany = true; } ++rulelen; $$ = mkstate( -anyccl ); } | fullccl { if ( ! cclsorted ) /* Sort characters for fast searching. We * use a shell sort since this list could * be large. */ cshell( ccltbl + cclmap[$1], ccllen[$1], true ); if ( useecs ) mkeccl( ccltbl + cclmap[$1], ccllen[$1], nextecm, ecgroup, csize, csize ); ++rulelen; $$ = mkstate( -$1 ); } | PREVCCL { ++rulelen; $$ = mkstate( -$1 ); } | '"' string '"' { $$ = $2; } | '(' re ')' { $$ = $2; } | CHAR { ++rulelen; if ( caseins && $1 >= 'A' && $1 <= 'Z' ) $1 = clower( $1 ); $$ = mkstate( $1 ); } ; fullccl : '[' ccl ']' { $$ = $2; } | '[' '^' ccl ']' { cclnegate( $3 ); $$ = $3; } ; ccl : ccl CHAR '-' CHAR { if ( caseins ) { if ( $2 >= 'A' && $2 <= 'Z' ) $2 = clower( $2 ); if ( $4 >= 'A' && $4 <= 'Z' ) $4 = clower( $4 ); } if ( $2 > $4 ) synerr( "negative range in character class" ); else { for ( i = $2; i <= $4; ++i ) ccladd( $1, i ); /* Keep track if this ccl is staying in * alphabetical order. */ cclsorted = cclsorted && ($2 > lastchar); lastchar = $4; } $$ = $1; } | ccl CHAR { if ( caseins && $2 >= 'A' && $2 <= 'Z' ) $2 = clower( $2 ); ccladd( $1, $2 ); cclsorted = cclsorted && ($2 > lastchar); lastchar = $2; $$ = $1; } | ccl ccl_expr { /* Too hard to properly maintain cclsorted. */ cclsorted = false; $$ = $1; } | { cclsorted = true; lastchar = 0; currccl = $$ = cclinit(); } ; ccl_expr: CCE_ALNUM { CCL_EXPR(isalnum) } | CCE_ALPHA { CCL_EXPR(isalpha) } | CCE_BLANK { CCL_EXPR(IS_BLANK) } | CCE_CNTRL { CCL_EXPR(iscntrl) } | CCE_DIGIT { CCL_EXPR(isdigit) } | CCE_GRAPH { CCL_EXPR(isgraph) } | CCE_LOWER { CCL_EXPR(islower) } | CCE_PRINT { CCL_EXPR(isprint) } | CCE_PUNCT { CCL_EXPR(ispunct) } | CCE_SPACE { CCL_EXPR(isspace) } | CCE_UPPER { if ( caseins ) CCL_EXPR(islower) else CCL_EXPR(isupper) } | CCE_XDIGIT { CCL_EXPR(isxdigit) } ; string : string CHAR { if ( caseins && $2 >= 'A' && $2 <= 'Z' ) $2 = clower( $2 ); ++rulelen; $$ = link_machines( $1, mkstate( $2 ) ); } | { $$ = mkstate( SYM_EPSILON ); } ; %% /* build_eof_action - build the "<>" action for the active start * conditions */ void build_eof_action() { register int i; char action_text[MAXLINE]; for ( i = 1; i <= scon_stk_ptr; ++i ) { if ( sceof[scon_stk[i]] ) format_pinpoint_message( "multiple <> rules for start condition %s", scname[scon_stk[i]] ); else { sceof[scon_stk[i]] = true; sprintf( action_text, "case YY_STATE_EOF(%s):\n", scname[scon_stk[i]] ); add_action( action_text ); } } line_directive_out( (FILE *) 0, 1 ); /* This isn't a normal rule after all - don't count it as * such, so we don't have any holes in the rule numbering * (which make generating "rule can never match" warnings * more difficult. */ --num_rules; ++num_eof_rules; } /* format_synerr - write out formatted syntax error */ void format_synerr( msg, arg ) char msg[], arg[]; { char errmsg[MAXLINE]; (void) sprintf( errmsg, msg, arg ); synerr( errmsg ); } /* synerr - report a syntax error */ void synerr( str ) char str[]; { syntaxerror = true; pinpoint_message( str ); } /* format_warn - write out formatted warning */ void format_warn( msg, arg ) char msg[], arg[]; { char warn_msg[MAXLINE]; (void) sprintf( warn_msg, msg, arg ); warn( warn_msg ); } /* warn - report a warning, unless -w was given */ void warn( str ) char str[]; { line_warning( str, linenum ); } /* format_pinpoint_message - write out a message formatted with one string, * pinpointing its location */ void format_pinpoint_message( msg, arg ) char msg[], arg[]; { char errmsg[MAXLINE]; (void) sprintf( errmsg, msg, arg ); pinpoint_message( errmsg ); } /* pinpoint_message - write out a message, pinpointing its location */ void pinpoint_message( str ) char str[]; { line_pinpoint( str, linenum ); } /* line_warning - report a warning at a given line, unless -w was given */ void line_warning( str, line ) char str[]; int line; { char warning[MAXLINE]; if ( ! nowarn ) { sprintf( warning, "warning, %s", str ); line_pinpoint( warning, line ); } } /* line_pinpoint - write out a message, pinpointing it at the given line */ void line_pinpoint( str, line ) char str[]; int line; { fprintf( stderr, "\"%s\", line %d: %s\n", infilename, line, str ); } /* yyerror - eat up an error message from the parser; * currently, messages are ignore */ void yyerror( msg ) char msg[]; { } flex-2.5.4/scan.l100444 75424 12 35474 6366532775 12207 0ustar kwzhstaff/* scan.l - scanner for flex input */ %{ /*- * Copyright (c) 1990 The Regents of the University of California. * All rights reserved. * * This code is derived from software contributed to Berkeley by * Vern Paxson. * * The United States Government has rights in this work pursuant * to contract no. DE-AC03-76SF00098 between the United States * Department of Energy and the University of California. * * Redistribution and use in source and binary forms with or without * modification are permitted provided that: (1) source distributions retain * this entire copyright notice and comment, and (2) distributions including * binaries display the following acknowledgement: ``This product includes * software developed by the University of California, Berkeley and its * contributors'' in the documentation or other materials provided with the * distribution and in all advertising materials mentioning features or use * of this software. Neither the name of the University nor the names of * its contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. */ /* $Header: /home/daffy/u0/vern/flex/RCS/scan.l,v 2.56 95/04/24 12:17:19 vern Exp $ */ #include "flexdef.h" #include "parse.h" #define ACTION_ECHO add_action( yytext ) #define ACTION_IFDEF(def, should_define) \ { \ if ( should_define ) \ action_define( def, 1 ); \ } #define MARK_END_OF_PROLOG mark_prolog(); #define YY_DECL \ int flexscan() #define RETURNCHAR \ yylval = (unsigned char) yytext[0]; \ return CHAR; #define RETURNNAME \ strcpy( nmstr, yytext ); \ return NAME; #define PUT_BACK_STRING(str, start) \ for ( i = strlen( str ) - 1; i >= start; --i ) \ unput((str)[i]) #define CHECK_REJECT(str) \ if ( all_upper( str ) ) \ reject = true; #define CHECK_YYMORE(str) \ if ( all_lower( str ) ) \ yymore_used = true; %} %option caseless nodefault outfile="scan.c" stack noyy_top_state %option nostdinit %x SECT2 SECT2PROLOG SECT3 CODEBLOCK PICKUPDEF SC CARETISBOL NUM QUOTE %x FIRSTCCL CCL ACTION RECOVER COMMENT ACTION_STRING PERCENT_BRACE_ACTION %x OPTION LINEDIR WS [[:blank:]]+ OPTWS [[:blank:]]* NOT_WS [^[:blank:]\n] NL \r?\n NAME ([[:alpha:]_][[:alnum:]_-]*) NOT_NAME [^[:alpha:]_*\n]+ SCNAME {NAME} ESCSEQ (\\([^\n]|[0-7]{1,3}|x[[:xdigit:]]{1,2})) FIRST_CCL_CHAR ([^\\\n]|{ESCSEQ}) CCL_CHAR ([^\\\n\]]|{ESCSEQ}) CCL_EXPR ("[:"[[:alpha:]]+":]") LEXOPT [aceknopr] %% static int bracelevel, didadef, indented_code; static int doing_rule_action = false; static int option_sense; int doing_codeblock = false; int i; Char nmdef[MAXLINE], myesc(); { ^{WS} indented_code = true; BEGIN(CODEBLOCK); ^"/*" ACTION_ECHO; yy_push_state( COMMENT ); ^#{OPTWS}line{WS} yy_push_state( LINEDIR ); ^"%s"{NAME}? return SCDECL; ^"%x"{NAME}? return XSCDECL; ^"%{".*{NL} { ++linenum; line_directive_out( (FILE *) 0, 1 ); indented_code = false; BEGIN(CODEBLOCK); } {WS} /* discard */ ^"%%".* { sectnum = 2; bracelevel = 0; mark_defs1(); line_directive_out( (FILE *) 0, 1 ); BEGIN(SECT2PROLOG); return SECTEND; } ^"%pointer".*{NL} yytext_is_array = false; ++linenum; ^"%array".*{NL} yytext_is_array = true; ++linenum; ^"%option" BEGIN(OPTION); return OPTION_OP; ^"%"{LEXOPT}{OPTWS}[[:digit:]]*{OPTWS}{NL} ++linenum; /* ignore */ ^"%"{LEXOPT}{WS}.*{NL} ++linenum; /* ignore */ ^"%"[^sxaceknopr{}].* synerr( _( "unrecognized '%' directive" ) ); ^{NAME} { strcpy( nmstr, yytext ); didadef = false; BEGIN(PICKUPDEF); } {SCNAME} RETURNNAME; ^{OPTWS}{NL} ++linenum; /* allows blank lines in section 1 */ {OPTWS}{NL} ACTION_ECHO; ++linenum; /* maybe end of comment line */ } { "*/" ACTION_ECHO; yy_pop_state(); "*" ACTION_ECHO; [^*\n]+ ACTION_ECHO; [^*\n]*{NL} ++linenum; ACTION_ECHO; } { \n yy_pop_state(); [[:digit:]]+ linenum = myctoi( yytext ); \"[^"\n]*\" { flex_free( (void *) infilename ); infilename = copy_string( yytext + 1 ); infilename[strlen( infilename ) - 1] = '\0'; } . /* ignore spurious characters */ } { ^"%}".*{NL} ++linenum; BEGIN(INITIAL); {NAME}|{NOT_NAME}|. ACTION_ECHO; {NL} { ++linenum; ACTION_ECHO; if ( indented_code ) BEGIN(INITIAL); } } { {WS} /* separates name and definition */ {NOT_WS}.* { strcpy( (char *) nmdef, yytext ); /* Skip trailing whitespace. */ for ( i = strlen( (char *) nmdef ) - 1; i >= 0 && (nmdef[i] == ' ' || nmdef[i] == '\t'); --i ) ; nmdef[i + 1] = '\0'; ndinstal( nmstr, nmdef ); didadef = true; } {NL} { if ( ! didadef ) synerr( _( "incomplete name definition" ) ); BEGIN(INITIAL); ++linenum; } }