704 lines
26 KiB
Plaintext
704 lines
26 KiB
Plaintext
Changes between release 2.4.7 (03Aug94) and release 2.4.6:
|
|
|
|
- Fixed serious bug in reading multiple files.
|
|
|
|
- Fixed bug in scanning NUL's.
|
|
|
|
- Fixed bug in input() returning 8-bit characters.
|
|
|
|
- Fixed bug in matching text with embedded NUL's when
|
|
using %array or lex compatibility.
|
|
|
|
- Fixed multiple invocations of YY_USER_ACTION when using '|'
|
|
continuation action.
|
|
|
|
- Minor prototyping fixes.
|
|
|
|
Changes between release 2.4.6 (04Jan94) and release 2.4.5:
|
|
|
|
- Linking with -lfl no longer required if your program includes
|
|
its own yywrap() and main() functions. (This change will cause
|
|
problems if you have a non-ANSI compiler on a system for which
|
|
sizeof(int) != sizeof(void*) or sizeof(int) != sizeof(size_t).)
|
|
|
|
- The use of 'extern "C++"' in FlexLexer.h has been modified to
|
|
get around an incompatibility with g++'s header files.
|
|
|
|
|
|
Changes between release 2.4.5 (11Dec93) and release 2.4.4:
|
|
|
|
- Fixed bug breaking C++ scanners that use REJECT or variable
|
|
trailing context.
|
|
|
|
- Fixed serious input problem for interactive scanners on
|
|
systems for which char is unsigned.
|
|
|
|
- Fixed bug in incorrectly treating '$' operator as variable
|
|
trailing context.
|
|
|
|
- Fixed bug in -CF table representation that could lead to
|
|
corrupt tables.
|
|
|
|
- Fixed fairly benign memory leak.
|
|
|
|
- Added `extern "C++"' wrapper to FlexLexer.h header. This
|
|
should overcome the g++ 2.5.X problems mentioned in the
|
|
NEWS for release 2.4.3.
|
|
|
|
- Changed #include of FlexLexer.h to use <> instead of "".
|
|
|
|
- Added feature to control whether the scanner attempts to
|
|
refill the input buffer once it's exhausted. This feature
|
|
will be documented in the 2.5 release.
|
|
|
|
|
|
Changes between release 2.4.4 (07Dec93) and release 2.4.3:
|
|
|
|
- Fixed two serious bugs in scanning 8-bit characters.
|
|
|
|
- Fixed bug in YY_USER_ACTION that caused it to be executed
|
|
inappropriately (on the scanner's own internal actions, and
|
|
with incorrect yytext/yyleng values).
|
|
|
|
- Fixed bug in pointing yyin at a new file and resuming scanning.
|
|
|
|
- Portability fix regarding min/max/abs macros conflicting with
|
|
function definitions in standard header files.
|
|
|
|
- Added a virtual LexerError() method to the C++ yyFlexLexer class
|
|
for reporting error messages instead of always using cerr.
|
|
|
|
- Added warning in flexdoc that the C++ scanning class is presently
|
|
experimental and subject to considerable change between major
|
|
releases.
|
|
|
|
|
|
Changes between release 2.4.3 (03Dec93) and release 2.4.2:
|
|
|
|
- Fixed bug causing fatal scanner messages to fail to print.
|
|
|
|
- Fixed things so FlexLexer.h can be included in other C++
|
|
sources. One side-effect of this change is that -+ and -CF
|
|
are now incompatible.
|
|
|
|
- libfl.a now supplies private versions of the the <string.h>/
|
|
<strings.h> string routines needed by flex and the scanners
|
|
it generates, to enhance portability to some BSD systems.
|
|
|
|
- More robust solution to 2.4.2's flexfatal() bug fix.
|
|
|
|
- Added ranlib of installed libfl.a.
|
|
|
|
- Some lint tweaks.
|
|
|
|
- NOTE: problems have been encountered attempting to build flex
|
|
C++ scanners using g++ version 2.5.X. The problem is due to an
|
|
unfortunate heuristic in g++ 2.5.X that attempts to discern between
|
|
C and C++ headers. Because FlexLexer.h is installed (by default)
|
|
in /usr/local/include and not /usr/local/lib/g++-include, g++ 2.5.X
|
|
decides that it's a C header :-(. So if you have problems, install
|
|
the header in /usr/local/lib/g++-include instead.
|
|
|
|
|
|
Changes between release 2.4.2 (01Dec93) and release 2.4.1:
|
|
|
|
- Fixed bug in libfl.a referring to non-existent "flexfatal" function.
|
|
|
|
- Modified to produce both compress'd and gzip'd tar files for
|
|
distributions (you probably don't care about this change!).
|
|
|
|
|
|
Changes between release 2.4.1 (30Nov93) and release 2.3.8:
|
|
|
|
- The new '-+' flag instructs flex to generate a C++ scanner class
|
|
(thanks to Kent Williams). flex writes an implementation of the
|
|
class defined in FlexLexer.h to lex.yy.cc. You may include
|
|
multiple scanner classes in your program using the -P flag. Note
|
|
that the scanner class also provides a mechanism for creating
|
|
reentrant scanners. The scanner class uses C++ streams for I/O
|
|
instead of FILE*'s (thanks to Tom Epperly). If the flex executable's
|
|
name ends in '+' then the '-+' flag is automatically on, so creating
|
|
a symlink or copy of "flex" to "flex++" results in a version of
|
|
flex that can be used exclusively for C++ scanners.
|
|
|
|
Note that without the '-+' flag, flex-generated scanners can still
|
|
be compiled using C++ compilers, though they use FILE*'s for I/O
|
|
instead of streams.
|
|
|
|
See the "GENERATING C++ SCANNERS" section of flexdoc for details.
|
|
|
|
- The new '-l' flag turns on maximum AT&T lex compatibility. In
|
|
particular, -l includes support for "yylineno" and makes yytext
|
|
be an array instead of a pointer. It does not, however, do away
|
|
with all incompatibilities. See the "INCOMPATIBILITIES WITH LEX
|
|
AND POSIX" section of flexdoc for details.
|
|
|
|
- The new '-P' option specifies a prefix to use other than "yy"
|
|
for the scanner's globally-visible variables, and for the
|
|
"lex.yy.c" filename. Using -P you can link together multiple
|
|
flex scanners in the same executable.
|
|
|
|
- The distribution includes a "texinfo" version of flexdoc.1,
|
|
contributed by Roland Pesch (thanks also to Marq Kole, who
|
|
contributed another version). It has not been brought up to
|
|
date, but reflects version 2.3. See MISC/flex.texinfo.
|
|
|
|
The flex distribution will soon include G.T. Nicol's flex
|
|
manual; he is presently bringing it up-to-date for version 2.4.
|
|
|
|
- yywrap() is now a function, and you now *must* link flex scanners
|
|
with libfl.a.
|
|
|
|
- Site-configuration is now done via an autoconf-generated
|
|
"configure" script contributed by Francois Pinard.
|
|
|
|
- Scanners now use fread() (or getc(), if interactive) and not
|
|
read() for input. A new "table compression" option, -Cr,
|
|
overrides this change and causes the scanner to use read()
|
|
(because read() is a bit faster than fread()). -f and -F
|
|
are now equivalent to -Cfr and -CFr; i.e., they imply the
|
|
-Cr option.
|
|
|
|
- In the blessed name of POSIX compliance, flex supports "%array"
|
|
and "%pointer" directives in the definitions (first) section of
|
|
the scanner specification. The former specifies that yytext
|
|
should be an array (of size YYLMAX), the latter, that it should
|
|
be a pointer. The array version of yytext is universally slower
|
|
than the pointer version, but has the advantage that its contents
|
|
remain unmodified across calls to input() and unput() (the pointer
|
|
version of yytext is, still, trashed by such calls).
|
|
|
|
"%array" cannot be used with the '-+' C++ scanner class option.
|
|
|
|
- The new '-Ca' option directs flex to trade off memory for
|
|
natural alignment when generating a scanner's tables. In
|
|
particular, table entries that would otherwise be "short"
|
|
become "long".
|
|
|
|
- The new '-h' option produces a summary of the flex flags.
|
|
|
|
- The new '-V' option reports the flex version number and exits.
|
|
|
|
- The new scanner macro YY_START returns an integer value
|
|
corresponding to the current start condition. You can return
|
|
to that start condition by passing the value to a subsequent
|
|
"BEGIN" action. You also can implement "start condition stacks"
|
|
by storing the values in an integer stack.
|
|
|
|
- You can now redefine macros such as YY_INPUT by just #define'ing
|
|
them to some other value in the first section of the flex input;
|
|
no need to first #undef them.
|
|
|
|
- flex now generates warnings for rules that can't be matched.
|
|
These warnings can be turned off using the new '-w' flag. If
|
|
your scanner uses REJECT then you will not get these warnings.
|
|
|
|
- If you specify the '-s' flag but the default rule can be matched,
|
|
flex now generates a warning.
|
|
|
|
- "yyleng" is now a global, and may be modified by the user (though
|
|
doing so and then using yymore() will yield weird results).
|
|
|
|
- Name definitions in the first section of a scanner specification
|
|
can now include a leading '^' or trailing '$' operator. In this
|
|
case, the definition is *not* pushed back inside of parentheses.
|
|
|
|
- Scanners with compressed tables are now "interactive" (-I option)
|
|
by default. You can suppress this attribute (which makes them
|
|
run slightly slower) using the new '-B' flag.
|
|
|
|
- Flex now generates 8-bit scanners by default, unless you use the
|
|
-Cf or -CF compression options (-Cfe and -CFe result in 8-bit
|
|
scanners). You can force it to generate a 7-bit scanner using
|
|
the new '-7' flag. You can build flex to generate 8-bit scanners
|
|
for -Cf and -CF, too, by adding -DDEFAULT_CSIZE=256 to CFLAGS
|
|
in the Makefile.
|
|
|
|
- You no longer need to call the scanner routine yyrestart() to
|
|
inform the scanner that you have switched to a new file after
|
|
having seen an EOF on the current input file. Instead, just
|
|
point yyin at the new file and continue scanning.
|
|
|
|
- You no longer need to invoke YY_NEW_FILE in an <<EOF>> action
|
|
to indicate you wish to continue scanning. Simply point yyin
|
|
at a new file.
|
|
|
|
- A leading '#' no longer introduces a comment in a flex input.
|
|
|
|
- flex no longer considers formfeed ('\f') a whitespace character.
|
|
|
|
- %t, I'm happy to report, has been nuked.
|
|
|
|
- The '-p' option may be given twice ('-pp') to instruct flex to
|
|
report minor performance problems as well as major ones.
|
|
|
|
- The '-v' verbose output no longer includes start/finish time
|
|
information.
|
|
|
|
- Newlines in flex inputs can optionally include leading or
|
|
trailing carriage-returns ('\r'), in support of several PC/Mac
|
|
run-time libraries that automatically include these.
|
|
|
|
- A start condition of the form "<*>" makes the following rule
|
|
active in every start condition, whether exclusive or inclusive.
|
|
|
|
- The following items have been corrected in the flex documentation:
|
|
|
|
- '-C' table compression options *are* cumulative.
|
|
|
|
- You may modify yytext but not lengthen it by appending
|
|
characters to the end. Modifying its final character
|
|
will affect '^' anchoring for the next rule matched
|
|
if the character is changed to or from a newline.
|
|
|
|
- The term "backtracking" has been renamed "backing up",
|
|
since it is a one-time repositioning and not a repeated
|
|
search. What used to be the "lex.backtrack" file is now
|
|
"lex.backup".
|
|
|
|
- Unindented "/* ... */" comments are allowed in the first
|
|
flex input section, but not in the second.
|
|
|
|
- yyless() can only be used in the flex input source, not
|
|
externally.
|
|
|
|
- You can use "yyrestart(yyin)" to throw away the
|
|
current contents of the input buffer.
|
|
|
|
- To write high-speed scanners, attempt to match as much
|
|
text as possible with each rule. See MISC/fastwc/README
|
|
for more information.
|
|
|
|
- Using the beginning-of-line operator ('^') is fairly
|
|
cheap. Using unput() is expensive. Using yyless() is
|
|
cheap.
|
|
|
|
- An example of scanning strings with embedded escape
|
|
sequences has been added.
|
|
|
|
- The example of backing-up in flexdoc was erroneous; it
|
|
has been corrected.
|
|
|
|
- A flex scanner's internal buffer now dynamically grows if needed
|
|
to match large tokens. Note that growing the buffer presently
|
|
requires rescanning the (large) token, so consuming a lot of
|
|
text this way is a slow process. Also note that presently the
|
|
buffer does *not* grow if you unput() more text than can fit
|
|
into the buffer.
|
|
|
|
- The MISC/ directory has been reorganized; see MISC/README for
|
|
details.
|
|
|
|
- yyless() can now be used in the third (user action) section
|
|
of a scanner specification, thanks to Ceriel Jacobs. yyless()
|
|
remains a macro and cannot be used outside of the scanner source.
|
|
|
|
- The skeleton file is no longer opened at run-time, but instead
|
|
compiled into a large string array (thanks to John Gilmore and
|
|
friends at Cygnus). You can still use the -S flag to point flex
|
|
at a different skeleton file, though if you use this option let
|
|
me know, as I plan to otherwise do away with -S in the near
|
|
future.
|
|
|
|
- flex no longer uses a temporary file to store the scanner's
|
|
actions.
|
|
|
|
- A number of changes have been made to decrease porting headaches.
|
|
In particular, flex no longer uses memset() or ctime(), and
|
|
provides a single simple mechanism for dealing with C compilers
|
|
that still define malloc() as returning char* instead of void*.
|
|
|
|
- Flex now detects if the scanner specification requires the -8 flag
|
|
but the flag was not given or on by default.
|
|
|
|
- A number of table-expansion fencepost bugs have been fixed,
|
|
making flex more robust for generating large scanners.
|
|
|
|
- flex more consistently identifies the location of errors in
|
|
its input.
|
|
|
|
- YY_USER_ACTION is now invoked only for "real" actions, not for
|
|
internal actions used by the scanner for things like filling
|
|
the buffer or handling EOF.
|
|
|
|
- The rule "[^]]" now matches any character other than a ']';
|
|
formerly it matched any character at all followed by a ']'.
|
|
This change was made for compatibility with AT&T lex.
|
|
|
|
- A large number of miscellaneous bugs have been found and fixed
|
|
thanks to Gerhard Wilhelms.
|
|
|
|
- The source code has been heavily reformatted, making patches
|
|
relative to previous flex releases no longer accurate.
|
|
|
|
|
|
Changes between 2.3 Patch #8 (21Feb93) and 2.3 Patch #7:
|
|
|
|
- Fixed bugs in dynamic memory allocation leading to grievous
|
|
fencepost problems when generating large scanners.
|
|
- Fixed bug causing infinite loops on character classes with 8-bit
|
|
characters in them.
|
|
- Fixed bug in matching repetitions with a lower bound of 0.
|
|
- Fixed bug in scanning NUL characters using an "interactive" scanner.
|
|
- Fixed bug in using yymore() at the end of a file.
|
|
- Fixed bug in misrecognizing rules with variable trailing context.
|
|
- Fixed bug compiling flex on Suns using gcc 2.
|
|
- Fixed bug in not recognizing that input files with the character
|
|
ASCII 128 in them require the -8 flag.
|
|
- Fixed bug that could cause an infinite loop writing out
|
|
error messages.
|
|
- Fixed bug in not recognizing old-style lex % declarations if
|
|
followed by a tab instead of a space.
|
|
- Fixed potential crash when flex terminated early (usually due
|
|
to a bad flag) and the -v flag had been given.
|
|
- Added some missing declarations of void functions.
|
|
- Changed to only use '\a' for __STDC__ compilers.
|
|
- Updated mailing addresses.
|
|
|
|
|
|
Changes between 2.3 Patch #7 (28Mar91) and 2.3 Patch #6:
|
|
|
|
- Fixed out-of-bounds array access that caused bad tables
|
|
to be produced on machines where the bad reference happened
|
|
to yield a 1. This caused problems installing or running
|
|
flex on some Suns, in particular.
|
|
|
|
|
|
Changes between 2.3 Patch #6 (29Aug90) and 2.3 Patch #5:
|
|
|
|
- Fixed a serious bug in yymore() which basically made it
|
|
completely broken. Thanks goes to Jean Christophe of
|
|
the Nethack development team for finding the problem
|
|
and passing along the fix.
|
|
|
|
|
|
Changes between 2.3 Patch #5 (16Aug90) and 2.3 Patch #4:
|
|
|
|
- An up-to-date version of initscan.c so "make test" will
|
|
work after applying the previous patches
|
|
|
|
|
|
Changes between 2.3 Patch #4 (14Aug90) and 2.3 Patch #3:
|
|
|
|
- Fixed bug in hexadecimal escapes which allowed only digits,
|
|
not letters, in escapes
|
|
- Fixed bug in previous "Changes" file!
|
|
|
|
|
|
Changes between 2.3 Patch #3 (03Aug90) and 2.3 Patch #2:
|
|
|
|
- Correction to patch #2 for gcc compilation; thanks goes to
|
|
Paul Eggert for catching this.
|
|
|
|
|
|
Changes between 2.3 Patch #2 (02Aug90) and original 2.3 release:
|
|
|
|
- Fixed (hopefully) headaches involving declaring malloc()
|
|
and free() for gcc, which defines __STDC__ but (often) doesn't
|
|
come with the standard include files such as <stdlib.h>.
|
|
Reordered #ifdef maze in the scanner skeleton in the hope of
|
|
getting the declarations right for cfront and g++, too.
|
|
|
|
- Note that this patch supercedes patch #1 for release 2.3,
|
|
which was never announced but was available briefly for
|
|
anonymous ftp.
|
|
|
|
|
|
Changes between 2.3 (full) release of 28Jun90 and 2.2 (alpha) release:
|
|
|
|
User-visible:
|
|
|
|
- A lone <<EOF>> rule (that is, one which is not qualified with
|
|
a list of start conditions) now specifies the EOF action for
|
|
*all* start conditions which haven't already had <<EOF>> actions
|
|
given. To specify an end-of-file action for just the initial
|
|
state, use <INITIAL><<EOF>>.
|
|
|
|
- -d debug output is now contigent on the global yy_flex_debug
|
|
being set to a non-zero value, which it is by default.
|
|
|
|
- A new macro, YY_USER_INIT, is provided for the user to specify
|
|
initialization action to be taken on the first call to the
|
|
scanner. This action is done before the scanner does its
|
|
own initialization.
|
|
|
|
- yy_new_buffer() has been added as an alias for yy_create_buffer()
|
|
|
|
- Comments beginning with '#' and extending to the end of the line
|
|
now work, but have been deprecated (in anticipation of making
|
|
flex recognize #line directives).
|
|
|
|
- The funky restrictions on when semi-colons could follow the
|
|
YY_NEW_FILE and yyless macros have been removed. They now
|
|
behave identically to functions.
|
|
|
|
- A bug in the sample redefinition of YY_INPUT in the documentation
|
|
has been corrected.
|
|
|
|
- A bug in the sample simple tokener in the documentation has
|
|
been corrected.
|
|
|
|
- The documentation on the incompatibilities between flex and
|
|
lex has been reordered so that the discussion of yylineno
|
|
and input() come first, as it's anticipated that these will
|
|
be the most common source of headaches.
|
|
|
|
|
|
Things which didn't used to be documented but now are:
|
|
|
|
- flex interprets "^foo|bar" differently from lex. flex interprets
|
|
it as "match either a 'foo' or a 'bar', providing it comes at the
|
|
beginning of a line", whereas lex interprets it as "match either
|
|
a 'foo' at the beginning of a line, or a 'bar' anywhere".
|
|
|
|
- flex initializes the global "yyin" on the first call to the
|
|
scanner, while lex initializes it at compile-time.
|
|
|
|
- yy_switch_to_buffer() can be used in the yywrap() macro/routine.
|
|
|
|
- flex scanners do not use stdio for their input, and hence when
|
|
writing an interactive scanner one must explictly call fflush()
|
|
after writing out a prompt.
|
|
|
|
- flex scanner can be made reentrant (after a fashion) by using
|
|
"yyrestart( yyin );". This is useful for interactive scanners
|
|
which have interrupt handlers that long-jump out of the scanner.
|
|
|
|
- a defense of why yylineno is not supported is included, along
|
|
with a suggestion on how to convert scanners which rely on it.
|
|
|
|
|
|
Other changes:
|
|
|
|
- Prototypes and proper declarations of void routines have
|
|
been added to the flex source code, courtesy of Kevin B. Kenny.
|
|
|
|
- Routines dealing with memory allocation now use void* pointers
|
|
instead of char* - see Makefile for porting implications.
|
|
|
|
- Error-checking is now done when flex closes a file.
|
|
|
|
- Various lint tweaks were added to reduce the number of gripes.
|
|
|
|
- Makefile has been further parameterized to aid in porting.
|
|
|
|
- Support for SCO Unix added.
|
|
|
|
- Flex now sports the latest & greatest UC copyright notice
|
|
(which is only slightly different from the previous one).
|
|
|
|
- A note has been added to flexdoc.1 mentioning work in progress
|
|
on modifying flex to generate straight C code rather than a
|
|
table-driven automaton, with an email address of whom to contact
|
|
if you are working along similar lines.
|
|
|
|
|
|
Changes between 2.2 Patch #3 (30Mar90) and 2.2 Patch #2:
|
|
|
|
- fixed bug which caused -I scanners to bomb
|
|
|
|
|
|
Changes between 2.2 Patch #2 (27Mar90) and 2.2 Patch #1:
|
|
|
|
- fixed bug writing past end of input buffer in yyunput()
|
|
- fixed bug detecting NUL's at the end of a buffer
|
|
|
|
|
|
Changes between 2.2 Patch #1 (23Mar90) and 2.2 (alpha) release:
|
|
|
|
- Makefile fixes: definition of MAKE variable for systems
|
|
which don't have it; installation of flexdoc.1 along with
|
|
flex.1; fixed two bugs which could cause "bigtest" to fail.
|
|
|
|
- flex.skel fix for compiling with g++.
|
|
|
|
- README and flexdoc.1 no longer list an out-of-date BITNET address
|
|
for contacting me.
|
|
|
|
- minor typos and formatting changes to flex.1 and flexdoc.1.
|
|
|
|
|
|
Changes between 2.2 (alpha) release of March '90 and previous release:
|
|
|
|
User-visible:
|
|
|
|
- Full user documentation now available.
|
|
|
|
- Support for 8-bit scanners.
|
|
|
|
- Scanners now accept NUL's.
|
|
|
|
- A facility has been added for dealing with multiple
|
|
input buffers.
|
|
|
|
- Two manual entries now. One which fully describes flex
|
|
(rather than just its differences from lex), and the
|
|
other for quick(er) reference.
|
|
|
|
- A number of changes to bring flex closer into compliance
|
|
with the latest POSIX lex draft:
|
|
|
|
%t support
|
|
flex now accepts multiple input files and concatenates
|
|
them together to form its input
|
|
previous -c (compress) flag renamed -C
|
|
do-nothing -c and -n flags added
|
|
Any indented code or code within %{}'s in section 2 is
|
|
now copied to the output
|
|
|
|
- yyleng is now a bona fide global integer.
|
|
|
|
- -d debug information now gives the line number of the
|
|
matched rule instead of which number rule it was from
|
|
the beginning of the file.
|
|
|
|
- -v output now includes a summary of the flags used to generate
|
|
the scanner.
|
|
|
|
- unput() and yyrestart() are now globally callable.
|
|
|
|
- yyrestart() no longer closes the previous value of yyin.
|
|
|
|
- C++ support; generated scanners can be compiled with C++ compiler.
|
|
|
|
- Primitive -lfl library added, containing default main()
|
|
which calls yylex(). A number of routines currently living
|
|
in the scanner skeleton will probably migrate to here
|
|
in the future (in particular, yywrap() will probably cease
|
|
to be a macro and instead be a function in the -lfl library).
|
|
|
|
- Hexadecimal (\x) escape sequences added.
|
|
|
|
- Support for MS-DOS, VMS, and Turbo-C integrated.
|
|
|
|
- The %used/%unused operators have been deprecated. They
|
|
may go away soon.
|
|
|
|
|
|
Other changes:
|
|
|
|
- Makefile enhanced for easier testing and installation.
|
|
- The parser has been tweaked to detect some erroneous
|
|
constructions which previously were missed.
|
|
- Scanner input buffer overflow is now detected.
|
|
- Bugs with missing "const" declarations fixed.
|
|
- Out-of-date Minix/Atari patches provided.
|
|
- Scanners no longer require printf() unless FLEX_DEBUG is being used.
|
|
- A subtle input() bug has been fixed.
|
|
- Line numbers for "continued action" rules (those following
|
|
the special '|' action) are now correct.
|
|
- unput() bug fixed; had been causing problems porting flex to VMS.
|
|
- yymore() handling rewritten to fix bug with interaction
|
|
between yymore() and trailing context.
|
|
- EOF in actions now generates an error message.
|
|
- Bug involving -CFe and generating equivalence classes fixed.
|
|
- Bug which made -CF be treated as -Cf fixed.
|
|
- Support for SysV tmpnam() added.
|
|
- Unused #define's for scanner no longer generated.
|
|
- Error messages which are associated with a particular input
|
|
line are now all identified with their input line in standard
|
|
format.
|
|
- % directives which are valid to lex but not to flex are
|
|
now ignored instead of generating warnings.
|
|
- -DSYS_V flag can now also be specified -DUSG for System V
|
|
compilation.
|
|
|
|
|
|
Changes between 2.1 beta-test release of June '89 and previous release:
|
|
|
|
User-visible:
|
|
|
|
- -p flag generates a performance report to stderr. The report
|
|
consists of comments regarding features of the scanner rules
|
|
which result in slower scanners.
|
|
|
|
- -b flag generates backtracking information to lex.backtrack.
|
|
This is a list of scanner states which require backtracking
|
|
and the characters on which they do so. By adding rules
|
|
one can remove backtracking states. If all backtracking states
|
|
are eliminated, the generated scanner will run faster.
|
|
Backtracking is not yet documented in the manual entry.
|
|
|
|
- Variable trailing context now works, i.e., one can have
|
|
rules like "(foo)*/[ \t]*bletch". Some trailing context
|
|
patterns still cannot be properly matched and generate
|
|
error messages. These are patterns where the ending of the
|
|
first part of the rule matches the beginning of the second
|
|
part, such as "zx*/xy*", where the 'x*' matches the 'x' at
|
|
the beginning of the trailing context. Lex won't get these
|
|
patterns right either.
|
|
|
|
- Faster scanners.
|
|
|
|
- End-of-file rules. The special rule "<<EOF>>" indicates
|
|
actions which are to be taken when an end-of-file is
|
|
encountered and yywrap() returns non-zero (i.e., indicates
|
|
no further files to process). See manual entry for example.
|
|
|
|
- The -r (reject used) flag is gone. flex now scans the input
|
|
for occurrences of the string "REJECT" to determine if the
|
|
action is needed. It tries to be intelligent about this but
|
|
can be fooled. One can force the presence or absence of
|
|
REJECT by adding a line in the first section of the form
|
|
"%used REJECT" or "%unused REJECT".
|
|
|
|
- yymore() has been implemented. Similarly to REJECT, flex
|
|
detects the use of yymore(), which can be overridden using
|
|
"%used" or "%unused".
|
|
|
|
- Patterns like "x{0,3}" now work (i.e., with lower-limit == 0).
|
|
|
|
- Removed '\^x' for ctrl-x misfeature.
|
|
|
|
- Added '\a' and '\v' escape sequences.
|
|
|
|
- \<digits> now works for octal escape sequences; previously
|
|
\0<digits> was required.
|
|
|
|
- Better error reporting; line numbers are associated with rules.
|
|
|
|
- yyleng is a macro; it cannot be accessed outside of the
|
|
scanner source file.
|
|
|
|
- yytext and yyleng should not be modified within a flex action.
|
|
|
|
- Generated scanners #define the name FLEX_SCANNER.
|
|
|
|
- Rules are internally separated by YY_BREAK in lex.yy.c rather
|
|
than break, to allow redefinition.
|
|
|
|
- The macro YY_USER_ACTION can be redefined to provide an action
|
|
which is always executed prior to the matched rule's action.
|
|
|
|
- yyrestart() is a new action which can be used to restart
|
|
the scanner after it has seen an end-of-file (a "real" one,
|
|
that is, one for which yywrap() returned non-zero). It takes
|
|
a FILE* argument indicating a new file to scan and sets
|
|
things up so that a subsequent call to yylex() will start
|
|
scanning that file.
|
|
|
|
- Internal scanner names all preceded by "yy_"
|
|
|
|
- lex.yy.c is deleted if errors are encountered during processing.
|
|
|
|
- Comments may be put in the first section of the input by preceding
|
|
them with '#'.
|
|
|
|
|
|
|
|
Other changes:
|
|
|
|
- Some portability-related bugs fixed, in particular for machines
|
|
with unsigned characters or sizeof( int* ) != sizeof( int ).
|
|
Also, tweaks for VMS and Microsoft C (MS-DOS), and identifiers all
|
|
trimmed to be 31 or fewer characters. Shortened file names
|
|
for dinosaur OS's. Checks for allocating > 64K memory
|
|
on 16 bit'ers. Amiga tweaks. Compiles using gcc on a Sun-3.
|
|
- Compressed and fast scanner skeletons merged.
|
|
- Skeleton header files done away with.
|
|
- Generated scanner uses prototypes and "const" for __STDC__.
|
|
- -DSV flag is now -DSYS_V for System V compilation.
|
|
- Removed all references to FTL language.
|
|
- Software now covered by BSD Copyright.
|
|
- flex will replace lex in subsequent BSD releases.
|