reading past 'stop' in various places when converting multibyte characters.
Reading too far caused truncation to not be detected when it should have
been, eventually causing regexec() to loop infinitely in with certain
combinations of patterns and strings in multibyte locales.
PR: 74020
MFC after: 4 weeks
multibyte character support:
- In CHadd(), avoid writing past the end of the character set bitmap when
the opposite-case counterpart of wide characters with values less than
NC have values greater than or equal to NC.
- In CHaddtype(), fix a braino that caused alphabetic characters to be
added to all character classes! (but only with REG_ICASE)
PR: 71367
idea is that we perform multibyte->wide character conversion while parsing
and compiling, then convert byte sequences to wide characters when they're
needed for comparison and stepping through the string during execution.
As with tr(1), the main complication is to efficiently represent sets of
characters in bracket expressions. The old bitmap representation is replaced
by a bitmap for the first 256 characters combined with a vector of individual
wide characters, a vector of character ranges (for [A-Z] etc.), and a vector
of character classes (for [[:alpha:]] etc.).
One other point of interest is that although the Boyer-Moore algorithm had
to be disabled in the general multibyte case, it is still enabled for UTF-8
because of its self-synchronizing nature. This greatly speeds up matching
by reducing the number of multibyte conversions that need to be done.
Only warnings that could be fixed without changing the generated object
code and without restructuring the source code have been handled.
Reviewed by: /sbin/md5
access an array beyond it's length. This only happens in the last iteration of
a loop, and the value fetched is not used then, so the bug is a relatively
innocent one. Fix this by not fetching any value on the last iteration of said
loop.
Submitted by: MKI <mki@mozone.net>
MFC after: 1 week
Avoid using parenthesis enclosure macros (.Pq and .Po/.Pc) with plain text.
Not only this slows down the mdoc(7) processing significantly, but it also
has an undesired (in this case) effect of disabling hyphenation within the
entire enclosed block.
of the processing of the recursion, "scan" would be pointing to O_CH
(or O_QUEST), which would then be interpreted as being the end character
for altoffset().
We avoid this by properly increasing scan before leaving the switch.
Without this, something like (a?b?)?cc would result in a g->moffset of
1 instead of 2.
I added a case to the soon-to-be-imported regex(3) test code to catch
this error.
string may be found (from the beginning of the pattern), the point
at which must is found minus that offset may actually point to some
place before the start of the text.
In that case, make start = start.
Alternatively, this could be tested for in the preceding if, but it
did not occur to me. :-)
Caught by: regex(3) test code
use a CHAR_MIN-based array, like elsewhere in the code.
Remove a number of unused variables (some due to the above change, one
that was left after a number of optimizing steps through the source).
Brucified by: bde
