The current localedef simply assumes that the locale headers on build system
are compatible with those on the target system which is not necessarily true.
It generally works on FreeBSD (as long as we don't change the locale headers),
but Linux and macOS provide completely different locale headers.
This change adds new bootstrap headers that namespace certain xlocale
structures defined or used by in the headers that localdef needs.
This is required since system headers *must* be able to include the "real"
locale headers for printf(), etc., but we also want to access the target
systems's internal locale structures.
Reviewed By: yuripv, brooks
Differential Revision: https://reviews.freebsd.org/D25229
for the known conflicts ("control" characters can't be "print"able).
POSIX doesn't explicitly forbid this, and actually includes <space>
character in "print".
PR: 225692
Reviewed by: bapt, cem (previous version), pfg (previous version)
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D17467
it appropriately when building share/ctypedef and share/colldef.
This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
build and in the native EB build match. Revert the changes done to libc
in r308170 as they are no longer needed.
PR: 231965
Reviewed by: bapt, emaste, sbruno, 0mp
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D17603
The localdef(1) changes are breaking world:
00:18:40.750 /usr/src/share/colldef/af_ZA.UTF-8.src: 2421: error: Bad file
descriptor
I will fix them offline.
Reported by: lwshu and many others
Also some small cleanups to match better current illumos.
CID: 1338540, 1338541, 1338557, 1338566
Obtained from: illumos
Discussed with: Yuri Pankov (@Nexenta)
MFC after: 5 days
This commit is from John Marino in dragonfly with the following commit log:
====
This was a CTYPE encoding error involving consecutive points of the same
ctype. It was reported by myself to Illumos over a year ago but I was
unsure if it was only happening on BSD. Given the cause, the bug is also
present on Illumos.
Basically, if consecutive points were of the exact same ctype, they would
be defined as a range regardless. For example, all of these would be
considered equivalent:
<A> ... <C>, <H> (converts to <A> .. <H>)
<A>, <B>, <H> (converts to <A> .. <H>)
<A>, <J> ... <H> (converts to <A> .. <H>)
So all the points that shouldn't have been defined got "bridged" by the
extreme points.
The effects were recently reported to FreeBSD on PR 213013. There are
countless places were the ctype flags are misdefined, so this is a major
fix that has to be MFC'd.
====
This reveals a bad change I did on the testsuite: while 0x07FF is a valid
unicode it is not used yet (reserved for future use)
PR: 213013
Submitted by: marino@
Reported by: Kurtis Rader <krader@skepticism.us>
Obtained from: Dragonfly
MFC after: 1 month
Illumos recently included space in 'print' class. We already had
this but the code had slight sorting differences. Move it some
lines up to reduce diffs with Illumos.
No functional change.
Reference:
https://illumos.org/issues/5227
of hexidecimal numbers) are all considered "numbers". (Note that while
all digits are numbers, not all numbers are digits).
Enhance localedef to automatically set the "number" characteristic when
it encounters a digit or xdigit definition. This fixes malfunctionning
isalnum(3)
Obtained from: DragonflyBSD
By having space automatically classified as "print" type, we can
eliminate the print section from ctype src files completely (they
are just "graph" plus "<space>".
Obtained from: Dragonfly
FreeBSD extended ctypes to include numbers (e.g. isnumber()) but never
actually implemented it. The isnumber() function was equivalent to the
isdigit() function in every case.
Now that DragonFly's ctype source files have number definitions, the
number ctype can finally be implemented. It's given a new flag _CTYPE_N.
The isalnum() and iswalnum() functions have been changed to use this
flag rather than the _CTYPE_D digit flag.
While isalnum(), isnumber(), and their wide equivalents now return
different values in locale cases, the ishexnumber() and iswhexnumber()
functions are unchanged. They are still aliases for isxdigit() and
iswxdigit().
Also change ctype.h for isdigit and isxdigit to use sbistype like the
other functions.
Obtained from: dragonfly
The localedef tool can read entire (and unmodified) CLDR posix definition
files, and generate all 6 LC categories: LC_COLLATE, LC_CTYPE, LC_TIME,
LC_NUMERIC, LC_MONETARY and LC_MESSAGES.
This tool has a long history with Solaris. The Nexenta developers
modified it to read CLDR files and created the much richer collation
formats. The libc collation functions have to be modified to read the
new format (called "BSD-1.0") and to handle the new data structures.
The result will be that locale-sensitive tools and functions will now
properly sort multibyte and unicode strings.
Obtained from: Dragonfly
