8 Commits

Author SHA1 Message Date
ache
a5038f060d The problem is: currently our single byte ctype(3) functions are broken
for wide characters locales in the argument range >= 0x80 - they may
return false positives.

Example 1: for UTF-8 locale we currently have:
iswspace(0xA0)==1 and isspace(0xA0)==1
(because iswspace() and isspace() are the same code)
but must have
iswspace(0xA0)==1 and isspace(0xA0)==0
(because there is no such character and all others in the range
0x80..0xff for the UTF-8 locale, it keeps ASCII only in the single byte
range because our internal wchar_t representation for UTF-8 is UCS-4).

Example 2: for all wide character locales isalpha(arg) when arg > 0xFF may
return false positives (must be 0).
(because iswalpha() and isalpha() are the same code)

This change address this issue separating single byte and wide ctype
and also fix iswascii() (currently iswascii() is broken for
arguments > 0xFF).
This change is 100% binary compatible with old binaries.

Reviewied by: i18n@
2007-10-13 16:28:22 +00:00
phantom
23d961a13f . Static'ize functions exported via function reference variables only.
. Replace inclusion of sys/param.h to sys/cdefs.h and sys/types.h where
  appropriate.
. move _*_init() prototypes to mblocal.h, and remove these prototypes
  from .c files
. use _none_init() in __setrunelocale() instead of duplicating code
. move __mb* variables from table.c to none.c allowing us to not to
  export _none_*() externs, and appropriately remove them from mblocal.h

Ok'ed by:	tjr
2005-02-27 15:11:09 +00:00
tjr
e3f042f4af Move prototypes of various encoding-related functions into a new header
file to avoid extern'ing them all over the place.
2004-05-12 14:09:04 +00:00
tjr
8f8a2ad179 Perform some basic validation of multibyte conversion state objects. 2004-04-12 13:09:18 +00:00
tjr
17077e5ae6 Don't cast away const qualifiers.
Spotted by:	bde
2004-04-10 00:27:52 +00:00
tjr
54a18fa1d6 Allow partial multibyte characters to accumulate in conversion state
objects passed to mbrtowc(), mbsrtowcs(), and mbrlen(), as required
by C99.
2004-04-07 10:48:19 +00:00
tjr
d98eda520c Reimplement the GB18030 encoding method using the new-style (mbrtowc()/
wcrtomb()) interface.
2004-04-04 11:00:42 +00:00
ache
2ecc37e253 Add support for gb18030 encoding
PR:             51729
Submitted by:   Kang Liu <liukang@bjpu.edu.cn>
2003-07-29 07:52:44 +00:00