Commit Graph

567 Commits

Author SHA1 Message Date
rodrigc
aa990038b1 Use ANSI C prototypes. Eliminates -Wold-style-definition warnings. 2015-09-20 20:50:18 +00:00
ed
94d0b79c32 Make UTF-8 parsing and generation more strict.
- in mbrtowc() we need to disallow codepoints above 0x10ffff.
- In wcrtomb() we need to disallow codepoints between 0xd800 and 0xdfff.

Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D3399
2015-08-25 09:16:09 +00:00
bapt
9e0d1c3e19 Remove 5 and 6 bytes sequences which are illegal in UTF-8 space. (part2)
Per rfc3629 value greater than 0x10ffff should be rejected

Suggested by:	jilles
2015-08-09 00:06:56 +00:00
bapt
df871b442a Remove 5 and 6 bytes sequences which are illegal in UTF-8 space.
Per rfc3629 value greater than 0x10ffff should be rejected

Suggested by:	jilles
2015-08-08 23:59:15 +00:00
theraven
b4f9c4540d __xlocale_C_ctype should not be const. It contains a reference count that is modified by newlocale / duplocale / freelocale.
MFC after:	1 week
2015-04-24 10:21:20 +00:00
theraven
8d131b2bfd Small changes to locale-related man pages.
Fix a missing .h and change the recommended include for the POSIX2008 functions from xlocale.h to locale.h.  Including xlocale.h is for legacy / Darwin compatibility so should not be encouraged.
2015-04-24 10:17:55 +00:00
tijl
b0813ee288 Remove the const qualifier from iconv(3) to comply with POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/iconv.html

Adjust all code that calls iconv.

PR:		199099
Exp-run by:	antoine
MFC after:	2 weeks
2015-04-15 09:09:20 +00:00
joel
716fc8f57e mdoc: improvements to SEE ALSO. 2014-12-27 08:31:52 +00:00
pfg
88b9fb0549 libc/locale: Remove a wrong comma.
This only had some effect when debugging.

Obtained from:	DragonflyBSD
MFC after:	3 days
2014-09-04 17:36:21 +00:00
pfg
005e0d33a7 minor perf enhancement for UTF-8
Reduce some duplicate code.

Reference:
https://www.illumos.org/issues/628

Obtained from:	Illumos
MFC after:	1 week
2014-07-04 22:39:39 +00:00
pfg
f0efccad2e citrus: Avoid invalid code points.
From the OpenBSD log:
The UTF-8 decoder should not accept byte sequences which decode to unicode
code positions U+D800 to U+DFFF (UTF-16 surrogates), U+FFFE, and U+FFFF.

http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8
http://unicode.org/faq/utf_bom.html#utf8-4

Reported by:	Stefan Sperling
Obtained from:	OpenBSD
MFC after:	5 days
2014-05-01 01:42:48 +00:00
pfg
aa4f79bd1b citrus: Avoid invalid code points.
From the OpenBSD log:
The UTF-8 decoder should not accept byte sequences which decode to unicode
code positions U+D800 to U+DFFF (UTF-16 surrogates), U+FFFE, and U+FFFF.

http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8
http://unicode.org/faq/utf_bom.html#utf8-4

Reported by:	Stefan Sperling
Obtained from:	OpenBSD
MFC after:	5 days
2014-04-29 15:25:57 +00:00
theraven
e9b954067d Fix an issue where the locale and rune locale could become out of sync,
causing mb* functions (and similar) to be called with the wrong data
(possibly a null pointer, causing a crash).

PR:		standards/188036
MFC after:	1 week
2014-04-02 11:10:46 +00:00
marcel
99c9726a00 Replace use of ${.CURDIR} by ${LIBC_SRCTOP} and define ${LIBC_SRCTOP}
if not already defined. This allows building libc from outside of
lib/libc using a reach-over makefile.

A typical use-case is to build a standard ILP32 version and a COMPAT32
version in a single iteration by building the COMPAT32 version using a
reach-over makefile.

Obtained from:	Juniper Networks, Inc.
2014-03-04 02:19:39 +00:00
peter
a02844e93b Replace the #define for "iconv" so it is for the function name instead of
a macro with parameters.  Remove a __DECONST hack and add consts instead
for gnu libiconv API compatability.  This makes it work with things like
devel/boost-libs that expects to use "iconv" as though it were a pointer.
2013-07-03 07:03:19 +00:00
ed
d6e16cad4c Add libiconv based versions of *c16*() and *c32*().
I initially thought wchar_t was locale independent, but this seems to be
only the case on Linux. This means that we cannot depend on the *wc*()
routines to implement *c16*() and *c32*(). Instead, use the Citrus
libiconv that is part of libc.

I'll see if there is anything I can do to make the existing functions
somewhat useful in case the system is built without libiconv in the
nearby future. If not, I'll simply remove the broken implementations.

Reviewed by:	jilles, gabor
2013-06-03 17:17:56 +00:00
ed
05bd8275f3 Add <uchar.h>.
The <uchar.h> header, part of C11, adds a small number of utility
functions for 16/32-bit "universal" characters, which may or may not be
UTF-16/32. As our wchar_t is already ISO 10646, simply add light-weight
wrappers around wcrtomb() and mbrtowc().

While there, also add (non-yet-standard) _l functions, similar to the
ones we already have for the other locale-dependent functions.

Reviewed by:	theraven
2013-05-21 19:59:37 +00:00
pluknet
0c8dbadb69 Document that the return type is different from 1003.1-2008.
MFC after:	1 week
2013-05-04 17:21:44 +00:00
pluknet
3b34730a0b mdoc: missing comma in .Dd macro. 2013-05-04 17:06:47 +00:00
pluknet
fd497a19e9 Also, add a missing period. 2013-05-03 13:27:13 +00:00
pluknet
c9503ea86d Remove an extra comma. 2013-05-03 12:45:45 +00:00
pluknet
fb11a49792 Remove the STANDARDS section.
querylocale is not part of IEEE Std 1003.1-2008.

MFC after:	3 days
2013-05-03 12:42:43 +00:00
jilles
6d5ca83e83 btowc(3), isblank(3): Correct prototypes for _l variants.
MFC after:	1 week
2013-03-27 21:31:40 +00:00
jilles
4e143eee29 libc: Make various internal file descriptors close-on-exec.
These are obtained via fopen().
2012-12-11 22:52:56 +00:00
brooks
8f8a6363fa Improve style(9) compliance of function declarations. 2012-12-10 17:34:33 +00:00
jilles
fba61ef227 libc: Use O_CLOEXEC for various internal file descriptors.
This fixes a race condition where another thread may fork() before CLOEXEC
is set, unintentionally passing the descriptor to the child process.

This commit only adds O_CLOEXEC flags to open() or openat() calls where no
fcntl(fd, F_SETFD, FD_CLOEXEC) follows. The separate fcntl() call still
leaves a race window so it should be fixed later.
2012-09-29 11:54:34 +00:00
joel
84ed784b36 Remove trailing whitespace. 2012-07-30 21:02:44 +00:00
issyl0
6a683d47fc Add more locale-specific functions to the relevant man pages and Makefile:
- lib/libc/locale/islower.3
  - lib/libc/locale/ispunct.3
  - lib/libc/locale/nl_langinfo.3
  - lib/libc/locale/isgraph.3
  - lib/libc/locale/isspace.3

Reviewed by:	bz
Approved by:	theraven
MFC after:	5 days
2012-07-30 20:56:19 +00:00
joel
dcf3898747 Start manpage with Dd macro and also remove a trailing whitespace
while here.
2012-07-26 14:46:19 +00:00
issyl0
430cf8efee Add a new man page containing details of new locale-specific functions for
wctype.h, iswalnum_l(3).  Add it and its functions to the Makefile.

Reviewed by:	gavin, jilles
Approved by:	theraven
MFC after:	5 days
2012-07-25 22:17:44 +00:00
theraven
3fd9819783 Restore the __collate_load_error global that was accidentally removed in the
xlocale refactoring.

MFC after:	1 week
2012-07-06 20:16:22 +00:00
jilles
1e8cef649c ctype_l(3): Note that not all these functions are specified by POSIX.1-2008.
The function isascii_l() is not in POSIX even though isascii() is,
probably because isascii() is marked as obsolete.  The other functions,
like digittoint_l() and ishexnumber_l(), are FreeBSD-specific just like
their non-_l versions.
2012-07-01 22:18:20 +00:00
theraven
1b6b7176ba Fix a leak when setting the global character locale to "C" from something else.
Reported by:	mm
2012-06-11 14:02:02 +00:00
gjb
8670397617 General mdoc(7) and typo fixes.
PR:		167734
Submitted by:	Nobuyuki Koganemaru (kogane!jp.freebsd.org)
MFC after:	3 days
2012-05-11 20:06:46 +00:00
dim
3be8548129 Fix copy/paste error in lib/libc/locale/toupper.c.
Submitted by:	Kohji Okuno <okuno.kohji@jp.panasonic.com>
2012-05-10 20:03:34 +00:00
theraven
aeabc6ab90 Fix some incorrect symbol versions.
Reported by:	das
2012-04-22 18:51:38 +00:00
joel
ccd4f1d859 mdoc: fix function type. 2012-04-07 09:26:21 +00:00
joel
0ef023ae5b mdoc: fix column names, indentation, column separation within each row, and
quotation. Also make sure we have the same amount of columns in each row as
the number of columns we specify in the head arguments.

Reviewed by:	brueffer
2012-04-07 09:05:30 +00:00
eadler
1ef5fe44d3 Remove trailing whitespace per mdoc lint warning
Disussed with:	gavin
No objection from:	doc
Approved by:	joel
MFC after:	3 days
2012-03-29 05:02:12 +00:00
joel
d37628cc98 mdoc: remove unknown macro. 2012-03-26 19:06:59 +00:00
joel
5879f1f169 mdoc: remove empty lines. 2012-03-26 15:30:28 +00:00
joel
96b099d4c4 mdoc: document title should be all caps. 2012-03-25 19:34:05 +00:00
joel
db7581ccf5 Make sure sections are sorted into conventional order. 2012-03-25 16:00:56 +00:00
joel
39c40cce12 Remove superfluous paragraph macro. 2012-03-25 12:13:24 +00:00
theraven
f3af7bf15c Make __get_locale() static inline, not just inline, so when compiling libc in
debug mode (without optimisations) it doesn't generate a linker failure.

Approved by:	dim (mentor)
2012-03-19 11:47:37 +00:00
theraven
984b6cc8df First set of xlocale man pages. More to follow...
Approved by:	dim (mentor)
2012-03-13 20:02:41 +00:00
theraven
75aeb0cde5 More xlocale cleanups.
Approved by:	dim (mentor)
2012-03-13 14:14:13 +00:00
dim
d62ff6ee49 Fix build of libc.so after r232620. This caused a duplicate definition
of __getCurrentRuneLocale().

Pointy hat to:	me
2012-03-06 21:56:30 +00:00
theraven
1c33114ffd Remove some duplicated copyright notices.
Approved by:	dim (mentor)
2012-03-06 12:53:44 +00:00
theraven
b0968176b3 Reapply 227753 (xlocale cleanup), plus some fixes so that it passes build
universe with gcc.

Approved by:	dim (mentor)
2012-03-04 15:31:13 +00:00