A future commit will actually implement //IGNORE so that applications
using base iconv can, e.g., sanitize UTF-8 strings. To do this, the
iconv_std module needs to be able to determine the minimum width for any
given encoding so that it can skip that many bytes in the input buffer.
This is mainly an issue for UTF-16 and UTF-32.
This commit bumps shlib versions to 5 for libiconv modules to reflect
the ABI change. It also fixes OptionalObsoleteFiles to remove the
libiconv modules if WITHOUT_ICONV is in use.
re: _ENCODING_MB_CUR_MIN, note that this file (citrus_stdenc_template.h)
is included at the bottom of an encoding *implementation*, so the
implementation is free to #define it prior. UTF1632 is a good example,
as it redefines the minimum to be a property on the encodinginfo, and
the minimum is set to 2 or 4 bytes for UTF-16 and UTF-32 respectively.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34344
Make it vaguely aware of options in the sense that it now knows that it
can zap any trailing //. It now copies the entire string in realsrc and
realdst, then terminates them at the options.
__bsd___iconv_open can now stop trying to allocate memory just for this
purpose, and the new version is technically more correct. GNU libiconv
will ignore options on the `in` codeset and still do the right thing.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34343
Checking there are still bytes left must be done before dereferencing
the pointer, not the other way round. This is harmless on traditional
architectures since the result will immediately be thrown away, and all
callers are in separate translation units so there is no potential for
optimising based on this out-of-bounds read. However, on CHERI, pointers
are bounded, and so this will trap if fed a string that does not have a
NUL within the first len bytes.
Found by: CHERI
Reviewed by: brooks
In _citrus_prop_read_TYPE_func_ generated functions, do not ignore parsed
'-' sign, negate the value as appropriate.
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33146
This option was added as a transition aide when symbol versioning was
first added. It was enabled by default in 2007 and is supported even
by the old GPLv2 binutils. Trying to disable it currently fails to
build in libc and at this point it isn't worth fixing the build.
Reported by: Michael Dexter
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D24637
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
reallocarray(3) is a non portable extension that originated in OpenBSD.
Given that it is already in FreeBSD's libc it is useful for the cases
where reallocation involves a multiplication.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D9955
This reduces build output, need for recalculating paths, and makes it clearer
which paths are relative to what areas in the source tree. The change in
performance over a locally mounted UFS filesystem was negligible in my testing,
but this may more positively impact other filesystems like NFS.
LIBC_SRCTOP was left alone so Juniper (and other users) can continue to
manipulate lib/libc/Makefile (and other Makefile.inc's under lib/libc) as
include Makefiles with custom options.
Discussed with: marcel, sjg
MFC after: 1 week
Reviewed by: emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D9207
It appears "sorted" may have not been implemented. Sorted or not,
we always follow the same action so simplify the code.
Leave a note for future generations.
CID: 1347084
convname and dst are guaranteed to be non-NULL by iconv_open(3).
src is an array. Remove these tests for NULL pointers.
While I'm here, eliminate a strlcpy with a correct but suspicious-looking
calculation for the third parameter (i.e. not a simple sizeof).
Compare the strings in-place instead of copying.
Found by: bdrewery
Found by: Coverity
CID: 1130050, 1130056
MFC after: 3 days
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6338
- Both curitem and curitem (via the names list) was always leaked.
- malloc(3) failures lead to some leaks.
- __bsd___iconv_get_list() failure lead to a crash since its error was not
handles and __bsd___iconv_free_list() is not NULL-safe.
I have slightly refactored this to avoid extra malloc and free logic in cases
of malloc(3) failing.
There are still bad assumptions here that I did not deal with. One of which is
that the data will always have a '/' so the strchr(3) will not return NULL.
Coverity CID: 1130055 1130054 1130053
We do not use iconv.alias file, so avoid using the vestiges
of the code that do.
Differential Revision: https://reviews.freebsd.org/D1729
Reviewed by: emaste
MFC after: 2 weeks
fields of a private struct such that variables of this type are
initialised correctly. Fixes conversion from ISO 2022.
Also do this in the BIG5 module to prevent similar errors in the future.
- In the libiconv module for EUC-TW replace 2^cs with 1<<cs. Fixes
conversion from EUC-TW.
- Synchronise iconv code with NetBSD. In most cases this only updates
the RCS id because the changes are already there or are NetBSD specific.
+ libc/iconv/citrus_csmapper.c: Add a comment.
+ libc/iconv/citrus_db_factory.c: Remove put16().
+ libc/iconv/citrus_iconv.c: Return EINVAL on error.
+ libc/iconv/citrus_mapper.c: Return EINVAL on error.
+ libc/iconv/citrus_memstream.c: Fix type of a variable.
+ libc/iconv/citrus_prop.h: Sync definition of _CITRUS_PROP_HINT_END.
+ libc/iconv/citrus_stdenc.c: Return EINVAL on error.
+ libiconv_modules/mapper_std/citrus_mapper_std.c: Plug memory leak.
Obtained from: NetBSD
MFC after: 2 weeks
if not already defined. This allows building libc from outside of
lib/libc using a reach-over makefile.
A typical use-case is to build a standard ILP32 version and a COMPAT32
version in a single iteration by building the COMPAT32 version using a
reach-over makefile.
Obtained from: Juniper Networks, Inc.
it is all in the one place again. Rename libc/iconv/iconv.c to
bsd_iconv.c. Compile the wrappers into libc.a so that WITHOUT_DYNAMICROOT
works again.
Discussed with: kib (and partly stolen from his patch)
when there is an invalid character in the output codeset while it is
valid in the input. However, POSIX requires iconv() to perform an
implementation-defined conversion on the character. So, Citrus iconv converts
such a character to a special character which means it is invalid in the
output codeset.
This is not a problem in most cases but some software like libxml2 depends
on GNU's behavior to determine if a character is output as-is or another form
such as a character entity (&#NNN;).
FreeBSD systems usually implemented this as a third party module and
our implementation hasn't played as nicely with the old way as it could
have.
To that end:
* Rename the iconv* symbols in libc.so.7 to have a __bsd_ prefix.
* Provide .symver compatability with existing 10.x+ binaries that
referenced the iconv symbols. All existing binaries should work.
* Like on Linux/glibc systems, add a libc_nonshared.a to the ldscript
at /usr/lib/libc.so.
* Move the "iconv*" wrapper symbols to libc_nonshared.a
This should solve the runtime ambiguity about which symbols resolve
to where. If you compile against the iconv in libc, your runtime
dependencies will be unambiguous.
Old 9.x libraries and binaries will always resolve against their
libiconv.so.3 like they did on 9.x. They won't resolve against libc.
Old 10.x binaries will be satisified by the .symver helpers.
This should allow ports to selectively compile against the libiconv
port if needed and it should behave without ambiguity now.
Discussed with: kib
good. This caused libc to spoof the ports libiconv namespace and
provide a colliding libiconv.so.3 to fool rtld. This should have
been removed some time ago.
_citrus_mapper_close again and result in a deadlock otherwise.
This is similar to NetBSD PR/24023 (fixed in their r1.5 of this file).
PR: bin/182994
Submitted by: Fabian Keil <fk fabiankeil de>
MFC after: 3 days
extensions and also tried to be link time compatible with ports libiconv.
This splits that functionality and enables the parts that shouldn't
interfere with the port by default.
WITH_ICONV (now on by default) - adds iconv.h, iconv_open(3) etc.
WITH_LIBICONV_COMPAT (off by default) adds the libiconv_open etc API, linker
symbols and even a stub libiconv.so.3 that are good enough to be able
to 'pkg delete -f libiconv' on a running system and reasonably expect it
to work.
I have tortured many machines over the last few days to try and reduce
the possibilities of foot-shooting as much as I can. I've successfully
recompiled to enable and disable the libiconv_compat modes, ports that use
libiconv alongside system iconv etc. If you don't enable the
WITH_LIBICONV_COMPAT switch, they don't share symbol space.
This is an extension of behavior on other system. iconv(3) is a standard
libc interface and libiconv port expects to be able to run alongside it on
systems that have it.
Bumped osreldate.
bypass mode when src == dst. Unfortunately, there are tools in ports
that pass byte streams through iconv to determine if the encodings
are valid. eg: gettext-0.18.3+.
Disable the optimization and behave like the other implementations.
a macro with parameters. Remove a __DECONST hack and add consts instead
for gnu libiconv API compatability. This makes it work with things like
devel/boost-libs that expects to use "iconv" as though it were a pointer.
