They are emitting characters which are triggering
a kyua bug which causes kyua to emit invalid XML.
This invalid XML is causing false failures in Jenkins.
On a separate note, kyua needs to be fixed with this:
https://github.com/jmmv/kyua/pull/148
or something similar.
When pmcstat exits after some samples were dropped, give the user an
idea of how many were lost. (Granted, these are global numbers, but
they may still help quantify the scope of the loss.)
Differential Revision: https://reviews.freebsd.org/D4123
Approved by: gnn (mentor)
MFC after: 1 month
Sponsored by: Juniper Networks
- Split up the testcases into C locale and ja_JP.eucJP testcases.
- Avoid a segfault in the event that setlocale fails, similar to r290843
- Replace `sizeof(x) / sizeof(*x)` pattern with `nitems(x)`
MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division
Provide more meaningful diagnostic messages if LC_CTYPE can't be set properly
instead of segfaulting, because setlocale returns NULL and strcmp(NULL, b) will
always segfault
Split up the testcases so one failing (in this case en_US.ISO8859-15) won't
cause the rest of the testcases to be skipped
Remove some unused variables
MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division
immediatelly as old code does, now for append modes too.
Real use case for such fallback is impossible (unless specially crafted).
2) Remove now unneded include I forgot to remove in prev. commits.
MFC after: 1 week
The NONE:US-ASCII case isn't necessary. The "NONE:" case will handle
US-ASCII, so let's remove the redundant handling.
Submitted by: marino
Obtained from: DragonflyBSD
The US-ASCII format was getting treated identically to POSIX. It is
supposed to throw an ILSEQ errno if a value of 0x80 or greater is
encountered, so let's bring back the "ASCII" handling.
While here, change nl_codeset to return US-ASCII only when the encoding
really is "US-ASCII". Before "C" and "POSIX" encoding returned this
string, so now they return "POSIX".
Discussed with: ache
Submitted by: marino
Obtained from: DragonflyBSD
as lib/libc/tests/gen
The code in test-fnmatch that was used for generating:
- bin/sh/tests/builtins/case2.0
- bin/sh/tests/builtins/case3.0
has been left undisturbed. The target `make sh-tests` has been moved over
from tools/regression/lib/libc/gen/Makefile to
lib/libc/tests/gen/Makefile and made into a PHONY target
case2.0 and case3.0 test input generation isn't being done automatically.
This needs additional discussion.
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- Remove a leftover printf from when this was a TAP based testcase
- Catch mmap failures properly
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
as lib/libc/tests/net
Also, fix eui64_aton_test:test_str(..). The test was comparing the result
of eui64_aton to a pointer of the expected result.
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
provided on amd64, but not i386. Add libm to DPADD/LDADD to unbreak the i386
tinderbox
Pointyhat to: ngie
MFC after: 1 week
X-MFC with: r290538
Sponsored by: EMC / Isilon Storage Division
as lib/libc/tests/stdlib
- Make the code a bit more style(9) compliant
- Convert a sizeof(x)/sizeof(x[0]) to nitems
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
libopenbsd is an internal library which
to bring in compatibility stuff from OpenBSD.
This will allow us to bring in more
OpenBSD utilities into the FreeBSD base system.
We similarly use libnetbsd for bringing in stuff from NetBSD.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D4078
Sorting eucJP text with "sort" resulted in an illegal sequence while
"gsort" worked. This was traced back to mbrtowc handling which was
broken for eucJP (probably eucCN, eucKR, and eucTW as well). This
small fix took hours to figure out. The OR operation to build the
wide character requires an unsigned character to work correctly. The
euc wcrtowc conversion is probably broken upstream in Illumos as well.
Triggered by: misc/freebsd-doc-ja in ports (encoded in eucJP)
Submitted by: marino
Obtained from: DragonflyBSD
The output of "locale charmap" is identical to the result of
nl_langinfo (CODESET) for any given locale. The logic for returning the
codeset was very simplistic. It just returned portion of the locale name
after the period (e.g. en_FR.ISO8859-1 returned "ISO8859-1").
When softlinks were added to locales, this broke. e.g.:
en_US returned ""
en_FR.UTF8 returned "UTF8"
en_FR.UTF-8 returned "UTF-8"
zh_Hant_HK.Big5HKSCS returned "Big5HKSCS"
zh_Hant_TW.Big5 returned "Big5"
es_ES@euro returned ""
In order to fix this properly, the named locale cannot be used to
determine the encoding. This information was almost available in the
rune data. Unfortunately, all the single byte encodings were listed
as "NONE" encoding.
So I adjusted localedef tool to provide more information about the
encoding. For example, instead of "NONE", the LC_CTYPE used by
fr_FR.ISO8859-15 is now encoded as "NONE:ISO8859-15". The locale
handlers now check if the first four characters of the encoding is
"NONE" and if so, treats it as a single-byte encoding.
The nl_langinfo handling of CODESET was adjusting accordingly. Now the
following is returned:
en_US returns "ISO8859-1"
fr_FR.UTF8 returns "UTF-8"
fr_FR.UTF-8 returns "UTF-8"
zh_Hant_HK.Big5HKSCS returns "Big5"
zh_Hant_TW.Big5 returns "Big5"
es_ES@euro returns "ISO8859-15"
as before, "C" and "POSIX" locales return "US-ASCII". This is a big
improvement. The result of nl_langinfo can never be a zero-length
string and it will always exclusively one of the values of the
character maps of /usr/src/tools/tools/locale/etc/final-maps.
Submitted by: marino
Obtained from: DragonflyBSD
Only enable h_raw on x86 targets for today so that a buildworld runs to
completion for clang enabled targets that are not x86. This should be
removed when validation of the sanitizer has occured for all targets
supported by FreeBSD and clang.
as lib/libc/rpc
This testcase requires rpcbind be up in running; otherwise the testcases
will time out and be skipped
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- Both curitem and curitem (via the names list) was always leaked.
- malloc(3) failures lead to some leaks.
- __bsd___iconv_get_list() failure lead to a crash since its error was not
handles and __bsd___iconv_free_list() is not NULL-safe.
I have slightly refactored this to avoid extra malloc and free logic in cases
of malloc(3) failing.
There are still bad assumptions here that I did not deal with. One of which is
that the data will always have a '/' so the strchr(3) will not return NULL.
Coverity CID: 1130055 1130054 1130053
Libedit's vi mode provides a v command to edit the current line in vi(1)
(hard-coded to vi, in fact).
When Unicode/wide character mode was added, this command started truncating
and/or corrupting the edited text.
This commit fixes v if the text fits into the buffer. If the text is longer,
it is truncated.
PR: 203743
Obtained from: NetBSD (originally submitted by me)
MK_NIS == no by converting `i` back to an int, and instead cast the loop
comparison to `int`
The loop comparison is iterating the len(ns_dtab)-1, because
the last element is the sentinel tuple { NULL, NULL, NULL, }, so when
both HESOID and NIS are off, len(ns_dtab)-1 == 1 - 1 == 0, and the loop
is skipped because the expression is tautologically false
While here, convert `(sizeof(x) / sizeof(x[0]))` to `nitems(x)`
Tested with: clang 3.7.0, gcc 4.2.1, and gcc 4.9.4 [*] with MK_NIS={no,yes}
and by running bash -lc 'id -u && id -g && id'
* gcc 4.9.4 needs another patch in order for the compile to succeed
with -Werror with lib/libc/gen/getgrent.c
Reported by: jhibbits
Through testing, the user noted that some Cyrillic characters were not
sorting correctly, and this was confirmed.
After extensive testing and review, the localedef tool was eliminated
as the culprit. The sustitutions were encoded correctly in LC_COLLATE.
The error was mainly in wcscoll where character expansions were
mishandled. The main directive pass routines had to be written to
go back for a new collation value when the "state" variable was set.
Before pointers were being advanced, the second lookup was gettting
applied to the wrong character, etc.
The "eat expansion codes" section on collate.c also had a bug. Later
own, the "state" variable logic was changed to only set if next
code was greater than zero (rather than >= 0).
Some additional cleanups got captured from previous work:
1) The previous commit moved the binary search comment from the
correct location to a wrong location because it's wrong upstream
in Illumos. The comment has little value so I just removed it.
2) Don't check if pointers are null before freeing, this is
redundant as free() handles null pointers.
3) The two binary search trees were standardized wrt initialization
4) On the binary search trees, a negative "high" exits rather than
checking the table count again.
Submitted by: marino
Obtained from: DragonflyBSD
In the past, _res was a global variable. Now, it's multiple function calls.
Several functions in the resolver use _res multiple times and therefore
call the function(s) far more than necessary.
Fix those callers to store the result of _res in a local variable.
Add __noinline to the definition of res_init() to avoid the code bloat
that these changes would have otherwise incurred. Thanks to jilles
for noticing this.
Reviewed by: jilles
MFC after: 1 week
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D3887
The main "fix" here is properly setting a collate loading error for each
early return. Tweaks include removing unnecessary null checks, adding
assertions (from Illumos) and a couple of variables to reduces code
differences and improve readability. For normal use, there are no
functional changes here.
Obtained from: DragonflyBSD, Illumos
It was not being used outside of META_MODE but this should make it more clear
that it is only for META_MODE.
Sponsored by: EMC / Isilon Storage Division
descriptor to avoid trashing valid file descriptors that access dev->fd at a
later point in time
PR: 192671
Submitted by: Scott Ferris <scott.ferris@isilon.com>
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- If the proxy returns a non-200 result, set the error code accordingly
so the caller / user gets a somewhat meaningful error message.
- Consume and discard any HTTP response header following the result line.
PR: 194483
Tested by: Fabian Keil <fk@fabiankeil.de>
MFC after: 1 week
On each resolver query, use stat(2) to see if the modification time
of /etc/resolv.conf has changed. If so, reload the file and reinitialize
the resolver library. However, only call stat(2) if at least two seconds
have passed since the last call to stat(2), since calling it on every
query could kill performance.
This new behavior is enabled by default. Add a "reload-period" option
to disable it or change the period of the test.
Document this behavior and option in resolv.conf(5).
Polish the man page just enough to appease igor.
https://lists.freebsd.org/pipermail/freebsd-arch/2015-October/017342.html
Reviewed by: kp, wblock
Discussed with: jilles, imp, alfred
MFC after: 1 month
Relnotes: yes
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D3867
FreeBSD extended ctypes to include numbers (e.g. isnumber()) but never
actually implemented it. The isnumber() function was equivalent to the
isdigit() function in every case.
Now that DragonFly's ctype source files have number definitions, the
number ctype can finally be implemented. It's given a new flag _CTYPE_N.
The isalnum() and iswalnum() functions have been changed to use this
flag rather than the _CTYPE_D digit flag.
While isalnum(), isnumber(), and their wide equivalents now return
different values in locale cases, the ishexnumber() and iswhexnumber()
functions are unchanged. They are still aliases for isxdigit() and
iswxdigit().
Also change ctype.h for isdigit and isxdigit to use sbistype like the
other functions.
Obtained from: dragonfly
the FreeBSD test suite
functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.
A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)
As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
netbsd-tests.test.mk (r289151)
- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
This removes the need for manually changing this flag for Google Chrome
users. It also improves compatibility with Linux applications running under
Linuxulator compatibility layer, and possibly also helps in porting software
from Linux.
Generally speaking, the flag allows applications to create the shared memory
segment, attach it, remove it, and then continue to use it and to reattach it
later. This means that the kernel will automatically "clean up" after the
application exits.
It could be argued that it's against POSIX. However, SUSv3 says this
about IPC_RMID: "Remove the shared memory identifier specified by shmid from
the system and destroy the shared memory segment and shmid_ds data structure
associated with it." From my reading, we break it in any case by deferring
removal of the segment until it's detached; we won't break it any more
by also deferring removal of the identifier.
This is the behaviour exhibited by Linux since... probably always, and
also by OpenBSD since the following commit:
revision 1.54
date: 2011/10/27 07:56:28; author: robert; state: Exp; lines: +3 -8;
Allow segments to be used even after they were marked for deletion with
the IPC_RMID flag.
This is permitted as an extension beyond the standards and this is similar
to what other operating systems like linux do.
MFC after: 1 month
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3603
This uses the kdump(1) utrace support code directly until a common library
is created.
This allows malloc(3) tracing with MALLOC_CONF=utrace:true and rtld tracing
with LD_UTRACE=1. Unknown utrace(2) data is just printed as hex.
PR: 43819 [inspired by]
Reviewed by: jhb
MFC after: 2 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D3819
the target is "make depend". This works around errors during
incremental make depend of some clang libraries, for example "don't know
how to make contrib/llvm/include/llvm/IR/IntrinsicsR600.td".
Reported by: emaste
This fix is spiritually similar to r287442 and was discovered thanks to
the KASSERT added in that revision.
NT_PROCSTAT_VMMAP output length, when packing kinfo structs, is tied to
the length of filenames corresponding to vnodes in the process' vm map
via vn_fullpath. As vnodes may move during coredump, this is racy.
We do not remove the race, only prevent it from causing coredump
corruption.
- Add a sysctl, kern.coredump_pack_vmmapinfo, to allow users to disable
kinfo packing for PROCSTAT_VMMAP notes. This avoids VMMAP corruption
and truncation, even if names change, at the cost of up to PATH_MAX
bytes per mapped object. The new sysctl is documented in core.5.
- Fix note_procstat_vmmap to self-limit in the second pass. This
addresses corruption, at the cost of sometimes producing a truncated
result.
- Fix PROCSTAT_VMMAP consumers libutil (and libprocstat, via copy-paste)
to grok the new zero padding.
Reported by: pho (https://people.freebsd.org/~pho/stress/log/datamove4-2.txt)
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3824
The functional_test.sh harness for each test subdir was inspired
by the version in bin/sh/tests/functional_test.sh
Some gymnastics were required to deal with implicit rules for
.c / .o -> .out as the suffix transformation rules were
incorrectly trying to create the test outputs from some of the
source files
Sponsored by: EMC / Isilon Storage Division
The latter is already defined in bsd.libnames.mk, so avoid the conflict
in case someone copy-pastes make variables
While here, switch path to the top of the source tree with SRCTOP
Shell syntax is too complicated to detect command substitution and unquoted
operators reliably without implementing much of sh's parser. Therefore, have
sh do this detection.
While changing sh's support anyway, also read input from a pipe instead of
arguments to avoid {ARG_MAX} limits and improve privacy, and output count
and length using 16 instead of 8 digits.
The basic concept is:
execl("/bin/sh", "sh", "-c", "freebsd_wordexp ${1:+\"$1\"} -f "$2",
"", flags & WRDE_NOCMD ? "-p" : "", <pipe with words>);
The WRDE_BADCHAR error is still implemented in libc. POSIX requires us to
fail strings containing unquoted braces with code WRDE_BADCHAR. Since this
is normally not a syntax error in sh, there is still a need for checking
code in libc, we_check().
The new we_check() is an optimistic check that all the characters
<newline> | & ; < > ( ) { }
are quoted. To avoid duplicating too much sh logic, such characters are
permitted when quoting characters are seen, even if the quoting characters
may themselves be quoted. This code reports all WRDE_BADCHAR errors; bad
characters that get past it and are a syntax error in sh return WRDE_SYNTAX.
Although many implementations of WRDE_NOCMD erroneously allow some command
substitutions (and ours even documented this), there appears to be code that
relies on its security (codesearch.debian.net shows quite a few uses).
Passing untrusted data to wordexp() still exposes a denial of service
possibility and a fairly large attack surface.
Reviewed by: wblock (man page only)
MFC after: 2 weeks
Relnotes: yes
Security: fixes command execution with wordexp(untrusted, WRDE_NOCMD)
The old code was exponential in the number of asterisks in the pattern.
However, once a match has been found upto the next asterisk, the previous
asterisks are no longer relevant.
FILES is not used when LIBRARIES_ONLY is set, which is used to build and
install the lib32 sysroot. All of the csu files do quality as "libraries"
for this case so just undefine LIBRARIES_ONLY.
This is still better than the previous realinstall handling as it does
not hook into META_MODE properly.
Sponsored by: EMC / Isilon Storage Division
This partially reverts r270170 for lib/csu/i386 while retaining the
change for using bsd.lib.mk.
These FILES groups could go into lib/csu/Makefile.inc but I've kept them
in the Makefiles for clarity.
Sponsored by: EMC / Isilon Storage Division
both in /usr/lib and /usr/local/lib, thus simplifying the use of modules
from ports, without breaking the compat32 case again.
PR: 191151
MFC after: 3 weeks
- In a PF_LOCAL address, "hostname" must begins with '/' and "servname"
is always NULL. All of ai_flags are ignored.
- PF_UNSPEC matches PF_LOCAL. EAI_SERVICE is not returned to make
AF-independent programming easier; "servname" is always ignored
in PF_LOCAL. In practice, PF_INET* and PF_LOCAL are
mutually-exclusive because a hostname which begins with '/' is invalid
in PF_INET*. No domain name resolution is performed for a PF_LOCAL address.
Differential Revision: https://reviews.freebsd.org/D3634
Note that the mountlist manipulations are somewhat fragile, and not very
pretty. The reason for this is to avoid changing vfs_mountroot(), which
is (obviously) rather mission-critical, but not very well documented,
and thus hard to test properly. It might be possible to rework it to use
its own simple root mount mechanism instead of vfs_mountroot().
Reviewed by: kib@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2698
This silence a warning brought up by valgrind whenever if_nametoindex
is used. This was already discussed in PR 166483, but the code
committed in r234329 guards the initilization with #ifdef PURIFY.
Therefore, valgrind still complains. Since this code is not performance
critical, always zero out the local variable to silence valgrind.
PR: 166483
Discussed with: eadler@
MFC after: 4 weeks
calling thread is supposed to see accesses issued by the initializer.
This means that the read of the once_control->state variable should
have an acquire semantic. Use atomic_thread_fence_acq() when the
value read is ONCE_DONE, instead of straightforward atomic_load_acq(),
to only put a barrier when needed (*).
On the other hand, the updates of the once_control->state with the
intermediate progress state do not need to synchronize with other
state accesses, remove _acq suffix.
Reviewed by: alc (previous version)
Suggested by: alc (*)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Coredump notes depend on being able to invoke dump routines twice; once
in a dry-run mode to get the size of the note, and another to actually
emit the note to the corefile.
When a note helper emits a different length section the second time
around than the length it requested the first time, the kernel produces
a corrupt coredump.
NT_PROCSTAT_FILES output length, when packing kinfo structs, is tied to
the length of filenames corresponding to vnodes in the process' fd table
via vn_fullpath. As vnodes may move around during dump, this is racy.
So:
- Detect badly behaved notes in putnote() and pad underfilled notes.
- Add a fail point, debug.fail_point.fill_kinfo_vnode__random_path to
exercise the NT_PROCSTAT_FILES corruption. It simply picks random
lengths to expand or truncate paths to in fo_fill_kinfo_vnode().
- Add a sysctl, kern.coredump_pack_fileinfo, to allow users to
disable kinfo packing for PROCSTAT_FILES notes. This should avoid
both FILES note corruption and truncation, even if filenames change,
at the cost of about 1 kiB in padding bloat per open fd. Document
the new sysctl in core.5.
- Fix note_procstat_files to self-limit in the 2nd pass. Since
sometimes this will result in a short write, pad up to our advertised
size. This addresses note corruption, at the risk of sometimes
truncating the last several fd info entries.
- Fix NT_PROCSTAT_FILES consumers libutil and libprocstat to grok the
zero padding.
With suggestions from: bjk, jhb, kib, wblock
Approved by: markj (mentor)
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3548
the buffer. (n == hostlen) also means the buffer length was
too short.
- Use sdl->sdl_data only when (sdl->sdl_nlen > 0 && sdl->sdl_alen == 0)
to prevent redundant output.
Connect it to the build.
The code assumed that SCHED_* constants form a contiguous set of
numbers, remove the assumption by using schedulers[] array in
get_different_scheduler(). This is no-op on FreeBSD, but improves
code portability.
The selection of different priority used the min/max priority range of
the current scheduler class, instead of the priority to be changed to.
The bug caused the test failure.
Remove duplication of POSIX_SPAWN_SETSIGDEF flag and now unused
duplications of MIN/MAX definitions.
Reviewed by: jilles, pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D3533
comment above, POSIX_SPAWN_SETSIGMASK and POSIX_SPAWN_SETSIGDEF
handlers used libthr interposed functions instead of syscalls.
Noted by: jilles
Sponsored by: The FreeBSD Foundation
MFC after: 6 days
are aliases for the syscall stubs and are plt-interposed, to the
libc-private aliases of internally interposed sigprocmask() etc.
Since e.g. _sigaction is not interposed by libthr, calling signal()
removes thr_sighandler() from the handler slot etc. The result was
breaking signal semantic and rtld locking.
The added __libc_sigprocmask and other symbols are hidden, they are
not exported and cannot be called through PLT. The setjmp/longjmp
functions for x86 were changed to use direct calls, and since
PIC_PROLOGUE only needed for functional PLT indirection on i386, it is
removed as well.
The PowerPC bug of calling the syscall directly in the setjmp/longjmp
implementation is kept as is.
Reported by: Pete French <petefrench@ingresso.co.uk>
Tested by: Michiel Boland <boland37@xs4all.nl>
Reviewed by: jilles (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Go ahead and defined -D_STANDALONE for all targets (only strictly
needed for some architecture, but harmless on those it isn't required
for). Also add -msoft-float to all architectures uniformly rather
that higgley piggley like it is today.
Differential Revision: https://reviews.freebsd.org/D3496
- in mbrtowc() we need to disallow codepoints above 0x10ffff.
- In wcrtomb() we need to disallow codepoints between 0xd800 and 0xdfff.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D3399
adjusted. This seems to be the case on all non-x86 architectures libproc
supports.
Reviewed by: kib
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3465
