The setkey() and encrypt() functions are part of XSI, not the POSIX base
definitions. There is no strict requirement for us to provide these,
especially if we're only going to keep these around as undocumented
stubs. The same holds for des_setkey() and des_cipher().
Instead of providing functions that only generate warnings when linking,
simply disallow linking against them. The impact of this is relatively
low. It only causes two leaf ports to break. I'll see what I can do to
help out to get those fixed.
PR: 211626
The sysctl cannot fail. If it does fail on some FreeBSD derivative or
after some future change, just abort() so that the problem will be found
and fixed.
It's preferable to provide an arc4random() function that cannot fail and
cannot return poor quality random data. While abort() is not normally
suitable for a library, it makes sense here.
Reviewed by: ed, jonathan, markm
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8077
It turns out that the path normalization that our brand new copy of
dirname(3) does is actually not allowed by the draft version of the
upcoming version of POSIX. It has to behave identically to the
dirname(1) utility.
This change replaces our new dirname(3) implementation by yet another
version that doesn't implement the path normalization logic; it merely
looks for the end of the directory name and overwrites that with a null
byte.
More details: See note #3370 at http://austingroupbugs.net/view.php?id=1073
PR: 212193
Reviewed by: emaste, jilles
Differential Revision: https://reviews.freebsd.org/D7790
In existing implementations including FreeBSD, there is no reason to use
readdir_r() in the common case where potentially multiple threads each list
their own directory. Code using readdir() is simpler.
What's more, lthough readdir_r() can safely be used on FreeBSD because
NAME_MAX is forced to 255, it cannot be used safely on systems where
{NAME_MAX} is not fixed. As a concrete example, FAT/NTFS filenames can be up
to 255 UTF-16 code units long, which can be up to 765 UTF-8 bytes.
Deprecating readdir_r() in POSIX has been proposed in
http://www.austingroupbugs.net/view.php?id=696
and glibc wants to deprecate it as well.
Reviewed by: ed, wblock
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D7678
As the xinstall(8) utility had to be patched up to work with the POSIXly
correct basename()/dirname() prototypes, we make it pretty hard to build
previous versions of FreeBSD on HEAD. xinstall(8) is part of the
bootstrap tools.
Add some logic to <libgen.h> to automatically detect bad calls to
dirname() based on the type of the argument. If the argument is of type
'const char *', we simply fall back to calling into dirname@FBSD_1.0
directly.
I'll also give basename() similar treatment when importing the
thread-safe version of that function.
Tested by: bdrewery, madpilot (thanks!)
Now that we've updated the prototypes of the basename(3) and dirname(3)
functions to conform to POSIX, let's go ahead and reimplement dirname(3)
in such a way that it's thread-safe, but also guaranteed to succeed. C
libraries like glibc, musl and the one that's part of Solaris already
follow such an approach.
Move the existing implementation to another source file,
freebsd11_dirname.c to keep existing users of the API that pass in a
constant string happy, using symbol versioning.
Put a new version of the function in dirname.c, obtained from CloudABI's
C library. This version scans through the pathname string from left to
right, normalizing it, while discarding the last pathname component.
Reviewed by: emaste, jilles
Differential Revision: https://reviews.freebsd.org/D7355
two sepatate functions to make glob(3) code less obscure and more simple.
There is no needs to make them inline since it is error path which supposed
to not happes often.
Just like with freelocale(3), I haven't been able to find any piece of
code that actually makes use of this function's return value, both in
base and in ports. The reason for this is that FreeBSD seems to be the
only operating system to have such a prototype. This is why I'm deciding
to not use symbol versioning for this.
It does seem that the pw(8) utility depends on the function's typing and
already had a switch in place to toggle between the FreeBSD and POSIX
variant of this function. Clean this up by always expecting the POSIX
variant.
There is also a single port that has a couple of local declarations of
setgrent(3) that need to be patched up. This is in the process of being
fixed.
PR: 211394 (exp-run)
to 0. Breaking this rule in 2001 NetBSD hack was imported which attempts
to workaround very limited glob() return codes amount. Use POSIX-compatible
workaround now with E2BIG which can't comes from other functions used
instead of prohibited 0.
Update the existing manual pages for basename(3) and dirname(3) to
mention that in future versions of FreeBSD, these functions will no
longer use internal buffers for storing the results.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D7356
POSIX allows these functions to be implemented in a way that the
resulting string is stored in the input buffer. Though some may find
this annoying, this has the advantage that it makes it possible to
implement this function in a thread-safe way. It also means that they
can be implemented in a way that they work for paths of arbitrary
length, as the output string of these functions is never longer than
max(1, len(input)).
Portable code already needs to be written with this in mind, so in my
opinion it makes very little sense to allow the existing behaviour.
Prevent the base system from falling back to this by switching over to
POSIX prototypes.
I'm not going to bump the __FreeBSD_version for this. The reason is that
it's possible to account for this change in a portable way, without
depending on a specific version of FreeBSD. An exp-run was done some
time ago. As far as I know, all regressions as a result of this have
already been fixed.
I'll give this change some time to settle. In the long run I want to
replace our copies by ones that are thread-safe and don't depend on
PATH_MAX/MAXPATHLEN.
case it fully constructed, but for half-constructed too, so have no
other choice to pass original pattern from glob() down to globextend()
instead of attempt to reconstruct I implement previously.
2) Instead of copy&paste the same big enough code, make function for it:
globfinal().
globexp1() recursive calls, but glob0() was not supposed to be called
repeatedly in the original code. It finalize results by possible adding
original pattern for no match case, may return GLOB_NOMATCH error and
by sorting all things. Original pattern adding or GLOB_NOMATCH error
can happens each time glob0() called repeatedly, and sorting happens
for one item only, all things are never sorted. Second, f.e. "a{a"
pattern does not match "a{a" file but match "a" file instead
(just one example, there are many). Third, some errors (f.e. for limits
or overflow) can be ignored by GLOB_BRACE code because it forces return (0).
Add non-finalizing flag to glob0() and make globexp0() wrapper around
recursively called globexp1() to finalize things like glob0() does.
Reorganize braces code to work correctly.
2) Don't allow MB_CUR_MAX * strlen overallocation hits GLOB_LIMIT_STRING
(ARG_MAX) limit, use final string length, not malloced space for it.
3) Revive DEBUG-ifdefed section.
unmodified, if no matches found. But our original code strips all '\'
returning it. Rewrite the code to allow to reconstruct exact the
original pattern with backslashes for this case.
2) Prevent to use truncated pattern if MAXPATHLEN exceeded, return
GLOB_NOMATCH instead.
3) Fix few end loop conditions filling Char arrays with mbrtowc(),
MB_CUR_MAX is unneeded in two places and condition is less by one
in other place.
4) Prevent to use truncated filenames match if MAXPATHLEN exceeded,
skip such directory entries.
5) Don't end *pathend with L'/' in glob3() if limit is reached, this
change will be not visible since error is returned.
6) If error happens in (*readdirfunc)(), do the same GLOB_ABORTED
processing as for g_opendir() as POSIX requires.
in any case and needed for further processing. For ~ expansion too.
2) Don't terminate *pathend with / when GLOB_LIMIT_STAT is reached, it will
be not visible outside in any case since error is returned.
3) Cosmetic: change if expression to better reflect its semantic.
since whole conversion needs a room for (len >= MB_CUR_MAX). It is no
difference when MB_CUR_MAX == 1, but for multi-byte locales last few chars
('\0' and before) may need just one byte, and the rest of MB_CUR_MAX - 1
space becomes unavailable in the MAXPATHLEN-sized buffer, which cause
conversion error on near MAXPATHLEN long pathes.
Increase g_Ctoc() conversion buffers to MB_LEN_MAX - 1.
as pattern meta chars.
2) GLOB_ERR and gl_errfunc are supposed to work only for real directories
per POSIX, so don't act on missing or plain files, for ENOENT or ENOTDIR
(as TODO in the code suggested).
3) Remove the hack in the manpage describing how to skip ENOENT and ENOTDIR
in gl_errfunc, it is unneeded now.
4) Set errno to ENAMETOOLONG if g_Ctoc() expansion fails in g_opendir(),
as in other places in the code which are wrappers around system functions.
open/read errors and with GLOB_ERR and gl_errfunc processing), so we can't
blindly return it on any MAXPATHLEN overflow. Even our manpage disagrees
with such GLOB_ABORTED usage. Use GLOB_NOSPACE for that now with errno is
set to 0 as for limits.
2) Return GLOB_NOSPACE when valid ~ expansion can't happens due to
MAXPATHLEN overflow too.
3) POSIX (and our manpage) says, if GLOB_ERR is set, GLOB_ABORTED should
be returned immediatelly, without using gl_errfunc. Implement it now.
them down. This hack still remains:
* 2. Illegal byte sequences in filenames are handled by treating them as
* single-byte characters with a values of such bytes of the sequence
* cast to wchar_t.
2) Reword the comment in the hack above to reflect implementation.
3) Protect signed wchar_t from sign extension when a signed char is assigned
to it in the hack above.
3) Corresponding backward hack in g_Ctoc() was not implemented, so all
pathes with illegal byte sequences are skipped as result, implement it now.
4) globtilde() forget to convert expanded user home dir from multibyte to
wchar.
5) Protect globtilde() from long expansion truncation.
6) Results was not sorted according to collate as POSIX requires.
locale (which cause core dump) by removing whole 'table' argument
by which it passed.
2) Restore __collate_range_cmp() in __sccl().
3) Collating [a-z] range in regcomp() only for single bytes locales
(we can't do it now for other ones). In previous state only first 256
wchars are considered and all others are just silently dropped from the
range.
Instead of changing whole course to another POSIX-permitted way
for consistency and uniformity I decide to completely ignore missing
regex fucntionality and concentrace on fixing bugs in what we have now,
too many small obstacles instead, counting ports.
Only first 256 wide chars are considered currently, all other are just
dropped from the range. Proper implementation require reverse tables
database lookup, since objects are really big as max UTF-8 (1114112
code points), so just the same scanning as it was for 256 chars will
slow things down.
POSIX does not require collation for [a-z] type ranges and does not
prohibit it for non-POSIX locales. POSIX require collation for ranges
only for POSIX (or C) locale which is equal to ASCII and binary for
other chars, so we already have it.
No other *BSD implements collation for [a-z] type ranges.
Restore ABI compatibility with unused now __collate_range_cmp() which
is visible from outside (will be removed later).
Setting time by seconds or microseconds may cause unexpected effects
especially if sysctl vfs.timestamp_precision=3 (not default).
Calling the obsolete functions with NULL timestamps is acceptable.
This support appears to have been documented in nsswitch.conf(5) for some
time. The implementation adds two NSS netgroup providers to libc. The
default, compat, provides the behaviour documented in netgroup(5), so this
change does not make any user-visible behaviour changes. A files provider
is also implemented.
innetgr(3) is implemented as an optional NSS method so that providers such
as NIS which are able to implement efficient reverse lookup can do so.
A fallback implementation is used otherwise. getnetgrent_r(3) is added for
convenience and to provide compatibility with glibc and Solaris.
With a small patch to net/nss_ldap, it's possible to specify an ldap
netgroup provider, allowing one to query nisNetgroupTriple entries.
Sponsored by: EMC / Isilon Storage Division
Handle an empty result from yp_match() by returning NULL, which is
consistent with the handling of an empty netgroup in /etc/netgroup.
setnetgrent(3) has no return value, so there is no particular need to
distinguish this case from an error.
PR: 26486
MFC after: 2 weeks
This adds stravis() and some new encoding flags VIS_SHELL, VIS_META,
and VIS_NOLOCALE.
Assorted cleanups and fixes includeing a manpage typo[0].
PR: 210013 [0]
Submitted by: pi [0]
If malloc() fails to allocate linep, then free olinep (if it exists)
before returning to avoid a memory leak.
Reported by: Coverity
CID: 1016716
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D6755
When collation support was brought in, the second and third
arguments in __collate_range_cmp() were changed from int to
wchar_t, breaking the ABI. Change them to a "char" type which
makes more sense and keeps the ABI compatible.
Also introduce __wcollate_range_cmp() which does work with wide
characters. This function is used only internally in libc so
we don't export it. Use the new function in glob(3), fnmatch(3),
and regexec(3).
PR: 179721
Suggested by: ache. jilles
MFC after: 3 weeks (perhaps partial only)
POSIX 2008 added the psignal() function which has already been part of
the BSDs for a long time. The only difference is, the POSIX version uses
an 'int' for the signal number, unlike our version which uses an
'unsigned int'. Fix up the function to use an 'int'. This should not
affect the ABI.
intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013.
A robust mutex is guaranteed to be cleared by the system upon either
thread or process owner termination while the mutex is held. The next
mutex locker is then notified about inconsistent mutex state and can
execute (or abandon) corrective actions.
The patch mostly consists of small changes here and there, adding
neccessary checks for the inconsistent and abandoned conditions into
existing paths. Additionally, the thread exit handler was extended to
iterate over the userspace-maintained list of owned robust mutexes,
unlocking and marking as terminated each of them.
The list of owned robust mutexes cannot be maintained atomically
synchronous with the mutex lock state (it is possible in kernel, but
is too expensive). Instead, for the duration of lock or unlock
operation, the current mutex is remembered in a special slot that is
also checked by the kernel at thread termination.
Kernel must be aware about the per-thread location of the heads of
robust mutex lists and the current active mutex slot. When a thread
touches a robust mutex for the first time, a new umtx op syscall is
issued which informs about location of lists heads.
The umtx sleep queues for PP and PI mutexes are split between
non-robust and robust.
Somewhat unrelated changes in the patch:
1. Style.
2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared
pi mutexes.
3. Removal of the userspace struct pthread_mutex m_owner field.
4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls
the lifetime of the shared mutex associated with a vnode' page.
Reviewed by: jilles (previous version, supposedly the objection was fixed)
Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects)
Tested by: pho
Sponsored by: The FreeBSD Foundation
According to style(9):
> normally, include <sys/types.h> OR <sys/param.h>, but not both.
(<sys/param.h> already includes <sys/types.h> when LOCORE is not defined).
We have an nitems() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
Given that it is available already without adding additional
headers and other parts of libc already use it, extend a bit
more its use.
In r103767 the kern.ps_strings sysctl was added as the preferred way to
locate the ps_strings struct and is available in any FreeBSD release
supported within the last decade.
Reviewed by: kib
While here also cleanup some surrounding code; particularly
drop some malloc() casts.
Found with devel/coccinelle.
Reviewed by: bde (previous version - all new bugs are mine)
do not participate in the global symbols namespace, but rtld locks are
still replaced and functions are interposed. In particular,
__pthread_map_stacks_exec is resolved to the libc version. If a
library is loaded later, which requires adjustment of the stack
protection mode, rtld calls into libc __pthread_map_stacks_exec due to
the symbols scope. The libc version might recurse into binder and
recursively acquire rtld bind lock, causing the hang.
Make libc __pthread_map_stacks_exec() interposed, which synchronizes
rtld locks and version of the stack exec hook when libthr loaded,
regardless of the symbol scope control or symbol resolution order.
The __pthread_map_stacks_exec() symbol is removed from the private
version in libthr since libc symbol now operates correctly in presence
of libthr.
Reported and tested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
wraps sendmsg(2) and recvmsg(2) into batch send and receive operation.
The goal of this implementation is only to provide API compatibility
with Linux.
The cancellation behaviour of the functions is not quite right, but
due to relative rare use of cancellation it is considered acceptable
comparing with the complexity of the correct implementation. If
functions are reimplemented as syscalls, the fix would come almost
trivial. The direct use of the syscall trampolines instead of libc
wrappers for sendmsg(2) and recvmsg(2) is to avoid data loss on
cancellation.
Submitted by: Boris Astardzhiev <boris.astardzhiev@gmail.com>
Discussed with: jilles (cancellation behaviour)
MFC after: 1 month
by application closing its stdin (i.e. STDIN_FILENO) prior to
calling readpassphrase WITHOUT setting RPP_STDIN. What happens
then is that the readpassphrase would open /dev/tty, and since
file descriptors are reused, the call would return first unused
fd, which is 0 which is also STDIN_FILENO. Then due to the usage
of "input != STDIN_FILENO" in the code to do its logic, that
would result in noecho flags not set on that file descriptor,
which was original issue I've been trying to fix.
In addition to that, the readpassphrase() would leak file
descriptor on its way out, so fix that one as well.
This problem can be tested with:
$ ssh-add - < /tmp/myprivate.key
The password will not be hidden as it should and ktrace will
show:
53326 ssh-add CALL open(0x80142443c,0x100002<O_RDWR|O_CLOEXEC>,<unused>0x165f030)
53326 ssh-add NAMI "/dev/tty"
53326 ssh-add RET open 0
53326 ssh-add CALL sigprocmask(SIG_SETMASK,0x802eb1324,0x7fffffffd5e0)
53326 ssh-add RET sigprocmask 0
53326 ssh-add CALL sigaction(SIGALRM,0x7fffffffd630,0x7fffffffd610)
Instead of:
57690 ssh-add CALL open(0x80142443c,0x100002<O_RDWR|O_CLOEXEC>,<unused>0x165f030)
57690 ssh-add NAMI "/dev/tty"
57690 ssh-add RET open 4
57690 ssh-add CALL ioctl(0x4,TIOCGETA,0x7fffffffd860)
57690 ssh-add RET ioctl 0
57690 ssh-add CALL ioctl(0x4,TIOCSETAF,0x7fffffffd680)
57690 ssh-add RET ioctl 0
57690 ssh-add CALL sigprocmask(SIG_SETMASK,0x802eb1324,0x7fffffffd620)
57690 ssh-add RET sigprocmask 0
57690 ssh-add CALL sigaction(SIGALRM,0x7fffffffd670,0x7fffffffd650)
For the case when the key is read from the file.
Technically this can also be workaround'ed at the application side
by not closing the STDIN_FILENO in the first place, but readpassphrase(3)
doesn't need to make any assumptions about that. Plus the file descriptor
leak confirms that this is an oversight, rather than a deliberate behaviour.
MFC after: 1 week
This bug could be reproduced easily by calling sem_open() with O_CREAT |
O_EXCL on a semaphore that is already open in the process. The struct
sem_nameinfo would be freed while still in sem_list and later calls to
sem_open() or sem_close() could access freed memory.
PR: 206396
MFC after: 5 days
Set _PATH_DEFPATH to
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin. This is the
path in the default class in the default /etc/login.conf,
excluding ~/bin which would not be expanded properly in a string
constant.
For normal logins, _PATH_DEFPATH is overridden by /etc/login.conf,
~/.login_conf or shell startup files. _PATH_DEFPATH is still used as a
default by execlp(), execvp(), posix_spawnp() and sh if PATH is not set, and
by cron. Especially the latter is a common trap (most recently in PR
204813).
PR: 204813
Reviewed by: secteam (delphij), alfred
i386 is the only current FreeBSD architecture that ever used a.out
format.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D4687
Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file. There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.
Sponsored by: EMC / Isilon Storage Division
Move fdopen() up near other resource allocation like malloc(); do proper
deallocation on failure later on in the function.
Submitted by: Ramachandra Topannavar <rtopannavar@panasas.com>
Reviewed by: jilles
Approved by: jhb (mentor)
MFC after: 2 weeks
Sponsored by: Panasas, Inc.
Differential Revision: https://reviews.freebsd.org/D4126
M lib/libc/gen/popen.c
MK_NIS == no by converting `i` back to an int, and instead cast the loop
comparison to `int`
The loop comparison is iterating the len(ns_dtab)-1, because
the last element is the sentinel tuple { NULL, NULL, NULL, }, so when
both HESOID and NIS are off, len(ns_dtab)-1 == 1 - 1 == 0, and the loop
is skipped because the expression is tautologically false
While here, convert `(sizeof(x) / sizeof(x[0]))` to `nitems(x)`
Tested with: clang 3.7.0, gcc 4.2.1, and gcc 4.9.4 [*] with MK_NIS={no,yes}
and by running bash -lc 'id -u && id -g && id'
* gcc 4.9.4 needs another patch in order for the compile to succeed
with -Werror with lib/libc/gen/getgrent.c
Reported by: jhibbits
Shell syntax is too complicated to detect command substitution and unquoted
operators reliably without implementing much of sh's parser. Therefore, have
sh do this detection.
While changing sh's support anyway, also read input from a pipe instead of
arguments to avoid {ARG_MAX} limits and improve privacy, and output count
and length using 16 instead of 8 digits.
The basic concept is:
execl("/bin/sh", "sh", "-c", "freebsd_wordexp ${1:+\"$1\"} -f "$2",
"", flags & WRDE_NOCMD ? "-p" : "", <pipe with words>);
The WRDE_BADCHAR error is still implemented in libc. POSIX requires us to
fail strings containing unquoted braces with code WRDE_BADCHAR. Since this
is normally not a syntax error in sh, there is still a need for checking
code in libc, we_check().
The new we_check() is an optimistic check that all the characters
<newline> | & ; < > ( ) { }
are quoted. To avoid duplicating too much sh logic, such characters are
permitted when quoting characters are seen, even if the quoting characters
may themselves be quoted. This code reports all WRDE_BADCHAR errors; bad
characters that get past it and are a syntax error in sh return WRDE_SYNTAX.
Although many implementations of WRDE_NOCMD erroneously allow some command
substitutions (and ours even documented this), there appears to be code that
relies on its security (codesearch.debian.net shows quite a few uses).
Passing untrusted data to wordexp() still exposes a denial of service
possibility and a fairly large attack surface.
Reviewed by: wblock (man page only)
MFC after: 2 weeks
Relnotes: yes
Security: fixes command execution with wordexp(untrusted, WRDE_NOCMD)
The old code was exponential in the number of asterisks in the pattern.
However, once a match has been found upto the next asterisk, the previous
asterisks are no longer relevant.
comment above, POSIX_SPAWN_SETSIGMASK and POSIX_SPAWN_SETSIGDEF
handlers used libthr interposed functions instead of syscalls.
Noted by: jilles
Sponsored by: The FreeBSD Foundation
MFC after: 6 days
are aliases for the syscall stubs and are plt-interposed, to the
libc-private aliases of internally interposed sigprocmask() etc.
Since e.g. _sigaction is not interposed by libthr, calling signal()
removes thr_sighandler() from the handler slot etc. The result was
breaking signal semantic and rtld locking.
The added __libc_sigprocmask and other symbols are hidden, they are
not exported and cannot be called through PLT. The setjmp/longjmp
functions for x86 were changed to use direct calls, and since
PIC_PROLOGUE only needed for functional PLT indirection on i386, it is
removed as well.
The PowerPC bug of calling the syscall directly in the setjmp/longjmp
implementation is kept as is.
Reported by: Pete French <petefrench@ingresso.co.uk>
Tested by: Michiel Boland <boland37@xs4all.nl>
Reviewed by: jilles (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Distinguish between WRDE_BADVAL and WRDE_SYNTAX based on when the error
occurred (parsing or execution), not based on whether WRDE_UNDEF was passed.
Also, return WRDE_NOSPACE for a few more unexpected results from sh.
The functionality of the wordexp builtin is easily replaced using normal
shell code, although performance is slightly worse.
This does not mean that wordexp() will remain shell-independent -- a fully
reliable implementation of WRDE_NOCMD is really only possible using
extensions to the shell, or by adding much of the shell's code to libc.
the 'user' sysctl tree, which have all been coming back 0 or empty
since r240176.
Differential Revision: https://reviews.freebsd.org/D2945
Reviewed by: sbruno
Approved by: jmallett (mentor)
MFC after: 3 days
