In existing implementations including FreeBSD, there is no reason to use
readdir_r() in the common case where potentially multiple threads each list
their own directory. Code using readdir() is simpler.
What's more, lthough readdir_r() can safely be used on FreeBSD because
NAME_MAX is forced to 255, it cannot be used safely on systems where
{NAME_MAX} is not fixed. As a concrete example, FAT/NTFS filenames can be up
to 255 UTF-16 code units long, which can be up to 765 UTF-8 bytes.
Deprecating readdir_r() in POSIX has been proposed in
http://www.austingroupbugs.net/view.php?id=696
and glibc wants to deprecate it as well.
Reviewed by: ed, wblock
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D7678
As the xinstall(8) utility had to be patched up to work with the POSIXly
correct basename()/dirname() prototypes, we make it pretty hard to build
previous versions of FreeBSD on HEAD. xinstall(8) is part of the
bootstrap tools.
Add some logic to <libgen.h> to automatically detect bad calls to
dirname() based on the type of the argument. If the argument is of type
'const char *', we simply fall back to calling into dirname@FBSD_1.0
directly.
I'll also give basename() similar treatment when importing the
thread-safe version of that function.
Tested by: bdrewery, madpilot (thanks!)
Now that we've updated the prototypes of the basename(3) and dirname(3)
functions to conform to POSIX, let's go ahead and reimplement dirname(3)
in such a way that it's thread-safe, but also guaranteed to succeed. C
libraries like glibc, musl and the one that's part of Solaris already
follow such an approach.
Move the existing implementation to another source file,
freebsd11_dirname.c to keep existing users of the API that pass in a
constant string happy, using symbol versioning.
Put a new version of the function in dirname.c, obtained from CloudABI's
C library. This version scans through the pathname string from left to
right, normalizing it, while discarding the last pathname component.
Reviewed by: emaste, jilles
Differential Revision: https://reviews.freebsd.org/D7355
two sepatate functions to make glob(3) code less obscure and more simple.
There is no needs to make them inline since it is error path which supposed
to not happes often.
Just like with freelocale(3), I haven't been able to find any piece of
code that actually makes use of this function's return value, both in
base and in ports. The reason for this is that FreeBSD seems to be the
only operating system to have such a prototype. This is why I'm deciding
to not use symbol versioning for this.
It does seem that the pw(8) utility depends on the function's typing and
already had a switch in place to toggle between the FreeBSD and POSIX
variant of this function. Clean this up by always expecting the POSIX
variant.
There is also a single port that has a couple of local declarations of
setgrent(3) that need to be patched up. This is in the process of being
fixed.
PR: 211394 (exp-run)
to 0. Breaking this rule in 2001 NetBSD hack was imported which attempts
to workaround very limited glob() return codes amount. Use POSIX-compatible
workaround now with E2BIG which can't comes from other functions used
instead of prohibited 0.
Update the existing manual pages for basename(3) and dirname(3) to
mention that in future versions of FreeBSD, these functions will no
longer use internal buffers for storing the results.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D7356
POSIX allows these functions to be implemented in a way that the
resulting string is stored in the input buffer. Though some may find
this annoying, this has the advantage that it makes it possible to
implement this function in a thread-safe way. It also means that they
can be implemented in a way that they work for paths of arbitrary
length, as the output string of these functions is never longer than
max(1, len(input)).
Portable code already needs to be written with this in mind, so in my
opinion it makes very little sense to allow the existing behaviour.
Prevent the base system from falling back to this by switching over to
POSIX prototypes.
I'm not going to bump the __FreeBSD_version for this. The reason is that
it's possible to account for this change in a portable way, without
depending on a specific version of FreeBSD. An exp-run was done some
time ago. As far as I know, all regressions as a result of this have
already been fixed.
I'll give this change some time to settle. In the long run I want to
replace our copies by ones that are thread-safe and don't depend on
PATH_MAX/MAXPATHLEN.
case it fully constructed, but for half-constructed too, so have no
other choice to pass original pattern from glob() down to globextend()
instead of attempt to reconstruct I implement previously.
2) Instead of copy&paste the same big enough code, make function for it:
globfinal().
globexp1() recursive calls, but glob0() was not supposed to be called
repeatedly in the original code. It finalize results by possible adding
original pattern for no match case, may return GLOB_NOMATCH error and
by sorting all things. Original pattern adding or GLOB_NOMATCH error
can happens each time glob0() called repeatedly, and sorting happens
for one item only, all things are never sorted. Second, f.e. "a{a"
pattern does not match "a{a" file but match "a" file instead
(just one example, there are many). Third, some errors (f.e. for limits
or overflow) can be ignored by GLOB_BRACE code because it forces return (0).
Add non-finalizing flag to glob0() and make globexp0() wrapper around
recursively called globexp1() to finalize things like glob0() does.
Reorganize braces code to work correctly.
2) Don't allow MB_CUR_MAX * strlen overallocation hits GLOB_LIMIT_STRING
(ARG_MAX) limit, use final string length, not malloced space for it.
3) Revive DEBUG-ifdefed section.
unmodified, if no matches found. But our original code strips all '\'
returning it. Rewrite the code to allow to reconstruct exact the
original pattern with backslashes for this case.
2) Prevent to use truncated pattern if MAXPATHLEN exceeded, return
GLOB_NOMATCH instead.
3) Fix few end loop conditions filling Char arrays with mbrtowc(),
MB_CUR_MAX is unneeded in two places and condition is less by one
in other place.
4) Prevent to use truncated filenames match if MAXPATHLEN exceeded,
skip such directory entries.
5) Don't end *pathend with L'/' in glob3() if limit is reached, this
change will be not visible since error is returned.
6) If error happens in (*readdirfunc)(), do the same GLOB_ABORTED
processing as for g_opendir() as POSIX requires.
in any case and needed for further processing. For ~ expansion too.
2) Don't terminate *pathend with / when GLOB_LIMIT_STAT is reached, it will
be not visible outside in any case since error is returned.
3) Cosmetic: change if expression to better reflect its semantic.
since whole conversion needs a room for (len >= MB_CUR_MAX). It is no
difference when MB_CUR_MAX == 1, but for multi-byte locales last few chars
('\0' and before) may need just one byte, and the rest of MB_CUR_MAX - 1
space becomes unavailable in the MAXPATHLEN-sized buffer, which cause
conversion error on near MAXPATHLEN long pathes.
Increase g_Ctoc() conversion buffers to MB_LEN_MAX - 1.
as pattern meta chars.
2) GLOB_ERR and gl_errfunc are supposed to work only for real directories
per POSIX, so don't act on missing or plain files, for ENOENT or ENOTDIR
(as TODO in the code suggested).
3) Remove the hack in the manpage describing how to skip ENOENT and ENOTDIR
in gl_errfunc, it is unneeded now.
4) Set errno to ENAMETOOLONG if g_Ctoc() expansion fails in g_opendir(),
as in other places in the code which are wrappers around system functions.
open/read errors and with GLOB_ERR and gl_errfunc processing), so we can't
blindly return it on any MAXPATHLEN overflow. Even our manpage disagrees
with such GLOB_ABORTED usage. Use GLOB_NOSPACE for that now with errno is
set to 0 as for limits.
2) Return GLOB_NOSPACE when valid ~ expansion can't happens due to
MAXPATHLEN overflow too.
3) POSIX (and our manpage) says, if GLOB_ERR is set, GLOB_ABORTED should
be returned immediatelly, without using gl_errfunc. Implement it now.
them down. This hack still remains:
* 2. Illegal byte sequences in filenames are handled by treating them as
* single-byte characters with a values of such bytes of the sequence
* cast to wchar_t.
2) Reword the comment in the hack above to reflect implementation.
3) Protect signed wchar_t from sign extension when a signed char is assigned
to it in the hack above.
3) Corresponding backward hack in g_Ctoc() was not implemented, so all
pathes with illegal byte sequences are skipped as result, implement it now.
4) globtilde() forget to convert expanded user home dir from multibyte to
wchar.
5) Protect globtilde() from long expansion truncation.
6) Results was not sorted according to collate as POSIX requires.
locale (which cause core dump) by removing whole 'table' argument
by which it passed.
2) Restore __collate_range_cmp() in __sccl().
3) Collating [a-z] range in regcomp() only for single bytes locales
(we can't do it now for other ones). In previous state only first 256
wchars are considered and all others are just silently dropped from the
range.
Instead of changing whole course to another POSIX-permitted way
for consistency and uniformity I decide to completely ignore missing
regex fucntionality and concentrace on fixing bugs in what we have now,
too many small obstacles instead, counting ports.
Only first 256 wide chars are considered currently, all other are just
dropped from the range. Proper implementation require reverse tables
database lookup, since objects are really big as max UTF-8 (1114112
code points), so just the same scanning as it was for 256 chars will
slow things down.
POSIX does not require collation for [a-z] type ranges and does not
prohibit it for non-POSIX locales. POSIX require collation for ranges
only for POSIX (or C) locale which is equal to ASCII and binary for
other chars, so we already have it.
No other *BSD implements collation for [a-z] type ranges.
Restore ABI compatibility with unused now __collate_range_cmp() which
is visible from outside (will be removed later).
Setting time by seconds or microseconds may cause unexpected effects
especially if sysctl vfs.timestamp_precision=3 (not default).
Calling the obsolete functions with NULL timestamps is acceptable.
This support appears to have been documented in nsswitch.conf(5) for some
time. The implementation adds two NSS netgroup providers to libc. The
default, compat, provides the behaviour documented in netgroup(5), so this
change does not make any user-visible behaviour changes. A files provider
is also implemented.
innetgr(3) is implemented as an optional NSS method so that providers such
as NIS which are able to implement efficient reverse lookup can do so.
A fallback implementation is used otherwise. getnetgrent_r(3) is added for
convenience and to provide compatibility with glibc and Solaris.
With a small patch to net/nss_ldap, it's possible to specify an ldap
netgroup provider, allowing one to query nisNetgroupTriple entries.
Sponsored by: EMC / Isilon Storage Division
Handle an empty result from yp_match() by returning NULL, which is
consistent with the handling of an empty netgroup in /etc/netgroup.
setnetgrent(3) has no return value, so there is no particular need to
distinguish this case from an error.
PR: 26486
MFC after: 2 weeks
This adds stravis() and some new encoding flags VIS_SHELL, VIS_META,
and VIS_NOLOCALE.
Assorted cleanups and fixes includeing a manpage typo[0].
PR: 210013 [0]
Submitted by: pi [0]
If malloc() fails to allocate linep, then free olinep (if it exists)
before returning to avoid a memory leak.
Reported by: Coverity
CID: 1016716
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D6755
When collation support was brought in, the second and third
arguments in __collate_range_cmp() were changed from int to
wchar_t, breaking the ABI. Change them to a "char" type which
makes more sense and keeps the ABI compatible.
Also introduce __wcollate_range_cmp() which does work with wide
characters. This function is used only internally in libc so
we don't export it. Use the new function in glob(3), fnmatch(3),
and regexec(3).
PR: 179721
Suggested by: ache. jilles
MFC after: 3 weeks (perhaps partial only)
POSIX 2008 added the psignal() function which has already been part of
the BSDs for a long time. The only difference is, the POSIX version uses
an 'int' for the signal number, unlike our version which uses an
'unsigned int'. Fix up the function to use an 'int'. This should not
affect the ABI.
intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013.
A robust mutex is guaranteed to be cleared by the system upon either
thread or process owner termination while the mutex is held. The next
mutex locker is then notified about inconsistent mutex state and can
execute (or abandon) corrective actions.
The patch mostly consists of small changes here and there, adding
neccessary checks for the inconsistent and abandoned conditions into
existing paths. Additionally, the thread exit handler was extended to
iterate over the userspace-maintained list of owned robust mutexes,
unlocking and marking as terminated each of them.
The list of owned robust mutexes cannot be maintained atomically
synchronous with the mutex lock state (it is possible in kernel, but
is too expensive). Instead, for the duration of lock or unlock
operation, the current mutex is remembered in a special slot that is
also checked by the kernel at thread termination.
Kernel must be aware about the per-thread location of the heads of
robust mutex lists and the current active mutex slot. When a thread
touches a robust mutex for the first time, a new umtx op syscall is
issued which informs about location of lists heads.
The umtx sleep queues for PP and PI mutexes are split between
non-robust and robust.
Somewhat unrelated changes in the patch:
1. Style.
2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared
pi mutexes.
3. Removal of the userspace struct pthread_mutex m_owner field.
4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls
the lifetime of the shared mutex associated with a vnode' page.
Reviewed by: jilles (previous version, supposedly the objection was fixed)
Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects)
Tested by: pho
Sponsored by: The FreeBSD Foundation
According to style(9):
> normally, include <sys/types.h> OR <sys/param.h>, but not both.
(<sys/param.h> already includes <sys/types.h> when LOCORE is not defined).
We have an nitems() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
Given that it is available already without adding additional
headers and other parts of libc already use it, extend a bit
more its use.
In r103767 the kern.ps_strings sysctl was added as the preferred way to
locate the ps_strings struct and is available in any FreeBSD release
supported within the last decade.
Reviewed by: kib
