Commit Graph

1987 Commits

Author SHA1 Message Date
Enji Cooper
5e3f43f2e0 getbsize(3): clarify that underflow/overflow warnings in regard to $BLOCKSIZE
gets output via warnx(3)

This helps set expectations for how one might deal with those messages, i.e.,
mute output from /dev/stderr today, since that's where vwarn(3) outputs messages
to today.

MFC after:	1 month
2017-06-12 00:43:14 +00:00
Pedro F. Giffuni
0523111725 fts_open: move bogus initialization further below, before it is used.
Move an unneeded initialization, introduced in r54770 to quiet down GCC,
to a place nearer to its first use. This has no practical effect, it just
keeps the garbage better sorted.

Hinted by:	OpenBSD (CVS rev. 1.56, without obfuscations)
2017-05-26 01:14:58 +00:00
Glen Barber
4adb408018 Update the "first appeared in" version in several manual pages.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-05-24 17:50:34 +00:00
Enji Cooper
7828a9ccb7 directory(3): add trailing comma after e.g., missed in r318699
Submitted by:	bjk
MFC after:	2 weeks
MFC with:	r318699
Sponsored by:	Dell EMC Isilon
2017-05-23 17:34:10 +00:00
Konstantin Belousov
6992112349 Commit the 64-bit inode project.
Extend the ino_t, dev_t, nlink_t types to 64-bit ints.  Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment.  Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.

ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks.  Unfortunately, not everything can be
fixed, especially outside the base system.  For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.

Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.

Struct xvnode changed layout, no compat shims are provided.

For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.

Update note: strictly follow the instructions in UPDATING.  Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.

Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb).  Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver.  Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem).  Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).

Sponsored by:	The FreeBSD Foundation (emaste, kib)
Differential revision:	https://reviews.freebsd.org/D10439
2017-05-23 09:29:05 +00:00
Enji Cooper
671ea87fba tcsendbreak(3): delete spurious blank line at the end of the man page
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-05-23 06:58:08 +00:00
Enji Cooper
3d54bd2294 dirname(3): fix section ordering
IMPLEMENTATION NOTES come before RETURN VALUES

MFC after:	2 weeks
Reported by:	make manlint
Sponsored by:	Dell EMC Isilon
2017-05-23 06:56:57 +00:00
Enji Cooper
a966418a66 directory(3): delete trailing whitespace and rewrite E.g. as e.g.
Reported by:	make manlint
Sponsored by:	Dell EMC Isilon
2017-05-23 06:55:51 +00:00
Enji Cooper
41a9662896 basename(3): fix section ordering
IMPLEMENTATION NOTES come before RETURN VALUES

MFC after:	2 weeks
Reported by:	make manlint
Sponsored by:	Dell EMC Isilon
2017-05-23 06:53:31 +00:00
Enji Cooper
1384163706 err(3): use NULL, aka (void*)0 per POSIX instead of (FILE *)0
This is being done to aid humans and static analysis checkers.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-05-23 06:14:02 +00:00
Eric van Gyzen
07f29d9f76 Remove old spinlock_debug code from libc
This no longer seems useful.  Remove it.

This was prompted by a "cast discards volatile qualifier" warning
in libthr when WARNS=6.

Reviewed by:	kib
MFC after:	3 days
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10832
2017-05-20 17:32:01 +00:00
Ed Maste
b47e69e69e getusershell: don't write past end of line buffer reading local shells
_local_initshells did not reset cp to the beginning of the line buffer
for every iteration that it called fgets(3), leading to writing past the
end of line with fairly long /etc/shells or excessively long line
lengths. Correct this by properly resetting cp.

PR:		192528
Submitted by:	Kyle Evans <kevans91@ksu.edu>
Reviewed by:	cem, jilles
Differential Revision:	https://reviews.freebsd.org/D10690
2017-05-15 17:57:09 +00:00
Jilles Tjoelker
8f932310ff glob: Fix comment about collapsing asterisks after r317749.
After r317749, collapsing adjacent asterisks is still required, but for a
different reason.
2017-05-07 19:52:56 +00:00
Conrad Meyer
1365421f09 glob: Fix an overlong line
Trivial style(9) fix, no functional change.  There are also some 81
characters lines below, but I don't see a good way to shorten them.

Sponsored by:	Dell EMC Isilon
2017-05-03 16:34:22 +00:00
Conrad Meyer
241eb37e29 libc glob: Avoid pathological exponential behavior
Adapt glob's match() routine to use a greedy algorithm that avoids
exponential runtime in byzantine inputs.

While here, add a testcase for the byzantine input.

Prompted by:	https://research.swtch.com/glob
Authored by:	Yves Orton <demerphq at gmail.com>
Obtained from:	Perl (33252c318625f3c6c89b816ee88481940e3e6f95)
Sponsored by:	Dell EMC Isilon
2017-05-03 15:55:29 +00:00
Konstantin Belousov
f7a6f6a2c0 Make semaphore names list mutex non-recursive.
The mutex is used in sem_open() and sem_close(), which cannot
recurse. The atfork handlers cannot collide with the open and close
code.

Reviewed by:	vangyzen
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D10545
2017-04-30 19:37:45 +00:00
Konstantin Belousov
e6a183a40e Restructure normal (non-error) control flow in sem_close().
Do not retest for the found semaphore after the loop to look it up.
Instead, handle both cases of last and non-last close simultaneously,
which allows to consolidate the list unlock and successful return.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-04-30 19:32:51 +00:00
Konstantin Belousov
3a1371626e Style.
- Use ANSI C function definitions.
- Remove redundand cast.
- Minor style compliance tweaks.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-04-30 10:47:59 +00:00
Konstantin Belousov
76c239924e getpagesize(3) cannot fail.
Sponsored by:	The FreeBSD Foundation
2017-04-26 14:28:27 +00:00
Konstantin Belousov
db4a195744 getpagesize(3) cannot fail.
The sysctl(HW_PAGESIZE) call cannot fail on FreeBSD kernels at least.
And even if it failed for some improbable reason, PAGE_SIZE is a safe
value to return.

Discussed with:	jilles
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-04-26 14:25:01 +00:00
Pedro F. Giffuni
5f2bd3bdac scandir(3): promote arraysz to size_t to match numitems.
The internal array size goes through a loop and is compared with numitems
which at its limits makes can be unreachably higher than arraysz.
Prevent an hypothetical overflow by matching the types.

MFC after:	1 week
2017-04-24 14:56:41 +00:00
Enji Cooper
3c3e9e1035 Note that getpagesize(3) can return -1 on failure
MFC after:	5 weeks
Sponsored by:	Dell EMC Isilon
2017-04-23 03:17:32 +00:00
Alexander Motin
f924326575 Optimize pathologic case of telldir() for Samba.
When application reads large directory, calling telldir() for each entry,
like Samba does, it creates exponential performance drop as number of
entries reach tenths to hundreds of thousands.  It is caused by full search
through the internal list, that never finds matches in that scenario, but
creates O(n^2) delays.  This patch optimizes that search, limiting it to
entries of the same buffer, turning time closer to O(n) in case of linear
directory scan.

PR:		218622
Reviewed by:	jhb, jilles
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D10408
2017-04-17 19:03:31 +00:00
Brooks Davis
8439a7220d Fix an out-of-bounds write when a zero-length buffer is passed.
Found with ttyname_test and CHERI bounds checking.

Reviewed by:	emaste
Obtained from:	CheriBSD
MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D10377
2017-04-13 15:52:45 +00:00
Konstantin Belousov
f6ef11cc75 Report _SC_SEM_NSEMS_MAX and _SC_SEM_VALUE_MAX which show parameters
of the current usermode implementation of the POSIX semaphores.

For NSEMS_MAX, return -1 without changing errno, which indicates that
the variable has no limit.  Before, sysconf(3) returned parameters
queried from the ksem(9) legacy implementation, which apparently has
low defaults for NSEMS_MAX.

Reported and tested by:	jbeich
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-04-12 19:27:14 +00:00
Warner Losh
62fd382c43 Simplify the code a little.
Submitted by: rppokala
Sponsored by: Netflix
2017-03-22 20:51:52 +00:00
Enji Cooper
c25e946c34 Unbreak world by adding sys/stat.h for stat(2)
X-MFC with:	r315733
Reported by:	O. Hartmann <ohartmann@walstatt.org>
Sponsored by:	Dell EMC Isilon
2017-03-22 19:23:13 +00:00
Enji Cooper
eea7b35fb9 style(9): sort #includes
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-22 19:17:13 +00:00
Warner Losh
37b5835028 Impelemnt ttys onifexists in init.
Implement a new init(8) option in /etc/ttys. If this option is present
on the entry in /etc/ttys, the entry will be active if and only if it
exists.  If the name starts with a '/', it will be considered an
absolute path. If not, it will be a path relative to /dev.

This allows one to turn off video console getty that aren't present
(while running a getty on them even when they aren't the system
console). Likewise with serial ports.

It differs from onifconsole in only requiring the device exist rather
than it be listed as one of the system consoles.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D10037
2017-03-22 19:00:41 +00:00
Conrad Meyer
0a8ff54eb1 scandir: Fix NULL dereference, uninitialized value use in error case
If opendir succeeds but malloc fails, numitems was used uninitialized in
error handling under the 'fail' label.  If it happened to have a non-zero
value, the NULL 'names' was dereferenced.

Reported by:	Coverity
CIDs:		1329566, 1372625
Sponsored by:	Dell EMC Isilon
2017-03-22 17:37:47 +00:00
Xin LI
91868665a9 Discard first 3072 bytes of RC4 keystream, this is a bandaid
that allows us to work on switching to a more modern PRNG.

Submitted by:	Steven Chamberlain <steven pyro eu org>
Approved by:	so
2017-03-14 06:00:44 +00:00
Pedro F. Giffuni
9f36610f9e libc: provide some bounds-checking through reallocarray(3).
reallocarray(3) is a non portable extension that originated in OpenBSD.
Given that it is already in FreeBSD's libc it is useful for the cases
where reallocation involves a multiplication.

MFC after:		2 weeks
Differential Revision:	https://reviews.freebsd.org/D9955
2017-03-12 16:03:34 +00:00
Pedro F. Giffuni
ada469063f libc: small cleanups.
Unsign setlen: it is local and will never be negative. Having one more bit
for growth is beneficial and it avoids a cast when it's going to be used
for allocation.

Reviewed by:	ngie
MFC after:	3 days
2017-03-12 03:26:25 +00:00
Pedro F. Giffuni
56362c6f81 libc: small cleanups.
Rename nitems to numitems: it shares the anme with an existing macro in
sys/params.h. Also initialize the value later which avoids asigning the
value if we exit early.

Reviewed by:	ngie
MFC after:	3 days
2017-03-12 03:22:18 +00:00
Conrad Meyer
8844cec8f3 fts: Fix a potential memory leak in error case
Dan Krejsa reports a potential memory leak in an fts_build error case,
detected by Coverity.  (It doesn't seem to show up in Coverity Scan, so I
don't have a CID to point to.)

I don't know whether it is actually possible to arrive in this case with a
non-empty 'head' list.  The cost is low, though.  One additional branch in a
terminal error case isn't the end of the world.

PR:		217125
Submitted by:	Dan Krejsa <dan.krejsa at gmail.com>
2017-03-04 20:46:57 +00:00
Warner Losh
fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Eric van Gyzen
81027fa594 Sort declaration of sem_clockwait_np
Also mention <time.h> in sem_timedwait(3), because POSIX does,
and because the user will need it for clockid_t, struct timespec,
and TIMER_ABSTIME.

Reported by:	bde
MFC after:	9 days
X-MFC with:	r314179
Sponsored by:	Dell EMC
2017-02-28 21:47:00 +00:00
Eric van Gyzen
b215ceaaec Add sem_clockwait_np()
This function allows the caller to specify the reference clock
and choose between absolute and relative mode.  In relative mode,
the remaining time can be returned.

The API is similar to clock_nanosleep(3).  Thanks to Ed Schouten
for that suggestion.

While I'm here, reduce the sleep time in the semaphore "child"
test to greatly reduce its runtime.  Also add a reasonable timeout.

Reviewed by:	ed (userland)
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D9656
2017-02-23 19:36:38 +00:00
Eric van Gyzen
f02396d5d9 Fix grammar in getpeereid(3) 2017-02-04 00:37:25 +00:00
Enji Cooper
d0fd0203fb Replace dot-dot relative pathing with SRCTOP-relative paths where possible
This reduces build output, need for recalculating paths, and makes it clearer
which paths are relative to what areas in the source tree. The change in
performance over a locally mounted UFS filesystem was negligible in my testing,
but this may more positively impact other filesystems like NFS.

LIBC_SRCTOP was left alone so Juniper (and other users) can continue to
manipulate lib/libc/Makefile (and other Makefile.inc's under lib/libc) as
include Makefiles with custom options.

Discussed with:	marcel, sjg
MFC after:	1 week
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D9207
2017-01-20 03:23:24 +00:00
Ian Lepore
f64342e354 Rework tty_drain() to poll the hardware for completion, and restore
drain timeout handling to historical freebsd behavior.

The primary reason for these changes is the need to have tty_drain() call
ttydevsw_busy() at some reasonable sub-second rate, to poll hardware that
doesn't signal an interrupt when the transmit shift register becomes empty
(which includes virtually all USB serial hardware).  Such hardware hangs
in a ttyout wait, because it never gets an opportunity to trigger a wakeup
from the sleep in tty_drain() by calling ttydisc_getc() again, after
handing the last of the buffered data to the hardware.

While researching the history of changes to tty_drain() I stumbled across
some email describing the historical BSD behavior of tcdrain() and close()
on serial ports, and the ability of comcontrol(1) to control timeout
behavior.  Using that and some advice from Bruce Evans as a guide, I've
put together these changes to implement the hardware polling and restore
the historical timeout behaviors...

 - tty_drain() now calls ttydevsw_busy() in a loop at 10 Hz to accomodate
   hardware that requires polling for busy state.

 - The "new historical" behavior for draining during close(2) is retained:
   the drain timeout is "1 second without making any progress".  When the
   1-second timeout expires, if the count of bytes remaining in the tty
   layer buffer is smaller than last time, the timeout is extended for
   another second.  Unfortunately, the same logic cannot be extended all
   the way down to the hardware, because the interface to that layer is a
   simple busy/not-busy indication.

 - Due to the previous point, an application that needs a guarantee that
   all data has been transmitted must use TIOCDRAIN/tcdrain(3) before
   calling close(2).

 - The historical behavior of honoring the drainwait setting for TIOCDRAIN
   (used by tcdrain(3)) is restored.

 - The historical kern.drainwait sysctl to control the global default
   drainwait time is restored, but is now named kern.tty_drainwait.

 - The historical default drainwait timeout of 300 seconds is restored.

 - Handling of TIOCGDRAINWAIT and TIOCSDRAINWAIT ioctls is restored
   (this also makes the comcontrol(1) drainwait verb work again).

 - Manpages are updated to document these behaviors.

Reviewed by:	bde (prior version)
2017-01-12 00:48:06 +00:00
John Baldwin
34ed0c63c8 Rename the 'flags' argument to getfsstat() to 'mode' and validate it.
This argument is not a bitmask of flags, but only accepts a single value.
Fail with EINVAL if an invalid value is passed to 'flag'.  Rename the
'flags' argument to getmntinfo(3) to 'mode' as well to match.

This is a followup to r308088.

Reviewed by:	kib
MFC after:	1 month
2016-12-27 20:21:11 +00:00
Ed Schouten
8ab00b8fbc Properly sign extend the result of jrand48() and mrand48().
These functions are supposed to return a value between [_2^31, 2^31).
This doesn't seem to work on 64-bit systems, where we return a value
between [0, 3^32). Patch up the function to use proper casts to int32_t.
While there, fix some other style bugs.

MFC after:	2 weeks
2016-12-06 19:08:29 +00:00
Ed Maste
134ede2dd2 remove unnecessary vm includes from setproctitle
vm headers were needed only for the PS_STRINGS fallback, which was
removed in r297888.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-11-22 16:00:18 +00:00
Ed Schouten
34168b28e9 Replace basename(3) by a thread-safe implementation.
Now that the changes to the dirname(3) function had some time to settle,
let's go ahead and use the same approach for replacing basename(3) by a
simple implementation that modifies the input string, thereby making it
thread-safe and guaranteed to succeed.

Unlike dirname(3), this function already had a thread-safe variant
basename_r(3). This function had its own set of problems, like having an
upper bound on the pathname length. Keep this function around for
compatibility, but remove most references from the man page. Make the
man page more similar to that of dirname(3).

As the basename_r(3) function is only provided by FreeBSD (and Bionic),
depending on its use is even more implementation defined than assuming
that basename(3) is thread-safe.

Reviewed by:	emaste
Differential Revision:	https://reviews.freebsd.org/D8382
2016-11-03 20:21:34 +00:00
John Baldwin
ab1b41edb5 Fix formatting of tables.
Specifically, use .Ta instead of tabs to separate column entries.  While
here fix a few other things:
- Use .Sy for all column headers (previously only the first column header
  was bold)
- Use .Dv to markup constants used for MIB names.
- Use "1234" and "4321" for the byte order descriptions without
  thousands separators.
- Mark up header files in the first table with .In.

MFC after:	2 weeks
2016-10-28 18:09:08 +00:00
Ed Maste
49a6e1ba32 Add comment on use of abort() in libc
Suggested by:	jonathan (in review D8133)
2016-10-12 13:56:14 +00:00
Konstantin Belousov
6886c70100 Correct indent.
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2016-10-11 11:46:02 +00:00
Konstantin Belousov
c3b83049d6 Fill msg_len for the initial element of msgvec.
Submitted by:	Daniel Salzman <daniel.salzman@nic.cz>
MFC after:	3 days
2016-10-11 11:45:12 +00:00
Ed Schouten
1a466ddc79 Remove setkey(), encrypt(), des_setkey() and des_cipher().
The setkey() and encrypt() functions are part of XSI, not the POSIX base
definitions. There is no strict requirement for us to provide these,
especially if we're only going to keep these around as undocumented
stubs. The same holds for des_setkey() and des_cipher().

Instead of providing functions that only generate warnings when linking,
simply disallow linking against them. The impact of this is relatively
low. It only causes two leaf ports to break. I'll see what I can do to
help out to get those fixed.

PR:		211626
2016-10-03 18:20:58 +00:00
Ed Maste
5c1ea1fcd0 libc arc4_stir: use only kern.arandom sysctl
The sysctl cannot fail. If it does fail on some FreeBSD derivative or
after some future change, just abort() so that the problem will be found
and fixed.

It's preferable to provide an arc4random() function that cannot fail and
cannot return poor quality random data. While abort() is not normally
suitable for a library, it makes sense here.

Reviewed by:	ed, jonathan, markm
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8077
2016-10-03 13:12:44 +00:00
Ed Schouten
fd85bff53e Replace dirname(3) by a copy that complies to POSIX.
It turns out that the path normalization that our brand new copy of
dirname(3) does is actually not allowed by the draft version of the
upcoming version of POSIX. It has to behave identically to the
dirname(1) utility.

This change replaces our new dirname(3) implementation by yet another
version that doesn't implement the path normalization logic; it merely
looks for the end of the directory name and overwrites that with a null
byte.

More details: See note #3370 at http://austingroupbugs.net/view.php?id=1073

PR:		212193
Reviewed by:	emaste, jilles
Differential Revision:	https://reviews.freebsd.org/D7790
2016-09-18 20:47:55 +00:00
Brooks Davis
466522c3ac Initialize lists of signals using C99 designators
Reviewed by:	jilles
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D7601
2016-09-02 00:16:19 +00:00
Jilles Tjoelker
89688ae708 directory(3): Deprecate readdir_r(). Clarify dirent buffers.
In existing implementations including FreeBSD, there is no reason to use
readdir_r() in the common case where potentially multiple threads each list
their own directory. Code using readdir() is simpler.

What's more, lthough readdir_r() can safely be used on FreeBSD because
NAME_MAX is forced to 255, it cannot be used safely on systems where
{NAME_MAX} is not fixed. As a concrete example, FAT/NTFS filenames can be up
to 255 UTF-16 code units long, which can be up to 765 UTF-8 bytes.

Deprecating readdir_r() in POSIX has been proposed in
http://www.austingroupbugs.net/view.php?id=696
and glibc wants to deprecate it as well.

Reviewed by:	ed, wblock
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D7678
2016-08-31 20:38:40 +00:00
Jilles Tjoelker
1221d42990 getvfsbyname(3): Remove [EFAULT] error.
Since r101651 in 2002, getvfsbyname() has written *vfc using userland
memcpy(), so the [EFAULT] error no longer occurs.

MFC after:	1 week
2016-08-30 21:43:57 +00:00
Ed Schouten
cd4dcac89a Improve compatibility of calls to dirname() on constant strings.
As the xinstall(8) utility had to be patched up to work with the POSIXly
correct basename()/dirname() prototypes, we make it pretty hard to build
previous versions of FreeBSD on HEAD. xinstall(8) is part of the
bootstrap tools.

Add some logic to <libgen.h> to automatically detect bad calls to
dirname() based on the type of the argument. If the argument is of type
'const char *', we simply fall back to calling into dirname@FBSD_1.0
directly.

I'll also give basename() similar treatment when importing the
thread-safe version of that function.

Tested by:	bdrewery, madpilot (thanks!)
2016-08-26 20:23:10 +00:00
Brooks Davis
2438d9a0e1 Avoid a redecleartion of __getosreldate().
Sponsored by:	DARPA, AFRL
2016-08-24 00:02:20 +00:00
Kevin Lo
0de6c9d651 - Add the 'restrict' type qualifier to match function prototype.
- Use .Lb libc rather than libpthread.

Reviewed by:	delphij
2016-08-17 07:25:50 +00:00
Ed Schouten
e2f6816100 Reimplement dirname(3) to be thread-safe.
Now that we've updated the prototypes of the basename(3) and dirname(3)
functions to conform to POSIX, let's go ahead and reimplement dirname(3)
in such a way that it's thread-safe, but also guaranteed to succeed. C
libraries like glibc, musl and the one that's part of Solaris already
follow such an approach.

Move the existing implementation to another source file,
freebsd11_dirname.c to keep existing users of the API that pass in a
constant string happy, using symbol versioning.

Put a new version of the function in dirname.c, obtained from CloudABI's
C library. This version scans through the pathname string from left to
right, normalizing it, while discarding the last pathname component.

Reviewed by:	emaste, jilles
Differential Revision:	https://reviews.freebsd.org/D7355
2016-08-12 07:03:58 +00:00
Sergey Kandaurov
18ac59f431 Grammar fixes. 2016-08-04 11:38:53 +00:00
Sergey Kandaurov
eb6a434243 mdoc: The .Fn function. 2016-08-04 11:22:51 +00:00
Andrey A. Chernov
20e37fa8ca Although the code amount is not big, move POSIX error processing into
two sepatate functions to make glob(3) code less obscure and more simple.
There is no needs to make them inline since it is error path which supposed
to not happes often.
2016-08-03 09:09:34 +00:00
Bryan Drewery
a8d890b4fb Add link for getnetgrent_r(3).
MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-02 23:46:32 +00:00
Ed Schouten
9c24291370 Fix up setgrent(3) to have a POSIX-compliant prototype.
Just like with freelocale(3), I haven't been able to find any piece of
code that actually makes use of this function's return value, both in
base and in ports. The reason for this is that FreeBSD seems to be the
only operating system to have such a prototype. This is why I'm deciding
to not use symbol versioning for this.

It does seem that the pw(8) utility depends on the function's typing and
already had a switch in place to toggle between the FreeBSD and POSIX
variant of this function. Clean this up by always expecting the POSIX
variant.

There is also a single port that has a couple of local declarations of
setgrent(3) that need to be patched up. This is in the process of being
fixed.

PR:		211394 (exp-run)
2016-07-31 08:05:15 +00:00
Andrey A. Chernov
000b8f832a In addition to prev. commit. Since potentially glob2() can return error
without setting errno, restore errno before its call.
2016-07-31 02:28:50 +00:00
Andrey A. Chernov
869eb80c16 Both C99 and POSIX directly prohibits any standard function to set errno
to 0. Breaking this rule in 2001 NetBSD hack was imported which attempts
to workaround very limited glob() return codes amount. Use POSIX-compatible
workaround now with E2BIG which can't comes from other functions used
instead of prohibited 0.
2016-07-31 01:14:06 +00:00
Andrey A. Chernov
15cb786674 Rework r303074 case 4. Don't immediatelly skip directory entries which
cause MAXPATHLEN exceeded. Process them first through gl_errfunc() and
GLOB_ERR.
2016-07-30 03:11:54 +00:00
Andrey A. Chernov
e04d8562b6 Reset errno for readdirfunc() before contunue. 2016-07-30 02:09:11 +00:00
Ed Schouten
329ecc7f94 Mention that basename(3) and dirname(3) will change in the future.
Update the existing manual pages for basename(3) and dirname(3) to
mention that in future versions of FreeBSD, these functions will no
longer use internal buffers for storing the results.

MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D7356
2016-07-29 16:25:09 +00:00
Ed Schouten
938809f941 Fix up prototypes of basename(3) and dirname(3) to comply to POSIX.
POSIX allows these functions to be implemented in a way that the
resulting string is stored in the input buffer. Though some may find
this annoying, this has the advantage that it makes it possible to
implement this function in a thread-safe way. It also means that they
can be implemented in a way that they work for paths of arbitrary
length, as the output string of these functions is never longer than
max(1, len(input)).

Portable code already needs to be written with this in mind, so in my
opinion it makes very little sense to allow the existing behaviour.
Prevent the base system from falling back to this by switching over to
POSIX prototypes.

I'm not going to bump the __FreeBSD_version for this. The reason is that
it's possible to account for this change in a portable way, without
depending on a specific version of FreeBSD. An exp-run was done some
time ago. As far as I know, all regressions as a result of this have
already been fixed.

I'll give this change some time to settle. In the long run I want to
replace our copies by ones that are thread-safe and don't depend on
PATH_MAX/MAXPATHLEN.
2016-07-28 16:20:27 +00:00
Andrey A. Chernov
a5ab035dd7 1) POSIX defines well when GLOB_NOMATCH or original pattern
(instead) should be returned, so we can't return GLOB_NOMATCH blindly
just because we dislike something in the pattern.

2) Remove extra condition.
2016-07-23 03:49:02 +00:00
Andrey A. Chernov
09264d7448 1) We need the original pattern (in the next round of changes) not only in
case it fully constructed, but for half-constructed too, so have no
other choice to pass original pattern from glob() down to globextend()
instead of attempt to reconstruct I implement previously.

2) Instead of copy&paste the same big enough code, make function for it:
globfinal().
2016-07-23 01:21:58 +00:00
Andrey A. Chernov
bd7a98506c 1) GLOB_BRACE was somewhat broken. First it repeatedly calls glob0() in
globexp1() recursive calls, but glob0() was not supposed to be called
repeatedly in the original code. It finalize results by possible adding
original pattern for no match case, may return GLOB_NOMATCH error and
by sorting all things. Original pattern adding or GLOB_NOMATCH error
can happens each time glob0() called repeatedly, and sorting happens
for one item only, all things are never sorted. Second, f.e. "a{a"
pattern does not match "a{a" file but match "a" file instead
(just one example, there are many). Third, some errors (f.e. for limits
or overflow) can be ignored by GLOB_BRACE code because it forces return (0).
Add non-finalizing flag to glob0() and make globexp0() wrapper around
recursively called globexp1() to finalize things like glob0() does.
Reorganize braces code to work correctly.

2) Don't allow MB_CUR_MAX * strlen overallocation hits GLOB_LIMIT_STRING
(ARG_MAX) limit, use final string length, not malloced space for it.

3) Revive DEBUG-ifdefed section.
2016-07-21 12:53:36 +00:00
Andrey A. Chernov
7455a07a9f In addition to r303074 case 1, search for protected L'/' too in globtilde() 2016-07-20 12:46:21 +00:00
Andrey A. Chernov
aed721ec51 1) Per POSIX (and glibc) GLOB_NOCHECK should return original pattern,
unmodified, if no matches found. But our original code strips all '\'
returning it. Rewrite the code to allow to reconstruct exact the
original pattern with backslashes for this case.

2) Prevent to use truncated pattern if MAXPATHLEN exceeded, return
GLOB_NOMATCH instead.

3) Fix few end loop conditions filling Char arrays with mbrtowc(),
MB_CUR_MAX is unneeded in two places and condition is less by one
in other place.

4) Prevent to use truncated filenames match if MAXPATHLEN exceeded,
skip such directory entries.

5) Don't end *pathend with L'/' in glob3() if limit is reached, this
change will be not visible since error is returned.

6) If error happens in (*readdirfunc)(), do the same GLOB_ABORTED
processing as for g_opendir() as POSIX requires.
2016-07-20 07:30:44 +00:00
Andrey A. Chernov
eef722c337 1) Don't protect \/ and \. even if user say so. They are not special chars
in any case and needed for further processing. For ~ expansion too.

2) Don't terminate *pathend with / when GLOB_LIMIT_STAT is reached, it will
be not visible outside in any case since error is returned.

3) Cosmetic: change if expression to better reflect its semantic.
2016-07-19 00:25:27 +00:00
Andrey A. Chernov
e9c0137235 g_Ctoc() conversion buffers are smaller than needed up to MB_CUR_MAX - 1
since whole conversion needs a room for (len >= MB_CUR_MAX). It is no
difference when MB_CUR_MAX == 1, but for multi-byte locales last few chars
('\0' and before) may need just one byte, and the rest of MB_CUR_MAX - 1
space becomes unavailable in the MAXPATHLEN-sized buffer, which cause
conversion error on near MAXPATHLEN long pathes.

Increase g_Ctoc() conversion buffers to MB_LEN_MAX - 1.
2016-07-18 20:24:13 +00:00
Andrey A. Chernov
d67355c507 Change patch from r303004 case 3. According to POSIX gl_errfunc should be
called first, then GLOB_ERR should be considered.
2016-07-18 19:20:49 +00:00
Andrey A. Chernov
196d61a92b 1) Add all characters from ~ expansion as protected to be not interpreted
as pattern meta chars.

2) GLOB_ERR and gl_errfunc are supposed to work only for real directories
per POSIX, so don't act on missing or plain files, for ENOENT or ENOTDIR
(as TODO in the code suggested).

3) Remove the hack in the manpage describing how to skip ENOENT and ENOTDIR
in gl_errfunc, it is unneeded now.

4) Set errno to ENAMETOOLONG if g_Ctoc() expansion fails in g_opendir(),
as in other places in the code which are wrappers around system functions.
2016-07-18 18:24:31 +00:00
Andrey A. Chernov
f4d4982ea6 1) POSIX defines well when GLOB_ABORTED can be returned (only for directory
open/read errors and with GLOB_ERR and gl_errfunc processing), so we can't
blindly return it on any MAXPATHLEN overflow. Even our manpage disagrees
with such GLOB_ABORTED usage. Use GLOB_NOSPACE for that now with errno is
set to 0 as for limits.

2) Return GLOB_NOSPACE when valid ~ expansion can't happens due to
MAXPATHLEN overflow too.

3) POSIX (and our manpage) says, if GLOB_ERR is set, GLOB_ABORTED should
be returned immediatelly, without using gl_errfunc. Implement it now.
2016-07-18 16:06:21 +00:00
Andrey A. Chernov
1cecacfe73 Reflect pathnames sorting in collation order. 2016-07-17 13:10:57 +00:00
Andrey A. Chernov
7e9488dda3 In g_Ctoc() apply CHAR() macro to *str to strip all flags. It gains nothing
right now, but some architectures theoretically may 64-bit wchar_t and the
code looks more correct.
2016-07-17 11:25:24 +00:00
Andrey A. Chernov
aa3d69a636 1) This file full of direct char <-> wchar_t assignment, not converted, cut
them down. This hack still remains:
 * 2. Illegal byte sequences in filenames are handled by treating them as
 *    single-byte characters with a values of such bytes of the sequence
 *    cast to wchar_t.

2) Reword the comment in the hack above to reflect implementation.

3) Protect signed wchar_t from sign extension when a signed char is assigned
to it in the hack above.

3) Corresponding backward hack in g_Ctoc() was not implemented, so all
pathes with illegal byte sequences are skipped as result, implement it now.

4) globtilde() forget to convert expanded user home dir from multibyte to
wchar.

5) Protect globtilde() from long expansion truncation.

6) Results was not sorted according to collate as POSIX requires.
2016-07-17 09:39:59 +00:00
Andrey A. Chernov
12eae8c8f3 1) Eliminate possibility to call __*collate_range_cmp() with inclomplete
locale (which cause core dump) by removing whole 'table' argument
by which it passed.

2) Restore __collate_range_cmp() in __sccl().

3) Collating [a-z] range in regcomp() only for single bytes locales
(we can't do it now for other ones). In previous state only first 256
wchars are considered and all others are just silently dropped from the
range.
2016-07-14 09:07:25 +00:00
Andrey A. Chernov
1daad8f5ad Back out non-collating [a-z] ranges.
Instead of changing whole course to another POSIX-permitted way
for consistency and uniformity I decide to completely ignore missing
regex fucntionality and concentrace on fixing bugs in what we have now,
too many small obstacles instead, counting ports.
2016-07-14 08:18:12 +00:00
Andrey A. Chernov
5a5807dd4c Remove broken support for collation in [a-z] type ranges.
Only first 256 wide chars are considered currently, all other are just
dropped from the range. Proper implementation require reverse tables
database lookup, since objects are really big as max UTF-8 (1114112
code points), so just the same scanning as it was for 256 chars will
slow things down.

POSIX does not require collation for [a-z] type ranges and does not
prohibit it for non-POSIX locales. POSIX require collation for ranges
only for POSIX (or C) locale which is equal to ASCII and binary for
other chars, so we already have it.

No other *BSD implements collation for [a-z] type ranges.

Restore ABI compatibility with unused now __collate_range_cmp() which
is visible from outside (will be removed later).
2016-07-10 03:49:38 +00:00
Jilles Tjoelker
f4aa4c78c7 utimes(2),utime(3): Add deprecation in favour of utimensat(2) and futimens(2).
Setting time by seconds or microseconds may cause unexpected effects
especially if sysctl vfs.timestamp_precision=3 (not default).

Calling the obsolete functions with NULL timestamps is acceptable.
2016-06-09 22:14:58 +00:00
Mark Johnston
714ac00292 Implement an NSS backend for netgroups and add getnetgrent_r(3).
This support appears to have been documented in nsswitch.conf(5) for some
time. The implementation adds two NSS netgroup providers to libc. The
default, compat, provides the behaviour documented in netgroup(5), so this
change does not make any user-visible behaviour changes. A files provider
is also implemented.

innetgr(3) is implemented as an optional NSS method so that providers such
as NIS which are able to implement efficient reverse lookup can do so.
A fallback implementation is used otherwise. getnetgrent_r(3) is added for
convenience and to provide compatibility with glibc and Solaris.

With a small patch to net/nss_ldap, it's possible to specify an ldap
netgroup provider, allowing one to query nisNetgroupTriple entries.

Sponsored by:	EMC / Isilon Storage Division
2016-06-09 01:28:44 +00:00
Mark Johnston
03ad7e450a Fix an infinite loop in setnetgrent(3) with NIS netgroups.
Handle an empty result from yp_match() by returning NULL, which is
consistent with the handling of an empty netgroup in /etc/netgroup.
setnetgrent(3) has no return value, so there is no particular need to
distinguish this case from an error.

PR:		26486
MFC after:	2 weeks
2016-06-09 01:11:48 +00:00
Mark Johnston
7f750d5034 Use a more common spelling for "(char *)0" in the getnetgrent man page.
MFC after:	3 days
2016-06-09 01:05:23 +00:00
Brooks Davis
8dfeba04eb Update to a June 8th snapshot of (un)vis form NetBSD.
This adds stravis() and some new encoding flags VIS_SHELL, VIS_META,
and VIS_NOLOCALE.

Assorted cleanups and fixes includeing a manpage typo[0].

PR:		210013 [0]
Submitted by:	pi [0]
2016-06-08 18:21:27 +00:00
Don Lewis
2b34ca7d10 Don't leak olinep if malloc() fails.
If malloc() fails to allocate linep, then free olinep (if it exists)
before returning to avoid a memory leak.

Reported by:	Coverity
CID:		1016716
Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D6755
2016-06-08 10:25:16 +00:00
Pedro F. Giffuni
3c2c0c0443 libc/locale: Fix type breakage in __collate_range_cmp().
When collation support was brought in, the second and third
arguments in __collate_range_cmp() were changed from int to
wchar_t, breaking the ABI. Change them to a "char" type which
makes more sense and keeps the ABI compatible.

Also introduce __wcollate_range_cmp() which does work with wide
characters. This function is used only internally in libc so
we don't export it. Use the new function in glob(3), fnmatch(3),
and regexec(3).

PR:		179721
Suggested by:	ache. jilles
MFC after:	3 weeks (perhaps partial only)
2016-06-05 19:12:52 +00:00
Ed Schouten
0977bd1e88 Fix the signature of the psignal() function.
POSIX 2008 added the psignal() function which has already been part of
the BSDs for a long time. The only difference is, the POSIX version uses
an 'int' for the signal number, unlike our version which uses an
'unsigned int'. Fix up the function to use an 'int'. This should not
affect the ABI.
2016-05-30 13:51:27 +00:00
Don Lewis
9b842193f9 Fix Coverity CID 1016714 Resource leak in process_file_actions_entry()
Don't leak a file descriptor of _dup2() fails (shouldn't happen).

Reported by:	Coverity
CID:		1016714
MFC after:	1 week
2016-05-25 07:13:53 +00:00
Don Lewis
015f4df218 Fix 1016718 Resource leak.
Don't leak a file descriptor if fchdir() fails.

Reported by:	Coverity
CID:		1016718
MFC after:	1 week
2016-05-25 06:55:53 +00:00
Bryan Drewery
487c4f4f36 FTS: Remove stale reference to nfs4 fs which was removed in r192578.
MFC after:	2 weeks
2016-05-21 01:31:41 +00:00
Konstantin Belousov
2a339d9e3d Add implementation of robust mutexes, hopefully close enough to the
intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013.

A robust mutex is guaranteed to be cleared by the system upon either
thread or process owner termination while the mutex is held.  The next
mutex locker is then notified about inconsistent mutex state and can
execute (or abandon) corrective actions.

The patch mostly consists of small changes here and there, adding
neccessary checks for the inconsistent and abandoned conditions into
existing paths.  Additionally, the thread exit handler was extended to
iterate over the userspace-maintained list of owned robust mutexes,
unlocking and marking as terminated each of them.

The list of owned robust mutexes cannot be maintained atomically
synchronous with the mutex lock state (it is possible in kernel, but
is too expensive).  Instead, for the duration of lock or unlock
operation, the current mutex is remembered in a special slot that is
also checked by the kernel at thread termination.

Kernel must be aware about the per-thread location of the heads of
robust mutex lists and the current active mutex slot.  When a thread
touches a robust mutex for the first time, a new umtx op syscall is
issued which informs about location of lists heads.

The umtx sleep queues for PP and PI mutexes are split between
non-robust and robust.

Somewhat unrelated changes in the patch:
1. Style.
2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared
   pi mutexes.
3. Removal of the userspace struct pthread_mutex m_owner field.
4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls
   the lifetime of the shared mutex associated with a vnode' page.

Reviewed by:	jilles (previous version, supposedly the objection was fixed)
Discussed with:	brooks, Martin Simmons <martin@lispworks.com> (some aspects)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2016-05-17 09:56:22 +00:00
Enji Cooper
6ea709b588 Remove trailing whitespace and use nitems(mib) instead of 2 when
calling sysctl(3)

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
2016-04-19 22:59:21 +00:00
Andriy Voskoboinyk
2cf5e9365d libc: do not include <sys/types.h> where <sys/param.h> was already included
According to style(9):
> normally, include <sys/types.h> OR <sys/param.h>, but not both.
(<sys/param.h> already includes <sys/types.h> when LOCORE is not defined).
2016-04-18 21:05:15 +00:00