followed are: Only 3 functions (pthread_cancel, pthread_setcancelstate,
pthread_setcanceltype) are required to be async-signal-safe by POSIX. None of
the rest of the pthread api is required to be async-signal-safe. This means
that only the three mentioned functions are safe to use from inside
signal handlers.
However, there are certain system/libc calls that are
cancellation points that a caller may call from within a signal handler,
and since they are cancellation points calls have to be made into libthr
to test for cancellation and exit the thread if necessary. So, the
cancellation test and thread exit code paths must be async-signal-safe
as well. A summary of the changes follows:
o Almost all of the code paths that masked signals, as well as locking the
pthread structure now lock only the pthread structure.
o Signals are masked (and left that way) as soon as a thread enters
pthread_exit().
o The active and dead threads locks now explicitly require that signals
are masked.
o Access to the isdead field of the pthread structure is protected by both
the active and dead list locks for writing. Either one is sufficient for
reading.
o The thread state and type fields have been combined into one three-state
switch to make it easier to read without requiring a lock. It doesn't need
a lock for writing (and therefore for reading either) because only the
current thread can write to it and it is an integer value.
o The thread state field of the pthread structure has been eliminated. It
was an unnecessary field that mostly duplicated the flags field, but
required additional locking that would make a lot more code paths require
signal masking. Any truly unique values (such as PS_DEAD) have been
reborn as separate members of the pthread structure.
o Since the mutex and condvar pthread functions are not async-signal-safe
there is no need to muck about with the wait queues when handling
a signal ...
o ... which also removes the need for wrapping signal handlers and sigaction(2).
o The condvar and mutex async-cancellation code had to be revised as a result
of some of these changes, which resulted in semi-unrelated changes which
would have been difficult to work on as a separate commit, so they are
included as well.
The only part of the changes I am worried about is related to locking for
the pthread joining fields. But, I will take a closer look at them once this
mega-patch is committed.
character, as some tar implementations incorrectly include a '/' with
the prefix.
Thanks to: Divacky Roman for the UnixWare 7 tarfile that
demonstrated this issue.
code available at tuhs.org, and found out that my chronology
is a bit off. In particular, Seventh Edition already used
the "linkflag" and "linkname" fields. Also, it appears that
there was no tar in Sixth Edition, contrary to what an earlier
tar.1 manpage claimed.
A few mdoc fixes also crept in here.
the size field for a hardlink entry. Specifically, ensure that
we do obey the size field for archives that we know are pax interchange
format archives, as required by POSIX.
Also, clarify the comment explaining why this is necessary and explain
the (very unusual) conditions under which it might fail.
a non-zero size but no body, some write a non-zero size and include
a body. To distinguish these cases, look for a valid tar header immediately
following a hardlink header with non-zero size.
of fcntl(2), flock(2), and lockf(3) advisory locks.
Add such a paragraph to the flock(2) manpage for the
sake of consistency.
Reviewed by: Cyrille Lefevre and Kirk McKusick on -arch
MFC after: 2 weeks
Correct my previous commit and add a comment to the manpage
indicating that the user must set errno to 0 if they wish to
distinguish "no such user" from "error".
Pointed out by: Jacques Vidrine (nectar@)
low bound, and the number of bytes remaining instead of storing the
raw byte sequence and deriving them every time mbrtowc() is called.
This is much faster -- about twice as fast in some crude benchmarks.
constants the wrong way on the VAX. Instead, use C99 hexadecimal
floating-point constants, which are guaranteed to be exact on binary
IEEE machines. (The correct hexadecimal values were already provided
in the source, but not used.) Also, convert the constants to
lowercase to work around a gcc bug that wasn't fixed until gcc 3.4.0.
Prompted by: stefanf
Main ones: mostly use conditional expressions in ifdefs instead of a
mixture of conditional expressions and nested ifdefs.
Nearby ones:
- don't do less than echo the code in the comment about libc_r
- fixed some internal insertion sort errors and indentation errors.
Remove "sys/types.h" as "sys/param.h" is already included
Use cast rather than back-pointer to convert from public to private
version of FTS data, and so avoid littering fts.h with any of the
details.
Pointed out By: bde, kientzle
"A trailing newline is added if none is present."
The code in syslogd, stderr, and console output always adds a newline
at the EOL. However, the existing code never actually removed a
trailing newline, and apparently relied on syslogd to convert it
into a space character. Thus, the existing newline was converted
to a trailing space at the EOL by syslogd, while stderr, and console
output resulted in an empty line.
MFC after: 2 weeks
RuneRange arrays. This is much faster when there are hundreds of
ranges (as is the case in UTF-8 locales) and was inspired by a
similar change made by Apple in Darwin.
of stat(2) calls by keeping an eye of the number of links a directory
has. It assumes that each subdirectory will have a hard link to its
parent, to represent the ".." node, and stops calling stat(2) when
all links are accounted for in a given directory.
This assumption is really only valid for UNIX-like filesystems: A
concrete example is NTFS. The NTFS "i-node" does contain a link
count, but most/all directories have a link count between 0 and 2
inclusive. The end result is that find on an NTFS volume won't
actually traverse the entire hierarchy of the directories passed
to it. (Those with a link count of two are not traversed at all)
The fix checks the "UFSness" of the filesystem before enabling the
optimisation.
Reviewed By: Tim Kientzle (kientzle@)
*printf() and *scanf(). Currently, this reduces the size of libc.so
by 9K on i386. But the real savings are for static binaries that use
*printf() or *scanf() but not strtod(); with an FP-disabled libc,
these binaries will not depend on the gdtoa routines, making each
binary about 22K smaller.
floating-point support, remove default definition of FLOATING_POINT
from the source, and change the compile-time option to
NO_FLOATING_POINT.
- Remove the HEXFLOAT option. It saves an insignificant amount of
space (<0.1% of the size of libc on i386) and complicates vfprintf()
and checkfmt().
-U flag to bsdtar. Essentially, this option breaks existing hard
links. According to SUSv2, tar is supposed to overwrite existing
files on extract by default which, in particular, preserves
existing hard links. Note that this is yet another bug in gtar; it
appears to always break existing links. (Maybe gtar's -U is broken?)
I'm unsure about how to handle this for other file types; the current
code always unlinks first unless the NO_OVERWRITE flag is specified.
I've commented this issue liberally and will come back to it later.
archive formats supported by libarchive, with some information about
the relative strengths and weaknesses of each format and notes about
issues with libarchive's support for those formats.
This page should make it unnecessary to list all of the libarchive
formats in the manpage of each program that uses libarchive.
Such programs can simply refer to libarchive-formats(5).
* little-endian old-style binary cpio archives
* big-endian old-style binary cpio archives
* SVR4 portable archives without CRC
* SVR4 portable archives with CRC
Note that I don't yet verify the CRC for the last one, and I'm
not quite certain I'm correctly parsing device numbers.
MS-CHAPv1 MPPE-keys).
- Added rad_demangle_mppe_key() for demangling mppe-keys (needed
for MPPE-keys).
- Added some typecasts for avoiding compiler warnings.
- Fix: better handle wrong usage of the lib (if the programmer
has not called rad_create_request() but rad_put_*(), then a
weird error message was returned).
- Added a new function for putting the Message-Authenticator.
- Verify the Message-Authenticator, if it was found inside a
response packet and silently drop the packet, if the validation
failed.
- Implicitly put the Message-Authenticator, if the EAP-Message
attribute was added.
- Added some missing defines.
Submitted by: Michael Bretterklieber
PR: 46555
The new fflags support in archive_entry supports Linux and FreeBSD
file flags and is a bit more gracious about unrecognized flag names
than strtofflags(3). This involves some minor API breakage.
The default tar format ("restricted pax") now enables pax extensions
when archiving files that have flags. In particular, copying dir
heirarchies with 'bsdtar cf - -C src . | bsdtar xpf - -C dest' now
preserves file flags. (Note the "p" on extract!)
While I'm here, fill in some additional explanation in the
archive_entry.3 manpage, fill in some missing MLINKS, mark some
overlooked internal functions 'static', and make a few minor style
fixes.
namespaces are visible. Previously, math.h failed to hide some C99-,
XSI-, and BSD-specific symbols in certain compilation environments.
The referenced PR has a nice listing of the appropriate conditions for
making symbols visible in math.h. The only non-stylistic difference
between the patch in the PR and this commit is that I superfluously
test for __BSD_VISIBLE in a few places to be more explicit about which
symbols have historically been part of the FreeBSD environment.
PR: 65939
Submitted by: Stefan Farfeleder <stefan@fafoe.narf.at>
makeing sure the spinlock isn't already in use might be a nice feature to
have in theory, it's hard to implement in practice since the passed in
pointer may not be NULL, but still be an invalid value (i.e. 1..2..3.. etc).
The original might have pointers to user-specified strings;
copying the string (instead of just the pointer) protects against
the client re-using their own buffers.
I'm trying hard to avoid dumping all of the 'set' string functions
in favor of slower, but more predictable 'copy' semantics.
after their change from an array of char to an array of enum.
This fixes problems that occurred when using positional arguments in
format strings, particularly with more than STATIC_ARG_TBL_SIZE (8)
of them.
PR: 65841
Submitted by: Steven Smith (mostly)
adjunct maps are used. One symtom of this bug is sshd saying:
login_get_lastlog: Cannot find account for uid X
when logging in. The problem here is caused by an incorrect reuse of the rv
variable when previous values are needed later.
High-resolution mtime/ctime/atime is not POSIX-standard, so hide
set/get of high-resolution time fields behind easily-mutable macros.
That makes it easier to change how those fields are accessed.
res_search only incremented got_servfail for h_errno == TRY_AGAIN *AND*
hp->rcode == SERVFAIL. However, there are cases such as timeouts where
rcode is not always set to SERVFAIL. This leads to inconsistent nameserver
operation during multi-domain and truncated dot searches, especially during
booting when portions of the network are being brought up simultanious with
dns lookups.
This patch attempts to correct the problem by unconditionally terminating
the search if TRY_AGAIN is returned (after res_query has gone through all
retries and name servers) instead of trying other domain elements in the
domain seach path.
This patch should fix reported problems (which I can reproduce) with some
NFS mounts failing during boot. This occured because mount_nfs thought the
host name lookup returned a definitive failure using a non-dotted host name
when, in fact, it timed out on the first part (host.search.domain.name) and
got a definitive host-not-found response on the second part (host.).
Generally speaking, search path name server timeouts can exceed 60 seconds
per element and most machines which consistently timeout on earlier portions
of a search path are effectively non-operational due to the imposed delays.
It is more important for DNS lookups to return the proper error code then
to be able to recover a valid lookup in later portions of the search path
in these situations.
Obtained from: DragonFly
MFC after: 3 weeks
Earlier versions of FreeBSD don't support ACLs.
Note that the ACL support code in archive_entry is standalone code and
unaffected by this. (In particular, it should be possible to
manipulate archives containing ACLs even if the ACLs cannot be
restored on the current system.)
* Re-use a single buffer for shar output formatting rather
than hammering the heap. (archive_write_set_format_shar.c)
* Fix a handful of minor memory leaks and clean up some of the
memory-management code.
try to set ACLs even if fflag restore fails, first cut at reading
Solaris tar ACLs
Code improvement: merge gnu tar read support into main tar reader;
this eliminates a lot of duplicate code and generalizes the tar
reader to handle formats with GNU-like extensions.
Style: Makefile cleanup, eliminate 'dmalloc' references, remove 'tartype'
from archive_entry (this makes archive_entry more format-agnostic)
Thanks to: David Magda for providing Solaris tar test files
mode (where the forked thread is the one and only thread and
is marked as system scope), set the system scope flag before
initializing the signal mask. This prevents trying to use
internal locks that haven't yet been initialized.
Reported by: Dan Nelson <dnelson at allantgroup.com>
Reviewed by: davidxu
* ACL storage is no longer erased before a group of entries are added.
* ACL text creation no longer tries to skip over non-existent text.
* UTF8 encoder no longer blows up on invalid wide characters.
* Fixed ACL state management for default ACLs.
Also, publicize function for obtaining text-format ACL in various
formats. The interface is now extensible through a "flags" argument
that allows you to select a variant format.
with 'star' ACL handling, though there's still a
bit more work needed in this area.
Added 'write_open_fd' and 'read_open_fd' to simplify, e.g.,
tar's u and r modes. Eliminated old 'write_open_file_position'
as a bad idea. (It required closing/reopening files to
do updates, which led to unpleasant implications.)
Various other minor fixes, API tweaks, etc.
on temporary nameserver failure. This is necessary to get
getipnodebyname(3) to correctly return h_errno=TRY_AGAIN instead
of HOST_NOT_FOUND.
Reviewed by: green, thomas
MFC after: 1 week
case where an /etc/nsswitch.conf file was present, but could not
be opened (e.g. due to permissions). Previously, the open failure
condition was suppressed, and the built-in defaults were used. In
revision 1.11, however, propagated the open failure causing all
nsdispatch() invocations to return NS_UNAVAIL, and thus many APIs
including getpwnam and gethostbyname unconditionally failed.
This commit restores the previous behavior.
Pointy hat: nectar (+1 for obstinance; ache had to use clue bat)
Reported by: ache
solved by a simple 'make world'. The signalcontext function was going
to the trouble of generating an even 16 byte alignment, but in fact it
needed to be odd aligned to simulate the 8-byte return address having
been pushed by the caller. This fixes yet another group of crashes in
applications using libpthread. And yet again, it was my fault all along.
While here, rename the duplicate internal ctx_wrapper() functions to
makectx_wrapper() and sigctx_wrapper() so that traces aren't ambiguous.
library, it may pull in that thread library at run time. If the
process started out single-threaded, this could cause attempts to
release locks that do not exist. Guard against this possibility by
checking __isthreaded before invoking thread primitives.
A similar problem remains if the process is linked against one thread
library, but the NSS module is linked against another. This can only
be avoided by careful design of the NSS module.
Submitted by: Sean McNeil <sean@mcneil.com> (mostly; bugs are mine)
functionality spelled out in SUSv3.
o Signal of 0 means do everything except send the signal
o Check that the signal is not invalid
o Check that the target thread is not dead/invalid
sigprocmask no longer needs to be wrapped.
o raise(3) is applied to the calling thread in a threaded program.
o In the sigaction wrapper reference the correct structure.
o Don't treat SIGTHR especially anymore (infact it won't exist in
a little while).
we still have to DTRT when an asynchronously cancellable thread is
cancelled while waiting for a mutex.
o While dequeueing a waiting mutex don't skip a thread if it has
a cancel pending. Only skip it if it is also async cancellable.
the cause of any bugs because it is *always* indirectly set
in the for...loop, but better to be explicit about it.
o Check the magic number of the passed in thread only after it has
been found in the active thread list. Otherwise, if the check is done
at the very beginning we may end up pointing to garbage if the
thread was once a valid thread, but has now been destroyed.
* Disabled shared-library building, as some API breakage is
still likely. (I didn't realize it was turned on by default.) If
you have an existing /usr/lib/libarchive.so.2, I recommend deleting it.
* Pax interchange format now correctly stores and reads UTF8
for extended attributes. In particular, pax format can portably
handle arbitrarily long pathnames containing arbitrary characters.
* Library compiles cleanly at -O2, -O3, and WARNS=6 on all
FreeBSD-CURRENT platforms.
* Minor portability improvements inspired by Juergen Lock
and Greg Lewis. (Less reliance on stdint.h, isolating of
various portability-challenged constructs.)
* archive_entry transparently converts multi-byte <-> wide character
strings, allowing clients and format handlers to deal with either
one, as appropriate.
* Support for reading 'L' and 'K' entries in standard tar archives
for star compatibility.
* Recognize (but don't yet handle) ACL entries from Solaris tar.
* Pushed format-specific data for format readers down into
format-specific storage and out of library-global storage. This
should make it easier to maintain individual formats without mucking
with the core library management.
* Documentation updates to track the above changes.
* Updates to tar.5 to correct a few mistakes and add some additional
information about GNU tar and Solaris tar formats.
Notes:
* The basic 'tar' reader is getting more general; there's not much
point in keeping the 'gnutar' reader separate. Merging the two
would lose a bunch of duplicate code.
* The libc ACL support is looking increasingly inadequate for my needs
here. I might need to assemble some fairly significant code for
parsing and building ACLs. <sigh>
ferror(), fileno() and clearerr(), using the value of __isthreaded to
decide between the fast inline single-threaded code and the more
general function equivalent. This gives most of the performance
benefits of the old unsafe macros while preserving thread safety.
addresses. For arch's with 64-bit longs, this is a nop, but for i386 this
allows sysinstall to properly handle disks and filesystems > 1 TB.
Changes from the original patch include:
- Use d_addr_t rather than inventing a blkcnt type based on int64_t.
- Use strtoimax() rather than strtoull() to parse d_addr_t's from config
files.
- Use intmax_t casts and %jd rather than %llu to printf d_addr_t values.
Tested on: i386
Tested by: kuriyama
Submitted by: julian
MFC after: 1 month
MATH_ERREXCEPTION and math_errhandling, so that C99 applications at
least have the possibility of determining that errno is not set for
math functions. Set math_errhandling to the non-standard-conforming
value of 0 for now to indicate that we don't support either method
of reporting errors. We intentionally don't support MATH_ERRNO
because errno is a mistake, and we are missing support for
MATH_ERREXCEPTION (<fenv.h>, compiler support for <fenv.h>, and
actually setting the exception flags correctly).
- Add DECL wrappers to libgeom.h.
- Rename structure members in libgeom.h to use a lg_ prefix for member
names. This is required because a few structures had members named
'class' which made g++ very unhappy.
- Catch gstat(8) and gconcat(8) up to these API changes.
Reviewed by: phk
Portability: Thanks to Juergen Lock, libarchive now compiles cleanly
on Linux. Along the way, I cleaned up a lot of error return codes and
reorganized some code to simplify conditional compilation of certain
sections.
Bug fixes:
* pax format now actually stores filenames that are 101-154
characters long.
* pax format now allows newline characters in extended attributes
(this fixes a long-standing bug in ACL handling)
* mtime/atime are now restored for directories
* directory list is now sorted prior to fix-up to permit
correct restore of non-writable dir heirarchies
structure and call stdio functions. In 5.X this was broken when FILE
locking was introduced into libc.
This change makes most (relevant) stdio functions work again when the
_extra file in FILE isn't initialised (and can't be without a libc
function to do it since the __sFILEX structure is private to libc).
This doesn't yet address the issue of selective restore
of hardlinked files. With cpio format, it's possible to correctly
restore any linked file; the API doesn't yet fully support this.
(There's no way for the library to inform a client whether or not
there's a file body associated with this entry. The assumption
right now is that "hardlink" entries have no file body.)
cleanups, handling 'ls -l-', handling '--*'
Note this is in the same time back out of our v1.3
"Don't print an error message if the bad option is '?'"
because it directly violates POSIX.
the size in the archive_entry object to zero if that format doesn't
store a body for that file type. This allows the client to determine
whether or not it should feed the file body to the archive. In
particular, cpio stores the file body for hardlinks, tar and shar
don't. With this change, bsdtar now correctly archives hardlinks in all
supported formats.
While I'm here, make shar output be more aggressive about creating directories.
Before this, commands such as:
bsdtar -cv -F shar some/explicit/path/to/a/file
wouldn't create the directory. Some simple logic to remember the last
directory creation helps reduce unnecessary mkdirs here.
At this point, I think the only flaw in libarchive's cpio support is
the failure to recognize hardlinks when reading.
While I'm here, fix a bug in reading filenames from
cpio files. (Copy should count the length of the name,
not the number of bytes available for input.)
that this provokes. "Wherever possible" means "In the kernel OR NOT
C++" (implying C).
There are places where (void *) pointers are not valid, such as for
function pointers, but in the special case of (void *)0, agreement
settles on it being OK.
Most of the fixes were NULL where an integer zero was needed; many
of the fixes were NULL where ascii <nul> ('\0') was needed, and a
few were just "other".
Tested on: i386 sparc64
Further contemplation has convinced me that this was
not going to really solve the problem of environment-poisoning
without raising serious administrative headaches. There
must be a better way...
- Fix syntax
- Remove the (slightly wrong) duplicate explanation of the error condition
- Change reference to invalid multibyte character into invalid wide character
This function removes all environment variables except
the ones listed on a "whitelist."
The function accepts two whitelist arguments.
If the first is NULL, a built-in default list will be
used. This allows callers to get a variety of behaviors:
* Default screening: provide NULL for both lists
* Custom screening: provide a custom list for the first argument
* Modified default screening: provide NULL for first arg,
list of additional variables to preserve in the second arg
Idea from: Jacques Vidrine
MFC after: 2 weeks
-static to CFLAGS). It just turned rev.1.5 into an obfuscated no-op.
As explained in the log for rev.1.5, testing should be done in the
host environment but there is a problem in cross-compilation environments.
As not explained in the log for rev.1.6, there was apparently a practical
problem with cross-compiling (makeworld should have set -static in
LDFLAGS but apparently didn't). Cross-compilation was especially
complicated because the relevant programs are test programs that were
run at beforeinstall time -- dynamic libraries might or might not exist
depending on the build options. The complications became moot in
rev.1.8 when beforeinstall was renamed "test".
The getaddrinfo(3), getipnodebyname(3) and resolver(3) can coincide now
with what should be totally reentrant, and h_errno values will now
be preserved correctly, but this does not affect interfaces such as
gethostbyname(3) which are still mostly non-reentrant.
In all of these relevant functions, the thread-safety has been pushed
down as far as it seems possible right now. This means that operations
that are selected via nsdispatch(3) (i.e. files, yp, dns) are protected
still under global locks that getaddrinfo(3) defines, but where possible
the locking is greatly reduced. The most noticeable improvement is
that multiple DNS lookups can now be run at the same time, and this
shows major improvement in performance of DNS-lookup threaded programs,
and solves the "Mozilla tab serialization" problem.
No single-threaded applications need to be recompiled. Multi-threaded
applications that reference "_res" to change resolver(3) options will
need to be recompiled, and ones which reference "h_errno" will also
if they desire the correct h_errno values. If the applications already
understood that _res and h_errno were not thread-safe and had their own
locking, they will see no performance improvement but will not
actually break in any way.
Please note that when NSS modules are used, or when nsdispatch(3)
defaults to adding any lookups of its own to the individual libc
_nsdispatch() calls, those MUST be reentrant as well.
the caller does not specify the rule number -- instead, the kernel
module is probed for the next available rule, which is then used.
Obtained from: TrustedBSD Project
Sponsored by: DARPA, McAfee Research
