avoid creating bad entries in the grp list as a result of memory allocation
failures while building new entries.
PR: bin/83340
Reviewed by: delphij (prior version of patch)
Introduce dirfd() libc exported symbol replacing macro with same name,
preserve _dirfd() macro for internal use.
Replace dirp->dd_fd with dirfd() call. Avoid using dirfd as variable
name to prevent shadowing global symbol.
Sponsored by: Google Summer Of Code 2011
The NAND Flash environment consists of several distinct components:
- NAND framework (drivers harness for NAND controllers and NAND chips)
- NAND simulator (NANDsim)
- NAND file system (NAND FS)
- Companion tools and utilities
- Documentation (manual pages)
This work is still experimental. Please use with caution.
Obtained from: Semihalf
Supported by: FreeBSD Foundation, Juniper Networks
[ENOENT] A component of file_name does not name an existing file or
file_name points to an empty string.
[ENOTDIR] A component of the path prefix is not a directory, or the
file_name argument contains at least one non- <slash> character
and ends with one or more trailing <slash> characters and the last
pathname component names an existing file that is neither a
directory nor a symbolic link to a directory.
Add checks for the listed conditions, and set errno accordingly.
Update the realpath(3) manpage to mention SUS behaviour. Remove the
requirement to include sys/param.h before stdlib.h.
PR: 128933
MFC after: 3 weeks
in the last import. They are sometimes needed when you want to use
advanced instructions.
Also, add clang's internal stdalign.h header to ObsoleteFiles.inc, since
it is redundant: we already have a stdalign.h header in /usr/include.
Pointy hat to: dim
PR: kern/167574
Submitted by: jkim
Reported by: Oliver Hartmann <ohartman@zedat.fu-berlin.de>
MFC after: 2 weeks
Enqueue thread in LIFO, this can cause starvation, but it gives better
performance. Use _thr_queuefifo to control the frequency of FIFO vs LIFO,
you can use environment string LIBPTHREAD_QUEUE_FIFO to configure the
variable.
when there is no timeout, because read(2) will return immediately if there
is no data waiting in the TCP buffer, causing fetch_read() to busy-loop on
slow connections.
MFC after: 3 weeks
Noticed by: Yanhui Shen <shen.elf@gmail.com>
The typical case was:
static __inline int
convert_ccl(FILE *fp, char * __restrict p, [...])
{
[...]
if (p == SUPPRESS_PTR) {
[...]
} else {
[...]
}
[...]
}
This qualifier says that the pointer is the only one at that time
pointing to the resource.
Here, clang considers that "p" will never match "SUPPRESS_PTR" and
optimize the if{} block out. This leads to segfaults in programs calling
vfscanf(3) and vfwscanf(3) with just the format string (no arguments
following it).
The following softwares were reported to abort with segmentation fault
and this patch fixes it:
o cmake
o smartd
o devel/ORBit2
dim@ opened an LLVM PR to discuss this clang optimization:
http://llvm.org/bugs/show_bug.cgi?id=12656
Tested by: bsam@
conversions. Both the specification and the documentation say the
width is interpreted as the max number of wide characters to read, but
the implementation was interpreting it as the number of bytes to convert.
(See also r105317.)
This change has security implications for any applications that depend
on the buggy behavior, but the impact in practice is probably nil.
Any such application would already be buggy on other platforms that
get the semantics right. Also, these conversions are rarely used;
%ls, %lc, and %l[ are more appropriate.
reading wide characters manually. With this change, they now use
fgetwc(). To make this work, we use an internal version of fgetwc()
with a few extensions: it takes an mbstate * because non-wide streams
don't have a built-in mbstate, and it indicates the number of bytes
read.
vfscanf() now resembles vfwscanf() more closely. Minor functional
improvements include working xlocale support in vfscanf(), setting the
stream error indicator on encoding errors, and proper handling of
shift-based encodings. (Actually, making shift-based encodings work
with non-wide streams is hopeless, but the implementation now matches
the broken specification.)
This tool changes the default buffering behaviour of standard
stdio streams.
It only works on dynamic binaries. To make it work for static
ones it would require cluttering stdio because there no single
entry point.
PR: 166660
Reviewed by: current@, jhb
Approved by: kib (mentor)
MFC after: 1 week
1. The licensing terms for the MD2 routines from RFC is not under a BSD-like
license. Instead it is only granted for non-commercial Internet
Privacy-Enhanced Mail.
2. MD2 is quite deprecated as it is no longer considered a cryptographically
strong algorithm.
Discussed with: so (cperciva), core
This is useful for GNU/kFreeBSD and the libusb2debian port.
Applications using the asynchronous API of LibUSB 1.0 needs
to be recompiled after this update.
Found by: lme @
indicates the avaliability of FILE, to prevent possible reordering of
the writes as seen by other CPUs.
Reported by: Fengwei yin <yfw.bsd gmail com>
Reviewed by: jhb
MFC after: 1 week
by separate conversion functions. This will hopefully make bugs more
noticeable (I noticed several already) and provide opportunities to
reduce code duplication.
prior to 3.0.0 release). This fixes several bugs related to memory
initialization.
Mangle __jemalloc_a0{malloc,calloc,free}() just like all the other
library-internal symbols in jemalloc, and adjust the tls allocation code
in libc to use the mangled names.
(i.e., the return value would overflow), set errno to EOVERFLOW
and return an error. This improves the chances that buggy
applications -- for instance, ones that pass in a negative integer
as the size due to a bogus calculation -- will fail in safe ways.
Returning an error in these situations is specified by POSIX, but
POSIX appears to have an off-by-one error that isn't duplicated in
this change.
Previously, some of these functions would silently cap the size at
INT_MAX+1, and others would exit with an error after writing more
than INT_MAX characters.
PR: 39256
MFC after: 2 weeks
true if the size is zero.
- Fix a claim that sprintf() is the same as snprintf() with an
infinite size. It's equivalent to snprintf() with a size of
INT_MAX + 1.
- Document the return values in the return values section.
- Document the possible errno value of EOVERFLOW.
MFC after: 2 weeks
infinite loop pretty much unconditionally. It's remarkable that the
patch that introduced the bug was never tested, but even more
remarkable that nobody noticed for over two years.
PR: 167039
MFC after: 3 days
way: first they are compiled to assembly, then some sed'ing is done on
the assembly, and lastly the assembly is compiled to an object file.
This last step is done using ${CC}, and not ${AS}, because when the
compiler is clang, it outputs directives that are too advanced for our
old gas. So we use clang's integrated assembler instead. (When the
compiler is gcc, it just calls gas, and nothing is different, except one
extra fork.)
However, in the .s to .o rules in lib/csu/$ARCH/Makefile, I still passed
CFLAGS to the compiler, instead of ACFLAGS, which are specifically for
compiling .s files.
In case you are using '-g' for debug info anywhere in your CFLAGS, it
causes the .s files to already contain debug information in the assembly
itself. In the next step, the .s files are also compiled using '-g',
and if the compiler is clang, it complains: "error: input can't have
.file dwarf directives when -g is used to generate dwarf debug info for
assembly code".
Fix this by using ${ACFLAGS} for compiling the .s files instead.
Reported by: jasone
MFC after: 1 week
In this case it doesn't really matter, as long as we turn a TTY name
into a set of shuffled bytes. Still, for correctness we should use the
proper function.
MFC after: 2 weeks
prior to 3.0.0 release) as contrib/jemalloc, and integrate it into libc.
The code being imported by this commit diverged from
lib/libc/stdlib/malloc.c in March 2010, which means that a portion of
the jemalloc 1.0.0 ChangeLog entries are relevant, as are the entries
for all subsequent releases.
Before, we took the first argument to pam_exec(8). With the addition of
options in front of the command, this could be wrong.
Now, options are parsed before calling _pam_exec() and messages contain
the proper command name.
While here, fix a warning.
Sponsored by: Yakaz (http://www.yakaz.com)
RFC 1738 specifies that any ":", "@", or "/" within a user name or
password in a URL is percent-encoded, to avoid ambiguity with the use
of those characters as URL component separators.
Reviewed by: rstone@
MFC after: 1 month
outside the range of valid file descriptors
PR: kern/164970
Submitted by: Peter Jeremy <peterjeremy@acm.org>
Reviewed by: jilles
Approved by: cperciva
MFC after: 1 week
usermode context switches (long jumps and ucontext operations). If these
are used across threads, multiple threads can end up with the same TLS base.
Madness will then result.
This makes behavior on PPC match that on x86 systems and on Linux.
MFC after: 10 days
compatibility, it broke programs using devstat, under 32-bit compatibility and
not.
It's very difficult to fix the identifiers used by devstat, so this change is
simply being backed out. Since changes to 3rd-party code seem likely, and may be
necessary to properly fix 32-bit binaries on 64-bit kernel, it would seem better
to make more invasive changes to fix GEOM's problems with 32-bit compatibility in
general.
The right thing to do is to replace all of the use of pointers as opaque
identifiers with a fixed-size (64-bit or even 32-bit should be enough for tracking
unique GEOM elments) field. That probably maintains source compatibility with
most GEOM consumers, and allows xml2tree to make better assumptions about how to
decode the identifiers.
Submitted by: Anton Yuzhaninov
privilege attempts to toggle SF_SETTABLE flags.
- Use the '^' operator in the SF_SNAPSHOT anti-toggling check.
Flags are now stored to ip->i_flags in one place after all checks.
Submitted by: bde
is already open in this process.
If the named semaphore is already open, sem_open() only increments a
reference count and did not take the flags into account (which otherwise
happens by passing them to open()). Add an extra check for O_CREAT|O_EXCL.
PR: kern/166706
Reviewed by: davidxu
MFC after: 10 days
quotation. Also make sure we have the same amount of columns in each row as
the number of columns we specify in the head arguments.
Reviewed by: brueffer
correct sign when the remainder was 0.
Fix a separate bug in remquo alone, in which the remainder and
quotient were both off by a bit in certain cases involving subnormal
remainders.
The bugs affected all platforms except amd64 and i386, on which the
routines are implemented in assembly.
PR: 166463
Submitted by: Ilya Burylov
MFC after: 2 weeks
application destroys semaphore after sem_wait returns. Just enter
kernel to wake up sleeping threads, only update _has_waiters if
it is safe. While here, check if the value exceed SEM_VALUE_MAX and
return EOVERFLOW if this is true.
a mutex after a thread has unlocked it, it event writes data to the mutex
memory to clear contention bit, there is a race that other threads
can lock it and unlock it, then destroy it, so it should not write
data to the mutex memory if there isn't any waiter.
The new operation UMTX_OP_MUTEX_WAKE2 try to fix the problem. It
requires thread library to clear the lock word entirely, then
call the WAKE2 operation to check if there is any waiter in kernel,
and try to wake up a thread, if necessary, the contention bit is set again
by the operation. This also mitgates the chance that other threads find
the contention bit and try to enter kernel to compete with each other
to wake up sleeping thread, this is unnecessary. With this change, the
mutex owner is no longer holding the mutex until it reaches a point
where kernel umtx queue is locked, it releases the mutex as soon as
possible.
Performance is improved when the mutex is contensted heavily. On Intel
i3-2310M, the runtime of a benchmark program is reduced from 26.87 seconds
to 2.39 seconds, it even is better than UMTX_OP_MUTEX_WAKE which is
deprecated now. http://people.freebsd.org/~davidxu/bench/mutex_perf.c
via procstat(1) and fstat(1):
- Change shm file descriptors to track the pathname they are associated
with and add a shm_path() method to copy the path out to a caller-supplied
buffer.
- Use the fo_stat() method of shared memory objects and shm_path() to
export the path, mode, and size of a shared memory object via
struct kinfo_file.
- Add a struct shmstat to the libprocstat(3) interface along with a
procstat_get_shm_info() to export the mode and size of a shared memory
object.
- Change procstat to always print out the path for a given object if it
is valid.
- Teach fstat about shared memory objects and to display their path,
mode, and size.
MFC after: 2 weeks
Kernel pointer values are used as opaque unique identifiers, which are then
used to reconstruct references between various providers, classes, etc., inside
libgeom from the source XML. Unfortunately, they're converted to pointer-width
integers (in the form of pointers) to do this, and 32-bit userland pointers
cannot hold sensible representations (however opaque) of 64-bit kernel pointers
on all systems.
In the case where the leading bits are zero and 32 distinct bits of pointer can
be identified, this will happen to work. On systems where the upper 32-bits of
kernel pointers are non-zero and the same for all kernel pointers, this will
result in double frees and all kinds of bizarre crashes and linkage between
objects inside libgeom.
To mitigate this problem, treat the opaque identifiers in the XML as C strings
instead, and internalize them to give unique and consistent per-object pointer
values in userland for each identifier in the XML. This allows us to keep the
libgeom logic the same with only minor changes to initial setup and parsing.
It might be more sensible for speed reasons to treat the identifiers as numbers
of a large size (uintmax_t, say) rather than strings, but strings seem fine for
now.
(As an added side-effect, this makes it slightly easier to identify unresolved
references, but nothing has been added to inform the user of those.)
New kernel events can be added at various location for sampling or counting.
This will for example allow easy system profiling whatever the processor is
with known tools like pmcstat(8).
Simultaneous usage of software PMC and hardware PMC is possible, for example
looking at the lock acquire failure, page fault while sampling on
instructions.
Sponsored by: NETASQ
MFC after: 1 month
pam_exec(8) now accepts a new option "return_prog_exit_status". When
set, the program exit status is used as the pam_exec return code. It
allows the program to tell why the step failed (eg. user unknown).
However, if it exits with a code not allowed by the calling PAM service
module function (see $PAM_SM_FUNC below), a warning is logged and
PAM_SERVICE_ERR is returned.
The following changes are related to this new feature but they apply no
matter if the "return_prog_exit_status" option is set or not.
The environment passed to the program is extended:
o $PAM_SM_FUNC contains the name of the PAM service module function
(eg. pam_sm_authenticate).
o All valid PAM return codes' numerical values are available
through variables named after the return code name. For instance,
$PAM_SUCCESS, $PAM_USER_UNKNOWN or $PAM_PERM_DENIED.
pam_exec return code better reflects what went on:
o If the program exits with !0, the return code is now
PAM_PERM_DENIED, not PAM_SYSTEM_ERR.
o If the program fails because of a signal (WIFSIGNALED) or doesn't
terminate normally (!WIFEXITED), the return code is now
PAM_SERVICE_ERR, not PAM_SYSTEM_ERR.
o If a syscall in pam_exec fails, the return code remains
PAM_SYSTEM_ERR.
waitpid(2) is called in a loop. If it returns because of EINTR, do it
again. Before, it would return PAM_SYSTEM_ERR without waiting for the
child to exit.
Several log messages now include the PAM service module function name.
The man page is updated accordingly.
Reviewed by: gleb@, des@
Sponsored by: Yakaz (http://www.yakaz.com)
MFC after: 2 weeks
- Rename pmc.mips to pmc.mips24k since it covers just one CPU,
no whole architecture
- Add documetnations for Octeon's PMC counters
- Remove CAVEATS section from pmc.mips24k page: PMC for MIPS supports
sampling now.
Because the utmpx interface is generally not required to be thread-safe,
but it is nice to have, if easy to do so. Therefore don't make a mess
out of the code and only use it if __NO_TLS is not defined.
several new kerberos related libraries and applications to FreeBSD:
o kgetcred(1) allows one to manually get a ticket for a particular service.
o kf(1) securily forwards ticket to another host through an authenticated
and encrypted stream.
o kcc(1) is an umbrella program around klist(1), kswitch(1), kgetcred(1)
and other user kerberos operations. klist and kswitch are just symlinks
to kcc(1) now.
o kswitch(1) allows you to easily switch between kerberos credentials if
you're running KCM.
o hxtool(1) is a certificate management tool to use with PKINIT.
o string2key(1) maps a password into key.
o kdigest(8) is a userland tool to access the KDC's digest interface.
o kimpersonate(8) creates a "fake" ticket for a service.
We also now install manpages for some lirbaries that were not installed
before, libheimntlm and libhx509.
- The new HEIMDAL version no longer supports Kerberos 4. All users are
recommended to switch to Kerberos 5.
- Weak ciphers are now disabled by default. To enable DES support (used
by telnet(8)), use "allow_weak_crypto" option in krb5.conf.
- libtelnet, pam_ksu and pam_krb5 are now compiled with error on warnings
disabled due to the function they use (krb5_get_err_text(3)) being
deprecated. I plan to work on this next.
- Heimdal's KDC now require sqlite to operate. We use the bundled version
and install it as libheimsqlite. If some other FreeBSD components will
require it in the future we can rename it to libbsdsqlite and use for these
components as well.
- This is not a latest Heimdal version, the new one was released while I was
working on the update. I will update it to 1.5.2 soon, as it fixes some
important bugs and security issues.
no waiters, we still increase and decrease count in user mode without
entering kernel, once there is a waiter, sem_post will enter kernel to
increase count and wake thread up, this is atomicy and allow us to
gracefully destroy semaphore after sem_wait returned.
pathnames.
With the current API (no *at functions), FTS_NOCHDIR requires that the
fts_accpath start with the original path passed to fts_open(); therefore,
the depth that can be reached is limited by the {PATH_MAX} constraint on
this pathname.
MFC after: 1 week
example, it uses a serialization point like following:
pthread_mutex_lock(&mutex);
pthread_mutex_unlock(&mutex);
pthread_mutex_destroy(&muetx);
They think a previous lock holder should have already left the mutex and
is no longer referencing it, so they destroy it. To be maximum compatible
with such code, we use IA64 version to unlock the mutex in kernel, remove
the two steps unlocking code.
Just like kill(2), it is impossible for killpg(0, ...) to fail with
ESRCH, as a process always has a process group.
Discussed on: arch@
MFC after: 1 week
On FreeBSD, all processes have a process group, so it is impossible for
kill(2) to fail this way. POSIX also doesn't mention this error
condition.
Discussed on: arch@
MFC after: 3 weeks
properly reloaded when calling _fini() in large binaries with multiple
TOC sections (e.g. GCC), leading to a segmentation fault. Adding -mlongcall
to crt1 flags causes the compiler to emit explicit TOC load instructions
for all function calls, including _fini().
Reviewed by: kib
Pointy hat to: kib
call preinit, init and fini arrays methods from crt1 for static binaries.
Mark new crt1 with FreeBSD-specific ELF note.
Move some common crt1 code into new MI file ignore_init.c, to reduce
duplication. Also, conservatively adjust nearby sources for style.
Reviewed by: kan
Tested by: andrew (arm), flo (sparc64)
MFC after: 3 weeks
- Fix TLS allocation for Variant I: both rtld and libc allocators
assume that tls_static_space includes space for TLS structure.
So increment calculated static size by the size of it.
syscall. Before r5958, seekdir() was called for its side effect of
freeing memory allocated by opendir() for rewinddir(), but that revision
added _reclaim_telldir() that frees all memory allocated by telldir()
calls, making this call redundant.
This introduces a slight change. If an application duplicated the descriptor
obtained through dirfd(), it can no longer rely on file position to be
reset to the start of file after a call to closedir(). It's believed to
be safe because neither POSIX, nor any other OS I've tested (NetBSD, Linux,
OS X) rewind the file offset pointer on closedir().
Reported by: Igor Sysoev
They were made excessive in r205424 by opening with O_DIRECTORY.
Also eliminated the fcntl() call used to set FD_CLOEXEC by opening
with O_CLOEXEC.
(fdopendir() still checks that the passed descriptor is a directory,
and sets FD_CLOEXEC on it.)
Reviewed by: ed
get rid of testing explicitly for clang (using ${CC:T:Mclang}) in
individual Makefiles.
Instead, use the following extra macros, for use with clang:
- NO_WERROR.clang (disables -Werror)
- NO_WCAST_ALIGN.clang (disables -Wcast-align)
- NO_WFORMAT.clang (disables -Wformat and friends)
- CLANG_NO_IAS (disables integrated assembler)
- CLANG_OPT_SMALL (adds flags for extra small size optimizations)
As a side effect, this enables setting CC/CXX/CPP in src.conf instead of
make.conf! For clang, use the following:
CC=clang
CXX=clang++
CPP=clang-cpp
MFC after: 2 weeks
o Wrap sentences on to new lines
o Rewrap lines where possible while trying to keep the diff to a
minimum
Found with: textproc/igor
MFC after: 1 week
X-MFC-With: r232157
Some of new features:
- New readers: RAR, LHA/LZH, CAB reader, 7-Zip
- New writers: ISO9660, XAR
- Improvements to many formats, especially including ISO9660 and Zip
- Stackable write filters to write, e.g., tar.gz.uu in a single pass
- Exploit seekable input; new "seekable" Zip reader can exploit the Zip
Central Directory when it's available; the old "streamable" Zip reader
is still fully supported for cases where seeking is not possible.
Full release notes available at:
https://github.com/libarchive/libarchive/wiki/ReleaseNotes
according to POSIX document, the clock ID may be dynamically allocated,
it unlikely will be in 64K forever. To make it future compatible, we
pack all timeout information into a new structure called _umtx_time, and
use fourth argument as a size indication, a zero means it is old code
using timespec as timeout value, but the new structure also includes flags
and a clock ID, so the size argument is different than before, and it is
non-zero. With this change, it is possible that a thread can sleep
on any supported clock, though current kernel code does not have such a
POSIX clock driver system.
UMTX_OP_WAIT. Upper 16bits is enough to hold a clock id, and lower
16bits is used to pass flags. The change saves a clock_gettime() syscall
from libthr.
Advantages:
- Reduces the number of different license versions in the tree
- Eliminates a typo
- Removes some incorrect author attributions due to c/p
- Removes c/p error potential for future pmc manpages
Approved by: jkoshy, gnn, rpaulo, fabient (copyright holders)
MFC after: 1 week
initialize the cache of the system information as it was done for the
dynamic libc. This removes several sysctls from the static binary
startup.
Use the aux vector to fill the single struct dl_phdr_info describing
the static binary itself, to implement dl_iterate_phdr(3) for the
static binaries. [1]
Based on the submission by: John Marino <draco marino st> [1]
Tested by: flo (sparc64)
MFC after: 2 weeks
- Address performance regressions encountered by das@ by caching per-thread
data in TLS where available.
- Add a __NO_TLS flag to cdefs.h to indicate where not available.
- Reorganise the xlocale.h definitions into xlocale/*.h so that they can be
included from multiple places.
- Export the POSIX2008 subset of xlocale when POSIX2008 says it should be
exported, independently of whether xlocale.h is included.
- Fix the bug where programs using ctype functions always assumed ASCII unless
recompiled.
- Fix some style(9) violations.
Reviewed by: brooks (mentor)
Approved by: dim (mentor)
The reasoning behind this, is that if we are consistent in our
documentation about the uint*_t stuff, people will be less tempted to
write new code that uses the non-standard types.
I am not going to bump the man page dates, as these changes can be
considered style nits. The meaning of the man pages is unaffected.
MFC after: 1 month
Ignore the NAT_T extension types so we can at least dump the SADB from
the in-base libipsec/setkey without error when NAT_T support is present
in the kernel, though not printing the additional information yet.
However in case there is no NAT_T support in kernel still consider them
to be an error.
MFC after: 8 weeks
At first, I added a utility called utxrm(8) to remove stale entries from
the user accounting database. It seems there are cases in which we need
to perform different operations on the database as well. Simply rename
utxrm(8) to utx(8) and place the old code under the "rm" command.
In addition to "rm", this tool supports "boot" and "shutdown", which are
going to be used by an rc-script which I am going to commit separately.
If the utmpx database gets updated while an application is reading it,
there is a chance the reading application processes partially
overwritten entries. To solve this, make sure we always read a multiple
of sizeof(struct futx) at a time.
MFC after: 2 weeks
conditional code parts not used by or applicable to FreeBSD.
The new implementation is supposed to be able to cope with changes to
the 'l' versions of the msghdr structs now used as well as to if_data
allowing future changes without breaking things.
This restores carp(4) config support in HEAD after r231504.
Reviewed by: glebius, brooks
MFC After: 3 months
on extended and extensible structs if_msghdrl and ifa_msghdrl. This
will allow us to extend both the msghdrl structs and eventually if_data
in the future without breaking the ABI.
Bump __FreeBSD_version to allow ports to more easily detect the new API.
Reviewed by: glebius, brooks
MFC after: 3 days
