with the user's namespace.
- Correct size and position variables type from long to size_t.
- Do not set errno to ENOMEM on malloc failure, as malloc already does so.
- Implement the concept of "buffer data length", which mandates what SEEK_END
refers to and the allowed extent for a read.
- Use NULL as read-callback if the buffer is opened in write-only mode.
Conversely, use NULL as write-callback when opened in read-only mode.
- Implement the handling of the ``b'' character in the mode argument. A binary
buffer differs from a text buffer (default mode if ``b'' is omitted) in that
NULL bytes are never appended to writes and that the "buffer data length"
equals to the size of the buffer.
- Remove shall from the man page. Use indicative instead. Also, specify that
the ``b'' flag does not conform with POSIX but is supported by glibc.
- Update the regression test so that the ``b'' functionality and the "buffer
data length" concepts are tested.
- Minor style(9) corrections.
Suggested by: jilles
Reviewed by: cognet
Approved by: cognet
This commit adds a new mode option 'e' that must follow any 'b', '+' and/or
'x' options. C11 is clear about the 'x' needing to follow 'b' and/or '+' and
that is what we implement; therefore, require a strict position for 'e' as
well.
For freopen() with a non-NULL path argument and fopen(), the close-on-exec
flag is set iff the 'e' mode option is specified. For freopen() with a NULL
path argument and fdopen(), the close-on-exec flag is turned on if the 'e'
mode option is specified and remains unchanged otherwise.
Although the same behaviour for fopen() can be obtained by open(O_CLOEXEC)
and fdopen(), this needlessly complicates the calling code.
Apart from the ordering requirement, the new option matches glibc.
PR: kern/169320
this fflush may fail to write data in the buffer.
PR: kern/137819
Submitted by: Eric Blake <ebb9@byu.net>
Reviewed by: theraven
Approved by: cperciva
MFC after: 2 weeks
The typical case was:
static __inline int
convert_ccl(FILE *fp, char * __restrict p, [...])
{
[...]
if (p == SUPPRESS_PTR) {
[...]
} else {
[...]
}
[...]
}
This qualifier says that the pointer is the only one at that time
pointing to the resource.
Here, clang considers that "p" will never match "SUPPRESS_PTR" and
optimize the if{} block out. This leads to segfaults in programs calling
vfscanf(3) and vfwscanf(3) with just the format string (no arguments
following it).
The following softwares were reported to abort with segmentation fault
and this patch fixes it:
o cmake
o smartd
o devel/ORBit2
dim@ opened an LLVM PR to discuss this clang optimization:
http://llvm.org/bugs/show_bug.cgi?id=12656
Tested by: bsam@
conversions. Both the specification and the documentation say the
width is interpreted as the max number of wide characters to read, but
the implementation was interpreting it as the number of bytes to convert.
(See also r105317.)
This change has security implications for any applications that depend
on the buggy behavior, but the impact in practice is probably nil.
Any such application would already be buggy on other platforms that
get the semantics right. Also, these conversions are rarely used;
%ls, %lc, and %l[ are more appropriate.
reading wide characters manually. With this change, they now use
fgetwc(). To make this work, we use an internal version of fgetwc()
with a few extensions: it takes an mbstate * because non-wide streams
don't have a built-in mbstate, and it indicates the number of bytes
read.
vfscanf() now resembles vfwscanf() more closely. Minor functional
improvements include working xlocale support in vfscanf(), setting the
stream error indicator on encoding errors, and proper handling of
shift-based encodings. (Actually, making shift-based encodings work
with non-wide streams is hopeless, but the implementation now matches
the broken specification.)
This tool changes the default buffering behaviour of standard
stdio streams.
It only works on dynamic binaries. To make it work for static
ones it would require cluttering stdio because there no single
entry point.
PR: 166660
Reviewed by: current@, jhb
Approved by: kib (mentor)
MFC after: 1 week
indicates the avaliability of FILE, to prevent possible reordering of
the writes as seen by other CPUs.
Reported by: Fengwei yin <yfw.bsd gmail com>
Reviewed by: jhb
MFC after: 1 week
by separate conversion functions. This will hopefully make bugs more
noticeable (I noticed several already) and provide opportunities to
reduce code duplication.
(i.e., the return value would overflow), set errno to EOVERFLOW
and return an error. This improves the chances that buggy
applications -- for instance, ones that pass in a negative integer
as the size due to a bogus calculation -- will fail in safe ways.
Returning an error in these situations is specified by POSIX, but
POSIX appears to have an off-by-one error that isn't duplicated in
this change.
Previously, some of these functions would silently cap the size at
INT_MAX+1, and others would exit with an error after writing more
than INT_MAX characters.
PR: 39256
MFC after: 2 weeks
true if the size is zero.
- Fix a claim that sprintf() is the same as snprintf() with an
infinite size. It's equivalent to snprintf() with a size of
INT_MAX + 1.
- Document the return values in the return values section.
- Document the possible errno value of EOVERFLOW.
MFC after: 2 weeks
infinite loop pretty much unconditionally. It's remarkable that the
patch that introduced the bug was never tested, but even more
remarkable that nobody noticed for over two years.
PR: 167039
MFC after: 3 days
load of _l suffixed versions of various standard library functions that use
the global locale, making them take an explicit locale parameter. Also
adds support for per-thread locales. This work was funded by the FreeBSD
Foundation.
Please test any code you have that uses the C standard locale functions!
Reviewed by: das (gdtoa changes)
Approved by: dim (mentor)
- If precision is 0, don't print period followed by no digits.
- If precision is 0 stop printing units as soon as possible
(eg. if we have three years and five days and precision is 0
print only 3y5d).
- If precision is not 0, print all units (eg. 3y0d0h0m0s.00).
MFC after: 2 weeks
the existing file descriptor. Instead, let dup2() atomically close the
old file descriptor when assigning the newly opened file to the same
descriptor. This closes a race in a multithreaded application where a
concurrent open() could allocate the existing file descriptor in between
the calls to close() and dup2().
PR: threads/79887
Submitted by: Dmitrij Tejblum tejblum of yandex-team.ru
Reviewed by: davidxu
MFC after: 1 week
bottom of the manpages and order them consistently.
GNU groff doesn't care about the ordering, and doesn't even mention
CAVEATS and SECURITY CONSIDERATIONS as common sections and where to put
them.
Found by: mdocml lint run
Reviewed by: ru
by *sprintf(), etc.
- Explicitly initialize _fl_mutex to PTHREAD_MUTEX_INITIALIZER for all FILE
objects. This is currently a nop on FreeBSD, but is import for other
platforms (or in the future) where PTHREAD_MUTEX_INITIALIZER is not simply
zero.
PR: threads/141198
Reported by: Jeremy Huddleston @ Apple
MFC after: 2 weeks
Prior to this commit, fread/fwrite calls with size * nmemb > SIZE_MAX
were handled by reading or writing (size_t)(size * nmemb) bytes; for
example, on 32-bit platforms, fread(ptr, 641, 6700417, f) would read 1
byte and indicate that the requested 6700417 blocks had been read.
This commit adds a check for such integer overflows, and treats them as
if an overly large request was passed to read/write; i.e., it sets errno
to EINVAL, sets the error indicator on the file, and returns a short
object count (0, to be specific).
The overflow check involves an integer division, so as a performance
optimization we check first to see if both size and nmemb are less than
2^16; if they are, no overflow is possible and we avoid the division.
We assume here that size_t is at least 32 bits; this appears to be true
on all platforms FreeBSD supports.
Although this commit fixes an integer overflow, it is not likely to have
any security implications, since any program which would be affected by
this bug fix is quite clearly already very confused.
Reviewed by: kib
MFC after: 1 month
and moving the default initialization of prec into the else clause.
The clang static analyzer erroneously thought that nsec can be used
uninitialized here; it was not actually possible, but better to make
the code clearer. (Clang can't know that sprintf() won't modify *pi
behind the scenes.)
uninitialized. Initialize it to a safe value so that there's no
chance of returning an error if stack garbage happens to be equal to
(size_t)-1 or (size_t)-2.
Found by: Clang static analyzer
MFC after: 7 days
- Tolerate applications that pass a NULL pointer for the buffer and
claim that the capacity of the buffer is nonzero.
- If an application passes in a non-NULL buffer pointer and claims the
buffer has zero capacity, we should free (well, realloc) it
anyway. It could have been obtained from malloc(0), so failing to
free it would be a small memory leak.
MFC After: 2 weeks
Reported by: naddy
PR: ports/138320
Right now nmemb is returned when size is 0. In newer versions of the
standards, it is explicitly required that fwrite() should return 0.
Submitted by: Christoph Mallon
Approved by: re (kib)
because it means getdelim() returns -1 for both error and EOF, and
never returns 0. However, this is what the original GNU implementation
does, and POSIX inherited the bug.
Reported by: marcus@
dprintf() is a simple wrapper around another function, so we may as
well implement it. But also like getline(), we can't prototype it by
default right now because it would break too many ports.
wcscasecmp(), and wcsncasecmp().
- Make some previously non-standard extensions visible
if POSIX_VISIBLE >= 200809.
- Use restrict qualifiers in stpcpy().
- Declare off_t and size_t in stdio.h.
- Bump __FreeBSD_version in case the new symbols (particularly
getline()) cause issues with ports.
Reviewed by: standards@
The integer thousands' separator code is rewritten in order to
avoid having to preallocate a buffer for the largest possible
digit string with the most possible instances of the longest
possible multibyte thousands' separator. The new version inserts
thousands' separators for integers using the same code as floating point.
sets up a fake buffered FILE and then effectively calls itself
recursively. Unfortunately, gcc doesn't know how to do tail call
elimination in this case, and actually makes things worse by
inlining __sbprintf(). This means that f[w]printf() to stderr was
allocating about 5k of stack on 64-bit platforms, much of which was
never used.
I've reorganized things to eliminate the waste. In addition to saving
some stack space, this improves performance in my tests by anywhere
from 5% to 17% (depending on the test) when -fstack-protector is
enabled. I found no statistically significant performance difference
when stack protection is turned off. (The tests redirected stderr to
/dev/null.)
to get rid of restrict qualifier discarding. This lets libc compile
cleanly in gnu99 mode.
Suggested by: kib, christoph.mallon at gmx.de
Approved by: kib (mentor)
slightly less evil inline functions, and move the buffering state into
a struct. This will make it possible for helper routines to produce
output for printf() directly, making it possible to untangle the code
somewhat.
In wprintf(), use the same buffering mechanism to reduce diffs to
printf(). This has the side-effect of causing wprintf() to catch write
errors that it previously ignored.
mkstemps(), and mkdtemp().
- Add proper range checking for the 'slen' parameter passed to mkstemps().
- Try all possible permutations of a template if a collision is encountered.
Previously, once a single template character reached 'z', it would not wrap
around to '0' and keep going until it encountered the original starting
letter. In the edge case that the randomly generated starting name used
all 'z' characters, only that single name would be tried before giving up.
PR: standards/66531
Submitted by: Jim Luther
Obtained from: Apple
MFC after: 1 week
by moving the positional argument handling code to a new file,
printf-pos.c, and moving common definitions to printflocal.h.
No functional change intended.
In particular, encapsulate the state of the type table in a struct,
and add inline functions to initialize, free, and manipulate that
state. This replaces some ugly macros that made proper error handling
impossible.
While here, remove an unneeded test for NULL and a variable that is
initialized (many times!) but never used. The compiler didn't catch
these because of rampant use of the same variable to mean different
things in different places.
This commit should not cause any changes in functionality.
accessor functions for its benefit now thaat FILE is opaque.
I'm sure there's a better way. I leave that for people to work
on in a src tree that isn't broken.
Pointy hat: jhb
move the definition of the type backing FILE (struct __sFILE) into an
internal header.
- Remove macros to inline certain operations from stdio.h. Applications
will now always call the functions instead.
- Move the various foo_unlocked() functions from unlocked.c into foo.c.
This lets some of the inlining macros (e.g. __sfeof()) move into
foo.c.
- Update a few comments.
- struct __sFILE can now go back to using mbstate_t, pthread_t, and
pthread_mutex_t instead of knowing about their private, backing types.
MFC after: 1 month
Reviewed by: kan
__sFILE. This was supposed to be done in 6.0. Some notes:
- Where possible I restored the various lines to their pre-__sFILEX state.
- Retire INITEXTRA() and just initialize the wchar bits (orientation and
mbstate) explicitly instead. The various places that used INITEXTRA
didn't need the locking fields or _up initialized. (Some places needed
_up to exist and not be off the end of a NULL or garbage pointer, but
they didn't require it to be initialized to a specific value.)
- For now, stdio.h "knows" that pthread_t is a 'struct pthread *' to
avoid namespace pollution of including all the pthread types in stdio.h.
Once we remove all the inlines and make __sFILE private it can go back
to using pthread_t, etc.
- This does not remove any of the inlines currently and does not change
any of the public ABI of 'FILE'.
MFC after: 1 month
Reviewed by: peter
them. Thus, any fd whose value is greater than SHRT_MAX is handled
incorrectly (the short value is sign-extended when converted to an int).
An unpleasant side effect is that if fopen() opens a file and gets a
backing fd that is greater than SHRT_MAX, fclose() will fail and the file
descriptor will be leaked. Better handle this by fixing fopen(), fdopen(),
and freopen() to fail attempts to use a fd greater than SHRT_MAX with
EMFILE.
At some point in the future we should look at expanding the file descriptor
in FILE to an int, but that is a bit complicated due to ABI issues.
MFC after: 1 week
Discussed on: arch
Reviewed by: wollman
Remove the const qualifier from ap argument for __v2printf, that induced
that breakage, and seems to be the real reason for bad code. ap is modified
inside the __v2printf body by va_arg macro.
Pointy hat to: kib
Approved by: pjd (mentor)
i386 with default optimization level (-O2), va_list pointer ap in the
__v2printf function is advanced before the use. That cause argument
shift and garbage instead last argument in printf-family when xprintf is
activated.
The nsswitch is easy victim of the bug.
Reviewed by: kan
Approved by: kan (mentor)
MFC after: 1 week
Issue __sflush() before possible setting O_APPEND mode or ftruncate(),
write to wrong place may occurse oserwise.
Use simplified _sseek() to the start, if no O_APPEND is set, instead
of _fseeko() (_sseek() to the end, if O_APPEND, occurse later, as for
file != NULL).
Don't check seek error return, as original fopen() and freopen() never
does.
file != NULL:
Add missing _sseek() to the end.
FBSDprivate locale symbols. These functions are needed by
libcompat.
Add _cleanup to the list of stdio FBSDprivate symbols. Some
third party applications use this. This will be removed and
replaced by fcloseall() once libc version is bumped.
Add _res to the list of resolv symbols.
Found by: portbuilder runs (thanks Kris!)
Add %M{essage} extension which prints an errno value as the
corresponding string if possible or numerically otherwise.
It is not currently possible to do the syslog(3) like %m extension
because errno would need to get capatured on entry to the first
function in the printf family, so %M requires you to supply errno
as an argument.
Add %Q{uote} extension which will print a string in double quotes with
appropriate back-slash escapes (only) if necessary.
on probationary terms: it may go away again if it transpires it is
a bad idea.
This extensible printf version will only be used if either
environment variable USE_XPRINTF is defined
or
one of the extension functions are called.
or
the global variable __use_xprintf is set greater than zero.
In all other cases our traditional printf implementation will
be used.
The extensible version is slower than the default printf, mostly
because less opportunity for combining I/O operation exists when
faced with extensions. The default printf on the other hand
is a bad case of spaghetti code.
The extension API has a GLIBC compatible part and a FreeBSD version
of same. The FreeBSD version exists because the GLIBC version may
run afoul of our FILE * locking in multithreaded programs and it
even further eliminate the opportunities for combining I/O operations.
Include three demo extensions which can be enabled if desired: time
(%T), hexdump (%H) and strvis (%V).
%T can format time_t (%T), struct timeval (%lT) and struct timespec (%llT)
in one of two human readable duration formats:
"%.3llT" -> "20349.245"
"%#.3llT" -> "5h39m9.245"
%H will hexdump a sequence of bytes and takes a pointer and a length
argument. The width specifies number of bytes per line.
"%4H" -> "65 72 20 65"
"%+4H" -> "0000 65 72 20 65"
"%#4H" -> "65 72 20 65 |er e|"
"%+#4H" -> "0000 65 72 20 65 |er e|"
%V will dump a string in strvis format.
"%V" -> "Hello\tWor\377ld" (C-style)
"%0V" -> "Hello\011Wor\377ld" (octal)
"%+V" -> "Hello%09Wor%FFld" (http-style)
Tests, comments, bugreports etc are most welcome.
allocate a memory block. sscanf calls __svfscanf which in turn calls
fread, fread triggers mutex initialization but the mutex is not
destroyed in sscanf, this leads to memory leak. To avoid the memory
leak and performance issue, we create a none MT-safe version of fread:
__fread, and instead let __svfscanf call __fread.
PR: threads/90392
Patch submitted by: dhartmei
MFC after: 7 days
For some denormalized long double values, a bug in __hldtoa() (called
from *printf()'s %A format) results in a base 16 digit being rounded
up from 0xf to 0x10.
When this digit is subsequently converted to string format, an index
of 10 reaches past the end of the uppper-case hex/char array, picking
up whatever the code segment happen to contain at that address.
This mostly seem to be some character from the upper half of the
byte range.
When using the %a format instead of %A, the first character past
the end of the lowercase hex/char table happens to be index 0 in
the uppercase hex/char table hextable and therefore the string
representation features a '0', which is supposedly correct.
This leads me to belive that the proper fix _may_ be as simple as
masking all but the lower four bits off after incrementing a hex-digit
in libc/gdtoa/_hdtoa.c:roundup(). I worry however that the upper
bit in 0x10 indicates a carry not carried.
Until das@ or bde@ finds time to visit this issue, extend the
hexdigit arrays with a 17th index containing '?' so that we get a
invalid but consistent and printable output in both %a and %A formats
whenever this bug strikes.
This unmasks the bug in the %a format therefore solving the real
issue may both become easier and more urgent.
Possibly related to: PR 85080
With help by: bde@
(((truncate to zero length) or (create)) (text file)) (for writing)
and not
((truncate file to zero length) or (create text file)) (for writing)
MFC after: 1 week
convenient when the source string isn't null-terminated.
Implement the other conversion functions (mbstowcs(), mbsrtowcs(), wcstombs(),
wcsrtombs()) in terms of these new functions.
writable. Affected callers include fwrite(), put?(), and *printf().
The issue of whether this is the right errno for funopened streams is
unresolved, but that's an obscure case, and some errno is better than
no errno.
Discussed with: bde, jkh
through byte by byte with mbrtowc(). In the usual case (buffer is big
enough to contain the multibyte character, character does not straddle
buffer boundary) this results in only one call to mbrtowc() for each
wide character read.
to the initial state when a stream is opened or seeked upon. Use the
stream's conversion state object instead of a freshly-zeroed one in
fgetwc(), fputwc() and ungetwc().
This is only a performance improvement for now, but it would also be
required in order to support state-dependent encodings.
floating-point support, remove default definition of FLOATING_POINT
from the source, and change the compile-time option to
NO_FLOATING_POINT.
- Remove the HEXFLOAT option. It saves an insignificant amount of
space (<0.1% of the size of libc on i386) and complicates vfprintf()
and checkfmt().
after their change from an array of char to an array of enum.
This fixes problems that occurred when using positional arguments in
format strings, particularly with more than STATIC_ARG_TBL_SIZE (8)
of them.
PR: 65841
Submitted by: Steven Smith (mostly)
ferror(), fileno() and clearerr(), using the value of __isthreaded to
decide between the fast inline single-threaded code and the more
general function equivalent. This gives most of the performance
benefits of the old unsafe macros while preserving thread safety.
