- Move CSRG IDs into __SCCSID().
- When a file has been copied, consistently use 'From: <tag>' for strings
referencing the version of the source file copied from in the license
block comment.
- Some of the 'From:' tags were using $FreeBSD$ that was being expanded on
each checkout. Fix those to hardcode the FreeBSD tag from the file that
was copied at the time of the copy.
- When multiple strings are present list them in "chronological" order,
so CSRG (__SCCSID) before FreeBSD (__FBSDID). If a file came from
OtherBSD and contains a CSRG ID from the OtherBSD file, use the order
CSRG -> OtherBSD -> FreeBSD.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D15831
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
Move an unneeded initialization, introduced in r54770 to quiet down GCC,
to a place nearer to its first use. This has no practical effect, it just
keeps the garbage better sorted.
Hinted by: OpenBSD (CVS rev. 1.56, without obfuscations)
Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment. Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.
ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks. Unfortunately, not everything can be
fixed, especially outside the base system. For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.
Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.
Struct xvnode changed layout, no compat shims are provided.
For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.
Update note: strictly follow the instructions in UPDATING. Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.
Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver. Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).
Sponsored by: The FreeBSD Foundation (emaste, kib)
Differential revision: https://reviews.freebsd.org/D10439
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
Because fts keeps internal file descriptors open across calls, making such
descriptors close-on-exec helps not only multi-threaded applications but
also single-threaded applications.
In particular, this prevents passing a temporary file descriptor for saving
the current directory to processes created via find -exec.
Introduce dirfd() libc exported symbol replacing macro with same name,
preserve _dirfd() macro for internal use.
Replace dirp->dd_fd with dirfd() call. Avoid using dirfd as variable
name to prevent shadowing global symbol.
Sponsored by: Google Summer Of Code 2011
The "FTS_NOSTAT" option can avoid a lot of calls to stat(2) if it knows that a
directory could not possibly have subdirectories. This is decided by looking at
the link count: a subdirectory would increment its parent's link count by
virtue of its own ".." entry. This assumption only holds for UFS-like
filesystems that implement links and directories this way, so we must punt for
others.
It looks like ZFS is a UFS-like file system, as the above also holds for ZFS.
Add ZFS to the list of file systems that allow for such optimization.
MFC after: 1 month
fields in FTS and FTSENT structs being too narrow. In addition,
the narrow types creep from there into fts.c. As a result, fts(3)
consumers, e.g., find(1) or rm(1), can't handle file trees an ordinary
user can create, which can have security implications.
To fix the historic implementation of fts(3), OpenBSD and NetBSD
have already changed <fts.h> in somewhat incompatible ways, so we
are free to do so, too. This change is a superset of changes from
the other BSDs with a few more improvements. It doesn't touch
fts(3) functionality; it just extends integer types used by it to
match modern reality and the C standard.
Here are its points:
o For C object sizes, use size_t unless it's 100% certain that
the object will be really small. (Note that fts(3) can construct
pathnames _much_ longer than PATH_MAX for its consumers.)
o Avoid the short types because on modern platforms using them
results in larger and slower code. Change shorts to ints as
follows:
- For variables than count simple, limited things like states,
use plain vanilla `int' as it's the type of choice in C.
- For a limited number of bit flags use `unsigned' because signed
bit-wise operations are implementation-defined, i.e., unportable,
in C.
o For things that should be at least 64 bits wide, use long long
and not int64_t, as the latter is an optional type. See
FTSENT.fts_number aka FTS.fts_bignum. Extending fts_number `to
satisfy future needs' is pointless because there is fts_pointer,
which can be used to link to arbitrary data from an FTSENT.
However, there already are fts(3) consumers that require fts_number,
or fts_bignum, have at least 64 bits in it, so we must allow for them.
o For the tree depth, use `long'. This is a trade-off between making
this field too wide and allowing for 64-bit inode numbers and/or
chain-mounted filesystems. On the one hand, `long' is almost
enough for 32-bit filesystems on a 32-bit platform (our ino_t is
uint32_t now). On the other hand, platforms with a 64-bit (or
wider) `long' will be ready for 64-bit inode numbers, as well as
for several 32-bit filesystems mounted one under another. Note
that fts_level has to be signed because -1 is a magic value for it,
FTS_ROOTPARENTLEVEL.
o For the `nlinks' local var in fts_build(), use `long'. The logic
in fts_build() requires that `nlinks' be signed, but our nlink_t
currently is uint16_t. Therefore let's make the signed var wide
enough to be able to represent 2^16-1 in pure C99, and even 2^32-1
on a 64-bit platform. Perhaps the logic should be changed just
to use nlink_t, but it can be done later w/o breaking fts(3) ABI
any more because `nlinks' is just a local var.
This commit also inludes supporting stuff for the fts change:
o Preserve the old versions of fts(3) functions through libc symbol
versioning because the old versions appeared in all our former releases.
o Bump __FreeBSD_version just in case. There is a small chance that
some ill-written 3-rd party apps may fail to build or work correctly
if compiled after this change.
o Update the fts(3) manpage accordingly. In particular, remove
references to fts_bignum, which was a FreeBSD-specific hack to work
around the too narrow types of FTSENT members. Now fts_number is
at least 64 bits wide (long long) and fts_bignum is an undocumented
alias for fts_number kept around for compatibility reasons. According
to Google Code Search, the only big consumers of fts_bignum are in
our own source tree, so they can be fixed easily to use fts_number.
o Mention the change in src/UPDATING.
PR: bin/104458
Approved by: re (quite a while ago)
Discussed with: deischen (the symbol versioning part)
Reviewed by: -arch (mostly silence); das (generally OK, but we didn't
agree on some types used; assuming that no objections on
-arch let me to stick to my opinion)
permission), try to continue in FTS_DONTCHDIR mode. Of course this
won't work for long paths, but we can't descend more than one pathname
component beyond the directory anyway if we lack search permission.
Here is a transcript demonstrating the change, where oldls is ls(1)
linked with the old fts(3):
das@VARK:~> mkdir t && touch t/{a,b,c} && chmod u-x t
das@VARK:~> oldls t
a b c
das@VARK:~> oldls -l t
das@VARK:~> \ls t
a b c
das@VARK:~> \ls -l t
ls: a: Permission denied
ls: b: Permission denied
ls: c: Permission denied
I had forgotten about this patch until bde reminded me. He reports
using it without problems for over a year.
PR: 45723
Remove "sys/types.h" as "sys/param.h" is already included
Use cast rather than back-pointer to convert from public to private
version of FTS data, and so avoid littering fts.h with any of the
details.
Pointed out By: bde, kientzle
of stat(2) calls by keeping an eye of the number of links a directory
has. It assumes that each subdirectory will have a hard link to its
parent, to represent the ".." node, and stops calling stat(2) when
all links are accounted for in a given directory.
This assumption is really only valid for UNIX-like filesystems: A
concrete example is NTFS. The NTFS "i-node" does contain a link
count, but most/all directories have a link count between 0 and 2
inclusive. The end result is that find on an NTFS volume won't
actually traverse the entire hierarchy of the directories passed
to it. (Those with a link count of two are not traversed at all)
The fix checks the "UFSness" of the filesystem before enabling the
optimisation.
Reviewed By: Tim Kientzle (kientzle@)
hack, thereby allowing future extensions to the structure (e.g., for extended
attributes) without rebreaking the ABI. FTSENT now contains a pointer to the
parent stream, which fts_compar() can then take advantage of, avoiding the
undefined behavior previously warned about. As a consequence of this change,
the prototype of the comparison function passed to fts_open() has changed
to reflect the required amount of constness for its use. All callers in the
tree are updated to use the correct prototype.
Comparison functions can now make use of the new parent pointer to access
the new stream-specific private data pointer, which is intended to assist
creation of reentrant library routines which use fts(3) internally.
Not objected to in spirit by: -arch
there and compare the inode and device numbers to the values we remember,
to guard against the directory having been moved around in the meantime.
Reported by: Nick Cleaton <nick@cleaton.net>
adding (weak definitions to) stubs for some of the pthread
functions. If the threads library is linked in, the real
pthread functions will pulled in.
Use the following convention for system calls wrapped by the
threads library:
__sys_foo - actual system call
_foo - weak definition to __sys_foo
foo - weak definition to __sys_foo
Change all libc uses of system calls wrapped by the threads
library from foo to _foo. In order to define the prototypes
for _foo(), we introduce namespace.h and un-namespace.h
(suggested by bde). All files that need to reference these
system calls, should include namespace.h before any standard
includes, then include un-namespace.h after the standard
includes and before any local includes. <db.h> is an exception
and shouldn't be included in between namespace.h and
un-namespace.h namespace.h will define foo to _foo, and
un-namespace.h will undefine foo.
Try to eliminate some of the recursive calls to MT-safe
functions in libc/stdio in preparation for adding a mutex
to FILE. We have recursive mutexes, but would like to avoid
using them if possible.
Remove uneeded includes of <errno.h> from a few files.
Add $FreeBSD$ to a few files in order to pass commitprep.
Approved by: -arch
just use _foo() <-- foo(). In the case of a libpthread that doesn't do
call conversion (such as linuxthreads and our upcoming libpthread), this
is adequate. In the case of libc_r, we still need three names, which are
now _thread_sys_foo() <-- _foo() <-- foo().
Convert all internal libc usage of: aio_suspend(), close(), fsync(), msync(),
nanosleep(), open(), fcntl(), read(), and write() to _foo() instead of foo().
Remove all internal libc usage of: creat(), pause(), sleep(), system(),
tcdrain(), wait(), and waitpid().
Make thread cancellation fully POSIX-compliant.
Suggested by: deischen
points. For library functions, the pattern is __sleep() <--
_libc_sleep() <-- sleep(). The arrows represent weak aliases. For
system calls, the pattern is _read() <-- _libc_read() <-- read().
patch to stop the core dumps while others come up with a better
reviewed patch which may also fix other problems. We do illegal
pointer arithmetic, but it should be OK since FreeBSD only supports
machines with flat address spaces.
Submitted by: bde
When fts_open is used with option FTS_NOCHDIR the full
path entry of type FTS_DP is returned with a trailing
'/' if the final directory is empty.
This fix coresponds to netbsd's __fts13.c v. 1.16
In some cases replace if (a == null) a = malloc(x); else a =
realloc(a, x); with simple reallocf(a, x). Per ANSI-C, this is
guaranteed to be the same thing.
I've been running these on my system here w/o ill effects for some
time. However, the CTM-express is at part 6 of 34 for the CAM
changes, so I've not been able to do a build world with the CAM in the
tree with these changes. Shouldn't impact anything, but...
tree. Also merge in fix to NetBSD PR #1495. These represent 1.3-1.9 in
the OpenBSD tree. Make minor KNF changes to new code (which is in the
OpenBSD as 1.10). This avoids the symlink race problems.
These patches should go into 2.2.5 before the ship if they don't
break anything in -current.
Reviewed by: Bruce Evans
Obtained from: OpenBSD
