nvlist_find_string() would return a copy of the found value, but callers
assumed they would have to make their own copy. It's simpler to change
nvlist_find_string() than it is to change callers, so do that.
Reported by: Coverity
This adds missing includes, uses the standard dirent.h rather than the
BSD-specific sys/dirent.h subset (which works on macOS but not Linux)
and works around Linux's lack of st_birthtim.
This allows usr.sbin/makefs to be added to LOCAL_XTOOL_DIRS again on
macOS and Linux so that disk images can be cross-built.
Reviewed by: markj
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36135
Currently the code copies a struct timespec's raw bits as a pair of
uint64_t. On 64-bit systems this has the same representation, but on
32-bit issues there are two issues:
1. tv_sec is a time_t which is 32-bit on i386 specifically
2. tv_nsec is a long not a 64-bit integer
On i386, this means the assertion should fire as the size doesn't match.
On other 32-bit systems there are 4 bytes of padding after tv_nsec,
which in practice are probably 0, as this data is ultimately coming from
the kernel, so it's deterministic (though the padding bytes are not
required to be preserved by the compiler, so are strictly unspecified).
However, on 32-bit big-endian systems, the padding bytes are in the
wrong half to be harmless, resulting in the nanoseconds being multiplied
by 2^32.
Fix this all by marshalling via a real uint64_t pair like is done by the
real ZFS_TIME_ENCODE.
Reviewed by: markj
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36131
The fsnode tree traversal routines used in ZFS mode assume that all
children of a (directory) fsnode can be accessed using a directory fd
for the parent and the child name. This is true when populating the
image using an mtree manifest or from a single staging directory, but
doesn't work when multiple staging directories are specified.
Change the traversal routines to use absolute path lookups when an mtree
manifest is not in use. This isn't ideal, but it's the simplest way to
fix the problem.
Reported by: imp
Sponsored by: The FreeBSD Foundation
Space used by the MOS is summed after all on-disk data structures are
finalized, so the MOS DSL directory represents a special case when
counting bytes used for each directory. The DSL layer failed to update
the MOS DSL directory's parent, the root directory, accordingly, so the
root directory reported less space used than was used in reality. Be
careful to update the root directory after the MOS is finalized.
PR: 265849
Sponsored by: The FreeBSD Foundation
Check the canmount property before building the mountpoint string.
Reported by: Coverity
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Sponsored by: The FreeBSD Foundation
nvstring_get() returns a copy of the string, not a pointer into the
nvlist's internal buffer.
Reported by: Coverity
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Sponsored by: The FreeBSD Foundation
This allows one to take a staged directory tree and create a file
consisting of a ZFS pool with one or more datasets that contain the
contents of the directory tree. This is useful for creating virtual
machine images without using the kernel to create a pool; "zpool create"
requires root privileges and currently is not permitted in jails.
makefs -t zfs also provides reproducible images by using a fixed seed
for pseudo-random number generation, used for generating GUIDs and hash
salts. makefs -t zfs requires relatively little by way of machine
resources.
The "zpool_reguid" rc.conf setting can be used to ask a FreeBSD guest to
generate a unique pool GUID upon first boot.
A small number of pool and dataset properties are supported. The pool
is backed by a single disk vdev. Data is always checksummed using
Fletcher-4, no redundant copies are made, and no compression is used.
The manual page documents supported pool and filesystem properties.
The implementation uses a few pieces of ZFS support from with the boot
loader, especially definitions for various on-disk structures, but is
otherwise standalone and in particular doesn't depend on OpenZFS.
This feature should be treated as experimental for now, i.e., important
data shouldn't be trusted to a makefs-created pool, and the command-line
interface is subject to change.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35248
When a new FS image is created we need to calculate how much space each
file is going to consume.
Fix two bugs in that logic:
1) Count the space needed for indirect blocks for large files.
1) Normally the trailing data of a file is written to a block of frag
size, 4 kB by default.
However for files that use indirect blocks a full block is allocated,
32kB by default. Take that into account.
Adjust size calculations to match what is done in ffs_mkfs routine:
* Depending on the UFS version the superblock is stored at a different
offset. Take that into account.
* Add the cylinder group block size.
* All of the above has to be aligned to the block size.
Finally, Remove "ncg" variable. It's always 1 and it was used to
multiply stuff.
PR: 229929
Reviewed by: mckusick
MFC after: 2 weeks
Sponsored by: Semihalf
Submitted by: Kornel Dulęba <mindal@semihalf.com>
Differential Revision: https://reviews.freebsd.org/D35131
Differential Revision: https://reviews.freebsd.org/D35132
Leave -Wcast-align disabled, at least for now, since there are numerous
instances of that warning in places where buffer pointers are cast to
pointers to various filesystem structures. Fixing this properly would
be too much work for too little gain.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Change its return type to void, because its result is ignored in both
call sites. Remove oldcnp argument as well, it is NULL always.
Suggested and reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33721
Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.
Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].
Reviewed by: mckusick, jhb
Differential Revision: https://reviews.freebsd.org/D33650
LK_EXCLUSIVE must be passed always, some consumers need the ability to
specify LK_NOWAIT
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31464
Apparently some large-file systems out there, such as my powerpc64le
Linux box, define daddr_t as a 32-bit type, which is sad and stymies
cross-building disk images. Cast daddr_t to off_t before doing
arithmetic that overflows.
Reviewed by: arichardson, jrtc27, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D27458
By default, makefs(8) has very few spare inodes in its output images,
which is fine for static filesystems, but not so great for VM images
where many more files will be added. Make makefs(8) use the same
default settings as newfs(8) when creating images with free space --
there isn't much point to leaving free space on the image if you
can't put files there. If no free space is requested, use current
behavior of a minimal number of available inodes.
Reviewed by: manu
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D29492
An install using -DNO_ROOT emits mtree entries containing tags used by
pkgbase. makefs(8) can safely ignore them, so do that rather than
emitting a warning for each entry.
Reviewed by: brooks, imp
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29384
The makefs msdosfs code includes fs/msdosfs/denode.h which directly uses
struct buf from <sys/buf.h> rather than the makefs struct m_buf.
To work around this problem provide a local denode.h that includes
ffs/buf.h and defines buf as an alias for m_buf.
Reviewed By: kib, emaste
Differential Revision: https://reviews.freebsd.org/D28835
Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.
Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition. Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D28679
The st variable is used as a shortcut for &node->inode->st, but in one
branch just before the exit we update node->inode without changing st.
Reported by: AddressSanitizer
Reviewed By: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28570
The macOS assert.h header does not define static_assert when compiling in
C99 mode. To fix this compile with -std=c11.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D25928
The added print was very helpful for debugging failed disk image creation.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D23200
This breaks the build on macOS where this header doesn't exist. I could
also add a compat header to tools/build/cross-build but since it's not
needed removing it seems like the better solution.
In order to do so we need to install the msdosfs headers to the bootstrap
sysroot and avoid includes of kernel headers that may not exist on every
host (e.g. sys/lockmgr.h). This change should allow bootstrapping of makefs
on FreeBSD 11+ as well as Linux and macOS.
We also have to avoid using the IO_SYNC macro since that may not be
available. In makefs it is only used to switch between calling
bwrite() and bdwrite() which both call the same function. Therefore we
can simply always call bwrite().
For our CheriBSD builds we always bootstrap makefs by setting
LOCAL_XTOOL_DIRS='lib/libnetbsd usr.sbin/makefs' and use the makefs binary
from the build tree to create a bootable disk image.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D23201
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22494
Since st_birthtime doesn't exists on Linux (unless you use statx(2)), we
instead populate it with the st_ctime value.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D22386
Setting the B_INVALONERR flag before a synchronous write causes the buf
cache to forcibly invalidate contents if the write fails (BIO_ERROR).
This is intended to be used to allow layers above the buffer cache to make
more informed decisions about when discarding dirty buffers without
successful write is acceptable.
As a proof of concept, use in msdosfs to handle failures to mark the on-disk
'dirty' bit during rw mount or ro->rw update.
Extending this to other filesystems is left as future work.
PR: 210316
Reviewed by: kib (with objections)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D21539
