If bread returns an error there is no bp to brelse. One of these
changes was taken from NetBSD commit 0a62dad69f62 ("This works well
enough to populate..."), the rest were found by looking for the same
pattern.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39069
Add missing brelse(bp). Without it the cache grows and we have a n^2
lookup. I'm not entirely sure why we read the block before we write it
back out, since the only side effect of that is to allocate memory,
clear the memory, read it in from disk, throw it away with the contents
of the file being written out. We likely should just do a getblk() here
instead, but even with all that, this takes the time it takes to create
a 150MB msdos fs image down from 5 minutes to 30 seconds.
Old: 317.663u 0.685s 5:18.34 100.0% 198+360k 0+19io 1009pf+0w
New: 7.330u 23.841s 0:31.17 100.0% 198+360k 0+250522io 4pf+0w
See code review for how we got this. tl;dr: netbsd move brelse
into bwrite and we picked up msdos code after that, but not the
move. That change should be picked up later.
Sponsored by: Netflix
Reviewed by: emaste
MFC After: 1 day (13.2 is coming fast)
Differential Revision: https://reviews.freebsd.org/D39025
If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need to exist. Just let openat fail in that case. It's conceivable
this will make debugging some cases weird, but it's sufficent to handle
the way we add /root/.ssh in CheriBSD VM images.
This is a recommit of 794154149f with
bugfixes.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38029
If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need to exist. Just let openat fail in that case. It's conceivable
this will make debugging some cases weird, but it's sufficent to handle
the way we add /root/.ssh in CheriBSD VM images.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38029
When a link target is specified use it rather than attempting to read
a potentially non-existant file.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38028
Commit d7eec79b70 overlooked the fact that
nvlist_find(DATA_TYPE_STRING) does not provide a nul-terminated string.
Fix the leak a different way.
Fixes: d7eec79b70 ("makefs: Plug a memory leak")
nvlist_find_string() would return a copy of the found value, but callers
assumed they would have to make their own copy. It's simpler to change
nvlist_find_string() than it is to change callers, so do that.
Reported by: Coverity
This adds missing includes, uses the standard dirent.h rather than the
BSD-specific sys/dirent.h subset (which works on macOS but not Linux)
and works around Linux's lack of st_birthtim.
This allows usr.sbin/makefs to be added to LOCAL_XTOOL_DIRS again on
macOS and Linux so that disk images can be cross-built.
Reviewed by: markj
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36135
Currently the code copies a struct timespec's raw bits as a pair of
uint64_t. On 64-bit systems this has the same representation, but on
32-bit issues there are two issues:
1. tv_sec is a time_t which is 32-bit on i386 specifically
2. tv_nsec is a long not a 64-bit integer
On i386, this means the assertion should fire as the size doesn't match.
On other 32-bit systems there are 4 bytes of padding after tv_nsec,
which in practice are probably 0, as this data is ultimately coming from
the kernel, so it's deterministic (though the padding bytes are not
required to be preserved by the compiler, so are strictly unspecified).
However, on 32-bit big-endian systems, the padding bytes are in the
wrong half to be harmless, resulting in the nanoseconds being multiplied
by 2^32.
Fix this all by marshalling via a real uint64_t pair like is done by the
real ZFS_TIME_ENCODE.
Reviewed by: markj
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36131
The fsnode tree traversal routines used in ZFS mode assume that all
children of a (directory) fsnode can be accessed using a directory fd
for the parent and the child name. This is true when populating the
image using an mtree manifest or from a single staging directory, but
doesn't work when multiple staging directories are specified.
Change the traversal routines to use absolute path lookups when an mtree
manifest is not in use. This isn't ideal, but it's the simplest way to
fix the problem.
Reported by: imp
Sponsored by: The FreeBSD Foundation
Space used by the MOS is summed after all on-disk data structures are
finalized, so the MOS DSL directory represents a special case when
counting bytes used for each directory. The DSL layer failed to update
the MOS DSL directory's parent, the root directory, accordingly, so the
root directory reported less space used than was used in reality. Be
careful to update the root directory after the MOS is finalized.
PR: 265849
Sponsored by: The FreeBSD Foundation
Check the canmount property before building the mountpoint string.
Reported by: Coverity
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Sponsored by: The FreeBSD Foundation
nvstring_get() returns a copy of the string, not a pointer into the
nvlist's internal buffer.
Reported by: Coverity
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Sponsored by: The FreeBSD Foundation
This allows one to take a staged directory tree and create a file
consisting of a ZFS pool with one or more datasets that contain the
contents of the directory tree. This is useful for creating virtual
machine images without using the kernel to create a pool; "zpool create"
requires root privileges and currently is not permitted in jails.
makefs -t zfs also provides reproducible images by using a fixed seed
for pseudo-random number generation, used for generating GUIDs and hash
salts. makefs -t zfs requires relatively little by way of machine
resources.
The "zpool_reguid" rc.conf setting can be used to ask a FreeBSD guest to
generate a unique pool GUID upon first boot.
A small number of pool and dataset properties are supported. The pool
is backed by a single disk vdev. Data is always checksummed using
Fletcher-4, no redundant copies are made, and no compression is used.
The manual page documents supported pool and filesystem properties.
The implementation uses a few pieces of ZFS support from with the boot
loader, especially definitions for various on-disk structures, but is
otherwise standalone and in particular doesn't depend on OpenZFS.
This feature should be treated as experimental for now, i.e., important
data shouldn't be trusted to a makefs-created pool, and the command-line
interface is subject to change.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35248
When a new FS image is created we need to calculate how much space each
file is going to consume.
Fix two bugs in that logic:
1) Count the space needed for indirect blocks for large files.
1) Normally the trailing data of a file is written to a block of frag
size, 4 kB by default.
However for files that use indirect blocks a full block is allocated,
32kB by default. Take that into account.
Adjust size calculations to match what is done in ffs_mkfs routine:
* Depending on the UFS version the superblock is stored at a different
offset. Take that into account.
* Add the cylinder group block size.
* All of the above has to be aligned to the block size.
Finally, Remove "ncg" variable. It's always 1 and it was used to
multiply stuff.
PR: 229929
Reviewed by: mckusick
MFC after: 2 weeks
Sponsored by: Semihalf
Submitted by: Kornel Dulęba <mindal@semihalf.com>
Differential Revision: https://reviews.freebsd.org/D35131
Differential Revision: https://reviews.freebsd.org/D35132
Leave -Wcast-align disabled, at least for now, since there are numerous
instances of that warning in places where buffer pointers are cast to
pointers to various filesystem structures. Fixing this properly would
be too much work for too little gain.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Change its return type to void, because its result is ignored in both
call sites. Remove oldcnp argument as well, it is NULL always.
Suggested and reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33721
Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.
Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].
Reviewed by: mckusick, jhb
Differential Revision: https://reviews.freebsd.org/D33650
LK_EXCLUSIVE must be passed always, some consumers need the ability to
specify LK_NOWAIT
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31464
Apparently some large-file systems out there, such as my powerpc64le
Linux box, define daddr_t as a 32-bit type, which is sad and stymies
cross-building disk images. Cast daddr_t to off_t before doing
arithmetic that overflows.
Reviewed by: arichardson, jrtc27, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D27458
By default, makefs(8) has very few spare inodes in its output images,
which is fine for static filesystems, but not so great for VM images
where many more files will be added. Make makefs(8) use the same
default settings as newfs(8) when creating images with free space --
there isn't much point to leaving free space on the image if you
can't put files there. If no free space is requested, use current
behavior of a minimal number of available inodes.
Reviewed by: manu
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D29492
An install using -DNO_ROOT emits mtree entries containing tags used by
pkgbase. makefs(8) can safely ignore them, so do that rather than
emitting a warning for each entry.
Reviewed by: brooks, imp
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29384
The makefs msdosfs code includes fs/msdosfs/denode.h which directly uses
struct buf from <sys/buf.h> rather than the makefs struct m_buf.
To work around this problem provide a local denode.h that includes
ffs/buf.h and defines buf as an alias for m_buf.
Reviewed By: kib, emaste
Differential Revision: https://reviews.freebsd.org/D28835
Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.
Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition. Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D28679
The st variable is used as a shortcut for &node->inode->st, but in one
branch just before the exit we update node->inode without changing st.
Reported by: AddressSanitizer
Reviewed By: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28570
The macOS assert.h header does not define static_assert when compiling in
C99 mode. To fix this compile with -std=c11.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D25928
