getopt(3) returns '?' when it encounters a flag not present in the in
the optstring or if a flag is missing its option argument. We can
handle this case with the "default" failure case with no loss of
legibility.
Obtained from: OpenBSD makefs.c 1.22
We previously attempted to emit Rock Ridge NM records only when the name
represented by the Rock Ridge extensions would actually differ. We would
omit the record for an all-upper-case directory name, however Linux (and
perhaps other operating systems) map names with no NM record to
lowercase.
This affected only directories, as file names have an implicit ";1"
version number appended and thus always differ. To solve, just emit NM
records for all entries other than DOT and DOTDOT .
We could continue to omit the NM record for directories that would avoid
mapping (for example, one named 1234.567) but this does not seem worth
the complexity.
PR: 203531
Reported by: Thomas Schmitt <scdbackup@gmx.net
Reviewed by: kevans
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39258
The boot catalog pointer is a DWord, but we previously populated it via
cd9660_bothendian_dword which overwrote four unused bytes following it.
See El Torito 1.0 (1995) Figure 7 for details.
PR: 203531
Reported by: Coverity Scan
Reported by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed by: kevans
CID: 977470
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39231
These are not actually used by makefs (yet), but ought to match the
spec.
See RRIP 4.1.4 Description of the "NM" System Use Entry for details.
PR: 203531
Sponsored by: The FreeBSD Foundation
The bit definitions for the TF_* timestamp bits (TF_MODIFY, etc.) were
incorrect, and timestamps were written in the wrong order.
See RRIP 4.1.6 Description of the "TF" System Use Entry for details.
PR: 203531
Reported by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed by: jrtc27, kevans
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39221
If bread returns an error there is no bp to brelse. One of these
changes was taken from NetBSD commit 0a62dad69f62 ("This works well
enough to populate..."), the rest were found by looking for the same
pattern.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39069
Add missing brelse(bp). Without it the cache grows and we have a n^2
lookup. I'm not entirely sure why we read the block before we write it
back out, since the only side effect of that is to allocate memory,
clear the memory, read it in from disk, throw it away with the contents
of the file being written out. We likely should just do a getblk() here
instead, but even with all that, this takes the time it takes to create
a 150MB msdos fs image down from 5 minutes to 30 seconds.
Old: 317.663u 0.685s 5:18.34 100.0% 198+360k 0+19io 1009pf+0w
New: 7.330u 23.841s 0:31.17 100.0% 198+360k 0+250522io 4pf+0w
See code review for how we got this. tl;dr: netbsd move brelse
into bwrite and we picked up msdos code after that, but not the
move. That change should be picked up later.
Sponsored by: Netflix
Reviewed by: emaste
MFC After: 1 day (13.2 is coming fast)
Differential Revision: https://reviews.freebsd.org/D39025
If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need to exist. Just let openat fail in that case. It's conceivable
this will make debugging some cases weird, but it's sufficent to handle
the way we add /root/.ssh in CheriBSD VM images.
This is a recommit of 794154149f with
bugfixes.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38029
If a type=dir entry exists and all contents are directories, files
added with contents=, or symlinks with link= attributes then it doesn't
need to exist. Just let openat fail in that case. It's conceivable
this will make debugging some cases weird, but it's sufficent to handle
the way we add /root/.ssh in CheriBSD VM images.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38029
When a link target is specified use it rather than attempting to read
a potentially non-existant file.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D38028
Commit d7eec79b70 overlooked the fact that
nvlist_find(DATA_TYPE_STRING) does not provide a nul-terminated string.
Fix the leak a different way.
Fixes: d7eec79b70 ("makefs: Plug a memory leak")
nvlist_find_string() would return a copy of the found value, but callers
assumed they would have to make their own copy. It's simpler to change
nvlist_find_string() than it is to change callers, so do that.
Reported by: Coverity
This adds missing includes, uses the standard dirent.h rather than the
BSD-specific sys/dirent.h subset (which works on macOS but not Linux)
and works around Linux's lack of st_birthtim.
This allows usr.sbin/makefs to be added to LOCAL_XTOOL_DIRS again on
macOS and Linux so that disk images can be cross-built.
Reviewed by: markj
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36135
Currently the code copies a struct timespec's raw bits as a pair of
uint64_t. On 64-bit systems this has the same representation, but on
32-bit issues there are two issues:
1. tv_sec is a time_t which is 32-bit on i386 specifically
2. tv_nsec is a long not a 64-bit integer
On i386, this means the assertion should fire as the size doesn't match.
On other 32-bit systems there are 4 bytes of padding after tv_nsec,
which in practice are probably 0, as this data is ultimately coming from
the kernel, so it's deterministic (though the padding bytes are not
required to be preserved by the compiler, so are strictly unspecified).
However, on 32-bit big-endian systems, the padding bytes are in the
wrong half to be harmless, resulting in the nanoseconds being multiplied
by 2^32.
Fix this all by marshalling via a real uint64_t pair like is done by the
real ZFS_TIME_ENCODE.
Reviewed by: markj
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Differential Revision: https://reviews.freebsd.org/D36131
The fsnode tree traversal routines used in ZFS mode assume that all
children of a (directory) fsnode can be accessed using a directory fd
for the parent and the child name. This is true when populating the
image using an mtree manifest or from a single staging directory, but
doesn't work when multiple staging directories are specified.
Change the traversal routines to use absolute path lookups when an mtree
manifest is not in use. This isn't ideal, but it's the simplest way to
fix the problem.
Reported by: imp
Sponsored by: The FreeBSD Foundation
Space used by the MOS is summed after all on-disk data structures are
finalized, so the MOS DSL directory represents a special case when
counting bytes used for each directory. The DSL layer failed to update
the MOS DSL directory's parent, the root directory, accordingly, so the
root directory reported less space used than was used in reality. Be
careful to update the root directory after the MOS is finalized.
PR: 265849
Sponsored by: The FreeBSD Foundation
Check the canmount property before building the mountpoint string.
Reported by: Coverity
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Sponsored by: The FreeBSD Foundation
nvstring_get() returns a copy of the string, not a pointer into the
nvlist's internal buffer.
Reported by: Coverity
Fixes: 240afd8c1f ("makefs: Add ZFS support")
Sponsored by: The FreeBSD Foundation
This allows one to take a staged directory tree and create a file
consisting of a ZFS pool with one or more datasets that contain the
contents of the directory tree. This is useful for creating virtual
machine images without using the kernel to create a pool; "zpool create"
requires root privileges and currently is not permitted in jails.
makefs -t zfs also provides reproducible images by using a fixed seed
for pseudo-random number generation, used for generating GUIDs and hash
salts. makefs -t zfs requires relatively little by way of machine
resources.
The "zpool_reguid" rc.conf setting can be used to ask a FreeBSD guest to
generate a unique pool GUID upon first boot.
A small number of pool and dataset properties are supported. The pool
is backed by a single disk vdev. Data is always checksummed using
Fletcher-4, no redundant copies are made, and no compression is used.
The manual page documents supported pool and filesystem properties.
The implementation uses a few pieces of ZFS support from with the boot
loader, especially definitions for various on-disk structures, but is
otherwise standalone and in particular doesn't depend on OpenZFS.
This feature should be treated as experimental for now, i.e., important
data shouldn't be trusted to a makefs-created pool, and the command-line
interface is subject to change.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35248
When a new FS image is created we need to calculate how much space each
file is going to consume.
Fix two bugs in that logic:
1) Count the space needed for indirect blocks for large files.
1) Normally the trailing data of a file is written to a block of frag
size, 4 kB by default.
However for files that use indirect blocks a full block is allocated,
32kB by default. Take that into account.
Adjust size calculations to match what is done in ffs_mkfs routine:
* Depending on the UFS version the superblock is stored at a different
offset. Take that into account.
* Add the cylinder group block size.
* All of the above has to be aligned to the block size.
Finally, Remove "ncg" variable. It's always 1 and it was used to
multiply stuff.
PR: 229929
Reviewed by: mckusick
MFC after: 2 weeks
Sponsored by: Semihalf
Submitted by: Kornel Dulęba <mindal@semihalf.com>
Differential Revision: https://reviews.freebsd.org/D35131
Differential Revision: https://reviews.freebsd.org/D35132
Leave -Wcast-align disabled, at least for now, since there are numerous
instances of that warning in places where buffer pointers are cast to
pointers to various filesystem structures. Fixing this properly would
be too much work for too little gain.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Change its return type to void, because its result is ignored in both
call sites. Remove oldcnp argument as well, it is NULL always.
Suggested and reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33721
Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.
Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].
Reviewed by: mckusick, jhb
Differential Revision: https://reviews.freebsd.org/D33650
LK_EXCLUSIVE must be passed always, some consumers need the ability to
specify LK_NOWAIT
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31464
