This matches NetBSD and rationalizes makefs with the kernel API.
This reverts commit 370e009188.
Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
Obtained from: NetBSD 0a62dad69f62, 0c4125e1a19f, cb6a5a3575fd
Differential Revision: https://reviews.freebsd.org/D39070
When a new FS image is created we need to calculate how much space each
file is going to consume.
Fix two bugs in that logic:
1) Count the space needed for indirect blocks for large files.
1) Normally the trailing data of a file is written to a block of frag
size, 4 kB by default.
However for files that use indirect blocks a full block is allocated,
32kB by default. Take that into account.
Adjust size calculations to match what is done in ffs_mkfs routine:
* Depending on the UFS version the superblock is stored at a different
offset. Take that into account.
* Add the cylinder group block size.
* All of the above has to be aligned to the block size.
Finally, Remove "ncg" variable. It's always 1 and it was used to
multiply stuff.
PR: 229929
Reviewed by: mckusick
MFC after: 2 weeks
Sponsored by: Semihalf
Submitted by: Kornel Dulęba <mindal@semihalf.com>
Differential Revision: https://reviews.freebsd.org/D35131
Differential Revision: https://reviews.freebsd.org/D35132
Leave -Wcast-align disabled, at least for now, since there are numerous
instances of that warning in places where buffer pointers are cast to
pointers to various filesystem structures. Fixing this properly would
be too much work for too little gain.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.
Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].
Reviewed by: mckusick, jhb
Differential Revision: https://reviews.freebsd.org/D33650
By default, makefs(8) has very few spare inodes in its output images,
which is fine for static filesystems, but not so great for VM images
where many more files will be added. Make makefs(8) use the same
default settings as newfs(8) when creating images with free space --
there isn't much point to leaving free space on the image if you
can't put files there. If no free space is requested, use current
behavior of a minimal number of available inodes.
Reviewed by: manu
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D29492
Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.
Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition. Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D28679
Since st_birthtime doesn't exists on Linux (unless you use statx(2)), we
instead populate it with the st_ctime value.
Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D22386
to go to the FS image we are making cannot be read (e.g. EPERM).
Current behaviour when we issue waring but still proceeed and
return success is definitely not correct: masking out error
condition as well as making a slighly inconsistent FS where
attempt to access the file in question ends up in EBADF. See
linked DR for details.
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D18584
Although the ffs (and later msdosfs) implementation in makefs is
independent of the one in kernel, it makes sense to keep differences to
a minimum in order to ease comparison and porting changes across.
Submitted by: Siva Mahadevan
Sponsored by: The FreeBSD Foundation
r222319 in newfs raised the default blocksize for UFS/FFS filesystems
from 16K to 32K and the default fragment size from 2K to 4K, with a
rationale that most disks were now running with 4K sectors.
MFC after: 2 weeks
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
I need these tools in order to install the crossbuilt FreeBSD and create a
disk image. Linux does not have a st_flags in struct stat so unfortunately
I need a bunch of ugly ifdefs. The resulting binaries allow me to
sucessfully install a MIPS64 world and create a disk-image that boots.
Reviewed By: brooks, bdrewery, emaste
Approved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13307
We may need to initialize multiple inode blocks before writing a given
inode. makefs(8) was only initializing a single block at a time, so
certain inode allocation patterns could lead to a situation where it
wrote an inode to an uninitialized block. That inode might be clobbered
by a later initialization, resulting in a filesystem image containing
directory entries that point to a seemingly unused inode.
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D13505
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
Initially, only tag files that use BSD 4-Clause "Original" license.
RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133
it has nothing to do with ffs and will eventually be moved.
gc sectorsize.
This is a corrected version of r317744.
NetBSD versions:
ffs.c 1.58
ffs/buf.c 1.14 1.18
ffs/buf.h 1.8
Submitted by: Siva Mahadevan <smahadevan@freebsdfoundation.org>
Obtained from: NetBSD
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10803
Add the ffs option to enable soft-updates.
The option is only processed is ufs2 has been selected.
Reviewed by: emaste, bapt (earlier version), allanjude (earlier version)
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D10773
it has nothing to do with ffs and will eventually be moved.
gc sectorsize.
NetBSD versions:
ffs.c 1.58
ffs/buf.c 1.14 1.18
ffs/buf.h 1.8
Obtained from: NetBSD
Sponsored by: The FreeBSD Foundation
The emalloc set of error-checking memory allocation routines were added
to libnetbsd in r316572. Use them in makefs to reduce differences with
NetBSD.
NetBSD revs:
cd9660.c 1.39
ffs.c 1.56
makefs.c 1.42
walk.c 1.27
cd9660/cd9660_archimedes.c 1.2
cd9660/cd9660_eltorito.c 1.20
cd9660/cd9660_write.c 1.16
cd9660/iso9660_rrip.c 1.12
ffs/buf.c 1.17
ffs/mkfs.c 1.26
Obtained from: NetBSD
From NetBSD christos Sat Jan 26 00:19:39 2013 +0000
make the buffer functions look exactly like the kernel ones and add other
cruft to make the kernel files compile.
ffs.c 1.54
ffs/buf.c 1.13
ffs/buf.h 1.3
ffs/ffs_alloc.c 1.21
ffs/ffs_balloc.c 1.15
Reviewed by: marcel, ngie
Obtained from: NetBSD
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8404
from struct stat. We don't necessarily have permissions to see the
generation number and the host OS may not have st_gen in struct stat
anyway. Since the kernel assigns random numbers, there's nothing
meaningful about the generation that requires us to preserve it when
the file system image is created. With this change, all generation
numbers come from random() and that makes it easier to add support
for reproducible builds at some time in the future (i.e. by adding
an argument to makefs that changes the behaviour of random() so that
it always returns 0 or some predictable sequence).
Differential Revision: https://reviews.freebsd.org/D8418
May 22 21:51:39 2011 +0000 (christos):
From Nathan Whitehorn (nwhitehorn at freebsd dot org):
Add code to generate bootable ISOs on Powermac and CHRP systems.
Synthesize some partition maps (APM and MBR, respectively) pointing
to (a) the whole disk, and (b) relevant El Torito boot images that
have been added by other code. These partition maps are a little
bit funny looking, but they seem to work. FreeBSD has been using
this successfully in their release generation on powerpc, as well
as generating all non-SPARC install media. SPARC support could
probably be added as an extension of this patch.
makefs.8 1.33
Tue Aug 23 17:09:11 2011 +0000 (christos):
PR/45285: Martin Matuska: makefs does not properly convert ISO level 1 and 2
filenames (buffer overflow)
makefs does not properly verify the maximum filename length in the
special "." case for both ISO level 1 and ISO level 2 filename
conversion. This creates broken images or causes a buffer overflow
(ISO level 2).
ISO level 1:
If a filename contains only dots or up to 8 characters followed by
dots the 8+3 limit check doesn't work.
ISO level 2:
If a filename contains a dot in the first 30 characters and a dot
on the 30th character, the length limit check doesn't work and the
buffer is overflowed.
$ mkdir level1
$ touch level1/12345............
$ makefs -t cd9660 -o isolevel=1 test.iso level1
$ mkdir level2
$ touch level2/1234567890.2345678901234567.....34567890123456789012345
$ makefs -t cd9660 -o isolevel=2 test.iso level2
cd9660.c 1.32
Sun Oct 9 21:33:43 2011 +0000 (christos):
add support for setting the ufs label. (Nathan Whitehorn)
ffs.c 1.45
ffs.h 1.2
mkfs.c 1.22
makefs.8 1.37
Obtained from: NetBSD
This is taken from the NetBSD versions listed below and adapted to the
makefs version in FreeBSD, along with a bug fix from cem@ that will be
sent to NetBSD.
Reviewed by: pfg
Approved by: re (gjb)
Obtained from: NetBSD
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D6835
The generation number is uint32_t so we can fit the complete range
of random(3). We could have used arc4random() but the result would
be unpredictable and it would prohibit reproducible builds.
While here add a comment where seeding is done: this affects
reproducible builds and might have to be re-visited to use a
release dependent value.
MFC after: 2 weeks
image up to.
From ticket:
While trying to run FreeBSD/mips on some device having very small flash media,
one is forced to compress file system with mkulzma(8) utility. It is desirable
to specify small UFS block/fragment sizes like 4096/512 bytes for makefs(8)
and big compression block size like 65535 bytes to mkulzma at the same time.
Then one obtains very good comression ratios (like 75% and more) but faces
the following problem.
geom_uncompress kernel module reports GEOM provider size rounded up to its
compression block size. Generally, this changes original media size and now
it fails to match the size of embedded UFS file system that leads to other
problems, f.e. geom_label kernel module does not like this and skips the
file system while tasting the GEOM and looking for UFS label.
This makes it impossible to refer to the file system using known UFS label
instead of something like /dev/map/rootfs.uncompress.
The following patch introduces new command line option "-r roundup" for makefs
that makes it round up the image to specified block size. Hence, geom_uncompress
does not change GEOM media size for images rounded that way and geom_label
accepts such GEOMs just fine.
With the patch applied, one can use following commands:
$ makefs -t ffs -r 65536 -o bsize=4096,fsize=512,label=flash optimization=space fs.img fs
$ mkulzma -s 65536 -o fs.img.ulzma fs.img
PR: bin/203707
Submitted by: <eugen@grosbein.net>
the minimum image size specified is always less than the maximum
image size. If makefs(1) is invoked specifying minimum image size,
but not maximum one, the program exits with an error. Example:
# sudo -E makefs -M 538968064 -B be /home/davide/disk.img $DESTDIR
makefs: `/home/davide/tftproot/mips' minsize of 538968064 rounded up
to ffs bsize of 8192 exceeds maxsize 0. Lower bsize, or round the
minimum and maximum sizes to bsize.
Assert then that minsize < maxsize iff maxsize is specified.
This change allows me to build MIPS images using makefs(1) and following
what specified in the wiki again.
Reviewed by: jmallett, ngie
would exceed the maximum size. This can be a difficult problem to diagnose
if one is, for instance, using -s with a fixed size in a script and the bsize
calculated for a filesystem image changes, necessitating a re-rounding of the
image size or a hand-setting of the bsize. Previously one would get a
cryptic message about how the size exceeded the maximum size, which normally
only happens if the contents of the image are larger than specified.
to be created. The support is based on mtree version 2.0, as used
in libarchive, but adds new features on top of it.
The current implementation is fully functional, but is envisioned
to grow at least the following additional features over time:
o Add support for the /include special command so that manifest
files can be constructed using includable fragments.
o Add support specifying a search path to locate content files.
o Content file filters: commands that provide file contents on
stdout.
The manifest file eliminates the need to first construct a tree
as root in order to create an image and allows images (releases)
to be created directly from object trees and/or source trees.
Reviewed by: deo
Sponsored by: Juniper Networks, Inc
include sys/time.h instead of time.h. This include is incorrect as
per the manpages for the APIs and the POSIX definitions. This commit
replaces sys/time.h where necessary with time.h.
The commit also includes some minor style(9) header fixup in newfs.
This commit is part of a larger effort by Garrett Cooper started in
//depot/user/gcooper/posix-conformance-work/ -- to make FreeBSD more
POSIX compliant.
Submitted by: Garrett Cooper yanegomi at gmail dot com