Commit Graph

223 Commits

Author SHA1 Message Date
Nathan Whitehorn
afb6a168f8 Allocate extra inodes in makefs when leaving free space in UFS images.
By default, makefs(8) has very few spare inodes in its output images,
which is fine for static filesystems, but not so great for VM images
where many more files will be added. Make makefs(8) use the same
default settings as newfs(8) when creating images with free space --
there isn't much point to leaving free space on the image if you
can't put files there. If no free space is requested, use current
behavior of a minimal number of available inodes.

Reviewed by:	manu
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D29492
2021-04-06 13:43:29 -04:00
Mark Johnston
ed42b22abc makefs: Ignore the "tags" keyword in mtree manifests
An install using -DNO_ROOT emits mtree entries containing tags used by
pkgbase.  makefs(8) can safely ignore them, so do that rather than
emitting a warning for each entry.

Reviewed by:	brooks, imp
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D29384
2021-03-23 14:38:40 -04:00
Alex Richardson
ba2cfa80e1 Fix makefs bootstrap after d485c77f20
The makefs msdosfs code includes fs/msdosfs/denode.h which directly uses
struct buf from <sys/buf.h> rather than the makefs struct m_buf.
To work around this problem provide a local denode.h that includes
ffs/buf.h and defines buf as an alias for m_buf.

Reviewed By:	kib, emaste
Differential Revision: https://reviews.freebsd.org/D28835
2021-02-22 17:55:45 +00:00
Konstantin Belousov
d485c77f20 Remove #define _KERNEL hacks from libprocstat
Make sys/buf.h, sys/pipe.h, sys/fs/devfs/devfs*.h headers usable in
userspace, assuming that the consumer has an idea what it is for.
Unhide more material from sys/mount.h and sys/ufs/ufs/inode.h,
sys/ufs/ufs/ufsmount.h for consumption of userspace tools, with the
same caveat.

Remove unacceptable hack from usr.sbin/makefs which relied on sys/buf.h
being unusable in userspace, where it override struct buf with its own
definition.  Instead, provide struct m_buf and struct m_vnode and adapt
code to use local variants.

Reviewed by:	mckusick
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D28679
2021-02-21 11:38:21 +02:00
Alex Richardson
12ad8bdb34 usr.sbin/makefs: fix use-after-free in read_mtree_keywords()
The st variable is used as a shortcut for &node->inode->st, but in one
branch just before the exit we update node->inode without changing st.

Reported by:	AddressSanitizer
Reviewed By:	emaste
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D28570
2021-02-10 15:24:25 +00:00
Alex Richardson
fe41c64b57 Fix -Wpointer-sign warnings in makefs and mkimg
Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D27175
2021-01-07 09:26:21 +00:00
Ed Maste
53d8e037b1 makefs: connect cd9660 El Torito EFI boot image system type
Sponsored by:	The FreeBSD Foundation
2020-09-17 19:41:10 +00:00
Alex Richardson
6424881cc8 Fix makefs bootstrap on macOS after D25563
The macOS assert.h header does not define static_assert when compiling in
C99 mode. To fix this compile with -std=c11.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D25928
2020-08-25 13:30:24 +00:00
Alex Richardson
06e20d1bab makefs (msdosfs): Use fprintf instead of debug print for errors
The added print was very helpful for debugging failed disk image creation.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D23200
2020-08-24 09:20:27 +00:00
Alex Richardson
b4e38a41f5 makefs: Drop unnecessary sys/clock.h include
This breaks the build on macOS where this header doesn't exist. I could
also add a compat header to tools/build/cross-build but since it's not
needed removing it seems like the better solution.
2020-08-07 16:04:06 +00:00
Li-Wen Hsu
bfd735d96e Revert r362390, those tests are fixed by r362418
PR:		247425
Sponsored by:	The FreeBSD Foundation
2020-06-23 19:14:38 +00:00
Kirk McKusick
972670e132 Allocate an fs_summary_info structure when creating a UFS filesystem
needed since introduced in -r362358.

PR:           247425
Sponsored by: Netflix
2020-06-19 23:32:40 +00:00
Li-Wen Hsu
865c7b713c Skip ufs related tests in fstyp(8) and makefs(8) temporarily
They are failing after r362358 and r362359.

PR:		247425
Sponsored by:	The FreeBSD Foundation
2020-06-19 17:32:30 +00:00
Brooks Davis
10dd04d39f Fix an indentation bug in r357169. 2020-01-30 18:34:08 +00:00
Alex Richardson
162ae9c834 Allow bootstrapping makefs on older FreeBSD hosts and Linux/macOS
In order to do so we need to install the msdosfs headers to the bootstrap
sysroot and avoid includes of kernel headers that may not exist on every
host (e.g. sys/lockmgr.h). This change should allow bootstrapping of makefs
on FreeBSD 11+ as well as Linux and macOS.

We also have to avoid using the IO_SYNC macro since that may not be
available. In makefs it is only used to switch between calling
bwrite() and bdwrite() which both call the same function. Therefore we
can simply always call bwrite().

For our CheriBSD builds we always bootstrap makefs by setting
LOCAL_XTOOL_DIRS='lib/libnetbsd usr.sbin/makefs' and use the makefs binary
from the build tree to create a bootable disk image.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D23201
2020-01-27 12:02:41 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Alex Richardson
1e9f67e2e4 makefs: Also set UFS di_birthtime when building on Linux
Since st_birthtime doesn't exists on Linux (unless you use statx(2)), we
instead populate it with the st_ctime value.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D22386
2019-11-15 18:34:30 +00:00
Alex Richardson
62a3510f1f Fix contents= being ignored in msdosfs makefs mtree
I noticed this while trying to build an EFI boot image

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D22387
2019-11-15 18:34:23 +00:00
Conrad Meyer
aaa3852435 buf: Add B_INVALONERR flag to discard data
Setting the B_INVALONERR flag before a synchronous write causes the buf
cache to forcibly invalidate contents if the write fails (BIO_ERROR).

This is intended to be used to allow layers above the buffer cache to make
more informed decisions about when discarding dirty buffers without
successful write is acceptable.

As a proof of concept, use in msdosfs to handle failures to mark the on-disk
'dirty' bit during rw mount or ro->rw update.

Extending this to other filesystems is left as future work.

PR:		210316
Reviewed by:	kib (with objections)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D21539
2019-09-11 21:24:14 +00:00
Ed Maste
840aca2880 makefs: share msdosfsmount.h between kernel msdosfs and makefs
Sponsored by:	The FreeBSD Foundation
2019-09-01 16:55:33 +00:00
Xin LI
8651679a5c Properly update FSInfo block after generation.
After populating the filesystem, write a FSInfo block with
proper information.

Reviewed by:	emaste, cem
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D21363
2019-08-23 05:23:45 +00:00
Ed Maste
2037e9880c makefs: diff reduction to sys/fs/msdosfs
No functional change.
2019-08-22 17:49:34 +00:00
Ed Maste
59e9b3695a makefs: Verify that the BPB media descriptor and FAT ID match
From r322982 in sys/fs/msdosfs.
2019-08-21 19:09:40 +00:00
Ed Maste
476b0ab758 makefs: share denode.h between kernel msdosfs and makefs
There is no need to duplicate this file when it can be trivially
shared (just exposing sections previously under #ifdef _KERNEL).

MFC with:	r351273
Differential Revision:	The FreeBSD Foundation
2019-08-21 19:07:13 +00:00
Ed Maste
51e79affa3 makefs: share fat.h between kernel msdosfs and makefs
There is no reason to duplicate this file when it can be trivially
shared (just exposing one section previously under #ifdef _KERNEL).

Reviewed by:	imp, cem
MFC with:	r351273
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21346
2019-08-21 02:21:40 +00:00
Ed Maste
ef63362055 makefs: use char * not void * for buf b_data, drop casts in msdos
(The kernel uses caddr_t.)

Suggested by:	cem
Reviewed by:	cem
MFC with:	r351273
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21348
2019-08-21 01:45:29 +00:00
Ed Maste
35284c22e9 makefs.8: update history
- ported to FreeBSD and first appeared in 8.0
- Christos Zoulas added the FAT support that I imported
2019-08-20 21:14:44 +00:00
Ed Maste
b350417ae6 makefs: avoid "dereferencing 'void *' pointer" warnings
On GCC 4.2.1 archs

MFC with:	r351273
Sponsored by:	The FreeBSD Foundation
2019-08-20 20:04:16 +00:00
Ed Maste
98dc8da58c makefs: add msdosfs (FAT) support
Add FAT support to makefs by copying some files from sys/fs/msdosfs/ and
updating others with changes from NetBSD.

The six files copied from sys/fs/msdosfs at r348251 and modified are:
denode.h direntry.h fat.h msdosfs_fat.c msdosfs_lookup.c msdosfsmount.h

I would prefer to avoid the duplication, but reluctance to doing so was
expressed in a previous review (D11197); for now copy the files and
revisit in the future.

Submitted by:	Siva Mahadevan
Discussed with:	cem, imp
MFC after:	1 month
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D16438
2019-08-20 18:20:45 +00:00
Ed Maste
5059f7c3ed makefs.8: expand description of image size
Submitted by:	ryan_freqlabs.com, Siva Mahadevan
Differential Revision:	https://reviews.freebsd.org/D21316
2019-08-19 21:38:10 +00:00
Ed Maste
4d393b6c64 makefs.8: style updates from igor
Sponsored by:	The FreeBSD Foundation
2019-08-19 21:21:32 +00:00
Ed Maste
286258a9a0 makefs: Fix "time" mtree attribute handling
When processing mtree(5) MANIFEST files, makefs(8) previously threw an
error if it encountered an entry whose "time" attribute contained a
non-zero subsecond component (e.g. time=1551620152.987220000).

Update the handling logic to properly assign the subsecond component if
built with nanosecond support, or silently discard it otherwise.

Also, re-enable the time attribute for the kyua tests.

PR:		194703
Submitted by:	Mitchell Horne <mhorne063@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D19627
2019-03-18 19:26:36 +00:00
Maxim Sobolev
781e78182d Improve error handling: bail out if one of the files scheduled
to go to the FS image we are making cannot be read (e.g. EPERM).
Current behaviour when we issue waring but still proceeed and
return success is definitely not correct: masking out error
condition as well as making a slighly inconsistent FS where
attempt to access the file in question ends up in EBADF. See
linked DR for details.

MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D18584
2019-02-25 23:45:36 +00:00
Ed Maste
5b292f9a2d makefs: use FreeBSD brelse function signature
Although the ffs (and later msdosfs) implementation in makefs is
independent of the one in kernel, it makes sense to keep differences to
a minimum in order to ease comparison and porting changes across.

Submitted by:	Siva Mahadevan
Sponsored by:	The FreeBSD Foundation
2018-07-26 13:33:10 +00:00
Ed Maste
5add92953e makefs: whitespace cleanup in msdos files
Sponsored by:	The FreeBSD Foundation
2018-07-25 13:27:20 +00:00
Alan Somers
d3f229a463 makefs(8): add test case for PR 229929
Fix two failing makefs test cases by adding "-M 1m", which was already used
for every other FFS test case.  Add a new test case for the underlying
issue: with no -M, -m, or -s options, makefs can underestimate image size.

PR:		229929
Reported by:	Jenkins
MFC after:	2 weeks
2018-07-21 17:24:14 +00:00
Ed Maste
ab6f27aac1 makefs: ANSIfy 2018-05-16 02:58:05 +00:00
Pedro F. Giffuni
e2ec2f75f2 makefs: Use ENODATA instead of ENOMSG as a translation for missing ENOATTR.
This is consistent with what some linux filesystems do and has been
adopted in our linuxulator.

MFC after:	3 days
2018-04-25 02:43:53 +00:00
Ed Maste
daddfa7cc7 makefs: tidy up reach-over source
- cd9660 relies on an #include "iso.h" but does not build any .c files
  out of source, so remove reach-over .PATH
- ffs does not rely on any sys/ headers, so remove -I from CFLAGS.
- ffs_tables from sys/ is used by ffs; move the SRCS entry from the top-
  level Makefile to ffs' Makefile.inc.

Sponsored by:	The FreeBSD Foundation
2018-04-20 22:23:38 +00:00
Benno Rice
756e49c303 Synchronise with NetBSD's version of EFI handling for El Torito images.
When I implemented my EFI support I failed to check if the upstream version
of makefs in NetBSD had done the same. Override my version with theirs to
make it easier to stay in sync with them in the future.

Reviewed by:	imp, mav
Obtained from:	NetBSD
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14913
2018-03-31 15:04:41 +00:00
Ed Maste
c93ff841aa makefs: sync fragment and block size with newfs
r222319 in newfs raised the default blocksize for UFS/FFS filesystems
from 16K to 32K and the default fragment size from 2K to 4K, with a
rationale that most disks were now running with 4K sectors.

MFC after:	2 weeks
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
2018-03-30 03:38:08 +00:00
Benno Rice
633cb765d8 Don't try to modify El Torito section headers if there aren't any.
MFC after:	1 week
2018-03-23 22:59:45 +00:00
Benno Rice
ebf1c08960 Correctly mark the last El Torito section header.
Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	emaste, imp
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14812
2018-03-23 22:52:26 +00:00
Benno Rice
8dbc390581 Allow makefs to properly tag UEFI El Torito boot images. Use them in amd64 ISOs.
UEFI booting requires an EFI System Partition (ESP). On most storage devices
this will be in a specific partition type. To allow booting from CD/ISO
filesystems, UEFI will look for an ESP in the form of a FAT filesystem image
embedded in the image. Historically FreeBSD has added one of these to its
amd64 ISO images but marked it as simply another i386 boot image. Luckily for
us most UEFI implementations are rather forgiving and work this out for us.

This change adds the ability to mark a boot image as being a UEFI image. It
also modifies our ISO generation to use this marking for the UEFI image we
embed.

Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	emaste, imp
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14809
2018-03-23 20:56:18 +00:00
Brooks Davis
46b59ac82b Check for cd9660 support before attempting to mount created images
This extends the set in r316028 to allow all tests to pass or be skipped
on a system without cd9660 support.

A better approach using tar is possible, but this works today.

Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D10516
2018-02-02 23:34:33 +00:00
Alex Richardson
5f40118235 Allow xinstall and makefs to be crossbuilt on Linux and Mac
I need these tools in order to install the crossbuilt FreeBSD and create a
disk image. Linux does not have a st_flags in struct stat so unfortunately
I need a bunch of ugly ifdefs. The resulting binaries allow me to
sucessfully install a MIPS64 world and create a disk-image that boots.

Reviewed By:	brooks, bdrewery, emaste
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13307
2018-01-16 21:43:46 +00:00
Eitan Adler
dae3a64fb9 userland: Fix several typos and minor errors
- duplicate words
- typos
- references to old versions of FreeBSD

Reviewed by:	imp, benno
2017-12-27 03:23:01 +00:00
Mark Johnston
3836e3592e Fix a logic bug in makefs lazy inode initialization.
We may need to initialize multiple inode blocks before writing a given
inode. makefs(8) was only initializing a single block at a time, so
certain inode allocation patterns could lead to a situation where it
wrote an inode to an uninitialized block. That inode might be clobbered
by a later initialization, resulting in a filesystem image containing
directory entries that point to a seemingly unused inode.

Reviewed by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D13505
2017-12-16 20:19:00 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Pedro F. Giffuni
8a16b7a18f General further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:49:47 +00:00