Commit Graph

213 Commits

Author SHA1 Message Date
Li-Wen Hsu
bfd735d96e Revert r362390, those tests are fixed by r362418
PR:		247425
Sponsored by:	The FreeBSD Foundation
2020-06-23 19:14:38 +00:00
Kirk McKusick
972670e132 Allocate an fs_summary_info structure when creating a UFS filesystem
needed since introduced in -r362358.

PR:           247425
Sponsored by: Netflix
2020-06-19 23:32:40 +00:00
Li-Wen Hsu
865c7b713c Skip ufs related tests in fstyp(8) and makefs(8) temporarily
They are failing after r362358 and r362359.

PR:		247425
Sponsored by:	The FreeBSD Foundation
2020-06-19 17:32:30 +00:00
Brooks Davis
10dd04d39f Fix an indentation bug in r357169. 2020-01-30 18:34:08 +00:00
Alex Richardson
162ae9c834 Allow bootstrapping makefs on older FreeBSD hosts and Linux/macOS
In order to do so we need to install the msdosfs headers to the bootstrap
sysroot and avoid includes of kernel headers that may not exist on every
host (e.g. sys/lockmgr.h). This change should allow bootstrapping of makefs
on FreeBSD 11+ as well as Linux and macOS.

We also have to avoid using the IO_SYNC macro since that may not be
available. In makefs it is only used to switch between calling
bwrite() and bdwrite() which both call the same function. Therefore we
can simply always call bwrite().

For our CheriBSD builds we always bootstrap makefs by setting
LOCAL_XTOOL_DIRS='lib/libnetbsd usr.sbin/makefs' and use the makefs binary
from the build tree to create a bootable disk image.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D23201
2020-01-27 12:02:41 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Alex Richardson
1e9f67e2e4 makefs: Also set UFS di_birthtime when building on Linux
Since st_birthtime doesn't exists on Linux (unless you use statx(2)), we
instead populate it with the st_ctime value.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D22386
2019-11-15 18:34:30 +00:00
Alex Richardson
62a3510f1f Fix contents= being ignored in msdosfs makefs mtree
I noticed this while trying to build an EFI boot image

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D22387
2019-11-15 18:34:23 +00:00
Conrad Meyer
aaa3852435 buf: Add B_INVALONERR flag to discard data
Setting the B_INVALONERR flag before a synchronous write causes the buf
cache to forcibly invalidate contents if the write fails (BIO_ERROR).

This is intended to be used to allow layers above the buffer cache to make
more informed decisions about when discarding dirty buffers without
successful write is acceptable.

As a proof of concept, use in msdosfs to handle failures to mark the on-disk
'dirty' bit during rw mount or ro->rw update.

Extending this to other filesystems is left as future work.

PR:		210316
Reviewed by:	kib (with objections)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D21539
2019-09-11 21:24:14 +00:00
Ed Maste
840aca2880 makefs: share msdosfsmount.h between kernel msdosfs and makefs
Sponsored by:	The FreeBSD Foundation
2019-09-01 16:55:33 +00:00
Xin LI
8651679a5c Properly update FSInfo block after generation.
After populating the filesystem, write a FSInfo block with
proper information.

Reviewed by:	emaste, cem
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D21363
2019-08-23 05:23:45 +00:00
Ed Maste
2037e9880c makefs: diff reduction to sys/fs/msdosfs
No functional change.
2019-08-22 17:49:34 +00:00
Ed Maste
59e9b3695a makefs: Verify that the BPB media descriptor and FAT ID match
From r322982 in sys/fs/msdosfs.
2019-08-21 19:09:40 +00:00
Ed Maste
476b0ab758 makefs: share denode.h between kernel msdosfs and makefs
There is no need to duplicate this file when it can be trivially
shared (just exposing sections previously under #ifdef _KERNEL).

MFC with:	r351273
Differential Revision:	The FreeBSD Foundation
2019-08-21 19:07:13 +00:00
Ed Maste
51e79affa3 makefs: share fat.h between kernel msdosfs and makefs
There is no reason to duplicate this file when it can be trivially
shared (just exposing one section previously under #ifdef _KERNEL).

Reviewed by:	imp, cem
MFC with:	r351273
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21346
2019-08-21 02:21:40 +00:00
Ed Maste
ef63362055 makefs: use char * not void * for buf b_data, drop casts in msdos
(The kernel uses caddr_t.)

Suggested by:	cem
Reviewed by:	cem
MFC with:	r351273
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21348
2019-08-21 01:45:29 +00:00
Ed Maste
35284c22e9 makefs.8: update history
- ported to FreeBSD and first appeared in 8.0
- Christos Zoulas added the FAT support that I imported
2019-08-20 21:14:44 +00:00
Ed Maste
b350417ae6 makefs: avoid "dereferencing 'void *' pointer" warnings
On GCC 4.2.1 archs

MFC with:	r351273
Sponsored by:	The FreeBSD Foundation
2019-08-20 20:04:16 +00:00
Ed Maste
98dc8da58c makefs: add msdosfs (FAT) support
Add FAT support to makefs by copying some files from sys/fs/msdosfs/ and
updating others with changes from NetBSD.

The six files copied from sys/fs/msdosfs at r348251 and modified are:
denode.h direntry.h fat.h msdosfs_fat.c msdosfs_lookup.c msdosfsmount.h

I would prefer to avoid the duplication, but reluctance to doing so was
expressed in a previous review (D11197); for now copy the files and
revisit in the future.

Submitted by:	Siva Mahadevan
Discussed with:	cem, imp
MFC after:	1 month
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D16438
2019-08-20 18:20:45 +00:00
Ed Maste
5059f7c3ed makefs.8: expand description of image size
Submitted by:	ryan_freqlabs.com, Siva Mahadevan
Differential Revision:	https://reviews.freebsd.org/D21316
2019-08-19 21:38:10 +00:00
Ed Maste
4d393b6c64 makefs.8: style updates from igor
Sponsored by:	The FreeBSD Foundation
2019-08-19 21:21:32 +00:00
Ed Maste
286258a9a0 makefs: Fix "time" mtree attribute handling
When processing mtree(5) MANIFEST files, makefs(8) previously threw an
error if it encountered an entry whose "time" attribute contained a
non-zero subsecond component (e.g. time=1551620152.987220000).

Update the handling logic to properly assign the subsecond component if
built with nanosecond support, or silently discard it otherwise.

Also, re-enable the time attribute for the kyua tests.

PR:		194703
Submitted by:	Mitchell Horne <mhorne063@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D19627
2019-03-18 19:26:36 +00:00
Maxim Sobolev
781e78182d Improve error handling: bail out if one of the files scheduled
to go to the FS image we are making cannot be read (e.g. EPERM).
Current behaviour when we issue waring but still proceeed and
return success is definitely not correct: masking out error
condition as well as making a slighly inconsistent FS where
attempt to access the file in question ends up in EBADF. See
linked DR for details.

MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D18584
2019-02-25 23:45:36 +00:00
Ed Maste
5b292f9a2d makefs: use FreeBSD brelse function signature
Although the ffs (and later msdosfs) implementation in makefs is
independent of the one in kernel, it makes sense to keep differences to
a minimum in order to ease comparison and porting changes across.

Submitted by:	Siva Mahadevan
Sponsored by:	The FreeBSD Foundation
2018-07-26 13:33:10 +00:00
Ed Maste
5add92953e makefs: whitespace cleanup in msdos files
Sponsored by:	The FreeBSD Foundation
2018-07-25 13:27:20 +00:00
Alan Somers
d3f229a463 makefs(8): add test case for PR 229929
Fix two failing makefs test cases by adding "-M 1m", which was already used
for every other FFS test case.  Add a new test case for the underlying
issue: with no -M, -m, or -s options, makefs can underestimate image size.

PR:		229929
Reported by:	Jenkins
MFC after:	2 weeks
2018-07-21 17:24:14 +00:00
Ed Maste
ab6f27aac1 makefs: ANSIfy 2018-05-16 02:58:05 +00:00
Pedro F. Giffuni
e2ec2f75f2 makefs: Use ENODATA instead of ENOMSG as a translation for missing ENOATTR.
This is consistent with what some linux filesystems do and has been
adopted in our linuxulator.

MFC after:	3 days
2018-04-25 02:43:53 +00:00
Ed Maste
daddfa7cc7 makefs: tidy up reach-over source
- cd9660 relies on an #include "iso.h" but does not build any .c files
  out of source, so remove reach-over .PATH
- ffs does not rely on any sys/ headers, so remove -I from CFLAGS.
- ffs_tables from sys/ is used by ffs; move the SRCS entry from the top-
  level Makefile to ffs' Makefile.inc.

Sponsored by:	The FreeBSD Foundation
2018-04-20 22:23:38 +00:00
Benno Rice
756e49c303 Synchronise with NetBSD's version of EFI handling for El Torito images.
When I implemented my EFI support I failed to check if the upstream version
of makefs in NetBSD had done the same. Override my version with theirs to
make it easier to stay in sync with them in the future.

Reviewed by:	imp, mav
Obtained from:	NetBSD
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14913
2018-03-31 15:04:41 +00:00
Ed Maste
c93ff841aa makefs: sync fragment and block size with newfs
r222319 in newfs raised the default blocksize for UFS/FFS filesystems
from 16K to 32K and the default fragment size from 2K to 4K, with a
rationale that most disks were now running with 4K sectors.

MFC after:	2 weeks
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
2018-03-30 03:38:08 +00:00
Benno Rice
633cb765d8 Don't try to modify El Torito section headers if there aren't any.
MFC after:	1 week
2018-03-23 22:59:45 +00:00
Benno Rice
ebf1c08960 Correctly mark the last El Torito section header.
Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	emaste, imp
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14812
2018-03-23 22:52:26 +00:00
Benno Rice
8dbc390581 Allow makefs to properly tag UEFI El Torito boot images. Use them in amd64 ISOs.
UEFI booting requires an EFI System Partition (ESP). On most storage devices
this will be in a specific partition type. To allow booting from CD/ISO
filesystems, UEFI will look for an ESP in the form of a FAT filesystem image
embedded in the image. Historically FreeBSD has added one of these to its
amd64 ISO images but marked it as simply another i386 boot image. Luckily for
us most UEFI implementations are rather forgiving and work this out for us.

This change adds the ability to mark a boot image as being a UEFI image. It
also modifies our ISO generation to use this marking for the UEFI image we
embed.

Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	emaste, imp
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14809
2018-03-23 20:56:18 +00:00
Brooks Davis
46b59ac82b Check for cd9660 support before attempting to mount created images
This extends the set in r316028 to allow all tests to pass or be skipped
on a system without cd9660 support.

A better approach using tar is possible, but this works today.

Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D10516
2018-02-02 23:34:33 +00:00
Alex Richardson
5f40118235 Allow xinstall and makefs to be crossbuilt on Linux and Mac
I need these tools in order to install the crossbuilt FreeBSD and create a
disk image. Linux does not have a st_flags in struct stat so unfortunately
I need a bunch of ugly ifdefs. The resulting binaries allow me to
sucessfully install a MIPS64 world and create a disk-image that boots.

Reviewed By:	brooks, bdrewery, emaste
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13307
2018-01-16 21:43:46 +00:00
Eitan Adler
dae3a64fb9 userland: Fix several typos and minor errors
- duplicate words
- typos
- references to old versions of FreeBSD

Reviewed by:	imp, benno
2017-12-27 03:23:01 +00:00
Mark Johnston
3836e3592e Fix a logic bug in makefs lazy inode initialization.
We may need to initialize multiple inode blocks before writing a given
inode. makefs(8) was only initializing a single block at a time, so
certain inode allocation patterns could lead to a situation where it
wrote an inode to an uninitialized block. That inode might be clobbered
by a later initialization, resulting in a filesystem image containing
directory entries that point to a seemingly unused inode.

Reviewed by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D13505
2017-12-16 20:19:00 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Pedro F. Giffuni
8a16b7a18f General further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:49:47 +00:00
Pedro F. Giffuni
df57947f08 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Eitan Adler
a2aef24aa3 Update several more URLs
- Primarily http -> https
- Primarily FreeBSD project URLs
2017-10-29 08:17:03 +00:00
Benno Rice
88e2cc9eb5 Replace makefs' hand-rolled unescaping with strunvis
mtree path names and link attributes are encoded, generally using strvis. Newer
versions of mtree will use C-style escapes but previously the accepted form was
octal escapes. makefs' mtree code spots the C-style escapes but fails to deal
with octal escapes correctly.

Remove mtree's escape-decoding code (except for a few instances where it's
needed) and instead pass pathnames and link targets through strunvis prior to
use.

Reviewed by:	marcel
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12104
2017-08-25 16:10:16 +00:00
Enji Cooper
86cc58dc96 MFhead@r321960 2017-08-02 22:28:12 +00:00
Enji Cooper
d2435d7c28 Require strings(1) with :o_flag_preparer and :o_flag_publisher
strings(1) might not be installed on the system, e.g., if MK_TOOLCHAIN == no

MFC after:	1 week
2017-08-02 20:42:39 +00:00
Enji Cooper
d511b20a69 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Ed Maste
0661ef2af2 makefs: add copies of NetBSD makefs msdos source files
We do not treat makefs as contrib code.  Import copies of makefs msdos
files from NetBSD so that we can track our changes to these files.

These are copied from NetBSD, with only a change to use __FBSDID and
$FreeBSD$ instead of __KERNEL_RCSID and $NetBSD$.  A copy of the
original $NetBSD$ tag remains in each source file.

These two files were missed in r320212.  Also remove a stray blank line
added in msdosfs_vfsops.c.

Submitted by:	Siva Mahadevan
Sponsored by:	The FreeBSD Foundation
2017-06-23 18:58:28 +00:00
Ed Maste
237d1b14f2 makefs: add copies of NetBSD makefs msdos source files
We do not treat makefs as contrib code.  Import copies of makefs msdos
files from NetBSD so that we can track our changes to these files.

These are copied from NetBSD, with only a change to use __FBSDID and
$FreeBSD$ instead of __KERNEL_RCSID and $NetBSD$.  A copy of the
original $NetBSD$ tag remains in each source file.

Submitted by:	Siva Mahadevan
Sponsored by:	The FreeBSD Foundation
2017-06-22 02:46:36 +00:00