device nodes
In particular, use st_rdev (the device type), not st_dev (the device inode),
and fix the comparison to be correct with the st_rdev field
Bug 203648
MFC after: 2 weeks
Submitted by: Thomas Schmitt <scdbackup@gmx.net>
Coverity CID: 1008927
Sponsored by: EMC / Isilon Storage Division
General changes:
- Parameterize out the mount command.
- Use mtree to verify the contents of an image (check_image_contents) instead
of using diff (diff verifies content, but not file metadata).
- Move common logic out to functions (common_cleanup, mount_image,
check_image_contents)
- Add stub testcases for makefs -D (crashes with SIGBUS, similar to bug # 192839)
- Add a note about the ISO-9660 and rockridge specs
- Add testcases that exercise:
-- Creating disk images from an mtree and multiple directories.
-- -F flag use (not really an extensive testcase right now)
cd9660-specific test changes:
- Remove an XXX comment about symlinks; I forgot that non-rockridge images turn
symlinks into hardlinks.
- Add testcases that exercise:
-- -o allow-deep-trees
-- -o allow-max-name stub testcase (doesn't seem to be implemented in makefs)
-- -o preparer (existence in image; not conformance to spec)
-- -o publisher (existence in image; not conformance to spec)
-- -o rockridge (basic)
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
these testcases don't need to be nested as much as bin/ls/ls_tests.sh
do when verifying ls -a, ls -A, etc. This allows the tests to make
all paths relative to the top of the temporary directory instead of
always tacking on $ATF_TMPDIR, thus complicating things unnecessarily
Create non-empty files in create_test_inputs as well now, similar to
create_test_inputs2 in bin/ls/ls_tests.sh
Compare the input files to the output file contents using diff where
possible:
- Skip over the fifo comparison for now because it always fails
- Skip over the symlink comparison on cd9660 because it always fails
today
Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks
X-MFC with: r289739
buffer_head needs to be freed -- not buffer
Detected by jemalloc, i.e. running makefs failed the arena assert
because my copy of malloc on CURRENT is compiled with the default
!MALLOC_PRODUCTION asserts on
Pointyhat to: ngie
PR: 203647
X-MFC with: r289687
Sponsored by: EMC / Isilon Storage Division
leaking it after returning from the function
MFC after: 1 week
PR: 203647
Submitted by: Thomas Schmitt <scdbackup@gmx.net>
Coverity CID: 978431
Sponsored by: EMC / Isilon Storage Division
checked with dumpfs(8). This may lead to other problems, f.e. geom_label kernel
module sanity checks do not like zero fs_old_size value and skips such UFS1
file system while tasting (fs_old_size derives from sblock.fs_providersize).
PR: 203704
Submitted by: eugen@grosbein.net
Reviewed by: marcel
image up to.
From ticket:
While trying to run FreeBSD/mips on some device having very small flash media,
one is forced to compress file system with mkulzma(8) utility. It is desirable
to specify small UFS block/fragment sizes like 4096/512 bytes for makefs(8)
and big compression block size like 65535 bytes to mkulzma at the same time.
Then one obtains very good comression ratios (like 75% and more) but faces
the following problem.
geom_uncompress kernel module reports GEOM provider size rounded up to its
compression block size. Generally, this changes original media size and now
it fails to match the size of embedded UFS file system that leads to other
problems, f.e. geom_label kernel module does not like this and skips the
file system while tasting the GEOM and looking for UFS label.
This makes it impossible to refer to the file system using known UFS label
instead of something like /dev/map/rootfs.uncompress.
The following patch introduces new command line option "-r roundup" for makefs
that makes it round up the image to specified block size. Hence, geom_uncompress
does not change GEOM media size for images rounded that way and geom_label
accepts such GEOMs just fine.
With the patch applied, one can use following commands:
$ makefs -t ffs -r 65536 -o bsize=4096,fsize=512,label=flash optimization=space fs.img fs
$ mkulzma -s 65536 -o fs.img.ulzma fs.img
PR: bin/203707
Submitted by: <eugen@grosbein.net>
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
the minimum image size specified is always less than the maximum
image size. If makefs(1) is invoked specifying minimum image size,
but not maximum one, the program exits with an error. Example:
# sudo -E makefs -M 538968064 -B be /home/davide/disk.img $DESTDIR
makefs: `/home/davide/tftproot/mips' minsize of 538968064 rounded up
to ffs bsize of 8192 exceeds maxsize 0. Lower bsize, or round the
minimum and maximum sizes to bsize.
Assert then that minsize < maxsize iff maxsize is specified.
This change allows me to build MIPS images using makefs(1) and following
what specified in the wiki again.
Reviewed by: jmallett, ngie
1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.
We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.
Reported by: kib
This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.
Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]
Mark known build failures as NO_PIE for now.
The only known runtime failure was rtld.
[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]
Put the superblock in the correct possition for UFS2... There is a bug
in FFS that if we don't put it here (for UFS2), it will forcefully
relocate the superblock, and I believe cause data loss..
I have a fix for that, but w/ how many releases are broken, we won't be
able to switch to the better _FLOPPY (block 0) for this for a while..
i. e. the POSIX:5.6.1 st_ino field, which can be used to detect hard links
in the file system. This is also the default in mkisofs(8) and according to
its man page, no system only being able to cope with Rock Ridge version 1.10
is known to exist.
PR: 185138
Submitted by: Kurt Lidl
MFC after: 1 week
would exceed the maximum size. This can be a difficult problem to diagnose
if one is, for instance, using -s with a fixed size in a script and the bsize
calculated for a filesystem image changes, necessitating a re-rounding of the
image size or a hand-setting of the bsize. Previously one would get a
cryptic message about how the size exceeded the maximum size, which normally
only happens if the contents of the image are larger than specified.
rather than NetBSD.
- Correctly set the Expiration Time in the Primary Volume Descriptor;
according to ISO 9660 8.4.26.1 unspecified date and time are denoted
by the digit 0 in RBP 1 to 16 but the number 0 in RBP 17. [1]
- Merge iso9660_rrip.c rev. 1.11 from NetBSD: name_len should be read
as unsigned byte. [2]
Note: This is according to ISO 9660 9.1.10.
- Rock Ridge TF entries should use a length of 5, because after the 4
bytes of generic SUSP header there is one byte of flags. See typedef
of ISO_RRIP_TF in iso9660_rrip.h. [1]
Submitted by: Thomas Schmitt [1]
Obtained from: NetBSD [2]
MFC after: 3 days
is picked up when defined. Previously, ffs_subr.c was always built without
support for opposite endianess as it doesn't include makefs.h on its own.
MFC after: 3 days
The cd9660 file system uses that field for the link count and it
was 0. This impacts pwd_mkdb(8) as it checks for st_nlink not being
0 as part of closing a race.
- Fill the needed pmPartStatus flags. At least the OpenBIOS
implementation relies on these flags.
This commit fixes the panic seen on OS-X when inserting a FreeBSD/ppc disc.
Additionally OpenBIOS recognizes the partition where the boot code is located.
This lets us load a FreeBSD/ppc PowerMac kernel inside qemu.
PR: powerpc/162091
MFC after: 1 week
error: variable 'parentrecnum' set but not used
Approved by: dim, cperciva (mentor, blanket for pre-mentorship already-approved commits)
MFC after: 3 days
error: variable 'temp' set but not used
Approved by: dim
Approved by: cperciva (mentor, blanket for pre-mentorship already-approved commits)
MFC after: 3 days
- Let panic() use _Noreturn and __printflike() instead of GCC-specific
attributes.
- Remove prototype from ffs_subr.c and let it include ffs_extern.h.
- Forward declare struct inode to make ffs_extern.h not depend on it.
o cd9960 -> cd9660
o Move inclusion of sys/endian.h from cd9660_eltorito.c to cd9660.h
since actual user is not cd9660_eltorito.c but iso.h and
cd9660_eltorito.h.
Actually, include order/place of sys/endian.h doesn't matter on
netbsd since it is always included by sys/types.h but it's not
true on other system. This should fix cross build breakage on
freebsd introduced by rev. 1.16 of cd9660_eltorito.c.
Problem reported and fix suggested on twitter.
o Fix fd leaks in error cases. Found by cppcheck.
o RRIP RE length should be 4, not 0
o Apply fixes for PR bin/44114 (makefs(8) -t cd9660 -o rockridge creates
corrupted cd9660fs), iso9660_rrip.c part:
- cd9660_rrip_finalize_node() should check rr_real_parent in node->parent,
not in node itself in RRIP_PL case
- cd9660_rrip_initialize_node() should update only node passed as arg
so handle RRIP_PL in DOTDOT case
Fixes malformed dotdot entries in deep (more than 8 level) directories
moved into .rr_moved dir.
Should be pulled up to netbsd-5.
(no official ISO has such deep dirs, but cobalt restorecd is affected)
Reviewed by: mm
Approved by: re (kib)
Obtained from: NetBSD
MFC after: 3 days
makefs with -t cd9660 -o rockridge against directories with
deep structure creates a corrupted cd9660 image.
http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=44114
Fix NetBSD PR bin/45217:
makefs creates ISO9660 images that violate the ECMA-119 (ISO9660)
specification. This is caused by erroneously writing 32 bytes
with value 0x20 to the volume_set_id field and 128 bytes with value 0x20
to the the following 37-byte fields in the PVD:
copyright_file_id, abstract_file_id, bibliographic_file_id
This causes, among other unwanted results the reserved4 field to be
overwritten with the value 0x20. To comply with the specification,
this field muse be zero. As a result, all FreeBSD distribution
images created with makefs have not been 100% valid ISO9660 files.
http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=45217
Reviewed by: kientzle
Approved by: re (kib)
Obtained from: NetBSD
MFC after: 3 days
to be created. The support is based on mtree version 2.0, as used
in libarchive, but adds new features on top of it.
The current implementation is fully functional, but is envisioned
to grow at least the following additional features over time:
o Add support for the /include special command so that manifest
files can be constructed using includable fragments.
o Add support specifying a search path to locate content files.
o Content file filters: commands that provide file contents on
stdout.
The manifest file eliminates the need to first construct a tree
as root in order to create an image and allows images (releases)
to be created directly from object trees and/or source trees.
Reviewed by: deo
Sponsored by: Juniper Networks, Inc
and usr.sbin/makefs/ffs/ffs_subr.c as they have no need of anything in that
file. No other programs or libraries include <ufs/ffs/ffs_extern.h> (nor
should they as it is totally in-kernel interfaces). For added protection
I enclosed the entire contents of <ufs/ffs/ffs_extern.h> in ifdef _KERNEL.
Feedback from: Bruce Evans and Tai-hwa Liang
All firmwares can read it, we need to write one anyway for old OF
versions, and if a 512-byte one is present, OS X ignores the 2048-byte one
in favor of it.
bootstrap partitions from the ISO9660 boot catalog. This preserves OS X's
ability to mount the CD, while allowing us a way to provide HFS-ified
bootstrap code for Open Firmware.
include sys/time.h instead of time.h. This include is incorrect as
per the manpages for the APIs and the POSIX definitions. This commit
replaces sys/time.h where necessary with time.h.
The commit also includes some minor style(9) header fixup in newfs.
This commit is part of a larger effort by Garrett Cooper started in
//depot/user/gcooper/posix-conformance-work/ -- to make FreeBSD more
POSIX compliant.
Submitted by: Garrett Cooper yanegomi at gmail dot com
brings in support for an optional intent log which eliminates the need
for background fsck on unclean shutdown.
Sponsored by: iXsystems, Yahoo!, and Juniper.
With help from: McKusick and Peter Holm
