From NetBSD christos Sat Jan 26 00:19:39 2013 +0000
make the buffer functions look exactly like the kernel ones and add other
cruft to make the kernel files compile.
ffs.c 1.54
ffs/buf.c 1.13
ffs/buf.h 1.3
ffs/ffs_alloc.c 1.21
ffs/ffs_balloc.c 1.15
Reviewed by: marcel, ngie
Obtained from: NetBSD
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8404
from struct stat. We don't necessarily have permissions to see the
generation number and the host OS may not have st_gen in struct stat
anyway. Since the kernel assigns random numbers, there's nothing
meaningful about the generation that requires us to preserve it when
the file system image is created. With this change, all generation
numbers come from random() and that makes it easier to add support
for reproducible builds at some time in the future (i.e. by adding
an argument to makefs that changes the behaviour of random() so that
it always returns 0 or some predictable sequence).
Differential Revision: https://reviews.freebsd.org/D8418
depend on namespace pollution and as such become more portable. This
means including headers like <sys/types.h> or <stdlib.h>, but also
making sure we include system/host headers before local headers.
While here: define ENOATTR as ENOMSG in mtree.c. There is no ENOATTR
on Linux.
With this, makefs is ready for compilation on macOS and Linux.
a seperate directory hierarchy used to build tools). This boils
down to replacing the use of ${.CURDIR} with either ${SRCDIR}
or ${SRCTOP}. SRCDIR is defined as the directory in which the
Makefile lives that bmake(1) is currently reading. Use SRCTOP
when reaching outside of makefs's directory.
May 22 21:51:39 2011 +0000 (christos):
From Nathan Whitehorn (nwhitehorn at freebsd dot org):
Add code to generate bootable ISOs on Powermac and CHRP systems.
Synthesize some partition maps (APM and MBR, respectively) pointing
to (a) the whole disk, and (b) relevant El Torito boot images that
have been added by other code. These partition maps are a little
bit funny looking, but they seem to work. FreeBSD has been using
this successfully in their release generation on powerpc, as well
as generating all non-SPARC install media. SPARC support could
probably be added as an extension of this patch.
makefs.8 1.33
Tue Aug 23 17:09:11 2011 +0000 (christos):
PR/45285: Martin Matuska: makefs does not properly convert ISO level 1 and 2
filenames (buffer overflow)
makefs does not properly verify the maximum filename length in the
special "." case for both ISO level 1 and ISO level 2 filename
conversion. This creates broken images or causes a buffer overflow
(ISO level 2).
ISO level 1:
If a filename contains only dots or up to 8 characters followed by
dots the 8+3 limit check doesn't work.
ISO level 2:
If a filename contains a dot in the first 30 characters and a dot
on the 30th character, the length limit check doesn't work and the
buffer is overflowed.
$ mkdir level1
$ touch level1/12345............
$ makefs -t cd9660 -o isolevel=1 test.iso level1
$ mkdir level2
$ touch level2/1234567890.2345678901234567.....34567890123456789012345
$ makefs -t cd9660 -o isolevel=2 test.iso level2
cd9660.c 1.32
Sun Oct 9 21:33:43 2011 +0000 (christos):
add support for setting the ufs label. (Nathan Whitehorn)
ffs.c 1.45
ffs.h 1.2
mkfs.c 1.22
makefs.8 1.37
Obtained from: NetBSD
From NetBSD, Mon Aug 15 14:45:01 2011 +0000 (wiz)
Re-order `usage' alphabetically;
rename option arguments in the manpage's `SYNOPSIS' section to
match those from `usage' (not the other way around; the `usage'-line
(and other parts of makefs.c) contain the correct names);
minor punctuation improvements.
From Snader_LB.
makefs.8 1.36
makefs.c 1.30
Obtained from: NetBSD
This is taken from the NetBSD versions listed below and adapted to the
makefs version in FreeBSD, along with a bug fix from cem@ that will be
sent to NetBSD.
Reviewed by: pfg
Approved by: re (gjb)
Obtained from: NetBSD
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D6835
The generation number is uint32_t so we can fit the complete range
of random(3). We could have used arc4random() but the result would
be unpredictable and it would prohibit reproducible builds.
While here add a comment where seeding is done: this affects
reproducible builds and might have to be re-visited to use a
release dependent value.
MFC after: 2 weeks
after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
In some cases the test system might not have mount_cd9660(8). Don't
implicitly rely on it while testing cd9660 support; explicitly rely
on it
MFC after: 1 week
Reported by: mjohnston
Sponsored by: EMC / Isilon Storage Division
- Don't use contractions (don't -> do not)
- Change "throw away" to "discard" when describing the -o keep-bad-images option
- Revert author e-mail split I brought over from NetBSD, effectively reverting
the change bapt made in r267668
MFC after: 3 days
Submitted by: bjk
Sponsored by: EMC / Isilon Storage Division
Verify the filesystem type using dumpfs. Add preliminary support
for NetBSD (needs to be validated)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- Parameterize the mtree keywords as $DEFAULT_MTREE_KEYWORDS
- Test with the extra mtree keywords, `mode,gid,uid`.
- Add a note about mtrees with time support not working with makefs right now
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
-- -o isolevel=1 currently fails because of path comparison issues,
so mark it as an expected failure.
-- -o isolevel=3 is not implemented, so expect it to fail as an out
of bounds value [*].
PR: 203645
MFC after: 1 week
X-MFC with: r290264
Sponsored by: EMC / Isilon Storage Division
dereferencing a NULL function pointer
Add some asserts to ensure that isolevel is always either 1 or 2.
PR: 203645
Reported by: Thomas Schmitt <scdbackup@gmx.net>
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
Note which options have been implemented and which options haven't
been implemented
Submitted as the following NetBSD PRs: bin/50390 and bin/50392
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- Rename -r to -R to avoid the clash with makefs -r in NetBSD
- Note that -R is an FFS-specific option because it's not implemented
in cd9660 today
- Rename the roundup variable to "roundup-size" in the manpage and help
text for consistency with other variables.
- Bump .Dd (missed in r289203)
PR: 203707
MFC after: 1 week
X-MFC with: r289203
Differential Revision: https://reviews.freebsd.org/D3959
Reviewed by: adrian (earlier patch), emaste
Sponsored by: EMC / Isilon Storage Division
device nodes
In particular, use st_rdev (the device type), not st_dev (the device inode),
and fix the comparison to be correct with the st_rdev field
Bug 203648
MFC after: 2 weeks
Submitted by: Thomas Schmitt <scdbackup@gmx.net>
Coverity CID: 1008927
Sponsored by: EMC / Isilon Storage Division
General changes:
- Parameterize out the mount command.
- Use mtree to verify the contents of an image (check_image_contents) instead
of using diff (diff verifies content, but not file metadata).
- Move common logic out to functions (common_cleanup, mount_image,
check_image_contents)
- Add stub testcases for makefs -D (crashes with SIGBUS, similar to bug # 192839)
- Add a note about the ISO-9660 and rockridge specs
- Add testcases that exercise:
-- Creating disk images from an mtree and multiple directories.
-- -F flag use (not really an extensive testcase right now)
cd9660-specific test changes:
- Remove an XXX comment about symlinks; I forgot that non-rockridge images turn
symlinks into hardlinks.
- Add testcases that exercise:
-- -o allow-deep-trees
-- -o allow-max-name stub testcase (doesn't seem to be implemented in makefs)
-- -o preparer (existence in image; not conformance to spec)
-- -o publisher (existence in image; not conformance to spec)
-- -o rockridge (basic)
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
these testcases don't need to be nested as much as bin/ls/ls_tests.sh
do when verifying ls -a, ls -A, etc. This allows the tests to make
all paths relative to the top of the temporary directory instead of
always tacking on $ATF_TMPDIR, thus complicating things unnecessarily
Create non-empty files in create_test_inputs as well now, similar to
create_test_inputs2 in bin/ls/ls_tests.sh
Compare the input files to the output file contents using diff where
possible:
- Skip over the fifo comparison for now because it always fails
- Skip over the symlink comparison on cd9660 because it always fails
today
Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks
X-MFC with: r289739
buffer_head needs to be freed -- not buffer
Detected by jemalloc, i.e. running makefs failed the arena assert
because my copy of malloc on CURRENT is compiled with the default
!MALLOC_PRODUCTION asserts on
Pointyhat to: ngie
PR: 203647
X-MFC with: r289687
Sponsored by: EMC / Isilon Storage Division
leaking it after returning from the function
MFC after: 1 week
PR: 203647
Submitted by: Thomas Schmitt <scdbackup@gmx.net>
Coverity CID: 978431
Sponsored by: EMC / Isilon Storage Division
checked with dumpfs(8). This may lead to other problems, f.e. geom_label kernel
module sanity checks do not like zero fs_old_size value and skips such UFS1
file system while tasting (fs_old_size derives from sblock.fs_providersize).
PR: 203704
Submitted by: eugen@grosbein.net
Reviewed by: marcel
image up to.
From ticket:
While trying to run FreeBSD/mips on some device having very small flash media,
one is forced to compress file system with mkulzma(8) utility. It is desirable
to specify small UFS block/fragment sizes like 4096/512 bytes for makefs(8)
and big compression block size like 65535 bytes to mkulzma at the same time.
Then one obtains very good comression ratios (like 75% and more) but faces
the following problem.
geom_uncompress kernel module reports GEOM provider size rounded up to its
compression block size. Generally, this changes original media size and now
it fails to match the size of embedded UFS file system that leads to other
problems, f.e. geom_label kernel module does not like this and skips the
file system while tasting the GEOM and looking for UFS label.
This makes it impossible to refer to the file system using known UFS label
instead of something like /dev/map/rootfs.uncompress.
The following patch introduces new command line option "-r roundup" for makefs
that makes it round up the image to specified block size. Hence, geom_uncompress
does not change GEOM media size for images rounded that way and geom_label
accepts such GEOMs just fine.
With the patch applied, one can use following commands:
$ makefs -t ffs -r 65536 -o bsize=4096,fsize=512,label=flash optimization=space fs.img fs
$ mkulzma -s 65536 -o fs.img.ulzma fs.img
PR: bin/203707
Submitted by: <eugen@grosbein.net>
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
the minimum image size specified is always less than the maximum
image size. If makefs(1) is invoked specifying minimum image size,
but not maximum one, the program exits with an error. Example:
# sudo -E makefs -M 538968064 -B be /home/davide/disk.img $DESTDIR
makefs: `/home/davide/tftproot/mips' minsize of 538968064 rounded up
to ffs bsize of 8192 exceeds maxsize 0. Lower bsize, or round the
minimum and maximum sizes to bsize.
Assert then that minsize < maxsize iff maxsize is specified.
This change allows me to build MIPS images using makefs(1) and following
what specified in the wiki again.
Reviewed by: jmallett, ngie
1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.
We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.
Reported by: kib
This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.
Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]
Mark known build failures as NO_PIE for now.
The only known runtime failure was rtld.
[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]
Put the superblock in the correct possition for UFS2... There is a bug
in FFS that if we don't put it here (for UFS2), it will forcefully
relocate the superblock, and I believe cause data loss..
I have a fix for that, but w/ how many releases are broken, we won't be
able to switch to the better _FLOPPY (block 0) for this for a while..
i. e. the POSIX:5.6.1 st_ino field, which can be used to detect hard links
in the file system. This is also the default in mkisofs(8) and according to
its man page, no system only being able to cope with Rock Ridge version 1.10
is known to exist.
PR: 185138
Submitted by: Kurt Lidl
MFC after: 1 week
would exceed the maximum size. This can be a difficult problem to diagnose
if one is, for instance, using -s with a fixed size in a script and the bsize
calculated for a filesystem image changes, necessitating a re-rounding of the
image size or a hand-setting of the bsize. Previously one would get a
cryptic message about how the size exceeded the maximum size, which normally
only happens if the contents of the image are larger than specified.
rather than NetBSD.
- Correctly set the Expiration Time in the Primary Volume Descriptor;
according to ISO 9660 8.4.26.1 unspecified date and time are denoted
by the digit 0 in RBP 1 to 16 but the number 0 in RBP 17. [1]
- Merge iso9660_rrip.c rev. 1.11 from NetBSD: name_len should be read
as unsigned byte. [2]
Note: This is according to ISO 9660 9.1.10.
- Rock Ridge TF entries should use a length of 5, because after the 4
bytes of generic SUSP header there is one byte of flags. See typedef
of ISO_RRIP_TF in iso9660_rrip.h. [1]
Submitted by: Thomas Schmitt [1]
Obtained from: NetBSD [2]
MFC after: 3 days
is picked up when defined. Previously, ffs_subr.c was always built without
support for opposite endianess as it doesn't include makefs.h on its own.
MFC after: 3 days
The cd9660 file system uses that field for the link count and it
was 0. This impacts pwd_mkdb(8) as it checks for st_nlink not being
0 as part of closing a race.
