Commit Graph

127 Commits

Author SHA1 Message Date
Sevan Janiyan
a7b3f36393 Fix null dereference when running out of nodes during rename.
PR:		212370
Submitted by:	Sascha Wildner <swildner AT gmail>
Reviewed by:	bcr (mentor)
Approved by:	bapt
Obtained from:	NetBSD
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D8643
2016-11-26 13:26:29 +00:00
Marcel Moolenaar
4ad8d73326 Assign a random number to di_gen (for FFS), instead of extracting it
from struct stat.  We don't necessarily have permissions to see the
generation number and the host OS may not have st_gen in struct stat
anyway.  Since the kernel assigns random numbers, there's nothing
meaningful about the generation that requires us to preserve it when
the file system image is created.  With this change, all generation
numbers come from random() and that makes it easier to add support
for reproducible builds at some time in the future (i.e. by adding
an argument to makefs that changes the behaviour of random() so that
it always returns 0 or some predictable sequence).

Differential Revision:	https://reviews.freebsd.org/D8418
2016-11-05 16:23:33 +00:00
Marcel Moolenaar
5d71efbecd Be more precise when including headers so that we're less likely to
depend on namespace pollution and as such become more portable. This
means including headers like <sys/types.h> or <stdlib.h>, but also
making sure we include system/host headers before local headers.

While here: define ENOATTR as ENOMSG in mtree.c. There is no ENOATTR
on Linux.

With this, makefs is ready for compilation on macOS and Linux.
2016-10-25 16:29:15 +00:00
Marcel Moolenaar
46783b12e3 Allow building makefs(8) from another Makefile (such as one in
a seperate directory hierarchy used to build tools). This boils
down to replacing the use of ${.CURDIR} with either ${SRCDIR}
or ${SRCTOP}. SRCDIR is defined as the directory in which the
Makefile lives that bmake(1) is currently reading. Use SRCTOP
when reaching outside of makefs's directory.
2016-10-25 16:21:38 +00:00
Ed Maste
71cc9a9f0e makefs: diff reduction with NetBSD
Obtained from:	NetBSD usr.sbin/makefs/cd9660.c 1.33
2016-10-07 19:12:15 +00:00
Ed Maste
881e506b36 makefs: sync NetBSD IDs with upstream for changes that we already have
May 22 21:51:39 2011 +0000 (christos):

    From Nathan Whitehorn (nwhitehorn at freebsd dot org):
    Add code to generate bootable ISOs on Powermac and CHRP systems.
    Synthesize some partition maps (APM and MBR, respectively) pointing
    to (a) the whole disk, and (b) relevant El Torito boot images that
    have been added by other code. These partition maps are a little
    bit funny looking, but they seem to work. FreeBSD has been using
    this successfully in their release generation on powerpc, as well
    as generating all non-SPARC install media. SPARC support could
    probably be added as an extension of this patch.

makefs.8 1.33

Tue Aug 23 17:09:11 2011 +0000 (christos):

    PR/45285: Martin Matuska: makefs does not properly convert ISO level 1 and 2
    filenames (buffer overflow)

    makefs does not properly verify the maximum filename length in the
    special "." case for both ISO level 1 and ISO level 2 filename
    conversion.  This creates broken images or causes a buffer overflow
    (ISO level 2).

    ISO level 1:
    If a filename contains only dots or up to 8 characters followed by
    dots the 8+3 limit check doesn't work.

    ISO level 2:
    If a filename contains a dot in the first 30 characters and a dot
    on the 30th character, the length limit check doesn't work and the
    buffer is overflowed.

    $ mkdir level1
    $ touch level1/12345............
    $ makefs -t cd9660 -o isolevel=1 test.iso level1

    $ mkdir level2
    $ touch level2/1234567890.2345678901234567.....34567890123456789012345
    $ makefs -t cd9660 -o isolevel=2 test.iso level2

cd9660.c 1.32

Sun Oct 9 21:33:43 2011 +0000 (christos):

    add support for setting the ufs label. (Nathan Whitehorn)

ffs.c 1.45
ffs.h 1.2
mkfs.c 1.22
makefs.8 1.37

Obtained from:	NetBSD
2016-07-19 18:40:54 +00:00
Ed Maste
298d081c1b makefs: reorder 'usage' alphabetically
From NetBSD, Mon Aug 15 14:45:01 2011 +0000 (wiz)

    Re-order `usage' alphabetically;
    rename option arguments in the manpage's `SYNOPSIS' section to
    match those from `usage' (not the other way around; the `usage'-line
    (and other parts of makefs.c) contain the correct names);
    minor punctuation improvements.

    From Snader_LB.

makefs.8 1.36
makefs.c 1.30

Obtained from:	NetBSD
2016-07-19 18:15:22 +00:00
Ed Maste
7b03d164da makefs: Provide a -T option to set timestamps to a consistent value
This is taken from the NetBSD versions listed below and adapted to the
makefs version in FreeBSD, along with a bug fix from cem@ that will be
sent to NetBSD.

Reviewed by:	pfg
Approved by:	re (gjb)
Obtained from:	NetBSD
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D6835
2016-06-14 14:03:28 +00:00
Simon J. Gerraty
1c8bdd84b8 Initialize date of rootNode.
Reviewed by:	jtl
2016-05-18 22:24:52 +00:00
Pedro F. Giffuni
00b8e9fe52 makefs(8): Clarify the comment concerning seeding.
Avoid giving the impression makefs currently supports reproduceable
builds.
2016-05-18 00:22:52 +00:00
Pedro F. Giffuni
40d39b4bf5 makefs(1): use all the random(3) range.
The generation number is uint32_t so we can fit the complete range
of random(3). We could have used arc4random() but the result would
be unpredictable and it would prohibit reproducible builds.

While here add a comment where seeding is done: this affects
reproducible builds and might have to be re-visited to use a
release dependent value.

MFC after:	2 weeks
2016-05-17 18:20:33 +00:00
Pedro F. Giffuni
9f36c1fa71 makefs(8): use NULL instead of zero for pointers. 2016-05-17 16:58:39 +00:00
Don Lewis
e29182f6c0 Fix an off by one error to avoid overflowing rp[].
Reported by:	Coverity
CID:		1007579
2016-05-16 16:16:46 +00:00
Conrad Meyer
16f6b6f174 ffs_bswap: Copy one UFS dinode member at a time
No functional change.

Reported by:	Coverity
CIDs:		974635, 974636, 977396, 977397, 977398, 977399
Sponsored by:	EMC / Isilon Storage Division
2016-05-11 16:42:13 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Marcelo Araujo
a8945bbf62 Use MIN/MAX macros from sys/param.h.
MFC after:	2 weeks.
2016-05-02 01:47:13 +00:00
Marcelo Araujo
bb0bd931dc The malloc will return NULL if it can't allocate memory.
MFC after:	2 weeks.
2016-04-18 05:19:45 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Enji Cooper
0bd9b70036 Require /sbin/mount_cd9660 when running the cd9660 tests
In some cases the test system might not have mount_cd9660(8). Don't
implicitly rely on it while testing cd9660 support; explicitly rely
on it

MFC after: 1 week
Reported by: mjohnston
Sponsored by: EMC / Isilon Storage Division
2016-01-27 06:14:20 +00:00
Enji Cooper
5e25b19e29 Fix getopt(3) argument after r290180; I forgot to change -r to -R
by accident

MFC after: 3 days
Pointyhat to: ngie
Reported by: vangyzen
Sponsored by: EMC / Isilon Storage Division
2015-12-29 20:17:40 +00:00
Ed Maste
203b6f692c makefs: use ENTRY macro for diff reduction with NetBSD
Sponsored by:	The FreeBSD Foundation
2015-12-21 16:12:41 +00:00
Ed Maste
6d90cbf05d Remove clauses 3 and 4 from makefs newfs_extern.h
Obtained from:	NetBSD
2015-11-19 02:39:46 +00:00
Enji Cooper
042c83fdc8 Bump .Dd 2015-11-09 09:39:59 +00:00
Enji Cooper
daf5e00077 Follow up to r290259 dealing with makefs(8)
- Don't use contractions (don't -> do not)
- Change "throw away" to "discard" when describing the -o keep-bad-images option
- Revert author e-mail split I brought over from NetBSD, effectively reverting
  the change bapt made in r267668

MFC after: 3 days
Submitted by: bjk
Sponsored by: EMC / Isilon Storage Division
2015-11-09 09:39:07 +00:00
Enji Cooper
126b73dd8b Add testcases for -t ffs -o version=[12]
Verify the filesystem type using dumpfs. Add preliminary support
for NetBSD (needs to be validated)

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-11-02 11:06:51 +00:00
Enji Cooper
fc062a4bd7 Sync minor whitespace / type changes in ffs_csum_swap and ffs_sb_swap with
src/sys/ufs/ffs/ffs_bswap.c@1.39

Obtained from: NetBSD
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-11-02 10:09:09 +00:00
Enji Cooper
c213f0dbec Clean up mtree keyword support a slight bit and add a few more default keywords
- Parameterize the mtree keywords as $DEFAULT_MTREE_KEYWORDS
- Test with the extra mtree keywords, `mode,gid,uid`.
- Add a note about mtrees with time support not working with makefs right now

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-11-02 10:08:00 +00:00
Enji Cooper
10bd272504 Add testcases for -t cd9660 -o isolevel=[1-3]
-- -o isolevel=1 currently fails because of path comparison issues,
   so mark it as an expected failure.
-- -o isolevel=3 is not implemented, so expect it to fail as an out
   of bounds value [*].

PR: 203645
MFC after: 1 week
X-MFC with: r290264
Sponsored by: EMC / Isilon Storage Division
2015-11-02 09:16:51 +00:00
Enji Cooper
bc06965ace Limit isoLevel to 1 and 2 to avoid segfaulting when isoLevel is set to 3 by
dereferencing a NULL function pointer

Add some asserts to ensure that isolevel is always either 1 or 2.

PR: 203645
Reported by: Thomas Schmitt <scdbackup@gmx.net>
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-11-02 08:43:15 +00:00
Enji Cooper
eac300b82d Fix spelling of isolevel cd9660 option
MFC after: 1 week
X-MFC with: r290260
Sponsored by: EMC / Isilon Storage Division
2015-11-02 07:46:47 +00:00
Enji Cooper
3d765991ed Bump .Dd for r290259 and r290260 2015-11-02 07:42:42 +00:00
Enji Cooper
6dbfd3d67c Document undocumented long options for -t cd9660
Note which options have been implemented and which options haven't
been implemented

Submitted as the following NetBSD PRs: bin/50390 and bin/50392

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-11-02 07:40:18 +00:00
Enji Cooper
93c2e8006a Sync makefs(8) content a bit with src/usr.sbin/makefs/makefs.8@1.53
Sections involving unimplemented filesystems (chfs, msdosfs, udf, v7fs) and
options have been omitted.

MFC after: 1 week
Obtained from: NetBSD
Sponsored by: EMC / Isilon Storage Division
2015-11-02 07:36:42 +00:00
Enji Cooper
78d2334f0d Follow up to roundup feature addition in r289203
- Rename -r to -R to avoid the clash with makefs -r in NetBSD
- Note that -R is an FFS-specific option because it's not implemented
  in cd9660 today
- Rename the roundup variable to "roundup-size" in the manpage and help
  text for consistency with other variables.
- Bump .Dd (missed in r289203)

PR: 203707
MFC after: 1 week
X-MFC with: r289203
Differential Revision: https://reviews.freebsd.org/D3959
Reviewed by: adrian (earlier patch), emaste
Sponsored by: EMC / Isilon Storage Division
2015-10-30 05:55:56 +00:00
Enji Cooper
dc50fb3673 Add a regression test for r289899 to validate rockridge encoding
of device types

X-MFC with: r289899
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-10-24 23:21:08 +00:00
Enji Cooper
fb9ab5ec6b Remove an ls -l I was using for debugging
MFC after: 2 weeks
X-MFC with: r289897
Sponsored by: EMC / Isilon Storage Division
2015-10-24 23:19:24 +00:00
Enji Cooper
fc002adafd Import the fix from NetBSD kern/48852 (sic) to fix rockridge encoding of
device nodes

In particular, use st_rdev (the device type), not st_dev (the device inode),
and fix the comparison to be correct with the st_rdev field

Bug 203648
MFC after: 2 weeks
Submitted by: Thomas Schmitt <scdbackup@gmx.net>
Coverity CID: 1008927
Sponsored by: EMC / Isilon Storage Division
2015-10-24 23:16:13 +00:00
Enji Cooper
fc2b053656 Add more cd9660/FFS makefs testcases
General changes:
- Parameterize out the mount command.
- Use mtree to verify the contents of an image (check_image_contents) instead
  of using diff (diff verifies content, but not file metadata).
- Move common logic out to functions (common_cleanup, mount_image,
  check_image_contents)
- Add stub testcases for makefs -D (crashes with SIGBUS, similar to bug # 192839)
- Add a note about the ISO-9660 and rockridge specs
- Add testcases that exercise:
-- Creating disk images from an mtree and multiple directories.
-- -F flag use (not really an extensive testcase right now)

cd9660-specific test changes:

- Remove an XXX comment about symlinks; I forgot that non-rockridge images turn
  symlinks into hardlinks.
- Add testcases that exercise:
-- -o allow-deep-trees
-- -o allow-max-name stub testcase (doesn't seem to be implemented in makefs)
-- -o preparer (existence in image; not conformance to spec)
-- -o publisher (existence in image; not conformance to spec)
-- -o rockridge (basic)

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-10-24 22:12:23 +00:00
Enji Cooper
66845bf5c5 Revise "create_test_inputs" to simplify the file structure as
these testcases don't need to be nested as much as bin/ls/ls_tests.sh
do when verifying ls -a, ls -A, etc. This allows the tests to make
all paths relative to the top of the temporary directory instead of
always tacking on $ATF_TMPDIR, thus complicating things unnecessarily

Create non-empty files in create_test_inputs as well now, similar to
create_test_inputs2 in bin/ls/ls_tests.sh

Compare the input files to the output file contents using diff where
possible:
- Skip over the fifo comparison for now because it always fails
- Skip over the symlink comparison on cd9660 because it always fails
  today

Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks
X-MFC with: r289739
2015-10-22 07:22:05 +00:00
Enji Cooper
7bdfc3b75e Correctly reintroduce the rudimentary smoke tests I botched up
in r289684

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-10-22 05:32:45 +00:00
Enji Cooper
6cf8c0fe51 Revert r289694
I committed some other undesirable local changes by accident
2015-10-21 13:15:04 +00:00
Enji Cooper
b4c3ec123e Add some rudimentary [smoke] testcases for makefs
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-10-21 13:13:38 +00:00
Enji Cooper
42afd55d1e Unbreak makefs -t cd9660 after r289687
buffer_head needs to be freed -- not buffer

Detected by jemalloc, i.e. running makefs failed the arena assert
because my copy of malloc on CURRENT is compiled with the default
!MALLOC_PRODUCTION asserts on

Pointyhat to: ngie
PR: 203647
X-MFC with: r289687
Sponsored by: EMC / Isilon Storage Division
2015-10-21 12:54:15 +00:00
Enji Cooper
b73321f086 Free buffer before returning from cd9660_write_path_table to avoid
leaking it after returning from the function

MFC after: 1 week
PR: 203647
Submitted by: Thomas Schmitt <scdbackup@gmx.net>
Coverity CID: 978431
Sponsored by: EMC / Isilon Storage Division
2015-10-21 11:38:48 +00:00
Eitan Adler
463a577b27 Fix a ton of speelling errors
arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337
2015-10-21 05:37:09 +00:00
Enji Cooper
d2a489eb1a Don't check if val is NULL before calling free; free(3) already
handles this

MFC after: 1 week
PR: 203649
Submitted by: Thomas Schmitt <scdbackup@gmx.net>
Coverity CID: 1305659
Sponsored by: EMC / Isilon Storage Division
2015-10-19 18:45:14 +00:00
Sean Bruno
24484ca4bf makefs(8) leaves sblock.fs_providersize uninitialized (zero) that can be easily
checked with dumpfs(8). This may lead to other problems, f.e. geom_label kernel
module sanity checks do not like zero fs_old_size value and skips such UFS1
file system while tasting (fs_old_size derives from sblock.fs_providersize).

PR:	203704
Submitted by:	eugen@grosbein.net
Reviewed by:	marcel
2015-10-13 17:00:14 +00:00
Adrian Chadd
a08b904c5e makefs: introduce a new option to specify what to round the resulting
image up to.

From ticket:

While trying to run FreeBSD/mips on some device having very small flash media,
one is forced to compress file system with mkulzma(8) utility. It is desirable
to specify small UFS block/fragment sizes like 4096/512 bytes for makefs(8)
and big compression block size like 65535 bytes to mkulzma at the same time.
Then one obtains very good comression ratios (like 75% and more) but faces
the following problem.

geom_uncompress kernel module reports GEOM provider size rounded up to its
compression block size. Generally, this changes original media size and now
it fails to match the size of embedded UFS file system that leads to other
problems, f.e. geom_label kernel module does not like this and skips the
file system while tasting the GEOM and looking for UFS label.

This makes it impossible to refer to the file system using known UFS label
instead of something like /dev/map/rootfs.uncompress.

The following patch introduces new command line option "-r roundup" for makefs
that makes it round up the image to specified block size. Hence, geom_uncompress
does not change GEOM media size for images rounded that way and geom_label
accepts such GEOMs just fine.

With the patch applied, one can use following commands:

$ makefs -t ffs -r 65536 -o bsize=4096,fsize=512,label=flash optimization=space fs.img fs
$ mkulzma -s 65536 -o fs.img.ulzma fs.img

PR:		bin/203707
Submitted by:	<eugen@grosbein.net>
2015-10-13 02:32:15 +00:00