Commit Graph

95 Commits

Author SHA1 Message Date
Warner Losh
c30dcf40ba Add a new flag to mkimg (-a num) to specify the active partition for
those partitioning schemes that have this concept. Implement it as an
override for mbr's setting 0x80 in the flags for the first partition
when we have boot code.

Differential Revision: https://reviews.freebsd.org/D4403
2016-10-18 05:43:12 +00:00
Marcel Moolenaar
4299711473 o Provide a private definition for UUIDs (mkimg_uuid_t) because
UUIDs are not portable.
 o  Move mkimg_uuid() to a new file and merge both gpt_uuid_enc()
    and vhd_uuid_enc() into a single mkimg_uuid_enc() that lives
    in the same file.
 o  Move the OS-specific implementation of generating a UUID to
    osdep_uuidgen() and provide the implementations for FreeBSD,
    macOS and Linux.
 o  Expect the partitioning scheme headers to be found by having
    a search to the directory in which the headers live. This
    avoids conflicts on non-FreeBSD machines.
2016-10-18 01:55:07 +00:00
Marcel Moolenaar
8d249f5e2f Switch to using the portable partition scheme headers. 2016-10-16 02:55:52 +00:00
Marcel Moolenaar
4ac1e8d30f Replace OFF_MAX with INT64_MAX. The former is defined on Linux. 2016-10-03 04:00:30 +00:00
Marcel Moolenaar
6b1235713e Prefer <stdint.h> over <sys/types.h>. While here remove redundant
inclusion of <sys/queue.h>.

Move the inclusion of the disk partitioning headers out of order
and inbetween standard headers and local header. They will change
in a subsequent commit.
2016-10-03 02:37:28 +00:00
Marcel Moolenaar
1080fb197b Replace STAILQ with TAILQ. TAILQs are portable enough that they can
be used on both macOS and Linux. STAILQs are not. In particular,
STAILQ_LAST does not next on Linux. Since neither STAILQ_FOREACH_SAFE
nor TAILQ_FOREACH_SAFE exist on Linux, replace its use with a regular
TAILQ_FOREACH. The _SAFE variant was only used for having the next
pointer in a local variable.
2016-10-03 01:46:47 +00:00
Marcel Moolenaar
7b5a53ea4d Portability changes:
1.  macOS nor Linux have MAP_NOCORE nor MAP_NOSYNC. Define as 0.
2.  macOS doesn't have SEEK_DATA nor SEEK_HOLE. Define as -1
    so that lseek will return -1 (with errno set to EINVAL).
3.  gcc correctly warns that error is assigned but not used in
    image_copyout_region().  Fix by returning on the first error.
2016-09-26 04:14:00 +00:00
Marcel Moolenaar
5aad7d9a87 Avoid depending on the <sys/endian.h> header for le*enc and be*enc.
Not only is the header unportable, the encoding/decoding functions
are as well.  Instead, duplicate the handful of small inlines we
need into a private header called endian.h.

Aside: an alternative approach is to move the encoding/decoding
functions to a separate system header.  While the header is still
nonportable, such an approach would make it possible to re-use the
definitions by playing games with include paths. This may be the
preferred approach if more (build) utilities need this.  This
change does not preclude that.  In fact, it makes it easier.
2016-09-26 01:06:32 +00:00
Marcel Moolenaar
4039ea7c9b Eliminate the use of EDOOFUS. The error code was used to signal
programming errors, but is really a poor substitute for assert.
And less portable as well.
2016-09-26 00:41:08 +00:00
Marcel Moolenaar
5a1302ab2e Replace the use of linker sets with constructors for both the
formats and schemes.  Formats and schemes are registered at
runtime now, rather than collected at link time.
2016-09-25 22:57:59 +00:00
Marcel Moolenaar
ac3425511d Update local variable 'block' after calling capacity_resize(),
otherwise format_resize(), which is called right after, isn't
getting the current/actual image size. Rather than rounding up,
format_resize() could end up truncating the size and we don't
allow that by design.

MFC after:	1 week
2016-09-24 17:29:27 +00:00
Alan Somers
005156d7b3 Update mkimg(1) author's contact info
Reviewed by:	marcel, imp
MFC after:	3 days
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D7991
2016-09-21 18:42:17 +00:00
Pedro F. Giffuni
3af7c80523 mkimg(1): minor cleanups with argument order in calloc(3).
Generally the first argument in calloc is supposed to stand for a count
and the second for a size. Try to make that consistent. While here,
attempt to make some use of the overflow detection capability in
calloc(3).
2016-07-12 15:46:53 +00:00
Ed Maste
023075def4 mkimg: bump version to 20151211 after r292082
mkimg has had a number of functional additions after the last time the
version was incremented. Do so now, to r292082's commit date, so that
users can determine what is supported.

Reviewed by:	marcel
Approved by:	re (gjb)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6882
2016-06-17 18:49:42 +00:00
Mark Johnston
cc7f01a5da mkimg: Indicate that input file pages are unlikely to be reused.
mkimg(1) uses a swap file to back input file chunks. When the output file
is being written out, blocks of the swap file are mapped and their contents
copied. This causes the backing VM pages to enter the active queue, and when
the output file is large relative to system memory (as is generally the
case), can result in a shortfall of inactive memory. This causes the
pagedaemon to aggressively scan the active queue and swap out process
memory in an attempt to meet the shortfall. Because mkimg's input files
are typically the intermediate result of some build process, there's no
need to push them all through the active queue. Use madvise(2) to indicate
that the backing pages may be reclaimed in preference to active pages. In
the case of the swap file, these pages will be freed as soon as mkimg
exits anyway.

When using mkimg on a desktop-class system with large amounts of dirty
process memory, this change substantially improves mkimg runtime and
reduces swap usage.

Reviewed by:	marcel
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D6654
2016-06-01 02:30:06 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Warner Losh
4224509a36 Add ppcboot FAT type. Needed to create a bootable powerpc image.
Differential Review: https://reviews.freebsd.org/D4407
2015-12-11 05:39:42 +00:00
Bryan Drewery
b1f92fa229 META MODE: Update dependencies with 'the-lot' and add missing directories.
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so.  A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR.  These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package.  The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:23:19 +00:00
Eitan Adler
463a577b27 Fix a ton of speelling errors
arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337
2015-10-21 05:37:09 +00:00
Ed Maste
a4c8dbbd87 mkimg: support fat16b partitions (MBR type 06h)
Reviewed by:	marcel
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3894
2015-10-15 01:09:14 +00:00
Enji Cooper
b2d48be1bc Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
Christian Brueffer
99d43aee11 Fix a few mandoc warnings.
MFC after:	1 week
2015-08-12 10:34:05 +00:00
Marcel Moolenaar
8f1d6b6915 Fix typo introduced in previous commit.
Pointed out by: Nikolai Lifanov <lifanov at mail.lifanov.com>
2015-08-07 18:40:44 +00:00
Marcel Moolenaar
de93252622 o Fix a typo.
o  Describe the file formats mkimg can create.
2015-08-07 17:22:37 +00:00
Marcel Moolenaar
9286ca0227 Rebase after r286395: rounding fix for dynamic VHD 2015-08-07 04:35:43 +00:00
Marcel Moolenaar
c20f01c4ea Fix the dynamic VHD format to work with qemu. The size of the disk
is taken to match the geometry and only when the geometry is max'd
out, is the actual recorded size taken.

Note that qemu has the same logic for the fixed VHD format. However
that is known to conflict with Microsoft Azure, where the recorded
size of the image is what counts.

Pointed out by: gjb@
2015-08-07 04:27:51 +00:00
Marcel Moolenaar
ddaceed224 Make image_copyout_zeroes() an interface function. 2015-08-03 01:24:48 +00:00
Marcel Moolenaar
8adccff34f Add the ntfs alias and support it with the MBR and GPT schemes
as DOSPTYP_NTFS and GPT_ENT_TYPE_MS_BASIC_DATA (resp).
2015-06-27 03:28:04 +00:00
Marcel Moolenaar
186d96ca88 Rebase after r284658:
1.  Change creator OS to "wi2k"
2.  Bump tool version to 2.0
2015-06-21 02:55:25 +00:00
Marcel Moolenaar
8b37e7ca92 Microsoft Azure expects the creator OS to be "Wi2k" and not "FBSD".
The image is not accepted for provisioning otherwise. Bump the
VHD creator tool version and the version of mkimg to signify our
success in provisioning.

Note that this also imapcts the dynamic VHD images.

Tested by: gjb@
2015-06-21 02:45:31 +00:00
Marcel Moolenaar
a1fd05ee8a Rebase the fixed VHD images after the rounding fix. 2015-06-21 01:55:01 +00:00
Marcel Moolenaar
eebf77ccef Microsoft Azure demands that fixed VHD images are a whole number
of megabytes. This is on top of having the image rounded to the
matching geometry of the image size.
By rounding up to the next MB after rounding to the geometry, we
lost idempotency. Subsequent calls to resize the image will keep
increasing the image size.

Tested by: gjb@
2015-06-21 01:44:27 +00:00
Marcel Moolenaar
4537ca4ef6 Don't resize again prior to writing. Resizing may not be idempotent
and no scheme adjusts the size after the format resized the image
the first time.
2015-06-21 01:35:32 +00:00
Marcel Moolenaar
c9f3108294 Update tests after sizing changes to the fixed VHD format. 2015-06-11 15:19:59 +00:00
Marcel Moolenaar
f863f42be3 Handle the case in which ncyls is 0.
While here, update copyright.
2015-06-11 15:18:54 +00:00
Marcel Moolenaar
2697d1ee89 For the fixed VHD format, round the raw image size to the next
multiple of the cylinder size. This is what qemu-img seems to
be doing. Make sure to handle boundary cases where increasing
the image size by 1 cyclinder's worth would also result in a
change of geometry.
2015-06-11 14:30:31 +00:00
Marcel Moolenaar
e6679c1a71 Add the -c option for specifying the capacity of the disk image. When
a capcity is given, no partitions are required. When no partitions are
given, no scheme needs to be specified either. This makes it possible
to create an entirely empty disk image. To add an empty partitioning
table, specify the scheme.

Bump the version to 20150222.
2015-02-22 04:50:47 +00:00
Marcel Moolenaar
aba885c960 Don't require a scheme if no partitions are given. Change the code
to handle that case. Note that we still require partitions, so the
change is effectively a no-op.
2015-02-22 01:20:49 +00:00
Marcel Moolenaar
b5380f2a09 In scheme_write(), don't overwrite the end parameter with the
return value of image_get_size(). As it so happens, the only
caller of scheme_write() passes exactly that.
2015-02-22 01:01:02 +00:00
Marcel Moolenaar
29eac8d790 Remove extraneous parenthesis in assignment. 2015-02-22 00:58:10 +00:00
Ed Maste
8fb02d8847 mkimg: Add MBR EFI partition type
Sponsored by:	The FreeBSD Foundation
2015-01-09 19:34:48 +00:00
Marcel Moolenaar
0f49f14626 The size of the first level reference count table is given in terms of the
number of clusters it occupies. It's not the number of entries in the table,
as it is for the L1 cluster table.

For small images, the two are the same. With the unit tests based on small
images, this change has therefore no effect on the unit test.  For larger
images (like the FreeBSD 10.1-RELEASE image), this gives a discrepancy that
actually shows up when running "qemu-img check".

Bump the version number of mkimg.

While here, fix a white-space bug.

MFC after: 1 week
2014-12-12 06:13:31 +00:00
Baptiste Daroussin
3e11bd9e2a Convert to usr.bin/ to LIBADD
Reduce overlinking
2014-11-25 14:29:10 +00:00
Edward Tomasz Napierala
65dd075b31 Add missing commas to .Xr.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-11-14 08:58:06 +00:00
Marcel Moolenaar
9ba57342c9 SEEK_DATA has interesting behaviour for sparse files on ZFS. A sparse file
with 128K of random data and truncated to 800K can have SEEK_DATA return -1
when given an offset of 128K. On UFS, the SEEK_DATA returns 800K (the size
of the file). SEEK_HOLE on ZFS seems to behave the same as UFS.

To handle this, map -1 to the size of the file (`end') when lseek returns
this for either SEEK_HOLE or SEEK_DATA. When sparse files are not supported
by the file system both `hole' and `data' will now be equal to `end' and we
will treat the entire file as data. This way, the -1 return for SEEK_DATA
on ZFS will end up doing the right thing.

Reported by: gjb@

MFC after:	3 days
2014-11-12 00:10:27 +00:00
Marcel Moolenaar
ea6d9dfc53 Update baseline files for EBR, MBR and PC98 now that mkimg fills in
the CHS fields appropriately when -T and -H are given on the command
line.
2014-10-03 20:54:35 +00:00
Marcel Moolenaar
2c83b36f45 Add mkimg_chs() for those schemes that need the LBA broken down into
cylinder, head and track numbers. Return ~0U for these values when
mkimg wasn't given both -T and -H (i.e. no geometry) or the cylinder
would be larger than the provided maximum.

Use mkimgs_chs() for the EBR, MBR and PC98 schemes to fill in the
appropriate fields. Make sure to use a "rounded" size so that the
partition is always a multiple of the track size. We reserved the
room for it in the metadata callback so that's a valid thing to
do.

Bump the mkimg version number.
While doing that again: have mkimg.o depend on the Makefile so that
a version change triggers a rebuild as needed.
2014-10-03 20:48:11 +00:00