Commit Graph

119 Commits

Author SHA1 Message Date
Oleksandr Tymoshenko
e331252340 Fix invalid VHDX generation for image larger than 4Gb
- Part of BAT payload location was lost due to invalid
    BAT entry encoding type (32 bits instead of 64 bits)
- The sequence of PB/SB entries in BAT was broken due to
    off-by-one index check. It worked for smaller than
    4Gb because there were no SB entries in BAT.

MFC after:	1 day
2020-07-13 02:24:31 +00:00
Baptiste Daroussin
5b990a9463 Revert r362466
Such change should not have happen without prior discussion and review.

With hat:	transitioning core
2020-06-22 07:46:24 +00:00
Hans Petter Selasky
7747001b12 Improve wording to be more precise and clear.
No functional change intended.

s/Master Boot/Main Boot/ (also called MBR)

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-06-21 13:34:08 +00:00
Mateusz Piotrowski
19870105e1 Use Fl instead of Ar for long flags
Also, bump date after r361935.

MFC after:	1 week
2020-06-08 21:11:34 +00:00
Oleksandr Tymoshenko
c78cd98b8a Add VHDX support to mkimg(1)
VHDX is the successor of Microsoft's VHD file format. It increases
maximum capacity of the virtual drive to 64TB and introduces features
to better handle power/system failures.

VHDX is the required format for 2nd generation Hyper-V VMs.

Reviewed by:	marcel
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D25184
2020-06-08 20:28:32 +00:00
Alex Richardson
a963845304 Allow bootstrapping mkimg on macOS/Linux
On these systems the (u)int64_t typedefs will not be implicitly defined by the
previous includes, so include <stdint.h> in the header that uses uint64_t.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D23202
2020-01-16 14:14:55 +00:00
Marcel Moolenaar
370d612f5e Don't compress and uuencode the "hexdump -C" output files. Just
save them with the $FreeBSD$ tag prepended.  Changes to these
files are now a lot easier to comprehend, which makes diffs also
reviewable.
2019-03-09 02:03:07 +00:00
Marcel Moolenaar
835adc6898 Round # partitions up to fill the last GPT table sector
Set the number of partitions entries in the GPT header to a
multiple of the number of entries that fit in a sector.

PR:		236238
Reviewed by:	imp
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D19465
2019-03-05 22:55:33 +00:00
Xin LI
4dbd43cd05 Remove unneeded headers.
MFC after:	1 month
2019-01-06 20:39:23 +00:00
Benno Rice
10e155c08a Fix a conditional that got mucked up.
Sponsored by:	iXsystems, Inc.
2018-04-12 17:16:13 +00:00
Benno Rice
459a61fd2d Add the ability to specify absolute and relative offsets to size partitions.
To create hybrid boot media we want to specify a partition at a known location.
This extends the syntax of size partitions to include an optional offset that
can be absolute or relative. It also introduces validation to make sure that
this hasn't resulted in overlapping partitions. I haven't added this to the
file and process partition specifications yet but the mechanics are designed
such that if someone comes up with a good way of specifying the offset it
will be fairly easy to add in.

Reviewed by:	imp
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14916
2018-04-12 15:47:47 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Enji Cooper
d511b20a69 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Enji Cooper
c0c5c2f45a Update the usr.bin/mkimg golden test output files after ^/head@r319125
^/head@r319125 changed the location of the backup pmbr, requiring the
output files to be regenerated, since they're binary disk dumps.

The output files were regenerated with "make rebase"--fixed in
^/head@r319294.

MFC with:	r319125, r319294
PR:		219673
Sponsored by:	Dell EMC Isilon
2017-05-31 08:01:12 +00:00
Enji Cooper
301441dcd8 Fix "make rebase" after ^/head@r315776
"make rebase" can be used for rebasing the output files from mkimg
after making a change to mkimg. This will come in handy soon, per
bug 219673.

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-05-31 07:49:49 +00:00
Enji Cooper
605b213ee1 Formalize the dependent/dependency relationship for <foo>.gz.uu vs <foo>
This helps ensure that the output files are regenerated if the input files
change, after the output files have been created.

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-05-31 07:42:14 +00:00
Emmanuel Vadot
6b14aecae7 mkimg: Correct an off by one error in the PMBR size
The PMBR last sector should be number of sector - 1 (As stated in UEFI Spec
2.6 page 118 table 17).
This fixes warning printed by linux tools like parted or fdisk.

Sponsored by:	Gandi.net
2017-05-29 12:51:02 +00:00
Emmanuel Vadot
2dcfca637d mkimg: Add -C argument to specify maximum capacity
Add a -C option to specify a maximum capacity for the final image file.
It is useful to control the size of the generated image for sdcard or
when we will add dynamic size partition.

Add --capacity which is a shorthand to define min and max capacity at
the same time.

Reviewed by:	bapt, marcel, wblock (manpages)
Sponsored by:	Gandi.net
Differential Revision:	https://reviews.freebsd.org/D10509
2017-05-10 09:36:34 +00:00
Enji Cooper
458cbf0ae6 Rename tests from <foo> to <foo>_test to match the FreeBSD test suite
naming scheme

usr.bin/diff/diff_test was renamed to usr.bin/diff/netbsd_diff_test
to avoid collisions with the renamed FreeBSD test.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-23 03:28:24 +00:00
Pedro F. Giffuni
6613eaa42b mkimg(1): let calloc(3) do the multiplication.
Reviewed by:	marcel
MFC after:	1 week
2017-03-13 20:14:07 +00:00
Simon J. Gerraty
baf4abfc39 Allow building mkimg as cross-tool
For linux the mmap offset must also be page aligned, and we
need to disable macros like __FBSDID()

Change the linux osdep_uuidgen() to use more portable gettimeofday().

Reviewed by: marcel
2017-03-03 01:56:55 +00:00
Baptiste Daroussin
0e630e32be Add missing section in manpage reference
Reported by:	make manlint
MFC after:	2 days
2017-02-11 23:41:38 +00:00
Yoshihiro Takahashi
2b375b4edd Remove pc98 support completely.
I thank all developers and contributors for pc98.

Relnotes:	yes
2017-01-28 02:22:15 +00:00
Warner Losh
c30dcf40ba Add a new flag to mkimg (-a num) to specify the active partition for
those partitioning schemes that have this concept. Implement it as an
override for mbr's setting 0x80 in the flags for the first partition
when we have boot code.

Differential Revision: https://reviews.freebsd.org/D4403
2016-10-18 05:43:12 +00:00
Marcel Moolenaar
4299711473 o Provide a private definition for UUIDs (mkimg_uuid_t) because
UUIDs are not portable.
 o  Move mkimg_uuid() to a new file and merge both gpt_uuid_enc()
    and vhd_uuid_enc() into a single mkimg_uuid_enc() that lives
    in the same file.
 o  Move the OS-specific implementation of generating a UUID to
    osdep_uuidgen() and provide the implementations for FreeBSD,
    macOS and Linux.
 o  Expect the partitioning scheme headers to be found by having
    a search to the directory in which the headers live. This
    avoids conflicts on non-FreeBSD machines.
2016-10-18 01:55:07 +00:00
Marcel Moolenaar
8d249f5e2f Switch to using the portable partition scheme headers. 2016-10-16 02:55:52 +00:00
Marcel Moolenaar
4ac1e8d30f Replace OFF_MAX with INT64_MAX. The former is defined on Linux. 2016-10-03 04:00:30 +00:00
Marcel Moolenaar
6b1235713e Prefer <stdint.h> over <sys/types.h>. While here remove redundant
inclusion of <sys/queue.h>.

Move the inclusion of the disk partitioning headers out of order
and inbetween standard headers and local header. They will change
in a subsequent commit.
2016-10-03 02:37:28 +00:00
Marcel Moolenaar
1080fb197b Replace STAILQ with TAILQ. TAILQs are portable enough that they can
be used on both macOS and Linux. STAILQs are not. In particular,
STAILQ_LAST does not next on Linux. Since neither STAILQ_FOREACH_SAFE
nor TAILQ_FOREACH_SAFE exist on Linux, replace its use with a regular
TAILQ_FOREACH. The _SAFE variant was only used for having the next
pointer in a local variable.
2016-10-03 01:46:47 +00:00
Marcel Moolenaar
7b5a53ea4d Portability changes:
1.  macOS nor Linux have MAP_NOCORE nor MAP_NOSYNC. Define as 0.
2.  macOS doesn't have SEEK_DATA nor SEEK_HOLE. Define as -1
    so that lseek will return -1 (with errno set to EINVAL).
3.  gcc correctly warns that error is assigned but not used in
    image_copyout_region().  Fix by returning on the first error.
2016-09-26 04:14:00 +00:00
Marcel Moolenaar
5aad7d9a87 Avoid depending on the <sys/endian.h> header for le*enc and be*enc.
Not only is the header unportable, the encoding/decoding functions
are as well.  Instead, duplicate the handful of small inlines we
need into a private header called endian.h.

Aside: an alternative approach is to move the encoding/decoding
functions to a separate system header.  While the header is still
nonportable, such an approach would make it possible to re-use the
definitions by playing games with include paths. This may be the
preferred approach if more (build) utilities need this.  This
change does not preclude that.  In fact, it makes it easier.
2016-09-26 01:06:32 +00:00
Marcel Moolenaar
4039ea7c9b Eliminate the use of EDOOFUS. The error code was used to signal
programming errors, but is really a poor substitute for assert.
And less portable as well.
2016-09-26 00:41:08 +00:00
Marcel Moolenaar
5a1302ab2e Replace the use of linker sets with constructors for both the
formats and schemes.  Formats and schemes are registered at
runtime now, rather than collected at link time.
2016-09-25 22:57:59 +00:00
Marcel Moolenaar
ac3425511d Update local variable 'block' after calling capacity_resize(),
otherwise format_resize(), which is called right after, isn't
getting the current/actual image size. Rather than rounding up,
format_resize() could end up truncating the size and we don't
allow that by design.

MFC after:	1 week
2016-09-24 17:29:27 +00:00
Alan Somers
005156d7b3 Update mkimg(1) author's contact info
Reviewed by:	marcel, imp
MFC after:	3 days
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D7991
2016-09-21 18:42:17 +00:00
Pedro F. Giffuni
3af7c80523 mkimg(1): minor cleanups with argument order in calloc(3).
Generally the first argument in calloc is supposed to stand for a count
and the second for a size. Try to make that consistent. While here,
attempt to make some use of the overflow detection capability in
calloc(3).
2016-07-12 15:46:53 +00:00
Ed Maste
023075def4 mkimg: bump version to 20151211 after r292082
mkimg has had a number of functional additions after the last time the
version was incremented. Do so now, to r292082's commit date, so that
users can determine what is supported.

Reviewed by:	marcel
Approved by:	re (gjb)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6882
2016-06-17 18:49:42 +00:00
Mark Johnston
cc7f01a5da mkimg: Indicate that input file pages are unlikely to be reused.
mkimg(1) uses a swap file to back input file chunks. When the output file
is being written out, blocks of the swap file are mapped and their contents
copied. This causes the backing VM pages to enter the active queue, and when
the output file is large relative to system memory (as is generally the
case), can result in a shortfall of inactive memory. This causes the
pagedaemon to aggressively scan the active queue and swap out process
memory in an attempt to meet the shortfall. Because mkimg's input files
are typically the intermediate result of some build process, there's no
need to push them all through the active queue. Use madvise(2) to indicate
that the backing pages may be reclaimed in preference to active pages. In
the case of the swap file, these pages will be freed as soon as mkimg
exits anyway.

When using mkimg on a desktop-class system with large amounts of dirty
process memory, this change substantially improves mkimg runtime and
reduces swap usage.

Reviewed by:	marcel
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D6654
2016-06-01 02:30:06 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Warner Losh
4224509a36 Add ppcboot FAT type. Needed to create a bootable powerpc image.
Differential Review: https://reviews.freebsd.org/D4407
2015-12-11 05:39:42 +00:00
Bryan Drewery
b1f92fa229 META MODE: Update dependencies with 'the-lot' and add missing directories.
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so.  A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR.  These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package.  The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:23:19 +00:00
Eitan Adler
463a577b27 Fix a ton of speelling errors
arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337
2015-10-21 05:37:09 +00:00
Ed Maste
a4c8dbbd87 mkimg: support fat16b partitions (MBR type 06h)
Reviewed by:	marcel
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3894
2015-10-15 01:09:14 +00:00
Enji Cooper
b2d48be1bc Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
Christian Brueffer
99d43aee11 Fix a few mandoc warnings.
MFC after:	1 week
2015-08-12 10:34:05 +00:00
Marcel Moolenaar
8f1d6b6915 Fix typo introduced in previous commit.
Pointed out by: Nikolai Lifanov <lifanov at mail.lifanov.com>
2015-08-07 18:40:44 +00:00