Commit Graph

244 Commits

Author SHA1 Message Date
gjb
b70ef7c982 Initial rewrite to consolidate VM image build scripts into one.
There may be some very sharp edges here while refactoring.

 - Move amd64/mk-vmimage.sh -> scripts/mk-vmimage.sh.
 - Remove vm-base target from Makefile.vm.
 - In vm-image target, use getopts flags for argument passing.
 - Create tools/vmimage.subr, containing default and prototype
   for the following functions that are used to drive the build,
   run in this order:

   vm_install_base()
   vm_extra_install_base()
   vm_extra_install_packages()
   vm_extra_install_ports()
   vm_extra_enable_services()
   vm_extra_pre_umount()
   vm_create_disk()
   vm_extra_create_disk()

 - In tools/azure.conf, override:

   vm_extra_install_base()
   vm_extra_pre_umount()
   vm_extra_create_disk()

 - In tools/openstack.conf, override:

   vm_extra_install_base()
   vm_extra_pre_umount()

Sponsored by:	The FreeBSD Foundation
2014-11-05 13:22:19 +00:00
gjb
0b5eea342c Fix output file name for openstack images.
No further conversion is necessary for this
VM file target, so there is no need to append
the '.raw' suffix here.

Sponsored by:	The FreeBSD Foundation
2014-10-29 19:44:34 +00:00
gjb
be176f72fe Initial commit providing a mechanism to create
openstack images as part of the release build.

This mimics the way Microsoft Azure images are
built, with the addition of installing the
net/cloud-init package and adding a (commented)
rc.conf(5) entry for cloudinit.

Sponsored by:	The FreeBSD Foundation
2014-10-29 17:04:09 +00:00
gjb
a452e67d04 Remove a few vestiges of passing an exit code to
panic().

Sponsored by:	The FreeBSD Foundation
2014-10-29 16:20:49 +00:00
gjb
b2f4325812 Avoid hard-coding the Azure image file format.
While here, avoid using OSRELEASE for the output
file name.

Sponsored by:	The FreeBSD Foundation
2014-10-29 16:18:29 +00:00
gjb
8521a45a55 Output an informational message when mkimg(1) runs, so it
does not appear that the process has stopped while waiting
for a 'y/n' response when waagent is deprovisioned.

Sponsored by:	The FreeBSD Foundation
2014-10-14 15:18:22 +00:00
gjb
53cd002a85 Fix signal list to trigger umount(8).
Sponsored by:	The FreeBSD Foundation
2014-10-14 12:13:43 +00:00
gjb
d7ed1332f2 Add a separate make(1) target to release/Makefile to
build FreeBSD virtual machine disk images for use on
the Microsoft Azure service.

For now, this target is not directly connected to the
build, however can be manually invoked.

The 'vm-azure' target invokes {amd64,i386}/mk-azure.sh,
which does the heavy lifting to produce proper VHDs.
mk-azure.sh uses a configuration file, defaulting to
tools/azure.conf if otherwise unset.

Sponsored by:	The FreeBSD Foundation
2014-10-14 11:48:28 +00:00
gjb
19f69f6638 Revert r272793, r272794, r272795:
Remove implementation of vm_prebuild_setup(),
  vm_setup(), and vm_postbuild_setup().

  It does not scale well, and I am not happy with
  their implementation.

Sponsored by:	The FreeBSD Foundation
2014-10-14 11:20:08 +00:00
gjb
cea401550a Reference vm_prebuild_setup(), vm_setup(), and
vm_postbuild_setup() functions.

Sponsored by:	The FreeBSD Foundation
2014-10-09 04:10:58 +00:00
gjb
437ed46f88 Allow mk-vmimage.sh to accept a configuration file as
an argument to '-c'.

Sponsored by:	The FreeBSD Foundation
2014-10-09 03:50:38 +00:00
gjb
c7959ee139 Add prototypes for build-specific tuning:
- vm_prebuild_setup():
	Steps to run prior to creating the file-backed
	disk image.

 - vm_setup():
	Steps to run while the file-backed disk image
	is mounted.

 - vm_postbuild_setup():
	Steps to run after the file-backed disk image
	is dismounted.

The intention is to override as necessary via build
configuration files, as needed.

Sponsored by:	The FreeBSD Foundation
2014-10-09 03:29:52 +00:00
gjb
c40726934a Add /usr/local/bin and /usr/local/sbin to PATH, needed
if third-party software needs to use utilities outside
of the base system during post-install stages (indexinfo
is one culprit).

Sponsored by:	The FreeBSD Foundation
2014-10-09 03:20:28 +00:00
gjb
da5a9f35d4 If the vm-base target fails, prevent the vm-image target
from being run since it cannot possibly succeed.

Sponsored by:	The FreeBSD Foundation
2014-10-02 18:23:53 +00:00
gjb
c932f99a7a Remove the first argument to panic(), which was initially
intended to be the exit code, however when a non-zero exit
code was returned to release/Makefile, this would prevent
any remaining (and possibly successful) stages from being
attempted.

Sponsored by:	The FreeBSD Foundation
2014-10-02 18:20:01 +00:00
gjb
b2b02a9be6 Avoid using env(1) to set values passed to mk-vmimage.sh,
and instead pass the values as arguments to the script,
making it easier to run this by hand, without 'make release'.

Add usage_vm_base() and usage_vm_image() usage helpers.

Sponsored by:	The FreeBSD Foundation
2014-10-01 19:13:42 +00:00
gjb
78c1b9246c Separate release/scripts/mk-vmimage.sh to machine-specific
scripts, making it possible to mimic the functionality for
non-x86 targets.

Move echo output if MAKEFLAGS is empty outside of usage().

Remove TARGET/TARGET_ARCH evaluation.

Sponsored by:	The FreeBSD Foundation
2014-10-01 17:05:40 +00:00
gjb
32749243a0 Fix indentation for diff reduction with commits to follow.
Sponsored by:	The FreeBSD Foundation
2014-04-25 19:43:18 +00:00
gjb
e61c0f1d81 Revert r264907 and r264908:
Restore make-memstick.sh back to its original state to
unbreak booting for machines that do not support GPT.

I have in-progress work to keep the MBR layout and add
the EFI partition, but it is not yet ready, and does
need at least one full release build to be certain it
does not break.

Sponsored by:	The FreeBSD Foundation
2014-04-25 18:46:54 +00:00
gjb
0b6c70fead Style cleanup
- Indent 1 full tab where needed
 - Use $() for shell exec
 - Insert a space between '$(( ))' parens

MFC After:	1 week
X-MFC-With:	r264907
Sponsored by:	The FreeBSD Foundation
2014-04-25 01:55:14 +00:00
gjb
fe37b9e4c2 Refactor make-memstick.sh to avoid creating the 'dangerously
dedicated' partition scheme, reported to cause the memstick.img
to fail to boot.

Similar to how make-memstick.sh worked on stable/8, use makefs(8)
to create the actual filesystem.  Then calculate the size of the
resulting image file, create the GPT partition scheme, then dd(1)
the filesystem created with makefs(8) to the freebsd-ufs GPT
partition.

This was tested on a known-working machine[1] for regression, and
a known-not-working machine[2] to ensure the boot issue has been
resolved.

Testers:	myself [1], db [2]
MFC After:	1 week
Sponsored by:	The FreeBSD Foundation
2014-04-25 01:38:57 +00:00
gjb
284a0c8006 Update the pkg-stage target to be more compatible with pkg-1.2:
- Add a release-dvd.conf pkg(8) configuration file to override
   the default FreeBSD.conf configuration.
 - Remove architecture-specific pkg-stage.conf files, consolidate,
   and move their contents to scripts/pkg-stage.sh.
 - Use 'pkg -vv' to determine the ABI, which is used as the
   cache directory.

Prior to these changes, it would be possible for pkg-stage to fetch
conflicting binary packages from multiple repositories.

Tested against:	head@r260522, stable/10@r260522

MFC after:	3 days
X-Insta-MFC:	possibly
Sponsored by:	The FreeBSD Foundation
2014-01-16 16:12:09 +00:00
gjb
da2cc957a9 Simplify PKG_ABI for pkg-stage.sh.
Submitted by:	hrs
MFC after:	3 days
X-MFC-With:	r258310
Sponsored by:	The FreeBSD Foundation
2013-11-18 20:01:52 +00:00
gjb
47bd4a0525 Fix how ABI is evaluated so it matches more than a dot-zero
case.

MFC after:	3 days
X-MFC-With:	r258310
Sponsored by:	The FreeBSD Foundation
2013-11-18 17:52:18 +00:00
gjb
9f5a779e22 Set the PKG_CACHEDIR directory to 'dvd/' instead of 'release/'
in preparation of adding a 'dvd1.iso' target.

MFC after:	3 days
X-MFC-With:	r258305, r258307, r258308
Sponsored by:	The FreeBSD Foundation
2013-11-18 16:11:19 +00:00
gjb
fa17320479 Add a script and configuration files to fetch pre-built packages
from pkg.FreeBSD.org for inclusion on release medium (dvd1.iso).

The script sources ${.CURDIR}/${TARGET}/pkg-stage.conf, which sets
several environment variables, such as the pkg(8) ABI, PACKAGESITE,
PKG_DBDIR, and PKG_CACHEDIR.  PKG_CACHEDIR is set to the directory
on the release medium used by bsdconfig(8) (/packages/${ABI}).  ABI
is determined by output of 'make -C /usr/src/release -V REVISION'.
See pkg.conf(5) for descripton on other variables set here.

The list of packages to include are set within the configuration
file.

The script and configuration files are intended to be run by the
'make dvd' target within the release directory, and assume the
release is built within a chroot environment (such as by using
release.sh).

Relevant updates to release/Makefile will follow.

Sponsored by:	The FreeBSD Foundation
2013-11-18 15:22:55 +00:00
hrs
aa08976cf0 - Add CHECKSUM.* support in Makefile[1].
- Use ln -fs to create a symlink.
- Remove pkgadd for docports.
- Use WITHOUT_JADETEX=yes instead of WITH_JADETEX=no.
- Add {WORLD,KERNEL}_FLAGS to [BTWK]MAKE.
- Use makefs(8) and gpart(8) for sparc64 ISO image[2].
- Add publisher option to makefs(8)[2].

Based on work by:	gjb[1]
Discussed with:		marius, nwhitehorn[2]
2013-02-03 10:26:24 +00:00
hrs
adbb26d90e ISO 9660 specification allows only "d-characters" and "a-characters" in the
Volume Descriptor (section 7.4).  In short, upper-case alphanumeric + some
symbols only.  While the makefs utility automatically converts the characters,
$LABEL should be consistent in the scripts.
2013-01-08 21:13:58 +00:00
nwhitehorn
2fc07cb3a5 Use UFS labels and bsdlabels (like the 8.x memsticks) instead of GPT to
fix problems with some BIOSes.

MFC after:	3 days
2011-10-09 16:23:04 +00:00
nwhitehorn
190bd9f097 Farewall, sysinstall! You served us well for many years, but 10.0 is one
digit beyond your time.

Various sysinstall dependencies (e.g. libftpio, libdisk, libodialog, etc.)
will be cleaned up in coming days. Some will take longer than others due to
a few other consumers (tzsetup and sade).
2011-10-03 15:13:09 +00:00
nwhitehorn
7c1cc185fa Mount memsticks read-only by default to prevent them being filled by
user modifications and subsequently preventing a functioning installation.

Approved by:	re (kib)
2011-07-30 00:51:36 +00:00
nwhitehorn
3fa339365c Garbage collect some unused variables and stray debug code.
Submitted by:	marius
2011-05-05 13:55:43 +00:00
nwhitehorn
3a0f89209c Repair release CD generation on PC98 and sun4v after release building
changes, and backport the new logic (ISO images are TARGET dependant, not
TARGET_CPUARCH dependant) to Makefile.sysinstall. While modifying ISO
image scripts, change several archs to use makefs (from base) instead of
mkisofs (from ports) which makes release CD generation both faster and
self-hosting.
2011-05-04 23:41:11 +00:00
nwhitehorn
7e04216c28 Use labels to find release media instead of hard-coded device paths. This
makes booting more reliable (and working at all on USB sticks). While here,
move responsibility for setting up fstab into the various platform mk-*.sh
scripts.

Suggested by:	many
2011-03-22 01:14:53 +00:00
nwhitehorn
2397771613 Add generation of memstick images to the bsdinstall release makefile for
i386 and amd64. This involved moving the memstick generation script to
the arch directories from scripts/, in analogy to mkisoimages.sh. This
script was never called from /usr/src/release/Makefile, so that hasn't
been updated.
2011-03-13 01:38:24 +00:00
nyan
c962e71f7e Fix making a release. The old libdialog was renamed to libodialog. 2011-01-14 10:56:44 +00:00
delphij
c64406c79a Reorder to have -lcrypto after -larchive. Our linker doesn't seem to like
the current ordering :(
2010-05-26 18:55:40 +00:00
delphij
f57b5cd0bc libarchive now needs libcrypto and liblzma. 2010-05-25 17:48:17 +00:00
jhb
3e0f5e5d70 Remove slattach from the install mfsroot since it doesn't exist anymore
to quiet a warning from crunchgen.

MFC after:	1 week
2010-01-26 15:28:00 +00:00
ed
f79d0f71d8 Remove libulog from the bootstrap again.
libulog now only provides functions that are used by various packages
from the ports tree, namely the libutempter ones. There is no reason to
link it into the crunch/fixit binaries anymore.
2010-01-21 16:56:27 +00:00
ed
197649eead Don't forget to link the fixits against -lulog.
reboot(8) now depends on it.

Reported by:	nyan
2009-12-06 14:22:14 +00:00
jamie
fa263a2ed1 Add libjail to the boot_crunch binaries (for ifconfig).
PR:		136071
Submitted by:	Scot Hetzel
Approved by:	re (kensmith), bz (mentor)
2009-06-29 13:59:30 +00:00
thompsa
4aa75fa98d Install libusb20.so.1 as libusb.so.1, there will be a followup commit to the
ports tree so that programs use libusb from the base by default. Thanks to
Stanislav Sedov for sorting out the ports build.

Bump __FreeBSD_version to 800069

Help and testing by:	stas
2009-03-09 17:05:31 +00:00
nyan
36a2601b9b Catch up with USB2 change.
Remove the usbdevs and add the usbconfig.

Spotted by:     hrs
Tested on:      i386
2009-03-03 09:04:18 +00:00
nyan
3476eb5656 - Add libbsdxml and libsbuf required by bsdlabel.
- Remove the mknod from pc98 fixit and also the ftp from pc98 small fixit
  to avoid disk full.
2008-09-25 12:25:54 +00:00
kensmith
ebd94f5c0d Recent work on the gnu cpio seems to have introduced some bugs, it
complains about "Malformed numbers" while unpacking the dists and
what winds up on the disk isn't correct.  Use this as an opportunity
to switch over to bsdcpio since at this point we don't even build
and install the gnu cpio by default.  Note sysinstall needed to be
tweaked a bit (dropping tape block size setting) because it seems
bsdcpio doesn't do anything with block sizes, at least as far as
reading from archives goes.  That wasn't really a problem since
installations from tape have been broken for a while and the rest
of sysinstall's tape support code will be removed shortly.
2008-08-20 13:35:39 +00:00
thompsa
a82fbe5c0f libbsdxml is now required by ifconfig(8).
Submitted by:	Florian Smeets
2008-04-22 18:20:05 +00:00
kensmith
284acd0781 In 'fixit mode' running "fsck /dev/ad0s1a" typically complains that it
can't find fsck_4.2bsd because there was no fstab file saying what
filesystem type it is looking at so it got the filesystem type from
the disk's label.  When that fails admins who haven't been in this
situation before are most likely to try "fsck -t ufs /dev/ad0s1a" because
ufs is the type used in fstab files on working systems but that also fails
complaining it can't find fsck_ufs.

This just sets it up so /stand in the MFS image (/sbin is a symlink
to /stand) includes hard links pointing fsck_4.2bsd and fsck_ufs to
fsck_ffs which is what is present in /sbin on installed systems.

Prodded by:	obrien
MFC after:	1 day
2008-01-22 16:35:10 +00:00
kensmith
4d070b972f ed(1) no longer relies on crypto library when built for the fixit media.
Approved by:	re (hrs)
2007-07-02 14:01:43 +00:00
kensmith
db48437b8a The i386 "fixit" floppy began overflowing, most likely due to the new
gcc.  Replace vi(1) with ed(1) as the editor to make the contents fit
on a floppy again.
2007-06-15 14:41:11 +00:00