Commit Graph

107 Commits

Author SHA1 Message Date
Andrew Thompson
ad33b77611 ec2.conf and vmimage.subr can be used from the installation livecd after
install to prepare an AMI image. This can be used to create a ZFS AMI disk
image using a virtual machine.

Change ec2.conf to use the pkg tool from a chroot rather than trying to
bootstrap it and fail from the livecd readonly filesystem.

Reviewed by:	gjb
2017-03-09 01:26:10 +00:00
Glen Barber
a744f4edcd Increase the EC2 image size for 12-CURRENT. The recent snapshot
builds of EC2 images for 12-CURRENT failed due to a full filesystem
on the md(4) device during creation.

Sponsored by:	The FreeBSD Foundation
2017-03-02 17:31:59 +00:00
Colin Percival
9763fbf7ef Enable IPv6 networking on Amazon EC2.
MFC after:	1 week
2017-01-15 09:06:45 +00:00
Glen Barber
82fe834050 - Resize FreeBSD to the size of the OpenStack flavor (growfs).
- Speeds up the boot process by disabling sendmail.
- Allows an user to ssh as root with a public key.
- Make ssh(1) respond faster by disabling DNS lookups.
- Enable DHCP on the vtnet(4) interface.

Note: The CLOUDWARE list has not yet been changed to include the
OpenStack target by default yet.

Submitted by:	Diego Casati
PR:		215258
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-12-14 03:01:15 +00:00
Li-Wen Hsu
49b53bee5a - Use virtualbox-ose-additions-nox11 for vagrant image to reduce size
Reviewed by:	brd, gjb, swills
Approved by:	gjb
2016-10-30 15:46:24 +00:00
Glen Barber
f5355fadf7 Belatedly revert r303119, which was determined to not be
needed.

Sponsored by:	The FreeBSD Foundation
2016-10-24 21:16:21 +00:00
Navdeep Parhar
eb46c98711 Add the ability to override the size of the swap partition when building
VM images.  The default continues to be 1G.

Reviewed by:	gjb@
Sponsored by:	Chelsio Communications
2016-10-10 22:36:44 +00:00
Glen Barber
d82d5ebec6 Create a /usr/home -> /home symlink for the arm images to
avoid /usr/home confusingly being created as a directory.

Reported by:	Russell Haley
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2016-07-20 22:49:33 +00:00
Colin Percival
d53ee33e1c Now that potentially buggy versions of Xen are automatically detected
(see r302635), there is no need to force msix interrupt migration off
via loader.conf.

Reverts:	r302184
MFC after:	3 days
2016-07-16 08:04:00 +00:00
Colin Percival
4d1afa14d5 Enable indirect segment I/Os by default in the Xen blkfront driver when
running on EC2.  Due to improvements in EC2, the performance penalty which
was present on some EC2 instances no longer exists, and enabling this
feature now consistently yields ~20% higher throughput with equal or lower
latency.

Reverts:	r286063
Approved by:	re (gjb)
MFC after:	2 weeks
Relnotes:	Improved disk throughput on EC2
2016-06-29 23:56:24 +00:00
Colin Percival
60406292e4 Turn off MSI-X interrupt migration by default in EC2 instances; this works
around a bug in older versions of Xen and unbreaks SR-IOV (aka. "EC2
Enhanced Networking").

Approved by:	re (gjb)
Thanks to:	jhb, Jeremiah Lott
2016-06-24 23:39:44 +00:00
Emmanuel Vadot
c7161c4adc Bump /tmp from 30m to 50m for ARM release images.
30m isn't enough for pkg anymore to extract packagesite.txz.
 40m is fine for now but let's take a safer way as we don't know when pkg will need more.

Reported by:	many
Approved by:	re (gjb), andrew (mentor)
2016-06-16 03:02:27 +00:00
Colin Percival
f207bd78e6 Switch from console="comconsole" to boot_multicons="YES" in EC2.
Amazon recently introduced an API for capturing screenshots of an
emulated VGA device; this commit makes that (somewhat) useful.

MFC after:	3 weeks
2016-06-09 13:58:31 +00:00
Glen Barber
a99eb3157e Turn off nfsv4acls for arm/armv6 builds, which has an impact
on performance, especially with SD cards on certain SoCs.

Requested by:	trasz
Discussed with:	ian, kientzle

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2016-06-01 04:18:55 +00:00
Glen Barber
a9bf704474 Fix tmpfs(5) '/tmp' mount point.
A stray trailing space snuck in with one of the recent
changes, making r290550 and r290573 effectively no-op.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-11-25 19:10:04 +00:00
Glen Barber
0eea47e68b Remove noatime from tmpfs(5) fstab(5) entry.
Submitted by:		Howard Su
MFC after:		3 days
X-MFC-With:		r290550
Sponsored by:	The FreeBSD Foundation
2015-11-09 06:30:26 +00:00
Glen Barber
6a833a7a2c Use tmpfs(5) instead of md(4) for '/tmp' mount.
Submitted by:		Nikolai Lifanov
Differential Revision:	D3506
MFC after:		3 days
Sponsored by:	The FreeBSD Foundation
2015-11-08 18:02:39 +00:00
Glen Barber
91c42fff77 In vm_copy_base(), turn off SU+J on the resultant filesystem,
leaving only SU enabled.

Discussed with:	kib (a few weeks ago)
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-09-29 15:30:27 +00:00
Colin Percival
a5c1653b5c Disable blkif indirect segment I/Os in EC2 by default due to performance
issues on some EC2 instance types.  Users may want to experiment with
removing this from loader.conf and measuring the performance impact on
the EC2 instances they are using.
2015-07-30 04:01:00 +00:00
Glen Barber
6a277d6e8c Fix an out-of-order execution issue regarding pkg(8):
- pkg(8) cannot be removed before subsequent reinvocations
- The PKG_CACHEDIR cannot be cleaned after the repo*.sqlite
  has been removed
- pkg(8) cannot be removed as a precursor to any of the other
  steps involved here

MFC after:	3 days
X-MFC-With:	r285722
X-MFC-Before:	10.2-{BETA3,RC1} (whichever happens next)
Sponsored by:	The FreeBSD Foundation
2015-07-21 03:18:53 +00:00
Brad Davis
a54bd5953b Add support for building VirtualBox Vagrant images.
Abstract the build, package and upload to handle building either type.

Approved by:	re (gjb)
2015-07-20 16:27:44 +00:00
Glen Barber
a1912cc902 Do not mount /var/log and /var/tmp as md(4)-backed devices.
Discussed with:	ian
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-07-12 19:17:05 +00:00
Glen Barber
a1c0309edc Revert r285018, r285019, r285076, r285078, r285082.
The true cause of the missing UFS/MSDOSFS labels has been
identified, and only affects stable/10 at the moment.

An request for commit to stable/10 will be pending RE approval
after this commit.

MFC after:	1 day
X-MFC-Note:	never
X-MFC-Never:	r285018, r285019, r285076, r285078, r285082
Sponsored by:	The FreeBSD Foundation
2015-07-04 00:01:41 +00:00
Glen Barber
ef3a5d20dd Test for loader.rc existence before attempting to create
the symlink from loader.rc.sample.

Fix paths relative to the CHROOTDIR.

MFC after:	3 days
X-MFC-With:	r285076, r285078
X-MFC-Before:	10.2-BETA1
Sponsored by:	The FreeBSD Foundation
2015-07-03 06:15:54 +00:00
Glen Barber
174fe0cfce Revert r285018 and r285019, which attempted to resolve
UFS/MSDOSFS label issues on FreeBSD/arm builds, however
the real problem was addressed in r285076, which is due
to two separate issues, unrelated to md(4) stale device
existence.

MFC after:	3 days
X-MFC-With:	r285076
X-MFC-Before:	10.2-BETA1
Sponsored by:	The FreeBSD Foundation
2015-07-03 06:07:21 +00:00
Glen Barber
a6d3c7cdd5 Fix the real problem with missing UFS/MSDOSFS labels on the
FreeBSD/arm builds.  The problem stems from the loader.rc file
not existing, as well as geom_label not being loaded at boot.

For now, add the geom_label_load entry to loader.conf, and
symlink loader.rc.sample to loader.rc, both of which allowed
my BeagleBone Black to boot fine with a UFS label reference in
fstab(5).

MFC after:	3 days
X-MFC-Before:	10.2-BETA1
Sponsored by:	The FreeBSD Foundation
2015-07-03 06:01:37 +00:00
Glen Barber
e8d7333d7d Fix the path to the UFS/MSDOSFS filesystem labels, since they
cannot possibly exist within the chroot(8) before the target
filesystem actually exists.

MFC after:	3 days
X-MFC-With:	r285018
Sponsored by:	The FreeBSD Foundation
2015-07-02 02:33:58 +00:00
Glen Barber
e07ca0423d Implement an evil workaround that prevents UFS/MSDOS labels from being
written to disk with newfs(8) and newfs_msdosfs(8).

When iterating through snapshot builds in serial, it is possible for
a build failure to leave stale md(4) devices behind, in some cases, they
could have a UFS or MSDOS filesystem label assigned.

If the md(4) is not destroyed (or not able to be destroyed, as has
happened recently due to my own fault), the filesystem label that
already exists can interfere with a new md(4) device that is targeted to
have the same label.

This behavior, although admittedly a logic error in the wrapper build
scripts, has caused intermittent reports (in particular with the armv6
builds) of missing UFS/MSDOSFS labels, causing the image to fallback to
the mountroot prompt.  This appears to only happen when the backing
md(4) device is destroyed before the calling umount(8) on the target
mount, after which the UFS/MSDOSFS label persists.

The workaround is this:  If EVERYTHINGISFINE is set to non-empty value,
check for an existing ufs/rootfs and msdosfs/MSDOSBOOT filesystem label
in arm_create_disk(), and rm(1) them if they exist.

The EVERYTHINGISFINE variable is chosen because it is used in exactly
one other place - release/Makefile.mirrors - and there are big scary
warnings at the top of that file as well that it should *not* be used
under normal circumstances.  This should not destroy a build machine
that also uses '/dev/ufs/rootfs' as the UFS label, and I have verified
in extensive local testing that the destroyed label is recreated when
the md(4) is unmounted/mounted, but this really should not be enabled
by anyone.

Having said all that, I absolutely *do* plan MFC this to stable/10 for
the 10.2-RELEASE cycle, as so far, I have only observed this behavior
on stable/10, but this is a temporary solution until I can unravel all
of the failure paths to properly trap them.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-07-02 02:13:20 +00:00
Brad Davis
41f7db3a4f Add initial support for building Vagrant images for VMWare. Next steps will be adding Virtualbox support and uploading to Hashicorp Atlas for others to consume.
Approved by:	re (gjb)
2015-06-27 20:39:13 +00:00
Glen Barber
eb64cd58f5 Add initial support for automatically uploading GCE virtual
machine images to the Google Compute Engine platform.

By default, gcutil/gsutil requires an Oauth2 login generated
from a URL that must be opened in a browser, a verification
code copied back to the terminal from which it was invoked,
etc., etc., making it near impossible for automation.

I've hacked together an evil solution to work around this,
so unless GCE_LOGIN_SKIP is set to a non-empty value, this
Makefile will not do anything useful.

As a result of this commit, remove the gce-package.sh script
that was never, nor will ever be, used.

MFC after:	3 days
X-MFC-Note:	(hopefully)
Sponsored by:	The FreeBSD Foundation
2015-06-27 01:29:59 +00:00
Glen Barber
67076e4d58 Ensure ASSUME_ALWAYS_YES is set when deleting pkg(8) from the
DESTDIR, otherwise it can get stuck in ttyin.

This means no 10-STABLE EC2 images this week.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-24 21:30:35 +00:00
Glen Barber
798e3ec502 Chase r284656 and r284658:
Remove the Azure-local vm_extra_create_disk(), since we no longer
need qemu-img to convert the final VHD image to an Azure-compatible
format.

Although the waagent utility is installed from ports, create the
symlink to /usr/sbin, pending investigation on where this is
hard-coded, so it can be reported upstream.  In the meantime, this
is good enough.

MFC after:	3 days
X-MFC-Needs:	r284269, r284270, r284271, r284655,
		r284656, r284657, r284658, r284659
X-MFC-Note:	Required for 10.2-RELEASE, marcel@ has
		implicit approval for the required changes
Sponsored by:	The FreeBSD Foundation
2015-06-21 04:34:57 +00:00
Glen Barber
99b858f537 Fix path to waagent.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-19 23:03:05 +00:00
Glen Barber
717a45fcae Work around a potential bug in pw(8) when '-m' is specified.
According to the manual page, '-m' should create the user home
directory, however rigorous testing suggests it does not, and
it is unclear if this is an implementation or expectation issue.

Sponsored by:	The FreeBSD Foundation
2015-06-05 02:22:04 +00:00
Glen Barber
0bf3ddb7a7 Make sure /home exists within the DESTDIR before creating the
'freebsd' user account.

Sponsored by:	The FreeBSD Foundation
2015-06-04 23:11:24 +00:00
Glen Barber
b07d911f39 Use the new pw(8) '-R' option to create the users/groups.
MFC after:	1 week
X-MFC-after:	r283961
Sponsored by:	The FreeBSD Foundation
2015-06-03 19:19:25 +00:00
Glen Barber
92284ce807 Pass the '-b' argument to pw(8) so the '/home' directory is created
in the correct place.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-02 15:02:30 +00:00
Glen Barber
915f44d03b In arm_create_disk(), disable soft updates journaling.
Disabling soft updates journaling appears to resolve issues
with kernel panics, and may also be generally bad to have
enabled for SD cards.

Requested by:	ian
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-01 21:10:47 +00:00
Glen Barber
f3ca4d9170 Re-enable arm_create_user(), and pass the '-V DESTDIR/etc' to
pw(8) to set the correct /etc directory for the user/group
files.

Provided by:	ian (thanks!)
MFC after:	3 days
X-MFC-with:	r283894
Sponsored by:	The FreeBSD Foundation
2015-06-01 20:59:18 +00:00
Glen Barber
b506d010c2 Disable arm_create_user(). As-is, this does not create the
user in the userland for the target image, but creates the
user in the build chroot.

Before this is re-enabled, I want to figure out a clean way
to do this without requiring the overhead of third-party
utilities (such as qemu).

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-01 20:26:38 +00:00
Glen Barber
ccbf25027e Fix virtual machine disk format creating by passing VMFORMAT
to mkimg(1)

PR:		200068
Submitted by:	Jeremy Norris
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-05-11 14:53:07 +00:00
Glen Barber
88feae84be Merge ^/projects/release-arm-redux into ^/head.
Of note:

- This commit adds native FreeBSD/arm release build support without
  requiring out-of-tree utilities.

- Part of this merge removes the WANDBOARD-{SOLO,DUAL,QUAD} kernel
  configuration files, for which the IMX6 kernel configuration file
  should be used instead.

- The resulting images have a 'freebsd' user (password 'freebsd'),
  to allow ssh(1) access when console access is not available (VGA
  or serial).  The default 'root' user password is set to 'root'.

- The /etc/ttys file for arm images now enable both ttyv0 and ttyu0
  by default.

Help from:	many (boot testing, feedback, etc.)
Sponsored by:	The FreeBSD Foundation
2015-05-09 21:08:12 +00:00
Glen Barber
fef41f600c Fix path to /boot/msdos for the target image.
Sponsored by:	The FreeBSD Foundation
2015-05-08 10:38:32 +00:00
Glen Barber
869ef34722 Reorder / and /boot/msdos mount to avoid the following error:
Mounting local file systems:
mount: /boot/msdos: No such file or directory

Sponsored by:	The FreeBSD Foundation
2015-05-07 23:46:33 +00:00
Glen Barber
efc6ab44f2 Create the /boot/msdos directory so the FAT partition is mounted
properly.

Noticed by:	sbruno
Sponsored by:	The FreeBSD Foundation
2015-05-07 20:25:17 +00:00
Glen Barber
60bc17f490 Enable the growfs rc(8) script.
Sponsored by:	The FreeBSD Foundation
2015-05-07 18:49:43 +00:00
Glen Barber
04b7fd007b In arm_create_disk(), chroot(8) to the build environment
is necessary.

In arm_install_base(), chroot(8) when installing world
and kernel.  Fix paths for fstab(5) and rc.conf(5).

Sponsored by:	The FreeBSD Foundation
2015-05-07 17:35:48 +00:00
Glen Barber
befaadd56c In release.sh, create the /dev directory within the chroot
before attempting to mount(8) devfs.  Also, create the
.OBJDIR for the 'release' target, so files end up in the
correct location.

In tools/arm.subr, fix the target device when creating the
gpart partition scheme.

Sponsored by:	The FreeBSD Foundation
2015-05-07 00:27:28 +00:00
Glen Barber
4ad8b75d53 Add tools/arm.subr to contain common subroutines used for
building arm images.  This is similar to tools/vmimage.subr
used for building virtual machine disk images.  By default,
only arm_create_disk() and arm_install_base() contain real
functionality here, and arm_install_uboot() must be overridden
in the arm/KERNEL.conf file.

In release.sh, make create_arm_armv6_build_release() do
something now.

In arm/BEAGLEBONE.conf, set IMAGE_SIZE, PART_SCHEME, FAT_SIZE,
FAT_TYPE, and MD_ARGS, as well as make arm_install_uboot()
functional.

Parts of this were taken from disecting a previous BEAGLEBONE
image, and other parts obtained from Crochet sources.

Sponsored by:	The FreeBSD Foundation
2015-05-06 19:58:12 +00:00
Glen Barber
fea236302d Remove crochet configuration files and tools/arm/ directory.
Sponsored by:	The FreeBSD Foundation
2015-05-06 15:59:29 +00:00