Commit Graph

99 Commits

Author SHA1 Message Date
cperciva
1c836fb168 Enable IPv6 networking on Amazon EC2.
MFC after:	1 week
2017-01-15 09:06:45 +00:00
gjb
46f3c9c4a4 - Resize FreeBSD to the size of the OpenStack flavor (growfs).
- Speeds up the boot process by disabling sendmail.
- Allows an user to ssh as root with a public key.
- Make ssh(1) respond faster by disabling DNS lookups.
- Enable DHCP on the vtnet(4) interface.

Note: The CLOUDWARE list has not yet been changed to include the
OpenStack target by default yet.

Submitted by:	Diego Casati
PR:		215258
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-12-14 03:01:15 +00:00
lwhsu
72818dba17 - Use virtualbox-ose-additions-nox11 for vagrant image to reduce size
Reviewed by:	brd, gjb, swills
Approved by:	gjb
2016-10-30 15:46:24 +00:00
gjb
00202d1651 Belatedly revert r303119, which was determined to not be
needed.

Sponsored by:	The FreeBSD Foundation
2016-10-24 21:16:21 +00:00
np
fe1964c601 Add the ability to override the size of the swap partition when building
VM images.  The default continues to be 1G.

Reviewed by:	gjb@
Sponsored by:	Chelsio Communications
2016-10-10 22:36:44 +00:00
gjb
c3da6d9f13 Create a /usr/home -> /home symlink for the arm images to
avoid /usr/home confusingly being created as a directory.

Reported by:	Russell Haley
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2016-07-20 22:49:33 +00:00
cperciva
4ff5475d27 Now that potentially buggy versions of Xen are automatically detected
(see r302635), there is no need to force msix interrupt migration off
via loader.conf.

Reverts:	r302184
MFC after:	3 days
2016-07-16 08:04:00 +00:00
cperciva
94125daa17 Enable indirect segment I/Os by default in the Xen blkfront driver when
running on EC2.  Due to improvements in EC2, the performance penalty which
was present on some EC2 instances no longer exists, and enabling this
feature now consistently yields ~20% higher throughput with equal or lower
latency.

Reverts:	r286063
Approved by:	re (gjb)
MFC after:	2 weeks
Relnotes:	Improved disk throughput on EC2
2016-06-29 23:56:24 +00:00
cperciva
9ff7a7dab9 Turn off MSI-X interrupt migration by default in EC2 instances; this works
around a bug in older versions of Xen and unbreaks SR-IOV (aka. "EC2
Enhanced Networking").

Approved by:	re (gjb)
Thanks to:	jhb, Jeremiah Lott
2016-06-24 23:39:44 +00:00
manu
ddb2ec7808 Bump /tmp from 30m to 50m for ARM release images.
30m isn't enough for pkg anymore to extract packagesite.txz.
 40m is fine for now but let's take a safer way as we don't know when pkg will need more.

Reported by:	many
Approved by:	re (gjb), andrew (mentor)
2016-06-16 03:02:27 +00:00
cperciva
1db263cf30 Switch from console="comconsole" to boot_multicons="YES" in EC2.
Amazon recently introduced an API for capturing screenshots of an
emulated VGA device; this commit makes that (somewhat) useful.

MFC after:	3 weeks
2016-06-09 13:58:31 +00:00
gjb
caed01c611 Turn off nfsv4acls for arm/armv6 builds, which has an impact
on performance, especially with SD cards on certain SoCs.

Requested by:	trasz
Discussed with:	ian, kientzle

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2016-06-01 04:18:55 +00:00
gjb
b77609818e Fix tmpfs(5) '/tmp' mount point.
A stray trailing space snuck in with one of the recent
changes, making r290550 and r290573 effectively no-op.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-11-25 19:10:04 +00:00
gjb
44b9d011ea Remove noatime from tmpfs(5) fstab(5) entry.
Submitted by:		Howard Su
MFC after:		3 days
X-MFC-With:		r290550
Sponsored by:	The FreeBSD Foundation
2015-11-09 06:30:26 +00:00
gjb
7320cfca32 Use tmpfs(5) instead of md(4) for '/tmp' mount.
Submitted by:		Nikolai Lifanov
Differential Revision:	D3506
MFC after:		3 days
Sponsored by:	The FreeBSD Foundation
2015-11-08 18:02:39 +00:00
gjb
00e9f99f76 In vm_copy_base(), turn off SU+J on the resultant filesystem,
leaving only SU enabled.

Discussed with:	kib (a few weeks ago)
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-09-29 15:30:27 +00:00
cperciva
84ed76c811 Disable blkif indirect segment I/Os in EC2 by default due to performance
issues on some EC2 instance types.  Users may want to experiment with
removing this from loader.conf and measuring the performance impact on
the EC2 instances they are using.
2015-07-30 04:01:00 +00:00
gjb
70dd4560e3 Fix an out-of-order execution issue regarding pkg(8):
- pkg(8) cannot be removed before subsequent reinvocations
- The PKG_CACHEDIR cannot be cleaned after the repo*.sqlite
  has been removed
- pkg(8) cannot be removed as a precursor to any of the other
  steps involved here

MFC after:	3 days
X-MFC-With:	r285722
X-MFC-Before:	10.2-{BETA3,RC1} (whichever happens next)
Sponsored by:	The FreeBSD Foundation
2015-07-21 03:18:53 +00:00
brd
8f7cde25f2 Add support for building VirtualBox Vagrant images.
Abstract the build, package and upload to handle building either type.

Approved by:	re (gjb)
2015-07-20 16:27:44 +00:00
gjb
eb8d3d79be Do not mount /var/log and /var/tmp as md(4)-backed devices.
Discussed with:	ian
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-07-12 19:17:05 +00:00
gjb
b42f5bb715 Revert r285018, r285019, r285076, r285078, r285082.
The true cause of the missing UFS/MSDOSFS labels has been
identified, and only affects stable/10 at the moment.

An request for commit to stable/10 will be pending RE approval
after this commit.

MFC after:	1 day
X-MFC-Note:	never
X-MFC-Never:	r285018, r285019, r285076, r285078, r285082
Sponsored by:	The FreeBSD Foundation
2015-07-04 00:01:41 +00:00
gjb
5bc952ae8d Test for loader.rc existence before attempting to create
the symlink from loader.rc.sample.

Fix paths relative to the CHROOTDIR.

MFC after:	3 days
X-MFC-With:	r285076, r285078
X-MFC-Before:	10.2-BETA1
Sponsored by:	The FreeBSD Foundation
2015-07-03 06:15:54 +00:00
gjb
f1db22bf1a Revert r285018 and r285019, which attempted to resolve
UFS/MSDOSFS label issues on FreeBSD/arm builds, however
the real problem was addressed in r285076, which is due
to two separate issues, unrelated to md(4) stale device
existence.

MFC after:	3 days
X-MFC-With:	r285076
X-MFC-Before:	10.2-BETA1
Sponsored by:	The FreeBSD Foundation
2015-07-03 06:07:21 +00:00
gjb
33c66012c0 Fix the real problem with missing UFS/MSDOSFS labels on the
FreeBSD/arm builds.  The problem stems from the loader.rc file
not existing, as well as geom_label not being loaded at boot.

For now, add the geom_label_load entry to loader.conf, and
symlink loader.rc.sample to loader.rc, both of which allowed
my BeagleBone Black to boot fine with a UFS label reference in
fstab(5).

MFC after:	3 days
X-MFC-Before:	10.2-BETA1
Sponsored by:	The FreeBSD Foundation
2015-07-03 06:01:37 +00:00
gjb
f2ec46bf54 Fix the path to the UFS/MSDOSFS filesystem labels, since they
cannot possibly exist within the chroot(8) before the target
filesystem actually exists.

MFC after:	3 days
X-MFC-With:	r285018
Sponsored by:	The FreeBSD Foundation
2015-07-02 02:33:58 +00:00
gjb
7f9362f8f8 Implement an evil workaround that prevents UFS/MSDOS labels from being
written to disk with newfs(8) and newfs_msdosfs(8).

When iterating through snapshot builds in serial, it is possible for
a build failure to leave stale md(4) devices behind, in some cases, they
could have a UFS or MSDOS filesystem label assigned.

If the md(4) is not destroyed (or not able to be destroyed, as has
happened recently due to my own fault), the filesystem label that
already exists can interfere with a new md(4) device that is targeted to
have the same label.

This behavior, although admittedly a logic error in the wrapper build
scripts, has caused intermittent reports (in particular with the armv6
builds) of missing UFS/MSDOSFS labels, causing the image to fallback to
the mountroot prompt.  This appears to only happen when the backing
md(4) device is destroyed before the calling umount(8) on the target
mount, after which the UFS/MSDOSFS label persists.

The workaround is this:  If EVERYTHINGISFINE is set to non-empty value,
check for an existing ufs/rootfs and msdosfs/MSDOSBOOT filesystem label
in arm_create_disk(), and rm(1) them if they exist.

The EVERYTHINGISFINE variable is chosen because it is used in exactly
one other place - release/Makefile.mirrors - and there are big scary
warnings at the top of that file as well that it should *not* be used
under normal circumstances.  This should not destroy a build machine
that also uses '/dev/ufs/rootfs' as the UFS label, and I have verified
in extensive local testing that the destroyed label is recreated when
the md(4) is unmounted/mounted, but this really should not be enabled
by anyone.

Having said all that, I absolutely *do* plan MFC this to stable/10 for
the 10.2-RELEASE cycle, as so far, I have only observed this behavior
on stable/10, but this is a temporary solution until I can unravel all
of the failure paths to properly trap them.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-07-02 02:13:20 +00:00
brd
55a27553e8 Add initial support for building Vagrant images for VMWare. Next steps will be adding Virtualbox support and uploading to Hashicorp Atlas for others to consume.
Approved by:	re (gjb)
2015-06-27 20:39:13 +00:00
gjb
72250149ec Add initial support for automatically uploading GCE virtual
machine images to the Google Compute Engine platform.

By default, gcutil/gsutil requires an Oauth2 login generated
from a URL that must be opened in a browser, a verification
code copied back to the terminal from which it was invoked,
etc., etc., making it near impossible for automation.

I've hacked together an evil solution to work around this,
so unless GCE_LOGIN_SKIP is set to a non-empty value, this
Makefile will not do anything useful.

As a result of this commit, remove the gce-package.sh script
that was never, nor will ever be, used.

MFC after:	3 days
X-MFC-Note:	(hopefully)
Sponsored by:	The FreeBSD Foundation
2015-06-27 01:29:59 +00:00
gjb
710db0f35a Ensure ASSUME_ALWAYS_YES is set when deleting pkg(8) from the
DESTDIR, otherwise it can get stuck in ttyin.

This means no 10-STABLE EC2 images this week.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-24 21:30:35 +00:00
gjb
9d1d13b2d4 Chase r284656 and r284658:
Remove the Azure-local vm_extra_create_disk(), since we no longer
need qemu-img to convert the final VHD image to an Azure-compatible
format.

Although the waagent utility is installed from ports, create the
symlink to /usr/sbin, pending investigation on where this is
hard-coded, so it can be reported upstream.  In the meantime, this
is good enough.

MFC after:	3 days
X-MFC-Needs:	r284269, r284270, r284271, r284655,
		r284656, r284657, r284658, r284659
X-MFC-Note:	Required for 10.2-RELEASE, marcel@ has
		implicit approval for the required changes
Sponsored by:	The FreeBSD Foundation
2015-06-21 04:34:57 +00:00
gjb
10fa74d463 Fix path to waagent.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-19 23:03:05 +00:00
gjb
043d779d9b Work around a potential bug in pw(8) when '-m' is specified.
According to the manual page, '-m' should create the user home
directory, however rigorous testing suggests it does not, and
it is unclear if this is an implementation or expectation issue.

Sponsored by:	The FreeBSD Foundation
2015-06-05 02:22:04 +00:00
gjb
b86973a665 Make sure /home exists within the DESTDIR before creating the
'freebsd' user account.

Sponsored by:	The FreeBSD Foundation
2015-06-04 23:11:24 +00:00
gjb
de010910ed Use the new pw(8) '-R' option to create the users/groups.
MFC after:	1 week
X-MFC-after:	r283961
Sponsored by:	The FreeBSD Foundation
2015-06-03 19:19:25 +00:00
gjb
a82be33c48 Pass the '-b' argument to pw(8) so the '/home' directory is created
in the correct place.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-02 15:02:30 +00:00
gjb
7447f1a3e1 In arm_create_disk(), disable soft updates journaling.
Disabling soft updates journaling appears to resolve issues
with kernel panics, and may also be generally bad to have
enabled for SD cards.

Requested by:	ian
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-01 21:10:47 +00:00
gjb
e961ac3d2e Re-enable arm_create_user(), and pass the '-V DESTDIR/etc' to
pw(8) to set the correct /etc directory for the user/group
files.

Provided by:	ian (thanks!)
MFC after:	3 days
X-MFC-with:	r283894
Sponsored by:	The FreeBSD Foundation
2015-06-01 20:59:18 +00:00
gjb
911994babe Disable arm_create_user(). As-is, this does not create the
user in the userland for the target image, but creates the
user in the build chroot.

Before this is re-enabled, I want to figure out a clean way
to do this without requiring the overhead of third-party
utilities (such as qemu).

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-06-01 20:26:38 +00:00
gjb
f930e15914 Fix virtual machine disk format creating by passing VMFORMAT
to mkimg(1)

PR:		200068
Submitted by:	Jeremy Norris
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-05-11 14:53:07 +00:00
gjb
fb117627f6 Merge ^/projects/release-arm-redux into ^/head.
Of note:

- This commit adds native FreeBSD/arm release build support without
  requiring out-of-tree utilities.

- Part of this merge removes the WANDBOARD-{SOLO,DUAL,QUAD} kernel
  configuration files, for which the IMX6 kernel configuration file
  should be used instead.

- The resulting images have a 'freebsd' user (password 'freebsd'),
  to allow ssh(1) access when console access is not available (VGA
  or serial).  The default 'root' user password is set to 'root'.

- The /etc/ttys file for arm images now enable both ttyv0 and ttyu0
  by default.

Help from:	many (boot testing, feedback, etc.)
Sponsored by:	The FreeBSD Foundation
2015-05-09 21:08:12 +00:00
gjb
c8fa25fd06 Fix path to /boot/msdos for the target image.
Sponsored by:	The FreeBSD Foundation
2015-05-08 10:38:32 +00:00
gjb
463d845cba Reorder / and /boot/msdos mount to avoid the following error:
Mounting local file systems:
mount: /boot/msdos: No such file or directory

Sponsored by:	The FreeBSD Foundation
2015-05-07 23:46:33 +00:00
gjb
2bf3795a46 Create the /boot/msdos directory so the FAT partition is mounted
properly.

Noticed by:	sbruno
Sponsored by:	The FreeBSD Foundation
2015-05-07 20:25:17 +00:00
gjb
22dede213e Enable the growfs rc(8) script.
Sponsored by:	The FreeBSD Foundation
2015-05-07 18:49:43 +00:00
gjb
3d9f396598 In arm_create_disk(), chroot(8) to the build environment
is necessary.

In arm_install_base(), chroot(8) when installing world
and kernel.  Fix paths for fstab(5) and rc.conf(5).

Sponsored by:	The FreeBSD Foundation
2015-05-07 17:35:48 +00:00
gjb
af0be5ff22 In release.sh, create the /dev directory within the chroot
before attempting to mount(8) devfs.  Also, create the
.OBJDIR for the 'release' target, so files end up in the
correct location.

In tools/arm.subr, fix the target device when creating the
gpart partition scheme.

Sponsored by:	The FreeBSD Foundation
2015-05-07 00:27:28 +00:00
gjb
db697baa50 Add tools/arm.subr to contain common subroutines used for
building arm images.  This is similar to tools/vmimage.subr
used for building virtual machine disk images.  By default,
only arm_create_disk() and arm_install_base() contain real
functionality here, and arm_install_uboot() must be overridden
in the arm/KERNEL.conf file.

In release.sh, make create_arm_armv6_build_release() do
something now.

In arm/BEAGLEBONE.conf, set IMAGE_SIZE, PART_SCHEME, FAT_SIZE,
FAT_TYPE, and MD_ARGS, as well as make arm_install_uboot()
functional.

Parts of this were taken from disecting a previous BEAGLEBONE
image, and other parts obtained from Crochet sources.

Sponsored by:	The FreeBSD Foundation
2015-05-06 19:58:12 +00:00
gjb
1289a25572 Remove crochet configuration files and tools/arm/ directory.
Sponsored by:	The FreeBSD Foundation
2015-05-06 15:59:29 +00:00
gjb
7ac181d7c0 Fix building BEAGLEBONE images with Crochet using the
sysutils/u-boot-beaglebone port:

 - In arm/BEAGLEBONE.conf, set EMBEDDEDPORTS to the
   sysutils/u-boot-beaglebone port.
 - In arm/release.sh, remove BEAGLEBONE from setting WANT_UBOOT
 - In tools/arm/crochet-BEAGLEBONE.conf, override the
   beaglebone_check_uboot(), and set BEAGLEBONE_UBOOT to
   /tmp/external/u-boot-beaglebone, and create symlinks to the
   u-boot files in /usr/local/share/u-boot-beaglebone and the
   uEnv.txt file in crochet/board/Beaglebone/files.

Sponsored by:	The FreeBSD Foundation
2015-05-05 21:08:50 +00:00
gjb
0f66edf8be Add initial support for building RPI2 images.
In release.sh, allow overriding buildenv_setup() before
the handoff to arm/release.sh.

Copy arm/RPI-B.conf -> arm/RPI2.conf, set UBOOT_PORT and
the correct KERNEL, and add the buildenv_setup() override
to install the sysutils/u-boot-rpi2 port/package.

Copy tools/arm/crochet-RPI-B.conf -> tools/arm/crochet-RPI2.conf,
and set the correct entries for the RaspberryPi2 board.

Thanks to:	loos@
Sponsored by:	The FreeBSD Foundation
2015-05-05 16:28:04 +00:00