This is useful for adding extra packages to the build of an AMI.
For example:
env VM_EXTRA_PACKAGES="zsh" make -C release ec2ami
Approved by: gjb
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Currently pkg(8) will fail to install any package if one is missing, so
make this a loop to prevent one missing package from preventing the rest
from installing. Seen building an AWS AMI for aarch64 on main and
ebsnvme-id is not available in the repo at the moment.
Approved by: gjb
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
Per [(1), i.e. test(1), the string comparison operator should be `=`,
not `==` in sh(1) scripts.
No functional change. FreeBSD's test accepts `==` as an alias for `=`
for compatibility with gnu test's `==` extension.
Reviewed by: imp@, gjb@
Pull Request: https://github.com/freebsd/freebsd-src/pull/485
Remove tmpfs size limitation, this breaks make installworld and installation of some packages
Format generated fstab using tabs to make it consistent and readable
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D29283
truncate(1) is not case-sensitive with regard to setting the size
of a file. makefs(8), however, does not honor upper-case values.
Update release-specific files and the release(7) manual page to
reflect this.
MFC with: 1ca8842f3a
Submitted by: ehem_freebsd_m5p.com (original)
Differential Review: https://reviews.freebsd.org/D28979
Sponsored by: Rubicon Communications, LLC ("Netgate")
This had prevented the bootconfig step from determining if an ESP exists,
resulting in its unconditional setup. On BIOS-booted amd64, this wasn't
harmful, just unnecessary, but it resulted in failed installations on
non-EFI-supporting platforms like powerpc64.
MFC after: 3 days
Using makefs instead reduces the privileges needed to build VM images,
simplifies the script (no need to copy files to a fresh image at the end),
and improves portability by allowing generation of cross-endian images.
As a result of the last, this patch also adds support for generation of
powerpc64 and powerpc64le VM images.
No other changes to the output. Tested and working for both amd64 and
powerpc64 targets.
Reviewed by: gjb
Differential Revision: https://reviews.freebsd.org/D28912
RISC-V has the same booting requirements as arm64 (loader.efi, no legacy
boot options), so generated images for both architectures have the same
partition layout.
Reviewed by: gjb
Differential Revision: https://reviews.freebsd.org/D27044
Since the few existing RISC-V hardware platforms are single board
computers, we can piggyback off of arm/arm64's embedded build support
for generating SD card images.
I don't see a pressing need to change the naming in this file at this
time.
Reviewed by: gjb, manu
Differential Revision: https://reviews.freebsd.org/D27043
A lot of projects CI can't do FreeBSD tests currently.
The main reason is that the project CI infrastructure is runned on Linux
and that our images aren't modifiable from a Linux hosts.
Add a basic image specific for this case (called BASIC-CI for a lack of a
better name).
The image have no package pre-installed.
It only have a few modification to have dhcp client runned on the default
interface and sshd started with option to be able to log on without a password
as root.
Sponsored by: The FreeBSD Foundation
Reviewed by: re (gjb@)
Differential Revision: https://reviews.freebsd.org/D25598
These images were switched to MBR in r281876 as a way to cope with a
hard-coded partition GUID in QEMU's default EFI firmware. Enough time
has passed that this is no longer a problem; QEMU versions >= 4.0
include a copy of edk2 EFI firmware that can detect the root filesystem
properly. Alternatively, sysutils/u-boot-qemu-arm64 can be used.
Switch back to building these images with a GPT partition table, and
re-enable the swap partition.
Reviewed by: gjb, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26986
The return code of write_partition_layout() doesn't bubble up, so an
invocation of make vm-release for an incorrect/unsupported target will
appear to succeed while make vm-install will fail due to missing
files. This isn't a common point of failure, but is worth handling
properly.
Upgrade this case to print a message to stderr, and exit in place. This
is okay to do since at this point in the execution of mk-vmimage.sh,
cleanup() has already been run.
Reviewed by: gjb
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26985
De-duplicate the invocation of mkimg(1). No functional change.
Reviewed by: gjb
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26984
Prior to this commit, EC2 AMIs used a "dual-dhclient" tool which was
launched in place of dhclient and spawned both the base system dhclient
for IPv4 and the ISC dhclient from ports for IPv6.
Now that rtsold supports the "M bit" (managed configuration), we can go
back to having the base system dhclient spawned normally, and provide a
script to rtsold which spawns the ISC dhclient from ports when rtsold
decides that it is appropriate.
Thanks to: bz
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
The FreeBSD base system continues to expand. 4GB is now insufficient;
we passed 3 GB in May 2019; we passed 2 GB in August 2017. Over half
of the disk space used is in /usr/lib/debug/.
Without this change, instances boot but are unusable, since the first
thing which breaks when VM filesystems are too small is the "pkg install"
in the VM building process.
EFI support is a hard requirement for generating Hyper-V Gen2 VM images.
Reviewed by: gjb
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25655
This adjusts freebsd-update.conf and portsnap.conf files in EC2 AMIs to
point at the new AWS-hosted mirror network.
Approved by: re (delphij)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D25498
The ebsnvme-id utility exposes information about EC2 disks -- for
Elastic Block Store volumes, their volume IDs and "linux device
names", and for Instance Store (aka "Ephemeral") disks, their
serial numbers.
The dev_aws_disk rc.d script and associated devd.conf rule maintains
a tree under /dev/aws/disk:
/dev/aws/disk/ebs/<volume ID>
/dev/aws/disk/linuxname/<linux device name>
/dev/aws/disk/ephemeral/<serial number>
which are symlinks to the corresponding nda or nvd devices.
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
Since Amazon Elastic File System is only available within AWS, it seems
more appropriate to have this added only in EC2 AMIs rather than
"polluting" non-EC2 images with it.
Reviewed by: gjb
MFC after: 7 days
Relnotes: Amazon EFS filesystems can be automounted by enabling autofs
and placing "/efs -efs" into /etc/auto_master.
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D24791
FreeBSD is in the process of switching from nvd(4) to nda(4) as the disk
device front-end to NVMe. Changing the default in the kernel is tricky
since existing systems may have /dev/nvd* hard-coded e.g. in /etc/fstab;
however, there's no reason to not change the default in HEAD for *new*
systems.
At present I have no intention of MFCing this to stable branches, since
someone might reasonably expect scripts they use for launching and
configuring FreeBSD 12.1 instances to work with FreeBSD 12.2 AMIs, for
example.
Reviewed by: gjb, imp
Relnotes: NVMe disks in EC2 instances launched from 13.0 and later
now show up as nda(4) devices.
Differential Revision: https://reviews.freebsd.org/D24583
I forgot to do this as part of r345858 -- I added it to the
vm_extra_pre_umount in vmimage.subr but forgot that function
was overridden in the EC2 build.
MFC after: 2 weeks
Make stock FreeBSD more useful for people wishing to use them. The
QEMU folks suggested this change. It adds a serial console which
allows them to interact with FreeBSD from the earliest moments. This
allows them to configure FreeBSD via the serial port to set it up for
CI use.
Reviewed by: kevans@
Sponsored by: Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D22786
SRC_CONF, __MAKE_CONF and SRCCONF, respectively) through
to arm_install_base() and chroot_arm_build_release().
This prevents failures when the target image is intended
to be build with make.conf(5) and src.conf(5) overrides,
which are correctly handled for non-embedded image builds.
Reported and tested by: Daniel Engberg
PR: 238615
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
The fix to override the default python version when building
the sysutils/py-google-compute-engine did not work, and there
are still issues that need to be addressed in the port itself.
See bugzilla 238267 for additional details.
MFC after: 6 days
MFC with: r348438
MFC note: no-op to appease the merge tracker
Sponsored by: The FreeBSD Foundation
size is too small to bootstrap the firstboot_pkgs list.
While here, add the growfs(8) startup script to /etc/rc.conf,
as Vagrant images can be resized by modifying the Vagrantfile.
Reported by: dbaio
PR: 238226
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
The ports/head branch recently switched to python3 as the default,
which breaks the sysutils/py-google-compute-engine startup scripts,
as lang/python installs lang/python3{,.x} where lang/python2{,.x}
are needed.
Set DEFAULT_VERSIONS in release/tools/gce.conf to python=2.7, and
remove the lang/python3 inclusion in VM_EXTRA_PACKAGES.
Additionally, unset DEFAULT_VERSIONS in release/tools/vmimage.subr
to prevent persistence of DEFAULT_VERSIONS=python=2.7 in subsequent
VM/cloud image builds.
Note: at present, this affects only 13-CURRENT and 12-STABLE, as
the stable/11 branch had already switched to using the 2019Q2 branch
at the start of the 11.3-RELEASE cycle, so this does not immediately
affect 11.3-BETA, hence the 1-week merge timeout. This had been
manually tested on 13-CURRENT.
Reported by: ler (privately)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
AMIs have been breaking for the past month due to insufficient disk space.
Due to the small amount of overhead in the disk image, the EC2 AMIs end
up with the same (4GB) minimum disk size.
Reported by: Michal Krawczyk
30GB to 3GB. The raw images can be resized using truncate(1), and
other formats can be resized with tools included with other tools
included with other hypervisors.
Enable the growfs(8) rc(8) at firstboot if the disk was resized
prior to booting the virtual machine for the first time.
Discussed with: several
PR: 232313 (requested in other context)
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
If MACHINE_ARCH doesn't match TARGET_ARCH, and we're not in the special
case of building i386 images on an amd64 host, we need to pull in the
qemu-user-static package; this allows us to run some commands inside
the VM disk image chroot, most notably to install packages.
Reviewed by: gjb
MFC after: 2 weeks
Sponsored by: FreeBSD/EC2 patreon (https://www.patreon.com/cperciva)
This script broke around FreeBSD 11.0 as a result of SWAPMETA no longer
being reported by vmstat -z; but it also needs to be reworked due to the
arrival in EC2 of nvme ephemeral disks.
I'll turn this option back on after I've found time to rewrite the
script in question.
PR: 234686
Reported by: meta@
MFC after: 1 week
Currently, the installer uses pre-created 800KB FAT12 filesystems that
it dd's onto the ESP partition.
This changeset improves that by having the installer generate a FAT32
filesystem directly onto the ESP using newfs_msdos and then copying
loader.efi into /EFI/freebsd.
For live installs it then runs efibootmgr to add a FreeBSD boot entry
in the BIOS.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D17947
Add a configuration for PINEBOOK image.
Pinebook is a arm64 laptop based on a Pine64 board.
Since the usb trackpad need a quirk, add a common function for adding
quirk for arm board.
A default one is supplied as most board to not need quirks.
Reviewed by: gjb
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D18337
lualoader has moved to a model where the user is expected to disable color
as desired, rather than disabling it automatically for serial boots, due to
more wide-spread support for color sequences.
In a similar vain, though also to reduce special cases, lualoader no
longer disables the beastie menu automatically for !x86. This was done in
Forth land with a different loader.rc that simply didn't invoke the menu
routines, thus wasn't necessary.
This set of changes puts release images back to how they would've been
experienced prior to the switch to Lua.
Approved by: re (rgrimes)
the probing and attaching of the PS/2 mouse (not present on EC2) and
keyboard (emulated, but not accessible via EC2).
Note that we disable atkbd0 separately even though during device probing
it shows up as a child of atkbdc0; this is necessary because the device
is also initialized during the early console setup from hammer_time.
This change cuts the kernel boot time on an EC2 c5.4xlarge instance from
7259ms down to 4727 ms.
Approved by: re (marius)
