Commit Graph

7744 Commits

Author SHA1 Message Date
cperciva
574fb3705e Check that EC2SNSTOPIC and EC2PUBLIC are non-empty, along with being
defined, in order to avoid problems with thermite.sh.

Reported by:	gjb
2018-07-17 23:16:05 +00:00
cperciva
6fb664c333 Teach EC2 AMI release code to send an SNS notification after publishing
new AMIs if EC2SNSTOPIC is defined.

Reviewed by:	gjb
Requested by:	Amazon
2018-07-17 19:05:30 +00:00
brd
aad9103efa Simplify using bsd.endian.mk and have it provide CAP_MKDB_ENDIAN, since it is
the most common usage.

Approved by:	bapt (mentor)
2018-06-28 13:48:59 +00:00
brd
99a956a88b Chase the pwd_mkdb endian changes.
Approved by:	bapt (mentor)
2018-06-27 19:10:32 +00:00
trasz
5d969cbc56 Enable USB OTG serial terminal on ARM SD card images. This configures
the system to make use of USB device mode / USB OTG to provide a "virtual
serial port" on release images.

Reviewed by:	gjb@
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15602
2018-06-12 16:45:52 +00:00
marius
d07944164c - Once we have shifted arguments up to thrice, base-bits-dir is $1 rather
than $4. Introduce $BASEBITSDIR for clarity and to avoid repeating this
  mistake in the future. Fixing this ensures that we pick up newly built
  boot bits native to the target rather for/from the host.
- Apply some of the argument quoting fixes done in r287635 but missing in
  later revisions.
2018-06-07 18:24:25 +00:00
emaste
a0d7989cff switch amd64 memstick installer images to MBR
A good number of BIOSes have trouble booting from GPT in non-UEFI mode.
This is commonly reported with Lenovo desktops and laptops (including
X220, X230, T430, and E31) and Dell systems.  Although UEFI is the
preferred amd64 boot method on recent hardware, older hardware does not
support UEFI, a user may wish to boot via BIOS/CSM, and some systems
that support UEFI fail to boot FreeBSD via UEFI (such as an old
AMD FX-6100 that I have).

With this change amd64 memsticks remain dual-mode (booting from either
UEFI or CSM); the partitioning type is just switched from GPT to MBR.

The "vestigial swap partition" in the GPT scheme was added in r265017 to
work around some issue with loader's GPT support, so we should not need
it when using MBR.

There is some concern that future UEFI systems may not boot from MBR,
but I am not aware of any today.  In any case the likely path forward
for our installers is to migrate to CD/USB combo images, and if it
becomes necessary introduce a separate memstick specifically for the
MBR BIOS/CSM case.

PR:		227954
Reviewed by:	gjb, imp, tsoome
MFC after:	3 days
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15599
2018-05-29 15:06:13 +00:00
trasz
7f1860cf19 Revert r333493, which was a temporary fix for 11.2-RELEASE, and instead
switch the default kldxref_enable to YES.

The reason is that it's required for every image that's being cross-built,
as kldxref(8) cannot handle files for non-native architectures.  For the
one that is not - amd64 - having it on by default doesn't change anything;
the script is noop if the linker.hints already exists.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-05-26 11:13:17 +00:00
manu
745aef8c6a release: arm64: Use loader.efi as bootaa64.efi for RPI3 and PINE64
boot1.efi have some trouble to read MBR partitions, it needs them to be
aligned a certain way while loader.efi can cope with them either way.
We want to switch to loader.efi as the main efi loader everywhere, it seems
that arm64 using MBR partition will be the guinea pig.

Tested On: RPI3, Pine64
Reviewed by:	imp
Approved by:	gjb
2018-05-21 11:58:02 +00:00
manu
0114ba5b98 release: rpi3: Copy the special rpi3 config.txt
RPI* 32bits and RPI* 64bits have a different config.txt
Copy to correct config.txt to the fat partition of the release image.
Also copy pwm.dtbo as some people want to use it.

Reviewed by:	gjb
2018-05-17 19:10:13 +00:00
manu
11595a1939 release: arm: Format FAT partition as FAT16
r332674 raised the size of the FAT partition from 2MB to 41MB for some
boards. But we format them in FAT12 and this size appears to be to big
for FAT12 and some SoC bootrom cannot cope with that.
Format the msdosfs partition as FAT16,

PR:		228285
MFC after:	soon
2018-05-17 16:21:12 +00:00
trasz
6f31c71ba1 Set kldxref_enable="YES" for ARM images. Without it, the images are missing
the /boot/kernel/linker.hints file, which breaks loading some of the modules
with dependencies, eg cfiscsi.ko.

This is a minimal fix for ARM images, in order to safely MFC it before
11.2-RELEASE.  Afterwards, however, I believe we should actually just change
the default (as in, etc/defaults/rc.conf).  The reason is that it's required
for every image that's being cross-built, as kldxref(1) cannot handle files
for non-native architectures.  For the one that is not - amd64 - having it
on by default doesn't change anything - the script is noop if the linker.hints
already exists.

The long-term solution would be to rewrite kldxref(1) to handle other
architectures, and generate linker.hints at build time.

Reviewed by:	gjb@
MFC after:	3 days
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14534
2018-05-11 14:52:35 +00:00
gjb
06e9cdaa58 Add a special GCE_LICENSE variable to Makefile.gce, which when set,
will include license metadata in the resultant GCE image.

GCE_LICENSE is unset by default, as it primarily pertains to images
produced by the FreeBSD Project, but for downstream FreeBSD consumers,
it can be set in the make(1) environment in the format of:

 --licenses="projects/PROJECT_ID/global/licenses/LICENSE_NAME"

The "license" is not a license, per se, but required metadata that
is required by the GCE marketplace.  For the FreeBSD Project, the
license name is simply 'freebsd', with the description of 'FreeBSD'.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-05-10 21:46:58 +00:00
gjb
d6196e6379 Use vYYYYMMDD in the timestamp suffix for Google Compute Engine
snapshot images for consistency with other OSes.

MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
2018-05-08 17:15:12 +00:00
gjb
01f9440a78 Fix a typo.
Submitted by:	lidl
MFC after:	3 days
MFC with:	r333262
Sponsored by:	The FreeBSD Foundation
2018-05-04 21:17:29 +00:00
gjb
bd07da39ba Ensure the ports and src trees are available on GCE images,
satisfying a requirement to allow FreeBSD to be considered
a top-tier supported OS in Google Compute Engine.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-05-04 20:38:26 +00:00
benno
498102ca2b Allow etdump, makefs and mkimg to be overridden.
Recent changes to makefs and mkimg have led to situations where the
disconnect between this script and the versions installed on the host cause
failures. Provide a way to work around this that doesn't require the
installation of new versions to the host system if that's not desired.

With this change mkisoimages.sh will honour the $ETDUMP, $MAKEFS and $MKIMG
environment variables but fall back to the previous behaviour of finding them
within $PATH.

Reviewed by:	gjb
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D15181
2018-04-25 18:47:52 +00:00
gjb
f917d4fb89 Increase the msdosfs partition size on arm SoC images where the
current size may not be sufficiently large for development and/or
testing.

PR:		227548
Submitted by:	trasz
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-04-18 14:34:33 +00:00
cperciva
211c3ab0c5 Move debug.{trace,debugger}_on_panic and kern.panic_reboot_wait_time in
EC2 instances from sysctl.conf to loader.conf; these can all be set as
loader tunables, and setting them in loader.conf gives us the right
behaviour in the event of a kernel panic taking place prior to when
sysctl.conf is processed.

MFC after:	1 week
2018-04-18 05:58:27 +00:00
benno
55fb52d3af Generate hybrid ISO images for amd64.
This keeps the existing El Torito entries for BIOS and UEFI boot code and
adds a GPT in the ISO image's System Area containing boot code for BIOS that
will load /boot/loader from the ISO filesystem and execute it. We then use
etdump to find the EFI System Partition image in the El Torito catalog and
add an entry to the GPT that allows EFI to find it.

Reviewed by:	gjb, imp
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D15051
2018-04-17 22:33:41 +00:00
emaste
717780fd17 switch i386 memstick installer images to MBR
Some BIOSes have trouble booting from GPT in non-UEFI mode.  This is
commonly reported with Lenovo laptops, including my x220.  As we do not
currently support booting FreeBSD/i386 via UEFI there's no reason to
prefer GPT.

The "vestigial swap partition" was added in r265017 to work around an
issue with loader's GPT support, so we should not need it when using
MBR.

We may want to make the same change to amd64, although the issue there is
mitigated by such systems booting via UEFI in the common case.

PR:		227422
Reviewed by:	gjb
MFC after:	3 weeks
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
2018-04-12 19:00:22 +00:00
emaste
e6f9b529dc make-memstick.sh: put partition args on separate lines
This makes it easier to identify the individual partition types and
facilitates comparisons across architectures.

Reviewed by:	gjb
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-04-10 19:49:04 +00:00
gjb
0b8ed22c31 Following r331292, many of the files (such as the LICENSE file)
have moved from the u-boot-rpi3 share directory to the default
rpi-firmware share directory.  Remove the files from UBOOT_FILES
and append the DTB file to a DTB_FILES list so the correct path
is used, fixing a build failure.

Sponsored by:	The FreeBSD Foundation
2018-04-10 17:40:44 +00:00
benno
eee4209a63 Fix the position of $bootable so that -o platformid=efi applies correctly.
Sponsored by:	iXsystems, Inc.
Pointy hat to:	benno (again)
2018-04-10 02:49:53 +00:00
benno
d286d66602 Fix makefs invocation after r331843.
Reported by:	gjb
Reviewed by:	gjb
Sponsored by:	iXsystems, Inc.
Pointy hat to:	benno
2018-04-10 02:43:07 +00:00
gjb
73bfbf4b92 Add logic for "families" for GCE images.
This allows for GCE consumers to easily detect the latest major
version of FreeBSD when using the gcloud command line utility.

To ensure snapshot builds do not conflict with release-style
builds (ALPHA, BETA, RC, RELEASE), the '-snap' suffix is appended
to the GCE image family name.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-03-30 19:08:37 +00:00
manu
c86c565b9d release: arm: Copy boot.scr from ports
Latest u-boot update need u-boot script to load and start ubldr.
(See D14230 for more details)
Copy this file for our arm release on the fat partition.

Approved by:	gjb
2018-03-26 18:39:38 +00:00
gjb
8f5156d5ad Escape trailing newlines in a long variable list for consistency.
Submitted by:	garga
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-03-26 18:24:16 +00:00
benno
733307e991 Allow makefs to properly tag UEFI El Torito boot images. Use them in amd64 ISOs.
UEFI booting requires an EFI System Partition (ESP). On most storage devices
this will be in a specific partition type. To allow booting from CD/ISO
filesystems, UEFI will look for an ESP in the form of a FAT filesystem image
embedded in the image. Historically FreeBSD has added one of these to its
amd64 ISO images but marked it as simply another i386 boot image. Luckily for
us most UEFI implementations are rather forgiving and work this out for us.

This change adds the ability to mark a boot image as being a UEFI image. It
also modifies our ISO generation to use this marking for the UEFI image we
embed.

Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	emaste, imp
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14809
2018-03-23 20:56:18 +00:00
gjb
e62fa66429 Remove google_accounts_manager from VM_RC_LIST in the GCE configuration
file, no longer needed.

PR:		221714
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-03-22 17:49:27 +00:00
gjb
a8063f7679 Following ports commit r465175, avoid fetch(1) to obtain
overlays and DTB files for the RPI3.

Submitted by:	manu
Sponsored by:	The FreeBSD Foundation
2018-03-21 14:36:16 +00:00
kevans
1bbebcb5fb pkgbase: Fix post-install script for kernel packages
kernel.ucl uses a hardcoded boot/kernel for kldxref, which is the incorrect
directory when we're installing extra kernels that aren't the "default"
kernel (placed at boot/kernel).

Fix this by instead using a new %KERNELDIR% that we now replace in
Makefile.inc1 with "kernel" for the default kernel and "kernel.${_kernel}"
for these extra kernels so that, e.g. /boot/kernel.SHIVA, will get properly
kldxref'd upon update and avoid outdated linker.hints.

Reviewed by:	gjb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14689
2018-03-14 14:45:57 +00:00
gjb
9b98636c9d Bump the size of virtual machine disk images from 20G to 30G,
providing more space for a local buildworld to succeed without
attaching separate disks for /usr/src and /usr/obj.

Reported by:	mckusick
MFC after:	3 days

Sponsored by:	The FreeBSD Foundation
2018-02-26 19:26:59 +00:00
andrew
afe637bbfd Put the pine64 root filesystem on teh correct partition.
The Pine64 root filesystem was incorrectly created directly on the MBR
partition. This can cause the loader to get confused when loading the
kernel from this filesystem.

The loader will see this as a small partition meaning later checks to
ensure it doesn't read past the end of the disk incorrectly report a
failure. This seems to work mostly by accident with the released images as
they are smaller than the reported size, however after growfs has run the
image may no longer boot.

Reviewed by:	gjb, emaste, imp
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14343
2018-02-16 16:22:54 +00:00
swills
6f73837b39 Change installer default to not install ports tree
Reviewed by:	gjb, dteske, allanjude, bdrewery, mat
Approved by:	gjb
Differential Revision:	https://reviews.freebsd.org/D14064
2018-01-30 16:34:56 +00:00
gjb
54946249a5 Bump Copyright year following r328283.
MFC after:	3 days
MFC with:	r328283
Sponsored by:	The FreeBSD Foundation
2018-01-23 16:48:31 +00:00
gjb
5fc81d4e46 When CHROOTBUILD_SKIP is set, evaluate the existence of /bin/sh
within the CHROOTDIR.  If it does not exist, unset CHROOTBUILD_SKIP
to prevent build failures.

Requested by:	swills
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-01-23 16:41:31 +00:00
nwhitehorn
b678b9e681 After removal of loader.ps3, change petitboot configuration in release media
to directly kexec the kernel. Unlike the old loader.ps3 code, this also works
on PowerNV systems, which also use petitboot.

MFC after:	1 month
2018-01-01 03:33:01 +00:00
gjb
1f9b8d0bcc Fix the reldoc target in release/Makefile following doc commit r51264.
Sponsored by:	The FreeBSD Foundation
2017-12-07 20:00:19 +00:00
cperciva
e05626901c Make EC2 instances use Amazon's NTP service for time synchronization.
Since Amazon provides NTP servers within their network, this should
be far superior to using the default NTP pools; and since the service
is provided by Amazon there's very little risk in enabling it by
default.  (If someone is able to compromise Amazon's NTP servers and
exploit them to attack EC2 instances, they would almost certainly be
able to compromise EC2 instances even without ntpd running...)

MFC after:	1 week
Relnotes:	EC2 instances now keep their clocks synchronized using
		the Amazon Time Sync Service (aka. NTP).
2017-12-05 09:22:14 +00:00
cperciva
db1e9749f5 Resurrect r321659: Turn off ChallengeResponseAuthentication for EC2 AMIs.
EC2 instances are normally launched with an SSH public key specified,
which is then used for logging in (by default, as 'ec2-user').  Having
ChallengeResponseAuthentication enabled (as FreeBSD's default sshd_config
does) has no functional effect in a new EC2 instance, since you can't log
in using a password until a password has been set -- but having this
enabled results in alerts from automated scanning tools which can detect
that sshd advertises support for keyboard-interactive logins (since they
can't detect that accounts have no password set).

EC2 users who want to use passwords to log in to their instances will need
to set 'ChallengeResponseAuthentication yes' in FreeBSD 12.0 and later.

Discussed with:	gjb, gtetlow, emaste, des
Requested by:	Amazon
X-MFC:		No
Relnotes:	ChallengeResponseAuthentication is turned off by default in
		Amazon EC2 AMIs.
2017-12-05 09:08:48 +00:00
gjb
8b13b248ad Fix port build flags passed to make(1) after r326315, where
it was missed for embedded image builds.

MFC after:	3 days
MFC with:	r326315
Sponsored by:	The FreeBSD Foundation
2017-11-30 20:53:57 +00:00
gjb
421bc6a3f1 Fix an indentation nit.
Sponsored by:	The FreeBSD Foundation
2017-11-30 20:52:01 +00:00
gjb
fe40e02df7 Correct a comment after r326330.
MFC after:	3 days
MFC with:	r326330
Sponsored by:	The FreeBSD Foundation
2017-11-28 18:08:14 +00:00
gjb
157d6ec6df Add a comment to release/release.conf.sample documenting
EMBEDDEDPORTS. [1]

Remove and update stale documentation from release(7) while here.

PR:		206344 [1]
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-28 18:02:58 +00:00
gjb
d59286dc30 Set DISTDIR and WRKDIRPREFIX when building ports within the
chroot(8) to avoid mtime changes within the ports checkout,
which can cause checksum differences.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-28 16:00:27 +00:00
gjb
e54ec1c5ff Remove /etc/resolv.conf from virtual machine images, which is
copied from the build host.  It is renamed to /etc/resolv.conf.bak
on boot, so never used anyway.

Noticed by:	peter
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-21 18:02:18 +00:00
gjb
b798bfb481 Use chroot(8) when invoking realpath(1) when setting BOOTFILES,
otherwise realpath(1) exits due to a nonexistent directory.

Sponsored by:	The FreeBSD Foundation
2017-11-20 15:03:03 +00:00
brd
223aefb522 Add missing call to services_mkdb to build the services.db
Approved by:	gjb
2017-11-19 02:16:11 +00:00
pfg
9da7bdde06 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00