Commit Graph

7970 Commits

Author SHA1 Message Date
Glen Barber
a7976e867f Increase the default size of the GCE disk image from 3GB to 20GB,
as 3GB is too small as discovered in this week's snapshot builds.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-05-03 17:23:08 +00:00
Glen Barber
cbc5290710 Reduce the default image size for virtual machine disk images from
30GB to 3GB.  The raw images can be resized using truncate(1), and
other formats can be resized with tools included with other tools
included with other hypervisors.

Enable the growfs(8) rc(8) at firstboot if the disk was resized
prior to booting the virtual machine for the first time.

Discussed with:	several
PR:		232313 (requested in other context)
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-04-30 14:29:09 +00:00
Colin Percival
1ecaa3f75e Add support for cross-building cloudware images.
If MACHINE_ARCH doesn't match TARGET_ARCH, and we're not in the special
case of building i386 images on an amd64 host, we need to pull in the
qemu-user-static package; this allows us to run some commands inside
the VM disk image chroot, most notably to install packages.

Reviewed by:	gjb
MFC after:	2 weeks
Sponsored by:	FreeBSD/EC2 patreon (https://www.patreon.com/cperciva)
2019-04-03 21:54:47 +00:00
Glen Barber
6a9f9e9b95 Bump the IMAGE_SIZE for arm64 SoC images to prevent failures due
to full filesystem.  This makes the size of the arm64 SoC images
consistent with armv6 and armv7.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-03-21 14:17:55 +00:00
Colin Percival
b55bb76366 Only install amazon-ssm-agent into amd64 AMIs.
This package does not exist on aarch64 at present.
2019-03-20 07:24:21 +00:00
Colin Percival
01b7329605 Pass --arm64 to bsdec2-image-upload when building ARM64 AMIs.
Future commits will allow the resulting EC2 AMIs to actually boot and
be usable.
2019-03-20 07:19:09 +00:00
Colin Percival
827e6ec07d Fix sed script to insert Amazon NTP server into ntp.conf once rather
than twice.

Reported by:	Rafal Lukawiecki
MFC after:	1 week
2019-02-19 23:24:39 +00:00
Emmanuel Vadot
90b924a12e release: arm64: pine64-lts: Use the newly created u-boot-pine64-lts port
In U-Boot 2019.01 there is now a config for this board, use it for the
release image.

MFC after:	1 week
2019-02-04 18:30:47 +00:00
Emmanuel Vadot
ee8a8076a2 release: arm64: rpi3: Install the RPI3B+ DTB file
We should use the correct DTB file otherwise the firmware uses
the RPI3B one.

MFC after:	1 week
2019-02-04 18:29:23 +00:00
Colin Percival
8cb623ce6e Turn off ec2_ephemeralswap for now
This script broke around FreeBSD 11.0 as a result of SWAPMETA no longer
being reported by vmstat -z; but it also needs to be reworked due to the
arrival in EC2 of nvme ephemeral disks.

I'll turn this option back on after I've found time to rewrite the
script in question.

PR:		234686
Reported by:	meta@
MFC after:	1 week
2019-01-09 03:55:25 +00:00
Glen Barber
252373ac4c Escape a new line following r342283.
Sponsored by:	The FreeBSD Foundation
2019-01-02 16:22:33 +00:00
Rebecca Cran
db8b561345 Rework UEFI ESP generation
Currently, the installer uses pre-created 800KB FAT12 filesystems that
it dd's onto the ESP partition.
This changeset improves that by having the installer generate a FAT32
filesystem directly onto the ESP using newfs_msdos and then copying
loader.efi into /EFI/freebsd.
For live installs it then runs efibootmgr to add a FreeBSD boot entry
in the BIOS.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D17947
2018-12-20 19:39:37 +00:00
Ed Maste
eacff37a61 Remove setting of long-unused MK_NCP option
NCP was removed from the tree in 2013 (r248097) and disconnected some
time before.

Reported by:	bz
Sponsored by:	The FreeBSD Foundation
2018-12-20 15:53:27 +00:00
Yuri Pankov
709bc7e0c1 mkisoimages.sh: don't use -p flag when copying loader.efi to msdosfs.
This fixes 'cdrom' target in the case when world was built by user,
and not root.

Reviewed by:	imp
Differential revision:	https://reviews.freebsd.org/D18414
2018-12-03 22:31:57 +00:00
Emmanuel Vadot
d6c31218b6 release: arm64: Add opp dtbo to PINE* boards
MFC after:	1 month
2018-11-29 22:36:36 +00:00
Glen Barber
6b26cacc52 Fix NTP query on GCE due to unresolved hostname.
PR:		232456
Submitted by:	Lucas Kanashiro
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-11-26 17:00:39 +00:00
Emmanuel Vadot
9848333b6c release: arm64: Add PINEBOOK config
Add a configuration for PINEBOOK image.
Pinebook is a arm64 laptop based on a Pine64 board.

Since the usb trackpad need a quirk, add a common function for adding
quirk for arm board.
A default one is supplied as most board to not need quirks.

Reviewed by:	gjb
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D18337
2018-11-26 16:38:39 +00:00
Glen Barber
8643808a68 In followup to r340406, remove doc from the EXTRA_DISTRIBUTIONS
list and remaining references from the script used to create the
MANIFEST file used by bsdinstall(8).

No MFC is planned at this time.

Sponsored by:	The FreeBSD Foundation
2018-11-13 19:53:02 +00:00
Glen Barber
35f6972e41 The roff ascii.gz documentation installed to /usr/share/doc
was removed in r318881 when roff was removed from the base
system.

This results in the doc.txz distribution set containing a
single directory (./) which is empty.

Remove the "Additional documentation" option from the menu
selection of bsdinstall(8), as the plain-text documentation
installed in /usr/share/doc is installed as part of the
packageworld target.

The doc entry has not been removed from EXTRA_DISTRIBUTIONS
in Makefile.inc1, in case its removal triggers an issue with
freebsd-update(8), which is currently aware of the world/doc
component, so the empty doc.txz continues to be created as
a precaution.

Noticed by:	rgrimes
MFC after:	2 days
MFC before:	12.0-RC1
Sponsored by:	The FreeBSD Foundation
2018-11-13 19:51:22 +00:00
Glen Barber
0434c14c14 Pass _BRANCH in the reldoc target to ensure the correct
src branch is used when generating the hardware.html page.

MFC after:	3 days
MFC before:	12.0-BETA3
MFC with:	r339873
Sponsored by:	The FreeBSD Foundation
2018-10-29 21:03:43 +00:00
Glen Barber
4de042ec89 Set OPTIONS_UNSET in the argument list to env(1), and add
AVAHI to the list.  This fixes the textproc/docproj build
seemingly following FLAVORS being added.

Specifically, the problem with the dependency chain here is:
- textproc/docproj depends on print/cups, which sets AVAHI=on
  by default;
- net/avahi-app depends on devel/gobject-introspection, which
  requires python3+;
- graphics/netpbm depends on graphics/mesa-libs, which can
  only be built with python2.7;
- textproc/docproj depends on a number of graphics ports for
  font rendering, etc.

MFC after:	3 days
MFC before:	12.0-BETA3
Sponsored by:	The FreeBSD Foundation
2018-10-29 20:53:05 +00:00
Glen Barber
d4e77c31a2 Reduce the GCE image size to 27G to be lower than the free
quota limit.

PR:		232313
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-10-24 15:51:55 +00:00
Glen Barber
8c14a4095d Add debug.witness.trace=0 back to the installer sysctl.conf(5),
incorrectly removed from head when it should have been removed
from stable/12 post-branch.

Reported by:	bdrewery
Sponsored by:	The FreeBSD Foundation
2018-10-23 14:38:08 +00:00
Ed Maste
a7d9306a40 release: set -e to exit on error in iso image scripts
Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D17651
2018-10-22 19:39:20 +00:00
Glen Barber
b958317950 - Update head to 13.0-CURRENT.
- Bump MACHINE_TRIPLE, TARGET_TRIPLE, FBSD_MAJOR, FBSD_CC_VER,
  FREEBSD_CC_VERSION, OS_VERSION.
- Update comment in UPDATING regarding debugging options.
- Remove debug.witness.trace=0 from installation media.
- Bump __FreeBSD_version.

Approved by:	re (implicit)
Sponsored by:	The FreeBSD Foundation
2018-10-19 00:37:47 +00:00
Kyle Evans
6e49372ee0 Disable kernels_autodetect on installation media
This feature is disabled on install media as these generally won't have any
interesting kernels to be listed other than the default kernel, so the
potential performance penalty in these situations likely isn't worth it.

Approved by:	re (kib)
2018-10-11 17:18:49 +00:00
Glen Barber
e4a1a705bb Add graphics/drm-legacy-kmod and graphics/drm-stable-kmod to the
list of packages to include on the dvd installer.

Approved by:	re (marius, rgrimes)
Sponsored by:	The FreeBSD Foundation
2018-10-10 19:57:10 +00:00
Glen Barber
8046db8b49 Fix the hardware.{txt,html} build in the release/Makefile 'reldoc'
target.

The doc/share/mk/doc.commands.mk sets SVN to /usr/local/bin/svn
by default, which is not necessarily installed by the documentation
project textproc/docproj port.

Ensure SVN can be evaluated properly to include the hardware pages
by iterating through /usr/local/bin and /usr/bin and looking for
both svn and svnlite binaries, and pass the SVN variable explicitly
through env(1) in the reldoc target to avoid failures if it does not
exist.

Approved by:	re (rgrimes)
Sponsored by:	The FreeBSD Foundation
2018-10-04 01:46:56 +00:00
Glen Barber
dc39aae35e Update the pkg-stage.sh script used to populate packages on the
dvd1.iso installation medium from including KDE4 to KDE5, as the
KDE4-based ports have been marked as deprecated in the Ports
Collection.

MFC after:	3 days
Approved by:	re (rgrimes)
Sponsored by:	The FreeBSD Foundation
2018-09-18 16:57:04 +00:00
Kyle Evans
9de0425ed9 release.sh: disable colors and the beastie menu for ARM/ARM64 targets
lualoader has moved to a model where the user is expected to disable color
as desired, rather than disabling it automatically for serial boots, due to
more wide-spread support for color sequences.

In a similar vain, though also to reduce special cases, lualoader no
longer disables the beastie menu automatically for !x86. This was done in
Forth land with a different loader.rc that simply didn't invoke the menu
routines, thus wasn't necessary.

This set of changes puts release images back to how they would've been
experienced prior to the switch to Lua.

Approved by:	re (rgrimes)
2018-08-30 18:00:28 +00:00
Colin Percival
a1b042f79d Disable atkbd0 and atkdbc0 in EC2 AMIs. This has the effect of skipping
the probing and attaching of the PS/2 mouse (not present on EC2) and
keyboard (emulated, but not accessible via EC2).

Note that we disable atkbd0 separately even though during device probing
it shows up as a child of atkbdc0; this is necessary because the device
is also initialized during the early console setup from hammer_time.

This change cuts the kernel boot time on an EC2 c5.4xlarge instance from
7259ms down to 4727 ms.

Approved by:	re (marius)
2018-08-26 03:56:54 +00:00
Emmanuel Vadot
eeab916dad release: arm64: Enable overlays for PINE64 and PINE64-LTS
Approved by:	re (gjb)
2018-08-24 15:01:58 +00:00
Emmanuel Vadot
66e380123e release: arm: Setup overlays if board config defines some
Approved by:	re (gjb)
2018-08-24 15:01:22 +00:00
Colin Percival
7ac2ecc0af Quieten the svn (or svnlite) commands used to extract information from an
SVN checkout for placement into an EC2 AMI.  We only run these if there
is a .svn directory; but in the event that SVN was used to check out a
tree which is then exported over NFS, we were unnecessarily noisy.

Reported by:	Andrey Fesenko
MFC after:	3 days
X-MFC-With:	r336420, r336433, r336593, r336621,
		r336622, r336624, r337394, r337401
2018-08-21 15:30:47 +00:00
Warner Losh
3e6e16b4da Copy the boot loader from the new location for the co-existing
loaders.

Reviewed by: gjb@
2018-08-17 20:41:50 +00:00
Roger Pau Monné
2502c66bbd build: skip the database check when generating install media
There are several scripts and targets solely used to generate install
media, make sure DB_FROM_SRC is used in that case in order to prevent
checking the host database, which is irrelevant when generating
install binaries.

Sponsored by:		Citrix Systems R&D
PR:			230459
Reviewed by:		gjb
Differential revision:	https://reviews.freebsd.org/D16638
2018-08-17 07:27:15 +00:00
Glen Barber
024b984d17 Add a space between a variable and escaped new line.
MFC after:	3 days
MFC with:	r337717
Sponsored by:	The FreeBSD Foundation
2018-08-13 17:24:31 +00:00
Glen Barber
f75f16f8a7 Add lang/python2, lang/python3, and lang/python to GCE images
to help avoid hard-coding 'python<MAJOR>.<MINOR>' in several
scripts in the client-side scripts.

PR:		230248
MFC after:	3 days
Submitted by:	gustavo.scalet@collabora.com
Sponsored by:	The FreeBSD Foundation
2018-08-13 17:23:43 +00:00
Glen Barber
6ac4304a0f Invoke the growfs rc script for each boot on GCE.
PR:		230275
Submitted by:	gustavo.scalet@collabora.com
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-08-09 23:43:10 +00:00
Glen Barber
0366e18e5a Update and replace old rc daemons for GCE images.
PR:		229000
Submitted by:	helen.koike@collabora.com
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-08-09 23:31:18 +00:00
Colin Percival
4222a490f2 Fix copy-and-paste error in previous commit. 2018-08-07 00:51:49 +00:00
Colin Percival
a7ec419d93 Add EC2PUBLICSNAP option to EC2 builds; this passes a (recently added)
flag to bsdec2-image-upload instructing it to mark the snapshot of its
root disk as public (which is independent from marking the created AMIs
as public).

Requested by:	Amazon
2018-08-06 19:21:32 +00:00
Emmanuel Vadot
7b7dbfa9b3 release: arm: Copy the dtb to the fat partition
When booting via EFI on arm we have no way to know the dtb file to load
and we always use the one provided from the bootloader.
This works in most case but :

 U-Boot have some really old DTB for some boards, the sync from Linux isn't done automatically for all boards
 Some boards (like TI BeagleBone series) use one u-boot for all the model and it doesn't embed the DTBs
 Some boards (like IMX6 based ones), don't embed the DTB

We want u-boot to load and patch the DTB with the mac address or the display
node enabled or not.

Reviewed by:	gjb, imp
Differential Revision:	https://reviews.freebsd.org/D16596
2018-08-06 17:21:20 +00:00
Glen Barber
768a8d8d44 Fix the ftp-stage target for arm embedded builds.
The images were renamed from KERNCONF to BOARDNAME when
specified, which would result in an image name of:

 12.0-CURRENT-arm-armv7-GENERIC.img

which would then be renamed to use the BOARDNAME for the
SoC the image is targeted to use.  BOARDNAME was specified
for all images as of r336994, which now causes the ftp-stage
target to fail, as the rename is no longer necessary.

Sponsored by:	The FreeBSD Foundation
2018-08-02 18:51:44 +00:00
Emmanuel Vadot
5a88ea3214 release: arm64: Add PINE64-LTS configuration file
Pine64 isn't produced anymore but Pine64-LTS is.
This image works on the LTS release and the Sopine module.

Reviewed by:	gjb
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D16487
2018-07-31 19:14:52 +00:00
Emmanuel Vadot
a8d0ee755c release: arm: Enable multicons for arm64
Since we have now EFI framebuffer enabled for ARM64 if we boot on a board
with an screen, u-boot will set up a EFI GOP framebuffer and we won't boot
using the serial console.
Also on RPI3 the firmware always setup the framebuffer area resulting in u-boot
always setup the EFI GOP and FreeBSD never using the serial console.

Reviewed by:	gjb, lwshu (previous version)
Differential Revision:	https://reviews.freebsd.org/D16472
2018-07-31 19:13:50 +00:00
Emmanuel Vadot
33bec6889a release: arm: Always boot armv6/v7 image with EFI
boot.scr is a u-boot script that loads and execute ubldr.bin
If not present u-boot will automatically boot loader.efi which
is already installed.

This means that all armv6/armv7 images are now booted via EFI

Tested-On: RPI-B
Tested-On: RPI2
Tested-On: OrangePi One
Tested-On: All lot of other boards
MFC after:	Never
Relnotes:	yes
2018-07-31 19:12:31 +00:00
Emmanuel Vadot
89f81528b3 release: Restore copy of boot.scr for some board
This is not a problem for 12-CURRENT as EFI boot works but it doesn't
for 11.
While here some board arm_install_uboot also copy ubldr.bin et create
firstboot files but it's already done in arm_install_boot

Reviewed by:	gjb
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D16481
2018-07-31 19:10:50 +00:00
Emmanuel Vadot
8608db1eb6 release: Deinstall u-boot ports before installing
FORCE_PKG_REGISTER is broken so multiple invocation of release.sh for the
same board will fails if /scratch isn't cleaned.
Leave it but deinstall the package first.

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D16513
2018-07-31 18:57:11 +00:00
Emmanuel Vadot
3d62046d67 release: arm: Use BOARDNAME in the image name
Using KERNEL made sense when all boards had different kernel configuration.
Now that all of them are using GENERIC use the board name instead

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D16512
2018-07-31 18:55:44 +00:00
Emmanuel Vadot
2e81004069 release: rpi-b: add BOARDNAME variable
Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D16514
2018-07-31 18:54:06 +00:00
Emmanuel Vadot
8b66e85493 release: arm: Use GENERIC kernel for IMX6 based boards
Switch the IMX.6 based board to GENERIC the kernel config.

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D16511
2018-07-31 18:52:43 +00:00
Emmanuel Vadot
4b99ca8e4c release: arm: Add a GENERIC sdcard image
This produce a generic sdcard image using armv7 GENERIC kernel that
just need some u-boot (or none if the board have u-boot or a SPI flash
for example).

Reviewed by:	imp, gjb
Differential Revision:	https://reviews.freebsd.org/D16410
2018-07-28 20:45:36 +00:00
Conrad Meyer
858178a142 Remove insecure ciphers from GCE sshd configuration
They were added for unclear reasons in r277263.  The current OpenSSH
defaults (7.5+) are reasonable, and do not include the insecure rc4 cipher:

                   chacha20-poly1305@openssh.com,
                   aes128-ctr,aes192-ctr,aes256-ctr,
                   aes128-gcm@openssh.com,aes256-gcm@openssh.com,
                   aes128-cbc,aes192-cbc,aes256-cbc

I think I recall there being a reason for a specific list of ciphers on GCE
at the time, but I do not recall what it was, and cannot find any
current GCE documentation of such a list.

So, just revert the explicit configuration and use sane openssh defaults.

PR:		230092
Submitted by:	Gustavo Scalet <gustavo.scalet AT collabora.com>
MFC after:	3 days
Security:	yes
2018-07-28 19:35:49 +00:00
Warner Losh
c680645566 Remove the hopelessly confused GUMSTIX.conf config
This config never worked. At no time did u-boot match the kenrel match
the userland. As all the GUMSTIX gear we support is quite old and/or
not working, remove it. The duovero stuff might work, but nobody
has the hardware for it and GUMSTIX hasn't sold it in years.
2018-07-27 19:25:09 +00:00
Glen Barber
cc893d5822 Fix OL_DIR definition following r336721.
Sponsored by:	The FreeBSD Foundation
2018-07-26 19:42:23 +00:00
Emmanuel Vadot
d904ce8a52 release: Add OL_DIR forgot in r336600
Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D16439
2018-07-25 17:45:56 +00:00
Matt Macy
38c0c78e50 fix use of empty in Makefile.ec2
empty() takes a variable name - not the expanded value

Reported by:	sjg
2018-07-22 23:33:48 +00:00
Matt Macy
16bb9a49ad Correctly reference SRCTOP, avoid multiple invocations of svn
Reported by:	sjg
2018-07-22 23:20:24 +00:00
Matt Macy
87faf66318 fix build non-svn checkouts post r336593
The change made in r336593 assumes that the build is happening in a
svn checkout resulting in misleading debug output. Check that we're
actually working in an svn checkout before proceeding to call svn.
2018-07-22 22:57:52 +00:00
Emmanuel Vadot
016ba82470 release: Add rpi firmware dtb and mmc overlays for RPI-B and RPI2
This is needed with new u-boot that uses the rpi-firmware dtbs.

Reviewed by:	gjb
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D16240
2018-07-22 12:04:21 +00:00
Emmanuel Vadot
43e289df5a release: Add arm_install_boot to install the commit boot bits
This reduce the per-board arm_install_uboot to just install u-boot.
While here remove the installation of rpi.dtb and rpi2.dtb as we load
them from the UFS partition via ubldr.

Reviewed by:	gjb, imp (older version)
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D16239
2018-07-22 12:03:17 +00:00
Emmanuel Vadot
1f8b431d18 release: Uniform release arm image size
Create a 50m fat16 for all boards.

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D16238
2018-07-22 12:02:07 +00:00
Colin Percival
049775cef3 Use svn or svnlite, or ${SVN_CMD} if set, for extracting the SVN branch
and revision number announced in SNS notifications about new EC2 AMIs.

While I'm here, incorporate that information into the AMI "description"
fields, since it's more useful than simply echoing the information
already provided via the AMI "name".

Approved by:	gjb
2018-07-21 22:54:43 +00:00
Colin Percival
43fbd222cc Check that EC2SNSTOPIC and EC2PUBLIC are non-empty, along with being
defined, in order to avoid problems with thermite.sh.

Reported by:	gjb
2018-07-17 23:16:05 +00:00
Colin Percival
a6fe18b04b Teach EC2 AMI release code to send an SNS notification after publishing
new AMIs if EC2SNSTOPIC is defined.

Reviewed by:	gjb
Requested by:	Amazon
2018-07-17 19:05:30 +00:00
Brad Davis
f59e535254 Simplify using bsd.endian.mk and have it provide CAP_MKDB_ENDIAN, since it is
the most common usage.

Approved by:	bapt (mentor)
2018-06-28 13:48:59 +00:00
Brad Davis
7c3f17c5d5 Chase the pwd_mkdb endian changes.
Approved by:	bapt (mentor)
2018-06-27 19:10:32 +00:00
Edward Tomasz Napierala
4fe984fdae Enable USB OTG serial terminal on ARM SD card images. This configures
the system to make use of USB device mode / USB OTG to provide a "virtual
serial port" on release images.

Reviewed by:	gjb@
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15602
2018-06-12 16:45:52 +00:00
Marius Strobl
470f228f62 - Once we have shifted arguments up to thrice, base-bits-dir is $1 rather
than $4. Introduce $BASEBITSDIR for clarity and to avoid repeating this
  mistake in the future. Fixing this ensures that we pick up newly built
  boot bits native to the target rather for/from the host.
- Apply some of the argument quoting fixes done in r287635 but missing in
  later revisions.
2018-06-07 18:24:25 +00:00
Ed Maste
075bac9787 switch amd64 memstick installer images to MBR
A good number of BIOSes have trouble booting from GPT in non-UEFI mode.
This is commonly reported with Lenovo desktops and laptops (including
X220, X230, T430, and E31) and Dell systems.  Although UEFI is the
preferred amd64 boot method on recent hardware, older hardware does not
support UEFI, a user may wish to boot via BIOS/CSM, and some systems
that support UEFI fail to boot FreeBSD via UEFI (such as an old
AMD FX-6100 that I have).

With this change amd64 memsticks remain dual-mode (booting from either
UEFI or CSM); the partitioning type is just switched from GPT to MBR.

The "vestigial swap partition" in the GPT scheme was added in r265017 to
work around some issue with loader's GPT support, so we should not need
it when using MBR.

There is some concern that future UEFI systems may not boot from MBR,
but I am not aware of any today.  In any case the likely path forward
for our installers is to migrate to CD/USB combo images, and if it
becomes necessary introduce a separate memstick specifically for the
MBR BIOS/CSM case.

PR:		227954
Reviewed by:	gjb, imp, tsoome
MFC after:	3 days
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15599
2018-05-29 15:06:13 +00:00
Edward Tomasz Napierala
a4ccdf9091 Revert r333493, which was a temporary fix for 11.2-RELEASE, and instead
switch the default kldxref_enable to YES.

The reason is that it's required for every image that's being cross-built,
as kldxref(8) cannot handle files for non-native architectures.  For the
one that is not - amd64 - having it on by default doesn't change anything;
the script is noop if the linker.hints already exists.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-05-26 11:13:17 +00:00
Emmanuel Vadot
785724755a release: arm64: Use loader.efi as bootaa64.efi for RPI3 and PINE64
boot1.efi have some trouble to read MBR partitions, it needs them to be
aligned a certain way while loader.efi can cope with them either way.
We want to switch to loader.efi as the main efi loader everywhere, it seems
that arm64 using MBR partition will be the guinea pig.

Tested On: RPI3, Pine64
Reviewed by:	imp
Approved by:	gjb
2018-05-21 11:58:02 +00:00
Emmanuel Vadot
78f78afc70 release: rpi3: Copy the special rpi3 config.txt
RPI* 32bits and RPI* 64bits have a different config.txt
Copy to correct config.txt to the fat partition of the release image.
Also copy pwm.dtbo as some people want to use it.

Reviewed by:	gjb
2018-05-17 19:10:13 +00:00
Emmanuel Vadot
2397aecf28 release: arm: Format FAT partition as FAT16
r332674 raised the size of the FAT partition from 2MB to 41MB for some
boards. But we format them in FAT12 and this size appears to be to big
for FAT12 and some SoC bootrom cannot cope with that.
Format the msdosfs partition as FAT16,

PR:		228285
MFC after:	soon
2018-05-17 16:21:12 +00:00
Edward Tomasz Napierala
896eb70c80 Set kldxref_enable="YES" for ARM images. Without it, the images are missing
the /boot/kernel/linker.hints file, which breaks loading some of the modules
with dependencies, eg cfiscsi.ko.

This is a minimal fix for ARM images, in order to safely MFC it before
11.2-RELEASE.  Afterwards, however, I believe we should actually just change
the default (as in, etc/defaults/rc.conf).  The reason is that it's required
for every image that's being cross-built, as kldxref(1) cannot handle files
for non-native architectures.  For the one that is not - amd64 - having it
on by default doesn't change anything - the script is noop if the linker.hints
already exists.

The long-term solution would be to rewrite kldxref(1) to handle other
architectures, and generate linker.hints at build time.

Reviewed by:	gjb@
MFC after:	3 days
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14534
2018-05-11 14:52:35 +00:00
Glen Barber
c3955757a9 Add a special GCE_LICENSE variable to Makefile.gce, which when set,
will include license metadata in the resultant GCE image.

GCE_LICENSE is unset by default, as it primarily pertains to images
produced by the FreeBSD Project, but for downstream FreeBSD consumers,
it can be set in the make(1) environment in the format of:

 --licenses="projects/PROJECT_ID/global/licenses/LICENSE_NAME"

The "license" is not a license, per se, but required metadata that
is required by the GCE marketplace.  For the FreeBSD Project, the
license name is simply 'freebsd', with the description of 'FreeBSD'.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-05-10 21:46:58 +00:00
Glen Barber
43613f74dd Use vYYYYMMDD in the timestamp suffix for Google Compute Engine
snapshot images for consistency with other OSes.

MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
2018-05-08 17:15:12 +00:00
Glen Barber
ef470d031a Fix a typo.
Submitted by:	lidl
MFC after:	3 days
MFC with:	r333262
Sponsored by:	The FreeBSD Foundation
2018-05-04 21:17:29 +00:00
Glen Barber
d55f0e89c8 Ensure the ports and src trees are available on GCE images,
satisfying a requirement to allow FreeBSD to be considered
a top-tier supported OS in Google Compute Engine.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-05-04 20:38:26 +00:00
Benno Rice
6ea2984738 Allow etdump, makefs and mkimg to be overridden.
Recent changes to makefs and mkimg have led to situations where the
disconnect between this script and the versions installed on the host cause
failures. Provide a way to work around this that doesn't require the
installation of new versions to the host system if that's not desired.

With this change mkisoimages.sh will honour the $ETDUMP, $MAKEFS and $MKIMG
environment variables but fall back to the previous behaviour of finding them
within $PATH.

Reviewed by:	gjb
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D15181
2018-04-25 18:47:52 +00:00
Glen Barber
f8b865d1d6 Increase the msdosfs partition size on arm SoC images where the
current size may not be sufficiently large for development and/or
testing.

PR:		227548
Submitted by:	trasz
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-04-18 14:34:33 +00:00
Colin Percival
b6028b0ba1 Move debug.{trace,debugger}_on_panic and kern.panic_reboot_wait_time in
EC2 instances from sysctl.conf to loader.conf; these can all be set as
loader tunables, and setting them in loader.conf gives us the right
behaviour in the event of a kernel panic taking place prior to when
sysctl.conf is processed.

MFC after:	1 week
2018-04-18 05:58:27 +00:00
Benno Rice
0279ac04ed Generate hybrid ISO images for amd64.
This keeps the existing El Torito entries for BIOS and UEFI boot code and
adds a GPT in the ISO image's System Area containing boot code for BIOS that
will load /boot/loader from the ISO filesystem and execute it. We then use
etdump to find the EFI System Partition image in the El Torito catalog and
add an entry to the GPT that allows EFI to find it.

Reviewed by:	gjb, imp
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D15051
2018-04-17 22:33:41 +00:00
Ed Maste
bf249c5a0a switch i386 memstick installer images to MBR
Some BIOSes have trouble booting from GPT in non-UEFI mode.  This is
commonly reported with Lenovo laptops, including my x220.  As we do not
currently support booting FreeBSD/i386 via UEFI there's no reason to
prefer GPT.

The "vestigial swap partition" was added in r265017 to work around an
issue with loader's GPT support, so we should not need it when using
MBR.

We may want to make the same change to amd64, although the issue there is
mitigated by such systems booting via UEFI in the common case.

PR:		227422
Reviewed by:	gjb
MFC after:	3 weeks
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
2018-04-12 19:00:22 +00:00
Ed Maste
8ea90e075c make-memstick.sh: put partition args on separate lines
This makes it easier to identify the individual partition types and
facilitates comparisons across architectures.

Reviewed by:	gjb
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-04-10 19:49:04 +00:00
Glen Barber
32c434538a Following r331292, many of the files (such as the LICENSE file)
have moved from the u-boot-rpi3 share directory to the default
rpi-firmware share directory.  Remove the files from UBOOT_FILES
and append the DTB file to a DTB_FILES list so the correct path
is used, fixing a build failure.

Sponsored by:	The FreeBSD Foundation
2018-04-10 17:40:44 +00:00
Benno Rice
eb5c194032 Fix the position of $bootable so that -o platformid=efi applies correctly.
Sponsored by:	iXsystems, Inc.
Pointy hat to:	benno (again)
2018-04-10 02:49:53 +00:00
Benno Rice
d4bb7978c6 Fix makefs invocation after r331843.
Reported by:	gjb
Reviewed by:	gjb
Sponsored by:	iXsystems, Inc.
Pointy hat to:	benno
2018-04-10 02:43:07 +00:00
Glen Barber
8e29ef317f Add logic for "families" for GCE images.
This allows for GCE consumers to easily detect the latest major
version of FreeBSD when using the gcloud command line utility.

To ensure snapshot builds do not conflict with release-style
builds (ALPHA, BETA, RC, RELEASE), the '-snap' suffix is appended
to the GCE image family name.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-03-30 19:08:37 +00:00
Emmanuel Vadot
60fde7ce5d release: arm: Copy boot.scr from ports
Latest u-boot update need u-boot script to load and start ubldr.
(See D14230 for more details)
Copy this file for our arm release on the fat partition.

Approved by:	gjb
2018-03-26 18:39:38 +00:00
Glen Barber
5db02b50ce Escape trailing newlines in a long variable list for consistency.
Submitted by:	garga
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-03-26 18:24:16 +00:00
Benno Rice
8dbc390581 Allow makefs to properly tag UEFI El Torito boot images. Use them in amd64 ISOs.
UEFI booting requires an EFI System Partition (ESP). On most storage devices
this will be in a specific partition type. To allow booting from CD/ISO
filesystems, UEFI will look for an ESP in the form of a FAT filesystem image
embedded in the image. Historically FreeBSD has added one of these to its
amd64 ISO images but marked it as simply another i386 boot image. Luckily for
us most UEFI implementations are rather forgiving and work this out for us.

This change adds the ability to mark a boot image as being a UEFI image. It
also modifies our ISO generation to use this marking for the UEFI image we
embed.

Reported by:	Thomas Schmitt <scdbackup@gmx.net>
Reviewed by:	emaste, imp
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14809
2018-03-23 20:56:18 +00:00
Glen Barber
c0505015d6 Remove google_accounts_manager from VM_RC_LIST in the GCE configuration
file, no longer needed.

PR:		221714
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-03-22 17:49:27 +00:00
Glen Barber
3377818679 Following ports commit r465175, avoid fetch(1) to obtain
overlays and DTB files for the RPI3.

Submitted by:	manu
Sponsored by:	The FreeBSD Foundation
2018-03-21 14:36:16 +00:00
Kyle Evans
fb6d3be5db pkgbase: Fix post-install script for kernel packages
kernel.ucl uses a hardcoded boot/kernel for kldxref, which is the incorrect
directory when we're installing extra kernels that aren't the "default"
kernel (placed at boot/kernel).

Fix this by instead using a new %KERNELDIR% that we now replace in
Makefile.inc1 with "kernel" for the default kernel and "kernel.${_kernel}"
for these extra kernels so that, e.g. /boot/kernel.SHIVA, will get properly
kldxref'd upon update and avoid outdated linker.hints.

Reviewed by:	gjb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14689
2018-03-14 14:45:57 +00:00
Glen Barber
e89ef0620a Bump the size of virtual machine disk images from 20G to 30G,
providing more space for a local buildworld to succeed without
attaching separate disks for /usr/src and /usr/obj.

Reported by:	mckusick
MFC after:	3 days

Sponsored by:	The FreeBSD Foundation
2018-02-26 19:26:59 +00:00
Andrew Turner
e90c2c3638 Put the pine64 root filesystem on teh correct partition.
The Pine64 root filesystem was incorrectly created directly on the MBR
partition. This can cause the loader to get confused when loading the
kernel from this filesystem.

The loader will see this as a small partition meaning later checks to
ensure it doesn't read past the end of the disk incorrectly report a
failure. This seems to work mostly by accident with the released images as
they are smaller than the reported size, however after growfs has run the
image may no longer boot.

Reviewed by:	gjb, emaste, imp
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14343
2018-02-16 16:22:54 +00:00
Steve Wills
893c3ece10 Change installer default to not install ports tree
Reviewed by:	gjb, dteske, allanjude, bdrewery, mat
Approved by:	gjb
Differential Revision:	https://reviews.freebsd.org/D14064
2018-01-30 16:34:56 +00:00
Glen Barber
6c41d54855 Bump Copyright year following r328283.
MFC after:	3 days
MFC with:	r328283
Sponsored by:	The FreeBSD Foundation
2018-01-23 16:48:31 +00:00
Glen Barber
afac3ed6c3 When CHROOTBUILD_SKIP is set, evaluate the existence of /bin/sh
within the CHROOTDIR.  If it does not exist, unset CHROOTBUILD_SKIP
to prevent build failures.

Requested by:	swills
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-01-23 16:41:31 +00:00
Nathan Whitehorn
9fb36b5eae After removal of loader.ps3, change petitboot configuration in release media
to directly kexec the kernel. Unlike the old loader.ps3 code, this also works
on PowerNV systems, which also use petitboot.

MFC after:	1 month
2018-01-01 03:33:01 +00:00
Glen Barber
496a68519f Fix the reldoc target in release/Makefile following doc commit r51264.
Sponsored by:	The FreeBSD Foundation
2017-12-07 20:00:19 +00:00
Colin Percival
4a93691064 Make EC2 instances use Amazon's NTP service for time synchronization.
Since Amazon provides NTP servers within their network, this should
be far superior to using the default NTP pools; and since the service
is provided by Amazon there's very little risk in enabling it by
default.  (If someone is able to compromise Amazon's NTP servers and
exploit them to attack EC2 instances, they would almost certainly be
able to compromise EC2 instances even without ntpd running...)

MFC after:	1 week
Relnotes:	EC2 instances now keep their clocks synchronized using
		the Amazon Time Sync Service (aka. NTP).
2017-12-05 09:22:14 +00:00
Colin Percival
4ba35bc4db Resurrect r321659: Turn off ChallengeResponseAuthentication for EC2 AMIs.
EC2 instances are normally launched with an SSH public key specified,
which is then used for logging in (by default, as 'ec2-user').  Having
ChallengeResponseAuthentication enabled (as FreeBSD's default sshd_config
does) has no functional effect in a new EC2 instance, since you can't log
in using a password until a password has been set -- but having this
enabled results in alerts from automated scanning tools which can detect
that sshd advertises support for keyboard-interactive logins (since they
can't detect that accounts have no password set).

EC2 users who want to use passwords to log in to their instances will need
to set 'ChallengeResponseAuthentication yes' in FreeBSD 12.0 and later.

Discussed with:	gjb, gtetlow, emaste, des
Requested by:	Amazon
X-MFC:		No
Relnotes:	ChallengeResponseAuthentication is turned off by default in
		Amazon EC2 AMIs.
2017-12-05 09:08:48 +00:00
Glen Barber
30ba43e1a6 Fix port build flags passed to make(1) after r326315, where
it was missed for embedded image builds.

MFC after:	3 days
MFC with:	r326315
Sponsored by:	The FreeBSD Foundation
2017-11-30 20:53:57 +00:00
Glen Barber
c205468e00 Fix an indentation nit.
Sponsored by:	The FreeBSD Foundation
2017-11-30 20:52:01 +00:00
Glen Barber
cd9ef49f4f Correct a comment after r326330.
MFC after:	3 days
MFC with:	r326330
Sponsored by:	The FreeBSD Foundation
2017-11-28 18:08:14 +00:00
Glen Barber
20772e472f Add a comment to release/release.conf.sample documenting
EMBEDDEDPORTS. [1]

Remove and update stale documentation from release(7) while here.

PR:		206344 [1]
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-28 18:02:58 +00:00
Glen Barber
27879b9a70 Set DISTDIR and WRKDIRPREFIX when building ports within the
chroot(8) to avoid mtime changes within the ports checkout,
which can cause checksum differences.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-28 16:00:27 +00:00
Glen Barber
3103eac6a0 Remove /etc/resolv.conf from virtual machine images, which is
copied from the build host.  It is renamed to /etc/resolv.conf.bak
on boot, so never used anyway.

Noticed by:	peter
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-21 18:02:18 +00:00
Glen Barber
2833897f35 Use chroot(8) when invoking realpath(1) when setting BOOTFILES,
otherwise realpath(1) exits due to a nonexistent directory.

Sponsored by:	The FreeBSD Foundation
2017-11-20 15:03:03 +00:00
Brad Davis
6be8e27be0 Add missing call to services_mkdb to build the services.db
Approved by:	gjb
2017-11-19 02:16:11 +00:00
Pedro F. Giffuni
df57947f08 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
Glen Barber
f295b9db79 Add general configuration files used by release/release.sh for
big-iron installation images.

MFC after:	3 days
MFC with:	r325948, r325949, r325950, r325951
Sponsored by:	The FreeBSD Foundation
2017-11-17 18:00:52 +00:00
Glen Barber
41edb6080c Fix indentation.
Sponsored by:	The FreeBSD Foundation
2017-11-17 17:40:53 +00:00
Glen Barber
bcccd559e2 Sort variables for consistency.
Sponsored by:	The FreeBSD Foundation
2017-11-17 17:36:45 +00:00
Glen Barber
d025cec625 Sort variables in arm64 SoC configurations.
Remove an unneeded UBLDR_LOADADDR from RPI3.conf.

Sponsored by:	The FreeBSD Foundation
2017-11-17 17:34:52 +00:00
Glen Barber
14dc747a10 Remove stray SRCBRANCH included by mistake.
Sponsored by:	The FreeBSD Foundation
2017-11-17 17:33:06 +00:00
Glen Barber
d294a5246f Only copy /etc/resolv.conf to ${CHROOTDIR} if /etc/resolv.conf does
not already exist within ${CHROOTDIR}.  This allows re-using a build
chroot with CHROOTBUILD_SKIP set to a non-empty value and CHROOTDIR
set to '/' in release.conf.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-15 19:14:44 +00:00
Emmanuel Vadot
cac201b866 release: Update u-boot and firmware file for RPI2 target
The u-boot port for RPI-2 was updated to use u-boot-master, this cause
an update in u-boot version to v2017.09 and changing the filename.
The various firmware files for the RPI* are now in a common ports
sysutils/rpi-firmware as they are shared on all the RPI version.

Update the release files to copy the right files from the right location.

Reviewed by:	gjb
MFC after:	3 days
2017-11-15 19:04:23 +00:00
Emmanuel Vadot
e69ce19b6a release: Update u-boot and firmware file for RPI-B target
THe u-boot port for RPI-B was updated to use u-boot-master, this cause
an update in u-boot version to v2017.09 and changing the filename.
The various firmware files for the RPI* are now in a common ports
sysutils/rpi-firmware as they are shared on all the RPI version.

Update the release files to copy the right files from the right location.

Reviewed by:	gjb
MFC after:	3 days
2017-11-15 19:03:06 +00:00
Warner Losh
ca987d4641 Move sys/boot to stand. Fix all references to new location
Sponsored by:	Netflix
2017-11-14 23:02:19 +00:00
Glen Barber
760c3a19e2 Correct the path to the md(4)-backed UFS filesystem for pine64
images.

Boot-tested by:	lidl
Sponsored by:	The FreeBSD Foundation
2017-11-14 14:31:02 +00:00
Glen Barber
d412b1d7bc Update the GUMSTIX image build to use arm/arm TARGET/TARGET_ARCH.
Update the TARGET/TARGET_ARCH matching in release/release.sh and
release/Makefile.mirrors for simplification.

Discussed with:	mmel
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-11-13 19:31:51 +00:00
Glen Barber
bb0c3a6b03 Specify WITH_UNIFIED_OBJDIR in chroot_arm_build_release() to
ensure the output image is created in the correct .OBJDIR.

Sponsored by:	The FreeBSD Foundation
2017-11-06 17:59:04 +00:00
Bryan Drewery
f7ef435bb6 Rework r325076: Just use the pre-existing OBJDIR.
Like was done in usr.sbin/mergemaster in r249906, we can just use
the already-built OBJDIR for install(1).

Sponsored by:	Dell EMC Isilon
2017-11-05 22:29:34 +00:00
Emmanuel Vadot
1883caa7cc release/arm: Do not install ubldr
ubldr is the non-pie version of ubldr.bin, do not install two
copies of the same binary. This will allow us to remove ubldr
in the future.
All the u-boot ports know how to load ubldr.bin

Reviewed by:	gjb (earlier version)
2017-11-03 23:02:57 +00:00
Glen Barber
91fcd6f57c Fix an error in the PINE64.conf, where the ubldr is not needed
and the u-boot needs to be written to the partition with dd(1).

Reported by:	manu
Sponsored by:	The FreeBSD Foundation
2017-11-03 19:32:10 +00:00
Glen Barber
5399c35fcc Add a configuration file for building pine64 SoC images.
Parts obtained from:	crochet
Sponsored by:	The FreeBSD Foundation
2017-11-03 19:08:59 +00:00
Glen Barber
0dd40b17a8 Include TARGET and TARGET_ARCH in chroot_arm_build_release()
make(1) invocations following the OBJDIR restructuring to
ensure the output arm SoC image is in the correct directory.

Sponsored by:	The FreeBSD Foundation
2017-11-03 18:54:25 +00:00
Bryan Drewery
dfa099890c Add option UNIFIED_OBJDIR, on by default, which moves the default build OBJDIR.
This changes the build OBJDIR from the older style of /usr/obj/<srcdir> for
native builds, and /usr/obj/<target>.<target_arch>/<srcdir> for cross builds to
a new simpler format of /usr/obj/<srcdir>/<target>.<target_arch>.  This
new format is used regardless of cross or native build.  It allows
easier management of multiple source tree object directories.

The UNIFIED_OBJDIR option will be removed and its feature made permanent
for the 12.0 release.

Relnotes:	yes (don't note UNIFIED_OBJDIR option since it will be removed)
Prior work:	D3711 D874
Reviewed by:	gjb, sjg
Discussed at:	https://lists.freebsd.org/pipermail/freebsd-arch/2016-May/017805.html
Discussed with:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12840
2017-11-01 21:22:05 +00:00
Colin Percival
9d98ae7d5f Add the amazon-ssm-agent package to EC2 AMI builds. This makes it
immediately available on instances which are running without internet
access (or which can't rely on firstboot_pkgs to install it for some
other reason).

Note that this agent is not enabled by default; to enable it, add
amazon_ssm_agent_enable="YES" to /etc/rc.conf, e.g., by placing the lines
	>>/etc/rc.conf
	amazon_ssm_agent_enable="YES"
into the EC2 user-data.  In addition to being enabled, the agent requires
keys to be provided via IAM Roles; users are encouraged to be very careful
in using this functionality due to the inherent vulnerability in the idea
of providing credentials via a service accessible to any process which can
open an HTTP connection.

Requested by:	Amazon
No objection from:	re@
Relnotes:	FreeBSD/EC2 AMIs now include the Amazon EC2 Systems Manager
		(SSM) Agent.
2017-11-01 00:33:54 +00:00
Bryan Drewery
54a03abb68 Run mm-mtree with whatever -j value the build is using.
Reviewed by:	gjb
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12807
2017-10-31 00:03:29 +00:00
Glen Barber
2170fff8a5 Set a default hostname for virtual machine images.
A recent bug in security/sudo causes segmentation faults when
the system is not configured with a hostname, which causes issues
with some virtual machine setups, notably Vagrant.  Set the default
hostname to the output of 'uname -o'.

Submitted by:	Nicholas Fiorentini
Sponsored by:	The FreeBSD Foundation
2017-10-30 13:54:54 +00:00
Eitan Adler
a2aef24aa3 Update several more URLs
- Primarily http -> https
- Primarily FreeBSD project URLs
2017-10-29 08:17:03 +00:00
Bryan Drewery
30385347cf Rework r254951 to not reach into private OBJDIR areas.
The original change was dealing with the build wanting to run a newer
install(1) that was not yet installed.  The solution to look into the private
legacy directory of the existing build conflicts with 2 upcoming features: a
changed OBJDIR format, and splitting the host tools into arch-dependent and
arch-independent directories.  Rather than hardcoding and changing the paths in
this script, just let kernel-toolchain do the work, while disabling much of the
meat.  With -j15 this finishes in 25 seconds for me and 117 seconds with -j1.
All that is really needed is bootstrap-tools, but the system is not currently
written in a way that all previous dependent steps will have ran.  The previous
steps, such as _worldtmp, are being reworked and renamed and so cannot be
relied upon to be right.

Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:09 +00:00
Glen Barber
33bfd7db54 Increase the arm/armv6 and arm/armv7 images from 2.5GB to 3GB,
since the RPI2 (at least) does not fit in 2.5GB.

While here, add a missing BOARDNAME to RPI2.conf.

Sponsored by:	The FreeBSD Foundation
2017-10-13 15:16:57 +00:00
Glen Barber
977401bd3b Add arm/armv7 checks to env_check().
Sponsored by:	The FreeBSD Foundation
2017-10-12 20:31:10 +00:00
Glen Barber
62d028dcac Switch BEAGLEBONE, CUBIEBOARD, and PANDABOARD to use the GENERIC
kernel configuration.

Submitted by:		andrew
Differential Revision:	https://reviews.freebsd.org/D12484
Sponsored by:	The FreeBSD Foundation
2017-10-09 17:12:02 +00:00
Glen Barber
2116f70705 Catch up with r324340, switching relevant arm SoCs from armv6
to armv7.

Sponsored by:	The FreeBSD Foundation
2017-10-06 17:30:18 +00:00
Warner Losh
0b972ac92e Support armv7 builds for userland
Make armv7 as a new MACHINE_ARCH.

Copy all the places we do armv6 and add armv7 as basically an
alias. clang appears to generate code for armv7 by default. armv7 hard
float isn't supported by the the in-tree gcc, so it hasn't been
updated to have a new default.

Support armv7 as a new valid MACHINE_ARCH (and by extension
TARGET_ARCH).

Add armv7 to the universe build.

Differential Revision: https://reviews.freebsd.org/D12010
2017-10-05 23:01:33 +00:00
Glen Barber
b1bad39103 Remove release-related documentation from the base repository,
moved to the doc repository (see revisions r51045:r51061).

Sponsored by:	The FreeBSD Foundation
2017-10-05 20:54:31 +00:00
Glen Barber
7ea06b02c3 Fix the 'reldoc' target, following doc commit r51047.
Sponsored by:	The FreeBSD Foundation
2017-10-05 19:00:22 +00:00
Glen Barber
54a13912f6 Bump armv6 SoC images to 2.25GB. RPI-B is full, so increase all
image sizes for consistency.

Submitted by:	manu
Sponsored by:	The FreeBSD Foundation
2017-10-02 15:56:45 +00:00
Glen Barber
3f04d8c304 Revert r323812 from release/tools/arm.subr, which has broken the
build on arm/armv6 images.

Pointyhat:	gjb (myself)
MFC after:	immediate
MFC note:	releng/10.4 has broken because of this
Sponsored by:	The FreeBSD Foundation
2017-09-22 14:34:27 +00:00
Glen Barber
c52962c68e Bootstrap etcupdate(8) and mergemaster(8) databases when creating
virtual machine images and embedded images, similar to what is
done when extracting base.txz to the target root filesystem in
an new installation.

Noticed by:	marius
Tested with:	head@r323729
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-09-20 15:49:12 +00:00
Glen Barber
36e7acd92d Increase arm{,64} SoC image sizes to prevent "filesystem full" build
failures.

Sponsored by:	The FreeBSD Foundation
2017-09-13 14:30:30 +00:00
Ed Maste
355534688c make-memstick.sh: use UFSv2
There's not much practical difference as far as install media is
concerned but newfs creates UFSv2 by default and it is sensible to use
the contemporary UFS version.

I also intend to change makefs to create UFSv2 by default (to match
newfs) so we'll want make-memstick.sh to be explicit, rather than
relying on the host tool's default.

Reviewed by:	andrew, gjb, jhibbits
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D12231
2017-09-11 14:41:57 +00:00
Ed Maste
f20be204d6 revert unintentional changes from r323436 2017-09-11 14:35:43 +00:00
Ed Maste
e9346a94d1 boot1: remove BOOT1_MAXSIZE default value
This Makefile relies on Makefile.fat providing the correct value for
BOOT1_MAXSIZE and BOOT1_OFFSET. Since BOOT1_OFFSET had no default value
here the build would already fail if Makefile.fat did not provide
correct values.

Sponsored by:	The FreeBSD Foundation
2017-09-11 14:33:04 +00:00
Ed Maste
708c2585e3 make-memstick.sh: use 'set -e' to abort if any step fails
Also remove the now-redundant error handling that was only for makefs.

This change applies arm64's r308171 to the other make-memstick.sh
versions.

Reviewed by:	gjb
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D12195
2017-09-05 12:57:45 +00:00
Ed Maste
93cf995aef mkisoimages.sh: remove obsolete x$var convention
Ancient shells had trouble with empty variables but this has not been
relevant for FreeBSD for a very long time (decades?).
2017-09-04 22:37:28 +00:00
Glen Barber
505f20a67b Increase the Amazon EC2 AMI image size from 2GB to 3GB to prevent
image build failures due to a full md(4)-backed filesystem.

Sponsored by:	The FreeBSD Foundation
2017-08-28 14:49:26 +00:00
Glen Barber
83ffbdb133 Use py-google-compute-engine instead for releasing Google Compute
Engine (GCE) images with an updated version of Google's tools.

PR:		221714
Submitted by:	helen _dot_ koike _@_ collabora_dot_com (original)
MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2017-08-22 15:34:27 +00:00
Glen Barber
53d1eddd44 Revert r322327, r322352, r322358:
Disconnect the dependency on the kernel package from the runtime
package.  There are a number of problems here:

 1) The runtime package installed into a chroot or a jail would
    include the kernel package, changing the behavior of how jails
    work now [1];

 2) As result of (1), it is possible a binary may incorrectly
    resolve kernel symbols [2]; in addition, it is possible there
    will be unexpected fallout with 32-bit jails on a 64-bit host
    kernel [2].

Noticed by:	brd [1]
Discussed with:	kib [2]
MFC after:	3 days
MFC note:	record-only to wipe from the merge tracker
Sponsored by:	The FreeBSD Foundation
2017-08-15 15:13:33 +00:00
Glen Barber
dae40fc1b4 Always expand the full path to the configuration file specified
with the '-c' flag.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-08-15 15:08:19 +00:00
Glen Barber
ebd2eeb3f8 Add SVNVERSION_CMD to bsd.own.mk, adding the capability to include
svnversion metadata to the runtime and kernel packages.

Instead of traversing src/sys, as is done by newvers.sh for uname(1),
a full tree walk is done to prevent userland and/or modifications
from not being reflected in a modified tree (M).

MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2017-08-11 19:21:40 +00:00
Glen Barber
268db560d5 Further revise r322327 and r322352 in release/packages/kernel.ucl.
Use PPID and PID to kill off the pre-install and parent pkg(8)
processes unless 'Y' or 'y' are entered at the prompt if the user
wants to proceed with upgrading the kernel and userland at the same
time.

This restores some of the logic and intent of r322327, with the
caveat of printing "child process terminated unexpectedly."

MFC after:	5 days
MFC with:	r322327, r322352
Sponsored by:	The FreeBSD Foundation
2017-08-10 13:32:04 +00:00
Glen Barber
65fbf6644d Revise part of r322327 in release/packages/kernel.ucl.
It appears I misunderstand process forking and signal handling in
how the pre-/post-install scripts are executed internally by pkg(8).
In some cases (not all), ^C when prompted to cancel the kernel
package update will stop the pre-install script from executing, but
allow pkg(8) to continue extracting the package when it is not the
intent.

In order to keep somewhat of an anti-footshooting measure in place,
print the recommendation to install the kernel package first if
ASSUME_ALWAYS_YES is false and TERM is set, then sleep for 5 seconds
to allow the user to see the message.

MFC after:	5 days
MFC with:	r322327
X-MFC-Note:	Maybe not until I am happy with this..
Sponsored by:	The FreeBSD Foundation
2017-08-10 12:30:34 +00:00
Glen Barber
16398a1141 Add a dependency on the kernel package for the runtime package.
The idea here is that, provided upstream pkg(8) maintainers accept
the proposed change, the kernel.ucl will contain a post-install
script causing pkg(8) to emit a message informing to reboot the
system after the kernel is upgraded using 'pkg upgrade', so the
new userland is installed on the running new kernel.  At present,
this functionality does not exist in pkg(8), but will help ensure
the upgrade path follows that from UPDATING.  To work around this
for now, evaluate ASSUME_ALWAYS_YES, and prompt the user if they
wish to proceed if not set to true.

Since there is a kernel dependency, and a non-GENERIC kernel may
be in use, update Makefile.inc1 to replace '%KERNCONF%' in the
runtime.ucl with the first-built kernel set either via command line
or in make.conf(5).

MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2017-08-09 19:16:54 +00:00
Glen Barber
b200285a05 Revert r321659, re-enabling ChallengeResponseAuthentication, which was
discussed a while back between cperciva@ and so@, and I forgot.

Reported by:	cperciva
Sponsored by:	The FreeBSD Foundation
2017-07-28 18:46:02 +00:00
Glen Barber
5941ae31e0 Turn off ChallengeResponseAuthentication for EC2 AMIs, one of EC2's
requirements.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-07-28 18:27:30 +00:00
Glen Barber
b16b5f8ddc Bump the image size for BANANAPI and CUBIEBOARD2 to 1.5G, as
1G is no longer large enough.

Reported by:	manu
Sponsored by:	The FreeBSD Foundation
2017-07-27 13:37:21 +00:00
Glen Barber
8f229143b1 Allow passing NOPKG= to make(1) to prevent the pkg-stage target
from getting executed.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-07-06 17:54:03 +00:00
Glen Barber
b18ebc9201 Fix the ftp-stage target for RPI3 images by loosening the
constraints on the TARGET and TARGET_ARCH variables.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-07-05 16:55:45 +00:00
Glen Barber
5567474c4a Fix Vagrant image upload after recent API changes.
- Update ATLAS_UPLOAD_URL to avoid various regular expressions
  from failing to match due to redirections.
- Use ATLAS_UPLOAD_URL throughout the script.
- Adjust several regular expression patterns.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-07-03 14:31:38 +00:00
Glen Barber
654d0a3794 Remove CHROOT_MAKEENV from the RPI3 configuration file, to avoid
assuming the build host is amd64.

MFC after:	3 days
X-MFC-With:	r320252, r320253, r320254
X-MFC-Note:	maybe
Sponsored by:	The FreeBSD Foundation
2017-06-27 14:39:00 +00:00
Glen Barber
c2a583455e Revert r319603, r319608, and r319609. Creating a hard link to the
dtb file for the cubieboard2 is no longer needed in 12-CURRENT.

Sponsored by:	The FreeBSD Foundation
2017-06-26 20:17:48 +00:00
Glen Barber
81d49eb56e Include WORLD_FLAGS in CHROOT_IMAKEFLAGS and CHROOT_DMAKEFLAGS, to
allow passing '-jN' to the installworld and distributeworld targets.

Submitted by:	bdrewery
Sponsored by:	The FreeBSD Foundation
2017-06-23 18:42:47 +00:00
Glen Barber
514fd0b8d3 Fix, for the fourth or fifth time now, looking up where boot1.efi
is located.

MFC after:	5 days
X-MFC-Note:	maybe
X-MFC-With:	r320252, r320253
Sponsored by:	The FreeBSD Foundation
2017-06-23 00:27:40 +00:00
Glen Barber
abd18eca50 Remove two configuration entries used for debugging that snuck in.
MFC after:	5 days
X-MFC-Note:	maybe
X-MFC-With:	r320252
Sponsored by:	The FreeBSD Foundation
2017-06-23 00:11:34 +00:00
Glen Barber
751c20d8a5 In release/release.sh:
- Rename chroot_arm_armv6_build_release() to chroot_arm_build_release()
  and make it hardware agnostic (such as armv6 -vs- armv7 -vs- arm64).

- Evaluate EMBEDDED_TARGET differently so release/tools/arm.subr can
  be used for arm/armv6 and arm64/aarch64.

- Update comments and copyright.

In release/tools/arm.subr:
- In arm_create_disk(), change the default alignment from 63 to 512k,
  fixing a boot issue on arm64 and EFI. [1]

- Update comments and copyright.

Add a RPI3 configuration file, pieces obtained from Crochet.

Obtained from:	Crochet [1]
MFC after:	5 days
X-MFC-Note:	maybe
Sponsored by:	The FreeBSD Foundation
2017-06-23 00:08:36 +00:00
Baptiste Daroussin
738919c039 Remove groff from base
All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.

Also remove checknr(1) and colcrt(1) which are only useful with groff.

Approved by:	(no objections on the mailing lists)
2017-06-07 23:00:34 +00:00
Glen Barber
2c4227aaa2 Ensure ${_CW} is uppercase when passing '-c' to mk-vmimage.sh,
otherwise with 'CLOUDWARE=vagrant-virtualbox', the path to the
configuration file may be incorrect.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-06-06 14:08:54 +00:00
Glen Barber
92ed3d6edd Revert r319608, the dtb path was correct the first time. Sigh.
MFC after:	3 days
MFC with:	r319603, r319608
Sponsored by:	The FreeBSD Foundation
2017-06-05 20:18:55 +00:00
Glen Barber
9d349b6fc0 Fix the source path for the cubieboard2 dtb hard link.
MFC after:	3 days
MFC with:	r319603
Sponsored by:	The FreeBSD Foundation
2017-06-05 19:41:06 +00:00
Glen Barber
dc09cfbf08 Create a hard link for sun7i-a20-cubieboard2.dtb to allow the
cubieboard2 to find the correct dtb file.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-06-05 17:05:06 +00:00
Colin Percival
0edcc5b8d5 Turn on support for the Amazon "Elastic Network Adapter" in EC2 AMIs.
X-MFC-after:	318647 + fixes for some lock ordering warnings
2017-05-25 19:02:54 +00:00
Glen Barber
f8e26d73e1 Enable DHCP and IPv6 autoconfig on non-cloud VM images.
PR:		203653
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-05-25 12:53:49 +00:00
Glen Barber
fefd043963 Allow PORTSDIR to be overridden in the pkg-stage target.
Submitted by:	Johannes Jost Meixner (original)
PR:		218393
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-05-22 14:21:45 +00:00
Glen Barber
4d8276988e Update release/scripts/atlas-upload.sh to account for API changes
made recently by Atlas Hashicorp.  The data returned from GET and
POST requests has changed, which caused a number of regex patterns
to fail to be properly identified as 'success' or 'failure', which
ended up in upload/publish failures.

Tested with:	12-CURRENT
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-05-11 16:37:29 +00:00
Brooks Davis
0607c830b3 More ATM and NATM removal
Submitted by:	ak
Reviewed by:	ngie
Differential Revision:	https://reviews.freebsd.org/D10511
2017-04-27 16:05:12 +00:00
Ed Maste
868f280a54 Add sysutils/tmux to the DVD package set
The zsh + tmux combination is commonly wanted.

Approved by:	gjb
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2017-04-24 16:37:25 +00:00
Glen Barber
f56769dd00 Trim trailing '/release/..' when setting _OBJDIR so arm64/aarch64
boot1.efifat is properly located when creating virtual machine images.

Sponsored by:	The FreeBSD Foundation
2017-04-19 21:18:06 +00:00
Glen Barber
44ca01977c - Increase the image size for RPI2 and IMX6-based boards from 1G
to 1.5G.
- Use the 'conv=sync' dd(1) option to fix writing the u-boot.imx
  file to the md(4) device for IMX6-based boards.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-04-07 16:55:13 +00:00
Enji Cooper
623724bc3d Add cxgbe(4), ixl(4), and mlx4en(4) to the hardware release notes
MFC after:	1 week
Reviewed by:	gjb
Sponsored by:	Dell EMC Isilon
2017-03-23 00:56:52 +00:00
Glen Barber
6155b39007 Add mlx5en(4) to the hardware page. [1]
Belatedly bump copyright years after several changes.

Reported by:	rgrimes [1]
Sponsored by:	The FreeBSD Foundation
2017-03-22 23:47:15 +00:00
Glen Barber
0bcf19987a Fix hardware list build after r315742.
Sponsored by:	The FreeBSD Foundation
2017-03-22 22:29:14 +00:00
Andrew Thompson
ad33b77611 ec2.conf and vmimage.subr can be used from the installation livecd after
install to prepare an AMI image. This can be used to create a ZFS AMI disk
image using a virtual machine.

Change ec2.conf to use the pkg tool from a chroot rather than trying to
bootstrap it and fail from the livecd readonly filesystem.

Reviewed by:	gjb
2017-03-09 01:26:10 +00:00
Glen Barber
696f00b67f Fix permissions on /tmp when installing the runtime package.
Reported by:	many
Patch by:	Kyle Evans
PR:		217574
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2017-03-08 18:52:40 +00:00
Glen Barber
a744f4edcd Increase the EC2 image size for 12-CURRENT. The recent snapshot
builds of EC2 images for 12-CURRENT failed due to a full filesystem
on the md(4) device during creation.

Sponsored by:	The FreeBSD Foundation
2017-03-02 17:31:59 +00:00
Warner Losh
43b96a4eb5 Revert prior commit to restore the files mangled by my "fixing" merge
conflicts for a git rebase I tried to do.
2017-03-01 02:10:40 +00:00
Warner Losh
fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Glen Barber
f82185a08f Fix the hardware.html build.
Sponsored by:	The FreeBSD Foundation
2017-02-16 22:29:37 +00:00
Brad Davis
accb19a8bc Add some comments with examples on how to build Vagrant images.
Approved by:	re (gjb)
2017-02-01 16:15:23 +00:00
Brad Davis
ef7a48206d Add a new vagrant-box target, to make it easier and less redundant to only
produce the Vagrant box and not upload it.

Approved by:	re (gjb)
2017-01-31 21:59:10 +00:00
Brad Davis
bdc6b35a96 Force a few variables to be upper case when they are evaulated.
This allows specifying, for example: CLOUDWARE=vagrant-vmware

Approved by:	re (gjb)
2017-01-31 19:09:06 +00:00
Yoshihiro Takahashi
2b375b4edd Remove pc98 support completely.
I thank all developers and contributors for pc98.

Relnotes:	yes
2017-01-28 02:22:15 +00:00
Colin Percival
9763fbf7ef Enable IPv6 networking on Amazon EC2.
MFC after:	1 week
2017-01-15 09:06:45 +00:00