extra bits from an "xtra-bits-dir". This feature is unusable
from release/Makefile. Add an XTRADIR setting to use it.
Differential Revision: https://reviews.freebsd.org/D3633
Reviewed by: kmacy
MFC after: 3 weeks
X-MFC-to: stable/10
Relnotes: yes
builds, and prepend it to SNAPSHOT_DATE to prevent a trailing '-'
in the final box name for a release build.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
issues on some EC2 instance types. Users may want to experiment with
removing this from loader.conf and measuring the performance impact on
the EC2 instances they are using.
- pkg(8) cannot be removed before subsequent reinvocations
- The PKG_CACHEDIR cannot be cleaned after the repo*.sqlite
has been removed
- pkg(8) cannot be removed as a precursor to any of the other
steps involved here
MFC after: 3 days
X-MFC-With: r285722
X-MFC-Before: 10.2-{BETA3,RC1} (whichever happens next)
Sponsored by: The FreeBSD Foundation
The true cause of the missing UFS/MSDOSFS labels has been
identified, and only affects stable/10 at the moment.
An request for commit to stable/10 will be pending RE approval
after this commit.
MFC after: 1 day
X-MFC-Note: never
X-MFC-Never: r285018, r285019, r285076, r285078, r285082
Sponsored by: The FreeBSD Foundation
the symlink from loader.rc.sample.
Fix paths relative to the CHROOTDIR.
MFC after: 3 days
X-MFC-With: r285076, r285078
X-MFC-Before: 10.2-BETA1
Sponsored by: The FreeBSD Foundation
UFS/MSDOSFS label issues on FreeBSD/arm builds, however
the real problem was addressed in r285076, which is due
to two separate issues, unrelated to md(4) stale device
existence.
MFC after: 3 days
X-MFC-With: r285076
X-MFC-Before: 10.2-BETA1
Sponsored by: The FreeBSD Foundation
FreeBSD/arm builds. The problem stems from the loader.rc file
not existing, as well as geom_label not being loaded at boot.
For now, add the geom_label_load entry to loader.conf, and
symlink loader.rc.sample to loader.rc, both of which allowed
my BeagleBone Black to boot fine with a UFS label reference in
fstab(5).
MFC after: 3 days
X-MFC-Before: 10.2-BETA1
Sponsored by: The FreeBSD Foundation
cannot possibly exist within the chroot(8) before the target
filesystem actually exists.
MFC after: 3 days
X-MFC-With: r285018
Sponsored by: The FreeBSD Foundation
written to disk with newfs(8) and newfs_msdosfs(8).
When iterating through snapshot builds in serial, it is possible for
a build failure to leave stale md(4) devices behind, in some cases, they
could have a UFS or MSDOS filesystem label assigned.
If the md(4) is not destroyed (or not able to be destroyed, as has
happened recently due to my own fault), the filesystem label that
already exists can interfere with a new md(4) device that is targeted to
have the same label.
This behavior, although admittedly a logic error in the wrapper build
scripts, has caused intermittent reports (in particular with the armv6
builds) of missing UFS/MSDOSFS labels, causing the image to fallback to
the mountroot prompt. This appears to only happen when the backing
md(4) device is destroyed before the calling umount(8) on the target
mount, after which the UFS/MSDOSFS label persists.
The workaround is this: If EVERYTHINGISFINE is set to non-empty value,
check for an existing ufs/rootfs and msdosfs/MSDOSBOOT filesystem label
in arm_create_disk(), and rm(1) them if they exist.
The EVERYTHINGISFINE variable is chosen because it is used in exactly
one other place - release/Makefile.mirrors - and there are big scary
warnings at the top of that file as well that it should *not* be used
under normal circumstances. This should not destroy a build machine
that also uses '/dev/ufs/rootfs' as the UFS label, and I have verified
in extensive local testing that the destroyed label is recreated when
the md(4) is unmounted/mounted, but this really should not be enabled
by anyone.
Having said all that, I absolutely *do* plan MFC this to stable/10 for
the 10.2-RELEASE cycle, as so far, I have only observed this behavior
on stable/10, but this is a temporary solution until I can unravel all
of the failure paths to properly trap them.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
It was useful when working out several kinks when
testing automated image uploading when retrying was
necessary, but now it is making things much too messy.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
not set, which expands to '11.0-CURRENT', for example.
If the branch is -CURRENT, -STABLE, or -PRERELEASE, suffix
the VAGRANT_VERSION with the snapshot date.
MFC after: 3 days
X-MFC-Needs: r284893, r284895, r284896, r284897, r284942
Tested with: head@r284961 (patched)
Sponsored by: The FreeBSD Foundation
VAGRANT_PROVIDERS variable. Right now, it defaults to only
vmware_desktop, virtualbox support is to follow at some point.
While here, fix the hashicorp URL: s/vagrant/atlas/, which
was result of a sed(1) replace (and my fault).
Sponsored by: The FreeBSD Foundation
VAGRANT_${VAR} variables extracted from VAGRANT_UPLOAD_CONF.
Set ATLAS_${VAR} to VAGRANT_${VAR} if VAGRANT_UPLOAD_CONF
is set. There is intent to intentionally have separate
variants of configuration entries, but the defaults do not
yet have any reason to be different.
Sponsored by: The FreeBSD Foundation
machine images to the Google Compute Engine platform.
By default, gcutil/gsutil requires an Oauth2 login generated
from a URL that must be opened in a browser, a verification
code copied back to the terminal from which it was invoked,
etc., etc., making it near impossible for automation.
I've hacked together an evil solution to work around this,
so unless GCE_LOGIN_SKIP is set to a non-empty value, this
Makefile will not do anything useful.
As a result of this commit, remove the gce-package.sh script
that was never, nor will ever be, used.
MFC after: 3 days
X-MFC-Note: (hopefully)
Sponsored by: The FreeBSD Foundation
AMIs and Azure VM images. This is particularly helpful for
testing to avoid name collisions, but also useful for cases
where a necessary rebuild is done before the date changes.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Remove the Azure-local vm_extra_create_disk(), since we no longer
need qemu-img to convert the final VHD image to an Azure-compatible
format.
Although the waagent utility is installed from ports, create the
symlink to /usr/sbin, pending investigation on where this is
hard-coded, so it can be reported upstream. In the meantime, this
is good enough.
MFC after: 3 days
X-MFC-Needs: r284269, r284270, r284271, r284655,
r284656, r284657, r284658, r284659
X-MFC-Note: Required for 10.2-RELEASE, marcel@ has
implicit approval for the required changes
Sponsored by: The FreeBSD Foundation
While 480M is sufficient for 10-STABLE, 11-CURRENT images at
this size fail due to insufficient space.
This commit is solely for the sake of getting updated snapshot
builds out, after which I'll analyze the resulting images to
figure out what a more sane value is, even if the image size
for 11-CURRENT needs to differ from 10-STABLE.
Sponsored by: The FreeBSD Foundation
Since the images are effectively mostly zeros at 1G,
reduce the size to allow installation on smaller SD
cards, such as 512Mb.
While here, stop writing the /boot.txt file on the
WANDBOARD, which isn't used anyway.
Discussed with: imp
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
According to the manual page, '-m' should create the user home
directory, however rigorous testing suggests it does not, and
it is unclear if this is an implementation or expectation issue.
Sponsored by: The FreeBSD Foundation
Disabling soft updates journaling appears to resolve issues
with kernel panics, and may also be generally bad to have
enabled for SD cards.
Requested by: ian
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
pw(8) to set the correct /etc directory for the user/group
files.
Provided by: ian (thanks!)
MFC after: 3 days
X-MFC-with: r283894
Sponsored by: The FreeBSD Foundation
user in the userland for the target image, but creates the
user in the build chroot.
Before this is re-enabled, I want to figure out a clean way
to do this without requiring the overhead of third-party
utilities (such as qemu).
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Reduce a number of duplicated logic.
As of this commit, this file does exactly what it is needed to do.
MFC after: 3 days
X-MFC-Note: needs all previous changes
Sponsored by: The FreeBSD Foundation
which helps control some of the arm-specific bits a bit more
cleanly (but not really 'clean').
If BOARDNAME is defined (as is in the WANDBOARD configuration
RE uses), do some magic to work with the KERNCONF and BOARDNAME
to rename the file, making it a bit more intuitive for the
consumer to determine which they need.
Yes, it is ugly, that is why there is a big warning at the top.
It is, however, still much cleaner than the now 474-line shell
script, and this Makefile produces the hierarchy needed without
much evil.
MFC after: 1 week
X-MFC-Note: needs all previous Makefile.mirror commits
Sponsored by: The FreeBSD Foundation
to sh(1).
Include xz(1)-compressed images when renaming snapshot
builds.
Use OSRELEASE in place of REVISION-BRANCH for checksum
filenames.
Sponsored by: The FreeBSD Foundation
Without this, AWS rejects subsequent image uploads of a different
architecture because the name conflicts.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
For RE purposes, we use the default (/R within the chroot), so
this helps avoid copying files multiple times and xz(1)-compressing
additional times when not needed.
Again, this Makefile is not for general consumption.
Sponsored by: The FreeBSD Foundation
a 474-line kludge of a shell script to pre-create the directory
hierarchy on ftp-master.
This is not in any way connected to the build, and there is no
intention to do so. This only intent here is to try to make
things a little bit easier for me. But I've probably just made
things worse.
Sponsored by: The FreeBSD Foundation
Of note:
- This commit adds native FreeBSD/arm release build support without
requiring out-of-tree utilities.
- Part of this merge removes the WANDBOARD-{SOLO,DUAL,QUAD} kernel
configuration files, for which the IMX6 kernel configuration file
should be used instead.
- The resulting images have a 'freebsd' user (password 'freebsd'),
to allow ssh(1) access when console access is not available (VGA
or serial). The default 'root' user password is set to 'root'.
- The /etc/ttys file for arm images now enable both ttyv0 and ttyu0
by default.
Help from: many (boot testing, feedback, etc.)
Sponsored by: The FreeBSD Foundation
since it supports all of these board variants.
While here, remove the WANDBOARD-{QUAD,SOLO,DUAL} kernel
configuration files.
Discussed with: ian
Sponsored by: The FreeBSD Foundation
error:
root@releng2:/ # mount_msdosfs /dev/md5s1 /usr/obj/usr/src/release/WANDBOARD-QUAD/fat
mount_msdosfs: /dev/md5s1: File name too long
Sponsored by: The FreeBSD Foundation
Ian informed me a few months ago that the WANDBOARD-* kernels will
eventually be combined into one that will work across all these
boards, but for now, build them individually.
Sponsored by: The FreeBSD Foundation
standard 'install' location for other architectures), then
compress the image with xz(1), and generate the CHECKSUM
files.
Sponsored by: The FreeBSD Foundation
is necessary.
In arm_install_base(), chroot(8) when installing world
and kernel. Fix paths for fstab(5) and rc.conf(5).
Sponsored by: The FreeBSD Foundation
before attempting to mount(8) devfs. Also, create the
.OBJDIR for the 'release' target, so files end up in the
correct location.
In tools/arm.subr, fix the target device when creating the
gpart partition scheme.
Sponsored by: The FreeBSD Foundation
building arm images. This is similar to tools/vmimage.subr
used for building virtual machine disk images. By default,
only arm_create_disk() and arm_install_base() contain real
functionality here, and arm_install_uboot() must be overridden
in the arm/KERNEL.conf file.
In release.sh, make create_arm_armv6_build_release() do
something now.
In arm/BEAGLEBONE.conf, set IMAGE_SIZE, PART_SCHEME, FAT_SIZE,
FAT_TYPE, and MD_ARGS, as well as make arm_install_uboot()
functional.
Parts of this were taken from disecting a previous BEAGLEBONE
image, and other parts obtained from Crochet sources.
Sponsored by: The FreeBSD Foundation
EMBEDDEDBUILD, EMBEDDED_TARGET, EMBEDDED_TARGET_ARCH,
EMBEDDEDPORTS, and KERNEL.
In release.sh, set TARGET and TARGET_ARCH to the
EMBEDDED_* variants from the configuration file.
Sponsored by: The FreeBSD Foundation
While here, move CHROOT_* and RELEASE_* variables from
env_setup() to env_check() since they may change if
a release.conf file is used.
Sponsored by: The FreeBSD Foundation
clear the workflow:
- env_setup()
- env_check()
- chroot_setup()
- extra_chroot_setup()
- chroot_build_target()
- chroot_build_release()
There should be no functional changes at this point.
Sponsored by: The FreeBSD Foundation
sysutils/u-boot-beaglebone port:
- In arm/BEAGLEBONE.conf, set EMBEDDEDPORTS to the
sysutils/u-boot-beaglebone port.
- In arm/release.sh, remove BEAGLEBONE from setting WANT_UBOOT
- In tools/arm/crochet-BEAGLEBONE.conf, override the
beaglebone_check_uboot(), and set BEAGLEBONE_UBOOT to
/tmp/external/u-boot-beaglebone, and create symlinks to the
u-boot files in /usr/local/share/u-boot-beaglebone and the
uEnv.txt file in crochet/board/Beaglebone/files.
Sponsored by: The FreeBSD Foundation
In release.sh, allow overriding buildenv_setup() before
the handoff to arm/release.sh.
Copy arm/RPI-B.conf -> arm/RPI2.conf, set UBOOT_PORT and
the correct KERNEL, and add the buildenv_setup() override
to install the sysutils/u-boot-rpi2 port/package.
Copy tools/arm/crochet-RPI-B.conf -> tools/arm/crochet-RPI2.conf,
and set the correct entries for the RaspberryPi2 board.
Thanks to: loos@
Sponsored by: The FreeBSD Foundation
to be installed. [1]
For the cw-ec2-portinstall and ec2ami targets, touch the
.TARGET file after completion to prevent duplicate invocations.
Add cw-ec2-portinstall and ec2ami to CLEANFILES.
Submitted by: cperciva[1]
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
GPT scheme. UEFI needs to know the unique partition GUID
with GPT, which changes each time. Specifically, the QEMU
EFI BIOS file has this hard-coded.[1]
Since the GPT labels are now unavailable, unconditionally
label the root filesystem as 'rootfs' with newfs(8), since
it does not hurt anything anywhere else. For the arm64 case,
'/' is mounted from /dev/ufs/rootfs; for all other VM images,
'/' is mounted from /dev/gpt/rootfs.
Unfortunately, since the /dev/gpt/swapfs label is also lost,
set NOSWAP=1 for the arm64/aarch64 images. This is temporary,
until I figure out a scalable solution to this. But, a certain
piece of softare was written "very fast", and ended up living
for 15 years. We can deal with this for a week or so.
Information from: andrew, emaste [1]
Sponsored by: The FreeBSD Foundation
building FreeBSD/arm64 VM images and memstick.img installation
medium:
r281786, r281788, r281792:
r281786:
Add support for building arm64/aarch64 virtual machine images.
r281788:
Copy amd64/make-memstick.sh to arm64/make-memstick.sh for
aarch64 memory stick images.
Although arm64 does not yet have USB support, the memstick
image should be bootable with certain virtualization tools,
such as qemu.
r281792:
Add a buildenv_setup() prototype, intended to be overridden as
needed.
For example, the arm64/aarch64 build needs devel/aarch64-binutils,
so buildenv_setup() in the release.conf for this architecture
handles the installation of the port before buildworld/buildkernel.
Sponsored by: The FreeBSD Foundation
aarch64 memory stick images.
Although arm64 does not yet have USB support, the memstick
image should be bootable with certain virtualization tools,
such as qemu.
Sponsored by: The FreeBSD Foundation
copy the userland from one md(4)-mounted filesystem to a clean
filesystem to prevent remnants of files that were added and
removed from resulting in an unclean filesystem. When newfs(8)
creates the first filesystem with journaled soft-updates enabled,
the /.sujournal file in the new filesystem cannot be overwritten
by the /.sujournal in the original filesystem.
To avoid this particular error case, do not enable journaled
soft-updates when creating the md(4)-backed filesystems, and
instead use tunefs(8) to enable journaled soft-updates after
the new filesystem is populated in vm_copy_base().
While here, fix a long standing bug where the build environment
/boot files were used by mkimg(1) when creating the VM disk
images by using the files in .OBJDIR.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
* Remove vm_umount_base function which is currently unused.
* Add umount_loop function which loops attempting to unmount one filesystem.
* Replace calls to umount with calls to umount_loop.
* Don't attempt to unmount ${DESTDIR}/dev if it isn't mounted.
The looping is necessary because sometimes umount fails due to filesystems
being busy. The most common cause of such busyness is periodic(8) jobs
running `find / ...`.
Reviewed by: gjb
Call newfs(8) and mount the md(4) device to the target
directory.
Specify DESTDIR for installworld, distribution, and
installkernel targets.
Sponsored by: The FreeBSD Foundation
a new filesystem before packaging it into a disk image. This prevents
"remnants" of deleted files from showing up in the VM images, and reduces
their compressed size (by about 10% for the cloudware images) as a result.
Looks good to: gjb
Dynamically figure out the list of targets based on tags passed on the mtrees
First sanity check that all packages have existing manifests
Generate the packages
Please note that for now the mtree needs more work as it has duplicate entries,
everything is not yet tagged
The packages now have generic entries and needs to be customize
1/ Analyse the fields from mtree lines to extract:
- owner,
- group,
- mode,
- flags,
- tags,
2/ Ignore non tagged entries
3/ For now concat tags to form a futur package name
post-install packaging cloud provider images.
Add a 'gce-package.sh' script to generate the final output
image ready for upload to the GCE platform. Right now, this
is the only image that has a specific output format (GNU-tar),
and this implementation is expected to be temporary.
This is not directly connected to the other release targets.
MFC after: 1 week
X-MFC-with: r279249, r279250
Sponsored by: The FreeBSD Foundation
disk images to a directory within DESTDIR.
Add CLOUDINSTALL variable to automatically generate the
actual list of install targets. While here, rename the
cloud-specific names from 'vm-' to 'cw-' to avoid naming
collisions.
Add AZURE_DISK, GCE_DISK, and OPENSTACK_DISK output file
variables.
Add 'cloudware-install' to the 'install' target.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Separately document multi-threaded support to keep the
version update separate, so it remains correct if xz(1)
is updated again.
Sponsored by: The FreeBSD Foundation
libarchive(3) doesn't support the new liblzma API yet, but this change
allows us to enable multi-threaded xz compression.
``make release'' should now finish in half the time on a machine with
several cores and fast disks (our typical build server).
This behaviour only applies when building a release and it doesn't
affect buildworld/installworld. To disable threaded xz compression,
set XZ_THREADS=1.
Reviewed by: gjb
Tested by: gjb
update paths; and include everything in the "base" distribution.
The "games" distribution being optional made sense when there were more
games and we had small disks; but the "games-like" games were moved into
the ports tree a dozen years ago and the remaining "utility-like" games
occupy less than 0.001% of my laptop's small hard drive. Meanwhile every
new user is confronted by the question "do you want games installed" when
they they try to install FreeBSD.
The next steps will be:
2. Removing punch card (bcd, ppt), phase-of-moon (pom), clock (grdc), and
caesar cipher (caesar, rot13) utilities. I intend to keep fortune, factor,
morse, number, primes, and random, since there is evidence that those are
still being used.
3. Merging src/games into src/usr.bin.
This change will not be MFCed.
Reviewed by: jmg
Discussed at: EuroBSDCon
Approved by: gjb (release-affecting changes)
Remove a BOARDS entry left in while testing.
Add build-arm-ports${BOARD} target as a prerequisite when
ARMPORTS is set.
Add build-arm-KERNCONF and build-arm-ports to CLEANFILES.
Sponsored by: The FreeBSD Foundation
defaulting VMCONFIG to /dev/null, and additionally
ensuring VMCONFIG is not a character device before it
is sourced.
While here, be sure to exit if usage() is called.
This should effectively be no-op, but the usage() output
was discovered while investigating a larger issue.
MFC after: 1 week
X-MFC-with: r277458, r277536, r277606, r277609,
r277836, r278118, r278119, r278206
Sponsored by: The FreeBSD Foundation
I was going to use __FreeBSD_version to determine if
xz(1) should be multi-threaded by default, but doing
this will cause problems if/when the changes are merged
from head.
Sponsored by: The FreeBSD Foundation
Allow multi-threaded xz(1) to be turned off by specifying
NO_XZTHREADS, and allow number of threads to be overridden
by specifying XZ_THREADS=N.
MFC after: 1 week
X-MFC-needs: r278433
Sponsored by: The FreeBSD Foundation
pages in the release notes. Rather than duplicating
the information between various files, add two new
files to include in all pages that currently display
the information.
Sponsored by: The FreeBSD Foundation
VM_RC_LIST and deduplicate.
Evaluate if firstboot_freebsd_update should be enabled
based on UNAME_r, because it is not supported for the
-CURRENT or -STABLE branches.
MFC after: 2 weeks
X-MFC-with: r277458, r277536, r277606, r277609,
r277836
Sponsored by: The FreeBSD Foundation
With the previous layout, the majority of the release
notes were categorized under one main "What's New"
section, which in addition to making the page difficult
to parse relevant bits, also had a few rendering issues,
such as 'Note' blocks visually not appearing as if they
pertain to a particular change.
This change removes the "What's New" section, integrating
it with the introduction. In addition, each section now
provides a brief description of its contents, broken down
further into a subsection for each "topic", making it much
easier to find a specific topic in the page.
MFC after: 3 days
X-MFC-Note: direct commit, not a merge
Sponsored by: The FreeBSD Foundation
