Commit Graph

71 Commits

Author SHA1 Message Date
Glen Barber
41ea4cc219 Set FORCE_PKG_REGISTER=1 when installing packages to avoid failures
when re-using build chroot(8) environments.

This is based on the patch in the PR referenced below, but instead
of using 'reinstall' in two locations (one of which already uses
FORCE_PKG_REGISTER=1), changes the non-embedded behavior.

PR:		205998
Submitted by:	ngie
MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2016-01-09 00:45:38 +00:00
Glen Barber
34187a086a Deprecate MD5 checksum generation in favor of SHA512.
This was discussed during the 10.2-RELEASE cycle, however
since we were nearing the end of the cycle, we decided to
defer this change until after 10.2-RELEASE.

Reminded by:	so (delphij), jmg
MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2015-10-14 22:33:11 +00:00
Glen Barber
238397876d Append VMFORMATS and VMSIZE to RELEASE_RMAKEFLAGS only if
WITH_VMIMAGES is set.

MFC after:	3 days
X-MFC-With:	r288341, r288345
Sponsored by:	The FreeBSD Foundation
2015-09-28 19:19:53 +00:00
Glen Barber
a312aae568 In followup to r288341, ensure VMFORMATS and VMSIZE are not
set to empty values, which would result in nonintuitive build
errors.

MFC after:	3 days
X-MFC-With:	r288341
PR:		203420 (related to)
Sponsored by:	The FreeBSD Foundation
2015-09-28 18:39:21 +00:00
Glen Barber
3a8cff7804 Honor VMFORMATS and VMSIZE if set in release.conf.
PR:		203420
Submitted by:	luca <dot> pizzamiglio <at> gmail <dot> com
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-09-28 16:33:38 +00:00
Glen Barber
88feae84be Merge ^/projects/release-arm-redux into ^/head.
Of note:

- This commit adds native FreeBSD/arm release build support without
  requiring out-of-tree utilities.

- Part of this merge removes the WANDBOARD-{SOLO,DUAL,QUAD} kernel
  configuration files, for which the IMX6 kernel configuration file
  should be used instead.

- The resulting images have a 'freebsd' user (password 'freebsd'),
  to allow ssh(1) access when console access is not available (VGA
  or serial).  The default 'root' user password is set to 'root'.

- The /etc/ttys file for arm images now enable both ttyv0 and ttyu0
  by default.

Help from:	many (boot testing, feedback, etc.)
Sponsored by:	The FreeBSD Foundation
2015-05-09 21:08:12 +00:00
Glen Barber
9fcdd5a3c9 Sigh. Really fix CHECKSUM file creation.
Sponsored by:	The FreeBSD Foundation
2015-05-08 15:15:36 +00:00
Glen Barber
395475f31e Fix CHECKSUM file creation.
Sponsored by:	The FreeBSD Foundation
2015-05-08 15:10:00 +00:00
Glen Barber
4bf1e8aee0 Do not include '/R' in the path when generating checksums, since
that will cause '/R' to be included in the CHECKSUM files.

Sponsored by:	The FreeBSD Foundation
2015-05-08 12:55:25 +00:00
Glen Barber
99cf2362e2 Move the finished image build to the '/R' directory (the
standard 'install' location for other architectures), then
compress the image with xz(1), and generate the CHECKSUM
files.

Sponsored by:	The FreeBSD Foundation
2015-05-08 12:50:48 +00:00
Glen Barber
40847e360b Fix the name of the final image.
Sigh.  Thou shalt not commit before coffee.

Sponsored by:	The FreeBSD Foundation
2015-05-08 10:47:57 +00:00
Glen Barber
b3e7a59bdf Rename KERNEL.img -> OSRELEASE-KERNEL.img when complete.
Sponsored by:	The FreeBSD Foundation
2015-05-08 10:43:29 +00:00
Glen Barber
ac1401b2c3 Fix paths.
Remove the temporary directory when done with it.
Destroy the md(4) device when done with it.

Sponsored by:	The FreeBSD Foundation
2015-05-07 17:40:00 +00:00
Glen Barber
befaadd56c In release.sh, create the /dev directory within the chroot
before attempting to mount(8) devfs.  Also, create the
.OBJDIR for the 'release' target, so files end up in the
correct location.

In tools/arm.subr, fix the target device when creating the
gpart partition scheme.

Sponsored by:	The FreeBSD Foundation
2015-05-07 00:27:28 +00:00
Glen Barber
68bca5fb30 Use the correct variable for the tools/TARGET.conf file.
Sponsored by:	The FreeBSD Foundation
2015-05-06 23:26:51 +00:00
Glen Barber
4ad8b75d53 Add tools/arm.subr to contain common subroutines used for
building arm images.  This is similar to tools/vmimage.subr
used for building virtual machine disk images.  By default,
only arm_create_disk() and arm_install_base() contain real
functionality here, and arm_install_uboot() must be overridden
in the arm/KERNEL.conf file.

In release.sh, make create_arm_armv6_build_release() do
something now.

In arm/BEAGLEBONE.conf, set IMAGE_SIZE, PART_SCHEME, FAT_SIZE,
FAT_TYPE, and MD_ARGS, as well as make arm_install_uboot()
functional.

Parts of this were taken from disecting a previous BEAGLEBONE
image, and other parts obtained from Crochet sources.

Sponsored by:	The FreeBSD Foundation
2015-05-06 19:58:12 +00:00
Glen Barber
9056dae945 Evaluage chroot_build_release_cmd based on the TARGET/TARGET_ARCH
of the build target.

Move arm/armv6 'release' bits to chroot_arm_armv6_build_release().

Sponsored by:	The FreeBSD Foundation
2015-05-06 18:16:27 +00:00
Glen Barber
57dbde2ca8 buildenv_setup(), if overridden in the configuration file,
only needs to be invoked once.

Sponsored by:	The FreeBSD Foundation
2015-05-06 16:21:12 +00:00
Glen Barber
6848c01fad Remove Crochet invocation.
Sponsored by:	The FreeBSD Foundation
2015-05-06 16:04:11 +00:00
Glen Barber
cf582ac56f Update the arm/*.conf configuration files to only set
EMBEDDEDBUILD, EMBEDDED_TARGET, EMBEDDED_TARGET_ARCH,
EMBEDDEDPORTS, and KERNEL.

In release.sh, set TARGET and TARGET_ARCH to the
EMBEDDED_* variants from the configuration file.

Sponsored by:	The FreeBSD Foundation
2015-05-06 15:58:21 +00:00
Glen Barber
a03128832c In extra_chroot_setup(), use 'uname -U' to determine OSVERSION.
Move the port build for EMBEDDEDPORTS (if set) from arm/release.sh
to release.sh.

Remove arm/release.sh.

Sponsored by:	The FreeBSD Foundation
2015-05-06 15:41:43 +00:00
Glen Barber
8501ddab55 Add a return to chroot_build_target().
In chroot_build_release(), reverse the order evaluating
EMBEDDEDBUILD.

Sponsored by:	The FreeBSD Foundation
2015-05-06 15:20:29 +00:00
Glen Barber
7290c83913 Wrap long lines in comments after previous commits that indented
the file.

Sponsored by:	The FreeBSD Foundation
2015-05-06 15:10:50 +00:00
Glen Barber
3c9143a888 Ensure resolv.conf exists on the build host before attempting
to copy it to the build chroot.

Move where the ldconfig(8) startup script is invoked.

Sponsored by:	The FreeBSD Foundation
2015-05-06 15:05:38 +00:00
Glen Barber
81cfc2845b Move where the configuration file is sourced if '-c'
is specified.  If the configuration file does not exist,
proceed with the default options instead of exiting.

Sponsored by:	The FreeBSD Foundation
2015-05-06 14:59:16 +00:00
Glen Barber
43eee0751f Move the trap after env_check(), since CHROOTDIR may be
redefined.

Sponsored by:	The FreeBSD Foundation
2015-05-06 14:53:34 +00:00
Glen Barber
3ddb84b3bb Shorten long lines.
Sponsored by:	The FreeBSD Foundation
2015-05-06 14:53:01 +00:00
Glen Barber
5dbf38dcca Move PATH and export to the same line.
Handle svn or svnlite being installed in /usr/bin or
/usr/local/bin.

Sponsored by:	The FreeBSD Foundation
2015-05-06 14:48:35 +00:00
Glen Barber
a12fa24967 Add documentation for newly-added functions.
While here, move CHROOT_* and RELEASE_* variables from
env_setup() to env_check() since they may change if
a release.conf file is used.

Sponsored by:	The FreeBSD Foundation
2015-05-06 14:35:43 +00:00
Glen Barber
fb539361f7 Add the following functions to release.sh to organize and make
clear the workflow:
 - env_setup()
 - env_check()
 - chroot_setup()
 - extra_chroot_setup()
 - chroot_build_target()
 - chroot_build_release()

There should be no functional changes at this point.

Sponsored by:	The FreeBSD Foundation
2015-05-06 14:21:51 +00:00
Glen Barber
2252f43bdc MFH: r282315-r282534
Sponsored by:	The FreeBSD Foundation
2015-05-06 11:50:30 +00:00
Glen Barber
20a67b0520 Add initial support for building RPI2 images.
In release.sh, allow overriding buildenv_setup() before
the handoff to arm/release.sh.

Copy arm/RPI-B.conf -> arm/RPI2.conf, set UBOOT_PORT and
the correct KERNEL, and add the buildenv_setup() override
to install the sysutils/u-boot-rpi2 port/package.

Copy tools/arm/crochet-RPI-B.conf -> tools/arm/crochet-RPI2.conf,
and set the correct entries for the RaspberryPi2 board.

Thanks to:	loos@
Sponsored by:	The FreeBSD Foundation
2015-05-05 16:28:04 +00:00
Glen Barber
7263c8c099 MFH: r280643-r281852
Sponsored by:	The FreeBSD Foundation
2015-04-22 12:58:16 +00:00
Glen Barber
6b1cf1b2c2 Merge the following from ^/projects/release-arm64 to allow
building FreeBSD/arm64 VM images and memstick.img installation
medium:
r281786, r281788, r281792:

 r281786:
  Add support for building arm64/aarch64 virtual machine images.

 r281788:
  Copy amd64/make-memstick.sh to arm64/make-memstick.sh for
  aarch64 memory stick images.

  Although arm64 does not yet have USB support, the memstick
  image should be bootable with certain virtualization tools,
  such as qemu.

 r281792:

  Add a buildenv_setup() prototype, intended to be overridden as
  needed.

  For example, the arm64/aarch64 build needs devel/aarch64-binutils,
  so buildenv_setup() in the release.conf for this architecture
  handles the installation of the port before buildworld/buildkernel.

Sponsored by:	The FreeBSD Foundation
2015-04-21 00:48:35 +00:00
Glen Barber
e6e746bfb0 MFH: r278968-r280640
Sponsored by:	The FreeBSD Foundation
2015-03-25 22:35:08 +00:00
Glen Barber
1db297199a Default XZ_THREADS=0, and add to RELEASE_RMAKEFLAGS
so it is passed to the 'release' make(1) target if
it needs to be adjusted to a non-zero integer.

Sponsored by:	The FreeBSD Foundation
2015-02-20 18:51:52 +00:00
Glen Barber
53f2fbcab4 MFH: r278202,r278205-r278590
Sponsored by:	The FreeBSD Foundation
2015-02-11 18:59:46 +00:00
Glen Barber
04c920b9f0 Bump copyright after r277458.
MFC after:	2 weeks
X-MFC-with:	r277458, r277536, r277606, r277609,
		r277836, r278118, r278119
Sponsored by:	The FreeBSD Foundation
2015-02-04 11:55:32 +00:00
Glen Barber
ccb7bf2623 Set a VERSION value.
Sponsored by:	The FreeBSD Foundation
2015-02-04 11:45:15 +00:00
Glen Barber
21f93ded5a Add examples for WITH_CLOUDWARE to release.conf.sample.
Add WITH_CLOUDWARE evaluation to RELEASE_RMAKEFLAGS.

Sponsored by:	The FreeBSD Foundation
2014-11-04 00:02:23 +00:00
Glen Barber
a9278c16be Add WITH_COMPRESSED_VMIMAGES variable, which when set enables
xz(1) compression of the virtual machine images.

This is intentionally separate to allow more fine-grained
tuning over which images are compressed, especially in cases
where compressing 20GB sparse images can take hours.

Sponsored by:	The FreeBSD Foundation
2014-10-02 00:34:03 +00:00
Glen Barber
4e187d25fa release/Makefile:
Connect the virtual machine image build to the release
  target if WITH_VMIMAGES is set to a non-empty value.

release/release.sh:
  Add WITH_VMIMAGES to RELEASE_RMAKEFLAGS.

release/release.conf.sample:
  Add commented entries for tuning the release build if the
  WITH_VMIMAGES make(1) environment variable is set to
  a non-empty value.

Sponsored by:	The FreeBSD Foundation
2014-09-28 17:53:10 +00:00
Glen Barber
9c6096d3a7 Wrap a long line.
MFC after:	3 days
X-MFC-with:	r270417, r270418, r270455
Sponsored by:	The FreeBSD Foundation
2014-08-24 12:51:34 +00:00
Glen Barber
f07e1c32aa Set OSREL and UNAME_r in release/release.sh when building
ports to prevent ports build failures from killing the
release build.

MFC after:	3 days
X-MFC-with:	r270417, r270418
Sponsored by:	The FreeBSD Foundation
2014-08-24 12:50:50 +00:00
Glen Barber
95d901dba9 Merge the following from ^/projects/release-noxdev:
r269549, r269551, r269552, r269553, r269554, r269555, r269558, r269559,
r269560, r269561, r269628, r269629, r269630, r269635, r269637:

r269549:
  Create a new project branch, release-noxdev, for
  a sandbox workspace outside of head/ to update the
  release bits for arm builds since the deprecation of
  the XDEV and XDEV_ARCH make(1) variables.

r269551:
  Define load_chroot_env() and load_target_env()
  prototypes.

r269552:
  Call load_chroot_env() and load_target_env() where
  they can be mutually conflicting with regard to
  TARGET, TARGET_ARCH, XDEV, and XDEV_ARCH.

r269553:
  Add shebang line to arm/*.conf files since these
  should be considered to be executable (albeit not
  on their own) shell scripts.

r269554:
  Redefine load_chroot_env() and load_target_env() in
  the arm/*.conf files, and reindent.

r269555:
  Simplify where load_chroot_env() and load_target_env() are
  called.

r269558:
  Provide example in release.conf.sample for overriding the
  load_chroot_env() and load_target_env() prototypes.

r269559:
  Remove a gratuitous newline.

r269560:
  Unset potentially conflicting variables in load_chroot_env()
  and load_target_env().

r269561:
  Make global variables global, and accessible outside of
  the functions within which they were once defined.

r269628:
  Remove XDEV/XDEV_ARCH evaluation if EMBEDDEDBUILD is set.

r269629:
  In arm/release.sh, switch TARGET/TARGET_ARCH back to the
  original XDEV/XDEV_ARCH make(1) variables.

  In theory, this should have been a no-op, but the TARGET and
  TARGET_ARCH are now unset in load_target_env() to avoid
  collision with the chroot userland.

r269630:
  Export variables in the arm/*.conf files because they
  need to be passed through release.sh to arm/release.sh.

  Set MK_TESTS=no for the xdev target.

r269635:
  As part of the XDEV/XDEV_ARCH deprecation, the
  'xdev-links' target was intentionally no longer
  invoked automatically.

  Invoke the xdev-links target after xdev, which
  creates, for example, /usr/bin/armv6-freebsd-cc
  symlink to /usr/armv6-freebsd/usr/bin/cc.

r269637:
  Set TARGET and TARGET_ARCH to the XDEV and XDEV_ARCH
  counterparts for the xdev and xdev-links make(1)
  targets.

Sponsored by:	The FreeBSD Foundation
2014-08-06 19:04:05 +00:00
Glen Barber
3e64b4c52d Add SRC_UPDATE_SKIP, DOC_UPDATE_SKIP, and PORTS_UPDATE_SKIP
variables.  These are intended to allow bypassing the
'svn co /usr/{src,doc,ports}' step in the chroot when the
tree exists from external means.

The use case here is that /usr/src, /usr/doc, and /usr/ports
in the chroot exist as result of zfs dataset clones, so it
is possible (and happens quite often) that the included
distributions may not be consistent.  (This is not the case
for -RELEASE builds, but does happen for snapshot builds.)

Tested on:	stable/9@r264319
MFC After:	3 days
Sponsored by:	The FreeBSD Foundation
2014-04-11 13:48:45 +00:00
Glen Barber
cbc0e08a7b Add a new release build variable, WITH_COMPRESSED_IMAGES.
When set to a non-empty value, the installation medium is
compressed with gzip(1) as part of the 'install' target in
the release/ directory.

With gzip(1) compression, downloadable image are reduced in
size quite significantly.  Build test against head@263927
shows the following:

 bootonly.iso:		64% smaller
 disc1.iso:		44% smaller
 memstick.img:		47% smaller
 mini-memstick.img:	65% smaller
 dvd1.iso:		untested

This option is off by default, I would eventually like to
turn it on by default, and remove the '-k' flag to gzip(1)
so only compressed images are published on FTP.

Requested by:	wkoszek
MFC After:	1 week
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:41:26 +00:00
Glen Barber
9362e2bc57 Add a hack-ish attempt to provide backwards-compatibility with
release.conf SVNROOT entries that do not have a trailing '/'.

In order to provide a mechanism to use !svn, the trailing '/'
was added to SVNROOT so there were no assumptions being made
about '/' being in the URL between SVNROOT and SRCBRANCH.

Sponsored by:	The FreeBSD Foundation
2014-03-06 18:48:02 +00:00
Glen Barber
e0436612a9 Merge ^/projects/release-embedded into ^/head.
After several months of testing and fixing (and breaking)
various parts of release/release.sh changes, it is now
possible to build FreeBSD/arm images as part of the release
process.

When EMBEDDEDBUILD is set in the release.conf file, release.sh
will create the build environment, then run a separate script
in release/${XDEV}/release.sh [1].  Currently, only arm is
supported.

The release/${XDEV}/release.sh configures the build environment
specific for the target image, such as installing gcc(1),
installing additional third-party software from the ports tree,
and fetching external sources.

Once the build environment is set up, release/${XDEV}/release.sh
runs Crochet, written by Tim Kientzle, which builds the userland
and kernel, and creates an image that can be written to an SD
card with dd(1).  Many thanks to Tim for his work on Crochet.

Sample configurations for FreeBSD/arm boards are in the
release/arm/ directory, and Crochet configuration files for each
board are located in release/tools/arm/.  Supported boards at this
time are: BEAGLEBONE, PANDABOARD, RPI-B, and WANDBOARD-QUAD.

Adding support for additional boards will continue in the
projects/release-embedded/ branch, and incrementally merged back
to head/.

Many thanks to the FreeBSD Foundation for the support and
sponsorship of this project.

[1] XDEV is used in order to keep the various configurations
    organized by architecture, but since TARGET and TARGET_ARCH
    are used to build the chroot, the values of those variables
    cannot be used.

MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-03-05 23:17:53 +00:00
Glen Barber
2e314957a9 Rework how WORLD_FLAGS and KERNEL_FLAGS are set, to remove an
unnecessary expr(1) call.

Inspired by:	hrs (a few months ago)
MFC after:	1 week
X-MFC-With:	r262499, r262505
Sponsored by:	The FreeBSD Foundation
2014-02-26 01:06:41 +00:00