Commit Graph

31 Commits

Author SHA1 Message Date
gjb
47dc31278a Add examples for WITH_CLOUDWARE to release.conf.sample.
Add WITH_CLOUDWARE evaluation to RELEASE_RMAKEFLAGS.

Sponsored by:	The FreeBSD Foundation
2014-11-04 00:02:23 +00:00
gjb
470a786adc Add WITH_COMPRESSED_VMIMAGES variable, which when set enables
xz(1) compression of the virtual machine images.

This is intentionally separate to allow more fine-grained
tuning over which images are compressed, especially in cases
where compressing 20GB sparse images can take hours.

Sponsored by:	The FreeBSD Foundation
2014-10-02 00:34:03 +00:00
gjb
2aaf766ec5 release/Makefile:
Connect the virtual machine image build to the release
  target if WITH_VMIMAGES is set to a non-empty value.

release/release.sh:
  Add WITH_VMIMAGES to RELEASE_RMAKEFLAGS.

release/release.conf.sample:
  Add commented entries for tuning the release build if the
  WITH_VMIMAGES make(1) environment variable is set to
  a non-empty value.

Sponsored by:	The FreeBSD Foundation
2014-09-28 17:53:10 +00:00
gjb
75bf853405 Wrap a long line.
MFC after:	3 days
X-MFC-with:	r270417, r270418, r270455
Sponsored by:	The FreeBSD Foundation
2014-08-24 12:51:34 +00:00
gjb
6cc6bc54dc Set OSREL and UNAME_r in release/release.sh when building
ports to prevent ports build failures from killing the
release build.

MFC after:	3 days
X-MFC-with:	r270417, r270418
Sponsored by:	The FreeBSD Foundation
2014-08-24 12:50:50 +00:00
gjb
0cf80d1f4d Merge the following from ^/projects/release-noxdev:
r269549, r269551, r269552, r269553, r269554, r269555, r269558, r269559,
r269560, r269561, r269628, r269629, r269630, r269635, r269637:

r269549:
  Create a new project branch, release-noxdev, for
  a sandbox workspace outside of head/ to update the
  release bits for arm builds since the deprecation of
  the XDEV and XDEV_ARCH make(1) variables.

r269551:
  Define load_chroot_env() and load_target_env()
  prototypes.

r269552:
  Call load_chroot_env() and load_target_env() where
  they can be mutually conflicting with regard to
  TARGET, TARGET_ARCH, XDEV, and XDEV_ARCH.

r269553:
  Add shebang line to arm/*.conf files since these
  should be considered to be executable (albeit not
  on their own) shell scripts.

r269554:
  Redefine load_chroot_env() and load_target_env() in
  the arm/*.conf files, and reindent.

r269555:
  Simplify where load_chroot_env() and load_target_env() are
  called.

r269558:
  Provide example in release.conf.sample for overriding the
  load_chroot_env() and load_target_env() prototypes.

r269559:
  Remove a gratuitous newline.

r269560:
  Unset potentially conflicting variables in load_chroot_env()
  and load_target_env().

r269561:
  Make global variables global, and accessible outside of
  the functions within which they were once defined.

r269628:
  Remove XDEV/XDEV_ARCH evaluation if EMBEDDEDBUILD is set.

r269629:
  In arm/release.sh, switch TARGET/TARGET_ARCH back to the
  original XDEV/XDEV_ARCH make(1) variables.

  In theory, this should have been a no-op, but the TARGET and
  TARGET_ARCH are now unset in load_target_env() to avoid
  collision with the chroot userland.

r269630:
  Export variables in the arm/*.conf files because they
  need to be passed through release.sh to arm/release.sh.

  Set MK_TESTS=no for the xdev target.

r269635:
  As part of the XDEV/XDEV_ARCH deprecation, the
  'xdev-links' target was intentionally no longer
  invoked automatically.

  Invoke the xdev-links target after xdev, which
  creates, for example, /usr/bin/armv6-freebsd-cc
  symlink to /usr/armv6-freebsd/usr/bin/cc.

r269637:
  Set TARGET and TARGET_ARCH to the XDEV and XDEV_ARCH
  counterparts for the xdev and xdev-links make(1)
  targets.

Sponsored by:	The FreeBSD Foundation
2014-08-06 19:04:05 +00:00
gjb
8606a0a992 Add SRC_UPDATE_SKIP, DOC_UPDATE_SKIP, and PORTS_UPDATE_SKIP
variables.  These are intended to allow bypassing the
'svn co /usr/{src,doc,ports}' step in the chroot when the
tree exists from external means.

The use case here is that /usr/src, /usr/doc, and /usr/ports
in the chroot exist as result of zfs dataset clones, so it
is possible (and happens quite often) that the included
distributions may not be consistent.  (This is not the case
for -RELEASE builds, but does happen for snapshot builds.)

Tested on:	stable/9@r264319
MFC After:	3 days
Sponsored by:	The FreeBSD Foundation
2014-04-11 13:48:45 +00:00
gjb
0fbab8558b Add a new release build variable, WITH_COMPRESSED_IMAGES.
When set to a non-empty value, the installation medium is
compressed with gzip(1) as part of the 'install' target in
the release/ directory.

With gzip(1) compression, downloadable image are reduced in
size quite significantly.  Build test against head@263927
shows the following:

 bootonly.iso:		64% smaller
 disc1.iso:		44% smaller
 memstick.img:		47% smaller
 mini-memstick.img:	65% smaller
 dvd1.iso:		untested

This option is off by default, I would eventually like to
turn it on by default, and remove the '-k' flag to gzip(1)
so only compressed images are published on FTP.

Requested by:	wkoszek
MFC After:	1 week
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:41:26 +00:00
gjb
4f9a1ea514 Add a hack-ish attempt to provide backwards-compatibility with
release.conf SVNROOT entries that do not have a trailing '/'.

In order to provide a mechanism to use !svn, the trailing '/'
was added to SVNROOT so there were no assumptions being made
about '/' being in the URL between SVNROOT and SRCBRANCH.

Sponsored by:	The FreeBSD Foundation
2014-03-06 18:48:02 +00:00
gjb
f40d4b8f24 Merge ^/projects/release-embedded into ^/head.
After several months of testing and fixing (and breaking)
various parts of release/release.sh changes, it is now
possible to build FreeBSD/arm images as part of the release
process.

When EMBEDDEDBUILD is set in the release.conf file, release.sh
will create the build environment, then run a separate script
in release/${XDEV}/release.sh [1].  Currently, only arm is
supported.

The release/${XDEV}/release.sh configures the build environment
specific for the target image, such as installing gcc(1),
installing additional third-party software from the ports tree,
and fetching external sources.

Once the build environment is set up, release/${XDEV}/release.sh
runs Crochet, written by Tim Kientzle, which builds the userland
and kernel, and creates an image that can be written to an SD
card with dd(1).  Many thanks to Tim for his work on Crochet.

Sample configurations for FreeBSD/arm boards are in the
release/arm/ directory, and Crochet configuration files for each
board are located in release/tools/arm/.  Supported boards at this
time are: BEAGLEBONE, PANDABOARD, RPI-B, and WANDBOARD-QUAD.

Adding support for additional boards will continue in the
projects/release-embedded/ branch, and incrementally merged back
to head/.

Many thanks to the FreeBSD Foundation for the support and
sponsorship of this project.

[1] XDEV is used in order to keep the various configurations
    organized by architecture, but since TARGET and TARGET_ARCH
    are used to build the chroot, the values of those variables
    cannot be used.

MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-03-05 23:17:53 +00:00
gjb
46aa6a2ffd Rework how WORLD_FLAGS and KERNEL_FLAGS are set, to remove an
unnecessary expr(1) call.

Inspired by:	hrs (a few months ago)
MFC after:	1 week
X-MFC-With:	r262499, r262505
Sponsored by:	The FreeBSD Foundation
2014-02-26 01:06:41 +00:00
gjb
e70125acff release.sh:
- Add a VCSCMD variable that defaults to 'svn checkout',
  and update places 'svn co' is used directly.
- After sourcing a configuration file, prefix SRCBRANCH,
  PORTBRANCH, and DOCBRANCH with the SVNROOT.
- Properly capitalize 'FreeBSD.org' in the default SVNROOT.
- Update Copyright.

release.conf.sample:
- Add an example to use git instead of svn, by nullifying
  SVNROOT, and setting SRCBRANCH, DOCBRANCH, and PORTBRANCH
  to the URL fo a git repository.

release.7:
- Document VCSCMD.

Submitted by:	Rick Miller (based on)
Sponsored by:	The FreeBSD Foundation
2014-02-25 22:13:48 +00:00
gjb
eb1a5f8de9 Move ^/user/gjb/hacking/release-embedded up one directory, and remove
^/user/gjb/hacking since this is likely to be merged to head/ soon.

Sponsored by:	The FreeBSD Foundation
2014-02-21 03:35:43 +00:00
gjb
df1e5281df Move build_doc_ports() to the if...fi block from which it is called.
Sponsored by:	The FreeBSD Foundation
2013-12-27 17:36:43 +00:00
gjb
6b01bbf146 Copy head@r259933 -> user/gjb/hacking/release-embedded for initial
inclusion of (at least) arm builds with the release.

Sponsored by:	The FreeBSD Foundation
2013-12-27 04:23:07 +00:00
jmmv
8e72067a86 Put the release objdir inside the chroot.
When building the system from outside the chroot as part of the release
process, ensure the objdir is within the chroot so that the whole output
of the release.sh script is self-contained in a single directory.

Use CHROOTDIR/tmp/obj instead of CHROOTDIR/usr/obj to minimize possible
interferences with the output of the build itself.

Reviewed by:	gjb
Approved by:	rpaulo (mentor)
2013-12-25 18:25:58 +00:00
jmmv
b8dac04a8f Delay copying of resolv.conf into the chroot until /etc exists.
With an unpopulated chroot dir, release building was failing because
the script was attempting to copy resolv.conf into a non-existent /etc
directory of the chroot.  Fix this by copying the file only after the
installworld has happened, which will create this directory.

Reviewed by:	gjb
Approved by:	rpaulo (mentor)
2013-12-25 13:57:23 +00:00
gjb
e5fc4e1ef1 Add WITH_DVD to RELEASE_RMAKEFLAGS, otherwise it is not actually
passed to 'make release'.

MFC after:	3 days
X-Before-RC2:	yes
Sponsored by:	The FreeBSD Foundation
2013-12-07 19:39:38 +00:00
gjb
e5b8a30889 Move OPTIONS_UNSET outside of the PBUILD_FLAGS variable, otherwise the
textproc/docproj port build fails.

Pointyhat:	gjb
MFC after:	3 days
X-Before-RC1:	yes
Sponsored by:	The FreeBSD Foundation
2013-11-30 23:56:26 +00:00
gjb
5c3fff5c5f Unconditionally copy the build host /etc/resolv.conf into
the chroot directory, since hostname resolution may be
needed in the case of building a dvd image (with packages)
and also setting 'NOPORTS=1'.

MFC after:	3 days
X-MFC-With:	r258305, r258307
Sponsored by:	The FreeBSD Foundation
2013-11-18 16:04:04 +00:00
gjb
a1d0c0dabb Since the doc/ toolchain conversion to docbook 5.0, JADETEX
is no longer a valid option, so remove 'WITHOUT_JADETEX=yes'
from PBUILD_FLAGS.

While here, also remove 'WITHOUT_X11=yes', since it will
cause a dependency conflict by requiring both print/ghostscript9
and print/ghostscript9-nox11.

Switch to OPTIONSNG format (WITHOUT_* -> OPTIONS_UNSET='FOO'),
and unset the FOP option, which requires Java (although it is
disabled by default).  Also unset the 'IGOR' option, since
textproc/igor is a validation tool, not necessarily a dependency
of the doc/ build itself.

While here, reduce the line length of PBUILD_FLAGS.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-11-13 17:49:07 +00:00
gjb
190bdc75e8 Fix a few style nits.
MFC after:	3 days
X-MFC-After:	10.0-RELEASE
Sponsored by:	The FreeBSD Foundation
2013-11-11 03:30:14 +00:00
gjb
0b050e8ff3 When building the textproc/docproj port, the ports-mgmt/pkg port
needs /var/run/ld-elf*.so.hints, which is not automatically created.

Fix reldoc build by running the ldconfig(8) startup script in the
chroot directory before starting the port build phase.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-11-04 16:28:29 +00:00
gjb
627feb6294 Remove hash generation from release.sh, as it is run as part of
the 'install' target in the release/Makefile.

MFC after:	3 days
Submitted by:	jhb
Approved by:	re (hrs)
Sponsored by:	The FreeBSD Foundation
2013-10-09 22:00:03 +00:00
gjb
d60e48989e Remove extra 'MAKE_FLAGS' line.
Submitted by:	jhb
Approved by:	re (marius)
Sponsored by:	The FreeBSD Foundation
2013-09-19 19:49:31 +00:00
gjb
9a09e7fb6b Remove get_rev_branch(), functionality exists in the release/Makefile.
Submitted by:	hrs
2013-08-13 21:01:23 +00:00
gjb
cfc7739bb4 - Only set ARCH_FLAGS (TARGET/TARGET_ARCH) if specified, otherwise
allow the toolchain to detect the correct values.

- Remove {SRC,DOC,PORT}REVISION variables, and use 'branch@rNNNNNN'
  as the {SRC,DOC,PORT}BRANCH variables.

- Only set default KERNEL_FLAGS and WORLD_FLAGS make(1) jobs if the
  number of CPUs is greater than 1.

Submitted by:	hrs
2013-08-13 20:16:14 +00:00
gjb
3d6c89b2d6 - Add SRC_FORCE_CHECKOUT configuration option to force svn to checkout
the src/ tree into a directory that contains files/directories, such
  as a case where a custom kernel configuration file is specified.

- Allow specification of multiple KERNCONFs to pass to 'make release'.

- Move evaluation of NODOCS/NOPORTS earlier, and set based on how the
  release process expects these options to be evaluated.

- Wrap KERNCONF specification in double quotes, and use 'eval' so multiple
  kernel configurations do not cause the build to fail in strange ways.

- Set WITHOUT_X11 for the port build flags for the documentation toolchain
  build.  Also run 'clean distclean' targets during port build.

PR:		180192
Submitted by:	Anes Mukhametov
MFC after:	3 days
Approved by:	kib (mentor, implicit)
2013-07-05 22:04:49 +00:00
gjb
e08b36714d If MAKE_CONF and/or SRC_CONF are set and not character devices,
copy to chroot to use with release build.

Submitted by:	Garrett Cooper (original version)
Approved by:	kib (mentor)
MFC after:	1 week
2013-06-22 22:12:24 +00:00
gjb
9241c8f59f Fix OSVERSION variable within PBUILD_FLAGS.
Submitted by:	Garrett Cooper (in part)
Approved by:	kib (mentor)
MFC After:	1 week
X-MFC-With:	r251652
X-MFC-To:	stable/9 only
2013-06-16 17:54:56 +00:00
gjb
ceed11ef6c Add new script to run release builds. The script is heavily based upon
the release/generate-release.sh script by nwhitehorn.

This script can use optional configuration file to override defaults,
making running multiple serialzied builds with different specific
configurations or architecture-specific tunings quite easy.

Sample overrideable options are included in the release.conf.sample
file.

Approved by:	kib (mentor)
MFC After:	2 weeks
X-MFC-To:	stable/9 only
2013-06-12 13:15:28 +00:00