Commit Graph

1019 Commits

Author SHA1 Message Date
gjb
f9cdf3e5ad Fix VOLUME_LABEL when BRANCH contains non-alpha
characters, in particular '-' and '.'.

Submitted by:	des
Tested by:	des
MFC after:	3 days
X-MFC-10.1:	yes
Sponsored by:	The FreeBSD Foundation
2014-11-04 17:24:13 +00:00
gjb
c23ea90a43 Fix a few issues with creating VOLUME_LABEL for the
installation ISOs:

 - TYPE, BRANCH, and REVISION are only defined if
   OSRELEASE is not defined, so in situations where
   one might set OSRELEASE for an in-house ISO build,
   VOLUME_LABEL would be empty.

 - makefs(8) limits the volume label to 32 characters,
   which for the powerpc64 case, OSRELEASE expands to
   FreeBSD-11.0-CURRENT-powerpc-powerpc64.  Even with
   removing the prefixing 'FreeBSD-', the string is 30
   characters long, leaving zero room for suffixing the
   type of ISO media (BO for bootonly, CD for cdrom, and
   DVD for dvdrom).

Resolve these by defining VOLUME_LABEL when defining
OSRELEASE if unset.  If OSRELEASE is defined by the
builder, use the OSRELEASE from that definition as the
VOLUME_LABEL.

In addition, for cases where both TARGET and TARGET_ARCH
are used for the VOLUME_LABEL, use TARGET_ARCH if it
differs from TARGET.

There are probably a few sharp edges here yet, but these
problems are going to affect the powerpc/powerpc64 builds
for 10.1-RELEASE, so the immediate concern is fixing the
underlying problem at hand quickly, and less so about the
elegance of the fix.

MFC after:	3 days
X-MFC-10.1:	yes, asap
Sponsored by:	The FreeBSD Foundation
2014-10-26 01:41:54 +00:00
gjb
679a7cd5ab Add more descriptive metadata to the ISO images.
PR: 		146079
Submitted by: 	Roman Bogorodskiy
MFC after: 	3 days
X-MFC-10.1: 	yes
Sponsored by:	The FreeBSD Foundation
2014-10-17 03:55:33 +00:00
gjb
d7ed1332f2 Add a separate make(1) target to release/Makefile to
build FreeBSD virtual machine disk images for use on
the Microsoft Azure service.

For now, this target is not directly connected to the
build, however can be manually invoked.

The 'vm-azure' target invokes {amd64,i386}/mk-azure.sh,
which does the heavy lifting to produce proper VHDs.
mk-azure.sh uses a configuration file, defaulting to
tools/azure.conf if otherwise unset.

Sponsored by:	The FreeBSD Foundation
2014-10-14 11:48:28 +00:00
gjb
040f030a24 Remove two stray comments added during the initial iterations
of testing, no longer needed.

Sponsored by:	The FreeBSD Foundation
2014-10-02 16:05:01 +00:00
gjb
470a786adc Add WITH_COMPRESSED_VMIMAGES variable, which when set enables
xz(1) compression of the virtual machine images.

This is intentionally separate to allow more fine-grained
tuning over which images are compressed, especially in cases
where compressing 20GB sparse images can take hours.

Sponsored by:	The FreeBSD Foundation
2014-10-02 00:34:03 +00:00
gjb
7403b4f22a After evaluating WITH_VMIMAGES is non-empty, ensure
the mk-vmimage.sh script exists before running it.

Sponsored by:	The FreeBSD Foundation
2014-10-01 19:25:02 +00:00
gjb
b2b02a9be6 Avoid using env(1) to set values passed to mk-vmimage.sh,
and instead pass the values as arguments to the script,
making it easier to run this by hand, without 'make release'.

Add usage_vm_base() and usage_vm_image() usage helpers.

Sponsored by:	The FreeBSD Foundation
2014-10-01 19:13:42 +00:00
gjb
78c1b9246c Separate release/scripts/mk-vmimage.sh to machine-specific
scripts, making it possible to mimic the functionality for
non-x86 targets.

Move echo output if MAKEFLAGS is empty outside of usage().

Remove TARGET/TARGET_ARCH evaluation.

Sponsored by:	The FreeBSD Foundation
2014-10-01 17:05:40 +00:00
gjb
73ba8a8869 release/Makefile:
Include .OBJDIR in DESTDIR in the vm-base target.

release/release.sh:
  Provide the full path to mddev.

Sponsored by:	The FreeBSD Foundation
2014-09-28 20:06:02 +00:00
gjb
2aaf766ec5 release/Makefile:
Connect the virtual machine image build to the release
  target if WITH_VMIMAGES is set to a non-empty value.

release/release.sh:
  Add WITH_VMIMAGES to RELEASE_RMAKEFLAGS.

release/release.conf.sample:
  Add commented entries for tuning the release build if the
  WITH_VMIMAGES make(1) environment variable is set to
  a non-empty value.

Sponsored by:	The FreeBSD Foundation
2014-09-28 17:53:10 +00:00
gjb
2bb4ac8630 Use VMBASE in place of a hard-coded filename in the CLEANFILES list.
Sponsored by:	The FreeBSD Foundation
2014-09-28 01:53:02 +00:00
gjb
c2a91dd928 Initial commit to include virtual machine images as part
of the FreeBSD release builds.

This adds a make(1) environment variable requirement,
WITH_VMIMAGES, which triggers the virtual machine image
targets when not defined to an empty value.

Relevant user-driven variables include:

 o VMFORMATS:	The virtual machine image formats to create.
   Valid formats are provided by running 'mkimg --formats'

 o VMSIZE:	The size of the resulting virtual machine
   image.  Typical compression is roughly 140Mb, regardless
   of the target size (10GB, 15GB, 20GB, 40GB sizes have been
   tested with the same result).

 o VMBASE:	The prefix of the virtual machine disk images.
   The VMBASE make(1) environment variable is suffixed with
   each format in VMFORMATS for each individual disk image, as
   well as '.img' for the source UFS filesystem passed to
   mkimg(1).

This also includes a new script, mk-vmimage.sh, based on how
the VM images for 10.0-RELEASE, 9.3-RELEASE, and 10.1-RELEASE
were created (mk-vmimage.sh in ^/user/gjb/thermite/).

With the order in which the stages need to occur, as well as
sanity-checking error cases, it makes much more sense to
execute a shell script called from make(1), using env(1) to
set specific parameters for the target image than it does to
do this in make(1) directly.

Sponsored by:	The FreeBSD Foundation
2014-09-28 01:45:20 +00:00
gjb
be29e6f09d Merge ^/projects/release-debugdist into ^/head:
r262491, r262493, r262516, r267345, r267397:

r262491:
  Add DEBUG_DISTRIBUTIONS, and set it to include base and
  EXTRA_DISTRIBUTIONS, excluding 'doc', since the documentation
  distribution does not have corresponding debug information.

  Use DEBUG_DISTRIBUTIONS in the 'distributeworld installworld'
  and 'packageworld' targets, to reduce the number of occurances
  of excluding distributions that do not have .debug files.

r262493:
  In release/Makefile, explicitly set WITHOUT_DEBUG_FILES=1
  for dvdrom and cdrom targets.  (Later reverted.)

  Exclude the *.debug.txz distributions from dvdrom and
  cdrom images, but include them for ftp distribution.

r262516:
  Rename ${dist}.debug.txz to ${dist}-dbg.txz to prevent the
  following output:
	eval: ${base....}: Bad substitution
	eval: ${doc....}: Bad substitution
	eval: ${games....}: Bad substitution
	eval: ${lib32....}: Bad substitution

  This also follows other naming conventions seen in the
  wild.

r267345:
  Explicitly set MK_DEBUG_FILES=no, which overrides the
  WITH_DEBUG_FILES=1 and WITHOUT_DEBUG_FILES=1 collisions
  previously experienced.

  This change allows us to create the {base,kernel}_debug.txz
  distributions without accidentally installing the *.debug
  files on the medium itself.

r267397:
  Remove evaluations of MK_DEBUG_FILES where not needed.
  If DEBUG_DISTRIBUTIONS is empty, which is true if
  MK_DEBUG_FILES evaluates to 'no' above, the loop does
  nothing.

MFC after:	1 month
Tested on:	head@r267801
Reviewed by:	brooks [1], emaste, imp [1]
		[1] earlier version
Sponsored by:	The FreeBSD Foundation
2014-07-01 19:04:04 +00:00
gjb
9b3c9d611e Switch release/Makefile knobs from WITHOUT_FOO=1 to MK_FOO=no.
This allows greater granular control over the release
build, and avoids WITH_FOO=1 and WITHOUT_FOO=1 collision.

This change was verified to produce correct results
compared to an earlier build by diffing find(1) output
of the resulting bootonly/, dvd/, and release/ directories.

Tested on:	head@r267449
MFC after:	1 month
X-MFC-Note:	Requires commits not yet MFC'd, so likely
		longer than 1 month.
Sponsored by:	The FreeBSD Foundation
2014-06-14 01:03:24 +00:00
jmmv
7726c9bd9f Strip out the test suite from the installation media.
This affects the disc1.iso, dvd1.iso and bootonly.iso files.

Obtained from:	gjb
2014-06-10 17:05:41 +00:00
gjb
24c8aa65fd Fix indentation level.
MFC after:	3 days
X-MFC-With:	r267326
Sponsored by:	The FreeBSD Foundation
2014-06-10 15:20:41 +00:00
gjb
bca56c387a Add empty pkg-stage file to CLEANFILES if WITH_DVD=1.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2014-06-10 15:19:28 +00:00
gjb
bb7c7a9614 Turn off the full witness trace on console output.
On head/, or more specifically, when WITNESS is in
the kernel config, the console is spammed excessively
with lock order reversal between isofs and devfs.

Set debug.witness.trace=0 in the installer sysctl.conf
to avoid printing the full KDB stack backtrace.  This
does not prevent printing the lock order reversal has
happened, only lessens the console spam.

Sponsored by:	The FreeBSD Foundation
2014-04-27 16:40:40 +00:00
gjb
abe0b75d08 Allow overriding xz(1) path.
Requested by:	marck
MFC After:	5 days
X-MFC-With:	r264027,r264028,r264029,r264030,r264046
Sponsored by:	The FreeBSD Foundation
2014-04-03 12:04:05 +00:00
gjb
4a1f79beb5 Use xz(1) instead of gzip(1) to compress release images
when WITH_COMPRESSED_IMAGES is used.

Requested by:	delphij, brooks, Nikolai Lifanov
MFC After:	1 week
X-MFC-With:	r264027,r264028,r264029,r264030
Sponsored by:	The FreeBSD Foundation
2014-04-02 16:53:07 +00:00
gjb
6acd43a12d If WITH_COMPRESSED_IMAGES is set, add the compressed images to
the CLEANFILES list.

MFC after:	1 week
X-MFC-With:	r264027,r264028,r264029
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:54:54 +00:00
gjb
11f7679c0d Fix logic error: s/||/&&/
Pointyhat:	gjb
MFC after:	1 week
X-MFC-With:	r264027,r264028
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:52:27 +00:00
gjb
5e982f9855 Clean up trailing whitespace in release/Makefile.
MFC after:	1 week
X-MFC-With:	r264027
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:46:46 +00:00
gjb
0fbab8558b Add a new release build variable, WITH_COMPRESSED_IMAGES.
When set to a non-empty value, the installation medium is
compressed with gzip(1) as part of the 'install' target in
the release/ directory.

With gzip(1) compression, downloadable image are reduced in
size quite significantly.  Build test against head@263927
shows the following:

 bootonly.iso:		64% smaller
 disc1.iso:		44% smaller
 memstick.img:		47% smaller
 mini-memstick.img:	65% smaller
 dvd1.iso:		untested

This option is off by default, I would eventually like to
turn it on by default, and remove the '-k' flag to gzip(1)
so only compressed images are published on FTP.

Requested by:	wkoszek
MFC After:	1 week
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:41:26 +00:00
gjb
aa58833be6 In release/Makefile, remove exclusion of CVS directories in the
src/ and ports/ distributions.

While I am thinking about it, exclude .git directories for src/
and ports/, as somewhat of a followup to r262499.

Sponsored by:	The FreeBSD Foundation
2014-02-26 00:52:01 +00:00
jhay
dcf30b256c etcupdate should use the src tree from where the release is built, not
the default (/usr/src) tree.

MFC after:	2 weeks
2014-02-17 12:29:17 +00:00
gjb
284a0c8006 Update the pkg-stage target to be more compatible with pkg-1.2:
- Add a release-dvd.conf pkg(8) configuration file to override
   the default FreeBSD.conf configuration.
 - Remove architecture-specific pkg-stage.conf files, consolidate,
   and move their contents to scripts/pkg-stage.sh.
 - Use 'pkg -vv' to determine the ABI, which is used as the
   cache directory.

Prior to these changes, it would be possible for pkg-stage to fetch
conflicting binary packages from multiple repositories.

Tested against:	head@r260522, stable/10@r260522

MFC after:	3 days
X-Insta-MFC:	possibly
Sponsored by:	The FreeBSD Foundation
2014-01-16 16:12:09 +00:00
gjb
47ed2e7236 Bootstrap etcupdate(8) as part of the release build, similar
to what is done for mergemaster(8).  This allows etcupdate(8)
to work out-of-box after the first upgrade of a system.

Submitted by:	jhb
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-12-22 16:12:47 +00:00
gjb
f5110df30b Add a pkg(8) repository configuration file for cdrom-based package
installation.

As part of the 'pkg-stage' target, copy the configuration file
to the 'packages/repos/' directory on the DVD filesystem.

MFC after:	3 days
X-MFC-Before:	-RC3
Sponsored by:	The FreeBSD Foundation
2013-12-15 20:38:07 +00:00
gjb
c6b603dccf Reduce disc1.iso size by 74MB by removing lib32 libraries, sendmail,
and atf.

MFC after:	3 days
X-MFC-To-10:	immediate (-RC2 requirement)
Sponsored by:	The FreeBSD Foundation
2013-12-14 22:41:17 +00:00
gjb
8801a18040 Ensure WITH_DVD is not empty to prevent 'WITH_DVD= ' from evaluating
to true.

Submitted by:	hrs
MFC after:	3 days
X-MFC-to-10:	immediate
Sponsored by:	The FreeBSD Foundation
2013-12-05 00:56:03 +00:00
gjb
f289f4d4b8 Turn off the dvdrom target by default. dvd1.iso is now built by
specifying 'WITH_DVD=1' during 'make release'.

This caused some problems during the freebsd-update builds for
10.0-BETA4.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-12-02 18:47:57 +00:00
hrs
89adebbdbc Use standard CLEANFILES/CLEANDIRS and clean target in bsd.obj.mk. 2013-12-01 15:24:35 +00:00
hrs
3af21b4c79 - Prevent TARGET and TARGET_ARCH from being defined as empty when
TARGET="" and/or TARGET_ARCH="" is specified.
- Remove extra ().
2013-12-01 15:06:30 +00:00
hrs
bcd91960e5 Add NOPKG to disable pkg-stage. 2013-11-30 17:07:53 +00:00
gjb
78f08ebb35 Document the dvdrom target and NODVD variable, forgotten with r258310.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-11-27 03:05:24 +00:00
gjb
f4030e9ee6 Remove WITHOUT_PROFILE=1 for the dvd1.iso medium.
Requested by:	hrs
MFC after:	3 days
X-MFC-With:	r258310
Sponsored by:	The FreeBSD Foundation
2013-11-18 19:57:11 +00:00
gjb
f240b29444 Add the 'dvd1.iso' target. This mimics the 'release.iso' target,
with the additional step of fetching packages for inclusion on the
dvd image.

The 'pkg-stage' target is used to run 'scripts/pkg-stage.sh' if
the '${TARGET}/pkg-stage.conf' configuration file exists (currently
only amd64 and i386).

Allow dvd1.iso to be skipped if NODVD=1.

MFC after:	3 days
X-MFC-With:	r258305, r258307, r258308, r258309
Sponsored by:	The FreeBSD Foundation
2013-11-18 16:25:56 +00:00
gjb
1c46d4842f Use the IMAGES variable to determine which image files to remove
as part of 'make -C /usr/src/release clean'.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-11-18 15:11:13 +00:00
gjb
e34cfa1b16 Remove extra target from 'memstick' that I forgot to remove before
previous commit.

MFC after:	1 week
X-MFC-After:	10.0-RELEASE
X-MFC-With:	r257805
Sponsored by:	The FreeBSD Foundation
2013-11-07 18:59:06 +00:00
gjb
d03c16de07 Add a 'mini-memstick.img' release target, which will use the
'bootonly.iso' components to create a smaller memory stick image.

This is useful for system recovery, where a full memstick.img image
is not necessarily needed (or wanted).  In addition, it is possible to
do bootonly-style installation, where the base.txz, kernel.txz, etc.
are fetched from a remote source.

Provide backwards-compatible target (mini-memstick), to keep in sync
with the targets documented in release/Makefile.

Prompted by:	wblock
MFC after:	1 week
X-MFC-after:	10.0-RELEASE
Sponsored by:	The FreeBSD Foundation
2013-11-07 18:53:55 +00:00
gjb
05dba22165 Reduce disc1.iso image size by installing the userland with
the WITHOUT_PROFILE=1 option set, trimming 56MB from the image.

Approved by:	re (glebius)
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2013-10-13 15:49:50 +00:00
des
aa2e4b623c Remove BIND.
Approved by:	re (gjb)
2013-09-30 17:23:45 +00:00
gjb
50d0b7bd21 For consistency with previous releases, name the cdrom install
cd 'disc1.iso'.

Discussed with:	re@ team
Approved by:	kib (mentor)
MFC after:	3 days
X-MFC-To:	stable/9 only
2013-07-22 12:07:57 +00:00
gjb
25c503b103 Create memstick images with the '.img' filename extension.
Requested by:	many
Approved by:	kib (mentor)
MFC after:	3 days
X-MFC-To:	stable/9 only
2013-07-21 05:23:34 +00:00
tijl
960738a3a8 Convert old make variable modifiers :U and :L to bmake :tu and :tl.
Reviewed by:	sjg
2013-06-02 11:44:23 +00:00
hrs
aa08976cf0 - Add CHECKSUM.* support in Makefile[1].
- Use ln -fs to create a symlink.
- Remove pkgadd for docports.
- Use WITHOUT_JADETEX=yes instead of WITH_JADETEX=no.
- Add {WORLD,KERNEL}_FLAGS to [BTWK]MAKE.
- Use makefs(8) and gpart(8) for sparc64 ISO image[2].
- Add publisher option to makefs(8)[2].

Based on work by:	gjb[1]
Discussed with:		marius, nwhitehorn[2]
2013-02-03 10:26:24 +00:00
gjb
d019ce692f Prevent including .zfs snapshot directories in the src.txz
distribution.  This can happen if the src/ tree checkout is
within its own ZFS dataset, and the 'snapdir' ZFS property
is set to 'visible.'

Approved by:	hrs
MFC after:	3 days
X-MFC-To:	stable/9 only
2012-11-11 21:52:18 +00:00
emaste
01647c1f15 Exclude potential circular symlink
This does not occur with normal release builds but can happen when reusing
these release parts for other purposes.
2012-10-15 17:19:47 +00:00