Commit Graph

1017 Commits

Author SHA1 Message Date
Glen Barber
6513e474ac Move virtual machine / cloud provider targets and
options from release/Makefile to their own Makefile.

Sponsored by:	The FreeBSD Foundation
2014-10-29 14:57:30 +00:00
Glen Barber
3991f309ed Add a separate make(1) target to release/Makefile to
build FreeBSD virtual machine disk images for use on
the Microsoft Azure service.

For now, this target is not directly connected to the
build, however can be manually invoked.

The 'vm-azure' target invokes {amd64,i386}/mk-azure.sh,
which does the heavy lifting to produce proper VHDs.
mk-azure.sh uses a configuration file, defaulting to
tools/azure.conf if otherwise unset.

Sponsored by:	The FreeBSD Foundation
2014-10-14 11:48:28 +00:00
Glen Barber
cadb91b066 Remove two stray comments added during the initial iterations
of testing, no longer needed.

Sponsored by:	The FreeBSD Foundation
2014-10-02 16:05:01 +00:00
Glen Barber
a9278c16be Add WITH_COMPRESSED_VMIMAGES variable, which when set enables
xz(1) compression of the virtual machine images.

This is intentionally separate to allow more fine-grained
tuning over which images are compressed, especially in cases
where compressing 20GB sparse images can take hours.

Sponsored by:	The FreeBSD Foundation
2014-10-02 00:34:03 +00:00
Glen Barber
a2950cc4e3 After evaluating WITH_VMIMAGES is non-empty, ensure
the mk-vmimage.sh script exists before running it.

Sponsored by:	The FreeBSD Foundation
2014-10-01 19:25:02 +00:00
Glen Barber
0bc9be5e74 Avoid using env(1) to set values passed to mk-vmimage.sh,
and instead pass the values as arguments to the script,
making it easier to run this by hand, without 'make release'.

Add usage_vm_base() and usage_vm_image() usage helpers.

Sponsored by:	The FreeBSD Foundation
2014-10-01 19:13:42 +00:00
Glen Barber
fadf2a24fe Separate release/scripts/mk-vmimage.sh to machine-specific
scripts, making it possible to mimic the functionality for
non-x86 targets.

Move echo output if MAKEFLAGS is empty outside of usage().

Remove TARGET/TARGET_ARCH evaluation.

Sponsored by:	The FreeBSD Foundation
2014-10-01 17:05:40 +00:00
Glen Barber
f54137c331 release/Makefile:
Include .OBJDIR in DESTDIR in the vm-base target.

release/release.sh:
  Provide the full path to mddev.

Sponsored by:	The FreeBSD Foundation
2014-09-28 20:06:02 +00:00
Glen Barber
4e187d25fa release/Makefile:
Connect the virtual machine image build to the release
  target if WITH_VMIMAGES is set to a non-empty value.

release/release.sh:
  Add WITH_VMIMAGES to RELEASE_RMAKEFLAGS.

release/release.conf.sample:
  Add commented entries for tuning the release build if the
  WITH_VMIMAGES make(1) environment variable is set to
  a non-empty value.

Sponsored by:	The FreeBSD Foundation
2014-09-28 17:53:10 +00:00
Glen Barber
c925095689 Use VMBASE in place of a hard-coded filename in the CLEANFILES list.
Sponsored by:	The FreeBSD Foundation
2014-09-28 01:53:02 +00:00
Glen Barber
a68d274b86 Initial commit to include virtual machine images as part
of the FreeBSD release builds.

This adds a make(1) environment variable requirement,
WITH_VMIMAGES, which triggers the virtual machine image
targets when not defined to an empty value.

Relevant user-driven variables include:

 o VMFORMATS:	The virtual machine image formats to create.
   Valid formats are provided by running 'mkimg --formats'

 o VMSIZE:	The size of the resulting virtual machine
   image.  Typical compression is roughly 140Mb, regardless
   of the target size (10GB, 15GB, 20GB, 40GB sizes have been
   tested with the same result).

 o VMBASE:	The prefix of the virtual machine disk images.
   The VMBASE make(1) environment variable is suffixed with
   each format in VMFORMATS for each individual disk image, as
   well as '.img' for the source UFS filesystem passed to
   mkimg(1).

This also includes a new script, mk-vmimage.sh, based on how
the VM images for 10.0-RELEASE, 9.3-RELEASE, and 10.1-RELEASE
were created (mk-vmimage.sh in ^/user/gjb/thermite/).

With the order in which the stages need to occur, as well as
sanity-checking error cases, it makes much more sense to
execute a shell script called from make(1), using env(1) to
set specific parameters for the target image than it does to
do this in make(1) directly.

Sponsored by:	The FreeBSD Foundation
2014-09-28 01:45:20 +00:00
Glen Barber
ffc6fb79b6 Merge ^/projects/release-debugdist into ^/head:
r262491, r262493, r262516, r267345, r267397:

r262491:
  Add DEBUG_DISTRIBUTIONS, and set it to include base and
  EXTRA_DISTRIBUTIONS, excluding 'doc', since the documentation
  distribution does not have corresponding debug information.

  Use DEBUG_DISTRIBUTIONS in the 'distributeworld installworld'
  and 'packageworld' targets, to reduce the number of occurances
  of excluding distributions that do not have .debug files.

r262493:
  In release/Makefile, explicitly set WITHOUT_DEBUG_FILES=1
  for dvdrom and cdrom targets.  (Later reverted.)

  Exclude the *.debug.txz distributions from dvdrom and
  cdrom images, but include them for ftp distribution.

r262516:
  Rename ${dist}.debug.txz to ${dist}-dbg.txz to prevent the
  following output:
	eval: ${base....}: Bad substitution
	eval: ${doc....}: Bad substitution
	eval: ${games....}: Bad substitution
	eval: ${lib32....}: Bad substitution

  This also follows other naming conventions seen in the
  wild.

r267345:
  Explicitly set MK_DEBUG_FILES=no, which overrides the
  WITH_DEBUG_FILES=1 and WITHOUT_DEBUG_FILES=1 collisions
  previously experienced.

  This change allows us to create the {base,kernel}_debug.txz
  distributions without accidentally installing the *.debug
  files on the medium itself.

r267397:
  Remove evaluations of MK_DEBUG_FILES where not needed.
  If DEBUG_DISTRIBUTIONS is empty, which is true if
  MK_DEBUG_FILES evaluates to 'no' above, the loop does
  nothing.

MFC after:	1 month
Tested on:	head@r267801
Reviewed by:	brooks [1], emaste, imp [1]
		[1] earlier version
Sponsored by:	The FreeBSD Foundation
2014-07-01 19:04:04 +00:00
Glen Barber
ba7bd912bf Switch release/Makefile knobs from WITHOUT_FOO=1 to MK_FOO=no.
This allows greater granular control over the release
build, and avoids WITH_FOO=1 and WITHOUT_FOO=1 collision.

This change was verified to produce correct results
compared to an earlier build by diffing find(1) output
of the resulting bootonly/, dvd/, and release/ directories.

Tested on:	head@r267449
MFC after:	1 month
X-MFC-Note:	Requires commits not yet MFC'd, so likely
		longer than 1 month.
Sponsored by:	The FreeBSD Foundation
2014-06-14 01:03:24 +00:00
Julio Merino
cb3ba77815 Strip out the test suite from the installation media.
This affects the disc1.iso, dvd1.iso and bootonly.iso files.

Obtained from:	gjb
2014-06-10 17:05:41 +00:00
Glen Barber
fdbd636433 Fix indentation level.
MFC after:	3 days
X-MFC-With:	r267326
Sponsored by:	The FreeBSD Foundation
2014-06-10 15:20:41 +00:00
Glen Barber
cd15d7f34e Add empty pkg-stage file to CLEANFILES if WITH_DVD=1.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2014-06-10 15:19:28 +00:00
Glen Barber
73e46dba11 Turn off the full witness trace on console output.
On head/, or more specifically, when WITNESS is in
the kernel config, the console is spammed excessively
with lock order reversal between isofs and devfs.

Set debug.witness.trace=0 in the installer sysctl.conf
to avoid printing the full KDB stack backtrace.  This
does not prevent printing the lock order reversal has
happened, only lessens the console spam.

Sponsored by:	The FreeBSD Foundation
2014-04-27 16:40:40 +00:00
Glen Barber
3b485e461d Allow overriding xz(1) path.
Requested by:	marck
MFC After:	5 days
X-MFC-With:	r264027,r264028,r264029,r264030,r264046
Sponsored by:	The FreeBSD Foundation
2014-04-03 12:04:05 +00:00
Glen Barber
55af5cf123 Use xz(1) instead of gzip(1) to compress release images
when WITH_COMPRESSED_IMAGES is used.

Requested by:	delphij, brooks, Nikolai Lifanov
MFC After:	1 week
X-MFC-With:	r264027,r264028,r264029,r264030
Sponsored by:	The FreeBSD Foundation
2014-04-02 16:53:07 +00:00
Glen Barber
0c5359919c If WITH_COMPRESSED_IMAGES is set, add the compressed images to
the CLEANFILES list.

MFC after:	1 week
X-MFC-With:	r264027,r264028,r264029
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:54:54 +00:00
Glen Barber
510f330370 Fix logic error: s/||/&&/
Pointyhat:	gjb
MFC after:	1 week
X-MFC-With:	r264027,r264028
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:52:27 +00:00
Glen Barber
86bb4323ab Clean up trailing whitespace in release/Makefile.
MFC after:	1 week
X-MFC-With:	r264027
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:46:46 +00:00
Glen Barber
cbc0e08a7b Add a new release build variable, WITH_COMPRESSED_IMAGES.
When set to a non-empty value, the installation medium is
compressed with gzip(1) as part of the 'install' target in
the release/ directory.

With gzip(1) compression, downloadable image are reduced in
size quite significantly.  Build test against head@263927
shows the following:

 bootonly.iso:		64% smaller
 disc1.iso:		44% smaller
 memstick.img:		47% smaller
 mini-memstick.img:	65% smaller
 dvd1.iso:		untested

This option is off by default, I would eventually like to
turn it on by default, and remove the '-k' flag to gzip(1)
so only compressed images are published on FTP.

Requested by:	wkoszek
MFC After:	1 week
Sponsored by:	The FreeBSD Foundation
2014-04-01 22:41:26 +00:00
Glen Barber
23659c9d9d In release/Makefile, remove exclusion of CVS directories in the
src/ and ports/ distributions.

While I am thinking about it, exclude .git directories for src/
and ports/, as somewhat of a followup to r262499.

Sponsored by:	The FreeBSD Foundation
2014-02-26 00:52:01 +00:00
John Hay
d6d73d476b etcupdate should use the src tree from where the release is built, not
the default (/usr/src) tree.

MFC after:	2 weeks
2014-02-17 12:29:17 +00:00
Glen Barber
6f50c4ea71 Update the pkg-stage target to be more compatible with pkg-1.2:
- Add a release-dvd.conf pkg(8) configuration file to override
   the default FreeBSD.conf configuration.
 - Remove architecture-specific pkg-stage.conf files, consolidate,
   and move their contents to scripts/pkg-stage.sh.
 - Use 'pkg -vv' to determine the ABI, which is used as the
   cache directory.

Prior to these changes, it would be possible for pkg-stage to fetch
conflicting binary packages from multiple repositories.

Tested against:	head@r260522, stable/10@r260522

MFC after:	3 days
X-Insta-MFC:	possibly
Sponsored by:	The FreeBSD Foundation
2014-01-16 16:12:09 +00:00
Glen Barber
e9120a2560 Bootstrap etcupdate(8) as part of the release build, similar
to what is done for mergemaster(8).  This allows etcupdate(8)
to work out-of-box after the first upgrade of a system.

Submitted by:	jhb
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-12-22 16:12:47 +00:00
Glen Barber
0d0e89b794 Add a pkg(8) repository configuration file for cdrom-based package
installation.

As part of the 'pkg-stage' target, copy the configuration file
to the 'packages/repos/' directory on the DVD filesystem.

MFC after:	3 days
X-MFC-Before:	-RC3
Sponsored by:	The FreeBSD Foundation
2013-12-15 20:38:07 +00:00
Glen Barber
b1e3e96568 Reduce disc1.iso size by 74MB by removing lib32 libraries, sendmail,
and atf.

MFC after:	3 days
X-MFC-To-10:	immediate (-RC2 requirement)
Sponsored by:	The FreeBSD Foundation
2013-12-14 22:41:17 +00:00
Glen Barber
d8824524b7 Ensure WITH_DVD is not empty to prevent 'WITH_DVD= ' from evaluating
to true.

Submitted by:	hrs
MFC after:	3 days
X-MFC-to-10:	immediate
Sponsored by:	The FreeBSD Foundation
2013-12-05 00:56:03 +00:00
Glen Barber
f7f650c76a Turn off the dvdrom target by default. dvd1.iso is now built by
specifying 'WITH_DVD=1' during 'make release'.

This caused some problems during the freebsd-update builds for
10.0-BETA4.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-12-02 18:47:57 +00:00
Hiroki Sato
46b13263cf Use standard CLEANFILES/CLEANDIRS and clean target in bsd.obj.mk. 2013-12-01 15:24:35 +00:00
Hiroki Sato
464485c8d0 - Prevent TARGET and TARGET_ARCH from being defined as empty when
TARGET="" and/or TARGET_ARCH="" is specified.
- Remove extra ().
2013-12-01 15:06:30 +00:00
Hiroki Sato
9d22cb2e69 Add NOPKG to disable pkg-stage. 2013-11-30 17:07:53 +00:00
Glen Barber
5d27274a06 Document the dvdrom target and NODVD variable, forgotten with r258310.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-11-27 03:05:24 +00:00
Glen Barber
2b471d3acb Remove WITHOUT_PROFILE=1 for the dvd1.iso medium.
Requested by:	hrs
MFC after:	3 days
X-MFC-With:	r258310
Sponsored by:	The FreeBSD Foundation
2013-11-18 19:57:11 +00:00
Glen Barber
dc2445625a Add the 'dvd1.iso' target. This mimics the 'release.iso' target,
with the additional step of fetching packages for inclusion on the
dvd image.

The 'pkg-stage' target is used to run 'scripts/pkg-stage.sh' if
the '${TARGET}/pkg-stage.conf' configuration file exists (currently
only amd64 and i386).

Allow dvd1.iso to be skipped if NODVD=1.

MFC after:	3 days
X-MFC-With:	r258305, r258307, r258308, r258309
Sponsored by:	The FreeBSD Foundation
2013-11-18 16:25:56 +00:00
Glen Barber
849861d03f Use the IMAGES variable to determine which image files to remove
as part of 'make -C /usr/src/release clean'.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-11-18 15:11:13 +00:00
Glen Barber
cb95a96bee Remove extra target from 'memstick' that I forgot to remove before
previous commit.

MFC after:	1 week
X-MFC-After:	10.0-RELEASE
X-MFC-With:	r257805
Sponsored by:	The FreeBSD Foundation
2013-11-07 18:59:06 +00:00
Glen Barber
a9da1707e5 Add a 'mini-memstick.img' release target, which will use the
'bootonly.iso' components to create a smaller memory stick image.

This is useful for system recovery, where a full memstick.img image
is not necessarily needed (or wanted).  In addition, it is possible to
do bootonly-style installation, where the base.txz, kernel.txz, etc.
are fetched from a remote source.

Provide backwards-compatible target (mini-memstick), to keep in sync
with the targets documented in release/Makefile.

Prompted by:	wblock
MFC after:	1 week
X-MFC-after:	10.0-RELEASE
Sponsored by:	The FreeBSD Foundation
2013-11-07 18:53:55 +00:00
Glen Barber
58ddb0a684 Reduce disc1.iso image size by installing the userland with
the WITHOUT_PROFILE=1 option set, trimming 56MB from the image.

Approved by:	re (glebius)
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2013-10-13 15:49:50 +00:00
Dag-Erling Smørgrav
56b72efe82 Remove BIND.
Approved by:	re (gjb)
2013-09-30 17:23:45 +00:00
Glen Barber
8cec99b682 For consistency with previous releases, name the cdrom install
cd 'disc1.iso'.

Discussed with:	re@ team
Approved by:	kib (mentor)
MFC after:	3 days
X-MFC-To:	stable/9 only
2013-07-22 12:07:57 +00:00
Glen Barber
3c334738ab Create memstick images with the '.img' filename extension.
Requested by:	many
Approved by:	kib (mentor)
MFC after:	3 days
X-MFC-To:	stable/9 only
2013-07-21 05:23:34 +00:00
Tijl Coosemans
53089271a1 Convert old make variable modifiers :U and :L to bmake :tu and :tl.
Reviewed by:	sjg
2013-06-02 11:44:23 +00:00
Hiroki Sato
de17cbcc7e - Add CHECKSUM.* support in Makefile[1].
- Use ln -fs to create a symlink.
- Remove pkgadd for docports.
- Use WITHOUT_JADETEX=yes instead of WITH_JADETEX=no.
- Add {WORLD,KERNEL}_FLAGS to [BTWK]MAKE.
- Use makefs(8) and gpart(8) for sparc64 ISO image[2].
- Add publisher option to makefs(8)[2].

Based on work by:	gjb[1]
Discussed with:		marius, nwhitehorn[2]
2013-02-03 10:26:24 +00:00
Glen Barber
9f5907dbbe Prevent including .zfs snapshot directories in the src.txz
distribution.  This can happen if the src/ tree checkout is
within its own ZFS dataset, and the 'snapdir' ZFS property
is set to 'visible.'

Approved by:	hrs
MFC after:	3 days
X-MFC-To:	stable/9 only
2012-11-11 21:52:18 +00:00
Ed Maste
c4acc58d1d Exclude potential circular symlink
This does not occur with normal release builds but can happen when reusing
these release parts for other purposes.
2012-10-15 17:19:47 +00:00
Ed Maste
f0c51ba5c7 Exclude potential circular symlink when creating src tarball.
Normal release builds are not affected by this but it can happen if using
these release bits against a work tree.

Sponsored by: ADARA Networks
2012-10-11 18:41:32 +00:00
Nathan Whitehorn
6f4fc34468 Make sure not to tar up CVS directories if this ports tree is a CVS
checkout.

PR:		bin/165868
MFC after:	5 days
2012-03-09 02:23:03 +00:00