Commit Graph

404 Commits

Author SHA1 Message Date
Glen Barber
144d107fcd Move usage() from vmimage.subr to mk-vmimage.sh,
in case vmimage.subr has not been sourced.

Sponsored by:	The FreeBSD Foundation
2014-11-08 12:47:21 +00:00
Glen Barber
87f6c4ac27 Return if vm_create_disk() is unsuccessful.
Sponsored by:	The FreeBSD Foundation
2014-11-08 12:23:50 +00:00
Glen Barber
1e7c1f1742 Initial rewrite to consolidate VM image build scripts into one.
There may be some very sharp edges here while refactoring.

 - Move amd64/mk-vmimage.sh -> scripts/mk-vmimage.sh.
 - Remove vm-base target from Makefile.vm.
 - In vm-image target, use getopts flags for argument passing.
 - Create tools/vmimage.subr, containing default and prototype
   for the following functions that are used to drive the build,
   run in this order:

   vm_install_base()
   vm_extra_install_base()
   vm_extra_install_packages()
   vm_extra_install_ports()
   vm_extra_enable_services()
   vm_extra_pre_umount()
   vm_create_disk()
   vm_extra_create_disk()

 - In tools/azure.conf, override:

   vm_extra_install_base()
   vm_extra_pre_umount()
   vm_extra_create_disk()

 - In tools/openstack.conf, override:

   vm_extra_install_base()
   vm_extra_pre_umount()

Sponsored by:	The FreeBSD Foundation
2014-11-05 13:22:19 +00:00
Glen Barber
fadf2a24fe Separate release/scripts/mk-vmimage.sh to machine-specific
scripts, making it possible to mimic the functionality for
non-x86 targets.

Move echo output if MAKEFLAGS is empty outside of usage().

Remove TARGET/TARGET_ARCH evaluation.

Sponsored by:	The FreeBSD Foundation
2014-10-01 17:05:40 +00:00
Glen Barber
f6490fe596 Comment cleanup in panic() message when mkimg(1) does not support
the requested disk image format.

Sponsored by:	The FreeBSD Foundation
2014-09-29 01:17:42 +00:00
Glen Barber
1e2f9a549b Remove the UFS label from the root filesystem since it is added
by mkimg(1) as a gpt label, consistent with the fstab(5) entry.

Sponsored by:	The FreeBSD Foundation
2014-09-28 23:22:55 +00:00
Glen Barber
5baa1b2730 Remove comments left in accidentally while testing, so the
VM /etc/fstab is actually created.

Sponsored by:	The FreeBSD Foundation
2014-09-28 21:15:30 +00:00
Glen Barber
79a94b12ae Fix UFS label for the root filesystem.
Sponsored by:	The FreeBSD Foundation
2014-09-28 21:13:30 +00:00
Glen Barber
f54137c331 release/Makefile:
Include .OBJDIR in DESTDIR in the vm-base target.

release/release.sh:
  Provide the full path to mddev.

Sponsored by:	The FreeBSD Foundation
2014-09-28 20:06:02 +00:00
Glen Barber
8234f86da8 Remove a 'set -x' that snuck in during testing.
Sponsored by:	The FreeBSD Foundation
2014-09-28 17:16:45 +00:00
Glen Barber
a68d274b86 Initial commit to include virtual machine images as part
of the FreeBSD release builds.

This adds a make(1) environment variable requirement,
WITH_VMIMAGES, which triggers the virtual machine image
targets when not defined to an empty value.

Relevant user-driven variables include:

 o VMFORMATS:	The virtual machine image formats to create.
   Valid formats are provided by running 'mkimg --formats'

 o VMSIZE:	The size of the resulting virtual machine
   image.  Typical compression is roughly 140Mb, regardless
   of the target size (10GB, 15GB, 20GB, 40GB sizes have been
   tested with the same result).

 o VMBASE:	The prefix of the virtual machine disk images.
   The VMBASE make(1) environment variable is suffixed with
   each format in VMFORMATS for each individual disk image, as
   well as '.img' for the source UFS filesystem passed to
   mkimg(1).

This also includes a new script, mk-vmimage.sh, based on how
the VM images for 10.0-RELEASE, 9.3-RELEASE, and 10.1-RELEASE
were created (mk-vmimage.sh in ^/user/gjb/thermite/).

With the order in which the stages need to occur, as well as
sanity-checking error cases, it makes much more sense to
execute a shell script called from make(1), using env(1) to
set specific parameters for the target image than it does to
do this in make(1) directly.

Sponsored by:	The FreeBSD Foundation
2014-09-28 01:45:20 +00:00
Glen Barber
83f0a683f4 When populating the dvd/packages/ directory, create
a symlink to All/pkg-*.txz in the Latest/ directory.

This allows 'pkg bootstrap' to work out-of-box if
the REPOS_DIR environment is properly set.

Tested on:	stable/10@r271848
MFC after:	3 days
X-MFC-10.1:	yes
Sponsored by:	The FreeBSD Foundation
2014-09-19 17:23:44 +00:00
Glen Barber
53bdd39b8a Simplify dvd package population with pkg-1.3.
Submitted by:	bdrewery
MFC after:	3 days
X-MFC-With:	r271480, r271483
Sponsored by:	The FreeBSD Foundation
2014-09-12 22:20:07 +00:00
Glen Barber
b60a2f70da Fix duplicate PKG_ABI in the PKG_CACHEDIR path.
MFC after:	3 days
X-MFC-With:	r271480
Sponsored by:	The FreeBSD Foundation
2014-09-12 20:16:55 +00:00
Glen Barber
017717d87f Set PKG_CACHEDIR to an 'All/' directory one level lower
to fix 'pkg repo' generating repository metadata for the
on-disc packages.

MFC after:	3 days
X-MFC-to-10.1:	yes
Sponsored by:	The FreeBSD Foundation
2014-09-12 18:52:01 +00:00
Gavin Atkinson
ac1b139605 Remove an nunnecessary reference to csup. 2014-06-25 12:17:00 +00:00
Glen Barber
540a8eac2f Disable the main FreeBSD pkg(7) repositories in the dvd
repository configuration to avoid fetching from upstream
in case there may be conflicts.

MFC After:	3 days
Sponsored by:	The FreeBSD Foundation
2014-05-22 19:25:41 +00:00
Glen Barber
f0d60b2a4e Add forward-compatibility glue with pkg-1.3:
- Use ASSUME_ALWAYS_YES=YES instead of ASSUME_ALWAYS_YES=1
   since pkg-1.3 expects "yes" or "true" values.
 - Before exporting PKG_ABI, strip extra characters from what
   is parsed from 'pkg -vv'.  This causes problems further down
   when creating the packages directory for inclusion on the
   dvd1.iso.  Previously PKG_ABI would be 'freebsd:9:x86:64',
   but now is '"freebsd:9:x86:64";' in pkg-1.3

Tested on:	stable/9@r265858 with ports-mgmt/pkg-devel
MFC After:	3 days
Sponsored by:	The FreeBSD Foundation
2014-05-22 19:22:03 +00:00
Glen Barber
4c9e857efa Use 'return' instead of 'exit' in main(), since the script exits
with the last returned exit code.

Sponsored by:	The FreeBSD Foundation
2014-05-05 23:36:46 +00:00
Glen Barber
aa8426befc Add a helper script for parsing 'svn log' for the
'Relnotes:' tag in case 'yes' is not explicitly
the first string value following the tab.

As it turns out, a number of commits have bypassed
my filters (both email and 'svn log --search'), and
this script returns the results I want when doing
these searches.

Sponsored by:	The FreeBSD Foundation
2014-05-05 23:01:01 +00:00
Craig Rodrigues
82cac3982b Add a script which lists SVN log entries only for changesets
which have files which have been Added or Removed.

This can help identify changesets which should be
mentioned in Release Engineering's release notes.
2014-02-28 23:12:20 +00:00
Glen Barber
6f50c4ea71 Update the pkg-stage target to be more compatible with pkg-1.2:
- Add a release-dvd.conf pkg(8) configuration file to override
   the default FreeBSD.conf configuration.
 - Remove architecture-specific pkg-stage.conf files, consolidate,
   and move their contents to scripts/pkg-stage.sh.
 - Use 'pkg -vv' to determine the ABI, which is used as the
   cache directory.

Prior to these changes, it would be possible for pkg-stage to fetch
conflicting binary packages from multiple repositories.

Tested against:	head@r260522, stable/10@r260522

MFC after:	3 days
X-Insta-MFC:	possibly
Sponsored by:	The FreeBSD Foundation
2014-01-16 16:12:09 +00:00
Glen Barber
0d0e89b794 Add a pkg(8) repository configuration file for cdrom-based package
installation.

As part of the 'pkg-stage' target, copy the configuration file
to the 'packages/repos/' directory on the DVD filesystem.

MFC after:	3 days
X-MFC-Before:	-RC3
Sponsored by:	The FreeBSD Foundation
2013-12-15 20:38:07 +00:00
Glen Barber
89df2158c5 Prevent release build errors found during snapshot builds where if
NOPORTS=1, pkg-stage.sh cannot build the ports-mgmt/pkg port if
WITH_DVD=1.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-12-12 05:49:27 +00:00
Glen Barber
4e11928ded Provide reproducibility between builds by building pkg(8) from ports,
instead of using pkg-bootstrap.

This should resolve a problem that was discovered during 10.0-BETA4
freebsd-update(8) builds,

Reported by:	secteam (delphij)
MFC after:	3 days
X-MFC-to-10:	possibly immediately, pending freebsd-update(8) builds
Sponsored by:	The FreeBSD Foundation
2013-12-02 14:28:30 +00:00
Glen Barber
042a51f641 Add a script and configuration files to fetch pre-built packages
from pkg.FreeBSD.org for inclusion on release medium (dvd1.iso).

The script sources ${.CURDIR}/${TARGET}/pkg-stage.conf, which sets
several environment variables, such as the pkg(8) ABI, PACKAGESITE,
PKG_DBDIR, and PKG_CACHEDIR.  PKG_CACHEDIR is set to the directory
on the release medium used by bsdconfig(8) (/packages/${ABI}).  ABI
is determined by output of 'make -C /usr/src/release -V REVISION'.
See pkg.conf(5) for descripton on other variables set here.

The list of packages to include are set within the configuration
file.

The script and configuration files are intended to be run by the
'make dvd' target within the release directory, and assume the
release is built within a chroot environment (such as by using
release.sh).

Relevant updates to release/Makefile will follow.

Sponsored by:	The FreeBSD Foundation
2013-11-18 15:22:55 +00:00
Will Andrews
0ca8e70cb0 Fix 'make release' on older hosts: use buildworld legacy utilities.
Newer FreeBSD installs require an install(1) that supports the new flags.

This adds ${MAKEOBJDIRPREFIX}${.CURDIR}/tmp/legacy/{bin,usr/{bin,sbin}}
to the PATH while generating an mtree database for 'make release'.

Note that the problem only exists here because mm-mtree.sh generates
its own object tree to avoid mucking with the existing one, which
results in a PATH containing legacy utility dirs that are empty.
2013-08-27 04:42:42 +00:00
Nathan Whitehorn
e59e2d8ec7 Farewall, sysinstall! You served us well for many years, but 10.0 is one
digit beyond your time.

Various sysinstall dependencies (e.g. libftpio, libdisk, libodialog, etc.)
will be cleaned up in coming days. Some will take longer than others due to
a few other consumers (tzsetup and sade).
2011-10-03 15:13:09 +00:00
Nathan Whitehorn
b70047d413 Add generation of an installation manifest containing SHA256 checksums as
well as package descriptions and add code in the installer to check the
checksums.
2011-03-13 17:15:31 +00:00
Nathan Whitehorn
89033280b7 Add generation of memstick images to the bsdinstall release makefile for
i386 and amd64. This involved moving the memstick generation script to
the arch directories from scripts/, in analogy to mkisoimages.sh. This
script was never called from /usr/src/release/Makefile, so that hasn't
been updated.
2011-03-13 01:38:24 +00:00
Nathan Whitehorn
99ad39c822 Improve the distributeworld target in Makefile.inc1 and update the release
infrastructure to use it. make distributeworld can now be used without
preparing its environment first and installs games into its distribution
using the regular make distribute logic instead of post-processing with
a script.

Also add two new targets, packageworld and packagekernel, that tar up the
results of distributeworld and distributekernel (also new), respectively.
2011-03-02 14:39:26 +00:00
Brian Somers
bcc4111272 Remove an unnecessary cat.
PR:		145447
Submitted by:	u at netbeisser dot de
MFC after:	1 week
2010-06-19 09:33:11 +00:00
Ken Smith
c0f29c6e1d Provide a script that can be used to create the memstick images. For now
it isn't being integrated into 'make release' because for the forseeable
future the memstick images will be identical to what's on the DVD except
for which package set is provided.  If/when what's on the memstick diverges
from what's on the DVD it would make more sense to generate a "memstick"
directory in $CHROOT/R/cdrom and build the memstick image along with the
ISO images.

Reviewed by:	jhb, ru, Garrett Cooper (yanefbsd at gmail dot com)
2010-02-18 15:45:43 +00:00
Ken Smith
b34c18a7dc Update package list for 8.0-REL.
Reviewed by:	re@, portmgr@
MFC after:	1 day
2009-10-20 13:58:30 +00:00
Ken Smith
44a37a783b Bring the layout of package-split.py more in line with where we're going
with packages on the release media.  It looks like we'll be putting just
the doc packages on the new "memory stick" image as well as disc1.  There
will be no other packages on the CDROM-sized media.  The DVD sized media
will include the doc packages plus whatever other packages we decide to
make part of the release.

This commit just brings the basic structure in line with being able to
do this.  We still need to discuss with various people exactly which
packages will be included on the DVD.

If the environement variable "PKG_DVD" is set a tree suitable for the
DVD media is generated.  Otherwise a tree suitable for the "memory stick"
and disc1 is generated.

MFC after:	3 days
2009-09-07 17:54:20 +00:00
John Baldwin
7b0eef0b32 Fix a few issues with the lib32 dist so that it includes ldd32.
- Use a better find invocation to purge empty directories from all the dist
  trees during a release build.  The previous version did not purge
  directories whose contents were all empty directories.
- Explicitly blacklist a few files from the lib32 dist instead of using a
  whitelist.  A better longterm solution is to fix the few offenders to not
  install data files during a lib32 install.

MFC after:	3 days
2009-08-24 21:55:43 +00:00
Doug Barton
c61ff19489 Add a script to create the /var/db/mergemaster.mtree file for new
releases so that when users subsequently update their source trees
they can make use of mergemaster's -U option.
2009-08-23 05:42:50 +00:00
Marc Fonvieille
944bc81da9 - release/* update to use freebsd-doc-* packages instead of building
FreeBSD docset during 'make release' this will speed up release
  builds;
- sysinstall(8) has also been updated to use these packages with a new
  menu allowing people to choose what localized doc to install;
- mention in UPDATING that docs from the FreeBSD Documentation project
  are now installed in /usr/local/share/doc/freebsd instead of
  /usr/share/doc.

Approved by:	re (kensmith)
2009-06-28 08:59:46 +00:00
Yoshihiro Takahashi
60088513b4 Specify 4096 block-size and 512 fragment-size.
This fixes no space error.
2009-02-14 14:50:27 +00:00
Ken Smith
48a0e75812 Due to some recent-ish work on GEOM/G_PART the c partition (by tradition
the whole disk) isn't available any more since it was redundant.  Just
use /dev/md0 instead of /dev/md0c to build the filesystem on.

Consulted-with:		marcel
2009-02-10 15:07:57 +00:00
Ken Smith
ad005f6252 Fix minor typo. 2008-10-13 17:42:21 +00:00
Ruslan Ermilov
896eafd957 src/compat/ is gone back in March.
Reported by:	Mars G Miro
2008-06-25 20:29:22 +00:00
Ruslan Ermilov
8e3cedddca Teach source installation shell script and sysinstall(8) about
the 'scompat' source dist that holds src/compat/.

Reported by:	Mars G Miro
2007-11-09 08:50:02 +00:00
Kris Kennaway
56e4ceea65 Catch up to changes in the ports tree since last release 2007-11-07 09:11:45 +00:00
John Baldwin
05dd939ae6 Teach the 'src-install' script about the 'scddl' source dist that holds
/usr/src/cddl.

Approved by:	re (kensmith)
2007-06-28 18:26:35 +00:00
Ken Smith
8b33776a49 Shift to vim-lite package because vim package doesn't build at the moment.
MFC after:	3 days
2006-11-17 05:10:41 +00:00
Ken Smith
bcc506c582 Alpha no longer supported.
Noticed by:	ru@
2006-11-17 04:24:33 +00:00
Ken Smith
0dd900b35b Switch to emulators/linux_base-fc4 since that's the new default
Linux emulator.

MFC after:	3 days
2006-11-16 19:08:27 +00:00
Bruce A. Mah
d94e6ff183 Remove security/freebsd-update from package split; it's now in the
base system.

Pointed out by:	kris
2006-10-17 13:50:22 +00:00
Kris Kennaway
4ecca96d98 Catch up to xchat and samba package changes 2006-09-09 07:25:40 +00:00