464 Commits

Author SHA1 Message Date
Glen Barber
2303632301 Switch to x11/gnome3 now that x11/gnome2 no
longer exists in the ports tree.

MFC after:	3 days
X-MFC-With:	r276820, r276822, r276827
Sponsored by:	The FreeBSD Foundation
2015-01-08 15:50:10 +00:00
Glen Barber
0669938973 Ensure the ports directories exist for the list of
packages intended to be included on the DVD, and
remove any nonexistent ports from the final list.

Print the list of missing paths, and ensure that
DVD_PACKAGES is non-zero length (which should never
happen).

MFC after:	3 days
X-MFC-With:	r276820, r276822
Sponsored by:	The FreeBSD Foundation
2015-01-08 15:42:10 +00:00
Glen Barber
6b7ca552c3 Ensure /var/run/ld-elf.so.hints exists before building
pkg(8).

MFC after:	3 days
X-MFC-With:	r276820
Sponsored by:	The FreeBSD Foundation
2015-01-08 14:39:53 +00:00
Glen Barber
ef6db57978 Print informational output when NOPORTS is set,
which would otherwise cause pkg-stage.sh to
silently exit.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-01-08 14:29:03 +00:00
Glen Barber
83373daa35 Properly change directories before creating the
PKG_ALTABI -> PKG_ABI symlink.

MFC after:	3 days
X-MFC-With:	r276765
Sponsored by:	The FreeBSD Foundation
2015-01-06 20:46:29 +00:00
Glen Barber
8f85f8e66e Update pkg-stage.sh to be compatible with pkg-1.4.x.
In 1.3.x and earlier, ABI format is, for example,
freebsd:11:x86:64.

In 1.4.x, ABI format is FreeBSD:11:amd64, and a new
configuration entry, ALTABI, is: freebsd:11:x86:64.

Export PKG_ABI and PKG_ALTABI accordingly, and if
PKG_ALTABI is set, create a symlink within the dvd
PKG_CACHEDIR so both new and old ABI directories
exist.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-01-06 20:42:50 +00:00
Glen Barber
4c16f1b111 Adjust the 'Relnotes:' search to look for any non-empty
string following the tag.

Sponsored by:	The FreeBSD Foundation
2014-12-08 15:45:07 +00:00
Colin Percival
3b4252413d Unmount filesystem and destroy md before we read the vnode from disk and
package it into a disk image.  Otherwise we end up packaging an unclean
filesystem.
2014-11-20 00:16:55 +00:00
Glen Barber
742aade462 Call cleanup() after everything is done.
Sponsored by:	The FreeBSD Foundation
2014-11-08 16:52:07 +00:00
Glen Barber
07fe767eb8 Spell 'OPTARG' correctly.
Actually call vm_create_base().

Sponsored by:	The FreeBSD Foundation
2014-11-08 12:59:32 +00:00
Glen Barber
144d107fcd Move usage() from vmimage.subr to mk-vmimage.sh,
in case vmimage.subr has not been sourced.

Sponsored by:	The FreeBSD Foundation
2014-11-08 12:47:21 +00:00
Glen Barber
87f6c4ac27 Return if vm_create_disk() is unsuccessful.
Sponsored by:	The FreeBSD Foundation
2014-11-08 12:23:50 +00:00
Glen Barber
1e7c1f1742 Initial rewrite to consolidate VM image build scripts into one.
There may be some very sharp edges here while refactoring.

 - Move amd64/mk-vmimage.sh -> scripts/mk-vmimage.sh.
 - Remove vm-base target from Makefile.vm.
 - In vm-image target, use getopts flags for argument passing.
 - Create tools/vmimage.subr, containing default and prototype
   for the following functions that are used to drive the build,
   run in this order:

   vm_install_base()
   vm_extra_install_base()
   vm_extra_install_packages()
   vm_extra_install_ports()
   vm_extra_enable_services()
   vm_extra_pre_umount()
   vm_create_disk()
   vm_extra_create_disk()

 - In tools/azure.conf, override:

   vm_extra_install_base()
   vm_extra_pre_umount()
   vm_extra_create_disk()

 - In tools/openstack.conf, override:

   vm_extra_install_base()
   vm_extra_pre_umount()

Sponsored by:	The FreeBSD Foundation
2014-11-05 13:22:19 +00:00
Glen Barber
fadf2a24fe Separate release/scripts/mk-vmimage.sh to machine-specific
scripts, making it possible to mimic the functionality for
non-x86 targets.

Move echo output if MAKEFLAGS is empty outside of usage().

Remove TARGET/TARGET_ARCH evaluation.

Sponsored by:	The FreeBSD Foundation
2014-10-01 17:05:40 +00:00
Glen Barber
f6490fe596 Comment cleanup in panic() message when mkimg(1) does not support
the requested disk image format.

Sponsored by:	The FreeBSD Foundation
2014-09-29 01:17:42 +00:00
Glen Barber
1e2f9a549b Remove the UFS label from the root filesystem since it is added
by mkimg(1) as a gpt label, consistent with the fstab(5) entry.

Sponsored by:	The FreeBSD Foundation
2014-09-28 23:22:55 +00:00
Glen Barber
5baa1b2730 Remove comments left in accidentally while testing, so the
VM /etc/fstab is actually created.

Sponsored by:	The FreeBSD Foundation
2014-09-28 21:15:30 +00:00
Glen Barber
79a94b12ae Fix UFS label for the root filesystem.
Sponsored by:	The FreeBSD Foundation
2014-09-28 21:13:30 +00:00
Glen Barber
f54137c331 release/Makefile:
Include .OBJDIR in DESTDIR in the vm-base target.

release/release.sh:
  Provide the full path to mddev.

Sponsored by:	The FreeBSD Foundation
2014-09-28 20:06:02 +00:00
Glen Barber
8234f86da8 Remove a 'set -x' that snuck in during testing.
Sponsored by:	The FreeBSD Foundation
2014-09-28 17:16:45 +00:00
Glen Barber
a68d274b86 Initial commit to include virtual machine images as part
of the FreeBSD release builds.

This adds a make(1) environment variable requirement,
WITH_VMIMAGES, which triggers the virtual machine image
targets when not defined to an empty value.

Relevant user-driven variables include:

 o VMFORMATS:	The virtual machine image formats to create.
   Valid formats are provided by running 'mkimg --formats'

 o VMSIZE:	The size of the resulting virtual machine
   image.  Typical compression is roughly 140Mb, regardless
   of the target size (10GB, 15GB, 20GB, 40GB sizes have been
   tested with the same result).

 o VMBASE:	The prefix of the virtual machine disk images.
   The VMBASE make(1) environment variable is suffixed with
   each format in VMFORMATS for each individual disk image, as
   well as '.img' for the source UFS filesystem passed to
   mkimg(1).

This also includes a new script, mk-vmimage.sh, based on how
the VM images for 10.0-RELEASE, 9.3-RELEASE, and 10.1-RELEASE
were created (mk-vmimage.sh in ^/user/gjb/thermite/).

With the order in which the stages need to occur, as well as
sanity-checking error cases, it makes much more sense to
execute a shell script called from make(1), using env(1) to
set specific parameters for the target image than it does to
do this in make(1) directly.

Sponsored by:	The FreeBSD Foundation
2014-09-28 01:45:20 +00:00
Glen Barber
83f0a683f4 When populating the dvd/packages/ directory, create
a symlink to All/pkg-*.txz in the Latest/ directory.

This allows 'pkg bootstrap' to work out-of-box if
the REPOS_DIR environment is properly set.

Tested on:	stable/10@r271848
MFC after:	3 days
X-MFC-10.1:	yes
Sponsored by:	The FreeBSD Foundation
2014-09-19 17:23:44 +00:00
Glen Barber
53bdd39b8a Simplify dvd package population with pkg-1.3.
Submitted by:	bdrewery
MFC after:	3 days
X-MFC-With:	r271480, r271483
Sponsored by:	The FreeBSD Foundation
2014-09-12 22:20:07 +00:00
Glen Barber
b60a2f70da Fix duplicate PKG_ABI in the PKG_CACHEDIR path.
MFC after:	3 days
X-MFC-With:	r271480
Sponsored by:	The FreeBSD Foundation
2014-09-12 20:16:55 +00:00
Glen Barber
017717d87f Set PKG_CACHEDIR to an 'All/' directory one level lower
to fix 'pkg repo' generating repository metadata for the
on-disc packages.

MFC after:	3 days
X-MFC-to-10.1:	yes
Sponsored by:	The FreeBSD Foundation
2014-09-12 18:52:01 +00:00
Gavin Atkinson
ac1b139605 Remove an nunnecessary reference to csup. 2014-06-25 12:17:00 +00:00
Glen Barber
540a8eac2f Disable the main FreeBSD pkg(7) repositories in the dvd
repository configuration to avoid fetching from upstream
in case there may be conflicts.

MFC After:	3 days
Sponsored by:	The FreeBSD Foundation
2014-05-22 19:25:41 +00:00
Glen Barber
f0d60b2a4e Add forward-compatibility glue with pkg-1.3:
- Use ASSUME_ALWAYS_YES=YES instead of ASSUME_ALWAYS_YES=1
   since pkg-1.3 expects "yes" or "true" values.
 - Before exporting PKG_ABI, strip extra characters from what
   is parsed from 'pkg -vv'.  This causes problems further down
   when creating the packages directory for inclusion on the
   dvd1.iso.  Previously PKG_ABI would be 'freebsd:9:x86:64',
   but now is '"freebsd:9:x86:64";' in pkg-1.3

Tested on:	stable/9@r265858 with ports-mgmt/pkg-devel
MFC After:	3 days
Sponsored by:	The FreeBSD Foundation
2014-05-22 19:22:03 +00:00
Glen Barber
4c9e857efa Use 'return' instead of 'exit' in main(), since the script exits
with the last returned exit code.

Sponsored by:	The FreeBSD Foundation
2014-05-05 23:36:46 +00:00
Glen Barber
aa8426befc Add a helper script for parsing 'svn log' for the
'Relnotes:' tag in case 'yes' is not explicitly
the first string value following the tab.

As it turns out, a number of commits have bypassed
my filters (both email and 'svn log --search'), and
this script returns the results I want when doing
these searches.

Sponsored by:	The FreeBSD Foundation
2014-05-05 23:01:01 +00:00
Craig Rodrigues
82cac3982b Add a script which lists SVN log entries only for changesets
which have files which have been Added or Removed.

This can help identify changesets which should be
mentioned in Release Engineering's release notes.
2014-02-28 23:12:20 +00:00
Glen Barber
6f50c4ea71 Update the pkg-stage target to be more compatible with pkg-1.2:
- Add a release-dvd.conf pkg(8) configuration file to override
   the default FreeBSD.conf configuration.
 - Remove architecture-specific pkg-stage.conf files, consolidate,
   and move their contents to scripts/pkg-stage.sh.
 - Use 'pkg -vv' to determine the ABI, which is used as the
   cache directory.

Prior to these changes, it would be possible for pkg-stage to fetch
conflicting binary packages from multiple repositories.

Tested against:	head@r260522, stable/10@r260522

MFC after:	3 days
X-Insta-MFC:	possibly
Sponsored by:	The FreeBSD Foundation
2014-01-16 16:12:09 +00:00
Glen Barber
0d0e89b794 Add a pkg(8) repository configuration file for cdrom-based package
installation.

As part of the 'pkg-stage' target, copy the configuration file
to the 'packages/repos/' directory on the DVD filesystem.

MFC after:	3 days
X-MFC-Before:	-RC3
Sponsored by:	The FreeBSD Foundation
2013-12-15 20:38:07 +00:00
Glen Barber
89df2158c5 Prevent release build errors found during snapshot builds where if
NOPORTS=1, pkg-stage.sh cannot build the ports-mgmt/pkg port if
WITH_DVD=1.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2013-12-12 05:49:27 +00:00
Glen Barber
4e11928ded Provide reproducibility between builds by building pkg(8) from ports,
instead of using pkg-bootstrap.

This should resolve a problem that was discovered during 10.0-BETA4
freebsd-update(8) builds,

Reported by:	secteam (delphij)
MFC after:	3 days
X-MFC-to-10:	possibly immediately, pending freebsd-update(8) builds
Sponsored by:	The FreeBSD Foundation
2013-12-02 14:28:30 +00:00
Glen Barber
042a51f641 Add a script and configuration files to fetch pre-built packages
from pkg.FreeBSD.org for inclusion on release medium (dvd1.iso).

The script sources ${.CURDIR}/${TARGET}/pkg-stage.conf, which sets
several environment variables, such as the pkg(8) ABI, PACKAGESITE,
PKG_DBDIR, and PKG_CACHEDIR.  PKG_CACHEDIR is set to the directory
on the release medium used by bsdconfig(8) (/packages/${ABI}).  ABI
is determined by output of 'make -C /usr/src/release -V REVISION'.
See pkg.conf(5) for descripton on other variables set here.

The list of packages to include are set within the configuration
file.

The script and configuration files are intended to be run by the
'make dvd' target within the release directory, and assume the
release is built within a chroot environment (such as by using
release.sh).

Relevant updates to release/Makefile will follow.

Sponsored by:	The FreeBSD Foundation
2013-11-18 15:22:55 +00:00
Will Andrews
0ca8e70cb0 Fix 'make release' on older hosts: use buildworld legacy utilities.
Newer FreeBSD installs require an install(1) that supports the new flags.

This adds ${MAKEOBJDIRPREFIX}${.CURDIR}/tmp/legacy/{bin,usr/{bin,sbin}}
to the PATH while generating an mtree database for 'make release'.

Note that the problem only exists here because mm-mtree.sh generates
its own object tree to avoid mucking with the existing one, which
results in a PATH containing legacy utility dirs that are empty.
2013-08-27 04:42:42 +00:00
Nathan Whitehorn
e59e2d8ec7 Farewall, sysinstall! You served us well for many years, but 10.0 is one
digit beyond your time.

Various sysinstall dependencies (e.g. libftpio, libdisk, libodialog, etc.)
will be cleaned up in coming days. Some will take longer than others due to
a few other consumers (tzsetup and sade).
2011-10-03 15:13:09 +00:00
Nathan Whitehorn
b70047d413 Add generation of an installation manifest containing SHA256 checksums as
well as package descriptions and add code in the installer to check the
checksums.
2011-03-13 17:15:31 +00:00
Nathan Whitehorn
89033280b7 Add generation of memstick images to the bsdinstall release makefile for
i386 and amd64. This involved moving the memstick generation script to
the arch directories from scripts/, in analogy to mkisoimages.sh. This
script was never called from /usr/src/release/Makefile, so that hasn't
been updated.
2011-03-13 01:38:24 +00:00
Nathan Whitehorn
99ad39c822 Improve the distributeworld target in Makefile.inc1 and update the release
infrastructure to use it. make distributeworld can now be used without
preparing its environment first and installs games into its distribution
using the regular make distribute logic instead of post-processing with
a script.

Also add two new targets, packageworld and packagekernel, that tar up the
results of distributeworld and distributekernel (also new), respectively.
2011-03-02 14:39:26 +00:00
Brian Somers
bcc4111272 Remove an unnecessary cat.
PR:		145447
Submitted by:	u at netbeisser dot de
MFC after:	1 week
2010-06-19 09:33:11 +00:00
Ken Smith
c0f29c6e1d Provide a script that can be used to create the memstick images. For now
it isn't being integrated into 'make release' because for the forseeable
future the memstick images will be identical to what's on the DVD except
for which package set is provided.  If/when what's on the memstick diverges
from what's on the DVD it would make more sense to generate a "memstick"
directory in $CHROOT/R/cdrom and build the memstick image along with the
ISO images.

Reviewed by:	jhb, ru, Garrett Cooper (yanefbsd at gmail dot com)
2010-02-18 15:45:43 +00:00
Ken Smith
b34c18a7dc Update package list for 8.0-REL.
Reviewed by:	re@, portmgr@
MFC after:	1 day
2009-10-20 13:58:30 +00:00
Ken Smith
44a37a783b Bring the layout of package-split.py more in line with where we're going
with packages on the release media.  It looks like we'll be putting just
the doc packages on the new "memory stick" image as well as disc1.  There
will be no other packages on the CDROM-sized media.  The DVD sized media
will include the doc packages plus whatever other packages we decide to
make part of the release.

This commit just brings the basic structure in line with being able to
do this.  We still need to discuss with various people exactly which
packages will be included on the DVD.

If the environement variable "PKG_DVD" is set a tree suitable for the
DVD media is generated.  Otherwise a tree suitable for the "memory stick"
and disc1 is generated.

MFC after:	3 days
2009-09-07 17:54:20 +00:00
John Baldwin
7b0eef0b32 Fix a few issues with the lib32 dist so that it includes ldd32.
- Use a better find invocation to purge empty directories from all the dist
  trees during a release build.  The previous version did not purge
  directories whose contents were all empty directories.
- Explicitly blacklist a few files from the lib32 dist instead of using a
  whitelist.  A better longterm solution is to fix the few offenders to not
  install data files during a lib32 install.

MFC after:	3 days
2009-08-24 21:55:43 +00:00
Doug Barton
c61ff19489 Add a script to create the /var/db/mergemaster.mtree file for new
releases so that when users subsequently update their source trees
they can make use of mergemaster's -U option.
2009-08-23 05:42:50 +00:00
Marc Fonvieille
944bc81da9 - release/* update to use freebsd-doc-* packages instead of building
FreeBSD docset during 'make release' this will speed up release
  builds;
- sysinstall(8) has also been updated to use these packages with a new
  menu allowing people to choose what localized doc to install;
- mention in UPDATING that docs from the FreeBSD Documentation project
  are now installed in /usr/local/share/doc/freebsd instead of
  /usr/share/doc.

Approved by:	re (kensmith)
2009-06-28 08:59:46 +00:00
Yoshihiro Takahashi
60088513b4 Specify 4096 block-size and 512 fragment-size.
This fixes no space error.
2009-02-14 14:50:27 +00:00
Ken Smith
48a0e75812 Due to some recent-ish work on GEOM/G_PART the c partition (by tradition
the whole disk) isn't available any more since it was redundant.  Just
use /dev/md0 instead of /dev/md0c to build the filesystem on.

Consulted-with:		marcel
2009-02-10 15:07:57 +00:00