Release the evil twin of nanobsd.sh: sysbuild.sh

quoth the README:

I have been running -current on my laptop since before FreeBSD 2.0 was
released and along the way developed this little trick to making the
task easier.

sysbuild.sh is a way to build a new FreeBSD system on a computer from
a specification, while leaving the current installation intact.

sysbuild.sh assume you have two partitions that can hold your rootfs
and can be booted, and roughly speaking, all it does is build a new
system into the one you don't use, from the one you do use.

A partition named /freebsd is assumed to be part of your layout, and
that is where the sources and ports will be found.

If you know how nanobsd works, you will find a lot of similarity.
This commit is contained in:
Poul-Henning Kamp 2009-01-17 20:30:06 +00:00
parent a1805f7bb9
commit 68d566e62e
2 changed files with 681 additions and 0 deletions

153
tools/tools/sysbuild/README Normal file
View File

@ -0,0 +1,153 @@
$FreeBSD$
About sysbuild.sh
=================
I have been running -current on my laptop since before FreeBSD 2.0 was
released and along the way developed this little trick to making the
task easier.
sysbuild.sh is a way to build a new FreeBSD system on a computer from
a specification, while leaving the current installation intact.
sysbuild.sh assume you have two partitions that can hold your rootfs
and can be booted, and roughly speaking, all it does is build a new
system into the one you don't use, from the one you do use.
A partition named /freebsd is assumed to be part of your layout, and
that is where the sources and ports will be found.
If you know how nanobsd works, you will find a lot of similarity.
HOWTO
=====
In all likelyhood, it is easier if we imagine you start with a blank
computer.
Grab a FreeBSD install ISO and boot it.
Create four disk slices:
ad0s1 = 5GB
ad0s2 = 5GB
ad0s3 = 5GB
ad0s4 = the rest
Create a root filesystem in s1a filling the entire ad0s1 slice.
Create a swap partition, if you want one, in ad0s4b.
Install the boot0 bootmanager.
Install the "Minimal" FreeBSD system into ad0s1a.
Reboot from the newly installed system.
Run these commands to set up the other partitions sysbuild.sh cares about:
# /freebsd filesystem
newfs -b 4096 -f 512 -O2 -U /dev/ad0s3
echo "/dev/ad0s3 /freebsd ufs rw 2 2" >> /etc/fstab
mkdir /freebsd
mount /freebsd
# deputy rootfilesystem
bsdlabel -B -w /dev/ad0s2
newfs -O2 -U /dev/ad0s2a
Next, install ports and sources:
cd /usr
rm -rf ports src
ln -s /freebsd/src
ln -s /freebsd/ports
cd /freebsd
mkdir ports src packages
# Or use svn if you prefer
csup -h cvsup.???.freebsd.org /usr/share/examples/cvsup/ports-supfile
csup -h cvsup.???.freebsd.org /usr/share/examples/cvsup/stable-supfile
And we should be ready to try a shot:
cd /root
cp /usr/src/tools/tools/sysbuild/sysbuild.sh
sh sysbuild.sh |& tee _.sb
If it succeeds, you should be able to:
boot0cfg -s 2 -v /dev/ad0
reboot
And come up with your newly built system.
Next time you want a new system, you just run sysbuild.sh again
and boot slice 1 when it's done.
TWEAKS
======
The sysbuild.sh script takes various parameters:
-c specfile # configure stuff, see below.
-w # skip buildworld, assume it was done earlier.
-k # skip buildkernel, ---//---
-b # skip both buildworld & buildkernel
-p # install cached packacges if found.
The specfile is a shellscript where you can override or set a number of
shell variables and functions.
A partial example:
# use a kernel different from GENERIC
KERNCONF=SMP
# Cache built packages, so we can use -p
PKG_DIR=/freebsd/packages
# Mount ports distfiles from another machine
REMOTEDISTFILES=fs:/rdonly/distfiles
# Fetch distfiles through a proxy
FTP_PROXY=http://127.0.0.1:3128/
HTTP_PROXY=http://127.0.0.1:3128/
export FTP_PROXY HTTP_PROXY
# We want these ports
PORTS_WE_WANT='
/usr/ports/archivers/unzip
/usr/ports/archivers/zip
/usr/ports/cad/linux-eagle
/usr/ports/comms/lrzsz
/usr/ports/databases/rrdtool
/usr/ports/devel/subversion-freebsd
'
# Files to move over
CONFIGFILES='
/root/.ssh
/etc/X11/xorg.conf
/etc/ssh/ssh_host*
/etc/rc.conf
/etc/rc.local
'
# Shell functions to tweak things
# (This makes commits to /etc mostly painless)
final_chroot() (
chpass -p "\$1\$IgMjWs2L\$Nu12OCsjfiwHHj0I7TmUN1" root
pw useradd phk -u 488 -d /home/phk -c "Poul-Henning Kamp" \
-G "wheel,operator,dialer" -s /bin/csh -w none
chpass -p "\$1\$VcM.9Ow8\$IcXHs0h9jsk27b8N64lOm/" phk
sed -i "" -e 's/^DS/DSorigo.freebsd.dk/' /etc/mail/sendmail.cf
sed -i "" -e '/console/s/^/#/' /etc/syslog.conf
echo "beastie_disable=YES" >> /boot/loader.conf
touch /root/.hushlogin
)

View File

@ -0,0 +1,528 @@
#!/bin/sh
#
# Copyright (c) 1994-2009 Poul-Henning Kamp.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
# $FreeBSD$
#
set -e
exec < /dev/null
if [ `uname -m` = "i386" ] ; then
TARGET_PART=`df / | sed '
1d
s/[ ].*//
s,/dev/,,
s,s1a,s3a,
s,s2a,s1a,
s,s3a,s2a,
'`
# Where our build-bits are to be found
FREEBSD_PART=`echo $TARGET_PART | sed 's/s[12]a/s3/'`
else
TARGET_PART=unknown
FREEBSD_PART=unknown
fi
# Relative to /freebsd
PORTS_PATH=ports
SRC_PATH=src
# OBJ_PATH=obj
# Name of kernel
KERNCONF=GENERIC
# srcconf
#SRCCONF="SRCCONF=/usr/src/src.conf"
# -j arg to make(1)
ncpu=`sysctl -n kern.smp.cpus`
if [ $ncpu -gt 1 ] ; then
JARG="-j $ncpu"
fi
# serial console ?
SERCONS=false
# Remotely mounted distfiles
# REMOTEDISTFILES=fs:/rdonly/distfiles
# Proxy
#FTP_PROXY=http://127.0.0.1:3128/
#HTTP_PROXY=http://127.0.0.1:3128/
#export FTP_PROXY HTTP_PROXY
PORTS_WE_WANT='
'
PORTS_OPTS="BATCH=YES MAKE_IDEA=YES A4=yes"
CONFIGFILES='
'
cleanup() (
)
before_ports() (
)
before_ports_chroot() (
)
final_root() (
)
final_chroot() (
)
#######################################################################
#######################################################################
usage () {
(
echo "Usage: $0 [-b/-k/-w] [-c config_file]"
echo " -b suppress builds (both kernel and world)"
echo " -k suppress buildkernel"
echo " -w suppress buildworld"
echo " -p used cached packages"
echo " -c specify config file"
) 1>&2
exit 2
}
#######################################################################
#######################################################################
if [ ! -f $0 ] ; then
echo "Must be able to access self ($0)" 1>&2
exit 1
fi
if grep -q 'Magic String: 0`0nQT40W%l,CX&' $0 ; then
true
else
echo "self ($0) does not contain magic string" 1>&2
exit 1
fi
#######################################################################
set -e
log_it() (
set +x
a="$*"
set `cat /tmp/_sb_log`
TX=`date +%s`
echo "$1 $TX" > /tmp/_sb_log
DT=`expr $TX - $1 || true`
DL=`expr $TX - $2 || true`
echo -n "### `date +%H:%M:%S`"
printf " ### %5d ### %5d ### %s\n" $DT $DL "$a"
)
#######################################################################
ports_recurse() (
set +x
for d
do
if [ ! -d $d ] ; then
echo "Missing port $d" 1>&2
exit 2
fi
if grep -q "^$d\$" /tmp/_.plist ; then
true
else
(
cd $d
ports_recurse `make -V _DEPEND_DIRS`
)
echo $d >> /tmp/_.plist
fi
done
)
ports_build() (
set +x
true > /tmp/_.plist
ports_recurse $PORTS_WE_WANT
# Now build & install them
for p in `cat /tmp/_.plist`
do
t=`echo $p | sed 's,/usr/ports/,,'`
pn=`cd $p && make package-name`
if [ "x${PKG_DIR}" != "x" -a -f ${PKG_DIR}/$pn.tbz ] ; then
if [ "x$use_pkg" = "x-p" ] ; then
log_it "install $p from ${PKG_DIR}/$pn.tbz"
pkg_add ${PKG_DIR}/$pn.tbz
fi
fi
i=`pkg_info -qO $t`
if [ -z "$i" ] ; then
log_it "build $p"
b=`echo $p | tr / _`
(
set -x
cd /usr/ports
cd $p
set +e
make clean
if make install ${PORTS_OPTS} ; then
if [ "x${PKG_DIR}" != "x" ] ; then
make package ${PORTS_OPTS}
mv *.tbz ${PKG_DIR}
fi
else
log_it FAIL build $p
fi
make clean
) > _.$b 2>&1 < /dev/null
date
fi
done
)
ports_prefetch() (
(
set +x
ports_recurse $PORTS_WE_WANT
# Now checksump/fetch them
for p in `cat /tmp/_.plist`
do
b=`echo $p | tr / _`
(
cd $p
if make checksum $PORTS_OPTS ; then
true
else
make distclean
make checksum $PORTS_OPTS || true
fi
) > /mnt/_.prefetch.$b 2>&1
done
)
)
#######################################################################
do_world=true
do_kernel=true
use_pkg=""
c_arg=""
set +e
args=`getopt bc:hkpw $*`
if [ $? -ne 0 ] ; then
usage
fi
set -e
set -- $args
for i
do
case "$i"
in
-b)
shift;
do_world=false
do_kernel=false
;;
-c)
c_arg=$2
if [ ! -f "$c_arg" ] ; then
echo "Cannot read $c_arg" 1>&2
usage
fi
. "$2"
shift
shift
;;
-h)
usage
;;
-k)
shift;
do_kernel=false
;;
-p)
shift;
use_pkg="-p"
;;
-w)
shift;
do_world=false
;;
--)
shift
break;
;;
esac
done
#######################################################################
if [ "x$1" = "xchroot_script" ] ; then
set +x
set -e
shift
before_ports_chroot
ports_build
exit 0
fi
if [ "x$1" = "xfinal_chroot" ] ; then
final_chroot
exit 0
fi
if [ $# -gt 0 ] ; then
echo "$0: Extraneous arguments supplied"
usage
fi
#######################################################################
T0=`date +%s`
echo $T0 $T0 > /tmp/_sb_log
log_it Unmount everything
(
( cleanup )
umount /freebsd/distfiles || true
umount /mnt/freebsd/distfiles || true
umount /dev/${FREEBSD_PART} || true
umount /mnt/freebsd || true
umount /mnt/dev || true
umount /mnt || true
umount /dev/${TARGET_PART} || true
) # > /dev/null 2>&1
log_it Prepare running image
mkdir -p /freebsd
mount /dev/${FREEBSD_PART} /freebsd
#######################################################################
if [ ! -d /freebsd/${PORTS_PATH} ] ; then
echo PORTS_PATH does not exist 1>&2
exit 1
fi
if [ ! -d /freebsd/${SRC_PATH} ] ; then
echo SRC_PATH does not exist 1>&2
exit 1
fi
log_it TARGET_PART $TARGET_PART
sleep 5
rm -rf /usr/ports
ln -s /freebsd/${PORTS_PATH} /usr/ports
rm -rf /usr/src
ln -s /freebsd/${SRC_PATH} /usr/src
if $do_world ; then
if [ "x${OBJ_PATH}" != "x" ] ; then
rm -rf /usr/obj
mkdir -p /freebsd/${OBJ_PATH}
ln -s /freebsd/${OBJ_PATH} /usr/obj
else
rm -rf /usr/obj
mkdir -p /usr/obj
fi
fi
#######################################################################
for i in ${PORTS_WE_WANT}
do
if [ ! -d $i ] ; then
echo "Port $i not found" 1>&2
exit 2
fi
done
export PORTS_WE_WANT
export PORTS_OPTS
#######################################################################
log_it Prepare destination partition
newfs -O2 -U /dev/${TARGET_PART} > /dev/null
mount /dev/${TARGET_PART} /mnt
mkdir -p /mnt/dev
mount -t devfs devfs /mnt/dev
if [ "x${REMOTEDISTFILES}" != "x" ] ; then
rm -rf /freebsd/${PORTS_PATH}/distfiles
ln -s /freebsd/distfiles /freebsd/${PORTS_PATH}/distfiles
mkdir -p /freebsd/distfiles
mount ${REMOTEDISTFILES} /freebsd/distfiles
fi
log_it "Start prefetch of ports distfiles"
ports_prefetch &
if $do_world ; then
(
cd /usr/src
log_it "Buildworld"
make ${JARG} -s buildworld ${SRCCONF} > /mnt/_.bw 2>&1
)
fi
if $do_kernel ; then
(
cd /usr/src
log_it "Buildkernel"
make ${JARG} -s buildkernel KERNCONF=$KERNCONF > /mnt/_.bk 2>&1
)
fi
log_it Installworld
(cd /usr/src && make ${JARG} installworld DESTDIR=/mnt ${SRCCONF} ) \
> /mnt/_.iw 2>&1
log_it distribution
(cd /usr/src/etc && make distribution DESTDIR=/mnt ${SRCCONF} ) \
> /mnt/_.dist 2>&1
log_it Installkernel
(cd /usr/src && make ${JARG} installkernel DESTDIR=/mnt KERNCONF=$KERNCONF ) \
> /mnt/_.ik 2>&1
if [ "x${OBJ_PATH}" != "x" ] ; then
rmdir /mnt/usr/obj
ln -s /freebsd/${OBJ_PATH} /mnt/usr/obj
fi
log_it Wait for ports prefetch
wait
log_it Move filesystems
if [ "x${REMOTEDISTFILES}" != "x" ] ; then
umount /freebsd/distfiles
fi
umount /dev/${FREEBSD_PART} || true
mkdir -p /mnt/freebsd
mount /dev/${FREEBSD_PART} /mnt/freebsd
if [ "x${REMOTEDISTFILES}" != "x" ] ; then
mount ${REMOTEDISTFILES} /mnt/freebsd/distfiles
fi
rm -rf /mnt/usr/ports || true
ln -s /freebsd/${PORTS_PATH} /mnt/usr/ports
rm -rf /mnt/usr/src || true
ln -s /freebsd/${SRC_PATH} /mnt/usr/src
log_it Build and install ports
# Make sure fetching will work in the chroot
if [ -f /etc/resolv.conf ] ; then
log_it copy resolv.conf
cp /etc/resolv.conf /mnt/etc
chflags schg /mnt/etc/resolv.conf
fi
if [ -f /etc/localtime ] ; then
log_it copy localtime
cp /etc/localtime /mnt/etc
fi
log_it copy ports config files
(cd / ; find var/db/ports -print | cpio -dumpv /mnt )
log_it ldconfig in chroot
chroot /mnt sh /etc/rc.d/ldconfig start
log_it before_ports
(
before_ports
)
log_it build ports
pwd
cp $0 /mnt/root
cp /tmp/_sb_log /mnt/tmp
b=`basename $0`
if [ "x$c_arg" != "x" ] ; then
cp $c_arg /mnt/root
chroot /mnt sh /root/$0 -c /root/`basename $c_arg` $use_pkg chroot_script
else
chroot /mnt sh /root/$0 $use_pkg chroot_script
fi
cp /mnt/tmp/_sb_log /tmp
log_it fixing fstab
sed "/[ ]\/[ ]/s;^[^ ]*[ ];/dev/${TARGET_PART} ;" \
/etc/fstab > /mnt/etc/fstab
log_it create all mountpoints
grep -v '^[ ]*#' /mnt/etc/fstab |
while read a b c
do
mkdir -p /mnt/$b
done
if [ "x$SERCONS" != "xfalse" ] ; then
log_it serial console
echo " -h" > /mnt/boot.config
sed -i "" -e /ttyd0/s/off/on/ /mnt/etc/ttys
sed -i "" -e /ttyu0/s/off/on/ /mnt/etc/ttys
sed -i "" -e '/^ttyv[0-8]/s/ on/ off/' /mnt/etc/ttys
fi
log_it move config files
(cd / && find ${CONFIGFILES} -print | cpio -dumpv /mnt)
log_it final_root
( final_root )
log_it final_chroot
cp /tmp/_sb_log /mnt/tmp
if [ "x$c_arg" != "x" ] ; then
chroot /mnt sh /root/$0 -c /root/`basename $c_arg` final_chroot
else
chroot /mnt sh /root/$0 final_chroot
fi
cp /mnt/tmp/_sb_log /tmp
log_it "Check these messages (if any):"
grep '^Stop' /mnt/_* || true
log_it DONE