The &os; Release Engineering Team $FreeBSD$ 2002 2003 2004 The &os; Release Engineering Team &tm-attrib.freebsd; &tm-attrib.intel; &tm-attrib.microsoft; &tm-attrib.sparc; &tm-attrib.general; This article describes the status of &os; &release.current;, from the standpoint of users who may be new to the &release.5x; series of releases or to &os; in general. It presents some background information on release engineering, some highlights of new features, and some possible drawbacks that might be faced by early adopters. It also contains some of the future release engineering plans for the 4-STABLE development branch and some tips on upgrading existing systems. &os; &release.5x; marks the first new major version of &os; in over two years. Besides a number of new features, it also contains a number of major developments in the underlying system architecture. Along with these advances, however, comes a system that incorporates a tremendous amount of new and not-widely-tested code. Compared to the existing line of &release.4x; releases, the first few &release.5x; releases may have regressions in areas of stability, performance, and occasionally functionality. For these reasons, the &a.re; specifically discourages users from updating from older &os; releases to &release.current; unless they are aware of (and prepared to deal with) possible regressions in the newer releases. Specifically, for more conservative users, we recommend running &release.4x; releases (such as &release.4last;) for the near-term future. We feel that such users are probably best served by upgrading to &release.5x; only after a 5-STABLE development branch has been created; this may be around the time of &release.5branchpoint;. (&os; &release.5x; suffers from what has been described as a chicken and egg problem. The entire project has a goal of producing releases that are as stable and reliable as possible. This stability and reliability requires widespread testing, particularly of the system's newer features. However, getting a large number of users to test the system, in a practical sense, means building and distributing a release first!) This article describes some of the issues involved in installing and running &os; &release.current;. We begin with a brief overview of the &os; release process. We then present some of the more noteworthy new features in &os; &release.current;, along with some areas that may prove troublesome for unwary users. For those users choosing to remain with 4-STABLE-based releases, we give some of the short- to medium-term plans for this development branch. Finally, we present some notes on upgrading existing &release.4x; systems to &release.current;. &os; employs a model of development that relies on multiple development branches within the source code repository. The main branch is called CURRENT, and is referred to in the CVS repository with the HEAD tag. New features are committed first to this branch; although this means that CURRENT is the first to see new functionality, it also means that it occasionally suffers from breakages as new features are added and debugged. Most &os; releases are made from one of several STABLE branches. Features are only added to these branches after some amount of testing in CURRENT. At the moment, only one STABLE branch is under active development; this branch is referred to as 4-STABLE, and all of the &os; &release.4x; releases were based on it. This branch has the tag RELENG_4 in the CVS repository. &os; 5.0, 5.1, and 5.2 are based on the CURRENT branch. The first of these releases was made after over two years of development (prior to these, the last release from HEAD was &os; 4.0, in March 2000). At some point after the release of &os; 5.0, a 5-STABLE branch will be created in the &os; CVS repository with the branch tag RELENG_5. The past two stable branches (3-STABLE and 4-STABLE) were created immediately after their respective dot-oh releases (3.0 and 4.0, respectively). In hindsight, this practice did not give sufficient time for either CURRENT to stabilize before the new branches were created. This in turn resulted in wasted effort porting bug fixes between branches, as well as some architectural changes that could not be ported between branches at all. Therefore, the release engineering team will only create the 5-STABLE branch in the CVS repository after we have found a relatively stable state to use as its basis. It is likely that there will be multiple releases in the &release.5x; series before this happens; we estimate that the 5-STABLE branch will be created around the time of &release.5branchpoint;. More information on &os; release engineering processes can be found on the Release Engineering Web pages and in the &os; Release Engineering article. Specific issues for the upcoming 5-STABLE development branch can be found in The Roadmap for 5-STABLE. A large attraction of &os; &release.5x; is a number of new features. These new features and functionality generally involve large architectural changes that were not feasible to port back to the &os; 4-STABLE development branch. (By contrast, many self-contained enhancements, such as new device drivers or userland utilities, have already been ported.) A brief, but not exhaustive list includes: SMPng: The next generation support for SMP machines (work in progress). Ongoing work aims to perform fine-grained locking of various kernel subsystems to increase the number of threads of execution that can be running in the kernel. More information can be found on the FreeBSD SMP Project page. KSE: Kernel Scheduled Entities allow a single process to have multiple kernel-level threads, similar to Scheduler Activations. The libkse and libthr threading libraries make this feature available to multi-threaded userland programs, using the &man.pthread.3; API. New architectures: Support for the sparc64, ia64, and amd64 architectures, in addition to the i386, pc98, and alpha. GCC: The compiler toolchain is now based on GCC 3.3.X, rather than GCC 2.95.X. MAC: Support for extensible, loadable Mandatory Access Control policies. GEOM: A flexible framework for transformations of disk I/O requests. The GBDE experimental disk encryption facility has been developed based on GEOM. FFS: The FFS filesystem now supports background &man.fsck.8; operations (for faster crash recovery) and filesystem snapshots. UFS2: A new UFS2 on-disk format has been added, which supports extended per-file attributes and larger file sizes. UFS2 is now the default format for &man.newfs.8;. On all platforms except for pc98, filesystems created from within &man.sysinstall.8; will use UFS2 by default. Cardbus: Support for Cardbus devices. Bluetooth: Support for Bluetooth devices. A more comprehensive list of new features can be found in the release notes for the various &os; &release.5x; releases. Along with the new features of &os; &release.5x; come some areas that can cause problems, or at least can lead to unexpected behavior. Generally, these come from the fact that a number of features are works-in-progress. A partial list of these areas of difficulty includes: A number of features are not yet finished. Examples from the feature list above include SMPng and KSE. While suitable for testing and experimentation, these features may not be ready for production use. Because of changes in kernel data structures and ABIs/APIs, third-party binary device drivers will require modifications to work correctly under &os; 5.0. There is a possibility of more minor ABI/API changes before the 5-STABLE branch is created, particularly on newer machine architectures. In some (hopefully rare) cases, user-visible structures may change, requiring recompiling of applications or reinstallation of ports/packages. Several parts of &os;'s base system functionality have been moved to the Ports Collection. Notable examples include Perl, UUCP, and most (but not all) games. While these programs are still supported, their removal from the base system may cause some confusion. Some parts of the &os; base system have fallen into a state of disrepair due to a lack of users and maintainers. These have been removed. Specific examples include the generation of a.out-style executables, XNS networking support, and the X-10 controller driver. A number of ports and packages do not build or do not run correctly under &os; &release.5x;, whereas they did under &os; 4-STABLE. Generally these problems are caused by compiler toolchain changes or cleanups of header files. In some cases they are caused by changes in kernel or device support. Many &os; &release.5x; features are seeing wide exposure for the first time. Many of these features (such as SMPng) have broad impacts on the kernel, and it may be difficult to gauge their effects on stability and performance. A certain amount of debugging and diagnostic code is still in place to help track down problems in &os; &release.5x;'s new features. This may cause &os; &release.5x; to perform more slowly than 4-STABLE. Features are only added to the 4-STABLE development branch after a settling time in -CURRENT. &os; &release.5x; does not have the stabilizing influence of a -STABLE branch. (It is likely that the 5-STABLE development branch will be created sometime after &release.5branchpoint;.) Documentation (such as the &os; Handbook and FAQ) may not reflect changes recently made to &os; &release.5x;. Because a number of these drawbacks affect system stability, the release engineering team recommends that more conservative sites and users stick to releases based on the 4-STABLE branch until the &release.5x; series is more polished. While we believe that many initial problems with stability have been fixed, some issues with performance are still being addressed by works-in-progress. We also note that best common practices in system administration call for trying operating system upgrades in a test environment before upgrading one's production, or mission-critical systems. It is important to note that even though releases are being made in the &release.5x; series, support for &release.4x; releases will continue for some time. Indeed, &os; 4.8 was released two months after 5.0, in April 2003, followed by 4.9, in October 2003, and 4.10 in May 2004. Future releases from the 4-STABLE branch (if any) will depend on several factors. The most important of these is the existence and stability of the 5-STABLE branch. If CURRENT is not sufficiently stable to allow the creation of a 5-STABLE branch, this may require and permit more releases from the 4-STABLE branch. Until the last declared release on the 4-STABLE branch, new features may be merged from HEAD at the discretion of developers, subject to existing release engineering policies. To some extent, the release engineering team (as well as the developer community as a whole) will take into account user demand for future 4-STABLE releases. This demand, however, will need to be balanced with release engineering resources (particularly developers' time, computing resources, and mirror archive space). We note that in general, the &os; community (both users and developers) has shown a preference for moving forward with new features in the &release.5x; branch and beyond, due to the difficulty involved in backporting (and maintaining) new functionality in &release.4x;. The &a.security-officer; will continue to support releases made from the 4-STABLE branch in accordance with their published policies, which can be found on the Security page on the &os; web site. Generally, the two most recent releases from any branch will be supported with respect to security advisories and security fixes. At its discretion, the team may support other releases for specific issues. At this point, the release engineering team has no specific plans for future releases from the 4-STABLE development branch, and we decided to maintain 4.10-RELEASE as Errata Branch. Previously a branch where a release is made has been maintained as Security Branch by the &a.security-officer; for a certain period, and they would only have critical security fixes applied. With FreeBSD 4.10-RELEASE the scope of fixes will be expanded to include local Denial of Service fixes as well as other significant and well-tested fixes that may not represent security issues. While those fixes will not likely to be published as a release from the branch, the &os; Errata Notice will cover them in the same manner as the &os; Security Advisory. For those users with existing &os; systems, this section offers a few notes on upgrading a &os; &release.4x; system to &release.5x;. As with any &os; upgrade, it is crucial to read the release notes and the errata for the version in question, as well as src/UPDATING in the case of source upgrades. Probably the most straightforward approach is that of backup everything, reformat, reinstall, and restore everything. This eliminates problems of incompatible or obsolete executables or configuration files polluting the new system. It allows new filesystems to be created to take advantage of new functionality (most notably, the UFS2 defaults). As of this time, the binary upgrade option in &man.sysinstall.8; has not been well-tested for cross-major-version upgrades. Using this feature is not recommended. In particular, a binary upgrade will leave behind a number of files that are present in &os; &release.4x; but not in &release.5x;. These obsolete files may create some problems. Examples of these files include old C++ headers, programs moved to the Ports Collection, or shared libraries that have moved to support dynamically-linked root filesystem executables. On the &i386; and pc98 platforms, a UserConfig utility exists on 4-STABLE to allow boot-time configuration of ISA devices when booting from installation media. Under &os; &release.5x;, this functionality has been replaced in part by the &man.device.hints.5; mechanism (it allows specifying the same parameters, but with a very different interface). Floppy-based binary installations may require downloading a third, new floppy image holding additional device drivers in kernel modules. This drivers.flp floppy image will generally be found in the same location as the usual kern.flp and mfsroot.flp floppy images. CDROM-based installations on the &i386; architecture now use a no-emulation boot loader. This allows, among other things, the use of a GENERIC kernel, rather than the stripped-down kernel on the floppy images. In theory, any system capable of booting the µsoft; &windowsnt; 4 installation CDROMs should be able to cope with the &os; &release.5x; CDROMs. Reading src/UPDATING is absolutely essential. The section entitled To upgrade from 4.x-stable to current contains a step-by-step update procedure. This procedure must be followed exactly, without making use of the shortcuts that some users occasionally employ. Perl has been removed from the base system, and should be installed either from a pre-built package or from the Ports Collection. Building Perl as a part of the base system created a number of difficulties which made updates problematic. The base system utilities that used Perl have either been rewritten (if still applicable) or discarded (if obsolete). &man.sysinstall.8; will now install the Perl package as a part of most distribution sets, so most users will not notice this change. It is generally possible to run old &release.4x; executables under &release.5x;, but this requires the compat4x distribution to be installed. Using old ports may be possible in some cases, although there are a number of known cases of backward incompatibility. As an example, the devel/gnomevfs2, mail/postfix, and security/cfs ports need to be recompiled due to changes in the statfs structure. When installing or upgrading over the top of an existing 4-STABLE-based system, it is extremely important to clear out old header files in /usr/include. Renaming or moving this directory before a binary installation or an installworld is generally sufficient. If this step is not taken, confusion may result (especially with C++ programs) as the compiler may wind up using a mixture of obsolete and current header files. MAKEDEV is no longer available, nor is it required. FreeBSD &release.5x; uses a device filesystem, which automatically creates device nodes on demand. For more information, please see &man.devfs.5;. UFS2 is the default on-disk format for file systems created using &man.newfs.8;. For all platforms except pc98, it is also the default for file systems created using the disk labeling screen within &man.sysinstall.8;. Because &os; &release.4x; only understands UFS1 (not UFS2), disk partitions that need to be accessed by both &release.5x; and &release.4x; must be created with UFS1. This can be specified using the -O1 option to &man.newfs.8;, or on the disk labeling screen in &man.sysinstall.8;. This situation most often arises with a a single machine that dual-boots &os; &release.4x; and &os; &release.5x;. Note that there is no way to convert file systems between the two on-disk formats (other than backing up, re-creating the file system, and restoring). While &os; &release.current; contains a number of new and exciting features, it may not be suitable for all users at this time. In this document, we presented some background on release engineering, some of the more notable new features of the &release.5x; series, and some drawbacks to early adoption. We also presented some future plans for the 4-STABLE development branch and some tips on upgrading for early adopters.