Contributed by &a.jraynard;. The FreeBSD Ports collection allows you to compile and install a very wide range of applications with a minimum of effort.

For all the hype about open standards, getting a program to work on different versions of Unix in the real world can be a tedious and tricky business, as anyone who's tried it will know. You may be lucky enough to find that the program you want will compile cleanly on your system, install itself in all the right places and run flawlessly ``out of the box'', but this is unfortunately rather rare. With most programs, you will find yourself doing a fair bit of head-scratching, and there are quite a few programs that will result in premature greying, or even chronic alopecia...

Some software distributions have attacked this problem by providing configuration scripts. Some of these are very clever, but they have an unfortunate tendency to triumphantly announce that your system is something you've never heard of and then ask you lots of questions that sound like a final exam in system-level Unix programming (``Does your system's gethitlist function return a const pointer to a fromboz or a pointer to a const fromboz? Do you have Foonix style unacceptable exception handling? And if not, why not?'').

Fortunately, with the Ports collection, all the hard work involved has already been done, and you can just type 'make install' and get a working program. Why have a Ports Collection?

The base FreeBSD system comes with a very wide range of tools and system utilities, but a lot of popular programs aren't in the base system, for good reasons:- ``I can't live without x y and z on my system'' type programs (eg a certain Lisp-based editor, or the mtools set of programs for dealing with DOS floppy disks), because it's too subjective (many people can't stand Emacs and/or never use DOS floppies and seem none the worse for it). Too specialised to put in the base system (CAD, databases). Programs which fall into the ``I wouldn't mind having a look at that when I get a spare minute'' category, rather than system-critical ones (some languages, perhaps). ``Wow fab this is way cool'' fun type programs that couldn't possibly be supplied with a serious operating system like FreeBSD ;-) However many programs you put in the base system, people will always want more, and a line has to be drawn somewhere (otherwise FreeBSD distributions would become absolutely enormous).

Obviously it would be unreasonable to expect everyone to port their favourite programs by hand (not to mention a tremendous amount of duplicated work), so the FreeBSD Project came up with an ingenious way of using standard tools that would automate the process.

Incidentally, this is an excellent illustration of how ``the Unix way'' works in practice by combining a set of simple but very flexible tools into something very powerful. How does the Ports collection work?

Programs are typically distributed on the Internet as a consisting of a Makefile and the source code for the program and usually some instructions (which are unfortunately not always as instructive as they could be), with perhaps a configuration script.

The standard scenario is that you FTP down the tarball, extract it somewhere, glance through the instructions, make any changes that seem necessary, run the configure script to set things up and use the standard `make' program to compile and install the program from the source.

FreeBSD ports still use the tarball mechanism, but use a to hold the "knowledge" of how to get the program working on FreeBSD, rather than expecting the user to be able to work it out. They also supply their own customised , so that almost every port can be built in the same way.

If you look at a port skeleton (either on or ) and expect to find all sorts of pointy-headed rocket science lurking there, you may be disappointed by the one or two rather unexciting-looking files and directories you find there. (We'll discuss in a minute how to go about ).

``How on earth can this do anything?'' I hear you cry. ``There isn't even any source code there!''

Fear not, gentle reader, all will become clear (hopefully). Let's see what happens if we try and install a port. I've chose `bash', also known as the Bourne-Again Shell, as that seems fairly typical. Note if you're trying this at home, you'll need to be root. # cd /usr/ports/shells/bash # make install Checksums OK. ===> Extracting for bash-1.14.5 ===> Patching for bash-1.14.5 ===> Applying FreeBSD patches for bash-1.14.5 ===> Configuring for bash-1.14.5 ===> Building for bash-1.14.5 [lots and lots of compiler output here...] ===> Installing for bash-1.14.5 make -f bash-Makefile bindir=/usr/local/bin prefix=/usr/local install (cd ./documentation/; make ) rm -f builtins.txt nroff -man builtins.1 > builtins.txt install -c -o bin -g bin -m 555 bash /usr/local/bin/bash install -c -o bin -g bin -m 555 bashbug /usr/local/bin/bashbug ( cd ./documentation/ ; make mandir=/usr/local/man/man1 man3dir=/usr/local/man/man3 infodir=/usr/local/info install ) [ -d /usr/local/man/man1 ] || mkdir /usr/local/man/man1 [ -d /usr/local/info ] || mkdir /usr/local/info ../support/install.sh -c -m 644 bash.1 /usr/local/man/man1 ../support/install.sh -c -m 644 builtins.1 /usr/local/man/man1/bash_builtins.1 ../support/install.sh -c -m 644 features.info /usr/local/info/bash.info gzip -9nf /usr/local/man/man1/bash.1 /usr/local/man/man1/bash_builtins.1 ===> Registering installation for bash-1.14.5

To avoid confusing the issue, I've slightly pruned the install output, as well as completely removing the build output. If you tried this yourself, you may well have got something like this at the start:- >> bash-1.14.5.tar.gz doesn't seem to exist on this system. >> Attempting to fetch from ftp://slc2.ins.cwru.edu/pub/dist/.

The `make' program has noticed that you didn't have a local copy of the source code and tried to FTP it down so it could get the job done (are you starting to feel impressed? 8-)). I already had the source handy in my example, so it didn't need to fetch it.

Let's go through this and see what the `make' program was doing. Locate the source code If it's not available locally, try to grab it from an FTP site. Run a test on the tarball to make sure it hasn't been tampered with, accidentally truncated, struck by neutrinos while in transit, etc. Extract the tarball into a temporary work directory. Apply any needed to get the source to compile and run under FreeBSD. Run any configuration script required by the build process and correctly answer any questions it asks. (Finally!) Compile the code. Install the program executable and other supporting files, man pages, etc. under the /usr/local hierarchy, where they won't get mixed up with system programs. This also makes sure that all the ports you install will go in the same place, instead of being flung all over your system. Register the installation in a database. This means that, if you don't like the program, you can cleanly all traces of it from your system.

See if you can match these steps to the make output. And if you weren't impressed before, you should be by now! Getting a FreeBSD Port

There are two ways of getting hold of the FreeBSD port for a program. One requires a , the other involves using an Compiling ports from CDROM

If you answered yes to the question ``Do you want to link the ports collection to your CDROM'' during the FreeBSD installation, the initial setting up will already have been done for you.

If not, make sure the # mkdir /usr/ports # cd /usr/ports # ln -s /cdrom/ports/distfiles distfiles to enable the ports make mechanism to find the tarballs (it expects to find them in /usr/ports/distfiles, which is why we sym-linked the CDROM's tarball directory to there).

Now, suppose you want to install the gnats program from the databases directory. Here's how to do it:- # cd /usr/ports # mkdir databases # cp -R /cdrom/ports/databases/gnats databases # cd databases/gnats # make install Or if you're a serious database user and you want to compare all the ones available in the Ports collection, do # cd /usr/ports # cp -R /cdrom/ports/databases . # cd databases # make install (yes, that really is a dot on its own after the cp command and not a mistake. It's Unix-ese for ``the current directory'')

and the ports make mechanism will automatically compile and install all the ports in the databases directory for you!

If you don't like this method, here's a completely different way of doing it:-

Create a "link tree" to it using the lndir(1) command that comes with the XFree86 distribution. Find a location with some free space and create a directory there, and make a symbolic link from /usr/ports to that directory. Then invoke the lndir(1) command with the full pathname of the ``ports'' directory on the CDROM as an argument (this might be, for example, something like: lndir /cdrom/ports). Then you can build ports directly off the CDROM by building them in the link tree you have created.

Note that there are some ports for which we cannot provide the original source in the CDROM due to licensing limitations. In that case, you will need to look at the section on Compiling ports from the Internet

If you don't have a CDROM, or you want to make sure you get the very latest version of the port you want, you'll need to download the for the port. Now this might sound like rather a fiddly job full of pitfalls, like downloading the patches into the pkg sub-directory by mistake, but it's actually very easy.

The key to it is that the FreeBSD FTP server can create on-the-fly for you. Here's how it works, with the gnats program in the databases directory as an example (the bits in square brackets are comments, don't type them in if you're trying this yourself!):- # cd /usr/ports # mkdir databases # cd databases # ftp ftp.freebsd.org [log in as `ftp' and give your email address when asked for a password. Remember to use binary (aka image) mode!] > cd /pub/FreeBSD/ports/databases > get gnats.tar.gz [tarballs up the gnats skeleton for us] > quit # tar xzf gnats.tar.gz [extract the gnats skeleton] # cd gnats # make install [build and install gnats] What happened here? We connected to the FTP server in the usual way and went to its databases sub-directory. When we gave it the command `get gnats.tar.gz', the FTP server up the gnats directory for us and even went to the trouble of compressing it before sending it so we could get our hands on it a little quicker.

We then extracted the gnats skeleton and went into the gnats directory to build the port. As we explained , the make process noticed we didn't have a copy of the source locally, so it fetched one before extracting, patching and building it.

Let's try something more ambitious now. Instead of getting a single port skeleton, let's get a whole sub-directory, for example all the database skeletons in the ports collection. It looks almost the same:- # cd /usr/ports # ftp ftp.freebsd.org [log in as `ftp' and give your email address when asked for a password. Remember to use binary (aka image) mode!] > cd /pub/FreeBSD/ports > get databases.tar.gz [tarballs up the databases directory for us] > quit # tar xzf databases.tar.gz [extract all the database skeletons] # cd databases # make install [build and install all the database ports] With half a dozen straightforward commands, we have now got a set of database programs on our FreeBSD machine! All we did that was different from getting a single port skeleton and building it was that we got a whole directory at once, and compiled everything in it at once. Pretty impressive, no?

If you expect to be installing more than one or two ports, it's probably worth downloading all the ports directories - this involves downloading 2 or 3MB, when they're compressed. However, don't get carried away and type 'get ports.tar.gz' unless you're prepared to download the distfiles directory as well - this contains the source code for every single port and will take a very long time to download! Skeletons

A team of compulsive hackers who've forgotten to eat in a frantic attempt to make a deadline? Something unpleasant lurking in the FreeBSD attic? No, a skeleton here is a minimal framework that supplies everything needed to make the ports magic work. Makefile

The most important component of a skeleton is the Makefile. This contains various statements that specify how the port should be compiled and installed. Here's the Makefile for bash:- # New ports collection makefile for: bash # Version required: 1.14.5 # Date created: 21 August 1994 # Whom: jkh # # Makefile,v 1.13 1995/10/04 14:45:01 asami Exp # DISTNAME= bash-1.14.5 CATEGORIES= shells MASTER_SITES= ftp://slc2.ins.cwru.edu/pub/dist/ MAINTAINER= ache@FreeBSD.ORG post-install: .if !defined(NOMANCOMPRESS) gzip -9nf ${PREFIX}/man/man1/bash.1 ${PREFIX}/man/man1/bash_builtins.1 .endif .include <bsd.port.mk> The lines beginning with a "#" sign are comments for the benefit of human readers (as in most Unix script files).

`DISTNAME" specifies the name of the , but without the extension.

`CATEGORIES" states what kind of program this is.

`MASTER_SITES" is the URL(s) of the master FTP site, which is used to retrieve the if it is not available on the local system. This is a site which is regarded as reputable, and is normally the one from which the program is officially distributed (in so far as any software is "officially" distributed on the Internet).

`MAINTAINER" is the email address of the person who is responsible for updating the skeleton if, for example a new version of the program comes out. (Note: The title of "maintainer" is mainly an administrative one; it does please mail &a.ports; and Skipping over the next few lines for a minute, the line .include says that the other statements and commands needed for this port are in a standard file called `bsd.port.mk". As these are the same for all ports, there is no point in duplicating them all over the place, so they are kept in a single standard file.

This is probably not the place to go into a detailed examination of how Makefiles work; suffice it to say that the lines starting with `post-install" over-ride the instructions in bsd.port.mk about what to do after installing the program, so that the man pages can be compressed after they have been put in their final destination. The files directory

The file containing the for the port is called "md5", after the MD5 algorithm used for ports checksums. It lives in a directory with the slightly confusing name of "files".

This directory can also contain other miscellaneous files that are required by the port and don't belong anywhere else. The patches directory

This directory contains the needed to make everything work properly under FreeBSD. The pkg directory

This program contains three quite useful files:- COMMENT - a one-line description of the program. DESCR - a more detailed description. PLIST - a list of all the files that will be created when the program is installed. It does not work?!

Oh. You can do one of four (4) things : Fix it yourself. Technical details can be found in Gripe. This is done by e-mail *ONLY*! The people at Walnut Creek are in no way responsible for the functionality (or lack thereof) of the FreeBSD system as a whole, and especially the ports system, which is mainly contributed by 3rd parties. (If you do not believe me, check the catalogue, especially the line saying "We cannot offer tech-support on this product") The e-mail address is the &a.ports;. Please include details of the port, where you got both the port source & distfile(s) from, and what the error was. Note: At time of writing, lang/Sather does not seem to work on Pentium machines due to the Intel Curse (aka the Floating Point Division Bug). Please do not tell us about this - gripe to Intel instead - it is their bug! Forget it. This is the easiest for most - very few of the programs in ports can be classified as `essential'! Grab the pre-compiled package from a ftp server. The ``master'' package collection is on FreeBSD's FTP server in the though check your local mirror first, please! These are more likely to work (on the whole) than trying to compile from source, and a lot faster! Use the pkg_add(1) or pkg_manage(1) program to install them to your system. I've got this program I'd like to make into a port...

Great! Please see the for detailed instructions on how to do this. Some Questions and Answers

Q. I thought this was going to be a discussion about modems??!

A. Ah. You must be thinking of the serial ports on the back of your computer. We're using `port' here to mean the result of `porting' a program from one version of Unix to another. (It's an unfortunate bad habit of computer people to use the same word to refer to several completely different things). Q. I thought you were supposed to use packages to install extra programs?

A. Yes, that's usually the quickest and easiest way of doing it. Q. So why bother with ports then?

A. Several reasons:- The licensing conditions on some software distributions require that they be distributed as source code, not binaries. Some people don't trust binary distributions. At least with source code you can (in theory) read through it and look for potential problems yourself. If you've got some local patches, you'll need the source to add them yourself. You might have opinions on how a program should be compiled that differ from the person who did the package - some people have strong views on what optimisation setting should be used, whether to build debug versions and then strip them or not, etc. etc. Some people like having code around, so they can read it if they get bored, hack around with it, borrow from it (licence terms permitting, of course!) and so on. If you ain't got the source, it ain't software! ;-) Q. What's a patch?

A. A patch is a small (usually) file that specifies how to go from one version of a file to another. It contains text that says, in effect, things like ``delete line 23'', ``add these two lines after line 468'' or ``change line 197 to this''. Also known as a `diff', since it's generated by a program of that name. Q. What's all this about tarballs?

A. It's a file ending in .tar.gz (with variations like .tar.Z, or even .tgz if you're trying to squeeze the names into a DOS filesystem).

Basically, it's a directory tree that's been archived into a single file (.tar) and then compressed (.gz). This technique was originally used for You can see what files are in them, or even extract them yourself, by using the standard Unix tar program, which comes with the base FreeBSD system, like this:- tar tvzf foobar.tar.gz # View contents of foobar.tar.gz tar xzvf foobar.tar.gz # Extract contents into the current directory Q. And a checksum?

A. It's a number generated by adding up all the data in the file you want to check. If any of the characters change, the checksum will no longer be equal to the total, so a simple comparison will allow you to spot the difference. (In practice, it's done in a more complicated way to spot problems like position-swapping, which won't show up with a simplistic addition). Q. I did what you said for and it worked great until I tried to install the kermit port:- # make install >> cku190.tar.gz doesn't seem to exist on this system. >> Attempting to fetch from ftp://kermit.columbia.edu/kermit/archives/. Why can't it find it? Have I got a dud CDROM?

A. The licensing terms for kermit don't allow us to put the tarball for it on the CDROM, so you'll have to fetch it by hand - sorry! The reason why you got all those error messages was because you weren't connected to the Internet at the time. Once you've downloaded it from any of the sites above, you can re-start the process (try and choose the nearest site to you, though, to save your time and the Internet's bandwidth). Q. I did that, but when I tried to put it into /usr/ports/distfiles I got some error about not having permission.

A. The ports mechanism looks for the tarball in /usr/ports/distfiles, but you won't be able to copy anything there because it's sym-linked to the CDROM, which is read-only. You can tell it to look somewhere else by doing DISTDIR=/where/you/put/it make install Q. Does the ports scheme only work if you have everything in /usr/ports? My system administrator says I must put everything under /u/people/guests/wurzburger, but it doesn't seem to work.

A. You can use the PORTSDIR and PREFIX variables to tell the ports mechanism to use different directories. For instance, PORTSDIR=/u/people/guests/wurzburger/ports make install will compile the port in /u/people/guests/wurzburger/ports and install everything under /usr/local. PREFIX=/u/people/guests/wurzburger/local make install will compile it in /usr/ports and install it in /u/people/guests/wurzburger/local. And of course PORTSDIR=.../ports PREFIX=.../local make install will combine the two (it's too long to fit on the page if I write it in full, but I'm sure you get the idea).

If you don't fancy typing all that in every time you install a port (and to be honest, who would?), it's a good idea to put these variables into your environment. Q. I don't have a FreeBSD CDROM, but I'd like to have all the tarballs handy on my system so I don't have to wait for a download every time I install a port. Is there an easy way to get them all at once?

A. To get every single tarball for the ports collection, do # cd /usr/ports # make fetch For all the tarballs for a single ports directory, do # cd /usr/ports/directory # make fetch and for just one port - well, I think you've guessed already. Q. I know it's probably faster to fetch the tarballs from one of the FreeBSD mirror sites close by. Is there any way to tell the port to fetch them from servers other than ones listed in the MASTER_SITES?

A. Yes. If you know, for example, ftp.FreeBSD.ORG is much closer than sites listed in MASTER_SITES, do as following example. # cd /usr/ports/directory # make MASTER_SITE_OVERRIDE=ftp://ftp.FreeBSD.ORG/pub/FreeBSD/distfiles/ fetch Q. I want to know what files make is going to need before it tries to pull them down.

A. 'make fetch-list' will display a list of the files needed for a port. Q. Is there any way to stop the port from compiling? I want to do some hacking on the source before I install it, but it's a bit tiresome having to watch it and hit control-C every time.

A. Doing 'make extract' will stop it after it has fetched and extracted the source code. Q. I'm trying to make my own port and I want to be able to stop it compiling until I've had a chance to see if my patches worked properly. Is there something like 'make extract', but for patches?

A. Yep, 'make patch' is what you want. And by the way, thank you for your efforts! Q. I've heard that some compiler options can cause bugs. Is this true? How can I make sure that I compile ports with the right settings?

A. Yes, with version 2.6.3 of gcc (the version shipped with FreeBSD 2.1.0 and 2.1.5), the -O2 option could result in buggy code unless you used the -fno-strength-reduce option as well. (Most of the ports don't use -O2). You # CFLAGS='-O2 -fno-strength-reduce' make install or by editing /etc/make.conf, but this doesn't always seem to get picked up. The surest way is to do 'make configure', then go into the source directory and inspect the Makefiles by hand, but this can get tedious if the source has lots of sub-directories, each with their own Makefiles. Q. There's so many ports it's hard to find the one I want. Is there a list anywhere of what ports are available?

A. Look in the INDEX file in /usr/ports. Q. I went to install the 'foo' port but the system suddenly stopped and starting compiling the 'bar' port. What's going on?

A. The 'foo' port needs something that's supplied with 'bar' - for instance, if 'foo' uses graphics, 'bar' might have a library with useful graphics processing routines. Or 'bar' might be a tool that's needed to compile the 'foo' port. Q. I installed the grizzle program from the ports and frankly it's a complete waste of disk space. I want to delete it but I don't know where it put all the files. Any clues?

A. No problem, just do pkg_delete grizzle-6.5 Q. Hang on a minute, you have to know the version number to use that command. You don't seriously expect me to remember that, do you??

A. Not at all, you can find it out by doing pkg_info -a | grep grizzle And it'll tell you:- Information for grizzle-6.5: grizzle-6.5 - the combined piano tutorial, LOGO interpreter and shoot 'em up arcade game. Q. Talking of disk space, the ports directory seems to be taking up an awful lot of room. Is it safe to go in there and delete things?

A. Yes, if you've installed the program and are fairly certain you won't need the source again, there's no point in keeping it hanging around. The best way to do this is # cd /usr/ports # make clean which will go through all the ports subdirectories and delete everything except the skeletons for each port. Q. I tried that and it still left all those tarballs or whatever you called them in the distfiles directory. Can I delete those as well?

A. Yes, if you're sure you've finished with them, those can go as well. Q. I like having lots and lots of programs to play with. Is there any way of installing all the ports in one go?

A. Just do # cd /usr/ports # make install Q. OK, I tried that, but I thought it would take a very long time so I went to bed and left it to get on with it. When I looked at the computer this morning, it had only done three and a half ports. Did something go wrong?

A. No, the problem is that some of the ports need to ask you questions that we can't answer for you (eg ``Do you want to print on A4 or US letter sized paper?'') and they need to have someone on hand to answer them. Q. I really don't want to spend all day staring at the monitor. Any better ideas?

A. OK, do this before you go to bed/work/the local park:- # cd /usr/ports # make -DBATCH install This will install every port that does # cd /usr/ports # make -DIS_INTERACTIVE install to finish the job. Q. At work, we're using frobble, which is in your ports collection, but we've altered it quite a bit to get it to do what we need. Is there any way of making our own packages, so we can distribute it more easily around our sites?

A. No problem, assuming you know how to make patches for your changes:- # cd /usr/ports/somewhere/frobble # make extract # cd work/frobble-2.8 [Apply your patches] # cd ../.. # make package Q. This ports stuff is really clever. I'm desperate to find out how you did it. What's the secret?

A. Nothing secret about it at all, just look at the bsd.ports.mk and bsd.ports.subdir.mk files in your (Note: readers with an aversion to intricate shell-scripts are advised not to follow this link...)