The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.
Significant changes and new features include:
o sa(4) driver status and parameter information is now exported via an
XML structure. This will allow for changes and improvements later
on that will not break userland applications. The old MTIOCGET
status ioctl remains, so applications using the existing interface
will not break.
o 'mt status' now reports drive-reported tape position information
as well as the previously available calculated tape position
information. These numbers will be different at times, because
the drive-reported block numbers are relative to BOP (Beginning
of Partition), but the block numbers calculated previously via
sa(4) (and still provided) are relative to the last filemark.
Both numbers are now provided. 'mt status' now also shows the
drive INQUIRY information, serial number and any position flags
(BOP, EOT, etc.) provided with the tape position information.
'mt status -v' adds information on the maximum possible I/O size,
and the underlying values used to calculate it.
o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.
The extra devices were originally added as place holders for
density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
and Solaris) have had device nodes that, when you write to them,
will automatically select a given density for particular tape drives.
This is a convenient way of switching densities, but it was never
implemented in FreeBSD. Only the device nodes were there, and that
sometimes confused users.
For modern tape devices, the density is generally not selectable
(e.g. with LTO) or defaults to the highest availble density when
the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
density selection won't be necessary. If they do need to select
the density, it is easy enough to use 'mt density' to change it.
o Protection information is now supported. This is either a
Reed-Solomon CRC or CRC32 that is included at the end of each block
read and written. On write, the tape drive verifies the CRC, and
on read, the tape drive provides a CRC for the userland application
to verify.
o New, extensible tape driver parameter get/set interface.
o Density reporting information. For drives that support it,
'mt getdensity' will show detailed information on what formats the
tape drive supports, and what formats the tape drive supports.
o Some mt(1) functionality moved into a new mt(3) library so that
external applications can reuse the code.
o The new mt(3) library includes helper routines to aid in parsing
the XML output of the sa(4) driver, and build a tree of driver
metadata.
o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
(write filemark immediate) ioctls needed by IBM's LTFS
implementation.
o Improve device departure behavior for the sa(4) driver. The previous
implementation led to hangs when the device was open.
o This has been tested on the following types of drives:
IBM TS1150
IBM TS1140
IBM LTO-6
IBM LTO-5
HP LTO-2
Seagate DDS-4
Quantum DLT-4000
Exabyte 8505
Sony DDS-2
contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
Add libmt.
lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
New mt(3) library that contains functions moved from mt(1) and
new functions needed to interact with the updated sa(4) driver.
This includes XML parser helper functions that application writers
can use when writing code to query tape parameters.
rescue/rescue/Makefile:
Add -lmt to CRUNCH_LIBS.
src/share/man/man4/mtio.4
Clarify this man page a bit, and since it contains what is
essentially the mtio.h header file, add new ioctls and structure
definitions from mtio.h.
src/share/man/man4/sa.4
Update BUGS and maintainer section.
sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
functions.
sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
Many tape driver changes, largely outlined above.
Increase the sa(4) driver read/write timeout from 4 to 32
minutes. This is based on the recommended values for IBM LTO
5/6 drives. This may also avoid timeouts for other tape
hardware that can take a long time to do retries and error
recovery. Longer term, a better way to handle this is to ask
the drive for recommended timeout values using the REPORT
SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
at least support that command, and it would allow for more
accurate timeout values.
Add XML status generation. This is done with a series of
macros to eliminate as much duplicate code as possible. The
new XML-based status values are reported through the new
MTIOCEXTGET ioctl.
Add XML driver parameter reporting, using the new MTIOCPARAMGET
ioctl.
Add a new driver parameter setting interface, using the new
MTIOCPARAMSET and MTIOCSETLIST ioctls.
Add a new MTIOCRBLIM ioctl to get block limits information.
Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
and scsi_read_position_10().
scsi_locate_10 implements the LOCATE command, as does the
existing scsi_set_position() command. It just supports
additional arguments and features. If/when we figure out a
good way to provide backward compatibility for older
applications using the old function API, we can just revamp
scsi_set_position(). The same goes for
scsi_read_position_10() and the existing scsi_read_position()
function.
Revamp sasetpos() to take the new mtlocate structure as an
argument. It now will use either scsi_locate_10() or
scsi_locate_16(), depending upon the arguments the user
supplies. As before, once we change position we don't have a
clear idea of what the current logical position of the tape
drive is.
For tape drives that support long form position data, we
read the current position and store that for later reporting
after changing the position. This should help applications
like Bacula speed tape access under FreeBSD once they are
modified to support the new ioctls.
Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
drives that report SCSI-2 or older, as well as drives that
report an Illegal Request type error for READ POSITION with
the long format. So we should automatically detect drives
that don't support the long form and stop asking for it after
an initial try.
Add a partition number to the sa(4) softc.
Improve device departure handling. The previous implementation
led to hangs when the device was open.
If an application had the sa(4) driver open, and attempted to
close it after it went away, the cam_periph_release() call in
saclose() would cause the periph to get destroyed because that
was the last reference to it. Because destroy_dev() was
called from the sa(4) driver's cleanup routine (sacleanup()),
and would block waiting for the close to happen, a deadlock
would result.
So instead of calling destroy_dev() from the cleanup routine,
call destroy_dev_sched_cb() from saoninvalidate() and wait for
the callback.
Acquire a reference for devfs in saregister(), and release it
in the new sadevgonecb() routine when all devfs devices for
the particular sa(4) driver instance are gone.
Add a new function, sasetupdev(), to centralize setting
per-instance devfs device parameters instead of repeating the
code in saregister().
Add an open count to the softc, so we know how many
peripheral driver references are a result of open
sessions.
Add the D_TRACKCLOSE flag to the cdevsw flags so
that we get a 1:1 mapping of open to close calls
instead of a N:1 mapping.
This should be a no-op for everything except the
control device, since we don't allow more than one
open on non-control devices.
However, since we do allow multiple opens on the
control device, the combination of the open count
and the D_TRACKCLOSE flag should result in an
accurate peripheral driver reference count, and an
accurate open count.
The accurate open count allows us to release all
peripheral driver references that are the result
of open contexts once we get the callback from devfs.
sys/sys/mtio.h:
Add a number of new mt(4) ioctls and the requisite data
structures. None of the existing interfaces been removed
or changed.
This includes definitions for the following new ioctls:
MTIOCRBLIM /* get block limits */
MTIOCEXTLOCATE /* seek to position */
MTIOCEXTGET /* get tape status */
MTIOCPARAMGET /* get tape params */
MTIOCPARAMSET /* set tape params */
MTIOCSETLIST /* set N params */
usr.bin/mt/Makefile:
mt(1) now depends on libmt, libsbuf and libbsdxml.
usr.bin/mt/mt.1:
Document new mt(1) features and subcommands.
usr.bin/mt/mt.c:
Implement support for mt(1) subcommands that need to
use getopt(3) for their arguments.
Implement a new 'mt status' command to replace the old
'mt status' command. The old status command has been
renamed 'ostatus'.
The new status function uses the MTIOCEXTGET ioctl, and
therefore parses the XML data to determine drive status.
The -x argument to 'mt status' allows the user to dump out
the raw XML reported by the kernel.
The new status display is mostly the same as the old status
display, except that it doesn't print the redundant density
mode information, and it does print the current partition
number and position flags.
Add a new command, 'mt locate', that will supersede the
old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
implements all of the functionality of the MTIOCEXTLOCATE
ioctl, and allows the user to change the logical position
of the tape drive in a number of ways. (Partition,
block number, file number, set mark number, end of data.)
The immediate bit and the explicit address bits are
implemented, but not documented in the man page.
Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
This allows the user to ask the drive to write a filemark
without waiting around for the operation to complete.
Add a new 'mt getdensity' command that gets the XML-based
tape drive density report from the sa(4) driver and displays
it. This uses the SCSI REPORT DENSITY SUPPORT command
to get comprehensive information from the tape drive about
what formats it is able to read and write.
Add a new 'mt protect' command that allows getting and setting
tape drive protection information. The protection information
is a CRC tacked on to the end of every read/write from and to
the tape drive.
Sponsored by: Spectra Logic
MFC after: 1 month
probes to userland programs and libraries without also needing to link
libelf.
dtrace -G places the __SUNW_dof symbol at the beginning of the DOF (DTrace
probe and provider metdata) section in the generated object file; drti.o
now just uses this symbol to locate the section. A complication occurs
when multiple dtrace-generated object files are linked together, since the
__SUNW_dof symbol defined in each file is global. This is handled by
using objcopy(1) to convert __SUNW_dof to a local symbol once drti.o has
been linked with the generated object file. Upstream, this is done using a
linker feature not present in GNU ld.
Differential Revision: https://reviews.freebsd.org/D1757
Reviewed by: rpaulo
MFC after: 1 month
Relnotes: yes
libarchive(3) doesn't support the new liblzma API yet, but this change
allows us to enable multi-threaded xz compression.
``make release'' should now finish in half the time on a machine with
several cores and fast disks (our typical build server).
This behaviour only applies when building a release and it doesn't
affect buildworld/installworld. To disable threaded xz compression,
set XZ_THREADS=1.
Reviewed by: gjb
Tested by: gjb
SHLIB_NAME_FULL so that the full binary is relinked when a dependency
changes. Right now the existing full binary is left as-is and only
the objcopy to remove debug symbols is run.
Differential Revision: https://reviews.freebsd.org/D1834
Reviewed by: emaste
MFC after: 3 days
update paths; and include everything in the "base" distribution.
The "games" distribution being optional made sense when there were more
games and we had small disks; but the "games-like" games were moved into
the ports tree a dozen years ago and the remaining "utility-like" games
occupy less than 0.001% of my laptop's small hard drive. Meanwhile every
new user is confronted by the question "do you want games installed" when
they they try to install FreeBSD.
The next steps will be:
2. Removing punch card (bcd, ppt), phase-of-moon (pom), clock (grdc), and
caesar cipher (caesar, rot13) utilities. I intend to keep fortune, factor,
morse, number, primes, and random, since there is evidence that those are
still being used.
3. Merging src/games into src/usr.bin.
This change will not be MFCed.
Reviewed by: jmg
Discussed at: EuroBSDCon
Approved by: gjb (release-affecting changes)
This brings support for multi-threaded compression. This brings close
N times faster compression where N is the number of CPU cores.
Because of this, liblzma now depends on libthr.
Soon libarchive will be modified to use the new lzma API.
Thanks to antoine@ for the exp-run.
Differential Revision: https://reviews.freebsd.org/D1786
Reviewed by: bapt
allows the user to request administrative changes to individual devices
such as attach or detaching drivers or disabling and re-enabling devices.
- Add a new /dev/devctl2 character device which uses ioctls for device
requests. The ioctls use a common 'struct devreq' which is somewhat
similar to 'struct ifreq'.
- The ioctls identify the device to operate on via a string. This
string can either by the device's name, or it can be a bus-specific
address. (For unattached devices, a bus address is the only way to
locate a device.) Bus drivers register an eventhandler to claim
unrecognized device names that the driver recognizes as a valid address.
Two buses currently support addresses: ACPI recognizes any device
in the ACPI namespace via its full path starting with "\" and
the PCI bus driver recognizes an address specification of
'pci[<domain>:]<bus>:<slot>:<func>' (identical to the PCI selector
strings supported by pciconf).
- To make it easier to cut and paste, change the PnP location string
in the PCI bus driver to output a full PCI selector string rather
than 'slot=<slot> function=<func>'.
- Add a devctl(3) interface in libdevctl which provides a wrapper around
the ioctls and is the preferred interface for other userland code.
- Add a devctl(8) program which is a simple wrapper around the requests
supported by devctl(3).
- Add a device_is_suspended() function to check DF_SUSPENDED.
- Add a resource_unset_value() function that can be used to remove a
hint from the kernel environment. This is used to clear a
hint.<driver>.<unit>.disabled hint when re-enabling a boot-time
disabled device.
Reviewed by: imp (parts)
Requested by: imp (changing PCI location string)
Relnotes: yes
__attribute__((format(...))), and the -fformat-extensions flag was
removed, introduce a new macro in bsd.sys.mk to choose the right variant
of compile flag for the used compiler, and use it.
Also add something similar to kern.mk, since including bsd.sys.mk from
that file will anger Warner. :-)
Note that bsd.sys.mk does not support the MK_FORMAT_EXTENSIONS knob used
in kern.mk, since that knob is only available in kern.opts.mk, not in
src.opts.mk. We might want to add it later, to more easily support
external compilers for building world (in particular, sys/boot).
triggers way too many times for the version of libc++ we have in base at
this point. While here, fix the compiler version check for
-Wno-unused-const-variable.
use floating point hardware instructions (because all armv6/7 systems we
support have fp hardware), but it passes args using a soft-float compatible
ABI. This should give noticible performance improvement (but not as much
as using the armv6hf arch).
Some users build FreeBSD as non-root in Perforce workspaces. By default,
Perforce sets files read-only unless they're explicitly being edited.
As a result, the -f argument must be used to cp in order to override the
read-only flag when copying source files to object directories. Bare use of
'cp' should be avoided in the future.
Update all current users of 'cp' in the src tree.
Reviewed by: emaste
MFC after: 1 week
Sponsored by: Spectra Logic
These tools are now from the ELF Tool Chain project:
* addr2line
* elfcopy (strip)
* nm
* size
* strings
The binutils versions are available by setting in src.conf:
WITHOUT_ELFTOOLCHAIN_TOOLS=yes
Thanks to antoine@ for multiple exp-runs and diagnosing many of the
failures.
PR: 195561 (ports exp-run)
Sponsored by: The FreeBSD Foundation
As a side-effect now info pages will always be built/installed if
MK_INFO == yes, whereas before their presence was conditional based on the
value of MK_INFO
This .mk file might be removed in the future, pending discussion on -arch. For
now unbreak its use outside of src (with the only use in ports according to
bapt being devel/cvs*)
X-MFC with: r276551, r276556
Reviewed by: bapt
Differential Revision: D1413
raft of new warnings that appear to be on by default in clang 3.5.0.
Fix RPI-B build issues with new clang not liking the ability to pass
arbitrary flags to as, since some flags are more arbitrary (and thus
verboten) than others.
These warnings should be actually fixed in the code, but this is a
band-aide to get things (almost) building again.
Recent binutils considered the .gnu.warning.symbol section as a fatal error when
run with --fatal-warnings which makes any users of "insecure" functions from
libc failing to build with recent binutils.
Introduce a new macro: LD_FATAL_WARNINGS=no to run ld(1) with
--no-fatal-warnings for the users of "insecure" functions
Differential Revision: https://reviews.freebsd.org/D1320
support, make this explicit in src.opts.mk, by updating the default
settings.
The defaults become as follows:
* If the host compiler is not C++11 capable, use gcc and disable clang.
* On x86, enable clang, make it the default cc, and disable gcc.
* On little-endian ARM, enable clang, but not the full build, make it
the default cc, and disable gcc.
* On PowerPC, enable clang, but enable gcc and make that the default cc.
* On everything else, use gcc, and disable clang.
This can be amended later, if we get e.g. sparc64 or big-endian ARM
working with clang.
Reviewed by: imp, brooks
Differential Revision: https://reviews.freebsd.org/D1294
Set WITH_ELFTOOLCHAIN_TOOLS in src.conf to use the elftoolchain version
of the following tools:
* addr2line
* elfcopy (strip / mcs)
* nm
* size
* strings
Reviewed by: bapt (earlier version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1224
- MK_FILE will conditionalize building lib/libmagic and usr.bin/file
- MK_SQLITE3 will conditionalize building lib/libsqlite3, and will disable
MK_SVN/MK_SVNLITE by proxy
implementation, merge ^/head r275078 through r275117.
Note that all the extraneous mergeinfo is there because Subversion
created it. I'll hopefully be able to remove it again when merging back
to head.
- bootparamd
- bootpd
- finger/fingerd
- ftp/ftpd
- hastctl/hastd
- iscsid, et al
- rbootd
- talk/talkd
- tcpd, et al
- tftp/tftpd
Add src.conf entries for the various components and do a best effort
at adding components to tools/build/mk/OptionalObsoleteFiles.inc
This is a thin wrapper around the kernel interface which should make
it easier to write GPIO applications. gpioctl(8) will be converted to
use this library in a separate commit.
Differential Revision: https://reviews.freebsd.org/D1183
Reviewed by: adrian, loos
Discussed on: arm@, embedded@
Relnotes: yes
LIBADD will automatically set DPADD and LDADD when needed including their
dependencies, LIBADD automatically handles private and internal libs so that
the end user Makefile does not have to care about it.
This allows to reduce overlinking on the base system leaving the framework get
the dependencies properly.
It also allows to built components binaries statically.
To use it:
Replace:
DPADD= ${LIBARCHIVE} ${LIBSSL}
LDADD= -larchive -lssl
by:
LIBADD= archive ssl
Differential Revision: https://reviews.freebsd.org/D1209
Reviewed by: brooks imp
Otherwise there's nothing for ctfconvert to do, and it ends up emitting an
error for each object file. Also remove some redundant checks from
bsd.prog.mk and bsd.lib.mk.
Differential Revision: https://reviews.freebsd.org/D1111
Reviewed by: imp
a kludge. However, it also effectively works around the issues for
high -j builds on systems that do not have the rm fixes.
A better fix would be to rmdir here, and fix the places where we're
sloppy and not list all the files we create in CLEANFILES, should
anybody have the time to chase them all to ground.
generating files from various sources instead of calling cat ${.ALLSRC} | sed
The perl case was skipped because it's not being used in the tree at this time
contrib/netbsd-tests
This Makefile snippet handles polluting testcases with -lnetbsd, specific
headers for ATF version differences, and does necessary rewriting for the
testcases to match the format discussed on the TestSuite wiki page
(t_<foo> -> <foo>_test)
One must define SRCTOP (inspired by projects/bmake), OBJTOP, and TESTSRC
(e.g. contrib/netbsd-tests/lib/libc/gen) to use the Makefile snippet
Test programs are specific either via NETBSD_ATF_TESTS_C or NETBSD_ATF_TESTS_SH
C++ analogs aren't currently implemented.
The imported testcases will be cleaned up to use this Makefile snippet pseudo
"API".
bsd.progs.mk generates a separate depend file for every program being
built, but then it does not properly tell each submake to use those
individual files. Properly propagate the depend file to use.
Discovered while preparing the update of atf to 0.21 and noticing that
the test programs were not being relinked to the new library.
This change is "make tinderbox" clean.
expression (:M) is empty, not the not matched (:N) is empty. The former case
means we have not found the TEST_SUBDIR value in SUBDIR
Reported by: rodrigc
X-MFC with: r273803
Pointyhat to: me (did not use a clean install root)
Sponsored by: EMC / Isilon Storage Division
appending the TESTS_SUBDIRS variable to SUBDIR
Duplicate directory entries can cause unexpected side effects, like
installing the same files multiple times. This can be easily
reproduced via the following testcase prior to this commit:
SUBDIR= dir
TESTS_SUBDIRS+= dir
.include <bsd.test.mk>
Sponsored by: EMC / Isilon Storage Division
Check for -static in LDFLAGS or LDFLAGS.<test>, then pass in the appropriate
dependency (LIBATF or LDATF)
This unbreaks the build with some of the NetBSD tests that need to be compiled
statically
Reviewed by: imp, jmmv
Phabric: D991
Sponsored by: EMC / Isilon Storage Division
issue. lib/atf isn't a prereq_lib, since it isn't required for other
libraries to build. Remove it. The old kludge of always building it
had effectively been retired. Since we don't want to build the
libraries with the tests when we're bootstrapping, invent
MK_TESTS_SUPPORT which normally defaults to the current MK_TESTS
value, except when explicitly defined. Make lib/atf depend on it being
yes. When building the libraries set MK_TESTS to no, and
MK_TESTS_SUPPORT to the current value of MK_TESTS so that later stages
of the build work correctly. This should fix (and does for me)
people's issues with parallel builds racing between lib/atf and
libexec/atf. Since lib/atf is built during the libraries phase, the
race disappears.
userland defaults to no. This caused issues for the automated option
documenation script. Turns out, this isn't used in userland at all, so
just remove it from here.
time and gnu.warnings.symbol are now being fatal preventing building world.
in the futur we want to investigate only making the gnu.warning.symbol non fatal
Reviewed by: imp
when building with an external gcc, we want to be able to pass the path to
the libc++ headers so dependencies are correctly computed for C++ source files.
Add a DEPFLAGS for that purpose
Reviewed by: imp
* anchor search strings appropriately,
* use .ALLSRC to pass the full path to the D script to dtrace(1),
* don't insert the auto-generated header into SRCS - it doesn't
accomplish anything, and we end up having to remove it from OBJS anyway.
Reviewed by: rpaulo
Differential Revision: https://reviews.freebsd.org/D978
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
for i386/amd64. Rather, it only works on i386/amd64 and should only be
built there. Rather than change the default based on which
architecutre, do things more directly by only building it on
i386/amd64 and having it always on. This is how we handle other
options that are relevant only for a few architectures.
This allows us to use libevent for other application in the future.
For now libevent is still INTERNALLIB and no shared library is installed.
MFC after: 1 month
This has the side effect of ensuring that realpath is also run for the
nominal case of PORTSDIR=/usr/ports (assuming .CURDIR is a ports directory
that relies on /usr/ports but is not rooted in it). This ensures that any
generated PORTSDIR used is always the actual location.
Submitted by: jkim (different implementation)
the oabi is still in the tree, but it is expected this will be removed
as developers work on surrounding code.
With this commit the ARM EABI is the only supported supported ABI by
FreeBSD on ARMa 32-bit processors.
X-MFC after: never
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D876
This will only take effect if PORTSDIR is not set, as previously supported.
Use .if exists(), for four specific possibilities relative to .CURDIR:
., .., ../.., and ../../.. The fourth possibility is primarily in case
ports ever grows a third level. If none of these paths exist, fall back to
the old default of /usr/ports.
This removes the need to set PORTSDIR explicitly (or via wrapper script) if
one is running out of a ports tree that is not in /usr/ports, but in a
home directory.
Reviewed by: bapt, bdrewery (older version)
CR: D799
MFC after: 1 week
Sponsored by: Spectra Logic
1. Do not install FILES/SCRIPTS multiple times if PROGS is specified; this is
already handled via bsd.prog.mk when it's called recursively (PR: 191055,
191955).
2. Some variables, like BINDIR and PROGNAME, default to a value if unset
whereas others get appended to, like CFLAGS. Add support for the former case
(PR: 191056)
3. Make "checkdpadd" and "clean" available targets for recursive execution.
Reviewed by: marcel, sjg
Phabric: D822
PR: 191055, 191056, 191955
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
many thanks for their continued support of FreeBSD.
While I'm there, also implement a new build knob, WITHOUT_HYPERV to
disable building and installing of the HyperV utilities when necessary.
The HyperV utilities are only built for i386 and amd64 targets.
This is a stable/10 candidate for inclusion with 10.1-RELEASE.
Submitted by: Wei Hu <weh microsoft com>
MFC after: 1 week
- Reduce DPADD and LDADD in checkdpadd to -l<foo>
- Skip over -Wl,[es]*-group because -Wl,--end-group and
-Wl,--start-group might be required to properly link objects (see
usr.bin/clang/lldb as an example)
This caveat has been present for a while with some components of
the build. However, these false positives were made more more apparent
after r269648.
Phabric: D635
Reviewed by: jmmv (an earlier version)
PR: 192730
MFC after: 2 weeks
This allows WITH_DEBUG_FILES to produce standalone debug for the ELF
runtime linker.
We previously disabled standalone debug files for bsd.prog.mk consumers
that included a non-default ${PROG} target, but this is not required.
Consumers that do not support standalone debug are still handled by
disabling it for statically linked binaries, and for those that specify
a non-default binary format.
Sponsored by: DARPA, AFRL
While there, add a NO_WTHREAD_SAFETY flag that can be used to disable
this specific warning flag. Disable it for auditdistd. We can easily
patch up auditdistd to have the right annotations to build, but as
auditdistd is intended to be portable across other operating systems,
it's not worth the effort.
Approved by: brueffer@
Add LIBSSP_NONSHARED to bsd.libnames.mk and append LIBSSP_NONSHARED to DPADD in
lib/libc when MK_SSP != no
Approved by: rpaulo (mentor)
MFC after: 3 days
Phabric: D675 (as part of a larger diff)
PR: 192728
1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.
We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.
Reported by: kib
the build starts.
This adds a new variable DHDRS that contains a list of all DTrace
header files. Then, we use the beforedepend hook to make sure the
heaeder files are built.
Introduce a beforebuild dependency (from projects/bmake) based on
feedback from Simon J. Gerraty. This lets us generate the header
files without running make depend.
Reviewed by: sjg, imp
MFC after: 3 days
This doesn't work like I expected and the if clause is never matched.
It turns out that the problem with the older make is unrelated.
MFC after: 3 days
* Include OBJDIR to make sure the generated file is found;
* Simplify the definition of OBJS;
* Add targets for shared objects and for profiled objects.
MFC after: 3 days
Make sure everything linking to a privatelib and/or an internallib does it directly
from the OBJDIR rather than DESTDIR.
Add src.libnames.mk so bsd.libnames.mk is not polluted by libraries not existsing
in final installation
Introduce the LD* variable which is what ld(1) is expecting (via LDADD) to link to
internal/privatelib
Directly link to the .so in case of private library to avoid having to complexify
LDFLAGS.
Phabric: https://phabric.freebsd.org/D553
Reviewed by: imp, emaste
get them flagged as .NOPATH. This hurts people who don't use obj dirs.
Since its clean target seprate rm's for things, use NOPATH_FILES as list
to collect things that need .NOPATH.
bsd.obj.mk will add CLEANFILES to NOPATH_FILES and do the deed if needed.
Reviewed by: sbruno
This is a redo of r267424, which was reverted in r267432 because it broke
"make buildworld" from FreeBSD 9.x. This issue has been resolved and this
change is still "make tinderbox" clean.
nothing more. Force it to be "no" when MK_CXX is "no" to simplify
usage. It no longer also means "build g++" since we no longer have a
platform where that's interesting now that pc98 no longer needs clang
and gcc, but not g++. pc98 now just uses clang after boot2 changes.
statically linked into consumers (GDB and variants) in the base
system, and the shared library is no longer installed.
That also allows ports to be able to use a modern version of readline
PR: 162948
Reviewed by: emaste
With the move of atf-sh into /usr/libexec in r267181, some of the
tests in the integration_test program broke because they could not
execute atf-sh from the path any longer.
This slipped through because I do have a local atf installation in
my home directory that appears in my path, hence the tests could
still execute my own version.
Fix this by forcing /usr/libexec to appear at the beginning of the
path when attempting to execute atf-sh.
To make upgrading easy (and to avoid an unnecessary entry in UPDATING),
make integration_test depend on the Makefile so that a rebuild of the
shell script is triggered. This requires a hack in the *.test.mk files
to ensure the Makefile is not treated as a source to the generated
program. Ugly, I know, but I don't have a better way of doing this at
the moment. Will think of one once I address the TODO in the *.test.mk
files that suggests generalizing the file generation functionality.
PR: 191052
Reviewed by: Garrett Cooper
This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation
This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h
Discussed at: BSDcan
When building world when WITH_DEBUG_FILES is set,
avoid building ar(1) archives with '-g'.
Tested on: head@r267801
Reviewed by: emaste, imp
Sponsored by: The FreeBSD Foundation
The _SUPPORT knobs have a consistent meaning which differs from the
behaviour controlled by this knob. As the knob is opt-out and has not
appeared in a release the impact should be low.
Suggested by: imp, wblock
MFC after: 1 week
during SUBDIR_PARALLEL builds. This augments the coarse .WAIT mechanism,
which is still useful if you've got a situation such as "almost everything
depends on A and B".
Because the parallel subdir mechanism uses non-obvious mangling of
target names, which should probably remain a private detail of the
implementation, it's not easy to do things like "libfoo: libbar", so
instead the new mechanism lets you set a variable that lists dependencies:
SUBDIR_DEPEND_libfoo= libgroodah libpouet
Note that while I'm using libraries as an example here, it really has
nothing to do with the generated library files. This is really saying
"build in directory libfoo after building in the libgroodah and libpouet
directories."
This updates lib/Makefile with dependency information based on the old
almost-accurate comment block and by combing through lib/* makefiles
looking for LDADD dependencies to other libraries within lib/*.
Reviewed by: Jia-Shiun Li <jiashiun@gmail.com>
This was never intended to be off by default but was done this way
because the initial patches broke the build. Things seem stable now
(have been so for a while too) and "make tinderbox" is clean so let's
try again.
Announced in freebsd-current; all reported shortcomings have been
addressed.
Force all the contents of /usr/tests to go into a separate distribution
file so that users of binary releases can easily choose to not install it.
To make this possible, we need two fixes:
- bsd.subdir.mk needs to properly honor NO_SUBDIR in all cases so that we
do not recurse into 'tests' subdirectories when we needn't. Otherwise,
we end up with some Kyuafiles in base.txz.
- etc/Makefile needs to skip installing tests in its 'distribute' target
so that a Kyuafile doesn't leak into base.txz.
Approved by: gjb
When FILES is defined in a Makefile that _also_ includes bsd.subdir.mk, the
build of the files (if any) was not properly triggered during the build
stage. This was because bsd.files.mk did not define the buildfiles target
if it was already defined... and bsd.subdir.mk defined this target on its
own, thus causing a conflict.
Fix this by unconditionally defining buildfiles from bsd.files.mk; this is
safe because nothing else in the tree needs to redefine this and because the
target itself contains no commands: all it does is define dependencies.
Also ensure that bsd.files.mk is always pulled in by bsd.test.mk regardless
of what bsd.prog.mk does.
These fixes allow "make installworld" to run cleanly on a system with
read-only src and obj trees.
This is "make tinderbox" clean.
Reviewed by: imp
Obtained from: jilles
This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.
Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]
Mark known build failures as NO_PIE for now.
The only known runtime failure was rtld.
[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]
In r266650, we made libatf-c and libatf-c++ private libraries so that no
components outside of the source tree could unintendedly depend on them.
This change does the same for the "atf-sh library" by moving the atf-sh
interpreter from its public location in /usr/bin/ to the private location
in /usr/libexec/. Our build system will ensure that our own test programs
use the right binary, but users won't be able to depend on atf-sh by
"mistake".
Committing this now to ride the UPDATING notice added with r267172 today.
This exists already in plain.test.mk and tap.test.mk and should have been
added to atf.test.mk too when the feature was first introduced.
(It is probably time to address the related TODOs but I will do that
separately.)
We should not be leaking these interfaces to the outside world given
that it's much easier for third-party components to use the devel/atf
package from ports.
As a side-effect, we can also drop the ATF pkgconfig and aclocal files
from the base system. Nothing in the base system needs these, and it
was quite ugly to have to get them installed only so that a few ports
could build. The offending ports have been fixed to depend on
devel/atf explicitly.
Reviewed by: bapt
Makefiles are evaluated without building things. In a normal build,
the prerequisites would be built, and CC would be an actual thing. In
an INDEX build, though, they don't exists. Redirect stderr to get rid
of annoying messages, and assume that the compiler version is 0 if the
actual compiler can't tell us. Do this in preference to guessing based
on numbers because gcc410 might be 4.10, or 4.1.0 and without
carefully crafted special knowledge we differentiate between them
easily (also ming-gcc has no clues at all). Elsewhere, don't trust
the compiler version if it is 0.
this allows make -n to do tree walks as expected without
doing anything else (as intended).
Use prefix _sub. to help avoid conflict with any real target.
Reviewed by: imp
Targets thus marked are supposed to run even with -n.
As such they should not do anything except run the sub-make.
Use an intermediate target _* to associate with _SUBDIR and which
depends on installincludes etc so that we get the correct behavior with -n.
Reviewed by: marcel
src/sys and the rest of the tree for builds.
o eliminate including bsd.mkopts.mk for the moment in kern.opts.mk
o No need to include src.opts.mk at all anymore. The reasons for it
are now coverted in sys.mk and src.sys.mk.
and MK_LLDB=no, so set those explicitly (now that we can do
that). Simplify tests for these variables as well, since we know they
will always be defined regardless of the phase of the build.
with clang 3.3. Useful for test building -current on a -stable system
in individual directories. Potentially useful if we ever want to
support, say, gcc 4.8 or 4.9's new warnings when building with an
external toolchain (but such support not yet committed). Document
the bsd.compiler.mk interface.
install it as fmake. This defaults to no. This should be viewed as the
first step towards evental migration of this historic code to ports
and removal from the tree.
versions. With its movement to src.opts.mk, bsd.prog.mk was testing
COMPILER_TYPE without including the bsd.compiler.mk anymore. In the
source tree, this caused no problems, for reasons that aren't clear,
but does cause problems outside of the source tree. Allow
bsd.compiler.mk to be included multiple times safely, and always
include bsd.compiler.mk at the top of bsd.prog.mk. Resist the urge to
put it in bsd.init.mk, since that would reintroduce the implicit
include.
options, so move their processing there. This fixes issues with
Makefiles that define NO_MAN=t and only inlcude bsd.*.mk files. A few
ports fell into this category, and they should be fixed by this change.
Also, for now, disable the warning about NO_foo being deprecated. More
work is needed than anticipated before we can do that, so kill the
noise for now.
