Commit Graph

17088 Commits

Author SHA1 Message Date
Ed Maste
015dfda6c6 Build and install userland .debug files by default
Debug data files are now built by default with 'make buildworld' and
installed with 'make installworld'. This facilitates debugging but
requires more disk space both during the build and for the installed
world. Debug files may be disabled by setting WITHOUT_DEBUG_FILES=yes
in src.conf(5).

Reviewed by:	bdrewery, eadler, vangyzen
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D4018
2015-12-07 19:15:52 +00:00
Christian Brueffer
a8b75f756d Add an MLINK for m_collapse.
PR:		204205
Submitted by:	avos
MFC after:	1 week
2015-12-07 19:08:33 +00:00
Christian Brueffer
24ae305fb7 Fix a typo in the CPUTYPE list.
PR:		205099
Submitted by:	xxjack12xx@gmail.com
MFC after:	1 week
2015-12-07 18:39:38 +00:00
Bryan Drewery
10f696a52d FAST_DEPEND: Only pass -MF if we care about the object being compiled.
This will save time generating dependency files that we didn't expect
due to cases where SRCS!=OBJS or for building custom targetted objects
in Makefiles that do not end up in the DEPENDOBJS list.

This uses a bmake trick to modify CFLAGS based on ${.TARGET}.  A
.PARSEDIR check is done for the sake of MFC safety.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-12-07 16:08:22 +00:00
Bryan Drewery
b9964333e0 FAST_DEPEND: Move handling code below yacc/lex/dtrace code that modified SRCS.
This fixes some of those newly added SRCS not having their depend files
included.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-12-07 16:08:19 +00:00
Bryan Drewery
69afbf0275 FAST_DEPEND: Only try to use dependencies from C/C++ SRCS as mkdep did.
Rather than try to guess at all of the OBJS variables just use SRCS
using the same patterns that mkdep does.  This also fixes a mistake
where dependencies were being generated with FAST_DEPEND when they were
not for mkdep.  This happens when OBJS!=SRCS as is the case in
gnu/lib/csu where SRCS has 1 file and OBJS has several other files that
does not even contain the 1 SRCS file.  Generally in these cases the
OBJS have custom dependencies defined in their Makefile.  If we generate
dependencies for those and then load a .depend file, then .IMPSRC may
contain duplicate sources and lead to errors such as:
  cc: error: cannot specify -o when generating multiple output files

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-12-07 16:08:16 +00:00
Enji Cooper
765086c5da Remove unused atf.test.mk variables
- ATF_BUILD_CC
- ATF_BUILD_CPP
- ATF_BUILD_CXX
- ATF_SHELL
- ATF_PREFIX

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-12-06 07:30:18 +00:00
Enji Cooper
7254383959 Remove redundant default TESTSDIR that is already defined in bsd.test.mk
after r289158

MFC after: 1 week
X-MFC with: r289158
Sponsored by: EMC / Isilon Storage Division
2015-12-06 07:00:23 +00:00
Enji Cooper
fb77db098d Use .Fx instead of explicitly spelling out FreeBSD
Fix several warnings reported by igor

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-12-06 06:53:37 +00:00
Conrad Meyer
3f07044f5b style.9: Add a small blurb about allowing bool
It was allowed before, but make it very explicit it is allowed now.  And
prefer 'bool' to older types that were used for the same purpose -- int and
boolean_t.

Like with the C99 fixed-width types, use common sense when changing old
code.

No igor regressions.

Suggested by:	bde <20151205031713.T3286@besplex.bde.org>
Reviewed by:	glebius, davide, bapt (earlier versions)
Reviewed by:	imp
Feedback from:	julian
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D4384
2015-12-05 17:01:38 +00:00
Andrew Rybchenko
9c2c444b10 sfxge: support for MCDI logging implemented
Submitted by:   Artem V. Andreev <Artem.Andreev at oktetlabs.ru>
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D4355
2015-12-05 07:04:11 +00:00
Christian Brueffer
7f8590a663 ARC-1203 is supported since the latest driver update. 2015-12-04 10:34:58 +00:00
Bryan Drewery
2231cd21d8 Fix 'install*' and many other missing targets with DIRDEPS_BUILD.
My changes in r291635 broke 'make install*' for DIRDEPS_BUILD but also
revealed that some other targets were not guaranteed to be created if
there was a SUBDIR defined.  One example is 'installfiles' was never
defined if SUBDIR was not empty.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 07:54:19 +00:00
Bryan Drewery
0368d219ac The .if redirection on .WAIT is no longer needed with bmake.
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 07:54:16 +00:00
Bryan Drewery
85874b565c Fix 'afterinstall' order not being respected after my changes in r291635.
The problem was that 'afterinstall' was not coming after SUBDIRs were
installed which was the expectation at least in sys/modules for kldxref.

Reported by:	np
Pointyhat to:	bdrewery
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 07:54:13 +00:00
Bryan Drewery
b4f9ba38ee Rearrange some common logic. 2015-12-04 07:54:04 +00:00
Bryan Drewery
7b861220df Add assertion for when LIBADD should be used rather than LDADD/DPADD.
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:59 +00:00
Bryan Drewery
eacae6dc66 Fix LDADD/DPADD that should be LIBADD.
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:47 +00:00
Bryan Drewery
2fd6394d34 Rework unknown LIBADD assertion to be more clear and to not suggest adding
DPADD/LDADD_<foo> variables that are a special case.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:24 +00:00
Bryan Drewery
3530922240 Support all of the CDDL/ZFS libraries for LIBADD.
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:21 +00:00
Bryan Drewery
7aab86d59d For INTERNALLIB always add in the corresponding _DP_ and use LIBADD in
the real build file.

This lessens the need to define DPADD_<lib> and LDADD_<lib> to just very
special cases.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:17 +00:00
Bryan Drewery
a9f9ec2435 Replace ln -s calls with INSTALL_SYMLINK
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:14 +00:00
Bryan Drewery
568f2ee158 Don't create a Makefile.depend in share/mk.
This would cause it to be included everywhere in the build since it is
the MAKESYSPATH.  This leads to including dirdeps.mk more times than
desired.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:10 +00:00
Bryan Drewery
249f51a492 DIRDEPS_BUILD: Install new Makefile.depend files atomically.
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:07 +00:00
Bryan Drewery
bd1944cacc DIRDEPS_BUILD: For the bootstrapped LIBADD from DPADD, resolve paths to RELDIR.
This allows the LIBDEPS/DPADD for the clang build to not have
../../../lib/clang/* in DIRDEPS.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:04 +00:00
Bryan Drewery
d6b837de97 Revert r288966 as it is redundant and not right.
bsd.prog.mk and bsd.lib.mk already make OBJS depend on headers when there is
not .OBJDIR/.depend file, which is still true for the initial meta mode builds.
If there was something to benefit the meta mode build here then it should be
extended to the non-meta mode build as well.

Some of the problems here were just DPSRCS being hooked up wrongly, fixed in
r291330.

The logic itself is flawed as 'buildfiles' is in a different part of the
dependency tree than the objects and headers are, so the objects will still be
built independent from 'buildfiles'.  'buildfiles' is not ordered in the build
before objects.

Sponsored by:	EMC / Isilon Storage Division
2015-12-03 22:39:42 +00:00
Kenneth D. Merry
a9934668aa Add asynchronous command support to the pass(4) driver, and the new
camdd(8) utility.

CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and
completed CCBs may be retrieved via the CAMIOGET ioctl.  User
processes can use poll(2) or kevent(2) to get notification when
I/O has completed.

While the existing CAMIOCOMMAND blocking ioctl interface only
supports user virtual data pointers in a CCB (generally only
one per CCB), the new CAMIOQUEUE ioctl supports user virtual and
physical address pointers, as well as user virtual and physical
scatter/gather lists.  This allows user applications to have more
flexibility in their data handling operations.

Kernel memory for data transferred via the queued interface is
allocated from the zone allocator in MAXPHYS sized chunks, and user
data is copied in and out.  This is likely faster than the
vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in
configurations with many processors (there are more TLB shootdowns
caused by the mapping/unmapping operation) but may not be as fast
as running with unmapped I/O.

The new memory handling model for user requests also allows
applications to send CCBs with request sizes that are larger than
MAXPHYS.  The pass(4) driver now limits queued requests to the I/O
size listed by the SIM driver in the maxio field in the Path
Inquiry (XPT_PATH_INQ) CCB.

There are some things things would be good to add:

1. Come up with a way to do unmapped I/O on multiple buffers.
   Currently the unmapped I/O interface operates on a struct bio,
   which includes only one address and length.  It would be nice
   to be able to send an unmapped scatter/gather list down to
   busdma.  This would allow eliminating the copy we currently do
   for data.

2. Add an ioctl to list currently outstanding CCBs in the various
   queues.

3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do
   that.

4. Test physical address support.  Virtual pointers and scatter
   gather lists have been tested, but I have not yet tested
   physical addresses or scatter/gather lists.

5. Investigate multiple queue support.  At the moment there is one
   queue of commands per pass(4) device.  If multiple processes
   open the device, they will submit I/O into the same queue and
   get events for the same completions.  This is probably the right
   model for most applications, but it is something that could be
   changed later on.

Also, add a new utility, camdd(8) that uses the asynchronous pass(4)
driver interface.

This utility is intended to be a basic data transfer/copy utility,
a simple benchmark utility, and an example of how to use the
asynchronous pass(4) interface.

It can copy data to and from pass(4) devices using any target queue
depth, starting offset and blocksize for the input and ouptut devices.
It currently only supports SCSI devices, but could be easily extended
to support ATA devices.

It can also copy data to and from regular files, block devices, tape
devices, pipes, stdin, and stdout.  It does not support queueing
multiple commands to any of those targets, since it uses the standard
read(2)/write(2)/writev(2)/readv(2) system calls.

The I/O is done by two threads, one for the reader and one for the
writer.  The reader thread sends completed read requests to the
writer thread in strictly sequential order, even if they complete
out of order.  That could be modified later on for random I/O patterns
or slightly out of order I/O.

camdd(8) uses kqueue(2)/kevent(2) to get I/O completion events from
the pass(4) driver and also to send request notifications internally.

For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR)
per CAM CCB on the reading side, and a scatter/gather list
(CAM_DATA_SG) on the writing side.  In addition to testing both
interfaces, this makes any potential reblocking of I/O easier.  No
data is copied between the reader and the writer, but rather the
reader's buffers are split into multiple I/O requests or combined
into a single I/O request depending on the input and output blocksize.

For the file I/O path, camdd(8) also uses a single buffer (read(2),
write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list
(readv(2), writev(2), preadv(2), pwritev(2)) on writes.

Things that would be nice to do for camdd(8) eventually:

1.  Add support for I/O pattern generation.  Patterns like all
    zeros, all ones, LBA-based patterns, random patterns, etc. Right
    Now you can always use /dev/zero, /dev/random, etc.

2.  Add support for a "sink" mode, so we do only reads with no
    writes.  Right now, you can use /dev/null.

3.  Add support for automatic queue depth probing, so that we can
    figure out the right queue depth on the input and output side
    for maximum throughput.  At the moment it defaults to 6.

4.  Add support for SATA device passthrough I/O.

5.  Add support for random LBAs and/or lengths on the input and
    output sides.

6.  Track average per-I/O latency and busy time.  The busy time
    and latency could also feed in to the automatic queue depth
    determination.

sys/cam/scsi/scsi_pass.h:
	Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue
	and fetch asynchronous CAM CCBs respectively.

	Although these ioctls do not have a declared argument, they
	both take a union ccb pointer.  If we declare a size here,
	the ioctl code in sys/kern/sys_generic.c will malloc and free
	a buffer for either the CCB or the CCB pointer (depending on
	how it is declared).  Since we have to keep a copy of the
	CCB (which is fairly large) anyway, having the ioctl malloc
	and free a CCB for each call is wasteful.

sys/cam/scsi/scsi_pass.c:
	Add asynchronous CCB support.

	Add two new ioctls, CAMIOQUEUE and CAMIOGET.

	CAMIOQUEUE adds a CCB to the incoming queue.  The CCB is
	executed immediately (and moved to the active queue) if it
	is an immediate CCB, but otherwise it will be executed
	in passstart() when a CCB is available from the transport layer.

	When CCBs are completed (because they are immediate or
	passdone() if they are queued), they are put on the done
	queue.

	If we get the final close on the device before all pending
	I/O is complete, all active I/O is moved to the abandoned
	queue and we increment the peripheral reference count so
	that the peripheral driver instance doesn't go away before
	all pending I/O is done.

	The new passcreatezone() function is called on the first
	call to the CAMIOQUEUE ioctl on a given device to allocate
	the UMA zones for I/O requests and S/G list buffers.  This
	may be good to move off to a taskqueue at some point.
	The new passmemsetup() function allocates memory and
	scatter/gather lists to hold the user's data, and copies
	in any data that needs to be written.  For virtual pointers
	(CAM_DATA_VADDR), the kernel buffer is malloced from the
	new pass(4) driver malloc bucket.  For virtual
	scatter/gather lists (CAM_DATA_SG), buffers are allocated
	from a new per-pass(9) UMA zone in MAXPHYS-sized chunks.
	Physical pointers are passed in unchanged.  We have support
	for up to 16 scatter/gather segments (for the user and
	kernel S/G lists) in the default struct pass_io_req, so
	requests with longer S/G lists require an extra kernel malloc.

	The new passcopysglist() function copies a user scatter/gather
	list to a kernel scatter/gather list.  The number of elements
	in each list may be different, but (obviously) the amount of data
	stored has to be identical.

	The new passmemdone() function copies data out for the
	CAM_DATA_VADDR and CAM_DATA_SG cases.

	The new passiocleanup() function restores data pointers in
	user CCBs and frees memory.

	Add new functions to support kqueue(2)/kevent(2):

	passreadfilt() tells kevent whether or not the done
	queue is empty.

	passkqfilter() adds a knote to our list.

	passreadfiltdetach() removes a knote from our list.

	Add a new function, passpoll(), for poll(2)/select(2)
	to use.

	Add devstat(9) support for the queued CCB path.

sys/cam/ata/ata_da.c:
	Add support for the BIO_VLIST bio type.

sys/cam/cam_ccb.h:
	Add a new enumeration for the xflags field in the CCB header.
	(This doesn't change the CCB header, just adds an enumeration to
	use.)

sys/cam/cam_xpt.c:
	Add a new function, xpt_setup_ccb_flags(), that allows specifying
	CCB flags.

sys/cam/cam_xpt.h:
	Add a prototype for xpt_setup_ccb_flags().

sys/cam/scsi/scsi_da.c:
	Add support for BIO_VLIST.

sys/dev/md/md.c:
	Add BIO_VLIST support to md(4).

sys/geom/geom_disk.c:
	Add BIO_VLIST support to the GEOM disk class.  Re-factor the I/O size
	limiting code in g_disk_start() a bit.

sys/kern/subr_bus_dma.c:
	Change _bus_dmamap_load_vlist() to take a starting offset and
	length.

	Add a new function, _bus_dmamap_load_pages(), that will load a list
	of physical pages starting at an offset.

	Update _bus_dmamap_load_bio() to allow loading BIO_VLIST bios.
	Allow unmapped I/O to start at an offset.

sys/kern/subr_uio.c:
	Add two new functions, physcopyin_vlist() and physcopyout_vlist().

sys/pc98/include/bus.h:
	Guard kernel-only parts of the pc98 machine/bus.h header with
	#ifdef _KERNEL.

	This allows userland programs to include <machine/bus.h> to get the
	definition of bus_addr_t and bus_size_t.

sys/sys/bio.h:
	Add a new bio flag, BIO_VLIST.

sys/sys/uio.h:
	Add prototypes for physcopyin_vlist() and physcopyout_vlist().

share/man/man4/pass.4:
	Document the CAMIOQUEUE and CAMIOGET ioctls.

usr.sbin/Makefile:
	Add camdd.

usr.sbin/camdd/Makefile:
	Add a makefile for camdd(8).

usr.sbin/camdd/camdd.8:
	Man page for camdd(8).

usr.sbin/camdd/camdd.c:
	The new camdd(8) utility.

Sponsored by:	Spectra Logic
MFC after:	1 week
2015-12-03 20:54:55 +00:00
Hans Petter Selasky
5b6a9ce8b3 Update the mlx5en(4) manual page.
MFC after:	1 week
Submitted by:	Mark Bloch <markb@mellanox.com>
Sponsored by:	Mellanox Technologies
Differential Revision:	https://reviews.freebsd.org/D4348
2015-12-03 10:17:01 +00:00
Enji Cooper
1c4ced8fe8 Fix a typo in a comment (spacial -> special)
Sponsored by: EMC / Isilon Storage Division
2015-12-03 07:42:00 +00:00
John Baldwin
fe2ebb7644 Add support for configuring additional virtual interfaces (VIs) on a port.
Each virtual interface has its own MAC address, queues, and statistics.
The dedicated netmap interfaces (ncxgbeX / ncxlX) were already implemented
as additional VIs on each port.  This change allows additional non-netmap
interfaces to be configured on each port.  Additional virtual interfaces
use the naming scheme vcxgbeX or vcxlX.

Additional VIs are enabled by setting the hw.cxgbe.num_vis tunable to a
value greater than 1 before loading the cxgbe(4) or cxl(4) driver.
NB: The first VI on each port is the "main" interface (cxgbeX or cxlX).

T4/T5 NICs provide a limited number of MAC addresses for each physical port.
As a result, a maximum of six VIs can be configured on each port (including
the "main" interface and the netmap interface when netmap is enabled).

One user-visible result is that when netmap is enabled, packets received
or transmitted via the netmap interface are no longer counted in the stats
for the "main" interface, but are not accounted to the netmap interface.

The netmap interfaces now also have a new-bus device and export various
information sysctl nodes via dev.n(cxgbe|cxl).X.

The cxgbetool 'clearstats' command clears the stats for all VIs on the
specified port along with the port's stats.  There is currently no way to
clear the stats of an individual VI.

Reviewed by:	np
MFC after:	1 month
Sponsored by:	Chelsio
2015-12-03 00:02:01 +00:00
John Baldwin
8d7e0f5889 The cdevpriv_dtr_t typedef was not able to be used in a function prototype
like the various d_*_t typedefs since it declared a function pointer rather
than a function.  Add a new d_priv_dtor_t typedef that declares the function
and can be used as a function prototype.  The previous typedef wasn't
useful outside of the cdevpriv implementation, so retire it.

The name d_priv_dtor_t was chosen to be more consistent with cdev methods
since it is commonly used in place of d_close_t even though it is not a
direct pointer in struct cdevsw.

Reviewed by:	kib, imp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D4340
2015-12-02 18:27:30 +00:00
Bryan Drewery
4b9378e3ba Add LIBNANDFS.
Sponsored by:	EMC / Isilon Storage Division
2015-12-02 05:23:12 +00:00
Bryan Drewery
03173d2ff0 bsd.subdir.mk: Only recurse on called targets, rather than dependencies.
This is to fix 'make all' causing it to recurse on both 'all' and 'buildconfig'
due to 'buildconfig' being in ALL_SUBDIR_TARGETS and being a dependency of
'all'.

This now adds all of the '*includes', '*files' targets as subdir targets,
allowing them to recurse.

This also removes the need for some 'realinstall' hacks in bsd.subdir.mk since
it no longer recurses; only 'install' will recurse and call the proper
'beforeinstall', 'realinstall', and 'afterinstall' in each sub-directory.

This fixes 'make includes' and 'make files' to not be a rerolled ${MAKE}
sub-shell but to rather just recurse on 'inclues' and 'files'.  This avoids
various issues such as the one fixed in r289462.  As such revert Makefile.inc1
back to using 'includes' which avoids an extra tree walk and parallelizes
the includes phases better.

Makefile.inc1 includes a guard so that 'make all' will not use SUBDIR_PARALLEL,
added in r289438.  This is so users do not get a probably broken build if they
run 'make all' from the top-level.  Before the change in this commit, the
workaround for 'make everything' was 'par-all' which would depend on 'all' and
cause a proper parallel recursion.  Now that will not work so a new
_PARALLEL_SUBUDIR_OK is used to allow it.

This is still part of an effort to combine bsd.(files|incs|confs).mk and move
some of its logic out of bsd.subdir.mk, as attempted in r289282 and reverted in
r289331.  This commit fixes the problems found there which was mostly double
recursing during 'includes' which would recurse on itself and 'buildincludes'
and 'installincludes', all in parallel.  The logic is still in bsd.subdir.mk
for now.

I've been cautious about this commit but have experienced no breakage on the
tree except for the 'par-all' case which was already a hack.  If something foo
is depending on something bar that should recurse, it is very likely that the
foo target is being recursed on already meaning that bar will still effectively
recurse once sub-directories call foo.

Discussed on:	arch@
MFC after:	never
Sponsored by:	EMC / Isilon Storage Division
2015-12-02 01:50:22 +00:00
Bryan Drewery
70d3d39e78 Revert r291633. Some files were missed. 2015-12-02 01:49:22 +00:00
Bryan Drewery
2cdf1c99f0 bsd.subdir.mk: Only recurse on called targets, rather than dependencies.
This is to fix 'make all' causing it to recurse on both 'all' and 'buildconfig'
due to 'buildconfig' being in ALL_SUBDIR_TARGETS and being a dependency of
'all'.

This now adds all of the '*includes', '*files' targets as subdir targets,
allowing them to recurse.

This also removes the need for some 'realinstall' hacks in bsd.subdir.mk since
it no longer recurses; only 'install' will recurse and call the proper
'beforeinstall', 'realinstall', and 'afterinstall' in each sub-directory.

This fixes 'make includes' and 'make files' to not be a rerolled ${MAKE}
sub-shell but to rather just recurse on 'inclues' and 'files'.  This avoids
various issues such as the one fixed in r289462.  As such revert Makefile.inc1
back to using 'includes' which avoids an extra tree walk and parallelizes
the includes phases better.

Makefile.inc1 includes a guard so that 'make all' will not use SUBDIR_PARALLEL,
added in r289438.  This is so users do not get a probably broken build if they
run 'make all' from the top-level.  Before the change in this commit, the
workaround for 'make everything' was 'par-all' which would depend on 'all' and
cause a proper parallel recursion.  Now that will not work so a new
_PARALLEL_SUBUDIR_OK is used to allow it.

This is still part of an effort to combine bsd.(files|incs|confs).mk and move
some of its logic out of bsd.subdir.mk, as attempted in r289282 and reverted in
r289331.  This commit fixes the problems found there which was mostly double
recursing during 'includes' which would recurse on itself and 'buildincludes'
and 'installincludes', all in parallel.  The logic is still in bsd.subdir.mk
for now.

I've been cautious about this commit but have experienced no breakage on the
tree except for the 'par-all' case which was already a hack.  If something foo
is depending on something bar that should recurse, it is very likely that the
foo target is being recursed on already meaning that bar will still effectively
recurse once sub-directories call foo.

Discussed on:	arch@
MFC after:	never
Sponsored by:	EMC / Isilon Storage Division
2015-12-02 01:47:27 +00:00
Bryan Drewery
5afa19725e Add assertions that capture invalid configurations for new libraries.
Fix current findings, which should fix cases of NO_SHARED not building
properly.

Given libfoo:
 - Ensure that a LIBFOO is set.  For INTERNALLIBS advise setting this in
   src.libnames.mk, otherwise bsd.libnames.mk.
 - Ensure that a LIBFOODIR is properly set.
 - Ensure that _DP_foo is set and matches the LIBADD in the build of foo's own
   Makefile

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 01:24:31 +00:00
Bryan Drewery
eaab62402b Don't overlink pthread to kerberos library consumers.
I'm not sure why this was here, none of these use pthread themselves and
none of the consumers are broken with removing this.

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 01:23:16 +00:00
Bryan Drewery
7bce873919 Don't overlink pthread to libsqlite3 consumers.
At least usr.bin/mandoc was overlink.

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 01:17:09 +00:00
Bryan Drewery
3d07982db6 Fix truncation of _DP_proc and add missing libelf.
Sponsored by:	EMC / Isilon Storage Division
2015-12-02 00:54:05 +00:00
Bryan Drewery
60648601bf Reduce overlinking of libdtrace consumers with libctf, libelf, libproc.
The proper place for this list is _DP_dtrace.

Due to removing the LDADD_dtrace, more LIBADD are needed in
cddl/usr.sbin/dtrace to prevent underlinking.

This fixes overlinking in cddl/usr.sbin/lockstat and
cddl/usr.sbin/plockstat.

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 00:49:49 +00:00
Bryan Drewery
ed8addefb2 Don't overlink libmt consumers with libsbuf.
This change came in r281332 which was reducing overlinking in mt(1) but
currently mt(1) is linked with sbuf when it does not need it due to the
LDADD_mt+=${LDADD_sbuf}.  Only libmt needs sbuf.

Add sbuf to _DP_mt so static linkage of libmt picks it up.

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 00:49:42 +00:00
Bryan Drewery
70b19675cd libssh uses libz even without MK_LDNS
Sponsored by:	EMC / Isilon Storage Division
2015-12-02 00:49:39 +00:00
Bryan Drewery
87a2755dee Revert incomplete r291623. 2015-12-02 00:17:13 +00:00
Bryan Drewery
eaa435e794 Add assertions that capture invalid configurations for new libraries.
Fix current findings.

Given libfoo:
 - Ensure that a LIBFOO is set.  For INTERNALLIBS advise setting this in
   src.libnames.mk, otherwise bsd.libnames.mk.
 - Ensure that a LIBFOODIR is properly set.
 - Ensure that _DP_foo is set and matches the LIBADD in the build of foo's own
   Makefile

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 00:01:09 +00:00
Bryan Drewery
de52e5bdca Define a LIB*SRCDIR for all known _LIBRARIES.
This is a follow-up to r291327 which added a LIB*DIR for all known
_LIBRARIES.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 22:19:17 +00:00
Bryan Drewery
945ceaafbc Only include src.conf if _WITHOUT_SRCCONF not defined.
This does not really fix anything currently since _WITHOUT_SRCCONF must be
defined in the environment or local.sys.*.mk, but is proper and needed for
downstream fixes.  I am working towards reworking src.conf inclusion still.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 20:56:16 +00:00
Bryan Drewery
f3dd2bc756 Support LOCAL_LIBRARIES for LIBADD.
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 20:50:14 +00:00
Bryan Drewery
4910373ca8 Add missing LIB80211 entry for DPADD needs.
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 18:09:40 +00:00
Bryan Drewery
3a42764af7 Fix underlinking in lib80211 and define static dependencies in src.libnames.mk
so NO_SHARED works properly.

Reported by:	Manfred Antar <null@pozo.com>
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 17:38:52 +00:00
Bryan Drewery
6feede08e1 FAST_DEPEND: Actually use -MP from DEPEND_MP, completing r291554.
X-MFC-With:	r291554
MFC after:	1 week
Pointyhat to:	bdrewery
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 15:50:35 +00:00
Bryan Drewery
4902050fa5 META MODE: Add some basic bootstrapping support for no Makefile.depend.
This will not work for bootstrapping dependencies, it will only
bootstrap the top-level build directory.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:29:30 +00:00
Bryan Drewery
b1f92fa229 META MODE: Update dependencies with 'the-lot' and add missing directories.
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so.  A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR.  These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package.  The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:23:19 +00:00
Bryan Drewery
3bed45ab81 META MODE: Rework [bootstrapped] tools PATH support.
- Support more of the toolchain from TOOLSDIR.
- This also improves 'make bootstrap-tools' to pass, for example,
  AS=/usr/bin/as to Makefile.inc1, which will tell cross-tools to use
  external toolchain support and avoid building things we won't be using
  in the build.
- Always set the PATH to contain the staged TOOLSDIR directories when
  not building the bootstrap targets.

  The previous version was only setting this at MAKE.LEVEL==0 and if the
  TOOLSDIR existed.  Both of these prevented using staged tools that were
  built during the build though as DIRDEPS with .host dependencies, such
  as the fix for needing usr.bin/localedef.host in r291311.
  This is not a common tool so we must build and use it during the build,
  and need to be prepared to change PATH as soon as it appears.

  This should also fix the issue of host dependencies disappearing from
  Makefile.depend and then reappearing due to the start of the fresh build not
  having the directory yet, resulting in the tools that were built not actually
  being used.
- Only use LEGACY_TOOLS while building in Makefile.inc1.  After r291317
  and r291546 there is no need to add LEGACY_TOOLS into the PATH for
  the pseudo/targets/toolchain build.
- Because the pseudo/targets/toolchain will now build its own
  [clang-]tblgen, the special logic in clang.build.mk is no longer needed.
- LEGACY_TOOLS is no longer used outside of targets/pseudo/bootstrap-tools
  so is no longer passed into the environment in its build.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:19:02 +00:00
Bryan Drewery
3c42abd89d META MODE: Don't trim out lib/clang/include dependency.
Doing this causes more trouble than it is worth regarding cyclic
dependencies.  It should not be needed after cleaning up MACHINE=host
builds in r291324.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:18:55 +00:00
Kevin Lo
e1b74f21f5 Add initial support for RTL8152 USB Fast Ethernet. RTL8152 supports
IPv4/IPv6 checksum offloading and VLAN tag insertion/stripping.

Since uether doesn't provide a way to announce driver specific offload
capabilities to upper stack, checksum offloading support needs more work
and will be done in the future.

Special thanks to Hayes Wang from RealTek who gave input.
2015-12-01 05:12:13 +00:00
Bryan Drewery
b6725ecdb2 FAST_DEPEND: Workaround hacks in the wild that have '..' or '/' in OBJS.
This is mostly working around the converts/iconv port having '../ces/file.o'
in its OBJS list which resulted in '.depend../ces/file.o'.  Now it will have
'.depend.._ces_file.o'.

Other implementations have :T which would result in '.depend.file.o' here, but
that could lead to collisions.

X-MFC-With:	r291554
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 03:09:54 +00:00
Bryan Drewery
593e465913 FAST_DEPEND: Allow not using -MP by setting DEPEND_MP blank.
-MP creates empty targets for all dependency files, which can be useful when a
dependency is deleted from the file system.  This would otherwise cause an
error for "don't know how to build FOO" since the .depend file is included
with the dependency registered.

This is mostly a workaround for the misc/dahdi-kmod port using '::' for one of
its dependencies, while -MP uses just ':'.  This results in an 'Inconsistent
operator for' error.

X-MFC-With:	r290433
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 03:00:10 +00:00
Bryan Drewery
c7f8dda118 META MODE: Always define HOST_*, even if not using them.
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 00:53:17 +00:00
Alexander Motin
aeef6b689a Use SPI name for parallel SCSI. 2015-11-30 22:09:55 +00:00
Adrian Chadd
5c15e5d044 Add missing lib declaration. 2015-11-30 06:56:25 +00:00
Hajimu UMEMOTO
26b79d5b83 Regen src.conf.5 for recent option changes. 2015-11-28 00:55:49 +00:00
John Baldwin
7f911abe54 Add support to libkvm for reading vmcores from other architectures.
- Add a kvaddr_type to represent kernel virtual addresses instead of
  unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
  that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
  is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles().  It drops the
  unused 'swapfile' argument and adds a new function pointer argument for
  a symbol resolving function.  Native kernels still use _fdnlist() from
  libc to resolve symbols if a resolver function is not supplied, but cross
  kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
  struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
  kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
  Each backend is responsible for implementing kvm_read2() for a given
  vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
  powerpc cores).
- Add internal helper routines for the common page offset hash table used
  by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
  to be cross-friendly by using private constants instead of ones that
  vary by platform (e.g. PAGE_SIZE).  Static assertions are present when
  a given backend is compiled natively to ensure the private constants
  match the real ones.
- Enable all of the existing vmcore backends on all platforms.  This means
  that libkvm on any platform should be able to perform KVA translation
  and read data from a vmcore of any platform.

Tested on:	amd64, i386, sparc64 (marius)
Differential Revision:	https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
Bryan Drewery
11ffa20fa0 Standardize on OBJTOP in and outside of META MODE.
Sponsored by:	EMC / Isilon Storage Division
2015-11-26 01:47:56 +00:00
Bryan Drewery
93e779a26c META MODE: These need object directories to handle staging.
Sponsored by:	EMC / Isilon Storage Division
2015-11-26 01:14:27 +00:00
George V. Neville-Neil
1576e8f871 Replace the retval.sh shell script with a native DTrace script.
Suggested by: markj
2015-11-26 00:53:39 +00:00
George V. Neville-Neil
669f9224ec Summary: A simple script to print the return value of any function,
with or without wild cards.
2015-11-25 22:59:41 +00:00
Bryan Drewery
8ffe95d60f Follow-up r291338 to handle .d, .y and .l files better as well.
X-MFC-With:	r291338
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 22:40:14 +00:00
Bryan Drewery
c3d4829ed1 Fix the "common object" handling to not depend on ".o" if SRCS only contains
headers.

This resulted in 'don't know how to make .o.' errors after the changes in
r289286.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 22:32:16 +00:00
Bryan Drewery
ebf076dfa5 Define a LIB<NAME>DIR for every library that LIBADD provides.
This is going to be used to allow DIRDEPS to be bootstrapped off of
LIBADD/DPADD.  It currently works for internal libraries which have a
DIR defined for them but also use the .a library from a src-mapped obj
directory.  It can also be useful for using -L without a --sysroot per
LIBADD to use the OBJDIR version of the libraries.

I didn't review every LIBADD, so it is possible this is missing some.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:49:55 +00:00
Bryan Drewery
2802302e64 META MODE: Fix MACHINE=host builds.
We need to not use -nostdinc since it breaks building of clang itself.  Use
-isystem rather than -I/usr/include and -nostdinc which gets us using
the stage include directory before searching the real host headers.
This allows removing more of the -I hacks to get host headers since the
headers are no longer excluded.  The -B seemed unneeded.

This fixes building of secure/lib/libcrypto which was looking at the
/usr/include/openssl/asn1.h header rather than the staged one.

This fixes building of clang which wants to find its own internal
headers in the STAGEDIR/usr/lib/clang/* path.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:45:01 +00:00
Bryan Drewery
cf10dd450d Stop looking up the same id(1) results in sub-makes.
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:58 +00:00
Bryan Drewery
4aa63711a7 META MODE: Fix 'make bootstrap-tools'.
The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.

This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.

For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds.  This has not yet been a problem due
to readelf not being built as a host tool in buildworld.  This is possible
in the meta build though when building the toolchain.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:51 +00:00
Bryan Drewery
b791fbe630 META MODE: Don't create .meta files when symlinking sources into the obj directory.
Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file.  There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:43 +00:00
Bryan Drewery
80dfd0d4a2 META MODE: Show PATH on errors.
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:31 +00:00
Bryan Drewery
08b20c637c META MODE: Don't add staged DPADDs to DIRDEPS when bootstrapping gendirdeps.
This was finding libraries that were installed into DESTDIR/usr/lib,
where DESTDIR is the stage directory, and then adding in usr/lib to
DIRDEPS.  Just exclude the STAGE_ROOT if defined.

Discussed with:	sjg
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:13:45 +00:00
Bryan Drewery
5e2f7161d7 META MODE: Define variables this file uses that were defined elsewhere.
Discussed with:	sjg
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:13:42 +00:00
Bryan Drewery
02d37cd3e4 META MODE: Don't truncate DIRDEPS for bootstrapping during gendirdeps.
Reviewed by:	sjg
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:13:39 +00:00
Bryan Drewery
bb8cd0c6f0 META MODE: Revert r287879 so that 'make -V' still uses AUTO_OBJ.
This has caused much confusion for myself as there are quite a lot of
variables that depend on having a proper ${.OBJDIR}.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:13:32 +00:00
Bryan Drewery
393608176b META MODE: Fix 'make the-lot' with recent locale changes
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:13:28 +00:00
Bryan Drewery
7b3ea376a2 META MODE: Prefer INSTALL=tools/install.sh to lessen the need for xinstall.host.
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:10:28 +00:00
Kevin Lo
ff6b30b9fa Add dependency to uether.
Reviewed by:	hselasky
2015-11-24 08:34:48 +00:00
Bryan Drewery
440ffb7e1e There seems to be no reason to duplicate CANONICALOBJDIR logic from bsd.obj.mk,
which is included for crunchgen builds.

No change in build output occurred with this change.

Sponsored by:	EMC / Isilon Storage Division
2015-11-23 23:04:52 +00:00
Alexander Motin
e3a0bc583b Remove "disable" hint, which duplicates system-wide "disabled". 2015-11-23 20:44:49 +00:00
Enji Cooper
ba23388ab8 Revert r291170
The mlx5* driver(s) are built [*]/installed separate from the OFED stack thanks
to recent refactoring done in the linuxkpi(4) module.

Always install the manpages instead of conditionally installing them if
MK_OFED != no

* Further refactoring of sys/ofed and linuxkpi(4) is pending to fully divorce
  mlx5* from ofed headers

MFC after: never
Requested by: hps
2015-11-23 19:44:39 +00:00
Alexander Motin
a4ccb5d682 Fix target mode support for Qlogic 2200 FC adapters.
Now target mode works for all supported FC adapters except ancient 2100,
which is not tested.
2015-11-23 15:49:50 +00:00
Dag-Erling Smørgrav
1765946ba9 Retire the NONE cipher option. 2015-11-23 12:48:13 +00:00
Alexander Motin
3e6deb330e Rip off target mode support for parallel SCSI QLogic adapters.
Hacks to enable target mode there complicated code, while didn't really
work.  And for outdated hardware fixing it is not really interesting.

Initiator mode tested with Qlogic 1080 adapter is still working fine.
2015-11-23 10:06:19 +00:00
Enji Cooper
c80e2a5d2b Install mce(4) and mlx5en(4) if MK_OFED != no
MFC after: 1 week
2015-11-23 00:43:23 +00:00
Enji Cooper
0c55686cd0 Bump .Dd 2015-11-23 00:40:08 +00:00
Enji Cooper
62d309bb84 - Fix bad double space between HW and LRO
- Fix improperly capitalized `interface`

MFC after: 1 week
2015-11-23 00:39:48 +00:00
Enji Cooper
6c4edcb26a Bump .Dd 2015-11-22 23:16:15 +00:00
Enji Cooper
d1241b1501 Recommend cc -Wall instead of gcc -Wall
MFC after: 1 week
2015-11-22 23:15:44 +00:00
Alexander Motin
6d2a1fbf23 Add API to obtain primary enclosure name and ID for /dev/sesX devices.
sesX device number may change between reboots, so to properly identify
the instance we need more data.  Name and ID reported here may mach ones
reported by SCSI device, but that is not really required by specs.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2015-11-21 10:22:01 +00:00
Brad Davis
59dad225f1 Remove a link to the now defunct "Release Engineering of Third Party Packages".
PR:		202803
Submitted by:	Tobias Kortkamp <tobias.kortkamp@gmail.com>
MFC after:	1 week
2015-11-21 00:35:40 +00:00
Warner Losh
dbd69b0dc4 Add aarch64 support to CPUTYPE 2015-11-20 21:49:49 +00:00
Ravi Pokala
e38d647a53 Add myself (rpokala) and my mentor (jhb)
Approved by:	jhb (mentor)
Differential Revision:	https://reviews.freebsd.org/D4231
2015-11-20 21:03:04 +00:00
Bryan Drewery
1da1353262 META MODE: Don't stage INTERNALPROGs.
This is only for sys/boot.  INTERNALPROG is not a build tool, it is just a way
to generate OBJS from a list of SRCS and use those objects elsewhere.

Sponsored by:	EMC / Isilon Storage Division
2015-11-20 00:22:55 +00:00
Bryan Drewery
86dd299005 Remove unsupported ia64.
Sponsored by:	EMC / Isilon Storage Division
2015-11-19 22:54:37 +00:00
Warner Losh
828b0885d9 Add arm CPUTYPE values typically used on FreeBSD. 2015-11-19 17:06:12 +00:00
Jonathan T. Looney
1067a2ba68 Consistently enforce the restriction against calling malloc/free when in a
critical section.

uma_zalloc_arg()/uma_zalloc_free() may acquire a sleepable lock on the
zone. The malloc() family of functions may call uma_zalloc_arg() or
uma_zalloc_free().

The malloc(9) man page currently claims that free() will never sleep.
It also implies that the malloc() family of functions will not sleep
when called with M_NOWAIT. However, it is more correct to say that
these functions will not sleep indefinitely. Indeed, they may acquire
a sleepable lock. However, a developer may overlook this restriction
because the WITNESS check that catches attempts to call the malloc()
family of functions within a critical section is inconsistenly
applied.

This change clarifies the language of the malloc(9) man page to clarify
the restriction against calling the malloc() family of functions
while in a critical section or holding a spin lock. It also adds
KASSERTs at appropriate points to make the enforcement of this
restriction more consistent.

PR:		204633
Differential Revision:	https://reviews.freebsd.org/D4197
Reviewed by:	markj
Approved by:	gnn (mentor)
Sponsored by:	Juniper Networks
2015-11-19 14:04:53 +00:00