Commit Graph

16843 Commits

Author SHA1 Message Date
bdrewery
493f9f8597 Rework r289778 to always parallelize known targets, without ordering.
- Rather than allow 'make clean*' to ignore dependencies, make a static
  list of targets in STANDALONE_SUBDIR_TARGETS that are known to be safe.
  This allows a user to override them if needed and avoids adding this feature
  to user-defined targets that are in ${SUBDIR_TARGETS}. [1]
- This now also allows to force SUBDIR_PARALLEL when calling these
  targets, since no dependencies are needed.

Reported by:	ian [1]
Sponsored by:	EMC / Isilon Storage Division
MFC after:	3 weeks
X-MFC-With:	r289778
2015-10-23 21:30:27 +00:00
bdrewery
fab547dc0c Fix regression from r289734 that caused crunchgen "subdirs" to not be
properly recursed.

The .for loop was defining a ${__dir} variable that was being set at a
different evaluation time than the target itself, so every 'cd ${__dir}'
became the last value that was in ${__dir}.  This resulted in 'make obj'
not properly being ran in the tree that would leave .depend files
scattered around when 'make all' was ran in rescue/.

To fix this, define a CRUNCH_SRCDIR_* for every prog if it does not
already have one and then use that variable in every relevant place.
This allows simplifying some logic as well.

Reported by:	emaste
X-MFC-With:	r289734
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-23 19:41:58 +00:00
cem
f317ccd8a1 sysctl(9): Document U8/U16 types from r289773
Suggested by:	ngie
Sponsored by:	EMC / Isilon Storage Division
2015-10-23 15:08:16 +00:00
bdrewery
7761ab52bc For SUBDIR_PARALLEL, when doing 'make clean*' or 'make obj' there is no need to
respect SUBDIR_DEPEND_* or .WAIT.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 23:41:56 +00:00
bdrewery
a305a56ce1 Fix installation of manpages with WITHOUT_COMPRESS broken since r284685.
This partially reverts r284685.  An attempt was made in r285295 to fix this
but was not enough.  There were still $${} vars in the code that should have
been using the ${_page} and ${_sect} vars, but the bigger problem was that
.for cannot be used on .ALLSRC as it is not defined when the .for is evaluated.

Using ${MAN} here in a .for loop doesn't work either as the paths are not
expanded right for lib/libc/ subdirs despite having a .PATH set for all
of them.

Add some comments around long .else and .endif as well.

Sponsored by:	EMC / Isilon Storage Division
2015-10-22 22:29:25 +00:00
cem
e7b0d63251 getenv.9: Document behavior of freeenv(NULL)
Fix a couple igor(1)-indicated spelling issues while here.

Suggested by:	jhb
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 16:46:30 +00:00
ed
e0d7cb1063 Add support for CloudABI on ARM64.
It turns out that it is pretty easy to make CloudABI work on ARM64. We
essentially only need to copy over the sysvec from AMD64 and ensure that
we use ARM64 specific registers.

As there is an overlap between function argument and return registers,
we do need to extend cloudabi64_schedtail() to only set its values if
we're actually forking. Not when we're creating a new thread.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D3917
2015-10-22 11:09:25 +00:00
bdrewery
a1c60621aa Fix incorrect defined() usage from style clean up in r289735.
Submitted by:	ngie
MFC after:	2 weeks
X-MFC-With:	r289735
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:52:37 +00:00
bdrewery
9a92f9cb47 Document that we use {} for variable expansion.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:50:52 +00:00
bdrewery
ee93b5fb91 Fix style. Namely use {} rather than ().
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:47:52 +00:00
bdrewery
9a27f36207 Rewrite crunchgen target handling for progs so that it can be parallelized.
This covers 'clean', 'cleandepend', 'cleandir', 'obj', 'objlink' and
'build-tools'.

This uses the same method as bsd.subdir.mk.

MFC after:	2 weeks
X-MFC-With:	r289731
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:42:17 +00:00
bdrewery
a2ebfeb0ad Clean up some bsd.crunchgen.mk issues.
- Remove handling of 'make -P' since that is for fmake only.
- Add '+' where appropriate for sub-make calls.
- Pass MK_TESTS=no to all of the sub-makes to prevent recursing into test
  directories for targets such as 'obj', 'clean', 'depend', etc.

Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:28:22 +00:00
bdrewery
f976589d2b Let SUBDIR_OVERRIDE with 'make buildworld' be more useful.
Now it can be used to effectively "build in a subdir".  It will use the
'cross-tools', 'libraries', and 'includes' phases of 'buildworld' to properly
setup a WORLDTMP to use.  Then it will build 'everything' only in the
listed SUBDIR_OVERRIDE directories.  It is still required to list custom
library directories in LOCAL_LIB_DIRS if SUBDIR_OVERRIDE is something
that contains libraries outside of the normal area (such as
SUBDIR_OVERRIDE=contrib/ofed needing LOCAL_LIB_DIRS=contrib/ofed/usr.lib)

Without these changes, SUBDIR_OVERRIDE with buildworld was broken or hit
obscure failures due to missing libraries, includes, or cross compiler.

SUBDIR_OVERRIDE with 'make <target that is not buildworld>' will continue to
work as it did before although its usefulness is questionable.

With a fully populated WORLDTMP, building with a SUBDIR_OVERRIDE with
-DNO_CLEAN only takes a few minutes to start building the target
directories.  This is still much better than building unneeded things via
'everything' when testing small subset changes.  A BUILDFAST or
SKIPWORLDTMP might make sense for this as well.

- Add in '_worldtmp' as we still need to create WORLDTMP as later targets,
  such as '_libraries' and '_includes' use it.  This probably was avoiding
  calling '_worldtmp' to not remove WORLDTMP for debugging purposes, but
  -DNO_CLEAN can be used for that.

- '_legacy' must be included since '_build-tools' uses -legacy.
  The SUBDIR_OVERRIDE change came in r95509, while -legacy being part
  of build-tools came in r113136.

- 'bootstrap-tools' is still skipped as this feature is not for
   upgrades.

- Fix buildworld combined with SUBDIR_OVERRIDE not installing all includes.

  The original change for SUBDIR_OVERRIDE in r95509 kept '_includes'
  and '_libraries' as building everything possible as the SUBDIR_OVERRIDE
  could need anything from them.  However in r96462 the real 'includes'
  target was changed from manual sub-makes to just recursing 'includes'
  on SUBDIR, thus not all includes have been installed into WORLDTMP since then
  when combined with 'buildworld'.

  This is not done unless calling 'make buildworld' as it would be
  unexpected to have it go into all directories when doing 'make
  SUBDIR_OVERRIDE=mydir includes'.

- Also need to build the cross-compiler so it is used with --sysroot.
  If this is burdensome then telling the build to use the local compiler
  as an external compiler (thus using a proper --sysroot to WORLDTMP) is
  possible by setting CC=/usr/bin/cc, CXX=/usr/bin/c++, etc.

- Don't build the lib32 distribution with SUBDIR_OVERRIDE in buildworld
  since it won't contain anything related to SUBDIR_OVERRIDE.  Testing
  of the lib32 build can be done with 'make build32'.

- Document these changes in build.7

Sponsored by:	EMC / Isilon Storage Division
MFC after:	2 weeks
2015-10-22 00:07:48 +00:00
avos
31af650a83 Add myself (avos) to committers-src.dot
Approved by:	adrian (mentor)
Differential Revision:	https://reviews.freebsd.org/D3941
2015-10-21 22:31:17 +00:00
bdrewery
82833332a3 Remove indirection of _sub target for using _SUBDIR.
This reverts r266473 as the need for it, working around .MAKE and '+'
issues, is no longer needed after r289460.  This avoids extra log
output in -j builds of '-- _sub.TARGET --' that are redundant with the
'-- TARGET --' and '-- TARGET_subdir_DIR --' entries already showing.

r266473 also made a subtle change in the ordering of _SUBDIR handling.  Before
the change, SUBDIRS were recursed into after building the TARGET due to the
.USE of _SUBDIR *appending* the commands onto the TARGET.  After the change
though the indirection caused TARGET to depend on _sub.TARGET which had the
_SUBDIR handling in it.  This TARGET would run after recursing.  However, the
SUBDIR_PARALLEL handling from r263778 has this ordering as well.  Since
this has so far not been a problem, for now make this behavior for
non-SUBDIR_PARALLEL use of _SUBDIR explicit by using .USEBEFORE.
Further research may change this back to .USE as well as the
SUBDIR_PARALLEL handling and bsd.progs.mk recursing.

Sponsored by:	EMC / Isilon Storage Division
2015-10-21 16:24:44 +00:00
bdrewery
9bff9cbf0c Fix building in a directory with SUBDIRs and SUBDIR_PARALLEL.
The SUBDIR_PARALLEL feature uses a .for dir in ${SUBDIR} loop.  The old code
here for recursing was setting SUBDIR= as a make *argument*.  The SUBDIR=
replacement was not actually handled until after the .for loop was unrolled.
This could be seen with a '.info ${SUBDIR} ${dir}' inside of the loop which
showed an empty ${SUBDIR} and a set ${dir}.  Setting NO_SUBIDR= before calling
${MAKE} as an *environment* variable handles the case fine and is a more
proper mechanism for disabling subdir handling.

This could be seen with 'make -C tests/sys/kern -j15 SUBDIR_PARALLEL=yes'.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-21 00:25:18 +00:00
cem
f7c12e6bb7 cpuset.9: Link to/from the new page
A follow-up to r289667.

Sponsored by:	EMC / Isilon Storage Division
2015-10-20 23:52:37 +00:00
cem
b7af74b7e7 Document cpuset(9)
A follow-up to r289467.

Coerced by:	jhb
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 23:48:14 +00:00
bdrewery
235f7c0a8e Improve safety of caching from r289659 by only importing of none of the
variables are already set.  This should cover odd cases such as the
COMPILER_TYPE override in lib/csu/powerpc64.

X-MFC-With:	r289659
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 20:37:00 +00:00
bdrewery
c197d1a9f7 Pass COMPILER_TYPE and COMPILER_VERSION to sub-makes to avoid redundant
lookups.

This uses a special variable name based on a hash of ${CC}, ${PATH}, and
${MACHINE} to ensure that a cached value is not used if any of these
values changes to use a new compiler.

Before this there were 34,620 fork/exec from bsd.compiler.mk during a buildworld.
After this there are 608.  More improvement is needed to cache a value from
the top-level before descending into subdirs in the various build phases.

Reviewed by:	brooks (earlier version)
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D3898
2015-10-20 20:15:25 +00:00
pluknet
1173dc8974 o NetBSD 7.0, OpenBSD 5.8, FreeBSD 10.2, OS X 10.11 added. 2015-10-20 11:37:16 +00:00
bdrewery
7e742cf0a0 Replace all of the duplicated logic for recursing into a subdir with one
implementation.  It is duplicated at run-time but is more easily
maintainable now.

Sponsored by:	EMC / Isilon Storage Division
2015-10-19 23:34:35 +00:00
bdrewery
dd315e14c5 Add missing .PHONY for parallel subdir target.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-19 22:27:32 +00:00
ngie
5c0da1966c Make libxo depend on libutil because it uses humanize_number after r287111
Remove overlinking in lib/libxo/tests, sbin/savecore, and
usr.bin/{iscsictl,wc,xo}

PR: 203673
Sponsored by: EMC / Isilon Storage Division
2015-10-18 07:30:50 +00:00
ngie
49d57a7487 Only enable -fstack-protector-strong on gcc 4.9+ and default to -fstack-protector
when -fstack-protector-strong is not available, like it was implicitly before
r288669

As noted by antoine@, devel/gcc (which is 4.8.5) lacks -fstack-protector-strong
support, whereas 4.8.4i (devel/gcc48) has the support.

Until a version is available which has -fstack-protector-strong support, be
conservative and only enable support with 4.9+.

Reviewed by: pfg
X-MFC with: r288669, r289465
Differential Revision: https://reviews.freebsd.org/D3924
2015-10-18 04:07:40 +00:00
cem
bb2ba2b058 Document bitset(9) 2015-10-17 19:55:58 +00:00
ngie
91fe36c7aa Only use -fstack-protector-strong with supported compilers
This includes clang 3.5.0+, gcc 4.2.1, gcc 4.8.4+

This allows me to do subdirectory makes again after setting
MAKESYSPATH on 10.2-RELEASE as it comes with clang 3.4.1.

As a sidenote: this isn't technically correct for all vintages
of gcc 4.2.1, but will be correct when gcc is rebuilt/reinstalled
after r286074, so this version check should be good enough.

X-MFC with: r288669
Differential Revision: https://reviews.freebsd.org/D3924
Reviewed by: emaste, pfg
2015-10-17 19:48:17 +00:00
bdrewery
d506826439 Rework the 'make -n -n' feature such that '-n' recurses and '-N' does not.
Bmake has a documented feature of '-N' to skip executing commands which is
specifically intended for debugging top-level builds and not recursing into
sub-directories.  This matches the older 'make -n' behavior we added which made
'-n -n' the recursing target and '-n' a non-recursing target.

Removing the '-n -n' feature allows the build to work as documented in
the bmake manpage with '-n' and '-N'.  The older '-n -n' feature was also
not documented anywhere that I could see.

Note that the ${_+_} var is still needed as currently bmake incorrectly
executes '+' commands when '-N' is specified.

The '-n' and '-n -n' features were broken for several reasons prior to this.
r251748 made '_+_' never expand with '-n -n' which resulted in many
sub-directories not being visited until fixed 2 years later in r288391, and
many targets were given .MAKE over the past few years which resulted in
non-sub-make commands, such as rm and ln and mtree, to be executed.

This should also allow removing some indirection hacks in bsd.subdir.mk and
other cases of .USE that have a .MAKE by using '+'.

Sponsored by:	EMC / Isilon Storage Division
Discussed on:	arch@ (mostly silence)
2015-10-17 16:42:54 +00:00
ngie
0a77d5c6e1 Install share/zoneinfo in a deterministic way by sorting the results from find
This helps produce deterministic METALOG output

PR: 200674
Submitted by: Fabian Keil <fk@fabiankeil.de>
Reviewed by: emaste
MFC after: 1 week
Obtained from: ElectroBSD
2015-10-17 09:26:16 +00:00
bdrewery
3e0a363d0d Similar to r289355, /usr/tests is within the base system so put the symbols
into /usr/lib/debug.

This covers some missing files:
  /usr/tests/libexec/rtld-elf/.debug
  /usr/tests/libexec/rtld-elf/.debug/libpythagoras.so.0.debug
  /usr/tests/lib/libc/tls/.debug
  /usr/tests/lib/libc/tls/.debug/libh_tls_dynamic.so.1.debug
  /usr/tests/lib/libc/tls/.debug/h_tls_dlopen.so.debug
  /usr/tests/lib/libthr/dlopen/.debug
  /usr/tests/lib/libthr/dlopen/.debug/h_pthread_dlopen.so.1.debug
  /usr/tests/lib/libxo/.debug
  /usr/tests/lib/libxo/.debug/libenc_test.so.debug

Sponsored by:	EMC / Isilon Storage Division
2015-10-16 05:06:43 +00:00
bdrewery
e1f3daf5e3 Rename the /usr/share/doc/legal files to driver.LICENSE to work around
bug of installing 'realtek' and 'intel_iwn' as files rather then as
a 'LICENSE' file in their directories.

Also add obsolete entries for the older names and names that existed in head
for a period of time.

Suggested by:	jmg
X-MFC-With:	r289391
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-16 00:38:05 +00:00
bdrewery
153205e33f Add more SUBDIR_PARALLEL.
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:55:08 +00:00
bdrewery
1b224bebd4 Make installing to a non-existent directory an error.
Before this, if a file was installed to DESTDIR/some/dir and that directory
was missing due to not having ran 'make distrib-dirs' yet, the file would
be installed as 'some/dir'.  For something like bsd.incs.mk with INCLUDEDIR
being a sub-directory of /usr/include, this could result in all of the headers
being installed to a file rather than getting a directory of them.

Now it will error that the file/directory does not exist rather than hide
the issue.

Another option being discussed is to implement GNU's install -D flag which
would auto create any missing directories.

This is a mitigation of the problem.  The proper order to the build is to
run 'make distrib-dirs' first, but that can be forgotten if building from
a sub-directory after updating the source code to the latest revision.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:49:56 +00:00
bdrewery
fc88096acf Fix wrong use of .for; the iteration variable is not used in the loop.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 18:55:43 +00:00
bdrewery
952d11b6c6 Remove excess .else
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 18:53:26 +00:00
emaste
e2f6f0979c /usr/tests is part of the base system (for *.debug files)
Sponsored by:	The FreeBSD Foundation
2015-10-15 02:52:30 +00:00
bdrewery
cca4c22082 Add missing targets to PHONY_NOTMAIN.
- buildconfig, installconfig (missed in r289085)
- files (missed in r241298)

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 20:38:51 +00:00
bdrewery
174c029e4d Recurse on 'buildconfig' and 'installconfig'. Remove the 'config' pseudo target.
The 'config' target isn't really needed right now so just remove it to avoid
any clashes with config(8) building.  It's also likely misspelled and should
be 'configs' if we decide to add it back.  This was just a convenience
target recently added.

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 20:30:32 +00:00
bdrewery
132e71dfe9 Re-indent the ALL_SUBDIR_TARGETS list 2015-10-14 20:28:15 +00:00
bdrewery
2606e8ae5c Revert r289282 for now as the interaction with a directory containing
bsd.files.mk and bsd.subdir.mk is recursing too many times.
2015-10-14 19:30:04 +00:00
emaste
d70c586bb8 /libexec subdirs are part of the base system (for *.debug files)
Sponsored by:	The FreeBSD Foundation
2015-10-14 19:19:44 +00:00
vangyzen
60458e70da resolver: automatically reload /etc/resolv.conf
On each resolver query, use stat(2) to see if the modification time
of /etc/resolv.conf has changed.  If so, reload the file and reinitialize
the resolver library.  However, only call stat(2) if at least two seconds
have passed since the last call to stat(2), since calling it on every
query could kill performance.

This new behavior is enabled by default.  Add a "reload-period" option
to disable it or change the period of the test.

Document this behavior and option in resolv.conf(5).

Polish the man page just enough to appease igor.

https://lists.freebsd.org/pipermail/freebsd-arch/2015-October/017342.html

Reviewed by:	kp, wblock
Discussed with:	jilles, imp, alfred
MFC after:	1 month
Relnotes:	yes
Sponsored by:	Dell Inc.
Differential Revision:	https://reviews.freebsd.org/D3867
2015-10-14 14:26:44 +00:00
bdrewery
c7b6fda55e Fix support for building a PROG_CXX, and PROG, directly.
For example in lib/atf/libatf-c++/tests/detail it is now possible to
run 'make application_test'.  This was intended to worked for PROGS,
but lacked support for PROGS_CXX.

Also fix redefining the main PROG target to recurse.  This isn't needed
since the main process is setting PROG/PROG_CXX to handle it directly
via bsd.prog.mk.

MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-14 05:50:16 +00:00
adrian
b79a4ad744 Fix date.
Noticed by:	bdrewery
2015-10-14 05:16:56 +00:00
bdrewery
3451a7cd5b Follow-up r288218 by ensuring common objects are built before recursing.
Some example where this is a problem:
  lib/atf/libatf-c++/tests/Makefile:SRCS.${_T}=   ${_T}.cpp test_helpers.cpp
  lib/atf/libatf-c++/tests/detail/Makefile:SRCS.${_T}=    ${_T}.cpp test_helpers.cpp
  lib/atf/libatf-c/tests/Makefile:SRCS.${_T}=     ${_T}.c test_helpers.c
  lib/atf/libatf-c/tests/detail/Makefile:SRCS.${_T}=      ${_T}.c test_helpers.c
  lib/libpam/libpam/tests/Makefile:SRCS.${test} = ${test}.c ${COMMONSRC}

A similar change may be needed for FILES, SCRIPTS, or INCS, but for now stay
with just SRCS.

Reported by:	rodrigc
MFC after:	3 weeks
X-MFC-With:	r288218
Sponsored by:	EMC / Isilon Storage Division
2015-10-14 04:42:05 +00:00
adrian
e146e19fb5 rsu(4) manpage updates: add me, add 802.11n support, update caveats.
* Add that I indeed added 802.11n support.
* Update caveats - we support 1x1, 1x2 and 2x2 operation now, but
  there's no transmit aggregation support.
2015-10-14 02:43:04 +00:00
bdrewery
e08f0663f7 Replace the out-of-place includes/files/config handling in bsd.subdir.mk with
more typical ALL_SUBDIR_TARGETS entries and target hooks in bsd.incs.mk,
bsd.files.mk and bsd.confs.mk.

This allows the targets to be NOPs if unneeded and still work with the
shortcut 'make includes' to build and then install in a parallel-safe manner.

Sort and re-indent the ALL_SUBDIR_TARGETS with the new entries.

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 02:37:30 +00:00
bdrewery
c54a8cc39c Correct a comment in bsd.incs.mk forgotten in r274662 and copied into bsd.confs.mk.
The bsd.confs.mk may be wrong but for now fix it.

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 00:43:29 +00:00
bdrewery
db1849f980 Add a note about the mysterious files/includes/config block.
This originated from r96668.
2015-10-14 00:36:33 +00:00
bdrewery
d7fe1b3fea bsd.subdir.mk: Move all of the targets into ALL_SUBDIR_TARGETS.
Also improve documentation.

The SUBDIR_TARGETS variable should really be named LOCAL_SUBDIR_TARGETS, but
renaming it may be a surprise for downstream vendors who use this variable.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-13 19:42:57 +00:00