This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.
Currently rescue/rescue and sys/modules are still not connected.
Sponsored by: EMC / Isilon Storage Division
- Support more of the toolchain from TOOLSDIR.
- This also improves 'make bootstrap-tools' to pass, for example,
AS=/usr/bin/as to Makefile.inc1, which will tell cross-tools to use
external toolchain support and avoid building things we won't be using
in the build.
- Always set the PATH to contain the staged TOOLSDIR directories when
not building the bootstrap targets.
The previous version was only setting this at MAKE.LEVEL==0 and if the
TOOLSDIR existed. Both of these prevented using staged tools that were
built during the build though as DIRDEPS with .host dependencies, such
as the fix for needing usr.bin/localedef.host in r291311.
This is not a common tool so we must build and use it during the build,
and need to be prepared to change PATH as soon as it appears.
This should also fix the issue of host dependencies disappearing from
Makefile.depend and then reappearing due to the start of the fresh build not
having the directory yet, resulting in the tools that were built not actually
being used.
- Only use LEGACY_TOOLS while building in Makefile.inc1. After r291317
and r291546 there is no need to add LEGACY_TOOLS into the PATH for
the pseudo/targets/toolchain build.
- Because the pseudo/targets/toolchain will now build its own
[clang-]tblgen, the special logic in clang.build.mk is no longer needed.
- LEGACY_TOOLS is no longer used outside of targets/pseudo/bootstrap-tools
so is no longer passed into the environment in its build.
Sponsored by: EMC / Isilon Storage Division
Doing this causes more trouble than it is worth regarding cyclic
dependencies. It should not be needed after cleaning up MACHINE=host
builds in r291324.
Sponsored by: EMC / Isilon Storage Division
IPv4/IPv6 checksum offloading and VLAN tag insertion/stripping.
Since uether doesn't provide a way to announce driver specific offload
capabilities to upper stack, checksum offloading support needs more work
and will be done in the future.
Special thanks to Hayes Wang from RealTek who gave input.
This is mostly working around the converts/iconv port having '../ces/file.o'
in its OBJS list which resulted in '.depend../ces/file.o'. Now it will have
'.depend.._ces_file.o'.
Other implementations have :T which would result in '.depend.file.o' here, but
that could lead to collisions.
X-MFC-With: r291554
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
-MP creates empty targets for all dependency files, which can be useful when a
dependency is deleted from the file system. This would otherwise cause an
error for "don't know how to build FOO" since the .depend file is included
with the dependency registered.
This is mostly a workaround for the misc/dahdi-kmod port using '::' for one of
its dependencies, while -MP uses just ':'. This results in an 'Inconsistent
operator for' error.
X-MFC-With: r290433
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- Add a kvaddr_type to represent kernel virtual addresses instead of
unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles(). It drops the
unused 'swapfile' argument and adds a new function pointer argument for
a symbol resolving function. Native kernels still use _fdnlist() from
libc to resolve symbols if a resolver function is not supplied, but cross
kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
Each backend is responsible for implementing kvm_read2() for a given
vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
powerpc cores).
- Add internal helper routines for the common page offset hash table used
by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
to be cross-friendly by using private constants instead of ones that
vary by platform (e.g. PAGE_SIZE). Static assertions are present when
a given backend is compiled natively to ensure the private constants
match the real ones.
- Enable all of the existing vmcore backends on all platforms. This means
that libkvm on any platform should be able to perform KVA translation
and read data from a vmcore of any platform.
Tested on: amd64, i386, sparc64 (marius)
Differential Revision: https://reviews.freebsd.org/D3341
headers.
This resulted in 'don't know how to make .o.' errors after the changes in
r289286.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
This is going to be used to allow DIRDEPS to be bootstrapped off of
LIBADD/DPADD. It currently works for internal libraries which have a
DIR defined for them but also use the .a library from a src-mapped obj
directory. It can also be useful for using -L without a --sysroot per
LIBADD to use the OBJDIR version of the libraries.
I didn't review every LIBADD, so it is possible this is missing some.
Sponsored by: EMC / Isilon Storage Division
We need to not use -nostdinc since it breaks building of clang itself. Use
-isystem rather than -I/usr/include and -nostdinc which gets us using
the stage include directory before searching the real host headers.
This allows removing more of the -I hacks to get host headers since the
headers are no longer excluded. The -B seemed unneeded.
This fixes building of secure/lib/libcrypto which was looking at the
/usr/include/openssl/asn1.h header rather than the staged one.
This fixes building of clang which wants to find its own internal
headers in the STAGEDIR/usr/lib/clang/* path.
Sponsored by: EMC / Isilon Storage Division
The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.
This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.
For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds. This has not yet been a problem due
to readelf not being built as a host tool in buildworld. This is possible
in the meta build though when building the toolchain.
Sponsored by: EMC / Isilon Storage Division
Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file. There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.
Sponsored by: EMC / Isilon Storage Division
This was finding libraries that were installed into DESTDIR/usr/lib,
where DESTDIR is the stage directory, and then adding in usr/lib to
DIRDEPS. Just exclude the STAGE_ROOT if defined.
Discussed with: sjg
Sponsored by: EMC / Isilon Storage Division
This has caused much confusion for myself as there are quite a lot of
variables that depend on having a proper ${.OBJDIR}.
Sponsored by: EMC / Isilon Storage Division
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.
Sponsored by: EMC / Isilon Storage Division
The mlx5* driver(s) are built [*]/installed separate from the OFED stack thanks
to recent refactoring done in the linuxkpi(4) module.
Always install the manpages instead of conditionally installing them if
MK_OFED != no
* Further refactoring of sys/ofed and linuxkpi(4) is pending to fully divorce
mlx5* from ofed headers
MFC after: never
Requested by: hps
Hacks to enable target mode there complicated code, while didn't really
work. And for outdated hardware fixing it is not really interesting.
Initiator mode tested with Qlogic 1080 adapter is still working fine.
sesX device number may change between reboots, so to properly identify
the instance we need more data. Name and ID reported here may mach ones
reported by SCSI device, but that is not really required by specs.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
This is only for sys/boot. INTERNALPROG is not a build tool, it is just a way
to generate OBJS from a list of SRCS and use those objects elsewhere.
Sponsored by: EMC / Isilon Storage Division
critical section.
uma_zalloc_arg()/uma_zalloc_free() may acquire a sleepable lock on the
zone. The malloc() family of functions may call uma_zalloc_arg() or
uma_zalloc_free().
The malloc(9) man page currently claims that free() will never sleep.
It also implies that the malloc() family of functions will not sleep
when called with M_NOWAIT. However, it is more correct to say that
these functions will not sleep indefinitely. Indeed, they may acquire
a sleepable lock. However, a developer may overlook this restriction
because the WITNESS check that catches attempts to call the malloc()
family of functions within a critical section is inconsistenly
applied.
This change clarifies the language of the malloc(9) man page to clarify
the restriction against calling the malloc() family of functions
while in a critical section or holding a spin lock. It also adds
KASSERTs at appropriate points to make the enforcement of this
restriction more consistent.
PR: 204633
Differential Revision: https://reviews.freebsd.org/D4197
Reviewed by: markj
Approved by: gnn (mentor)
Sponsored by: Juniper Networks
default and add a manual page for mlx5en. The mlx5 module contains
shared code for both infiniband and ethernet. The mlx5en module
contains specific code for ethernet functionality only. A mlx5ib
module is in the works for infiniband support.
Supported hardware:
- ConnectX-4: 10/20/25/40/50/56/100Gb/s speeds.
- ConnectX-4 LX: 10/25/40/50Gb/s speeds (low power consumption)
Refer to the mlx5en(4) manual page for a comprehensive list.
The team porting the mlx5 driver(s) to FreeBSD:
- Hans Petter Selasky <hselasky@freebsd.org>
- Oded Shanoon <odeds@mellanox.com>
- Meny Yossefi <menyy@mellanox.com>
- Shany Michaely <shanim@mellanox.com>
- Shahar Klein <shahark@mellanox.com>
- Daria Genzel <dariaz@mellanox.com>
- Mark Bloch <markb@mellanox.com>
Differential Revision: https://reviews.freebsd.org/D4163
Submitted by: Mark Block <markb@mellanox.com>
Sponsored by: Mellanox Technologies
Reviewed by: gnn @
MFC after: 3 days
This is useful in environments where system configuration is performed by
automated interaction with the system console, since unexpected witness
output makes such automation difficult. With this change, the new
debug.witness.output_channel sysctl allows one to specify that witness
output is to be printed to the kernel log (using log(9)) rather than the
console.
Reviewed by: cem, jhb
MFC after: 2 weeks
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D4183
Keep old CPUTYPEs around for compatibility. Also include
a list of typical values for FreeBSD.
# Split out from other changes in D4155
Differential Revision: https://reviews.freebsd.org/D4155
has not ran yet.
This fixes building objects directly, or skipping 'make depend', not generating
required headers first. This case did work without FAST_DEPEND so there's no
reason it should not work here as well.
An example of this can be seen building in gnu/usr.bin/binutils/libbfd
without running 'make depend' first to generate config.h.
Sponsored by: EMC / Isilon Storage Division
MFC after: 3 weeks
X-MFC-With: r290433
This allows most of the build to simply consider MK_META_MODE
Update to latest dirdeps.mk so we can do:
make -f dirdeps.mk bin/cat.i386
Reviewed by: bdrewery
The goal here is to make the upgrade seamless for users
Add aliases for zh_HK
Remove bad symlinks created by previous bad upgrade procedure.
Complete ObsoleteFiles.inc with more locales that have been removed
This allows META_FILES option to be renamed META_MODE.
Also add META_COOKIE_TOUCH for use in targets that can benefit
from a cookie when in meta mode.
Differential Revision: https://reviews.freebsd.org/D4153
Reviewed by: bdrewery
new return codes of -1 were mistakenly being considered "true". Callout_stop
now returns -1 to indicate the callout had either already completed or
was not running and 0 to indicate it could not be stopped. Also update
the manual page to make it more consistent no non-zero in the callout_stop
or callout_reset descriptions.
MFC after: 1 Month with associated callout change.
This also fixes truncating the path that the links were installed to, which
was most likely going to be the same directory the library was in anyhow.
Let bsd.sys.mk handle SYMLINKS via stage_symlinks. stage_libs continues to
handle the SHLIB_LINK though since it is not a SYMLINKS.
This fixes a race, seen easily in lib/libthr, where libpthread_p.a is created
by both stage_libs and stage_symlinks resulting in 'ln: File exists'.
Sponsored by: EMC / Isilon Storage Division
Discussed with: sjg
Discussing with sjg@, we agree the better fix is to be done in meta.stage.mk.
This also broke staging of SYMLINKS for non-shared libraries, such as for
lib/libcompiler_rt, which results in all Makefile.depend removing it.
Sponsored by: EMC / Isilon Storage Division
meta.stage.mk is handling ${SYMLINKS:T} for stage_libs already. The logic in
bsd.sys.mk to handle ${SYMLINKS} was brought in r247817 when it was moved out
of bsd.prog.mk and bsd.lib.mk into bsd.sys.mk. The logic previously was
limited to bsd.prog.mk.
This fixes a race, seen easily in lib/libthr, where libpthread_p.a is created
by both stage_libs and stage_symlinks resulting in 'ln: File exists'.
Sponsored by: EMC / Isilon Storage Division
Igor has many less complaints now. I think the two remaining are bogus, but I
am also not sure why Igor is producing them.
The page still needs more work.
Sponsored by: EMC / Isilon Storage Division
centralizes the handling of CC and HOST_CC.
This fixes a bug with WITH_CCACHE_BUILD when using MACHINE=host since
CC is overridden in local.init.mk via src.opts.mk long before bsd.compiler.mk
is included.
Originally the ccache implementation was placed in local.init.mk but moved
to bsd.compiler.mk as it seemed more proper and avoided other ordering
issues.
Sponsored by: EMC / Isilon Storage Division
directory symlink when the target directory does not exist. This will
cause an error instead of a broken setup.
Sponsored by: EMC / Isilon Storage Division
should be used by TCP for sure in its cleanup of the IN-PCB (will be coming shortly).
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D4076
- Move all section 5 bluetooth manpages under MK_BLUETOOTH != no
MFC after: 3 days
PR: 193260
Reported by: Philippe Michel <philippe.michel7@sfr.fr>
Sponsored by: EMC / Isilon Storage Division
Similar to the original reason for these dependency hints to be added,
in r124637, the missing-dependency file case can lead to building of the
wrong source.
A clear example of this is in gnu/lib/libstdc++ where the .PATH contains
both contrib/gcc and contrib/libstdc++/src.
contrib/gcc has a debug.c.
contrib/libstdc++/src has a debug.cc.
When building for the objects of debug.o, debug.So, and debug.po, it is
ambiguous for which src file to use due to the suffix transformation
rules, even though the proper one is listed first in .PATH.
This was normally avoided due to these dependency hints for the initial
build, and then mkdep would add an explicit 'debug.o: debug.cc'
dependency into the .depend file. WITH_FAST_DEPEND does not generate
the .depend file with these, but puts them into .depend.debug.o instead.
Rather than extending the exists() check to each object's .depend.*
file, just enable the hint when when using WITH_FAST_DEPEND. It fixes
the problem and seems to be safe enough to use since it is mapping SRCS
back to OBJS, rather than letting make make assumptions from OBJS to
SRCS.
A similar check mapping objects to headers is present in some mk files
but was not extended here for FAST_DEPEND since it has not yet been
found to be a problem.
X-MFC-With: r290433
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
used to achieve a safe pseudo-incremental build. This is explained in
more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the
cache, but with a full cache saves 30-50% in build times. When combined with
the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
resulting dependency file, which it does not do when using mkdep(1)/'CC
-E'. Stats are provided at the end of this message.
This removes the need to modify /etc/make.conf with the CC:= and CXX:=
lines which conflicted with external compiler support [3] (causing the
bootstrap compiler to not be built which lead to obscure failures [4]),
incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
Clang errors with parenthesis, and did not work with META_MODE.
The option name was picked to match the existing option in ports. This
feature is available for both in-src and out-of-src builds that use
/usr/share/mk.
Linking, assembly compiles, and pre-processing avoid using ccache since it is
only overhead. ccache does nothing special in these modes, although there is
no harm in calling it for them.
CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
compiler to hash the content of the compiler binary to determine if it
should be a cache miss. For external compilers the 'mtime' option is used
as it is more efficient and likely to be correct. Future work may optimize the
'content' check using the same checks as whether a bootstrap compiler is needed
to be built.
The CCACHE_CPP2 pessimization is currently default in our devel/ccache
port due to Clang requiring it. Clang's -Wparentheses-equality,
-Wtautological-compare, and -Wself-assign warnings do not mix well with
compiling already-pre-processed code that may have expanded macros that
trigger the warnings. GCC has so far not had this issue so it is allowed to
disable the CCACHE_CPP2 default in our port.
Sharing a cache between multiple checkouts, or systems, is explained in
the ccache manual. Sharing a cache over NFS would likely not be worth
it, but syncing cache directories between systems may be useful for an
organization. There is also a memcached backend available [5]. Due to using
an object directory outside of the source directory though you will need to
ensure that both are in the same prefix and all users use the same layout. A
possible working layout is as follows:
Source: /some/prefix/src1
Source: /some/prefix/src2
Source: /some/prefix/src3
Objdir: /some/prefix/obj
Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
all absolute paths to be relative. Using something like this is required due
to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
adds into its hash for the object without CCACHE_BASEDIR.
distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
I have not personally tested this and assume it will not mix well with
using the bootstrap compiler.
The cache from buildworld can be reused in a subdir by first running
'make buildenv' (from r290424).
Note that the cache is currently different depending on whether -j is
used or not due to ccache enabling -fdiagnostics-color automatically if
stderr is a TTY, which bmake only does if not using -j.
The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files and ccache objects.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
The WITH_FAST_DEPEND feature was used for comparison here as well to show
the dramatic time savings with a full cache.
buildworld:
x buildworld-before
+ buildworld-ccache-empty
* buildworld-ccache-full
% buildworld-ccache-full-fastdep
# buildworld-fastdep
+-------------------------------------------------------------------------------+
|% * # +|
|% * # +|
|% * # xxx +|
| |A |
| A|
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 4519 4525.04 4520.73 4521.59 3.1104823
Difference at 95.0% confidence
758.027 +/- 43.4565
20.1412% +/- 1.15466%
(Student's t, pooled s = 19.1726)
* 3 1823.08 1827.2 1825.62 1825.3 2.0785572
Difference at 95.0% confidence
-1938.26 +/- 43.298
-51.5007% +/- 1.15045%
(Student's t, pooled s = 19.1026)
% 3 1266.96 1279.37 1270.47 1272.2667 6.3971113
Difference at 95.0% confidence
-2491.3 +/- 44.3704
-66.1952% +/- 1.17895%
(Student's t, pooled s = 19.5758)
# 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)
buildkernel:
x buildkernel-before
+ buildkernel-ccache-empty
* buildkernel-ccache-empty-fastdep
% buildkernel-ccache-full
# buildkernel-ccache-full-fastdep
@ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|# @ % * |
|# @ % * x + |
|# @ % * xx ++|
| MA |
| MA|
| A |
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 727.97 731.91 728.06 729.31333 2.2492295
Difference at 95.0% confidence
156.88 +/- 4.17129
27.4058% +/- 0.728695%
(Student's t, pooled s = 1.84034)
* 3 527.1 528.29 528.08 527.82333 0.63516402
Difference at 95.0% confidence
-44.61 +/- 2.33254
-7.79305% +/- 0.407478%
(Student's t, pooled s = 1.02909)
% 3 400.4 401.05 400.62 400.69 0.3306055
Difference at 95.0% confidence
-171.743 +/- 2.16453
-30.0023% +/- 0.378128%
(Student's t, pooled s = 0.954969)
# 3 201.94 203.34 202.28 202.52 0.73020545
Difference at 95.0% confidence
-369.913 +/- 2.40293
-64.6212% +/- 0.419774%
(Student's t, pooled s = 1.06015)
@ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)
[1] https://ccache.samba.org/performance.html
[2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
[3] https://reviews.freebsd.org/D3484
[5] https://github.com/jrosdahl/ccache/pull/30
PR: 182944 [4]
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Relnotes: yes
This is because the .meta files generated from filemon already contain a
list of all files read to generate the object.
X-MFC-With: r290433
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
This is especially noticeable in the kernel obj directory since it
includes so many files.
X-MFC-With: r290433
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Add S8, S16, S32, and U32 types; add SYSCTL*() macros for them, as well
as for the existing 64-bit types. (While SYSCTL*QUAD and UQUAD macros
already exist, they do not take the same sort of 'val' parameter that
the other macros do.)
Clean up the documented "types" in the sysctl.9 document. (These are
macros and thus not real types, but the manual page documents intent.)
The sysctl_add_oid(9) arg2 has been bumped from intptr_t to intmax_t to
accommodate 64-bit types on 32-bit pointer architectures.
This is just the kernel support piece; the userspace sysctl(1) support
will follow in a later patch.
Submitted by: Ravi Pokala <rpokala@panasas.com>
Reviewed by: cem
Relnotes: no
Sponsored by: Panasas
Differential Revision: https://reviews.freebsd.org/D4091
Add net.link.lagg.lacp.default_strict_mode which defines
the default value for LACP strict compliance for created
lagg devices.
Also:
* Add lacp_strict option to ifconfig(8).
* Fix lagg(4) creation examples.
* Minor style(9) fix.
MFC after: 1 week
This speeds up buildworld by 16% on my system and buildkernel by 35%.
Rather than calling mkdep(1), which is just a wrapper around 'cc -E',
use the modern -MD -MT -MF flags to gather and generate dependencies during
compilation. This flag was introduced in GCC "a long time ago", in GCC 3.0,
and is also supported by Clang. (It appears that ICC also supports this but I
do not have access to test it). This avoids running the preprocessor *twice*
for every build, in both 'make depend' and 'make all'. This is especially
noticeable when using ccache since it does not cache preprocessor results from
mkdep(1) / 'cc -E', but still speeds up compilation with the -MD flags.
For 'make depend' a tree-walk is still done to ensure that all DPSRCS
are generated when expected, and that beforedepend/afterdepend and
_EXTRADEPEND are all still respected. In time this may change but for now
I've been conservative. The time for a tree-walk with -j combined with
SUBDIR_PARALLEL is not significant. For example, it takes about 9 seconds
with -j15 to walk all of src/ for 'make depend' now on my system.
A .depend file is still generated with the various rules that apply to
the final target, or custom rules. Otherwise there are now
per-built-object-file .depend files, such as .depend.filename.o. These
are included directly by make rather than populating .depend with a loop
and .depend lines, which only added overhead to the now almost-NOP 'make
depend' phase.
Before this I experimented with having mkdep(1) called in parallel per-file.
While this improved the kernel and lib/libc 'make depend' phase, it resulted
in slower build times overall.
The -M flags are removed from CFLAGS when linking since they have no effect.
Enabling this by default, for src or out-of-src, can be done once more testing
has been done, such as a ports exp-run, and with more compilers.
The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_FAST_DEPEND=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
buildworld:
x buildworld-before
+ buildworld-fastdep
+-------------------------------------------------------------------------------+
|+ |
|+ |
|+ xx x|
| |_MA___||
|A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)
buildkernel:
x buildkernel-before
+ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|+ x |
|++ xx|
| A||
|A| |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)
Sponsored by: EMC / Isilon Storage Division
MFC after: 3 weeks
Relnotes: yes
PCI-Express capability registers (that is, PCI config registers in the
standard PCI config space belonging to the PCI-Express capability
register set).
Note that all of the current PCI-e registers are either 16 or 32-bits,
so only widths of 2 or 4 bytes are supported.
Reviewed by: imp
MFC after: 1 week
Sponsored by: Chelsio
Differential Revision: https://reviews.freebsd.org/D4088
libopenbsd is an internal library which
to bring in compatibility stuff from OpenBSD.
This will allow us to bring in more
OpenBSD utilities into the FreeBSD base system.
We similarly use libnetbsd for bringing in stuff from NetBSD.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D4078
They were never supported, and the encoding they should use is somehow
controversial, they could be added later if we really need them and once
one has decided which encoding they should use
In retrospect, having an alias for UTF-8 does not bring any real benefits
and it can cause confusion. Let's remove this *.UTF8 locale symlinks
which is closer to the convention of the other BSDs.
GCC testsuite is also removing utf8 and UTF8 locales for portability
with BSD.
Submitted by: marino
Obtained from: DragonflyBSD
MK_USB != no
Add the manpages to OptionalObsoleteFiles.inc
As a side-effect, this also fixes installworld with MK_USB == no
X-MFC with: r290128
Sponsored by: EMC / Isilon Storage Division
Certain invalid operations trigger hardware error conditions. Error
conditions that only halt one channel can be detected and recovered by
resetting the channel. Error conditions that halt the whole device are
generally not recoverable.
Add a sysctl to inject channel-fatal HW errors,
'dev.ioat.<N>.force_hw_error=1'.
When a halt due to a channel error is detected, ioat(4) blocks new
operations from being queued on the channel, completes any outstanding
operations with an error status, and resets the channel before allowing
new operations to be queued again.
Update ioat.4 to document error recovery; document blockfill introduced
in r290021 while we are here; document ioat_put_dmaengine() added in
r289907; document DMA_NO_WAIT added in r289982.
Sponsored by: EMC / Isilon Storage Division
This can have important debugging information such as 'cc: not found' or
'ccache: error: Could not find compiler "cc" in PATH'.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
"one of many" targets, e.g. `make hello_world`, where hello_world is a C
program
Tested with: PROGS and PROGS_CXX
MFC after: 1 week
X-MFC with: r289289
Sponsored by: EMC / Isilon Storage Division
Using .USEBEFORE had the unintended side-effect of changing the directory for
the real target ran in the current directory. For example this meant that
the 'make clean' would run in one of the SUBDIR.
Sponsored by: EMC / Isilon Storage Division
Pointyhat to: bdrewery
window in number of segments on fly. It is set to 10 segments by default.
Remove net.inet.tcp.experimental.initcwnd10 which is now redundant. Also remove
the parent node net.inet.tcp.experimental as it's not needed anymore and also
because it was not well thought out.
Differential Revision: https://reviews.freebsd.org/D3858
In collaboration with: lstewart
Reviewed by: gnn (prev version), rwatson, allanjude, wblock (man page)
MFC after: 2 weeks
Relnotes: yes
Sponsored by: Limelight Networks
The "files" staging name is not the same as "bsd.files.mk" but seems to just be
a group of extra files that are not the essential includes or libraries, which
include .
Sponsored by: EMC / Isilon Storage Division
"Method of Reporting Informational Exceptions") in the SCSI mode database.
T10/04-371 revision 2 (revision 4; page 2, table 1) describes it as a
bit-field of 4 bits wide.
1. http://www.t10.org/ftp/t10/document.04/04-371r2.pdf
This a recommit of head@r289913 to fix the original commit message, in
particular:
- I incorrectly claimed that unit change was 'i' -> 't'.
- The spec I reference in this commit is 2 decades newer than the one noted in
r289913. The fields in the SCSI mode database are more complete in the newer
spec, so it'll be easier for someone to decipher this commit if need be
later.
- I screwed up the bug entry in the previous commit message
Pointyhat to: ngie (for botching up r289913)
PR: 200619
Reported by: Michael Baptist
Submitted by: Lars Skodje
Sponsored by: EMC / Isilon Storage Divisionf
"Method of Reporting Informational Exceptions") in the SCSI mode database as
the field described in X3T10/94-190 (revision 4; page 2, table 1) [1.] is
4 bits wide, not 4 bytes wide
1. http://ftp.t10.org/ftp/t10/document.94/94-190r4.pdf
Bug 200619
MFC after: 1 week
Reported by: Michael Baptist <mbaptist@isilon.com>
Submitted by: Lars Skodje <lskodje@isilon.com>
Sponsored by: EMC / Isilon Storage Division
This is because the previous version was very obscure about the fact
that despite having Clang "on by default" for architectures such as powerpc, it
does not actually build due to the GCC it uses not having C++11 support.
Using an external compiler that supports C++11 does allow this to work.
This whole block should be rethought more given "on by default" is not
really default without extra work which could actually be surprising for
why Clang is showing up when using a newer GCC.
Sponsored by: EMC / Isilon Storage Division
'buildconfig' is connected to 'all', but 'installconfig' is only called
manually. There is not much need to conditionalize this file right
now due to how it is hooked up and its impact on various build phases.
Sponsored by: EMC / Isilon Storage Division
- Rather than allow 'make clean*' to ignore dependencies, make a static
list of targets in STANDALONE_SUBDIR_TARGETS that are known to be safe.
This allows a user to override them if needed and avoids adding this feature
to user-defined targets that are in ${SUBDIR_TARGETS}. [1]
- This now also allows to force SUBDIR_PARALLEL when calling these
targets, since no dependencies are needed.
Reported by: ian [1]
Sponsored by: EMC / Isilon Storage Division
MFC after: 3 weeks
X-MFC-With: r289778
properly recursed.
The .for loop was defining a ${__dir} variable that was being set at a
different evaluation time than the target itself, so every 'cd ${__dir}'
became the last value that was in ${__dir}. This resulted in 'make obj'
not properly being ran in the tree that would leave .depend files
scattered around when 'make all' was ran in rescue/.
To fix this, define a CRUNCH_SRCDIR_* for every prog if it does not
already have one and then use that variable in every relevant place.
This allows simplifying some logic as well.
Reported by: emaste
X-MFC-With: r289734
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
This partially reverts r284685. An attempt was made in r285295 to fix this
but was not enough. There were still $${} vars in the code that should have
been using the ${_page} and ${_sect} vars, but the bigger problem was that
.for cannot be used on .ALLSRC as it is not defined when the .for is evaluated.
Using ${MAN} here in a .for loop doesn't work either as the paths are not
expanded right for lib/libc/ subdirs despite having a .PATH set for all
of them.
Add some comments around long .else and .endif as well.
Sponsored by: EMC / Isilon Storage Division
It turns out that it is pretty easy to make CloudABI work on ARM64. We
essentially only need to copy over the sysvec from AMD64 and ensure that
we use ARM64 specific registers.
As there is an overlap between function argument and return registers,
we do need to extend cloudabi64_schedtail() to only set its values if
we're actually forking. Not when we're creating a new thread.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D3917
This covers 'clean', 'cleandepend', 'cleandir', 'obj', 'objlink' and
'build-tools'.
This uses the same method as bsd.subdir.mk.
MFC after: 2 weeks
X-MFC-With: r289731
Sponsored by: EMC / Isilon Storage Division
- Remove handling of 'make -P' since that is for fmake only.
- Add '+' where appropriate for sub-make calls.
- Pass MK_TESTS=no to all of the sub-makes to prevent recursing into test
directories for targets such as 'obj', 'clean', 'depend', etc.
Sponsored by: EMC / Isilon Storage Division
Now it can be used to effectively "build in a subdir". It will use the
'cross-tools', 'libraries', and 'includes' phases of 'buildworld' to properly
setup a WORLDTMP to use. Then it will build 'everything' only in the
listed SUBDIR_OVERRIDE directories. It is still required to list custom
library directories in LOCAL_LIB_DIRS if SUBDIR_OVERRIDE is something
that contains libraries outside of the normal area (such as
SUBDIR_OVERRIDE=contrib/ofed needing LOCAL_LIB_DIRS=contrib/ofed/usr.lib)
Without these changes, SUBDIR_OVERRIDE with buildworld was broken or hit
obscure failures due to missing libraries, includes, or cross compiler.
SUBDIR_OVERRIDE with 'make <target that is not buildworld>' will continue to
work as it did before although its usefulness is questionable.
With a fully populated WORLDTMP, building with a SUBDIR_OVERRIDE with
-DNO_CLEAN only takes a few minutes to start building the target
directories. This is still much better than building unneeded things via
'everything' when testing small subset changes. A BUILDFAST or
SKIPWORLDTMP might make sense for this as well.
- Add in '_worldtmp' as we still need to create WORLDTMP as later targets,
such as '_libraries' and '_includes' use it. This probably was avoiding
calling '_worldtmp' to not remove WORLDTMP for debugging purposes, but
-DNO_CLEAN can be used for that.
- '_legacy' must be included since '_build-tools' uses -legacy.
The SUBDIR_OVERRIDE change came in r95509, while -legacy being part
of build-tools came in r113136.
- 'bootstrap-tools' is still skipped as this feature is not for
upgrades.
- Fix buildworld combined with SUBDIR_OVERRIDE not installing all includes.
The original change for SUBDIR_OVERRIDE in r95509 kept '_includes'
and '_libraries' as building everything possible as the SUBDIR_OVERRIDE
could need anything from them. However in r96462 the real 'includes'
target was changed from manual sub-makes to just recursing 'includes'
on SUBDIR, thus not all includes have been installed into WORLDTMP since then
when combined with 'buildworld'.
This is not done unless calling 'make buildworld' as it would be
unexpected to have it go into all directories when doing 'make
SUBDIR_OVERRIDE=mydir includes'.
- Also need to build the cross-compiler so it is used with --sysroot.
If this is burdensome then telling the build to use the local compiler
as an external compiler (thus using a proper --sysroot to WORLDTMP) is
possible by setting CC=/usr/bin/cc, CXX=/usr/bin/c++, etc.
- Don't build the lib32 distribution with SUBDIR_OVERRIDE in buildworld
since it won't contain anything related to SUBDIR_OVERRIDE. Testing
of the lib32 build can be done with 'make build32'.
- Document these changes in build.7
Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks
This reverts r266473 as the need for it, working around .MAKE and '+'
issues, is no longer needed after r289460. This avoids extra log
output in -j builds of '-- _sub.TARGET --' that are redundant with the
'-- TARGET --' and '-- TARGET_subdir_DIR --' entries already showing.
r266473 also made a subtle change in the ordering of _SUBDIR handling. Before
the change, SUBDIRS were recursed into after building the TARGET due to the
.USE of _SUBDIR *appending* the commands onto the TARGET. After the change
though the indirection caused TARGET to depend on _sub.TARGET which had the
_SUBDIR handling in it. This TARGET would run after recursing. However, the
SUBDIR_PARALLEL handling from r263778 has this ordering as well. Since
this has so far not been a problem, for now make this behavior for
non-SUBDIR_PARALLEL use of _SUBDIR explicit by using .USEBEFORE.
Further research may change this back to .USE as well as the
SUBDIR_PARALLEL handling and bsd.progs.mk recursing.
Sponsored by: EMC / Isilon Storage Division
The SUBDIR_PARALLEL feature uses a .for dir in ${SUBDIR} loop. The old code
here for recursing was setting SUBDIR= as a make *argument*. The SUBDIR=
replacement was not actually handled until after the .for loop was unrolled.
This could be seen with a '.info ${SUBDIR} ${dir}' inside of the loop which
showed an empty ${SUBDIR} and a set ${dir}. Setting NO_SUBIDR= before calling
${MAKE} as an *environment* variable handles the case fine and is a more
proper mechanism for disabling subdir handling.
This could be seen with 'make -C tests/sys/kern -j15 SUBDIR_PARALLEL=yes'.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
variables are already set. This should cover odd cases such as the
COMPILER_TYPE override in lib/csu/powerpc64.
X-MFC-With: r289659
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
lookups.
This uses a special variable name based on a hash of ${CC}, ${PATH}, and
${MACHINE} to ensure that a cached value is not used if any of these
values changes to use a new compiler.
Before this there were 34,620 fork/exec from bsd.compiler.mk during a buildworld.
After this there are 608. More improvement is needed to cache a value from
the top-level before descending into subdirs in the various build phases.
Reviewed by: brooks (earlier version)
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3898
when -fstack-protector-strong is not available, like it was implicitly before
r288669
As noted by antoine@, devel/gcc (which is 4.8.5) lacks -fstack-protector-strong
support, whereas 4.8.4i (devel/gcc48) has the support.
Until a version is available which has -fstack-protector-strong support, be
conservative and only enable support with 4.9+.
Reviewed by: pfg
X-MFC with: r288669, r289465
Differential Revision: https://reviews.freebsd.org/D3924
This includes clang 3.5.0+, gcc 4.2.1, gcc 4.8.4+
This allows me to do subdirectory makes again after setting
MAKESYSPATH on 10.2-RELEASE as it comes with clang 3.4.1.
As a sidenote: this isn't technically correct for all vintages
of gcc 4.2.1, but will be correct when gcc is rebuilt/reinstalled
after r286074, so this version check should be good enough.
X-MFC with: r288669
Differential Revision: https://reviews.freebsd.org/D3924
Reviewed by: emaste, pfg
Bmake has a documented feature of '-N' to skip executing commands which is
specifically intended for debugging top-level builds and not recursing into
sub-directories. This matches the older 'make -n' behavior we added which made
'-n -n' the recursing target and '-n' a non-recursing target.
Removing the '-n -n' feature allows the build to work as documented in
the bmake manpage with '-n' and '-N'. The older '-n -n' feature was also
not documented anywhere that I could see.
Note that the ${_+_} var is still needed as currently bmake incorrectly
executes '+' commands when '-N' is specified.
The '-n' and '-n -n' features were broken for several reasons prior to this.
r251748 made '_+_' never expand with '-n -n' which resulted in many
sub-directories not being visited until fixed 2 years later in r288391, and
many targets were given .MAKE over the past few years which resulted in
non-sub-make commands, such as rm and ln and mtree, to be executed.
This should also allow removing some indirection hacks in bsd.subdir.mk and
other cases of .USE that have a .MAKE by using '+'.
Sponsored by: EMC / Isilon Storage Division
Discussed on: arch@ (mostly silence)
bug of installing 'realtek' and 'intel_iwn' as files rather then as
a 'LICENSE' file in their directories.
Also add obsolete entries for the older names and names that existed in head
for a period of time.
Suggested by: jmg
X-MFC-With: r289391
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Before this, if a file was installed to DESTDIR/some/dir and that directory
was missing due to not having ran 'make distrib-dirs' yet, the file would
be installed as 'some/dir'. For something like bsd.incs.mk with INCLUDEDIR
being a sub-directory of /usr/include, this could result in all of the headers
being installed to a file rather than getting a directory of them.
Now it will error that the file/directory does not exist rather than hide
the issue.
Another option being discussed is to implement GNU's install -D flag which
would auto create any missing directories.
This is a mitigation of the problem. The proper order to the build is to
run 'make distrib-dirs' first, but that can be forgotten if building from
a sub-directory after updating the source code to the latest revision.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
The 'config' target isn't really needed right now so just remove it to avoid
any clashes with config(8) building. It's also likely misspelled and should
be 'configs' if we decide to add it back. This was just a convenience
target recently added.
Sponsored by: EMC / Isilon Storage Division
On each resolver query, use stat(2) to see if the modification time
of /etc/resolv.conf has changed. If so, reload the file and reinitialize
the resolver library. However, only call stat(2) if at least two seconds
have passed since the last call to stat(2), since calling it on every
query could kill performance.
This new behavior is enabled by default. Add a "reload-period" option
to disable it or change the period of the test.
Document this behavior and option in resolv.conf(5).
Polish the man page just enough to appease igor.
https://lists.freebsd.org/pipermail/freebsd-arch/2015-October/017342.html
Reviewed by: kp, wblock
Discussed with: jilles, imp, alfred
MFC after: 1 month
Relnotes: yes
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D3867
For example in lib/atf/libatf-c++/tests/detail it is now possible to
run 'make application_test'. This was intended to worked for PROGS,
but lacked support for PROGS_CXX.
Also fix redefining the main PROG target to recurse. This isn't needed
since the main process is setting PROG/PROG_CXX to handle it directly
via bsd.prog.mk.
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Some example where this is a problem:
lib/atf/libatf-c++/tests/Makefile:SRCS.${_T}= ${_T}.cpp test_helpers.cpp
lib/atf/libatf-c++/tests/detail/Makefile:SRCS.${_T}= ${_T}.cpp test_helpers.cpp
lib/atf/libatf-c/tests/Makefile:SRCS.${_T}= ${_T}.c test_helpers.c
lib/atf/libatf-c/tests/detail/Makefile:SRCS.${_T}= ${_T}.c test_helpers.c
lib/libpam/libpam/tests/Makefile:SRCS.${test} = ${test}.c ${COMMONSRC}
A similar change may be needed for FILES, SCRIPTS, or INCS, but for now stay
with just SRCS.
Reported by: rodrigc
MFC after: 3 weeks
X-MFC-With: r288218
Sponsored by: EMC / Isilon Storage Division
more typical ALL_SUBDIR_TARGETS entries and target hooks in bsd.incs.mk,
bsd.files.mk and bsd.confs.mk.
This allows the targets to be NOPs if unneeded and still work with the
shortcut 'make includes' to build and then install in a parallel-safe manner.
Sort and re-indent the ALL_SUBDIR_TARGETS with the new entries.
Sponsored by: EMC / Isilon Storage Division
Also improve documentation.
The SUBDIR_TARGETS variable should really be named LOCAL_SUBDIR_TARGETS, but
renaming it may be a surprise for downstream vendors who use this variable.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Before this, the target was unknown. Now it will recurse on subdirs and run
the target in the current directory. It is required to recurse as there
may be subdirs that have objs in their directory or in the object directory,
so it is not enough to just delete the objdir of the subdir parent.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
netbsd-tests.test.mk (r289151)
- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
When run from bin/ls/tests, for example, the value of TESTSDIR would be
${TESTSBASE}/${RELDIR:H} -> /usr/tests/bin/ls/tests/.. ->
/usr/tests/bin/ls
Document the new behavior in bsd.README.
While here, also document TESTSBASE
Relnotes: yes
Differential Revision: D1022
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
- projects/bmake and subsequent commits provide SRCTOP; there's no need to
manually specify it now.
- Compute a sane default for OBJTOP based on .OBJDIR and RELDIR. Manually
specifying this is probably no longer needed, but it persists just in case
(supporting commits will need to be made to move it out of some of the meta
.mk files).
- Compute a sane default for TESTSRC. Error out if the path cannot be found.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
This allows to set delete method via tunable, before device capabilities
are known. Also allow ZERO method for devices not reporting LBP, if user
explicitly requests it -- it may be useful if storage supports compression
and WRITE SAME, but does not support UNMAP.
MFC after: 2 weeks
Simplifying maintainance and options (only one place to deal with MK_DMAGENT)
This also makes packaging base less intrusive by getting a single point where
to add tags.
locales.
When using a Chinese locale, such as zh_TW.UTF-8 or zh_CN.UTF-8,
nl_langinfo(ABMON_*) only returned numbers. For instance,
nl_langinfo(ABMON_1) returns 1, nl_langinfo(ABMON_2) returns 2, and
so on.
This causes problems in applications that put the short month name
and the day of the month together. For example, 'Apr 14' in English
becomes '414日' in Chinese on the top bar of GNOME Shell.
This problem may be resolved by appending '月' to all short month
names and replacing %b with %_m in date_fmt. ja_JP.UTF-8 already
does this, and this matches the en_US.ISO8859-1 behavior, which
returns 'Oct'. The GNU C Library also returns values with '月'
appended.
PR: 199441
Submitted by: Ting-Wei Lan <lantw44 gmail com>
MFC after: 2 weeks
stage_* targets.
In non-jobs mode we can achieve the same result by simply introducing
the targets in the correct order.
Thus in bsd*.mk we simply add targets to STAGE_TARGETS which we
realize in meta.stage.mk
Reviewed by: bdrewery
This fix is spiritually similar to r287442 and was discovered thanks to
the KASSERT added in that revision.
NT_PROCSTAT_VMMAP output length, when packing kinfo structs, is tied to
the length of filenames corresponding to vnodes in the process' vm map
via vn_fullpath. As vnodes may move during coredump, this is racy.
We do not remove the race, only prevent it from causing coredump
corruption.
- Add a sysctl, kern.coredump_pack_vmmapinfo, to allow users to disable
kinfo packing for PROCSTAT_VMMAP notes. This avoids VMMAP corruption
and truncation, even if names change, at the cost of up to PATH_MAX
bytes per mapped object. The new sysctl is documented in core.5.
- Fix note_procstat_vmmap to self-limit in the second pass. This
addresses corruption, at the cost of sometimes producing a truncated
result.
- Fix PROCSTAT_VMMAP consumers libutil (and libprocstat, via copy-paste)
to grok the new zero padding.
Reported by: pho (https://people.freebsd.org/~pho/stress/log/datamove4-2.txt)
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3824
only when _WITHOUT_SRCCONF wasn't defined. Restore this
behavior because bsd.ports.mk depends on this in subtle
ways. The compat include of bsd.compiler.mk should
be removed in 12 anyway.
PR: 203540
The general stack protector is known to be weak and has pretty small
coverage. While setting stack-protector-all would give better protection
it would come with a performance cost: for this reason Google's Chrome OS
team developed a new stack-protector-strong variant.
In addition to the protections offered by -fstack-protector, the new option
will guard any function that declares any type or length of local array,
even those in structs or unions. It will also protect functions that use a
local variable's address in a function argument or on the right-hand side
of an assignment.
The option was introduced in GCC-4.9, but support for it has been
back-ported to our base GCC (r286074) and is also available in clang.
The change was tested with dbench and doesn't introduce performance
regressions. An exp-run over the ports tree revealed no failures when
using the stricter stack-protector-all. Thanks to all testers involved.
Reference:
https://outflux.net/blog/archives/2014/01/27/fstack-protector-strong/
Tested by: pho, portmgr (antoine)
Discussed with: secteam (delphij)
Differential Revision: https://reviews.freebsd.org/D3463
PR: 203394 (exp-run)
Relnotes: yes
MFC: no (not supported in older clang)
event loop and should sleep only when waiting for events (eg. via kevent(2)).
When a program is going to sleep in the kernel, the script will show its name,
PID, kernel stack trace and userland stack trace. Sleeping in kevent(2) is
ignored as it is expected to be valid.
Sample output:
# ./blocking lynxd
lynxd(15042) is blocking...
kernel`_cv_wait_sig+0x124
kernel`seltdwait+0xae
kernel`sys_poll+0x3a3
kernel`amd64_syscall+0x343
kernel`0xffffffff806c79ab
lynxd`poll+0xa
lynxd`pqSocketCheck+0xa2
lynxd`pqWaitTimed+0x29
lynxd`connectDBComplete+0xd7
lynxd`PQsetdbLogin+0x2ec
lynxd`db_connect+0x3c
lynxd`main+0x198
lynxd`_start+0x16f
0x2
lynxd(1925) is blocking...
kernel`_cv_wait+0x125
zfs.ko`zio_wait+0x5b
zfs.ko`dmu_buf_hold_array_by_dnode+0x1dc
zfs.ko`dmu_read+0xcb
zfs.ko`zfs_freebsd_getpages+0x37b
kernel`VOP_GETPAGES_APV+0xa7
kernel`vnode_pager_getpages+0x9a
kernel`vm_fault_hold+0x885
kernel`vm_fault+0x77
kernel`trap_pfault+0x211
kernel`trap+0x506
kernel`0xffffffff806c76c2
lynxd`EVP_add_cipher+0x13
lynxd`SSL_library_init+0x11
lynxd`main+0x94
lynxd`_start+0x16f
0x2
lynxd(1925) is blocking...
kernel`_cv_wait+0x125
zfs.ko`zio_wait+0x5b
zfs.ko`dbuf_read+0x791
zfs.ko`dbuf_findbp+0x12f
zfs.ko`dbuf_hold_impl+0xa2
zfs.ko`dbuf_hold+0x1b
zfs.ko`dmu_buf_hold_array_by_dnode+0x153
zfs.ko`dmu_read_uio+0x66
zfs.ko`zfs_freebsd_read+0x3a3
kernel`VOP_READ_APV+0xa1
kernel`vn_read+0x13a
kernel`vn_io_fault+0x10b
kernel`dofileread+0x95
kernel`kern_readv+0x68
kernel`sys_read+0x63
kernel`amd64_syscall+0x343
kernel`0xffffffff806c79ab
lynxd`_read+0xa
lynxd`__srefill+0x122
lynxd`fgets+0x78
lynxd`file_gets+0x1d
lynxd`BIO_gets+0x64
lynxd`PEM_read_bio+0xf5
lynxd`PEM_X509_INFO_read_bio+0x90
lynxd`X509_load_cert_crl_file+0x47
lynxd`by_file_ctrl+0x2e
lynxd`X509_STORE_load_locations+0x4a
lynxd`sslctx_init+0x255
lynxd`main+0x215
lynxd`_start+0x16f
0x2
Requested by: gnn
Obtained from: Wheel Systems http://wheelsystems.com
This fixes staging errors for non-parallel builds that have LINKS.
Creating hardlinks must always happen after the actual files are installed.
The staging code was protected by an .ORDER statement that only affected
parallel -j builds but not non-parallel builds. Fix this by making the
real stage_links.SET (stage_links.links, stage_links.mlinks, etc) targets
depend on the main targets for all of the other possible staging needs. For
example, stage_links.links will depend on stage_as and stage_files, which have
their own dependencies to stage_as.prog or stage_files.prog or stage_files.SET,
which is enough to satistfy the ordering.
Also remove the requirement that symlinks be created last, as they can
safely be made without the source being present unlike hardlinks. This also
fixes symlinks to come before hardlinks as it is possible, in theory, to
hardlink a symlink. This is not actually supported here though.
Sponsored by: EMC / Isilon Storage Division
This is converting the path usr/share/tmac.*stage to something else, but
nothing ever installs or reads from such a path. They might look in
stage.*usr/share/tmac, but that's not what this is matching. Additionally
the .dirdeps match all of the tmac files back to gnu/usr.bin/groff/tmac
fine.
Sponsored by: EMC / Isilon Storage Division
When bmake was initially imported at r241298 shell commands were no longer
ran with 'set -e' as they were before. This was fixed in r254980 so they
again always use 'set -e'.
Sponsored by: EMC / Isilon Storage Division
This is a NOP as r254419 enabled this by default in bmake. Add it here though
to ensure it is known that we are using this as a default and in case a
bmake import removes the default we have.
This tells bmake to always pass job tokens into sub-commands. Otherwise
it would only do so if the target being built depended on the special
.MAKE target (which causes _all_ commands to be executed with -n as well)
or if the command matches '${MAKE}/${.MAKE}/$(MAKE)/$(.MAKE)/make' (before
expansion, so ${LIB32WMAKE} would not qualify). Using '+' on a command
(which runs the command with -n) would not pass the job token even though it
is a documented way to achieve the .MAKE effect on a command.
Sponsored by: EMC / Isilon Storage Division
This fixes atf-c.h not properly being installed to /usr/include/ (in
the stagedir) via its override of 'INCSDIR_atf-c.h= ${INCLUDEDIR}'.
This fixes building things that depend on atf.
Staging seems to ignore OWN/GRP/MODE settings and needs further exploration.
Sponsored by: EMC / Isilon Storage Division
bsd.obj.mk handles the needs fine. When an objdir exists it will
just rm -Rf the objdir. When it does not exist though it will
call 'clean' and 'cleandepend', which properly recurse in bsd.progs.mk.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Refer to the usb_quirk(4) manual page for more details on how to use
this new feature.
Submitted by: Maxime Soule <btik-fbsd@scoubidou.com>
PR: 203249
MFC after: 2 weeks
