stage_* targets.
In non-jobs mode we can achieve the same result by simply introducing
the targets in the correct order.
Thus in bsd*.mk we simply add targets to STAGE_TARGETS which we
realize in meta.stage.mk
Reviewed by: bdrewery
This fix is spiritually similar to r287442 and was discovered thanks to
the KASSERT added in that revision.
NT_PROCSTAT_VMMAP output length, when packing kinfo structs, is tied to
the length of filenames corresponding to vnodes in the process' vm map
via vn_fullpath. As vnodes may move during coredump, this is racy.
We do not remove the race, only prevent it from causing coredump
corruption.
- Add a sysctl, kern.coredump_pack_vmmapinfo, to allow users to disable
kinfo packing for PROCSTAT_VMMAP notes. This avoids VMMAP corruption
and truncation, even if names change, at the cost of up to PATH_MAX
bytes per mapped object. The new sysctl is documented in core.5.
- Fix note_procstat_vmmap to self-limit in the second pass. This
addresses corruption, at the cost of sometimes producing a truncated
result.
- Fix PROCSTAT_VMMAP consumers libutil (and libprocstat, via copy-paste)
to grok the new zero padding.
Reported by: pho (https://people.freebsd.org/~pho/stress/log/datamove4-2.txt)
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3824
only when _WITHOUT_SRCCONF wasn't defined. Restore this
behavior because bsd.ports.mk depends on this in subtle
ways. The compat include of bsd.compiler.mk should
be removed in 12 anyway.
PR: 203540
The general stack protector is known to be weak and has pretty small
coverage. While setting stack-protector-all would give better protection
it would come with a performance cost: for this reason Google's Chrome OS
team developed a new stack-protector-strong variant.
In addition to the protections offered by -fstack-protector, the new option
will guard any function that declares any type or length of local array,
even those in structs or unions. It will also protect functions that use a
local variable's address in a function argument or on the right-hand side
of an assignment.
The option was introduced in GCC-4.9, but support for it has been
back-ported to our base GCC (r286074) and is also available in clang.
The change was tested with dbench and doesn't introduce performance
regressions. An exp-run over the ports tree revealed no failures when
using the stricter stack-protector-all. Thanks to all testers involved.
Reference:
https://outflux.net/blog/archives/2014/01/27/fstack-protector-strong/
Tested by: pho, portmgr (antoine)
Discussed with: secteam (delphij)
Differential Revision: https://reviews.freebsd.org/D3463
PR: 203394 (exp-run)
Relnotes: yes
MFC: no (not supported in older clang)
event loop and should sleep only when waiting for events (eg. via kevent(2)).
When a program is going to sleep in the kernel, the script will show its name,
PID, kernel stack trace and userland stack trace. Sleeping in kevent(2) is
ignored as it is expected to be valid.
Sample output:
# ./blocking lynxd
lynxd(15042) is blocking...
kernel`_cv_wait_sig+0x124
kernel`seltdwait+0xae
kernel`sys_poll+0x3a3
kernel`amd64_syscall+0x343
kernel`0xffffffff806c79ab
lynxd`poll+0xa
lynxd`pqSocketCheck+0xa2
lynxd`pqWaitTimed+0x29
lynxd`connectDBComplete+0xd7
lynxd`PQsetdbLogin+0x2ec
lynxd`db_connect+0x3c
lynxd`main+0x198
lynxd`_start+0x16f
0x2
lynxd(1925) is blocking...
kernel`_cv_wait+0x125
zfs.ko`zio_wait+0x5b
zfs.ko`dmu_buf_hold_array_by_dnode+0x1dc
zfs.ko`dmu_read+0xcb
zfs.ko`zfs_freebsd_getpages+0x37b
kernel`VOP_GETPAGES_APV+0xa7
kernel`vnode_pager_getpages+0x9a
kernel`vm_fault_hold+0x885
kernel`vm_fault+0x77
kernel`trap_pfault+0x211
kernel`trap+0x506
kernel`0xffffffff806c76c2
lynxd`EVP_add_cipher+0x13
lynxd`SSL_library_init+0x11
lynxd`main+0x94
lynxd`_start+0x16f
0x2
lynxd(1925) is blocking...
kernel`_cv_wait+0x125
zfs.ko`zio_wait+0x5b
zfs.ko`dbuf_read+0x791
zfs.ko`dbuf_findbp+0x12f
zfs.ko`dbuf_hold_impl+0xa2
zfs.ko`dbuf_hold+0x1b
zfs.ko`dmu_buf_hold_array_by_dnode+0x153
zfs.ko`dmu_read_uio+0x66
zfs.ko`zfs_freebsd_read+0x3a3
kernel`VOP_READ_APV+0xa1
kernel`vn_read+0x13a
kernel`vn_io_fault+0x10b
kernel`dofileread+0x95
kernel`kern_readv+0x68
kernel`sys_read+0x63
kernel`amd64_syscall+0x343
kernel`0xffffffff806c79ab
lynxd`_read+0xa
lynxd`__srefill+0x122
lynxd`fgets+0x78
lynxd`file_gets+0x1d
lynxd`BIO_gets+0x64
lynxd`PEM_read_bio+0xf5
lynxd`PEM_X509_INFO_read_bio+0x90
lynxd`X509_load_cert_crl_file+0x47
lynxd`by_file_ctrl+0x2e
lynxd`X509_STORE_load_locations+0x4a
lynxd`sslctx_init+0x255
lynxd`main+0x215
lynxd`_start+0x16f
0x2
Requested by: gnn
Obtained from: Wheel Systems http://wheelsystems.com
This fixes staging errors for non-parallel builds that have LINKS.
Creating hardlinks must always happen after the actual files are installed.
The staging code was protected by an .ORDER statement that only affected
parallel -j builds but not non-parallel builds. Fix this by making the
real stage_links.SET (stage_links.links, stage_links.mlinks, etc) targets
depend on the main targets for all of the other possible staging needs. For
example, stage_links.links will depend on stage_as and stage_files, which have
their own dependencies to stage_as.prog or stage_files.prog or stage_files.SET,
which is enough to satistfy the ordering.
Also remove the requirement that symlinks be created last, as they can
safely be made without the source being present unlike hardlinks. This also
fixes symlinks to come before hardlinks as it is possible, in theory, to
hardlink a symlink. This is not actually supported here though.
Sponsored by: EMC / Isilon Storage Division
This is converting the path usr/share/tmac.*stage to something else, but
nothing ever installs or reads from such a path. They might look in
stage.*usr/share/tmac, but that's not what this is matching. Additionally
the .dirdeps match all of the tmac files back to gnu/usr.bin/groff/tmac
fine.
Sponsored by: EMC / Isilon Storage Division
When bmake was initially imported at r241298 shell commands were no longer
ran with 'set -e' as they were before. This was fixed in r254980 so they
again always use 'set -e'.
Sponsored by: EMC / Isilon Storage Division
This is a NOP as r254419 enabled this by default in bmake. Add it here though
to ensure it is known that we are using this as a default and in case a
bmake import removes the default we have.
This tells bmake to always pass job tokens into sub-commands. Otherwise
it would only do so if the target being built depended on the special
.MAKE target (which causes _all_ commands to be executed with -n as well)
or if the command matches '${MAKE}/${.MAKE}/$(MAKE)/$(.MAKE)/make' (before
expansion, so ${LIB32WMAKE} would not qualify). Using '+' on a command
(which runs the command with -n) would not pass the job token even though it
is a documented way to achieve the .MAKE effect on a command.
Sponsored by: EMC / Isilon Storage Division
This fixes atf-c.h not properly being installed to /usr/include/ (in
the stagedir) via its override of 'INCSDIR_atf-c.h= ${INCLUDEDIR}'.
This fixes building things that depend on atf.
Staging seems to ignore OWN/GRP/MODE settings and needs further exploration.
Sponsored by: EMC / Isilon Storage Division
bsd.obj.mk handles the needs fine. When an objdir exists it will
just rm -Rf the objdir. When it does not exist though it will
call 'clean' and 'cleandepend', which properly recurse in bsd.progs.mk.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Refer to the usb_quirk(4) manual page for more details on how to use
this new feature.
Submitted by: Maxime Soule <btik-fbsd@scoubidou.com>
PR: 203249
MFC after: 2 weeks
This fixes the following errors:
make: don't know how to make bsd.README. Stop
make: don't know how to make auto.obj.mk. Stop
This is easily seen in sys/dev/*.
The new behavior is now the expected output:
make: no target to make.
This would happen as MAKESYSPATH (.../share/mk) is auto added to the -I list.
Any directory where make is ran in the src tree that has no local Makefile
would then try executing the target in share/mk/Makefile, which by default
was to build the first entry in FILES. Of course, because bsd.README and
auto.obj.mk are not in the current directory the error is shown.
This check only works for bmake, but I will still MFC it with an extra
'!defined(.PARSEDIR) ||' guard for stable/10.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
This avoids needing a large boot partition / file system in order to
accommodate multiple kernels, and provides consistency with userland
debug. This also simplifies the process of moving kernel debug files
to a separate package and installing them on demand.
In addition, change kernel debug file extension to .debug, to match
userland debug files.
When using the supported kernel installation method the
/usr/lib/debug/boot/kernel directory will be renamed (to kernel.old)
as is done with /boot/kernel.
Developers wishing to maintain the historical behavior of installing
debug files in /boot/kernel/ can set KERN_DEBUGDIR="" in src.conf(5).
Reviewed by: bdrewery, brooks, imp, markj
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1006
This mostly fixes an interaction with bsd.test.mk with PROGS and SCRIPTS.
This was most notable with 'make clean' and 'make install', which r281055
and r272055 attempted to address but were inadequate.
It also addresses similar issues in bsd.progs.mk when not using bsd.test.mk.
This also fixes cases of NOT running commands in the parent when using
bsd.progs.mk:
- 'make clean' was not run for the main process for Makefiles which had both
FILES and SUBDIR but no PROGS or SCRIPTS. This usually was just a
leftover Kyuafile.auto. One such example is usr.bin/bmake/tests/sysmk/t1/2.
- 'make obj' was not running in the current directory with bsd.test.mk due
to early inclusion of bsd.subdir.mk. This was not really a problem due to
the SUBDIRS using 'mkdir -p' for their objdirs.
There were subtle bugs causing this wrong behavior:
1. bsd.progs.mk needs to set SCRIPTS to empty when recursing to avoid
the sub-makes from installing, cleaning or building the SCRIPTS;
only the parent make should be doing this. r281055 effectively did
the same but wasn't enough.
2. CLEANFILES may contain (especially from *.test.mk) files which only
the parent should clean, such as from FILES and SCRIPTS. To resolve
sub-makes also cleaning these, reset CLEANFILES and CLEANDIRS in the
children before including bsd.prog.mk. A tempting alternative would be
to only handle CLEANFILES in the parent but then the child bsd.prog.mk
CLEANFILES of per-PROGS wouldn't be setup.
3. bsd.subdir.mk was included too soon in bsd.test.mk. It needs to be
included after bsd.prog.mk as the SCRIPTS logic is short-circuitted if
'install:' is already defined (which bsd.subdir.mk does). There is
actually no need to include bsd.subdir.mk from bsd.test.mk as bsd.prog.mk
and bsd.obj.mk will do so in the proper order. The description in r257095
covers this for FILES and was fixed differently, though changing the
handling of target(install) in bsd.prog.mk may make sense after more
research.
4. bsd.progs.mk had extra logic to handle recursing SCRIPTS if PROGS was
empty, which isn't its business to be doing. SCRIPTS is handled fine
by bsd.prog.mk. This mostly reverts and reworks the fix in r259209 and
partially reverts r272055.
5. bsd.progs.mk has no need to depend 'all:' on SCRIPTS and FILES. These
are handled by bsd.prog.mk/bsd.files.mk fine. This also partially reverts
r272055.
6. bsd.progs.mk was not drop-in safe for bsd.prog.mk. Move the PROGS
check from r273186 to allow it to be used safely.
Specific tested cases:
SCRIPTS:no PROGS:no FILES:yes SUBDIR:yes
usr.bin/bmake/tests/sysmk/t1/2
SCRIPTS:yes PROGS:no FILES:yes SUBDIR:no
usr.bin/bmake/tests/sysmk/t1/2/1
SCRIPTS:yes PROGS:yes FILES:yes SUBDIR:yes
lib/libthr/tests
SCRIPTS:yes PROGS:no FILES:yes SUBDIR:no
usr.bin/yacc/tests
libexec/atf/atf-sh/tests
A full buildworld/installworld/clean comparison with mtree was also done.
The only relevant difference was the new fixed behavior of removing
Kyuafile.auto from the objdir in 'clean'.
Converting SCRIPTS to be a special case FILES group will make this less
fragile and is being explored.
One known remaining issue is 'cleandepend' removing the tags files for
every recursive call.
Note that the 'make clean' command runs for the CURDIR last, which can make
it appear to run multiple times when cleaning in tests/, but each command is
for a SUBDIR returning up the chain. This is purely bsd.subdir.mk behavior.
PR: 191055
PR: 191955
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
In the first build the TOOLSDIR does not exit yet which causes CC to default
to the sys.mk version. Once a TOOLSDIR is created during the build though,
this logic was changing CC to ${TOOLSDIR}/usr/bin/cc even though that file
did not exist. Thus CC went from 'cc' to '/usr/bin/cc' which forced a
rebuild of everything while using the same compiler. Check that TOOLSDIR is
not empty to avoid this. If there is actually a TOOLSDIR cc then it will be
used and properly rebuild.
Sponsored by: EMC / Isilon Storage Division
Filemon(4) will record paths as they are seen, not as fully resolved. make(1)
will take the .MAKE.META.IGNORE_PATHS values and resolve them. This creates
a discrepancy if CCACHE_DIR is a symlink. Fix this by ensuring it is
resolved for its actual usage.
Submitted by: sjg
When inspecting this value it is more expected to have it show the
automatically-created directory value rather than CURDIR.
Sponsored by: EMC / Isilon Storage Division
This limits CLEANFILES removal to just bsd.obj.mk now and removes the need
for NOPATH_FILES.
This reverts r96529 which was done due to the command line being too long
for libc. Since then all architectures now use 256k for ARG_MAX (r170102).
Regardless of that, the libc CLEANFILES is only 72k now. Others
may be larger but not likely to hit the limit. If needed, we can improve
the bsd.obj.mk clean: target to split up the list via bmake features.
This also removes some workarounds that are no longer needed.
- a.out removal
- OBJS.tmp, which has not been needed since r117080.
- *.so, which has not been needed since a .so->.So rename in r42450.
This also fixes STATICOBJS and SHLIB_LINK not being in the .NOPATH list.
Sponsored by: EMC / Isilon Storage Division
