after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
Replace `make regress` (legacy test make target) and `make test` (incomplete
test make target added with the FreeBSD test suite) with make check as it's
consistent with other open source projects.
`make check` defaults to running tests from `.OBJDIR`, but can be overridden
with the `CHECKDIR` variable.
Add `make checkworld` target to simplify running the FreeBSD test suite from
`TESTSBASE` (i.e. the top-level tests directory), similar to buildworld.
Document `make check` and `make checkworld` in build(7).
Other minor changes:
- Rename intermediate file (`Kyuafile.auto`) to `Kyuafile` to simplify
`make check`.
- Remove terse warnings attached to `beforetest`/`aftertest`.
- Add kyua binary check to check target in suite.test.mk; error out if it's
not found
The MFC is [partly] contingent on other build related changes being MFCed.
Differential Revision: https://reviews.freebsd.org/D4406
MFC after: 2 months
X-MFC to: stable/10
Relnotes: yes
Reviewed by: bdrewery, Evan Cramer <eccramer@gmail.com>
Sponsored by: EMC / Isilon Storage Division
When bmake was initially imported at r241298 shell commands were no longer
ran with 'set -e' as they were before. This was fixed in r254980 so they
again always use 'set -e'.
Sponsored by: EMC / Isilon Storage Division
Allow Makefiles to define generic metadata settings that apply to all test
programs defined by a Makefile. The generic TEST_METADATA variable extends
the per-test program settings already supported via TEST_METADATA.<program>.
This feature will be useful to easily apply some settings to all programs
in a directory. In particular, Kyua 0.12 will support parallel execution
of test programs and a bunch of them will need to be tagged as is_exclusive
to indicate that they cannot be run in parallel with anything else due to
their side-effects. It will be reasonable to set this setting on whole
directories.
MFC after: 1 week
The new suite.test.mk file contains all the logic needed to install test
programs under /usr/tests/ and to support Kyua as the run-time engine.
This file is included by default by bsd.test.mk so Makefiles do not need
to care about its existence.
Specific Makefiles can define NOT_FOR_TEST_SUITE to indicate that whatever
test programs they are building are not supposed to be installed under
/usr/tests/ nor run by Kyua. (The effect of passing this setting is that
suite.test.mk is simply not included.)
NOT_FOR_TEST_SUITE should never be used by Makefiles in the base system.
This functionality is provided so that third-parties can hook in their
own test code, with different semantics, if they wish. This was asked
for by sjg@.
