d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Simplify the capsicum-test wrapper script

Browse Source 
Instead of running tests one-by-one with the shell wrapper we now run
the full gtest testsuite twice (once as root, once as non root). This
significantly speeds up running tests despite running them twice.
This change also passes the missing -u flag to capsicum-test that caused
test failures (https://bugs.freebsd.org/250178)

Previously, running the testsuite with the wrapper script took ~3s per
test on aarch64 QEMU, i.e. a total of almost 5 minutes.
Now it takes 6 seconds to run all tests twice.

Before:
root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional
94/96 passed (2 failed)
      309.97 real        58.46 user       244.31 sys

After:
root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional
functional:test_root  ->  passed  [2.659s]
functional:test_unprivileged  ->  passed  [2.391s]
2/2 passed (0 failed)
        5.48 real         1.06 user         2.52 sys

This overhead is caused by kyua + atf-sh spawning lots of additional
processes and can be avoided by just running the googletest test binary.
syscall                     seconds   calls  errors
fork                   39.810229456    1275       0
sigprocmask            13.546928736     572       0

i.e. 1275 processes spawned to run a single test.

Test Plan:	All tests pass with D28907.
PR:		250178
Reviewed By:	lwhsu
Differential Revision: https://reviews.freebsd.org/D29014
This commit is contained in:
Alex Richardson 2021-03-02 18:27:34 +00:00
parent 28cd3a673e
commit 53a535c1d8
1 changed files with 17 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

70
tests/sys/capsicum/functional.sh
View File

@ -29,69 +29,33 @@
#
# $FreeBSD$
SRCDIR=$(atf_get_srcdir)
CAPSICUM_TEST_BIN=capsicum-test
check()
{
local tc=${1}
atf_test_case "test_root"
test_root_head() {
atf_check -s exit:0 -o match:PASSED -e ignore \
${SRCDIR}/${CAPSICUM_TEST_BIN} --gtest_filter=${tc}
atf_set descr 'Run capsicum-test as root'
atf_set require.user root
}
skip()
{
local reason=${1}
atf_skip "${reason}"
test_root_body() {
atf_check -s exit:0 -o match:PASSED -e ignore \
"$(atf_get_srcdir)/${CAPSICUM_TEST_BIN}" -u "$(id -u tests)"
}
add_testcase()
{
local tc=${1}
local tc_escaped word
atf_test_case "test_unprivileged"
test_unprivileged_head() {
tc_escaped=$(echo ${tc} | sed -e 's/\./__/')
atf_test_case ${tc_escaped}
if [ "$(atf_config_get ci false)" = "true" ]; then
case "${tc_escaped}" in
ForkedOpenatTest_WithFlagInCapabilityMode___|OpenatTest__WithFlag)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/249960\"; }"
;;
PipePdfork__WildcardWait)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/244165\"; }"
;;
Capability__NoBypassDAC)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/250178\"; }"
;;
Pdfork__OtherUserForked)
eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/250179\"; }"
;;
*)
eval "${tc_escaped}_body() { check ${tc}; }"
;;
esac
else
eval "${tc_escaped}_body() { check ${tc}; }"
fi
atf_add_test_case ${tc_escaped}
atf_set descr 'Run capsicum-test as an unprivileged user'
atf_set require.user unprivileged
}
list_tests()
{
${SRCDIR}/${CAPSICUM_TEST_BIN} --gtest_list_tests | awk '
/^[^ ]/ { CAT=$0 }
/^[ ]/ { print CAT $1}'
test_unprivileged_body() {
atf_check -s exit:0 -o match:PASSED -e ignore \
"$(atf_get_srcdir)/${CAPSICUM_TEST_BIN}" -u "$(id -u)"
}
atf_init_test_cases()
{
local t
for t in `list_tests`; do
add_testcase $t
done
atf_init_test_cases() {
atf_add_test_case test_root
atf_add_test_case test_unprivileged
}