freebsd-dev/sbin
Alex Richardson 44ec023c17 Rewrite pfctl_test in C to reduce testsuite run time
The new C test takes 25 seconds on QEMU-RISC-V, wheras the shell version
takes 332 seconds.

Even with the latest optimizations to atf-sh this test still takes a few
seconds to startup in QEMU. Re-writing it in C reduces the runtime for a
single test from about 2-3 seconds to less than .5 seconds. Since there
are ~80 tests, this adds up to about 3-4 minutes.
This may not seem like a big speedup, but before the recent optimizations
to avoid atf_get_srcdir, each test took almost 100 seconds on QEMU RISC-V
instead of 3. This also significantly reduces the time it takes to list
the available test cases, which speeds up running the tests via kyua:

```
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test_old
...
158/158 passed (0 failed)
      332.08 real        42.58 user       286.17 sys
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test
158/158 passed (0 failed)
       24.96 real         9.75 user        14.26 sys

root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
        0.17 real         0.06 user         0.08 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old pf1001
pfctl_test_old: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_old: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id  Refs Name
141    1 pf
Executing command [ pfctl -o none -nvf - ]
passed
        1.73 real         0.25 user         1.41 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old -l > /dev/null
       24.36 real         2.26 user        21.86 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
        0.04 real         0.02 user         0.01 sys
```

The speedups are even more noticeable on CHERI-RISC-V (since QEMU runs
slower when emulating CHERI instructions):
```
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new -l > /dev/null
        0.51 real         0.49 user         0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
       34.20 real        32.69 user         0.16 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id  Refs Name
147    1 pf
Executing command [ pfctl -o none -nvf - ]
passed
        5.74 real         5.41 user         0.03 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new pf1001
pfctl_test_new: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_new: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
        0.68 real         0.66 user         0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl #
```

Reviewed By:	kp
Differential Revision: https://reviews.freebsd.org/D26779
2020-10-14 17:39:50 +00:00
..
adjkerntz
bectl loader: zfs should support bootonce an nextboot 2020-09-21 09:01:10 +00:00
bsdlabel
camcontrol camcontrol(8): Bugfixes for some issues reported by mandoc 2020-10-03 18:40:02 +00:00
ccdconfig
clri
comcontrol
conscontrol
ddb
decryptcore Do a sweep and remove most WARNS=6 settings 2020-10-01 01:10:51 +00:00
devd Document the rather suprising behavior with ' inside action rules. 2020-10-13 05:19:00 +00:00
devfs devfs.rules: unhide pf in vnet jails 2020-10-05 19:26:54 +00:00
devmatch devmatch: First appeared in 12.0 2020-10-13 05:32:00 +00:00
dhclient dhclient(8): Bugfixes for some issues reported by mandoc 2020-10-03 18:42:15 +00:00
dmesg
dump Clean up some function and variable names. 2020-06-23 21:11:40 +00:00
dumpfs Print the fs last-mounted time too. 2020-05-02 00:10:25 +00:00
dumpon dumpon.8: Update minimum key size language 2020-04-23 18:04:52 +00:00
etherswitchcfg
fdisk
ffsinfo
fsck Fix a few mandoc issues 2020-10-09 14:03:45 +00:00
fsck_ffs Update the libufs cgget() and cgput() interfaces to have a similar 2020-09-19 22:48:30 +00:00
fsck_msdosfs Use %ju and cast to (uintmax_t) to avoid using PRI* macros. 2020-09-28 04:30:31 +00:00
fsdb fsdb(8): Fix an issue reported by mandoc 2020-10-03 18:44:13 +00:00
fsirand
gbde
geom
ggate ggate: Fix ggated/ggatec debug print of offsets. 2020-09-09 12:58:19 +00:00
growfs Add missing cylinder group check-hash updates when doing large expansions 2020-09-22 03:57:48 +00:00
gvinum gvinum(8): Fix an issue reported by mandoc 2020-10-03 18:49:00 +00:00
hastctl
hastd
ifconfig Fix a few mandoc issues 2020-10-09 14:03:45 +00:00
init Remove risky compatability with old kernels 2020-09-02 21:14:36 +00:00
ipf
ipfw ipfw(8): Bugfixes for some issues reported by mandoc 2020-10-03 18:30:01 +00:00
iscontrol pkgbase: We can't easily have a package with either a - or a _ 2020-08-07 12:19:21 +00:00
kldconfig kldconfig(8): Add EXAMPLES to the man page 2020-09-29 17:52:15 +00:00
kldload
kldstat kldstat(8): Add EXAMPLES to the man page 2020-09-02 18:15:57 +00:00
kldunload
ldconfig ldconfig: add aout deprecation notice 2020-03-17 15:57:25 +00:00
md5 md5: return non-zero if built-in tests (-x) fail 2020-07-23 20:06:24 +00:00
mdconfig
mdmfs mdmfs(8): Fix an issue reported by mandoc 2020-10-03 18:47:50 +00:00
mknod
mksnap_ffs Deduplicate fsid comparisons 2020-05-21 01:55:35 +00:00
mount Move the mount name to bit mapping into sys/mount.h so it can be shared with the 2020-08-19 17:09:58 +00:00
mount_cd9660
mount_fusefs
mount_msdosfs Fix a few mandoc issues 2020-10-09 14:03:45 +00:00
mount_nfs mount_nfs(8): document alternate form of the gssname option 2020-07-17 21:55:24 +00:00
mount_nullfs
mount_udf
mount_unionfs
natd
newfs In the newfs(8) utility, use the more appropriate sbwrite() and cgwrite() 2020-09-13 22:57:50 +00:00
newfs_msdos Fix makefs bootstrap on macOS after D25563 2020-08-25 13:30:24 +00:00
nfsiod
nos-tun
nvmecontrol Use intmax_t to print uint64_t values. 2020-10-07 07:55:55 +00:00
pfctl Rewrite pfctl_test in C to reduce testsuite run time 2020-10-14 17:39:50 +00:00
pfilctl Do a sweep and remove most WARNS=6 settings 2020-10-01 01:10:51 +00:00
pflogd
ping Do a sweep and remove most WARNS=6 settings 2020-10-01 01:10:51 +00:00
ping6 Add -z "TOS" option to ping6, to test DSCP/ECN values 2020-09-10 00:50:18 +00:00
quotacheck
rcorder [rcorder] [crunch] Fix C function declarations to include void 2020-09-21 17:59:45 +00:00
reboot The nextboot(8) manual page currently says that the loader(8) would delete 2020-10-11 10:40:11 +00:00
recoverdisk Do a sweep and remove most WARNS=6 settings 2020-10-01 01:10:51 +00:00
resolvconf
restore Clean up global variable declarations in the dump and restore 2020-04-04 00:56:56 +00:00
route Skip sbin/route tests if jail not installed (WITHOUT_JAIL). 2020-08-04 21:34:13 +00:00
routed
rtsol
savecore savecore: accept device names without the /dev/ prefix 2020-06-29 22:12:23 +00:00
sconfig Fix the following -Werror warning from clang 10.0.0 in sconfig: 2020-02-13 18:37:17 +00:00
setkey Refer to AES-CBC as "aes-cbc" rather than "rijndael-cbc" for IPsec. 2020-06-04 22:58:37 +00:00
shutdown shutdown.8: Fix typo 2020-07-05 13:08:17 +00:00
spppcontrol
swapon The -F flag of swapon(8) requires -a to work. 2020-05-19 12:16:44 +00:00
sysctl Enable iterating all sysctls, even ones with CTLFLAG_SKIP 2020-10-05 20:13:22 +00:00
tests
tunefs Only set WARNS if not defined 2020-09-11 13:28:37 +00:00
umount Document that umount -A does not unmount /dev 2020-06-18 23:12:55 +00:00
veriexec veriexec(8): Bugfix for an issue reported by mandoc 2020-10-03 18:46:42 +00:00
zfsbootcfg loader: zfs should support bootonce an nextboot 2020-09-21 09:01:10 +00:00
Makefile
Makefile.amd64
Makefile.arm
Makefile.i386
Makefile.inc build: provide a default WARNS for all in-tree builds 2020-09-18 17:17:46 +00:00
Makefile.mips
Makefile.powerpc64