44ec023c17
The new C test takes 25 seconds on QEMU-RISC-V, wheras the shell version
takes 332 seconds.
Even with the latest optimizations to atf-sh this test still takes a few
seconds to startup in QEMU. Re-writing it in C reduces the runtime for a
single test from about 2-3 seconds to less than .5 seconds. Since there
are ~80 tests, this adds up to about 3-4 minutes.
This may not seem like a big speedup, but before the recent optimizations
to avoid atf_get_srcdir, each test took almost 100 seconds on QEMU RISC-V
instead of 3. This also significantly reduces the time it takes to list
the available test cases, which speeds up running the tests via kyua:
```
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test_old
...
158/158 passed (0 failed)
332.08 real 42.58 user 286.17 sys
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test
158/158 passed (0 failed)
24.96 real 9.75 user 14.26 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
0.17 real 0.06 user 0.08 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old pf1001
pfctl_test_old: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_old: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id Refs Name
141 1 pf
Executing command [ pfctl -o none -nvf - ]
passed
1.73 real 0.25 user 1.41 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old -l > /dev/null
24.36 real 2.26 user 21.86 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
0.04 real 0.02 user 0.01 sys
```
The speedups are even more noticeable on CHERI-RISC-V (since QEMU runs
slower when emulating CHERI instructions):
```
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new -l > /dev/null
0.51 real 0.49 user 0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
34.20 real 32.69 user 0.16 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id Refs Name
147 1 pf
Executing command [ pfctl -o none -nvf - ]
passed
5.74 real 5.41 user 0.03 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new pf1001
pfctl_test_new: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_new: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
0.68 real 0.66 user 0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl #
```
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D26779
|
||
|---|---|---|
| .. | ||
| Makefile | ||
| Makefile.depend | ||
| pf0001.in | ||
| pf0001.ok | ||
| pf0002.in | ||
| pf0002.ok | ||
| pf0003.in | ||
| pf0003.ok | ||
| pf0004.in | ||
| pf0004.ok | ||
| pf0005.in | ||
| pf0005.ok | ||
| pf0006.in | ||
| pf0006.ok | ||
| pf0007.in | ||
| pf0007.ok | ||
| pf0008.in | ||
| pf0008.ok | ||
| pf0009.in | ||
| pf0009.ok | ||
| pf0010.in | ||
| pf0010.ok | ||
| pf0011.in | ||
| pf0011.ok | ||
| pf0012.in | ||
| pf0012.ok | ||
| pf0013.in | ||
| pf0013.ok | ||
| pf0014.in | ||
| pf0014.ok | ||
| pf0016.in | ||
| pf0016.ok | ||
| pf0018.in | ||
| pf0018.ok | ||
| pf0019.in | ||
| pf0019.ok | ||
| pf0020.in | ||
| pf0020.ok | ||
| pf0022.in | ||
| pf0022.ok | ||
| pf0023.in | ||
| pf0023.ok | ||
| pf0024.in | ||
| pf0024.ok | ||
| pf0025.in | ||
| pf0025.ok | ||
| pf0026.in | ||
| pf0026.ok | ||
| pf0028.in | ||
| pf0028.ok | ||
| pf0030.in | ||
| pf0030.ok | ||
| pf0031.in | ||
| pf0031.ok | ||
| pf0032.in | ||
| pf0032.ok | ||
| pf0034.in | ||
| pf0034.ok | ||
| pf0035.in | ||
| pf0035.ok | ||
| pf0038.in | ||
| pf0038.ok | ||
| pf0039.in | ||
| pf0039.ok | ||
| pf0040.in | ||
| pf0040.ok | ||
| pf0041.in | ||
| pf0041.ok | ||
| pf0047.in | ||
| pf0047.ok | ||
| pf0048.in | ||
| pf0048.ok | ||
| pf0049.in | ||
| pf0049.ok | ||
| pf0050.in | ||
| pf0050.ok | ||
| pf0052.in | ||
| pf0052.ok | ||
| pf0053.in | ||
| pf0053.ok | ||
| pf0055.in | ||
| pf0055.ok | ||
| pf0056.in | ||
| pf0056.ok | ||
| pf0057.in | ||
| pf0057.ok | ||
| pf0060.in | ||
| pf0060.ok | ||
| pf0061.in | ||
| pf0061.ok | ||
| pf0065.in | ||
| pf0065.ok | ||
| pf0067.in | ||
| pf0067.ok | ||
| pf0069.in | ||
| pf0069.ok | ||
| pf0070.in | ||
| pf0070.ok | ||
| pf0071.in | ||
| pf0071.ok | ||
| pf0072.in | ||
| pf0072.ok | ||
| pf0074.in | ||
| pf0074.ok | ||
| pf0075.in | ||
| pf0075.ok | ||
| pf0077.in | ||
| pf0077.ok | ||
| pf0078.in | ||
| pf0078.ok | ||
| pf0079.in | ||
| pf0079.ok | ||
| pf0081.in | ||
| pf0081.ok | ||
| pf0082.in | ||
| pf0082.ok | ||
| pf0084.in | ||
| pf0084.ok | ||
| pf0085.in | ||
| pf0085.ok | ||
| pf0087.in | ||
| pf0087.ok | ||
| pf0088.in | ||
| pf0088.ok | ||
| pf0089.in | ||
| pf0089.ok | ||
| pf0090.in | ||
| pf0090.ok | ||
| pf0091.in | ||
| pf0091.ok | ||
| pf0092.in | ||
| pf0092.ok | ||
| pf0094.in | ||
| pf0094.ok | ||
| pf0095.in | ||
| pf0095.include | ||
| pf0095.ok | ||
| pf0096.in | ||
| pf0096.ok | ||
| pf0097.in | ||
| pf0097.ok | ||
| pf0098.in | ||
| pf0098.ok | ||
| pf0100.in | ||
| pf0100.ok | ||
| pf0101.in | ||
| pf0101.ok | ||
| pf0102.in | ||
| pf0102.ok | ||
| pf0104.in | ||
| pf0104.ok | ||
| pf1001.in | ||
| pf1001.ok | ||
| pf1002.in | ||
| pf1002.ok | ||
| pf1003.in | ||
| pf1003.ok | ||
| pf1004.in | ||
| pf1004.ok | ||
| pf1005.in | ||
| pf1005.ok | ||