The new C test takes 25 seconds on QEMU-RISC-V, wheras the shell version
takes 332 seconds.
Even with the latest optimizations to atf-sh this test still takes a few
seconds to startup in QEMU. Re-writing it in C reduces the runtime for a
single test from about 2-3 seconds to less than .5 seconds. Since there
are ~80 tests, this adds up to about 3-4 minutes.
This may not seem like a big speedup, but before the recent optimizations
to avoid atf_get_srcdir, each test took almost 100 seconds on QEMU RISC-V
instead of 3. This also significantly reduces the time it takes to list
the available test cases, which speeds up running the tests via kyua:
```
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test_old
...
158/158 passed (0 failed)
332.08 real 42.58 user 286.17 sys
root@qemu-riscv64-alex:~ # /usr/bin/time kyua test -k /usr/tests/sbin/pfctl/Kyuafile pfctl_test
158/158 passed (0 failed)
24.96 real 9.75 user 14.26 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
0.17 real 0.06 user 0.08 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old pf1001
pfctl_test_old: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_old: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id Refs Name
141 1 pf
Executing command [ pfctl -o none -nvf - ]
passed
1.73 real 0.25 user 1.41 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_old -l > /dev/null
24.36 real 2.26 user 21.86 sys
root@qemu-riscv64-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
0.04 real 0.02 user 0.01 sys
```
The speedups are even more noticeable on CHERI-RISC-V (since QEMU runs
slower when emulating CHERI instructions):
```
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new -l > /dev/null
0.51 real 0.49 user 0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test -l > /dev/null
34.20 real 32.69 user 0.16 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test pf1001
pfctl_test: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Id Refs Name
147 1 pf
Executing command [ pfctl -o none -nvf - ]
passed
5.74 real 5.41 user 0.03 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl # /usr/bin/time ./pfctl_test_new pf1001
pfctl_test_new: WARNING: Running test cases outside of kyua(1) is unsupported
pfctl_test_new: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
Running pfctl -o none -nvf /usr/tests/sbin/pfctl/./files/pf1001.in
---
binat on em0 inet6 from fc00::/64 to any -> fc00:0:0:1::/64
binat on em0 inet6 from any to fc00:0:0:1::/64 -> fc00::/64
---
passed
0.68 real 0.66 user 0.00 sys
root@qemu-cheri-alex:/usr/tests/sbin/pfctl #
```
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D26779
To prevent issues with odd shell characters appearing in, a surprising
shell feature is used. Document it and a workaround for it.
Differential Revision: https://reviews.freebsd.org/D26723
the /boot/nextboot.conf file or its contents which is 1) not the most user-
friendly way of working with custom configurations, and 2) simply not true
for both Forth and Lua implementations: they would not delete it, but just
change the setting to "NO", that is, disable it.
While at it, add one missing serial (Oxford) comma and fix some bogus line
wraps along the way.
Approved by: bcr (manpages)
Differential Revision: https://reviews.freebsd.org/D25971
Single quotes interfere with the workaround put in with r335753 and
aren't necessary in this case. I believe that all the underling issues
with r335753 have been corrected, but need to do more extensive
followup before reverting it as a bad idea.
PR: 240411
MFC After: 2 days (to give it time to get into 12.2)
- no blank before trailing delimiter
- whitespace at end of input line
- sections out of conventional order
- normalizing date format
- AUTHORS section without An macro
As we did in the kernel in r366112 replace the MS() macro with the version(s)
added to the kernel: _IEEE80211_MASKSHIFT(). Also provide its counter part.
This will later allow use to use other macros defined in net80211 headers
here in ifconfig.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Sort a few VHT160 and 80+80 lines, update some comments, and remove
a superfluous ','.
No functional changes intended.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Update the to log fetch operation for latest WDC NVMe products.
Tested on HGST SN100 (a few years old) and WDC SN720 (more recent).
Submitted by: Akhilesh Rn <Akhilesh.RN@wdc.com> (minor style tweak by me)
Github PR: 435
Add an "nextnoskip" sysctl that allows for listing of sysctls intended to be
normally skipped for cost reasons.
This makes it so the names/descriptions of those sysctls can be discovered with
sysctl -aN/sysctl -ad/sysctl -at.
It also makes it so children are visited when a node flagged with CTLFLAG_SKIP
is explicitly requested.
The intended use case is to mark the root "kstat" node with CTLFLAG_SKIP so that
the extensive and expensive stats are skipped by default but may still be easily
obtained without having to know them all (which may not even be possible) and
request each one-by-one.
Reviewed by: jhb
MFC after: 2 weeks
Relnotes: yes
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D26560
/dev/pf is usable in vnet jails, so don't hide the node there.
We shouldn't expose /dev/pf in regular jails, as that gives them control over
the host (or parent vnet jail) firewall.
Reviewed by: bz
Differential Revision: https://reviews.freebsd.org/D26537
Repeating the default WARNS here makes it slightly more difficult to
experiment with default WARNS changes, e.g. if we did something absolutely
bananas and introduced a WARNS=7 and wanted to try lifting the default to
that.
Drop most of them; there is one in the blake2 kernel module, but I suspect
it should be dropped -- the default WARNS in the rest of the build doesn't
currently apply to kernel modules, and I haven't put too much thought into
whether it makes sense to make it so.
Add EXAMPLES section to the man page showing the use of all flags except for
-S.
While here, clarify -f description. It not only suppresses diagnostic messages
but it also affects the exit status of the command itself. This is shown in two
of the examples.
Approved by: bcr@
Differential Revision: https://reviews.freebsd.org/D26588
Fix unquoted test for an empty value, which broke nextboot(8) on non-ZFS /boot
systems after r365938.
Discussed with: allanjude, tsoome
X-MFC-With: r365938
The purpose of checksize() is to verify that the referenced cluster
chain size matches the recorded file size (up to 2^32 - 1) in the
directory entry. We follow the cluster chain, then multiple the
cluster count by bytes per cluster to get the physical size, then
check it against the recorded size.
When a file is close to 4 GiB (between 4GiB - cluster size and 4GiB,
both non-inclusive), the product of cluster count and bytes per
cluster would be exactly 4 GiB. On 32-bit systems, because size_t
is 32-bit, this would wrap back to 0, which will cause the file be
truncated to 0.
Fix this by using 64-bit physicalSize instead.
This fix is inspired by an Android change request at
https://android-review.googlesource.com/c/platform/external/fsck_msdos/+/1428461
PR: 249533
Reviewed by: kevlo
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26524
This fixes a compile issue under gcc6 which complains about
legacy style C function declarations.
Differential Revision: https://reviews.freebsd.org/D26504
nvmecontrol tries to upload firmware in chunks as large as it thinks
the device permits. It fails to take into account the FWUG value used
by some drives to advertise the size and alignment limits for firmware
chunks.
- Use the firwmare update granularity value from the
- If the granularity is not reported or not restricted, fall back to
the previously existing logic that calculates the max transfer
size based on MDTS.
- Add firmware update granularity to the identify-controller output.
Reviewed by: imp (previous version), chuck
Obtained from: Dell EMC Isilon
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D26390
bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.
By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.
By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.
bootonce dataset name is recorded in boot pool labels, bootenv area.
in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.
bootonce and nextboot features are usable in both UEFI and BIOS boot.
To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).
At this time, only lua loader is updated.
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512
API to the sbget() and sbput() interfaces. Specifically they take
a file descriptor pointer rather than the struct uufsd *disk pointer
used by the libufs cgread() and cgwrite() interfaces. Update fsck_ffs
to use these revised interfaces.
No functional changes intended.
Sponsored by: Netflix
The current default is provided in various Makefile.inc in some top-level
directories and covers a good portion of the tree, but doesn't cover parts
of the build a little deeper (e.g. libcasper).
Provide a default in src.sys.mk and set WARNS to it in bsd.sys.mk if that
variable is defined. This lets us relatively cleanly provide a default WARNS
no matter where you're building in the src tree without breaking things
outside of the tree.
Crunchgen has been updated as a bootstrap tool to work on this change
because it needs r365605 at a minimum to succeed. The cleanup necessary to
successfully walk over this change on WITHOUT_CLEAN builds has been added.
There is a supplemental project to this to list all of the warnings that are
encountered when the environment has WARNS=6 NO_WERROR=yes:
https://warns.kevans.dev -- this project will hopefully eventually go away
in favor of CI doing a much better job than it.
Reviewed by: emaste, brooks, ngie (all earlier version)
Reviewed by: emaste, arichardson (depend-cleanup.sh change)
Differential Revision: https://reviews.freebsd.org/D26455
These are similar to the existing VLAN capabilities.
Reviewed by: kib@
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D25873
I have been trying to reduce the time that testsuite runs take for CheriBSD
on QEMU (currently about 22 hours). One of the slowest tests is pfctl_test:
Just listing the available test cases currently takes 98 seconds on a
CheriBSD RISC-V system due to all the processes being spawned. This trivial
patch reduces the time to 92 seconds. The better solution would be to
rewrite the test in C/C++ which I may do as a follow-up change.
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D26417
This would allow interested parties to do experimental runs with an
environment set appropriately to raise all the warnings throughout the
build; e.g. env WARNS=6 NO_WERROR=yes buildworld.
Not currently touching the numerous instances in ^/tools.
MFC after: 1 week
Properly handle the case, when some opcode keywords follow after
the `frag` opcode without additional options.
Reported by: Evgeniy Khramtsov <evgeniy at khramtsov org>
ping has the option to add the (deprecated) TOS byte
using the -z option. Adding the same option, with the
same (deprecated) Traffic Class Byte (nowadays actually
DSCP and ECN fields) to ping6 to validate proper QoS
processing in network switches.
Reviewed by: tuexen
MFC after: 2 weeks
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D26384
The request offset and length are always unsigned, so print them as
such.
Submitted by: Yoshihiro Ota <ota@j.email.ne.jp>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21388
o Enhance dependency loop logging: print full chain instead of the
last link competing the loop;
o Add -g option to generate dependency graph suitable for GraphViz
visualization, loops and other graph generation issues are highlighted
automatically;
o Add -p option that enables grouping items that can be processed in
parallel.
Submitted by: Boris Lytochkin <lytboris at gmail>
Reviewed by: melifaro
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25389
The badsys() handler for SIGSYS was added as a transtion aid for kernels
lacking sysctl() in 1993. It is unsafe and unsound so remove it rather
than running the risk of a privilege-dropping system call being silently
omitted.
This partially reverts SCCSID 6.12 (Berkeley) 03/03/93 "add code to
change the system security level".
Reviewed by: mckusick, imp, kevans
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26289
- Change the type of hw.pagesizes to OPAQUE, since it returns an array.
- Modify the handler to only truncate the returned length if the caller
supplied an output buffer. This allows use of the trick of passing a
NULL output buffer to fetch the output size, while preserving
compatibility if MAXPAGESIZES is increased.
- Add a "S,pagesize" formatter to sysctl(8).
Reviewed by: alc, kib
MFC after: 2 weeks
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26239
