Commit Graph

875 Commits

Author SHA1 Message Date
asomers
dc5eb6e3a5 fusefs: Fix some bugs regarding the size of the LISTXATTR list
* A small error in r338152 let to the returned size always being exactly
  eight bytes too large.

* The FUSE_LISTXATTR operation works like Linux's listxattr(2): if the
  caller does not provide enough space, then the server should return ERANGE
  rather than return a truncated list.  That's true even though in FUSE's
  case the kernel doesn't provide space to the client at all; it simply
  requests a maximum size for the list.  We previously weren't handling the
  case where the server returns ERANGE even though the kernel requested as
  much size as the server had told us it needs; that can happen due to a
  race.

* We also need to ensure that a pathological server that always returns
  ERANGE no matter what size we request in FUSE_LISTXATTR won't cause an
  infinite loop in the kernel.  As of this commit, it will instead cause an
  infinite loop that exits and enters the kernel on each iteration, allowing
  signals to be processed.

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21287
2019-08-28 04:19:37 +00:00
trasz
93937c31e9 Fix build on 32 bit archs. 2019-08-27 13:55:45 +00:00
trasz
8914da68f6 Introduce <sys/qmath.h>, a fixed-point math library from Netflix.
This makes it possible to perform mathematical operations
on
fractional values without using floating point. It operates on Q
numbers, which are integer-sized, opaque structures initialized
to hold a chosen number of integer and fractional
bits.


For a general description of the Q number system, see the "Fixed Point
Representation & Fractional Math" whitepaper[1]; for the actual
API see the qmath(3) man page.

This is one of dependencies for the upcoming stats(3) framework[2]
that will be applied to the TCP stack in a later commit.

1. https://www.superkits.net/whitepapers/Fixed%20Point%20Representation%20&%20Fractional%20Math.pdf
2. https://reviews.freebsd.org/D20477

Reviewed by:	bcr (man pages, earlier version), sef (earlier version)
Discussed with:	cem, dteske, imp, lstewart
Sponsored By:	Klara Inc, Netflix
Obtained from:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20116
2019-08-27 11:46:22 +00:00
kevans
71ab388e98 tests: shm_open(2): Verify FD_CLOEXEC
Motivated by the fact that I'm messing around near the implementation and
wanting to ensure this doesn't get messed up in the process.

MFC after:	1 week
2019-08-25 02:38:45 +00:00
lwhsu
7ed91a6e6f Fix failure test cases after r351423 due to ping6(8) options changed
Failure test cases:
    sys.netpfil.common.pass_block.pf_v6
    sys.netpfil.pf.pass_block.noalias
    sys.netpfil.pf.pass_block.v6

Sponsored by:	The FreeBSD Foundation
2019-08-24 01:28:39 +00:00
kp
dd5c45df1f netpfil tests: Add forward test for the three firewalls
Submitted by:	Ahsan Barkati
Reviewed by:	kp
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21321
2019-08-23 12:11:46 +00:00
kevans
450bfda5be mips: hide regnum definitions behind _KERNEL/_WANT_MIPS_REGNUM
machine/regnum.h ends up being included by sys/procfs.h and sys/ptrace.h via
machine/reg.h. Many of the regnum definitions are too short and too generic
to be exposing to any userland application including one of these two
headers. Moreover, these actively cause build failures in googletest
(template <typename T1 ...> expanding to template <typename 9 ...>).

Hide the definitions behind _KERNEL or _WANT_MIPS_REGNUM, and patch all of
the userland consumers to define as needed.

Discussed with:	imp, jhb
Reviewed by:	imp, jhb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21330
2019-08-22 21:43:21 +00:00
kevans
883b4d95e4 Fix the build with WITHOUT_GOOGLETEST
Attempting to build the fusefs tests WITHOUT_GOOGLETEST will result in an
error if the host system or sysroot doesn't already have googletest headers
in /usr/include/private (e.g. host built/installed WITHOUT_GOOGLETEST, clean
cross-buildworld WITHOUT_GOOGLETEST).

Reviewed by:	asomers
Differential Revision:	https://reviews.freebsd.org/D21367
2019-08-22 19:10:31 +00:00
lwhsu
48c02702bb Fix path issues after r351212
This fixes sys.netpfil.pf.forward.v4 and sys.netpfil.pf.icmp.cve_2019_5598
failures in CI system.

Sponsored by:	The FreeBSD Foundation
2019-08-22 12:08:35 +00:00
kp
567631864d netpfil tests: Add too many fragments test for pf, ipfw and ipf
Add test for checking that the packets are dropped if it is fragmented into
more than the defined value.

Submitted by:	Ahsan Barkati
Reviewed by:	kp
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21307
2019-08-20 14:46:12 +00:00
kp
d39fae8637 netpfil tests: Add the set ToS test for ipfw and pf
This test tests the following:

 - The firewall is able to set the tos bits
 - The firewall is able to set the DSCP bits when EN bits is already set and
   the EN bits remains unchanged.
 - The firewall is able to drop the packets based on ToS value

Submitted by:	Ahsan Barkati
Reviewed by:	kp
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21305
2019-08-20 14:31:22 +00:00
kp
20a19e32a4 netpfil tests: Move pft_ping.py and sniffer.py to the common test directory
The pft_ping.py and sniffer.py tool is moved from tests/sys/netpfil/pf to
tests/sys/netpfil/common directory because these tools are to be used in
common for all the firewalls.

Submitted by:	Ahsan Barkati
Reviewed by:	kp, thj
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21276
2019-08-19 10:48:27 +00:00
kib
7beefda00b sys.kern.pdeathsig.signal_delivered_ptrace: fix startup.
Inform D that C executed procctl(PROC_PDEATHSIG_CTL).  Otherwise D
might allow B to exit before C is set up to receive a signal on the
parent exit.  In this case, C waits forever for the signal and test
hangs.

PR:	237657
Reported and tested by:	lwhsu
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-08-19 09:33:09 +00:00
kib
c7867f406d sys.kern.pdeathsig.signal_delivered_ptrace: fix debugger detach
ptrace(PT_DETACH) requires stopped debuggee, otherwise it fails.  When
the call fails, the C process is left as debuggee of the process D,
and might be killed too early if process D exits occurs fast enough.

Since pipes are not closed in the forked children, this resulted in
the test hanging, since no write occured from C to wake A.

PR:	237657
Reported and tested by:	lwhsu
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-08-19 09:29:27 +00:00
thj
19ce7443f6 Add basic NAT test for pf, ipf and ipfw
Add common firewall NAT tests for pf, ipf and ipfw (using both in-kernel and
userspace NAT).

Submitted by:   Ahsan Barkati
Sponsored by:   Google, Inc. (GSoC 2019)
Reviewed by:    kp
Approved by:    bz (mentor)
MFC after:      2 weeks
Differential Revision: https://reviews.freebsd.org/D21199
2019-08-17 06:44:11 +00:00
asomers
f39a1bd142 fusefs: don't send the namespace during listextattr
The FUSE_LISTXATTR operation always returns the full list of a file's
extended attributes, in all namespaces. There's no way to filter the list
server-side. However, currently FreeBSD's fusefs driver sends a namespace
string with the FUSE_LISTXATTR request. That behavior was probably copied
from fuse_vnop_getextattr, which has an attribute name argument. It's
been there ever since extended attribute support was added in r324620. This
commit removes it.

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21280
2019-08-16 05:06:54 +00:00
0mp
ea36ac6047 pf tests: Fix accidental duplication of content
Some files got their contented duplicated in r345409. Some mistakes where
fixed in r345430. The only file that was left with a duplicated content was
CVE-2019-5598.py.

Reviewed by:	kp
Approved by:	src (kp)
Differential Revision:	https://reviews.freebsd.org/D21267
2019-08-15 12:00:59 +00:00
asomers
3d3ceb2be6 fusefs: fix conditional from r351061
The entirety of r351061 was a copy/paste error.  I'm sorry I've been
comitting so hastily.

Reported by:	rpokala
Reviewed by:	rpokala
MFC after:	2 weeks
MFC-With:	351061
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21265
2019-08-15 04:47:42 +00:00
asomers
060f336a18 fusefs: fix the 32-bit build after 351042
Reported by:	jhb
MFC after:	2 weeks
MFC-With:	351042
Sponsored by:	The FreeBSD Foundation
2019-08-15 00:23:03 +00:00
asomers
2026d5ae4a fusefs: Fix the size of fuse_getattr_in
In FUSE protocol 7.9, the size of the FUSE_GETATTR request has increased.
However, the fusefs driver is currently not sending the additional fields.
In our implementation, the additional fields are always zero, so I there
haven't been any test failures until now.  But fusefs-lkl requires the
request's length to be correct.

Fix this bug, and also enhance the test suite to catch similar bugs.

PR:		239830
MFC after:	2 weeks
MFC-With:	350665
Sponsored by:	The FreeBSD Foundation
2019-08-14 20:45:00 +00:00
asomers
f2c2e8bf45 fusefs: fix intermittency in the default_permissions.Unlink.ok test
The test needs to expect a FUSE_FORGET operation. Most of the time the test
would pass anyway, because by chance FUSE_FORGET would arrive after the
unmount.

MFC after:	2 weeks
MFC-With:	350665
Sponsored by:	The FreeBSD Foundation
2019-08-14 18:04:04 +00:00
asomers
c2eb452161 fusefs: skip some tests when unsafe aio is disabled
MFC after:      15 days
MFC-With:       r350665
Sponsored by:   The FreeBSD Foundation
2019-08-13 15:52:28 +00:00
asomers
8cbbd74e18 fusefs: add SVN Keywords to the test files
Reported by:	SVN pre-commit hooks
MFC after:	15 days
MFC-With:	r350665
Sponsored by:	The FreeBSD Foundation
2019-08-13 15:49:40 +00:00
ngie
6125c63e8c tests/sys/opencrypto: enable armv8crypto on aarch64
This change makes required modifications in runtests to also only require the
aesni module on Intel (i386/amd64) platforms, as it is an Intel specific
module.

MFC after:	1 month
MFC to:		^/stable/12 (support not present on ^/stable/11)
Submitted by:	Greg V <greg@unrelenting.technology>
Differential Revision:	https://reviews.freebsd.org/D21018
2019-08-10 15:53:42 +00:00
brooks
f01ed61a39 Don't add -Wno-class-memaccess with older gcc.
This is a gcc 8.0+ warning which needed to be silenced on for the riscv
build.  amd64-xtoolchain-gcc still uses gcc 6.4.0 and does not understand
this flag.

Reviewed by:	asomers
Feedback from:	imp
Differential Revision:	https://reviews.freebsd.org/D21195
2019-08-09 23:50:57 +00:00
lwhsu
0177685a5b Get configuration variable with default value for not breaking default setting
Reported by:	markj
Sponsored by:	The FreeBSD Foundation
2019-08-07 21:02:23 +00:00
asomers
059ede2b0f fusefs: merge from projects/fuse2
This commit imports the new fusefs driver. It raises the protocol level
from 7.8 to 7.23, fixes many bugs, adds a test suite for the driver, and
adds many new features. New features include:

* Optional kernel-side permissions checks (-o default_permissions)
* Implement VOP_MKNOD, VOP_BMAP, and VOP_ADVLOCK
* Allow interrupting FUSE operations
* Support named pipes and unix-domain sockets in fusefs file systems
* Forward UTIME_NOW during utimensat(2) to the daemon
* kqueue support for /dev/fuse
* Allow updating mounts with "mount -u"
* Allow exporting fusefs file systems over NFS
* Server-initiated invalidation of the name cache or data cache
* Respect RLIMIT_FSIZE
* Try to support servers as old as protocol 7.4

Performance enhancements include:

* Implement FUSE's FOPEN_KEEP_CACHE and FUSE_ASYNC_READ flags
* Cache file attributes
* Cache lookup entries, both positive and negative
* Server-selectable cache modes: writethrough, writeback, or uncached
* Write clustering
* Readahead
* Use counter(9) for statistical reporting

PR:		199934 216391 233783 234581 235773 235774 235775
PR:		236226 236231 236236 236291 236329 236381 236405
PR:		236327 236466 236472 236473 236474 236530 236557
PR:		236560 236844 237052 237181 237588 238565
Reviewed by:	bcr (man pages)
Reviewed by:	cem, ngie, rpokala, glebius, kib, bde, emaste (post-commit
		review on project branch)
MFC after:	3 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Pull Request:	https://reviews.freebsd.org/D21110
2019-08-07 00:38:26 +00:00
jilles
798cd23a8e Add a test for kill() on a zombie 2019-08-06 21:17:22 +00:00
oshogbo
a629021f11 procdesc: fix reparenting when the debugger is attached
The process is reparented to the debugger while it is attached.
  B          B
 /   ---->   |
A          A D

Every time when the process is reparented, it is added to the orphan list
of the previous parent:

A->orphan = B
D->orphan = NULL

When the A process will close the process descriptor to the B process,
the B process will be reparented to the init process.
  B            B - init
  |   ---->
A D          A   D

A->orphan = B
D->orphan = B

In this scenario, the B process is in the orphan list of A and D.

When the last process descriptor is closed instead of reparenting
it to the reaper let it stay with the debugger process and set
our previews parent to the reaper.

Add test case for this situation.
Notice that without this patch the kernel will crash with this test case:
panic: orphan 0xfffff8000e990530 of 0xfffff8000e990000 has unexpected oppid 1

Reviewed by:	markj, kib
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20361
2019-08-05 20:15:46 +00:00
thj
971524ae62 Add common firewall test suite
Add a common test suite for the firewalls included in the base system. The test
suite allows common test infrastructure to test pf, ipfw and ipf firewalls from
test files containing the setup for all three firewalls.

Add the pass block test for pf, ipfw and ipf. The pass block test checks the
allow/deny functionality of the firewalls tested.

Submitted by:   Ahsan Barkati
Sponsored by:   Google, Inc. (GSoC 2019)
Reviewed by:    kp
Approved by:    bz (co-mentor)
MFC after:      2 weeks
Differential Revision: https://reviews.freebsd.org/D21065
2019-08-05 11:47:34 +00:00
lwhsu
62a7fa5ebb Only skip test cases sometimes failing in CI when they are running in CI
Suggested by:	jhb
Sponsored by:	The FreeBSD Foundation
2019-08-01 18:19:16 +00:00
asomers
f16a67fc58 fusefs: fix building tests with GCC 8
GCC 8 objected to including C++-only flags in CWARNFLAGS

Sponsored by:	The FreeBSD Foundation
2019-07-30 19:47:45 +00:00
asomers
87ceed399d fusefs: nul-terminate some strings in the readdir test
Reported by:	GCC 8
Sponsored by:	The FreeBSD Foundation
2019-07-30 17:31:09 +00:00
asomers
10eed53afb fusefs: fix panic when writing with O_DIRECT and using writeback cache
When a fusefs file system is mounted using the writeback cache, the cache
may still be bypassed by opening a file with O_DIRECT.  When writing with
O_DIRECT, the cache must be invalidated for the affected portion of the
file.  Fix some panics caused by inadvertently invalidating too much.

Sponsored by:	The FreeBSD Foundation
2019-07-28 15:17:32 +00:00
asomers
90daad7031 MFHead @r350386
Sponsored by:	The FreeBSD Foundation
2019-07-28 04:02:22 +00:00
lwhsu
7879c1cd2f Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger

PR:		239425
Sponsored by:	The FreeBSD Foundation
2019-07-24 17:41:40 +00:00
lwhsu
5d004f4712 Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger

PR:		239399
Sponsored by:	The FreeBSD Foundation
2019-07-23 09:39:27 +00:00
lwhsu
ee2defc067 Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger

PR:		239397
Sponsored by:	The FreeBSD Foundation
2019-07-23 09:19:58 +00:00
lwhsu
3178aa4214 Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__PT_KILL_competing_stop

PR:		220841
Sponsored by:	The FreeBSD Foundation
2019-07-23 07:56:42 +00:00
lwhsu
f9a5985875 Temporarily skip sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4
on i386 as they are flakey on it

PR:		239380
Sponsored by:	The FreeBSD Foundation
2019-07-22 18:54:26 +00:00
lwhsu
66b0becfe5 Fix URL.
Sponsored by:	The FreeBSD Foundation
2019-07-22 18:43:46 +00:00
lwhsu
1872e290af Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger

PR:		239292
Sponsored by:	The FreeBSD Foundation
2019-07-22 10:37:56 +00:00
asomers
ba6d18a0c1 fusefs: fix warnings in the tests reported by GCC
Sponsored by:	The FreeBSD Foundation
2019-07-20 05:21:13 +00:00
asomers
67182c4a60 sendfile: don't panic when VOP_GETPAGES_ASYNC returns an error
PR:		236466
Sponsored by:	The FreeBSD Foundation
2019-07-19 18:03:30 +00:00
asomers
964ad03769 fusefs: add a intr/nointr mount option
FUSE file systems can optionally support interrupting outstanding
operations.  However, the file system does not identify to the kernel at
mount time whether it's capable of doing that.  Instead it signals its
noncapability by returning ENOSYS to the first FUSE_INTERRUPT operation it
receives.  That's a problem for reliable signal delivery, because the kernel
must choose which thread should get a signal before it knows whether the
FUSE server can handle interrupts.  The problem is even worse because the
FUSE protocol allows a file system to simply ignore all FUSE_INTERRUPT
operations.

Fix the signal delivery logic by making interruptibility an opt-in mount
option.  This will require a corresponding change to libfuse, but not to
most file systems that link to libfuse.

Bump __FreeBSD_version due to the new mount option.

Sponsored by:	The FreeBSD Foundation
2019-07-18 17:55:13 +00:00
asomers
9b3c8b4203 fusefs: multiple interruptility improvements
1) Don't explicitly not mask SIGKILL.  kern_sigprocmask won't allow it to be
   masked, anyway.

2) Fix an infinite loop bug.  If a process received both a maskable signal
   lower than 9 (like SIGINT) and then received SIGKILL,
   fticket_wait_answer would spin.  msleep would immediately return EINTR,
   but cursig would return SIGINT, so the sleep would get retried.  Fix it
   by explicitly checking whether SIGKILL has been received.

3) Abandon the sig_isfatal optimization introduced by r346357.  That
   optimization would cause fticket_wait_answer to return immediately,
   without waiting for a response from the server, if the process were going
   to exit anyway.  However, it's vulnerable to a race:

   1) fatal signal is received while fticket_wait_answer is sleeping.
   2) fticket_wait_answer sends the FUSE_INTERRUPT operation.
   3) fticket_wait_answer determines that the signal was fatal and returns
      without waiting for a response.
   4) Another thread changes the signal to non-fatal.
   5) The first thread returns to userspace.  Instead of exiting, the
      process continues.
   6) The application receives EINTR, wrongly believes that the operation
      was successfully interrupted, and restarts it.  This could cause
      problems for non-idempotent operations like FUSE_RENAME.

Reported by:    kib (the race part)
Sponsored by:   The FreeBSD Foundation
2019-07-17 22:45:43 +00:00
jhb
bd67f2ec6b Add ptrace op PT_GET_SC_RET.
This ptrace operation returns a structure containing the error and
return values from the current system call.  It is only valid when a
thread is stopped during a system call exit (PL_FLAG_SCX is set).

The sr_error member holds the error value from the system call.  Note
that this error value is the native FreeBSD error value that has _not_
been translated to an ABI-specific error value similar to the values
logged to ktrace.

If sr_error is zero, then the return values of the system call will be
set in sr_retval[0] and sr_retval[1].

Reviewed by:	kib
MFC after:	1 month
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D20901
2019-07-15 21:48:02 +00:00
jhb
64ee41ff55 Add a test for PT_GET_SC_ARGS.
Reviewed by:	kib
MFC after:	1 month
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D20899
2019-07-15 21:26:55 +00:00
asomers
317f02f891 projects/fuse2: build fixes
* Fix the kernel build with gcc by removing a redundant extern declaration
* In the tests, fix a printf format specifier that assumed LP64

Sponsored by:	The FreeBSD Foundation
2019-07-13 14:42:09 +00:00
lwhsu
7de5e75b50 Correct definitions in sys.opencrypto.runtests.main for 32bit platform
Reviewed by:	cem, jhb
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20894
2019-07-10 01:08:08 +00:00