Commit Graph

805 Commits

Author SHA1 Message Date
Li-Wen Hsu
7fb6c523bc Temporarily skip sys.sys.qmath_test.qdivq_s64q in CI because it is unstable
PR:		240219
Discussed with:	trasz
Sponsored by:	The FreeBSD Foundation
2019-09-03 10:49:13 +00:00
Edward Tomasz Napierala
3dd61c212c Fix qmath(3) build problems with GCC 8.
Discussed with:	asomers
Sponsored by:	Klara Systems
Differential Revision:	https://reviews.freebsd.org/D21442
2019-08-29 07:39:31 +00:00
Alan Somers
5e63333052 fusefs: Fix some bugs regarding the size of the LISTXATTR list
* A small error in r338152 let to the returned size always being exactly
  eight bytes too large.

* The FUSE_LISTXATTR operation works like Linux's listxattr(2): if the
  caller does not provide enough space, then the server should return ERANGE
  rather than return a truncated list.  That's true even though in FUSE's
  case the kernel doesn't provide space to the client at all; it simply
  requests a maximum size for the list.  We previously weren't handling the
  case where the server returns ERANGE even though the kernel requested as
  much size as the server had told us it needs; that can happen due to a
  race.

* We also need to ensure that a pathological server that always returns
  ERANGE no matter what size we request in FUSE_LISTXATTR won't cause an
  infinite loop in the kernel.  As of this commit, it will instead cause an
  infinite loop that exits and enters the kernel on each iteration, allowing
  signals to be processed.

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21287
2019-08-28 04:19:37 +00:00
Edward Tomasz Napierala
912b6fa37e Fix build on 32 bit archs. 2019-08-27 13:55:45 +00:00
Edward Tomasz Napierala
5a38af1fb8 Introduce <sys/qmath.h>, a fixed-point math library from Netflix.
This makes it possible to perform mathematical operations
on
fractional values without using floating point. It operates on Q
numbers, which are integer-sized, opaque structures initialized
to hold a chosen number of integer and fractional
bits.


For a general description of the Q number system, see the "Fixed Point
Representation & Fractional Math" whitepaper[1]; for the actual
API see the qmath(3) man page.

This is one of dependencies for the upcoming stats(3) framework[2]
that will be applied to the TCP stack in a later commit.

1. https://www.superkits.net/whitepapers/Fixed%20Point%20Representation%20&%20Fractional%20Math.pdf
2. https://reviews.freebsd.org/D20477

Reviewed by:	bcr (man pages, earlier version), sef (earlier version)
Discussed with:	cem, dteske, imp, lstewart
Sponsored By:	Klara Inc, Netflix
Obtained from:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20116
2019-08-27 11:46:22 +00:00
Kyle Evans
95aa96f3a8 tests: shm_open(2): Verify FD_CLOEXEC
Motivated by the fact that I'm messing around near the implementation and
wanting to ensure this doesn't get messed up in the process.

MFC after:	1 week
2019-08-25 02:38:45 +00:00
Li-Wen Hsu
74fa3a52ce Fix failure test cases after r351423 due to ping6(8) options changed
Failure test cases:
    sys.netpfil.common.pass_block.pf_v6
    sys.netpfil.pf.pass_block.noalias
    sys.netpfil.pf.pass_block.v6

Sponsored by:	The FreeBSD Foundation
2019-08-24 01:28:39 +00:00
Kristof Provost
39cae0d5ed netpfil tests: Add forward test for the three firewalls
Submitted by:	Ahsan Barkati
Reviewed by:	kp
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21321
2019-08-23 12:11:46 +00:00
Kyle Evans
e21f96a811 mips: hide regnum definitions behind _KERNEL/_WANT_MIPS_REGNUM
machine/regnum.h ends up being included by sys/procfs.h and sys/ptrace.h via
machine/reg.h. Many of the regnum definitions are too short and too generic
to be exposing to any userland application including one of these two
headers. Moreover, these actively cause build failures in googletest
(template <typename T1 ...> expanding to template <typename 9 ...>).

Hide the definitions behind _KERNEL or _WANT_MIPS_REGNUM, and patch all of
the userland consumers to define as needed.

Discussed with:	imp, jhb
Reviewed by:	imp, jhb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21330
2019-08-22 21:43:21 +00:00
Kyle Evans
92a8983bab Fix the build with WITHOUT_GOOGLETEST
Attempting to build the fusefs tests WITHOUT_GOOGLETEST will result in an
error if the host system or sysroot doesn't already have googletest headers
in /usr/include/private (e.g. host built/installed WITHOUT_GOOGLETEST, clean
cross-buildworld WITHOUT_GOOGLETEST).

Reviewed by:	asomers
Differential Revision:	https://reviews.freebsd.org/D21367
2019-08-22 19:10:31 +00:00
Li-Wen Hsu
cdac716946 Fix path issues after r351212
This fixes sys.netpfil.pf.forward.v4 and sys.netpfil.pf.icmp.cve_2019_5598
failures in CI system.

Sponsored by:	The FreeBSD Foundation
2019-08-22 12:08:35 +00:00
Kristof Provost
3e1ebe7fb8 netpfil tests: Add too many fragments test for pf, ipfw and ipf
Add test for checking that the packets are dropped if it is fragmented into
more than the defined value.

Submitted by:	Ahsan Barkati
Reviewed by:	kp
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21307
2019-08-20 14:46:12 +00:00
Kristof Provost
df3d236d96 netpfil tests: Add the set ToS test for ipfw and pf
This test tests the following:

 - The firewall is able to set the tos bits
 - The firewall is able to set the DSCP bits when EN bits is already set and
   the EN bits remains unchanged.
 - The firewall is able to drop the packets based on ToS value

Submitted by:	Ahsan Barkati
Reviewed by:	kp
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21305
2019-08-20 14:31:22 +00:00
Kristof Provost
9531253098 netpfil tests: Move pft_ping.py and sniffer.py to the common test directory
The pft_ping.py and sniffer.py tool is moved from tests/sys/netpfil/pf to
tests/sys/netpfil/common directory because these tools are to be used in
common for all the firewalls.

Submitted by:	Ahsan Barkati
Reviewed by:	kp, thj
Sponsored by:	Google, Inc. (GSoC 2019)
Differential Revision:	https://reviews.freebsd.org/D21276
2019-08-19 10:48:27 +00:00
Konstantin Belousov
de567a4bef sys.kern.pdeathsig.signal_delivered_ptrace: fix startup.
Inform D that C executed procctl(PROC_PDEATHSIG_CTL).  Otherwise D
might allow B to exit before C is set up to receive a signal on the
parent exit.  In this case, C waits forever for the signal and test
hangs.

PR:	237657
Reported and tested by:	lwhsu
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-08-19 09:33:09 +00:00
Konstantin Belousov
39ba52c0b7 sys.kern.pdeathsig.signal_delivered_ptrace: fix debugger detach
ptrace(PT_DETACH) requires stopped debuggee, otherwise it fails.  When
the call fails, the C process is left as debuggee of the process D,
and might be killed too early if process D exits occurs fast enough.

Since pipes are not closed in the forked children, this resulted in
the test hanging, since no write occured from C to wake A.

PR:	237657
Reported and tested by:	lwhsu
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-08-19 09:29:27 +00:00
Tom Jones
0d9da68f01 Add basic NAT test for pf, ipf and ipfw
Add common firewall NAT tests for pf, ipf and ipfw (using both in-kernel and
userspace NAT).

Submitted by:   Ahsan Barkati
Sponsored by:   Google, Inc. (GSoC 2019)
Reviewed by:    kp
Approved by:    bz (mentor)
MFC after:      2 weeks
Differential Revision: https://reviews.freebsd.org/D21199
2019-08-17 06:44:11 +00:00
Alan Somers
3a79e8e772 fusefs: don't send the namespace during listextattr
The FUSE_LISTXATTR operation always returns the full list of a file's
extended attributes, in all namespaces. There's no way to filter the list
server-side. However, currently FreeBSD's fusefs driver sends a namespace
string with the FUSE_LISTXATTR request. That behavior was probably copied
from fuse_vnop_getextattr, which has an attribute name argument. It's
been there ever since extended attribute support was added in r324620. This
commit removes it.

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21280
2019-08-16 05:06:54 +00:00
Mateusz Piotrowski
03d8a4b7d3 pf tests: Fix accidental duplication of content
Some files got their contented duplicated in r345409. Some mistakes where
fixed in r345430. The only file that was left with a duplicated content was
CVE-2019-5598.py.

Reviewed by:	kp
Approved by:	src (kp)
Differential Revision:	https://reviews.freebsd.org/D21267
2019-08-15 12:00:59 +00:00
Alan Somers
38a3e0bd6b fusefs: fix conditional from r351061
The entirety of r351061 was a copy/paste error.  I'm sorry I've been
comitting so hastily.

Reported by:	rpokala
Reviewed by:	rpokala
MFC after:	2 weeks
MFC-With:	351061
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21265
2019-08-15 04:47:42 +00:00
Alan Somers
f6b344e560 fusefs: fix the 32-bit build after 351042
Reported by:	jhb
MFC after:	2 weeks
MFC-With:	351042
Sponsored by:	The FreeBSD Foundation
2019-08-15 00:23:03 +00:00
Alan Somers
bf50749773 fusefs: Fix the size of fuse_getattr_in
In FUSE protocol 7.9, the size of the FUSE_GETATTR request has increased.
However, the fusefs driver is currently not sending the additional fields.
In our implementation, the additional fields are always zero, so I there
haven't been any test failures until now.  But fusefs-lkl requires the
request's length to be correct.

Fix this bug, and also enhance the test suite to catch similar bugs.

PR:		239830
MFC after:	2 weeks
MFC-With:	350665
Sponsored by:	The FreeBSD Foundation
2019-08-14 20:45:00 +00:00
Alan Somers
331884f291 fusefs: fix intermittency in the default_permissions.Unlink.ok test
The test needs to expect a FUSE_FORGET operation. Most of the time the test
would pass anyway, because by chance FUSE_FORGET would arrive after the
unmount.

MFC after:	2 weeks
MFC-With:	350665
Sponsored by:	The FreeBSD Foundation
2019-08-14 18:04:04 +00:00
Alan Somers
dbee856aff fusefs: skip some tests when unsafe aio is disabled
MFC after:      15 days
MFC-With:       r350665
Sponsored by:   The FreeBSD Foundation
2019-08-13 15:52:28 +00:00
Alan Somers
1fa8ebfbbb fusefs: add SVN Keywords to the test files
Reported by:	SVN pre-commit hooks
MFC after:	15 days
MFC-With:	r350665
Sponsored by:	The FreeBSD Foundation
2019-08-13 15:49:40 +00:00
Alan Somers
c2265ae7a8 fusefs: skip some tests when unsafe aio is disabled
MFC after:	16 days
MFC-With:	r350665
Sponsored by:	The FreeBSD Foundation
2019-08-12 20:00:21 +00:00
Enji Cooper
84a457c6c0 tests/sys/opencrypto: enable armv8crypto on aarch64
This change makes required modifications in runtests to also only require the
aesni module on Intel (i386/amd64) platforms, as it is an Intel specific
module.

MFC after:	1 month
MFC to:		^/stable/12 (support not present on ^/stable/11)
Submitted by:	Greg V <greg@unrelenting.technology>
Differential Revision:	https://reviews.freebsd.org/D21018
2019-08-10 15:53:42 +00:00
Brooks Davis
4a045a66fd Don't add -Wno-class-memaccess with older gcc.
This is a gcc 8.0+ warning which needed to be silenced on for the riscv
build.  amd64-xtoolchain-gcc still uses gcc 6.4.0 and does not understand
this flag.

Reviewed by:	asomers
Feedback from:	imp
Differential Revision:	https://reviews.freebsd.org/D21195
2019-08-09 23:50:57 +00:00
Li-Wen Hsu
84922a5d4c Get configuration variable with default value for not breaking default setting
Reported by:	markj
Sponsored by:	The FreeBSD Foundation
2019-08-07 21:02:23 +00:00
Alan Somers
0b4275accb fusefs: merge from projects/fuse2
This commit imports the new fusefs driver. It raises the protocol level
from 7.8 to 7.23, fixes many bugs, adds a test suite for the driver, and
adds many new features. New features include:

* Optional kernel-side permissions checks (-o default_permissions)
* Implement VOP_MKNOD, VOP_BMAP, and VOP_ADVLOCK
* Allow interrupting FUSE operations
* Support named pipes and unix-domain sockets in fusefs file systems
* Forward UTIME_NOW during utimensat(2) to the daemon
* kqueue support for /dev/fuse
* Allow updating mounts with "mount -u"
* Allow exporting fusefs file systems over NFS
* Server-initiated invalidation of the name cache or data cache
* Respect RLIMIT_FSIZE
* Try to support servers as old as protocol 7.4

Performance enhancements include:

* Implement FUSE's FOPEN_KEEP_CACHE and FUSE_ASYNC_READ flags
* Cache file attributes
* Cache lookup entries, both positive and negative
* Server-selectable cache modes: writethrough, writeback, or uncached
* Write clustering
* Readahead
* Use counter(9) for statistical reporting

PR:		199934 216391 233783 234581 235773 235774 235775
PR:		236226 236231 236236 236291 236329 236381 236405
PR:		236327 236466 236472 236473 236474 236530 236557
PR:		236560 236844 237052 237181 237588 238565
Reviewed by:	bcr (man pages)
Reviewed by:	cem, ngie, rpokala, glebius, kib, bde, emaste (post-commit
		review on project branch)
MFC after:	3 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Pull Request:	https://reviews.freebsd.org/D21110
2019-08-07 00:38:26 +00:00
Jilles Tjoelker
cb71f1fa01 Add a test for kill() on a zombie 2019-08-06 21:17:22 +00:00
Mariusz Zaborski
fd631bcd95 procdesc: fix reparenting when the debugger is attached
The process is reparented to the debugger while it is attached.
  B          B
 /   ---->   |
A          A D

Every time when the process is reparented, it is added to the orphan list
of the previous parent:

A->orphan = B
D->orphan = NULL

When the A process will close the process descriptor to the B process,
the B process will be reparented to the init process.
  B            B - init
  |   ---->
A D          A   D

A->orphan = B
D->orphan = B

In this scenario, the B process is in the orphan list of A and D.

When the last process descriptor is closed instead of reparenting
it to the reaper let it stay with the debugger process and set
our previews parent to the reaper.

Add test case for this situation.
Notice that without this patch the kernel will crash with this test case:
panic: orphan 0xfffff8000e990530 of 0xfffff8000e990000 has unexpected oppid 1

Reviewed by:	markj, kib
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20361
2019-08-05 20:15:46 +00:00
Tom Jones
f97a8a3615 Add common firewall test suite
Add a common test suite for the firewalls included in the base system. The test
suite allows common test infrastructure to test pf, ipfw and ipf firewalls from
test files containing the setup for all three firewalls.

Add the pass block test for pf, ipfw and ipf. The pass block test checks the
allow/deny functionality of the firewalls tested.

Submitted by:   Ahsan Barkati
Sponsored by:   Google, Inc. (GSoC 2019)
Reviewed by:    kp
Approved by:    bz (co-mentor)
MFC after:      2 weeks
Differential Revision: https://reviews.freebsd.org/D21065
2019-08-05 11:47:34 +00:00
Li-Wen Hsu
9777e3544a Only skip test cases sometimes failing in CI when they are running in CI
Suggested by:	jhb
Sponsored by:	The FreeBSD Foundation
2019-08-01 18:19:16 +00:00
Alan Somers
9f13765e42 fusefs: fix building tests with GCC 8
GCC 8 objected to including C++-only flags in CWARNFLAGS

Sponsored by:	The FreeBSD Foundation
2019-07-30 19:47:45 +00:00
Alan Somers
f0c07f0ce8 fusefs: nul-terminate some strings in the readdir test
Reported by:	GCC 8
Sponsored by:	The FreeBSD Foundation
2019-07-30 17:31:09 +00:00
Alan Somers
669a092af1 fusefs: fix panic when writing with O_DIRECT and using writeback cache
When a fusefs file system is mounted using the writeback cache, the cache
may still be bypassed by opening a file with O_DIRECT.  When writing with
O_DIRECT, the cache must be invalidated for the affected portion of the
file.  Fix some panics caused by inadvertently invalidating too much.

Sponsored by:	The FreeBSD Foundation
2019-07-28 15:17:32 +00:00
Alan Somers
a63915c2d7 MFHead @r350386
Sponsored by:	The FreeBSD Foundation
2019-07-28 04:02:22 +00:00
Li-Wen Hsu
1ab93d1f23 Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger

PR:		239425
Sponsored by:	The FreeBSD Foundation
2019-07-24 17:41:40 +00:00
Li-Wen Hsu
c2dc497a38 Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger

PR:		239399
Sponsored by:	The FreeBSD Foundation
2019-07-23 09:39:27 +00:00
Li-Wen Hsu
ea24861d5e Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger

PR:		239397
Sponsored by:	The FreeBSD Foundation
2019-07-23 09:19:58 +00:00
Li-Wen Hsu
7d1f74716c Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__PT_KILL_competing_stop

PR:		220841
Sponsored by:	The FreeBSD Foundation
2019-07-23 07:56:42 +00:00
Li-Wen Hsu
4f6d74c9c4 Temporarily skip sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4
on i386 as they are flakey on it

PR:		239380
Sponsored by:	The FreeBSD Foundation
2019-07-22 18:54:26 +00:00
Li-Wen Hsu
63b0609c12 Fix URL.
Sponsored by:	The FreeBSD Foundation
2019-07-22 18:43:46 +00:00
Li-Wen Hsu
37ba9b348b Temporarily skip flakey test case
sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger

PR:		239292
Sponsored by:	The FreeBSD Foundation
2019-07-22 10:37:56 +00:00
Alan Somers
5a0b9a2776 fusefs: fix warnings in the tests reported by GCC
Sponsored by:	The FreeBSD Foundation
2019-07-20 05:21:13 +00:00
Alan Somers
fca79580be sendfile: don't panic when VOP_GETPAGES_ASYNC returns an error
PR:		236466
Sponsored by:	The FreeBSD Foundation
2019-07-19 18:03:30 +00:00
Alan Somers
ed74f781c9 fusefs: add a intr/nointr mount option
FUSE file systems can optionally support interrupting outstanding
operations.  However, the file system does not identify to the kernel at
mount time whether it's capable of doing that.  Instead it signals its
noncapability by returning ENOSYS to the first FUSE_INTERRUPT operation it
receives.  That's a problem for reliable signal delivery, because the kernel
must choose which thread should get a signal before it knows whether the
FUSE server can handle interrupts.  The problem is even worse because the
FUSE protocol allows a file system to simply ignore all FUSE_INTERRUPT
operations.

Fix the signal delivery logic by making interruptibility an opt-in mount
option.  This will require a corresponding change to libfuse, but not to
most file systems that link to libfuse.

Bump __FreeBSD_version due to the new mount option.

Sponsored by:	The FreeBSD Foundation
2019-07-18 17:55:13 +00:00
Alan Somers
d26d63a4af fusefs: multiple interruptility improvements
1) Don't explicitly not mask SIGKILL.  kern_sigprocmask won't allow it to be
   masked, anyway.

2) Fix an infinite loop bug.  If a process received both a maskable signal
   lower than 9 (like SIGINT) and then received SIGKILL,
   fticket_wait_answer would spin.  msleep would immediately return EINTR,
   but cursig would return SIGINT, so the sleep would get retried.  Fix it
   by explicitly checking whether SIGKILL has been received.

3) Abandon the sig_isfatal optimization introduced by r346357.  That
   optimization would cause fticket_wait_answer to return immediately,
   without waiting for a response from the server, if the process were going
   to exit anyway.  However, it's vulnerable to a race:

   1) fatal signal is received while fticket_wait_answer is sleeping.
   2) fticket_wait_answer sends the FUSE_INTERRUPT operation.
   3) fticket_wait_answer determines that the signal was fatal and returns
      without waiting for a response.
   4) Another thread changes the signal to non-fatal.
   5) The first thread returns to userspace.  Instead of exiting, the
      process continues.
   6) The application receives EINTR, wrongly believes that the operation
      was successfully interrupted, and restarts it.  This could cause
      problems for non-idempotent operations like FUSE_RENAME.

Reported by:    kib (the race part)
Sponsored by:   The FreeBSD Foundation
2019-07-17 22:45:43 +00:00
John Baldwin
32451fb9fc Add ptrace op PT_GET_SC_RET.
This ptrace operation returns a structure containing the error and
return values from the current system call.  It is only valid when a
thread is stopped during a system call exit (PL_FLAG_SCX is set).

The sr_error member holds the error value from the system call.  Note
that this error value is the native FreeBSD error value that has _not_
been translated to an ABI-specific error value similar to the values
logged to ktrace.

If sr_error is zero, then the return values of the system call will be
set in sr_retval[0] and sr_retval[1].

Reviewed by:	kib
MFC after:	1 month
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D20901
2019-07-15 21:48:02 +00:00