Commit Graph

265 Commits

Author SHA1 Message Date
Enji Cooper
15b69ab272 Wait for /dev/ggate* to appear after calling ggatel create in :ggatel_{file,md}
The test assumed that `ggatel create` created a device on completion, but that's
incorrect. This squashes the race by waiting for the device to appear, as
`ggatel create` daemonizes before issuing an ioctl to geom_gate(4) if not called
with `-v`.

Discussed with:	asomers
MFC after:	1 week
PR:		204616
Sponsored by:	Dell EMC Isilon
2017-01-31 06:12:51 +00:00
Conrad Meyer
6be2ff7d3e calculate_crc32c: Add SSE4.2 implementation on x86
Derived from an implementation by Mark Adler.

The fast loop performs three simultaneous CRCs over subsets of the data
before composing them.  This takes advantage of certain properties of
the CRC32 implementation in Intel hardware.  (The CRC instruction takes 1
cycle but has 2-3 cycles of latency.)

The CRC32 instruction does not manipulate FPU state.

i386 does not have the crc32q instruction, so avoid it there.  Otherwise
the implementation is identical to amd64.

Add basic userland tests to verify correctness on a variety of inputs.

PR:		216467
Reported by:	Ben RUBSON <ben.rubson at gmail.com>
Reviewed by:	kib@, markj@ (earlier version)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D9342
2017-01-31 03:26:32 +00:00
Robert Watson
55e0d88afd Fix build of aio_test on MIPS, where the compiler warns about the local
variable 'err' shadowing the global function err(3).  Which it does.

Sponsored by:	DARPA, AFRL
2017-01-28 12:26:22 +00:00
Alan Somers
740a9bfadd Improve the aio tests
* Add tests for aio_suspend(2).
* Add tests for polled completion notification.
* Test the full matrix of file descriptor types and completion notification
  mechanisms.
* Don't bother with mkstemp, because ATF runs every test in its own temp dir.
* Fix some typos.
* Remove extraneous ATF_REQUIRE_KERNEL_MODULE calls.

Reviewed by:	jhb
MFC after:	4 weeks
Differential Revision:	https://reviews.freebsd.org/D9045
2017-01-28 04:33:51 +00:00
Enji Cooper
183ae521e5 Use _SED instead of hacking tests rewriting mknod ... p as mkfifo
Similar to r312297
2017-01-16 17:53:58 +00:00
Enji Cooper
267f10b14c Use _SED to rewrite mknod ... p command as mkfifo instead of
adding an unnecessary diff to the test
2017-01-16 17:49:53 +00:00
Enji Cooper
9b6d988185 Fix typo in r312216
I meant to replace "exp" with "exponent", not "expected"

MFC after:	13 days
Pointyhat to:	ngie
Submitted by:	bde
2017-01-15 10:29:53 +00:00
Enji Cooper
d75a788085 Revert r312119 and reword the intent to fix -Wshadow issues
between exp(3) and `exp` var.

The approach taken previously was not ideal for multiple
functional and stylistic reasons.

Add to existing sed call in Makefile to replace `exp` with
`exponent` instead.

MFC after:	13 days
Requested by:	bde
2017-01-15 09:25:33 +00:00
Enji Cooper
437999a767 Mark testcases which use cap_enter as expected failures until the
PR is resolved so those of us that run the tests don't have the
bogus failures counted against our overall results

PR:	215690
2017-01-15 09:13:41 +00:00
Enji Cooper
15dbd418f6 Add include Makefiles for tests/sys/{fs,kern,kqueue,mac}/...
The primary goal for doing this is to leverage the work done in r312114
for enabling WARNS to address trivial code quality issues with new tests

MFC after:	6 days
Tested with:	make tinderbox
Sponsored by:	Dell EMC Isilon
2017-01-14 20:29:26 +00:00
Enji Cooper
4944940b4e Add include Makefiles for tests/sys/{fs,kern,kqueue,mac}/...
The primary goal for doing this is to leverage the work done in r312114
for enabling WARNS to address trivial code quality issues with new tests

MFC after:	6 days
Tested with:	make tinderbox
Sponsored by:	Dell EMC Isilon
2017-01-14 20:21:21 +00:00
Enji Cooper
00ea8a5fc9 Fix -Wformat issue
Use %zu for printing out results from nitems, as it's size_t based

MFC after:	1 week
X-MFC with:	r312120
Reported by:	gcc (mips:mipsel tinderbox)
Sponsored by:	Dell EMC Isilon
2017-01-14 10:38:39 +00:00
Enji Cooper
dcdb30d8f8 Follow up to r312118
State that execve failed instead of just printing out the program name
and strerror(errno) via err(3).

MFC after:	3 days
X-MFC with:	r312118
Sponsored by:	Dell EMC Isilon
2017-01-14 05:24:35 +00:00
Enji Cooper
f614ceaf4a Fix warnings
- Staticize test_num
- Promote i to size_t to deal with -Wsign-compare issues

Tested with:	clang, gcc, gcc49
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-01-14 05:18:18 +00:00
Enji Cooper
dfa7f3a543 Fix -Wformat issue with zero-length format string passed to err(3)
MFC after:	3 days
Tested with:	clang, gcc 4.2.1, gcc 4.9
Sponsored by:	Dell EMC Isilon
2017-01-14 05:02:53 +00:00
Enji Cooper
bf8f6d7c8d Remove WARNS set globally by ../Makefile.inc now
Sponsored by:	Dell EMC Isilon
2017-01-14 04:20:42 +00:00
Enji Cooper
de81acbd15 Enable WARNS?= 6 across all of tests/sys
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-01-14 04:20:06 +00:00
Enji Cooper
54ff563fb9 Remove unused vars to fix -Wunused issues
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-01-14 04:10:04 +00:00
Enji Cooper
5fea45956e Fix -Wsign-compare warnings
The loop index (i) doesn't need to be size_t as its comparison is signed

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-01-14 04:09:01 +00:00
Enji Cooper
02820e5e4e Bump WARNS up to 6 again
Has not been tested (can't be after r312103 without cem's hacks
to atf/kyua)!
2017-01-14 04:00:26 +00:00
Conrad Meyer
5c9ce2504a Follow-up to r312103:
Revert r310995 as well.
2017-01-14 02:29:25 +00:00
Conrad Meyer
47f2efe4fb Revert r310994
Don't implement some terrible hack on a test by test basis.  The
framework fix is straightforward and can be chased up in the original
bug.

Reviewed by:	ngie ("be my guest")
2017-01-14 01:03:20 +00:00
Alan Somers
9a1fe327ae ATFify the geom gate tests.
This ensures their cleanup routines will be run even if they should timeout.

tests/sys/geom/class/gate/ggate_test.sh
tests/sys/geom/class/gate/Makefile
	Add an ATF test with three testcases, one for each TAP test. Use
	ATF-style cleanup functions, and convert sleeps to polling loops.

ObsoleteFiles.inc
tests/sys/geom/class/gate/conf.sh
tests/sys/geom/class/gate/1_test.sh
tests/sys/geom/class/gate/2_test.sh
tests/sys/geom/class/gate/3_test.sh
	Delete TAP test files

Reviewed by:	ngie
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D8891
2017-01-10 20:35:09 +00:00
Konstantin Belousov
1c32456953 Use type-independent formats for printing nlink_t and ino_t.
Extracted from:	ino64 work by gleb, mckusick
Discussed with:	mckusick
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-01-06 16:59:33 +00:00
Enji Cooper
e4162b7955 Bump WARNS to 6 to catch simple QA issues like some of the ones I squashed
in r310994
2017-01-01 04:02:06 +00:00
Enji Cooper
7f9785e873 Make sys/vfs/lookup_cap_dotdot actually work with "kyua test"
The tests don't work when reading/writing to file descriptors in the
sandbox after entering capability mode (and wouldn't have, regardless
of the framework), so adjust the tests so they function within the
framework.

For tests that enter capability mode over the course of the test, the
following is now done:

  1. Fork child process for capability mode test.
  2. In child...
     i.   Enter capability mode.
     ii.  Test invariants.
     iii. Exit after calling test function.
  3. Collect status for child and determine whether or not it completed
     successfully.

In order to test the invariants in the child process, they now use assert(3)
instead of ATF_REQUIRE*, as the atf-c-api functions right to results files
in the directories in order to determine where and how tests fail.

While in the area, fix several -Wshadow and -Wunused warnings found when I
bumped WARNS up to 6, and fix some minor style(9) issues with indentation
and type alignment.

PR:	215690
2017-01-01 04:01:27 +00:00
Alan Somers
162ea6dc66 ATFify the gnop tests
Also, add test cases for the -p, -P, and -s options to gnop create

Reviewed by:	ngie
MFC after:	4 weeks
Differential Revision:	https://reviews.freebsd.org/D8892
2016-12-30 02:22:38 +00:00
Alan Somers
f92ce0224a Reduce the runtime of the GELI tests
There is no reduction in test coverage.  On my system runtime is reduced
from 38m32s to 6m24s.

tests/sys/geom/class/eli/conf.sh
tests/sys/geom/class/eli/init_a_test.sh
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/integrity_copy_test.sh
tests/sys/geom/class/eli/integrity_data_test.sh
tests/sys/geom/class/eli/integrity_hmac_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
tests/sys/geom/class/eli/onetime_test.sh
	Move the looping code into common functions in conf.sh, and remove
	alias ciphers from the list.

tests/sys/geom/class/eli/init_a_test.sh
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/integrity_copy_test.sh
tests/sys/geom/class/eli/integrity_data_test.sh
tests/sys/geom/class/eli/integrity_hmac_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
	Move a few commands that don't need to be in the inner loop out.

tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
	Reduce the sector count

tests/sys/geom/class/eli/Makefile
tests/sys/geom/class/eli/init_alias_test.sh
	Add a test for initializing a GELI device using one of the cipher
	aliases, and check that the alias is correctly interpreted.

MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D8814
2016-12-29 20:28:50 +00:00
Jilles Tjoelker
b9a6fb9343 reaper: Make REAPER_KILL_SUBTREE actually work.
MFC after:	2 weeks
2016-12-14 22:49:20 +00:00
Jilles Tjoelker
fe56b17710 Add tests for reaper receiving SIGCHLD (r309886).
PR:		213928
MFC after:	2 weeks
2016-12-12 22:51:29 +00:00
Conrad Meyer
9f0136cb26 Add basic ATF tests for Capability mode .. lookups
A follow-up to r309887.

Several tests copied verbatim from
https://github.com/emaste/snippets/blob/master/test_openat.c .

Reviewed by:	kib@, ngie@ (earlier version)
X-MFC-With:	r309887
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D8748
2016-12-12 17:23:09 +00:00
Jilles Tjoelker
b1dd83d1d0 Add some tests for reaper functionality (in procctl()).
MFC after:	1 week
2016-12-10 22:05:24 +00:00
Conrad Meyer
695349e306 coredump_phnum_test: Make expected pheader count more flexible
Note: this test still requires binutils readelf (from ports) to pass, until a
few issues are resolved in elftoolchain.

PR:		215019
Reported by:	ngie@
Sponsored by:	Dell EMC Isilon
2016-12-04 03:50:57 +00:00
Enji Cooper
3d77b88dff Expect 01:main to fail
Changes were made to ZFS in the past year with respect to how ACLs
are handled, causing failures in this test. Mark it TODO so (hopefully)
someone more knowledgeable (like mav or trasz) will fix the code or the
test.

PR:	212323
2016-12-03 02:24:15 +00:00
Enji Cooper
c68f00edd6 Expect :coredump_phnum to fail
The number of program headers on my system (GENERIC-NODEBUG) don't match the
expected (hardcoded) number in the test

PR:	215019
2016-12-03 02:17:50 +00:00
Enji Cooper
cebf0ec58a Fix test coredump_phnum_test:coredump_phnum require.config specification
The requirement is set via `atf_set "require.config" "allow_sysctl_side_effects"',
not `atf_set "require.config" "allow_sysctl_side_effects"'

X-MFC with: r308177
Pointyhat to: cmeyer
2016-12-03 02:09:23 +00:00
Conrad Meyer
e8e39fc29e Add test case for >65535 segment coredumps
A long-belated follow-up to r303099.

With feedback from:	jmmv, ngie
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D7264
2016-11-01 19:18:16 +00:00
Ruslan Bukin
0a33140d06 Change fs image name so it will not be regenerated (we
have both big and little-endian images in tree).
Also we don't known the endianness of the platform the
image was generated on.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-10-25 18:43:36 +00:00
Enji Cooper
669c253531 Integrate contrib/netbsd-tests/fs/tmpfs into the FreeBSD test suite
as tests/sys/fs

These testcases exercise tmpfs support

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2016-10-21 05:24:08 +00:00
Ruslan Bukin
db636d72e3 Add big-endian uzip file system and choose right file system
to proceed tests with.

Reviewed by:	jmmv, ngie
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D8073
2016-10-19 10:01:04 +00:00
Ruslan Bukin
05533a6f19 Fix comment. We have different VM layout on MIPS, so test is skipped.
Requested by:	kib
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-10-18 10:20:58 +00:00
Ruslan Bukin
470d063a87 Skip test on MIPS as we don't have shared page implemented yet.
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-10-10 14:21:40 +00:00
Ruslan Bukin
21a4258d89 Ensure data in pipe is available to read.
Useful for latest (5th, direct mode) test only.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-10-10 13:40:05 +00:00
Konstantin Belousov
ddce1c3ddb Export the mq_getfd_np() symbol from librt.so, which allows to get
file descriptor for the given posix mqueue.  Export the
timer_oshandle_np() symbol to get ktimer id for the given posix timer.

Requested by:	Lewis Donzis <lew@perftech.com>
Reviewed by:	jilles
Discussed with:	kan
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-10-02 17:02:59 +00:00
Ruslan Bukin
090aa8d18b Increase timeouts for geli tests. It takes 2-3x more time to proceed the
tests on MIPS64EB in QEMU.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-09-28 08:11:00 +00:00
Ruslan Bukin
3a970562d7 Check if IPv6 supported before running the test, skip otherwise.
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-09-23 07:51:01 +00:00
Ruslan Bukin
d29c555195 Use bsdlabel as we don't have hardlink disklabel -> bsdlabel
on some platforms.

Reviewed by:	ngie
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D7968
2016-09-22 12:53:11 +00:00
Enji Cooper
bddfc749fa Remove change accidentally committed via r305963 for upcoming tests/sys/fs/...
work

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2016-09-19 09:15:12 +00:00
Enji Cooper
34ccb11831 Reindent TESTSDIR definition for consistency/readability
MFC after:		3 days
Sponsored by:		Dell EMC Isilon
2016-09-19 07:07:55 +00:00
Enji Cooper
a85d4fc7e4 Only chmod $TMPDIR if it's not /tmp
This is a safety belt to ensure that the /tmp sticky bit stuff doesn't
get whacked by accident if someone runs the script outright

MFC after:	1 week
X-MFC with:	r305916
Sponsored by:	Dell EMC Isilon
2016-09-18 05:10:15 +00:00
Enji Cooper
9cd2207265 Make sure $TMPDIR is created with 0755 permissions
This is required to ensure that the temporary script can be executed,
as the default mode is apparently too restrictive

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2016-09-18 05:06:15 +00:00
Jilles Tjoelker
d30e66e53a wait: Do not copyout uninitialized status/rusage/wrusage.
If wait4() or wait6() return 0 because of WNOHANG, the status, rusage and
wrusage information should not be returned.

PR:		212048
Reported by:	Casey Lucas
MFC after:	2 weeks
2016-09-09 21:58:48 +00:00
Enji Cooper
8f2b5a5469 Port contrib/netbsd-tests/kernel/kqueue/... as tests/sys/kqueue/...
proc2_test must be skipped because the invariant tested
(`ke.fflags & NOTE_TRACKERR`) doesn't pass.

MFC after:	58 days
Submitted by:	kib (earlier form)
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D6172
2016-09-06 08:50:21 +00:00
Enji Cooper
cb5fe245b1 Move tests/sys/kqueue/... to tests/sys/kqueue/libkqueue/...
This is being done to clearly distinguish the libkqueue tests
from the (soon to be imported) NetBSD tests.

MFC after:	58 days
Sponsored by:	EMC / Isilon Storage Division
2016-09-06 08:45:29 +00:00
Enji Cooper
3919472360 Skip testcases 9/10 if jail(8) isn't installed
These testcases require jail support

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-09-03 17:59:46 +00:00
Enji Cooper
46f4fe1eb8 Add a missing "Bail out!" if zpool create fails
This will make the exit info more meaningful if/when zpool create fails,
and establishes parity with the other 2 zfs acl testcases (01, 03).

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-09-03 17:31:13 +00:00
Bryan Drewery
1eb4ec9c62 DIRDEPS_BUILD: Add some missing dirctories to the build.
Sponsored by:	EMC / Isilon Storage Division
2016-08-31 19:30:59 +00:00
Julio Merino
f2ee07cfdc Make use of Kyua's work directories.
Change the vnode tests to use the current directory when creating temporary
files, which we can assume is a volatile work directory, and then make the
kqueue_test.sh driver _not_ abandon the directory created by Kyua.

This makes the various kqueue tests independent of each other, and ensures
the temporary file is cleaned up on failure.

Problem spotted by asomers@ when reviewing D4254.
2016-08-25 10:28:47 +00:00
Enji Cooper
ef5c8d5460 Only expect :encode_tv_random_million to fail on 64-bit platforms
It passes on i386

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-16 20:32:08 +00:00
Enji Cooper
708cc995e3 Redirect the output of the testcases to stderr instead of
redirecting it to /dev/null

This will aid in debugging failures

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-13 06:26:33 +00:00
John Baldwin
005ce8e4e6 Fix locking issues with aio_fsync().
- Use correct lock in aio_cancel_sync when dequeueing job.
- Add _locked variants of aio_set/clear_cancel_function and use those
  to avoid lock recursion when adding and removing fsync jobs to the
  per-process sync queue.
- While here, add a basic test for aio_fsync().

PR:		211390
Reported by:	Randy Westlund <rwestlun@gmail.com>
MFC after:	1 week
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7339
2016-07-29 18:26:15 +00:00
John Baldwin
fc4f075a1a Add PTRACE_VFORK to trace vfork events.
First, PL_FLAG_FORKED events now also set a PL_FLAG_VFORKED flag when
the new child was created via vfork() rather than fork().  Second, a
new PL_FLAG_VFORK_DONE event can now be enabled via the PTRACE_VFORK
event mask.  This new stop is reported after the vfork parent resumes
due to the child calling exit or exec.  Debuggers can use this stop to
reinsert breakpoints in the vfork parent process before it resumes.

Reviewed by:	kib
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D7045
2016-07-18 14:53:55 +00:00
John Baldwin
8d570f64aa Add a mask of optional ptrace() events.
ptrace() now stores a mask of optional events in p_ptevents.  Currently
this mask is a single integer, but it can be expanded into an array of
integers in the future.

Two new ptrace requests can be used to manipulate the event mask:
PT_GET_EVENT_MASK fetches the current event mask and PT_SET_EVENT_MASK
sets the current event mask.

The current set of events include:
- PTRACE_EXEC: trace calls to execve().
- PTRACE_SCE: trace system call entries.
- PTRACE_SCX: trace syscam call exits.
- PTRACE_FORK: trace forks and auto-attach to new child processes.
- PTRACE_LWP: trace LWP events.

The S_PT_SCX and S_PT_SCE events in the procfs p_stops flags have
been replaced by PTRACE_SCE and PTRACE_SCX.  PTRACE_FORK replaces
P_FOLLOW_FORK and PTRACE_LWP replaces P2_LWP_EVENTS.

The PT_FOLLOW_FORK and PT_LWP_EVENTS ptrace requests remain for
compatibility but now simply toggle corresponding flags in the
event mask.

While here, document that PT_SYSCALL, PT_TO_SCE, and PT_TO_SCX both
modify the event mask and continue the traced process.

Reviewed by:	kib
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D7044
2016-07-15 15:32:09 +00:00
John Baldwin
3340c45b96 Add a test for user signal delivery.
This test verifies we get the correct ptrace event details when a signal
is posted to a traced process from userland.

Reviewed by:	kib (part of D7044)
2016-07-15 15:28:33 +00:00
George V. Neville-Neil
15dbc1605b On FreeBSD there is a setsockopt option SO_USER_COOKIE which allows
setting a 32 bit value on each socket. This can be used by applications
and DTrace as a rendezvous point so that an applicaton's data can
more easily be captured at run time. Expose the user cookie via
DTrace by updating the translator in tcp.d and add a quick test
program, a TCP server, that sets the cookie on each connection
accepted.

Reviewed by:	hiren
MFC after:	1 week
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D7152
2016-07-08 23:44:09 +00:00
Alan Somers
39ba705baf Fix bitstring allocation on 32-bit platforms
sys/sys/bitstring.h
	Fix a rounding calculation that could undersize a bitstring on
	32-bit platforms.

tests/sys/sys/bitstring_test.h
	Add a test for bitstr_size

PR:		210260
Reported by:	Mark Millard
Reviewed by:	gibbs
Approved by:	re (marius)
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D6848
2016-06-24 21:44:46 +00:00
John Baldwin
b1012d8036 Account for AIO socket operations in thread/process resource usage.
File and disk-backed I/O requests store counts of read/written disk
blocks in each AIO job so that they can be charged to the thread that
completes an AIO request via aio_return() or aio_waitcomplete().  This
change extends AIO jobs to store counts of received/sent messages and
updates socket backends to set these counts accordingly.  Note that
the socket backends are careful to only charge a single messages for
each AIO request even though a single request on a blocking socket might
invoke sosend or soreceive multiple times.  This is to mimic the
resource accounting of synchronous read/write.

Adjust the UNIX socketpair AIO test to verify that the message resource
usage counts update accordingly for aio_read and aio_write.

Approved by:	re (hrs)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D6911
2016-06-21 22:19:06 +00:00
Alan Somers
1fe8c3077f Skip sys/acl tests on systems lacking perl
tests/sys/acl/Makefile
	add perl to the required_programs for all tests in this directory

Reviewed by:	ngie
Approved by:	re (gjb)
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D6870
2016-06-20 23:17:00 +00:00
Alan Somers
30da687794 Always create loopback routes on every fib
Always create loopback routes on every fib, for both IPv4 and IPv6

etc/rc.d/routing
	Create loopback IPv4 and IPv6 routes on every fib at boot. Revert
	278302; now that all FIBs have IPv6 loopback routes, the
	"route add -reject" commands won't fail.

tests/etc/rc.d/routing_test.sh
	Greatly simplify static_ipv6_loopback_route_for_each_fib. It was
	written under the assumption that loopback routes would be added to
	a given fib by the kernel as soon as an interface is configured on
	that fib. However, the logic can be much simpler now that we simply
	add loopback routes to all fibs at boot. This also removes the need
	to run the test as root, removes the restriction that
	net.add_addr_allfibs=0, and removes the need to configure fibs in
	kyua.conf.

	Also, add a test case for IPv4 loopback routes

Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D6582
2016-05-27 22:40:40 +00:00
John Baldwin
778ce4f297 Return the correct status when a partially completed request is cancelled.
After the previous changes to fix requests on blocking sockets to complete
across multiple operations, an edge case exists where a request can be
cancelled after it has partially completed.  POSIX doesn't appear to
dictate exactly how to handle this case, but in general I feel that
aio_cancel() should arrange to cancel any request it can, but that any
partially completed requests should return a partial completion rather
than ECANCELED.  To that end, fix the socket AIO cancellation routine to
return a short read/write if a partially completed request is cancelled
rather than ECANCELED.

Sponsored by:	Chelsio Communications
2016-05-24 21:09:05 +00:00
John Baldwin
1717b68af1 Don't prematurely return short completions on blocking sockets.
Always requeue an AIO job at the head of the socket buffer's queue if
sosend() or soreceive() returns EWOULDBLOCK on a blocking socket.
Previously, requests were only requeued if they returned EWOULDBLOCK
and completed no data.  Now after a partial completion on a blocking
socket the request is queued and the remaining request is retried when
the socket is ready.  This allows writes larger than the currently
available space on a blocking socket to fully complete.  Reads on a
blocking socket that satifsy the low watermark can still return a short
read (same as read()).

In order to track previously completed data, the internal 'status'
field of the AIO job is used to store the amount of previously
computed data.

Non-blocking sockets continue to return short completions for both
reads and writes.

Add a test for a "large" AIO write on a blocking socket that writes
twice the socket buffer size to a UNIX domain socket.

Sponsored by:	Chelsio Communications
2016-05-24 03:13:27 +00:00
Alan Somers
1b82e02f4d Add bit_count to the bitstring(3) api
Add a bit_count function, which efficiently counts the number of bits set in
a bitstring.

sys/sys/bitstring.h
tests/sys/sys/bitstring_test.c
share/man/man3/bitstring.3
	Add bit_alloc

sys/kern/subr_unit.c
	Use bit_count instead of a naive counting loop in check_unrhdr, used
	when INVARIANTS are enabled. The userland test runs about 6x faster
	in a generic build, or 8.5x faster when built for Nehalem, which has
	the POPCNT instruction.

sys/sys/param.h
	Bump __FreeBSD_version due to the addition of bit_alloc

UPDATING
	Add a note about the ABI incompatibility of the bitstring(3)
	changes, as suggested by lidl.

Suggested by:	gibbs
Reviewed by:	gibbs, ngie
MFC after:	9 days
X-MFC-With:	299090, 300538
Relnotes:	yes
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D6255
2016-05-23 20:29:18 +00:00
Alan Somers
fc13dfc8e2 Commit a missing change from 299090
tests/sys/kern/Makefile
	Reenable a disabled compiler warning, the need for which was
	eliminated by r299090.

Reviewed by:	ngie
MFC after:	4 weeks
X-MFC-With:	299090
Sponsored by:	Spectra Logic Corp
2016-05-23 20:19:07 +00:00
Conrad Meyer
079375d12b kern_descrip_test: Fix trivial buffer overrun with readlink(2)
Reported by:	Coverity
CID:		1229965, 1229972
Sponsored by:	EMC / Isilon Storage Division
2016-05-12 03:44:29 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Alan Somers
8907f744ff Improve performance and functionality of the bitstring(3) api
Two new functions are provided, bit_ffs_at() and bit_ffc_at(), which allow
for efficient searching of set or cleared bits starting from any bit offset
within the bit string.

Performance is improved by operating on longs instead of bytes and using
ffsl() for searches within a long. ffsl() is a compiler builtin in both
clang and gcc for most architectures, converting what was a brute force
while loop search into a couple of instructions.

All of the bitstring(3) API continues to be contained in the header file.
Some of the functions are large enough that perhaps they should be uninlined
and moved to a library, but that is beyond the scope of this commit.

sys/sys/bitstring.h:
        Convert the majority of the existing bit string implementation from
        macros to inline functions.

        Properly protect the implementation from inadvertant macro expansion
        when included in a user's program by prefixing all private
        macros/functions and local variables with '_'.

        Add bit_ffs_at() and bit_ffc_at(). Implement bit_ffs() and
        bit_ffc() in terms of their "at" counterparts.

        Provide a kernel implementation of bit_alloc(), making the full API
        usable in the kernel.

        Improve code documenation.

share/man/man3/bitstring.3:
        Add pre-exisiting API bit_ffc() to the synopsis.

        Document new APIs.

        Document the initialization state of the bit strings
        allocated/declared by bit_alloc() and bit_decl().

        Correct documentation for bitstr_size(). The original code comments
        indicate the size is in bytes, not "elements of bitstr_t". The new
        implementation follows this lead. Only hastd assumed "elements"
        rather than bytes and it has been corrected.

etc/mtree/BSD.tests.dist:
tests/sys/Makefile:
tests/sys/sys/Makefile:
tests/sys/sys/bitstring.c:
        Add tests for all existing and new functionality.

include/bitstring.h
	Include all headers needed by sys/bitstring.h

lib/libbluetooth/bluetooth.h:
usr.sbin/bluetooth/hccontrol/le.c:
        Include bitstring.h instead of sys/bitstring.h.

sbin/hastd/activemap.c:
        Correct usage of bitstr_size().

sys/dev/xen/blkback/blkback.c
        Use new bit_alloc.

sys/kern/subr_unit.c:
        Remove hard-coded assumption that sizeof(bitstr_t) is 1.  Get rid of
        unrb.busy, which caches the number of bits set in unrb.map.  When
        INVARIANTS are disabled, nothing needs to know that information.
        callapse_unr can be adapted to use bit_ffs and bit_ffc instead.
        Eliminating unrb.busy saves memory, simplifies the code, and
        provides a slight speedup when INVARIANTS are disabled.

sys/net/flowtable.c:
        Use the new kernel implementation of bit-alloc, instead of hacking
        the old libc-dependent macro.

sys/sys/param.h
        Update __FreeBSD_version to indicate availability of new API

Submitted by:   gibbs, asomers
Reviewed by:    gibbs, ngie
MFC after:      4 weeks
Sponsored by:   Spectra Logic Corp
Differential Revision:  https://reviews.freebsd.org/D6004
2016-05-04 22:34:11 +00:00
Alan Somers
794277da54 Automate the subr_unit test.
Build and install the subr_unit test program originally written by phk, and
run it with the other ATF tests.

tests/sys/kern/Makefile
	* Build and install the subr_unit test as a plain test

sys/kern/subr_unit.c
	* Reduce the default number of repetitions from 100 to 1, and add a
	  command-line parser to override it.
	* Don't be so noisy by default
	* Fix an include problem for the test build

Reviewed by:	ngie
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D6038
2016-04-29 21:11:31 +00:00
Glen Barber
49dae58b28 Fix including Kyuafile in packaged base system.
Fix a related typo while here.

Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses the PR as far
as I can tell in my tests.

PR:		209114
Submitted by:	ngie
Sponsored by:	The FreeBSD Foundation
2016-04-29 05:28:40 +00:00
Enji Cooper
94ebd6f5b4 Fix issues identified by Coverity
- Always munmap memory regions after mmap'ing them.
- Make sure getpagesize() returns a value greater than 0 and use a
  cached value instead of always calling getpagesize(3).
- Remove intermediate variable for assigning from $TMPDIR if set in the
  environment to eliminate warnings about pointer conversions with "/tmp",
  and to mute an invalid buffer overflow concern from Coverity
  (snprintf and tacking on a NUL terminator was alleviating that concern
  before).
- Remove useless self-test of psize before it's initialized.
- Check the return values of getrlimit/setrlimit.

Cosmetic changes:
- Replace a `(void*)0` with NULL.
- Do some minor whitespace clean up.
- Remove an unnecessary cast to mmap.
- Make all munmap calls use ATF_REQUIRE_MSG instead of using the:

  > if (munmap(..) == -1)
  >    atf_tc_fail(..)

  idiom. Employ the new idiom consistently when calling munmap.

CID: 1331351, 1331382-1331386, 1331513, 1331514, 1331565, 1331583, 1331694
Differential Revision: https://reviews.freebsd.org/D6012
MFC after: 2 weeks
Reported by: Coverity
Reviewed by: markj
Sponsored by: EMC / Isilon Storage Division
2016-04-19 23:15:47 +00:00
Enji Cooper
410757a4f4 Fix leaks and test for getpagesize() returning == -1
- close file descriptors after use.
- Always munmap memory regions after mmap'ing them.
- Make sure getpagesize() returns a value greater than 0 and use a
  cached value instead of always calling getpagesize(3).

CID: 1331374-1331377, 1331653-1331662
Differential Revision: https://reviews.freebsd.org/D6011
MFC after: 2 weeks
Reported by: Coverity
Reviewed by: cem
Sponsored by: EMC / Isilon Storage Division
2016-04-19 22:25:14 +00:00
Enji Cooper
9c7defc3bf Minor cosmetic cleanup
- Remove spurious trailing whitespace in licensing header
- Remove unnecessary semi-colon after comment [*]

MFC after: 3 days
Submitted by: pfg [*]
Sponsored by: EMC / Isilon Storage Division
2016-04-18 07:48:27 +00:00
Glen Barber
0edd2576c0 MFH
Sponsored by:	The FreeBSD Foundation
2016-04-16 02:32:12 +00:00
John Baldwin
d1ad1a7394 Add a test for cancelling an active AIO request on a socket.
The older AIO code awakened all pending AIO requests on a socket
when any data arrived.  This could result in AIO daemons blocking on
an empty socket buffer.  These requests could not be cancelled
which led to a deadlock during process exit.  This test reproduces
this case.  The newer AIO code is able to cancel the pending AIO
request correctly.

Reviewed by:	ngie (-ish)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D4363
2016-04-16 00:01:16 +00:00
Enji Cooper
abea2d5a7e Set test_argv to NULL, not 0, if not executing a specific test
MFC after: 1 week
Submitted by: pfg
Sponsored by: EMC / Isilon Storage Division
2016-04-15 02:53:52 +00:00
Glen Barber
d60840138f MFH
Sponsored by:	The FreeBSD Foundation
2016-04-04 23:55:32 +00:00
John Baldwin
c03094a4b6 Correct error messages in recently added test.
The large read test uses an empty file created via mkstemp() rather than
/dev/null as character devices are subject to two different clamping
sysctls.  However, I forgot to update some of the error messages after
changing to mkstemp() that were still referring to /dev/null.
2016-03-31 21:25:40 +00:00
Enji Cooper
da611c05f0 Use a SKIP testplan instead of bailing out if/when the tester isn't
root, or the geom class can't be loaded cleanly [*]

This makes sure that scenarios that are easy to hit aren't counted
as false positives with kyua test

MFC after: 1 week
PR: 208101
Sponsored by: EMC / Isilon Storage Division
2016-03-22 08:12:45 +00:00
John Baldwin
bb430bc740 Fully handle size_t lengths in AIO requests.
First, update the return types of aio_return() and aio_waitcomplete() to
ssize_t.

POSIX requires aio_return() to return a ssize_t so that it can represent
all return values from read() and write().  aio_waitcomplete() should use
ssize_t for the same reason.

aio_return() has used ssize_t in <aio.h> since r31620 but the manpage and
system call entry were not updated.  aio_waitcomplete() has always
returned int.

Note that this does not require new system call stubs as this is
effectively only an API change in how the compiler interprets the return
value.

Second, allow aio_nbytes values up to IOSIZE_MAX instead of just INT_MAX.

aio_read/write should now honor the same length limits as normal read/write.

Third, use longs instead of ints in the aio_return() and aio_waitcomplete()
system call functions so that the 64-bit size_t in the in-kernel aiocb
isn't truncated to 32-bits before being copied out to userland or
being returned.

Finally, a simple test has been added to verify the bounds checking on the
maximum read size from a file.
2016-03-21 21:37:33 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
52259a98ad MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
John Baldwin
f3215338ef Refactor the AIO subsystem to permit file-type-specific handling and
improve cancellation robustness.

Introduce a new file operation, fo_aio_queue, which is responsible for
queueing and completing an asynchronous I/O request for a given file.
The AIO subystem now exports library of routines to manipulate AIO
requests as well as the ability to run a handler function in the
"default" pool of AIO daemons to service a request.

A default implementation for file types which do not include an
fo_aio_queue method queues requests to the "default" pool invoking the
fo_read or fo_write methods as before.

The AIO subsystem permits file types to install a private "cancel"
routine when a request is queued to permit safe dequeueing and cleanup
of cancelled requests.

Sockets now use their own pool of AIO daemons and service per-socket
requests in FIFO order.  Socket requests will not block indefinitely
permitting timely cancellation of all requests.

Due to the now-tight coupling of the AIO subsystem with file types,
the AIO subsystem is now a standard part of all kernels.  The VFS_AIO
kernel option and aio.ko module are gone.

Many file types may block indefinitely in their fo_read or fo_write
callbacks resulting in a hung AIO daemon.  This can result in hung
user processes (when processes attempt to cancel all outstanding
requests during exit) or a hung system.  To protect against this, AIO
requests are only permitted for known "safe" files by default.  AIO
requests for all file types can be enabled by setting the new
vfs.aio.enable_usafe sysctl to a non-zero value.  The AIO tests have
been updated to skip operations on unsafe file types if the sysctl is
zero.

Currently, AIO requests on sockets and raw disks are considered safe
and are enabled by default.  aio_mlock() is also enabled by default.

Reviewed by:	cem, jilles
Discussed with:	kib (earlier version)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D5289
2016-03-01 18:12:14 +00:00
Glen Barber
317cec3c43 MFH
Sponsored by:	The FreeBSD Foundation
2016-02-22 12:28:23 +00:00
Mark Johnston
88c2beac9c Ensure that we test the event condition when a disabled kevent is enabled.
r274560 modified kqueue_register() to only test the event condition if the
corresponding knote is not disabled. However, this check takes place before
the EV_ENABLE flag is used to clear the KN_DISABLED flag on the knote, so
enabling a previously-disabled kevent would not result in a notification for
a triggered event. This change fixes the problem by testing for EV_ENABLED
before possibly checking the event condition.

This change also updates a kqueue regression test to exercise this case.

PR:		206368
Reviewed by:	kib
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D5307
2016-02-19 01:49:33 +00:00
Glen Barber
72c3aa02dc MFH
Sponsored by:	The FreeBSD Foundation
2016-02-18 00:37:58 +00:00
Bryan Drewery
d3157f0915 Test directories can build in parallel fine.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 02:13:59 +00:00
Glen Barber
0fe0fe112f MFH
Sponsored by:	The FreeBSD Foundation
2016-02-15 21:58:52 +00:00
Enji Cooper
8e89d3a42a Similar to r295116, add an additional 1 second sleep after calling ggatel
before calling dd to defeat a race when writing out to the geom_gate(4)
device

MFC after: 1 month
Reported by: Jenkins
Sponsored by: EMC / Isilon Storage Division
2016-02-10 17:57:24 +00:00
Glen Barber
bbb51924bb MFH
Sponsored by:	The FreeBSD Foundation
2016-02-08 12:16:01 +00:00
Enji Cooper
be484a1255 Use basenames for getfacl, setfacl, and zpool to work around the
fact that Jenkins hardcodes image sizes to 2GB with the FreeBSD_HEAD
job

This is to stop the unnecessary failure emails because we've gone
over the 2GB limit

MFC after: 1 week
X-MFC with: r295341
Sponsored by: EMC / Isilon Storage Division
2016-02-06 04:13:20 +00:00