Commit Graph

49 Commits

Author SHA1 Message Date
Alan Somers
88fa3a7649 Revert r337929
FreeBSD's mkstemp sets the temporary file's permissions to 600, and has ever
since mkstemp was added in 1987.  Coverity's warning is still relevant for
portable programs since OpenGroup does not require that behavior, and POSIX
didn't until 2008.  But none of these programs are portable.
2018-08-16 22:04:00 +00:00
Alan Somers
76f2606181 Fix Coverity warnings about mkstemp in tests
umask(2) should always be used prior to mkstemp(3) so the temporary file
won't be created with insecure permissions.

Reported by:	Coverity
CID:		1331605 1347173 1375366 1339800 1331604 1296056 1296060
CID:		1296057 1296062
MFC after:	2 weeks
2018-08-16 21:36:19 +00:00
Alan Somers
52c0983128 lio_listio: return EAGAIN instead of EIO when out of resources
This behavior is already documented by the man page, and suggested by POSIX.

Reviewed by:	jhb
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D15099
2018-04-16 18:12:15 +00:00
John Baldwin
7c13fd10c4 Revert r318180 and re-enable AIO tests on md(4) by default.
The 'physio' fast-path used by AIO requests on md(4) devices, is not
gated on the unsafe_aio knob.  Prior to r327755, some AIO requests could
fail the fast-path and fall back to the slow-path (requests for devices
not supporting unmapped I/O and requests which failed with EFAULT during
the fast-path).  However, those cases now return a suitable error rather
than using the slow-path.

PR:		217261
Reviewed by:	asomers
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D14742
2018-03-19 19:09:15 +00:00
Alan Somers
c189c73f01 aio_kqueue_test: fix the build when DEBUG=1
Also, make the debug output more verbose
2018-03-17 17:52:47 +00:00
Alan Somers
604bbb3f3c aio: add an ATF test case for aio_fsync error conditions 2018-03-17 17:48:21 +00:00
Alan Somers
c4708ce80c Fix build on arm after r325817
Reported by:	rpokala
MFC after:	3 weeks
X-MFC-With:	325817
Sponsored by:	Spectra Logic Corp
2017-11-14 21:11:55 +00:00
Alan Somers
c52ef9bbff AIO tests: increase limits
tests/sys/aio/aio_kqueue_test.c
	Instead of using a hard-coded queue depth, use
	vfs.aio.max_aio_queue_per_proc

tests/sys/aio/lio_kqueue_test.c
	The old, small limit on lio_listio's operation count was lifted by
	change 324941.  Raise the operation count as high as possible without
	exceeding the process's operation limit.

MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
2017-11-14 17:46:37 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Enji Cooper
67b82daf3e Fix cosmetic issue with error message
Add missing space in error message related to PR noted.

MFC after:	2 weeks
PR:		220398
2017-08-02 09:49:41 +00:00
Alan Somers
18ddf67c64 Implement SIGEV_THREAD notifications for lio_listio(2)
Our man pages have always indicated that this was supported, but in fact the
feature was never implemented for lio_listio(2).

Reviewed by:	jhb, kib (earlier version)
MFC after:	20 days
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D11680
2017-07-21 15:09:24 +00:00
Alan Somers
0eafa7078f Remove dead code that was killed by r320975
Reported by:	Coverity
CID:		1377977
MFC after:	15 days
X-MFC-With:	320975
Sponsored by:	Spectra Logic Corp
2017-07-19 15:22:10 +00:00
Alan Somers
7e3db62753 Add regression tests for bugs 220459 and 220398
Bug 220398 - lio_listio(2) never sends asynchronous notification if nent==0
Bug 220459 - lio_listio(2) doesn't support SIGEV_THREAD

PR:		220459
PR:		220398
Reviewed by:	cem, jhb
MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D11470
2017-07-17 18:33:30 +00:00
Alan Somers
a4ea52aab6 Fix the build with GCC after r320975
Reported by:	pfg
MFC after:	20 days
X-MFC-With:	320975
Sponsored by:	Spectra Logic Corp
2017-07-14 21:50:04 +00:00
Alan Somers
314b447f66 Add tests for aio(4) completion notification via signals and threads
Reviewed by:	jhb
MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D11468
2017-07-13 22:53:13 +00:00
Alan Somers
d600f474c6 Use ATF cleanup routines in aio_test.c
Remove aio_test's legacy timeout handling and cleanup routines.  Instead,
use ATF's builtin capabilities.  ATF automatically cleans up newly created
files, too, so we don't have to explicitly unlink them.  The only tests than
need a cleanup routine are the md(4) tests, which must destroy their md
device.

Reviewed by:	jhb
MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D11468
2017-07-13 22:49:55 +00:00
Jilles Tjoelker
9086b30bc4 tests/sys/aio: Add missing mode to open() calls with O_CREAT. 2017-06-04 21:39:37 +00:00
Alan Somers
95c91f3b1d Fix build of AIO tests with -DDEBUG
Also, redefine some constants for clarity.  No functional change.

MFC after:	1 week
2017-05-21 15:37:08 +00:00
Enji Cooper
20a438f31a Mark all md tests as requiring unsafe AIO in order to function
These tests have been flapping (failing<->passing) on Jenkins for months.
It passes reliably for me if unsafe AIO is permitted, but it doesn't
pass on Jenkins reliably if unsafe AIO is disabled (the current default).

Mark the tests as requiring unsafe AIO to mitigate the intermittent
failures when unsafe AIO isn't permitted. If the kernel code is changed
to reliably function with md(4) devices using unsafe AIO, this commit can
be reverted.

MFC after:	2 months
PR:		217261
Reported by:	Jenkins
Sponsored by:	Dell EMC Isilon
2017-05-11 08:06:46 +00:00
Enji Cooper
8e9450bb5a Remove unused constant (PATH_TEMPLATE)
It was made unnecessary in r312913.

MFC after:	3 weeks
MFC with:	r312913
Sponsored by:	Dell EMC Isilon
2017-05-09 20:26:43 +00:00
Enji Cooper
e9b7132b30 style(9): move function definition curly braces to column 0
MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
2017-05-09 19:23:14 +00:00
Enji Cooper
175e856fa2 Print out when unsafe AIO is enabled to debugging purposes
MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
2017-05-09 19:20:02 +00:00
Enji Cooper
5fec77f1d6 Refactor ATF_REQUIRE_UNSAFE_AIO and PLAIN_REQUIRE_UNSAFE_AIO
This is being done to reduce duplication between the two macros.

MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
2017-05-09 19:16:18 +00:00
Enji Cooper
da880f61ca style(9): clean up trailing whitespace
MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
2017-05-09 18:54:35 +00:00
Robert Watson
55e0d88afd Fix build of aio_test on MIPS, where the compiler warns about the local
variable 'err' shadowing the global function err(3).  Which it does.

Sponsored by:	DARPA, AFRL
2017-01-28 12:26:22 +00:00
Alan Somers
740a9bfadd Improve the aio tests
* Add tests for aio_suspend(2).
* Add tests for polled completion notification.
* Test the full matrix of file descriptor types and completion notification
  mechanisms.
* Don't bother with mkstemp, because ATF runs every test in its own temp dir.
* Fix some typos.
* Remove extraneous ATF_REQUIRE_KERNEL_MODULE calls.

Reviewed by:	jhb
MFC after:	4 weeks
Differential Revision:	https://reviews.freebsd.org/D9045
2017-01-28 04:33:51 +00:00
John Baldwin
005ce8e4e6 Fix locking issues with aio_fsync().
- Use correct lock in aio_cancel_sync when dequeueing job.
- Add _locked variants of aio_set/clear_cancel_function and use those
  to avoid lock recursion when adding and removing fsync jobs to the
  per-process sync queue.
- While here, add a basic test for aio_fsync().

PR:		211390
Reported by:	Randy Westlund <rwestlun@gmail.com>
MFC after:	1 week
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7339
2016-07-29 18:26:15 +00:00
John Baldwin
b1012d8036 Account for AIO socket operations in thread/process resource usage.
File and disk-backed I/O requests store counts of read/written disk
blocks in each AIO job so that they can be charged to the thread that
completes an AIO request via aio_return() or aio_waitcomplete().  This
change extends AIO jobs to store counts of received/sent messages and
updates socket backends to set these counts accordingly.  Note that
the socket backends are careful to only charge a single messages for
each AIO request even though a single request on a blocking socket might
invoke sosend or soreceive multiple times.  This is to mimic the
resource accounting of synchronous read/write.

Adjust the UNIX socketpair AIO test to verify that the message resource
usage counts update accordingly for aio_read and aio_write.

Approved by:	re (hrs)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D6911
2016-06-21 22:19:06 +00:00
John Baldwin
778ce4f297 Return the correct status when a partially completed request is cancelled.
After the previous changes to fix requests on blocking sockets to complete
across multiple operations, an edge case exists where a request can be
cancelled after it has partially completed.  POSIX doesn't appear to
dictate exactly how to handle this case, but in general I feel that
aio_cancel() should arrange to cancel any request it can, but that any
partially completed requests should return a partial completion rather
than ECANCELED.  To that end, fix the socket AIO cancellation routine to
return a short read/write if a partially completed request is cancelled
rather than ECANCELED.

Sponsored by:	Chelsio Communications
2016-05-24 21:09:05 +00:00
John Baldwin
1717b68af1 Don't prematurely return short completions on blocking sockets.
Always requeue an AIO job at the head of the socket buffer's queue if
sosend() or soreceive() returns EWOULDBLOCK on a blocking socket.
Previously, requests were only requeued if they returned EWOULDBLOCK
and completed no data.  Now after a partial completion on a blocking
socket the request is queued and the remaining request is retried when
the socket is ready.  This allows writes larger than the currently
available space on a blocking socket to fully complete.  Reads on a
blocking socket that satifsy the low watermark can still return a short
read (same as read()).

In order to track previously completed data, the internal 'status'
field of the AIO job is used to store the amount of previously
computed data.

Non-blocking sockets continue to return short completions for both
reads and writes.

Add a test for a "large" AIO write on a blocking socket that writes
twice the socket buffer size to a UNIX domain socket.

Sponsored by:	Chelsio Communications
2016-05-24 03:13:27 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber
0edd2576c0 MFH
Sponsored by:	The FreeBSD Foundation
2016-04-16 02:32:12 +00:00
John Baldwin
d1ad1a7394 Add a test for cancelling an active AIO request on a socket.
The older AIO code awakened all pending AIO requests on a socket
when any data arrived.  This could result in AIO daemons blocking on
an empty socket buffer.  These requests could not be cancelled
which led to a deadlock during process exit.  This test reproduces
this case.  The newer AIO code is able to cancel the pending AIO
request correctly.

Reviewed by:	ngie (-ish)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D4363
2016-04-16 00:01:16 +00:00
Glen Barber
d60840138f MFH
Sponsored by:	The FreeBSD Foundation
2016-04-04 23:55:32 +00:00
John Baldwin
c03094a4b6 Correct error messages in recently added test.
The large read test uses an empty file created via mkstemp() rather than
/dev/null as character devices are subject to two different clamping
sysctls.  However, I forgot to update some of the error messages after
changing to mkstemp() that were still referring to /dev/null.
2016-03-31 21:25:40 +00:00
John Baldwin
bb430bc740 Fully handle size_t lengths in AIO requests.
First, update the return types of aio_return() and aio_waitcomplete() to
ssize_t.

POSIX requires aio_return() to return a ssize_t so that it can represent
all return values from read() and write().  aio_waitcomplete() should use
ssize_t for the same reason.

aio_return() has used ssize_t in <aio.h> since r31620 but the manpage and
system call entry were not updated.  aio_waitcomplete() has always
returned int.

Note that this does not require new system call stubs as this is
effectively only an API change in how the compiler interprets the return
value.

Second, allow aio_nbytes values up to IOSIZE_MAX instead of just INT_MAX.

aio_read/write should now honor the same length limits as normal read/write.

Third, use longs instead of ints in the aio_return() and aio_waitcomplete()
system call functions so that the 64-bit size_t in the in-kernel aiocb
isn't truncated to 32-bits before being copied out to userland or
being returned.

Finally, a simple test has been added to verify the bounds checking on the
maximum read size from a file.
2016-03-21 21:37:33 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
52259a98ad MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
John Baldwin
f3215338ef Refactor the AIO subsystem to permit file-type-specific handling and
improve cancellation robustness.

Introduce a new file operation, fo_aio_queue, which is responsible for
queueing and completing an asynchronous I/O request for a given file.
The AIO subystem now exports library of routines to manipulate AIO
requests as well as the ability to run a handler function in the
"default" pool of AIO daemons to service a request.

A default implementation for file types which do not include an
fo_aio_queue method queues requests to the "default" pool invoking the
fo_read or fo_write methods as before.

The AIO subsystem permits file types to install a private "cancel"
routine when a request is queued to permit safe dequeueing and cleanup
of cancelled requests.

Sockets now use their own pool of AIO daemons and service per-socket
requests in FIFO order.  Socket requests will not block indefinitely
permitting timely cancellation of all requests.

Due to the now-tight coupling of the AIO subsystem with file types,
the AIO subsystem is now a standard part of all kernels.  The VFS_AIO
kernel option and aio.ko module are gone.

Many file types may block indefinitely in their fo_read or fo_write
callbacks resulting in a hung AIO daemon.  This can result in hung
user processes (when processes attempt to cancel all outstanding
requests during exit) or a hung system.  To protect against this, AIO
requests are only permitted for known "safe" files by default.  AIO
requests for all file types can be enabled by setting the new
vfs.aio.enable_usafe sysctl to a non-zero value.  The AIO tests have
been updated to skip operations on unsafe file types if the sysctl is
zero.

Currently, AIO requests on sockets and raw disks are considered safe
and are enabled by default.  aio_mlock() is also enabled by default.

Reviewed by:	cem, jilles
Discussed with:	kib (earlier version)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D5289
2016-03-01 18:12:14 +00:00
Glen Barber
2aa00a6001 More 'tests' package fixes.
Sponsored by:	The FreeBSD Foundation
2016-02-03 00:34:23 +00:00
Enji Cooper
0b0ac0beb2 - Fix an improperly sized buffer for pathname [1]
- Fix a -Wunused-but-set-variable warning [2]

MFC after: 1 week
Reported by: cppcheck [1], gcc 4.9 [2]
Sponsored by: EMC / Isilon Storage Division
2015-12-28 02:21:36 +00:00
Enji Cooper
fc8e5f5fd4 Fix style(9) a bit and ensure that error from initializing kqueue(2) is
sane

- Push the kqueue(2) initialization down so the errno will correspond with
  the failure instead of potentially being stomped on by functions called
  by `PLAIN_REQUIRE_KERNEL_MODULE`
- Delete trailing whitespace
- Add spaces between braces for conditional and control blocks (for/if)
- Use err/errx instead of perror+printf+exit/printf+exit.
- Remove braces for single-line conditionals

Tested with and without -DDEBUG

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-12-28 02:18:22 +00:00
Enji Cooper
e26e73f37f Place cancel and error under #ifdef DEBUG to mute
-Wunused-but-set-variable warnings reported by gcc 4.9

Remove some trailing whitespace as well

Tested with and without -DDEBUG

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-12-28 02:01:41 +00:00
Bryan Drewery
eacae6dc66 Fix LDADD/DPADD that should be LIBADD.
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:47 +00:00
Enji Cooper
941c3b8bd5 Adjust CFLAGS to find freebsd_test_suite/macros.h
MFC after: 6 days
2015-04-28 10:59:06 +00:00
Enji Cooper
4e5f38d634 - Use ATF_REQUIRE_KERNEL_MDOULE to require aio(4)
- Don't use /tmp as a basis for temporary files as it's outside of the ATF
  sandbox
- Don't override MAX macro in sys/param.h

MFC after: 6 days
2015-04-28 10:53:06 +00:00
Enji Cooper
c5dfa43a6c Use ATF_REQUIRE_KERNEL_MODULE instead of aio_available function
MFC after: 6 days
2015-04-28 10:51:12 +00:00
Enji Cooper
f44eb01339 Integrate tools/regression/aio/aiotest and tools/regression/aio/kqueue into the
FreeBSD test suite as tests/sys/aio

MFC after: 1 week
2015-04-27 08:51:40 +00:00