532a8c64da
Some classes of IOAT hardware prefetch reads. DMA operations that depend on the result of prior DMA operations must use the DMA_FENCE flag to prevent stale reads. (E.g., I've hit this personally on Broadwell-EP. The Broadwell-DE has a different IOAT unit that is documented to not pipeline DMA operations.) Sponsored by: EMC / Isilon Storage Division
271 lines
8.1 KiB
Groff
271 lines
8.1 KiB
Groff
.\" Copyright (c) 2015 EMC / Isilon Storage Division
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
.\" modification, are permitted provided that the following conditions
|
|
.\" are met:
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
|
|
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
|
|
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
.\" SUCH DAMAGE.
|
|
.\"
|
|
.\" $FreeBSD$
|
|
.\"
|
|
.Dd January 14, 2016
|
|
.Dt IOAT 4
|
|
.Os
|
|
.Sh NAME
|
|
.Nm I/OAT
|
|
.Nd Intel I/O Acceleration Technology
|
|
.Sh SYNOPSIS
|
|
To compile this driver into your kernel,
|
|
place the following line in your kernel configuration file:
|
|
.Bd -ragged -offset indent
|
|
.Cd "device ioat"
|
|
.Ed
|
|
.Pp
|
|
Or, to load the driver as a module at boot, place the following line in
|
|
.Xr loader.conf 5 :
|
|
.Bd -literal -offset indent
|
|
ioat_load="YES"
|
|
.Ed
|
|
.Pp
|
|
In
|
|
.Xr loader.conf 5 :
|
|
.Pp
|
|
.Cd hw.ioat.force_legacy_interrupts=0
|
|
.Pp
|
|
In
|
|
.Xr loader.conf 5 or
|
|
.Xr sysctl.conf 5 :
|
|
.Pp
|
|
.Cd hw.ioat.enable_ioat_test=0
|
|
.Cd hw.ioat.debug_level=0
|
|
(only critical errors; maximum of 3)
|
|
.Pp
|
|
.Ft typedef void
|
|
.Fn (*bus_dmaengine_callback_t) "void *arg" "int error"
|
|
.Pp
|
|
.Ft bus_dmaengine_t
|
|
.Fn ioat_get_dmaengine "uint32_t channel_index"
|
|
.Ft void
|
|
.Fn ioat_put_dmaengine "bus_dmaengine_t dmaengine"
|
|
.Ft int
|
|
.Fn ioat_get_hwversion "bus_dmaengine_t dmaengine"
|
|
.Ft size_t
|
|
.Fn ioat_get_max_io_size "bus_dmaengine_t dmaengine"
|
|
.Ft int
|
|
.Fn ioat_set_interrupt_coalesce "bus_dmaengine_t dmaengine" "uint16_t delay"
|
|
.Ft uint16_t
|
|
.Fn ioat_get_max_coalesce_period "bus_dmaengine_t dmaengine"
|
|
.Ft void
|
|
.Fn ioat_acquire "bus_dmaengine_t dmaengine"
|
|
.Ft int
|
|
.Fn ioat_acquire_reserve "bus_dmaengine_t dmaengine" "uint32_t n" "int mflags"
|
|
.Ft void
|
|
.Fn ioat_release "bus_dmaengine_t dmaengine"
|
|
.Ft struct bus_dmadesc *
|
|
.Fo ioat_copy
|
|
.Fa "bus_dmaengine_t dmaengine"
|
|
.Fa "bus_addr_t dst"
|
|
.Fa "bus_addr_t src"
|
|
.Fa "bus_size_t len"
|
|
.Fa "bus_dmaengine_callback_t callback_fn"
|
|
.Fa "void *callback_arg"
|
|
.Fa "uint32_t flags"
|
|
.Fc
|
|
.Ft struct bus_dmadesc *
|
|
.Fo ioat_copy_8k_aligned
|
|
.Fa "bus_dmaengine_t dmaengine"
|
|
.Fa "bus_addr_t dst1"
|
|
.Fa "bus_addr_t dst2"
|
|
.Fa "bus_addr_t src1"
|
|
.Fa "bus_addr_t src2"
|
|
.Fa "bus_dmaengine_callback_t callback_fn"
|
|
.Fa "void *callback_arg"
|
|
.Fa "uint32_t flags"
|
|
.Fc
|
|
.Ft struct bus_dmadesc *
|
|
.Fo ioat_blockfill
|
|
.Fa "bus_dmaengine_t dmaengine"
|
|
.Fa "bus_addr_t dst"
|
|
.Fa "uint64_t fillpattern"
|
|
.Fa "bus_size_t len"
|
|
.Fa "bus_dmaengine_callback_t callback_fn"
|
|
.Fa "void *callback_arg"
|
|
.Fa "uint32_t flags"
|
|
.Fc
|
|
.Ft struct bus_dmadesc *
|
|
.Fo ioat_null
|
|
.Fa "bus_dmaengine_t dmaengine"
|
|
.Fa "bus_dmaengine_callback_t callback_fn"
|
|
.Fa "void *callback_arg"
|
|
.Fa "uint32_t flags"
|
|
.Fc
|
|
.Sh DESCRIPTION
|
|
The
|
|
.Nm
|
|
driver provides a kernel API to a variety of DMA engines on some Intel server
|
|
platforms.
|
|
.Pp
|
|
There is a number of DMA channels per CPU package.
|
|
(Typically 4 or 8.)
|
|
Each may be used independently.
|
|
Operations on a single channel proceed sequentially.
|
|
.Pp
|
|
Blockfill operations can be used to write a 64-bit pattern to memory.
|
|
.Pp
|
|
Copy operations can be used to offload memory copies to the DMA engines.
|
|
.Pp
|
|
Null operations do nothing, but may be used to test the interrupt and callback
|
|
mechanism.
|
|
.Pp
|
|
All operations can optionally trigger an interrupt at completion with the
|
|
.Ar DMA_INT_EN
|
|
flag.
|
|
For example, a user might submit multiple operations to the same channel and
|
|
only enable an interrupt and callback for the last operation.
|
|
.Pp
|
|
The hardware can delay and coalesce interrupts on a given channel for a
|
|
configurable period of time, in microseconds.
|
|
This may be desired to reduce the processing and interrupt overhead per
|
|
descriptor, especially for workflows consisting of many small operations.
|
|
Software can control this on a per-channel basis with the
|
|
.Fn ioat_set_interrupt_coalesce
|
|
API.
|
|
The
|
|
.Fn ioat_get_max_coalesce_period
|
|
API can be used to determine the maximum coalescing period supported by the
|
|
hardware, in microseconds.
|
|
Current platforms support up to a 16.383 millisecond coalescing period.
|
|
Optimal configuration will vary by workflow and desired operation latency.
|
|
.Pp
|
|
All operations are safe to use in a non-blocking context with the
|
|
.Ar DMA_NO_WAIT
|
|
flag.
|
|
(Of course, allocations may fail and operations requested with
|
|
.Ar DMA_NO_WAIT
|
|
may return NULL.)
|
|
.Pp
|
|
Operations that depend on the result of prior operations should use
|
|
.Ar DMA_FENCE .
|
|
For example, such a scenario can happen when two related DMA operations are
|
|
queued.
|
|
First, a DMA copy to one location (A), followed directly by a DMA copy
|
|
from A to B.
|
|
In this scenario, some classes of I/OAT hardware may prefetch A for the second
|
|
operation before it is written by the first operation.
|
|
To avoid reading a stale value in sequences of dependent operations, use
|
|
.Ar DMA_FENCE .
|
|
.Pp
|
|
All operations, as well as
|
|
.Fn ioat_get_dmaengine ,
|
|
can return NULL in special circumstances.
|
|
For example, if the
|
|
.Nm
|
|
driver is being unloaded, or the administrator has induced a hardware reset, or
|
|
a usage error has resulted in a hardware error state that needs to be recovered
|
|
from.
|
|
.Pp
|
|
It is invalid to attempt to submit new DMA operations in a
|
|
.Fa bus_dmaengine_callback_t
|
|
context.
|
|
.Sh USAGE
|
|
A typical user will lookup the DMA engine object for a given channel with
|
|
.Fn ioat_get_dmaengine .
|
|
When the user wants to offload a copy, they will first
|
|
.Fn ioat_acquire
|
|
the
|
|
.Ar bus_dmaengine_t
|
|
object for exclusive access to enqueue operations on that channel.
|
|
Optionally, the user can reserve space by using
|
|
.Fn ioat_acquire_reserve
|
|
instead.
|
|
If
|
|
.Fn ioat_acquire_reserve
|
|
succeeds, there is guaranteed to be room for
|
|
.Fa N
|
|
new operations in the internal ring buffer.
|
|
Then, they will submit one or more operations using
|
|
.Fn ioat_blockfill ,
|
|
.Fn ioat_copy ,
|
|
or
|
|
.Fn ioat_null .
|
|
After queuing one or more individual DMA operations, they will
|
|
.Fn ioat_release
|
|
the
|
|
.Ar bus_dmaengine_t
|
|
to drop their exclusive access to the channel.
|
|
The routine they provided for the
|
|
.Fa callback_fn
|
|
argument will be invoked with the provided
|
|
.Fa callback_arg
|
|
when the operation is complete.
|
|
When they are finished with the
|
|
.Ar bus_dmaengine_t ,
|
|
the user should
|
|
.Fn ioat_put_dmaengine .
|
|
.Pp
|
|
Users MUST NOT block between
|
|
.Fn ioat_acquire
|
|
and
|
|
.Fn ioat_release .
|
|
Users SHOULD NOT hold
|
|
.Ar bus_dmaengine_t
|
|
references for a very long time to enable fault recovery and kernel module
|
|
unload.
|
|
.Pp
|
|
For an example of usage, see
|
|
.Pa src/sys/dev/ioat/ioat_test.c .
|
|
.Sh FILES
|
|
.Bl -tag
|
|
.It Pa /dev/ioat_test
|
|
test device for
|
|
.Xr ioatcontrol 8
|
|
.El
|
|
.Sh SEE ALSO
|
|
.Xr ioatcontrol 8
|
|
.Sh HISTORY
|
|
The
|
|
.Nm
|
|
driver first appeared in
|
|
.Fx 11.0 .
|
|
.Sh AUTHORS
|
|
The
|
|
.Nm
|
|
driver was developed by
|
|
.An \&Jim Harris Aq Mt jimharris@FreeBSD.org ,
|
|
.An \&Carl Delsey Aq Mt carl.r.delsey@intel.com ,
|
|
and
|
|
.An \&Conrad Meyer Aq Mt cem@FreeBSD.org .
|
|
This manual page was written by
|
|
.An \&Conrad Meyer Aq Mt cem@FreeBSD.org .
|
|
.Sh CAVEATS
|
|
Copy operation takes bus addresses as parameters, not virtual addresses.
|
|
.Pp
|
|
Buffers for individual copy operations must be physically contiguous.
|
|
.Pp
|
|
Copies larger than max transfer size (1MB, but may vary by hardware) are not
|
|
supported.
|
|
Future versions will likely support this by breaking up the transfer into
|
|
smaller sizes.
|
|
.Sh BUGS
|
|
The
|
|
.Nm
|
|
driver only supports blockfill, copy, and null operations at this time.
|
|
The driver does not yet support advanced DMA modes, such as XOR, that some
|
|
I/OAT devices support.
|