d/freebsd-nq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2600131bd6
freebsd-nq/share/man/man4/ioat.4

248 lines
7.3 KiB
Groff
Raw Normal View History

Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.\" Copyright (c) 2015 EMC / Isilon Storage Division
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.\" $FreeBSD$
.\"
ioat(4): Add an API to get HW revision Different revisions support different operations. Refer to Intel External Design Specifications to figure out what your hardware supports. Sponsored by: EMC / Isilon Storage Division
2015-12-17 23:21:37 +00:00
.Dd December 17, 2015
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Dt IOAT 4
.Os
.Sh NAME
.Nm I/OAT
.Nd Intel I/O Acceleration Technology
.Sh SYNOPSIS
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
To compile this driver into your kernel,
place the following line in your kernel configuration file:
.Bd -ragged -offset indent
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Cd "device ioat"
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.Ed
.Pp
Or, to load the driver as a module at boot, place the following line in
.Xr loader.conf 5 :
.Bd -literal -offset indent
ioat_load="YES"
.Ed
.Pp
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
In
.Xr loader.conf 5 :
.Pp
.Cd hw.ioat.force_legacy_interrupts=0
.Pp
In
.Xr loader.conf 5 or
.Xr sysctl.conf 5 :
.Pp
.Cd hw.ioat.enable_ioat_test=0
.Cd hw.ioat.debug_level=0
(only critical errors; maximum of 3)
.Pp
.Ft typedef void
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.Fn (*bus_dmaengine_callback_t) "void *arg" "int error"
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Pp
.Ft bus_dmaengine_t
.Fn ioat_get_dmaengine "uint32_t channel_index"
.Ft void
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.Fn ioat_put_dmaengine "bus_dmaengine_t dmaengine"
ioat(4): Add support for interrupt coalescing In I/OAT, this is done through the INTRDELAY register. On supported platforms, this register can coalesce interrupts in a set period to avoid excessive interrupt load for small descriptor workflows. The period is configurable anywhere from 1 microsecond to 16.38 milliseconds, in microsecond granularity. Sponsored by: EMC / Isilon Storage Division
2015-12-14 22:01:52 +00:00
.Ft int
ioat(4): Add an API to get HW revision Different revisions support different operations. Refer to Intel External Design Specifications to figure out what your hardware supports. Sponsored by: EMC / Isilon Storage Division
2015-12-17 23:21:37 +00:00
.Fn ioat_get_hwversion "bus_dmaengine_t dmaengine"
.Ft int
ioat(4): Add support for interrupt coalescing In I/OAT, this is done through the INTRDELAY register. On supported platforms, this register can coalesce interrupts in a set period to avoid excessive interrupt load for small descriptor workflows. The period is configurable anywhere from 1 microsecond to 16.38 milliseconds, in microsecond granularity. Sponsored by: EMC / Isilon Storage Division
2015-12-14 22:01:52 +00:00
.Fn ioat_set_interrupt_coalesce "bus_dmaengine_t dmaengine" "uint16_t delay"
.Ft uint16_t
.Fn ioat_get_max_coalesce_period "bus_dmaengine_t dmaengine"
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.Ft void
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Fn ioat_acquire "bus_dmaengine_t dmaengine"
.Ft void
.Fn ioat_release "bus_dmaengine_t dmaengine"
.Ft struct bus_dmadesc *
.Fo ioat_copy
.Fa "bus_dmaengine_t dmaengine"
.Fa "bus_addr_t dst"
.Fa "bus_addr_t src"
.Fa "bus_size_t len"
.Fa "bus_dmaengine_callback_t callback_fn"
.Fa "void *callback_arg"
.Fa "uint32_t flags"
.Fc
.Ft struct bus_dmadesc *
ioat(4): Add ioat_copy_8k_aligned KPI The hardware supports descriptors with two non-contiguous pages. This allows issuing one descriptor for an 8k copy from/to non-contiguous but otherwise page-aligned memory. Sponsored by: EMC / Isilon Storage Division
2015-12-09 22:45:51 +00:00
.Fo ioat_copy_8k_aligned
.Fa "bus_dmaengine_t dmaengine"
.Fa "bus_addr_t dst1"
.Fa "bus_addr_t dst2"
.Fa "bus_addr_t src1"
.Fa "bus_addr_t src2"
.Fa "bus_dmaengine_callback_t callback_fn"
.Fa "void *callback_arg"
.Fa "uint32_t flags"
.Fc
.Ft struct bus_dmadesc *
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.Fo ioat_blockfill
.Fa "bus_dmaengine_t dmaengine"
.Fa "bus_addr_t dst"
.Fa "uint64_t fillpattern"
.Fa "bus_size_t len"
.Fa "bus_dmaengine_callback_t callback_fn"
.Fa "void *callback_arg"
.Fa "uint32_t flags"
.Fc
.Ft struct bus_dmadesc *
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Fo ioat_null
.Fa "bus_dmaengine_t dmaengine"
.Fa "bus_dmaengine_callback_t callback_fn"
.Fa "void *callback_arg"
.Fa "uint32_t flags"
.Fc
.Sh DESCRIPTION
The
.Nm
driver provides a kernel API to a variety of DMA engines on some Intel server
platforms.
.Pp
There is a number of DMA channels per CPU package.
(Typically 4 or 8.)
Each may be used independently.
Operations on a single channel proceed sequentially.
.Pp
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
Blockfill operations can be used to write a 64-bit pattern to memory.
.Pp
Copy operations can be used to offload memory copies to the DMA engines.
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Pp
Null operations do nothing, but may be used to test the interrupt and callback
mechanism.
.Pp
All operations can optionally trigger an interrupt at completion with the
.Ar DMA_EN_INT
flag.
For example, a user might submit multiple operations to the same channel and
only enable an interrupt and callback for the last operation.
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.Pp
ioat(4): Add support for interrupt coalescing In I/OAT, this is done through the INTRDELAY register. On supported platforms, this register can coalesce interrupts in a set period to avoid excessive interrupt load for small descriptor workflows. The period is configurable anywhere from 1 microsecond to 16.38 milliseconds, in microsecond granularity. Sponsored by: EMC / Isilon Storage Division
2015-12-14 22:01:52 +00:00
The hardware can delay and coalesce interrupts on a given channel for a
configurable period of time, in microseconds.
This may be desired to reduce the processing and interrupt overhead per
descriptor, especially for workflows consisting of many small operations.
Software can control this on a per-channel basis with the
.Fn ioat_set_interrupt_coalesce
API.
The
.Fn ioat_get_max_coalesce_period
API can be used to determine the maximum coalescing period supported by the
hardware, in microseconds.
Current platforms support up to a 16.383 millisecond coalescing period.
Optimal configuration will vary by workflow and desired operation latency.
.Pp
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
All operations are safe to use in a non-blocking context with the
.Ar DMA_NO_WAIT
flag.
(Of course, allocations may fail and operations requested with
.Ar DMA_NO_WAIT
may return NULL.)
.Pp
All operations, as well as
.Fn ioat_get_dmaengine ,
can return NULL in special circumstances.
For example, if the
.Nm
driver is being unloaded, or the administrator has induced a hardware reset, or
a usage error has resulted in a hardware error state that needs to be recovered
from.
.Pp
It is invalid to attempt to submit new DMA operations in a
.Fa bus_dmaengine_callback_t
context.
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Sh USAGE
A typical user will lookup the DMA engine object for a given channel with
.Fn ioat_get_dmaengine .
When the user wants to offload a copy, they will first
.Fn ioat_acquire
the
.Ar bus_dmaengine_t
object for exclusive access to enqueue operations on that channel.
Then, they will submit one or more operations using
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.Fn ioat_blockfill ,
.Fn ioat_copy ,
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
or
.Fn ioat_null .
ioat(4): Add ioat_copy_8k_aligned KPI The hardware supports descriptors with two non-contiguous pages. This allows issuing one descriptor for an 8k copy from/to non-contiguous but otherwise page-aligned memory. Sponsored by: EMC / Isilon Storage Division
2015-12-09 22:45:51 +00:00
After queuing one or more individual DMA operations, they will
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Fn ioat_release
the
.Ar bus_dmaengine_t
to drop their exclusive access to the channel.
The routine they provided for the
.Fa callback_fn
argument will be invoked with the provided
.Fa callback_arg
when the operation is complete.
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
When they are finished with the
.Ar bus_dmaengine_t ,
the user should
.Fn ioat_put_dmaengine .
.Pp
Users MUST NOT block between
.Fn ioat_acquire
and
.Fn ioat_release .
Users SHOULD NOT hold
.Ar bus_dmaengine_t
references for a very long time to enable fault recovery and kernel module
unload.
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.Pp
For an example of usage, see
.Pa src/sys/dev/ioat/ioat_test.c .
.Sh FILES
Clean up issues reported by mandoc -Tlint
2015-12-14 13:01:36 +00:00
.Bl -tag
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
.It Pa /dev/ioat_test
test device for
.Xr ioatcontrol 8
.El
.Sh SEE ALSO
.Xr ioatcontrol 8
.Sh HISTORY
The
.Nm
driver first appeared in
.Fx 11.0 .
.Sh AUTHORS
The
.Nm
driver was developed by
.An \&Jim Harris Aq Mt jimharris@FreeBSD.org ,
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.An \&Carl Delsey Aq Mt carl.r.delsey@intel.com ,
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
and
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
.An \&Conrad Meyer Aq Mt cem@FreeBSD.org .
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
This manual page was written by
.An \&Conrad Meyer Aq Mt cem@FreeBSD.org .
.Sh CAVEATS
Copy operation takes bus addresses as parameters, not virtual addresses.
.Pp
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
Buffers for individual copy operations must be physically contiguous.
.Pp
Copies larger than max transfer size (1MB, but may vary by hardware) are not
supported.
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
Future versions will likely support this by breaking up the transfer into
smaller sizes.
.Sh BUGS
The
.Nm
ioat: Handle channel-fatal HW errors safely Certain invalid operations trigger hardware error conditions. Error conditions that only halt one channel can be detected and recovered by resetting the channel. Error conditions that halt the whole device are generally not recoverable. Add a sysctl to inject channel-fatal HW errors, 'dev.ioat.<N>.force_hw_error=1'. When a halt due to a channel error is detected, ioat(4) blocks new operations from being queued on the channel, completes any outstanding operations with an error status, and resets the channel before allowing new operations to be queued again. Update ioat.4 to document error recovery; document blockfill introduced in r290021 while we are here; document ioat_put_dmaengine() added in r289907; document DMA_NO_WAIT added in r289982. Sponsored by: EMC / Isilon Storage Division
2015-10-31 20:38:06 +00:00
driver only supports blockfill, copy, and null operations at this time.
Import ioat(4) driver I/OAT is also referred to as Crystal Beach DMA and is a Platform Storage Extension (PSE) on some Intel server platforms. This driver currently supports DMA descriptors only and is part of a larger effort to upstream an interconnect between multiple systems using the Non-Transparent Bridge (NTB) PSE. For now, this driver is only built on AMD64 platforms. It may be ported to work on i386 later, if that is desired. The hardware is exclusive to x86. Further documentation on ioat(4), including API documentation and usage, can be found in the new manual page. Bring in a test tool, ioatcontrol(8), in tools/tools/ioat. The test tool is not hooked up to the build and is not intended for end users. Submitted by: jimharris, Carl Delsey <carl.r.delsey@intel.com> Reviewed by: jimharris (reviewed my changes) Approved by: markj (mentor) Relnotes: yes Sponsored by: Intel Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D3456
2015-08-24 19:32:03 +00:00
The driver does not yet support advanced DMA modes, such as XOR, that some
I/OAT devices support.
Reference in New Issue Copy Permalink