Commit Graph

21 Commits

Author SHA1 Message Date
Ruslan Bukin
d987842d1e Enter the network epoch in the xdma interrupt handler if required
by a peripheral device driver.

Sponsored by:	DARPA, AFRL
2020-02-08 23:07:29 +00:00
Ruslan Bukin
a00eed9676 Fix a KASSERT since chained mbufs are accepted by the xdma bounce
buffer loader. m_copydata() will copy entire chain to a single buffer.

Sponsored by:	DARPA, AFRL
2020-02-08 21:02:20 +00:00
Ruslan Bukin
a8692c16c9 Fix xae(4) driver attachement on the Government Furnished Equipment (GFE)
riscv cores.

GFE cores come with standard DTS file that lacks standard 'dmas ='
property, which means xae(4) could not find a DMA controller to use.

The 'dmas' property could not be added to the DTS file because the
ethernet controller and DMA engine parts in Linux are implemented
in a single driver.

Instead of 'dmas' property the standard Xilinx 'axistream-connected'
property is provided, so fallback to use it instead.

Suggested by:	James Clarke <jrtc27@jrtc27.com>
Reviewed by:	James Clarke <jrtc27@jrtc27.com>
Sponsored by:	DARPA, AFRL
2020-02-07 14:36:28 +00:00
Ed Maste
d676fedfbc xdma: avoid NULL deref in error case
Reported by:	Dr Silvio Cesare of InfoSect
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-08-30 00:40:08 +00:00
Ruslan Bukin
e8e90fef03 Remove unused header.
Sponsored by:	DARPA, AFRL
2019-07-22 16:50:37 +00:00
Ruslan Bukin
951e058411 o Add support for BERI IOMMU device
o Add an experimental IOMMU support to xDMA framework

The BERI IOMMU device is the part of CHERI device-model project [1]. It
translates memory addresses for various BERI peripherals modelled in
software. It accepts FreeBSD/mips64 page directories format and manages
BERI TLB.

1. https://github.com/CTSRD-CHERI/device-model

Sponsored by:	DARPA, AFRL
2019-07-22 16:01:20 +00:00
Ruslan Bukin
17f1623d42 Fix transfers that don't use busdma or bounce buffer (e.g. software dmas).
Busdma data loaded by different function (the bug introduced in r349727).

Sponsored by:	DARPA, AFRL
2019-07-04 15:26:02 +00:00
Ruslan Bukin
0c340d7ed9 Negate the logic of XCHAN_CAP_NOBUFS macro and rename it to
XCHAN_CAP_BOUNCE.

The only application that uses bounce buffering for now is the Government
Furnished Equipment (GFE) P2's dma core (AXIDMA) with its own dedicated
cacheless bounce buffer.

Sponsored by:	DARPA, AFRL
2019-07-04 14:04:08 +00:00
Ruslan Bukin
33da49cd2e Don't copy the data from bounce buffer back to the mbuf if channel does
not use bounce buffering.

Sponsored by:	DARPA, AFRL
2019-05-29 16:01:34 +00:00
Conrad Meyer
e2e050c8ef Extract eventfilter declarations to sys/_eventfilter.h
This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions.  The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended).  Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed.  __FreeBSD_version has been bumped.
2019-05-20 00:38:23 +00:00
Conrad Meyer
f4ab98c597 xdma(4): Fix invalid pointer use (breaks arm.SOCFPGA build)
In xdma_handle_mem_node(), vmem_size_t and vmem_addr_t pointers were passed to
an FDT API that emits u_long values to the output parameter pointer.  This
broke on systems with both xdma and 32-bit vmem size/addr types (SOCFPGA).

Reported by:	tinderbox
Sponsored by:	Dell EMC Isilon
2019-05-16 17:34:36 +00:00
Ruslan Bukin
101869a8f0 o Implement a bounce buffer based on device reserved memory.
Grab device reserved physical memory regions from FDT using standard
  "memory-region" property and use vmem(9) to allocate buffers from it.
  The same vmem could be used by DMA engine drivers to allocate memory for
  DMA descriptors.
  This is required for platforms that provide uncached memory region
  reserved exclusively for DMA operations.
o Change sleepable sx(9) lock type to non-sleepable mutex(9) since
  network drivers usually hold mutex during DMA operations. So we don't
  take sleepable lock after non-sleepable.

Tested on U.S. Government Furnished Equipment (GFE) 64-bit RISC-V cores.

Sponsored by:	DARPA, AFRL
2019-05-08 15:22:27 +00:00
Ruslan Bukin
5a51e5e49d o Rewrite softdma_process_tx() of Altera SoftDMA engine driver
so it does not require a bounce buffer. The only need for this was
  to align the buffer address. Implement unaligned access and we don't
  need to copy data twice.
o Remove contigmalloc-based bounce buffer from xDMA code since it is
  not suitable for arbitrary memory provided by platform, which is
  sometimes a dedicated piece of memory that is not managed by OS at all.

Sponsored by:	DARPA, AFRL
2019-04-29 16:27:15 +00:00
Alan Cox
44d0efb215 Eliminate kmem_alloc_contig()'s unused arena parameter.
Reviewed by:	hselasky, kib, markj
Discussed with:	jeff
Differential Revision:	https://reviews.freebsd.org/D16799
2018-08-20 15:57:27 +00:00
Ruslan Bukin
c42f10a257 Add driver for ARM PrimeCell PL330 DMA engine.
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D10201
2018-04-13 12:43:54 +00:00
Ruslan Bukin
b06154ea03 Don't include sys/bus_dma.h directly, use machine/bus.h instead.
Sponsored by:	DARPA, AFRL
2018-04-13 12:41:51 +00:00
Ruslan Bukin
3d5b3b0a44 Tune xDMA interface slightly:
o Move descriptors allocation to DMA engine driver
o Add generic xdma_request() routine
o Add less-generic scatter-gather application based on xdma interface

Typical operation flow in peripheral device driver is:

1. Get xDMA controller
sc->xdma_tx = xdma_ofw_get(sc->dev, "tx");

2. Allocate virtual channel
sc->xchan_tx = xdma_channel_alloc(sc->xdma_tx, caps);

3. Setup transfer status callback
xdma_setup_intr(sc->xchan_tx, my_tx_intr, sc, &sc->ih_tx);

4. Request a transfer(s)
ret = xdma_request(sc->xchan_tx, &req);

5. Free the channel
xdma_channel_free(sc->xdma_tx);

6. Free the controller
xdma_put(sc->xdma_tx);

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14971
2018-04-12 15:36:24 +00:00
Jason A. Harmening
eb36b1d0bc Clean up MD pollution of bus_dma.h:
--Remove special-case handling of sparc64 bus_dmamap* functions.
  Replace with a more generic mechanism that allows MD busdma
  implementations to generate inline mapping functions by
  defining WANT_INLINE_DMAMAP in <machine/bus_dma.h>.  This
  is currently useful for sparc64, x86, and arm64, which all
  implement non-load dmamap operations as simple wrappers
  around map objects which may be bus- or device-specific.

--Remove NULL-checked bus_dmamap macros.  Implement the
  equivalent NULL checks in the inlined x86 implementation.
  For non-x86 platforms, these checks are a minor pessimization
  as those platforms do not currently allow NULL maps.  NULL
  maps were originally allowed on arm64, which appears to have
  been the motivation behind adding arm[64]-specific barriers
  to bus_dma.h, but that support was removed in r299463.

--Simplify the internal interface used by the bus_dmamap_load*
  variants and move it to bus_dma_internal.h

--Fix some drivers that directly include sys/bus_dma.h
  despite the recommendations of bus_dma(9)

Reviewed by:	kib (previous revision), marius
Differential Revision:	https://reviews.freebsd.org/D10729
2017-07-01 05:35:29 +00:00
Ruslan Bukin
436b590a4b Add parenthesis.
Noticed by:	hps
Sponsored by:	DARPA, AFRL
2016-12-30 16:26:54 +00:00
Ruslan Bukin
5b2e3029ff o Fix unlocking.
o Fix types for ds_addr/ds_len.

Sponsored by:	DARPA, AFRL
2016-12-30 16:06:05 +00:00
Ruslan Bukin
85debf7f6e Add xDMA -- the DMA abstraction layer, initial verison.
xDMA is a DMA framework designed to abstract the interaction
between device drivers and DMA engines.

Project wiki: https://wiki.freebsd.org/xdma

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D8807
2016-12-20 18:02:07 +00:00