Commit Graph

80 Commits

Author SHA1 Message Date
Tomasz Zawadzki
047c067c05 so_ver: increase all major versions
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 22.05
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id1a5358882dc496faa5b0b5c9a63b326c378c551
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-31 15:29:56 +00:00
Zhiqiang Liu
3630473789 ioat: fix potential double free problem in ioat_channel_start()
In ioat_channel_start(), if spdk_vtophys(ioat->comp_update) returns
SPDK_VTOPHYS_ERROR, spdk_free is called to free ioat->comp_update,
and ioat->comp_update is not set to NULL. However, the caller
ioat_attach() will also call ioat_channel_destruct() to free
ioat->comp_update, then double-free problem occurs.

Here, we will not free ioat->comp_update in ioat_channel_start(),
ioat_channel_destruct() will do that.

Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Change-Id: I3be19a3feec5c2188051ee67820bfd1e61de9b48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8300
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-06-16 09:00:37 +00:00
Tomasz Zawadzki
e4070ee0e0 so_ver: increase all major versions
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the quarterly releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 21.04
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I44d01154430a074103bd21c7084f44932e81fe72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6167
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-02-05 14:43:47 +00:00
Changpeng Liu
26ae3d66d8 ioat: hide 2MiB boundary memory check in spdk_vtophys()
Change-Id: I8ad61e398f02fc5989450415424f58ea60423f85
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5178
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-11-25 17:15:13 +00:00
Tomasz Zawadzki
4e8e97c886 log: remove internal log.h header
There is nothing left here, so remove it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib947d42bc577dbebb4650b1be885e05a80f8f8cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
2020-10-15 08:23:39 +00:00
Tomasz Zawadzki
2172c432cf log: simplify SPDK_LOG_REGISTER_COMPONENT
This patch removes the string from register component.
Removed are all instances in libs or hardcoded in apps.

Starting with this patch literal passed to register,
serves as name for the flag.

All instances of SPDK_LOG_* were replaced with just *
in lowercase.
No actual name change for flags occur in this patch.

Affected are SPDK_LOG_REGISTER_COMPONENT() and
SPDK_*LOG() macros.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I002b232fde57ecf9c6777726b181fc0341f1bb17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
2020-10-14 08:00:35 +00:00
Shuhei Matsumoto
e2b236ed50 lib/ioat: Update the major SO version for libspdk_ioat.so
ABI dependency check tool complained the following:

00:06:10.012  ************************************
00:06:10.012  START TEST confirm_abi_deps
00:06:10.012  ************************************
00:06:11.559  No corresponding object for libspdk_bdev_uring.so in canonical directory. Skipping.
00:06:12.237  No corresponding object for libspdk_event_sock.so in canonical directory. Skipping.
00:06:12.485  Please update the major SO version for libspdk_ioat.so. A header accesible type has been modified since last release.
00:06:15.388  No corresponding object for libspdk_rdma.so in canonical directory. Skipping.
00:06:15.663  No corresponding object for libspdk_sock_uring.so in canonical directory. Skipping.
00:06:15.954  Processed 55 objects.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2a6cc91ac56ed693ae0676cf64e01ed3ff620ffc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3511
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 07:25:47 +00:00
paul luse
6a9e923da2 lib/ioat: add mew API to get the max # of descriptors supported
Needed by follow-on patches to aupport accel_fw batching in IOAT
module.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie39718f42756c10af131ff7893874d19ea390ecc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3154
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-23 22:26:39 +00:00
Maciej Szwed
eb05cbd677 pollers: Fix pollers to return correct busy status
Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.

Active pollers should return BUSY status only if they
did any meangful work besides checking some conditions
(e.g. processing requests, do some complicated operations).

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id4636a0997489b129cecfe785592cc97b50992ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-07 07:29:31 +00:00
Seth Howell
d18e63206a mk/lib: add a check that major and minor version is set for libs.
Also, while we are here, consolidate setting SO_SUFFIX to one spot.

Previously, it was possible for a library to slip through
without an SO version.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4db5fa5839502d266c6259892e5719b05134518c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-21 09:19:00 +00:00
Seth Howell
12a3278a2e lib/ioat: add a map file.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If7c9db16ab0408280a5ef789556f751f1e0cc8c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2210
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-11 12:29:07 +00:00
Darek Stojaczyk
e03861f138 memory.h: move to public headers
There's no reason not to publish those. Especially if
they're needed in other public headers.

Change-Id: I7dfc6922fcc0dfc46822ad8a16a375f997b98e84
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-19 08:50:45 +00:00
Seth Howell
193927830d make: rev SO versions individually for libraries.
This will allow us to keep track of compatibility issues on a
per-library basis.

Change-Id: Ib0c796adb1efe1570212a503ed660bef6f142b6e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-18 08:02:30 +00:00
paul luse
75c2135de5 module/ioat: fix pci device cleanup on exit
Using bdevperf, for example, with CBDMA on a malloc backend
with zcopy disabled, 16 messages like this would appear on
exist (each for a different BDF):

Device 0000:00:04.0 is still attached at shutdown!

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I05dd84c9f6a7c398ac1ee705b1c292530d283042
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:38:09 +00:00
Ziye Yang
7961de4341 lib/ioat: Fix the error message when enabling ioat during application shutdown.
The printed error message could be the folowing example.

Device 0000:00:04.0 is still attached at shutdown!
Device 0000:00:04.1 is still attached at shutdown!
Device 0000:00:04.2 is still attached at shutdown!
Device 0000:00:04.3 is still attached at shutdown!
Device 0000:00:04.4 is still attached at shutdown!
Device 0000:00:04.5 is still attached at shutdown!
Device 0000:00:04.6 is still attached at shutdown!
Device 0000:00:04.7 is still attached at shutdown!
Device 0000:80:04.0 is still attached at shutdown!
Device 0000:80:04.1 is still attached at shutdown!
Device 0000:80:04.2 is still attached at shutdown!
Device 0000:80:04.3 is still attached at shutdown!
Device 0000:80:04.4 is still attached at shutdown!
Device 0000:80:04.5 is still attached at shutdown!
Device 0000:80:04.6 is still attached at shutdown!
Device 0000:80:04.7 is still attached at shutdown!

I think this issue exists a long time in our CI pool.
And the root cause is: "spdk_pci_device_detach" function
is not called in "ioat_channel_destruct".

This patch should solve this issue.

Change-Id: I47b3f1104a407c5b8390c9389c2f231ae018d16a
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478837
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-30 03:09:01 +00:00
Darek Stojaczyk
e05a487129 ioat: allocate device struct using regular calloc
spdk_dma_malloc() is not required here, as the device
struct is neither DMA-able nor shared between processes.

While here, also allocate various ioat buffers using
spdk_zmalloc() instead of spdk_dma_zmalloc(), as
spdk_dma_*malloc() is about to be deprecated.

Change-Id: I4ffa40a59f48b2854ba3b35a759e4778007b47b7
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450258
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-09 06:30:42 +00:00
Darek Stojaczyk
3aa204fb31 ioat: don't rely on phys_addr retrieved from spdk_malloc()
The phys_addr param in spdk_*malloc() is about to be
deprecated, so use a separate spdk_vtophys() call to
retrieve physical addresses.

This patch also adds error checks against SPDK_VTOPHYS_ERROR.
The error handling paths are already there to account for
spdk_*malloc() failures themselves, so reuse them in case
of vtophys failures.

Change-Id: Ic1d9194a9a2052a46fc6f7990f53aa91b4a8bbc3
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/416994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-20 01:06:09 +00:00
Jim Harris
b01a393382 ioat: reduce completion writebacks
Do not set the completion_update bit except on
the last descriptor built before the dmacount doorbell
is written.  This allows much better batching of
completions (to match batching of the submissions).

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idd0281fb2e9e1ad2eb0f65f097c54fc051dfd935

Reviewed-on: https://review.gerrithub.io/c/444974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-18 07:44:17 +00:00
Jim Harris
c258d73feb ioat: add APIs to only build descriptors
Add spdk_ioat_build_copy and spdk_ioat_build_fill
which mirror the existing spdk_ioat_submit_copy
and spdk_ioat_submit_fill.  These new functions
*only* build the descriptors in the ring - they do
not write the doorbell.  This enables batching
which can significantly improve performance by
reducing the number of MMIO writes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia3539f936924b7f833f4a7b963d06ffefa68379f

Reviewed-on: https://review.gerrithub.io/c/444973
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-18 07:44:17 +00:00
Jim Harris
ec95646a61 ioat: make ioat_flush a public spdk_ioat_flush API
This will enable batching of doorbell writes in
future commits.  For now, just make the API public.

This is the first in a series of patches that
drastically improves performance for high queue
depth CB-DMA workloads.  Some basic tests on
my Xeon E5-v3 platform shows about 4x improvement
for 512B transfers.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia8d28a63f5020ae8644c1efdec7f68740bb6920c

Reviewed-on: https://review.gerrithub.io/c/444972
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-18 07:44:17 +00:00
Darek Stojaczyk
eecc6dc8e6 memory: replace all hardcoded 0x200000 with a define
Makes the code slightly more readable.

Change-Id: Iebf8fb07bceacf433d4bdad0a30419a3faab7eee
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439370
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-13 00:47:26 +00:00
Darek Stojaczyk
3edc4db7a7 env: move VALUE_2MB defines into a new internal header
We use those values in various places in SPDK,
so let's define them in a single place now.

Change-Id: Iad9a5745d69166a6e6032370d4e5a0e604914e45
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-13 00:47:26 +00:00
Darek Stojaczyk
8c4ed83b49 vtophys: add length parameter to the vtophys function
This follows the same trend as the mem_map APIs.

Currently, most of the spdk_vtophys() callers manually
detect physically noncontiguous buffers to split them
into multiple physically contiguous chunks. This patch
is a first step towards encapsulating most of that logic
in a single place - in spdk_vtophys() itself.

This patch doesn't change any functionality on its own,
it only extends the API.

Change-Id: I16faa9dea270c370f2a814cd399f59055b5ccc3d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-09 08:24:59 +00:00
Darek Stojaczyk
41130653b7 pci: remove ioat-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: Ia92bd8f4525712bd27ade16ead67435c5e0fbe7a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Rami Rosen
87eacb6829 ioat: change the return type of ioat_channel_destruct() to be void
The static method ioat_channel_destruct() never fails. This cleanup patch
changes its return type to be void instead of int.

Change-Id: I0db464b6a810eadc3f58550114c4532ca33d03b5
Signed-off-by: Rami Rosen <ramirose@gmail.com>
Reviewed-on: https://review.gerrithub.io/425936
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-18 22:41:00 +00:00
Rami Rosen
412d89cd6f ioat: change return type of spdk_ioat_detach() to be void
The method spdk_ioat_detach() never fails. This patch changes its
return type to be void instead of int.

Change-Id: I6f5bda9ab330c02ae0535bed8b78d416c7b1e264
Signed-off-by: Rami Rosen <ramirose@gmail.com>
Reviewed-on: https://review.gerrithub.io/425252
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-11 21:16:48 +00:00
Dariusz Stojaczyk
9b172afdb6 ioat: clear the internal channel error register on reset
I/OAT device will refuse to resume from the Halted
state until we clear both CHANERR and CHANERR_INT
registers.

We only cleared CHANERR so far. So after the I/OAT
device encountered an error, SPDK would not be able
to initialize it ever again unless it's rebound to
the ioatdma driver.

Unlike CHANERR, CHANERR_INT is only accessible through
PCI config space.

CHANERR_INT is only available on I/OAT devices with
version < 3.3.

Change-Id: Ib369ca76f58c4868fe61ff9532b2e7947e5091d3
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422026
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-13 16:59:18 +00:00
Dariusz Stojaczyk
f75cc493d9 ioat: fix typo on IOAT_DEFAULT_ORDER comment
Change-Id: I9fa3fe1f8252746fbad9462e0bc18c191081d357
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/417808
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-05 16:24:56 +00:00
Daniel Verkamp
104fa6e722 ioat: allow ring to be physically discontiguous
The IOAT "ring" is actually just a circular linked list of descriptors;
the descriptors do not need to be in a single physically contiguous
region.  This can be accomodated by calling spdk_vtophys() on each
descriptor rather than assuming they are all in a single contiguous
region.

Also store the physical address of each descriptor in its associated
software descriptor context to avoid the need to call spdk_vtophys()
during runtime.

Change-Id: Ic8636bbc61deb496a0c6d0ea56b75d298f5f426c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/417782
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-05 08:06:44 +00:00
Daniel Verkamp
5aa5f3dc22 ioat: fix "XFCERCAP" typo in comment
Change-Id: I9e7ff07de8b06afb45b1059d455c8fe854035640
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/417783
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-04 02:36:45 +00:00
Dariusz Stojaczyk
a784e88e35 ioat: do not downcast device pointer to void*
This code was created when spdk_pci_device wasn't
available yet. Now we can use spdk_pci_device* instead
of void* for extra code clarity.

Change-Id: I81d440720b22a484ae3d2739e0510a021bebbafe
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/416995
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-03 16:29:42 +00:00
Dariusz Stojaczyk
962b15d0c4 ioat: split FILL tasks on hugepage boundary
This was obviously omitted in the original implementation.

Change-Id: Ifba049cdb8d1c1f24e30adb542946a2fa83c7464
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/416655
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-02 18:23:15 +00:00
Daniel Verkamp
8a6ba58cb4 scripts/check_format: check for spaces before tabs
Automatically detect more whitespace errors.

All existing cases are fixed; only whitespace change (verify with
diff -w) except for one comment style fixup in include/spdk/nvme.h.

Change-Id: If750e54b9c8e3421ea6feda5f20184a31431631e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402360
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-05 11:09:13 -05:00
Daniel Verkamp
543cb17248 build: remove $(ENV_CFLAGS) where not necessary
Only Makefiles for libraries that directly depend on DPDK (rather than
the SPDK env abstraction) should add $(ENV_CFLAGS).

Change-Id: Ifdf44d3ef8c42bbf7f20edd524b330d00658235b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/392818
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-03 12:05:59 -05:00
Daniel Verkamp
59970a89be astyle: enforce braces around single-line statements
Require braces around all conditional statements, e.g.:

    if (cond)
        statement();

becomes:

    if (cond) {
        statement();
    }

This is the style used through most of the SPDK code, but several
exceptions crept in over time.  Add the astyle option to make sure we
are consistent.

Change-Id: I5a71980147fe8dfb471ff42e8bc06db2124a1a7f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/390914
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-12-11 11:19:32 -05:00
Daniel Verkamp
ea1c15791f log: rename SPDK_TRACE_* to SPDK_LOG_*
Disambiguate the log components from the trace functionality
(include/spdk/trace.h).

The internal spdk_trace_flag structure and related functions will be
renamed in a later commit - this is just a find and replace on
SPDK_TRACE_* and SPDK_LOG_REGISTER_TRACE_FLAG().

Change-Id: I617bd5a9fbe35ffb44ae6020b292658c094a0ad6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/376421
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-07 12:23:19 -05:00
Jim Harris
a0d7056f11 copy: return 0 on success and appropriate errno on failure
This code was still using an old paradigm of returning the
number of bytes associated with a successful submission.
Just return 0 on success instead - if caller needs the
number of bytes for some reason they have the information
to get it.

While here, return an appropriate negated errno where possible -
we especially want ENOMEM returned when an ioat channel runs out
of descriptors.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5858ccd6cff916b6c80fda7d2c9fce96fb39ef89

Reviewed-on: https://review.gerrithub.io/378858
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-09-25 13:56:29 -04:00
John Meneghini
8a44220b1a env: Rename spdk_malloc/zmalloc/realloc/free to spdk_dma_(func)
- rename spdk_malloc_socket to spdk_dma_malloc_socket
  - rename spdk_malloc to spdk_dma_malloc
  - rename spdk_zmalloc to spdk_dma_zmalloc
  - rename spdk_realloc to spdk_dma_realloc
  - rename spdk_free to spdk_dma_free

Change-Id: I52a11b7a4243281f9c56f503e826fd7c4a1fd883
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/362604
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-05-31 15:30:27 -04:00
Ben Walker
b961d9cc12 include: Move the remainder of the code base to stdinc.h
Change-Id: I6a142feeaad3117bd3c75e7c5cb7231a1cfa78ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-08 13:20:36 -07:00
Daniel Verkamp
84d904841f util: move common helper functions to util.h
These were repeated a few different places, so pull them into a common
header file.

Change-Id: Id807fa2cfec0de2e0363aeb081510fb801781985
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-02-10 09:38:52 -07:00
Ziye Yang
4a5a24d537 ioat: cleanup logic in spdk_ioat_submit_copy
Change-Id: I90614b93e1eb6b7b09ca7a21efe5a782f08a9da6
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-01-23 16:09:11 -07:00
Daniel Verkamp
f93fd72680 env: split PCI drivers into individual files
Change the PCI enumeration API to individual functions per device type
so that only the drivers that are actually in use get linked into the
final executable.  All of the common code is still shared internally in
the env_dpdk library.

Change-Id: I2ba83afe59202a510f999a0674e23e60b6581221
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-06 09:30:55 -07:00
Daniel Verkamp
d27b24c94b log: split internal TRACELOG macro into new header
The SPDK_TRACELOG macro depends on a CONFIG setting (DEBUG), so it
should not be part of the public API.

Create a new include/spdk_internal directory for headers that should
only be used within SPDK, not exported for public use.

Change-Id: I39b90ce57da3270e735ba32210c4b3a3468c460b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-16 13:33:51 -07:00
Ben Walker
e0196e8124 ioat: Remove ioat_impl, use swappable env lib instead.
Change-Id: I377e41503b349b34c9ffc911840cb8fb9b13f322
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-11 13:34:08 -07:00
Ben Walker
7c3a6d8c43 env: Move tsc functions to env.
Change-Id: Ieb1caabd76b1af9fdc7a95698ae09c86dce134bd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-06 09:07:41 -07:00
Ben Walker
0dd80395f3 env: Move pci.c from util to env
This allows users to swap their PCI library from
libpciaccess/dpdk to another mechanism using the standard
method for swapping out the env library.

Change-Id: Ib2248f8b43754a540de2ec01897e571f0302b667
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:53:24 -07:00
Ben Walker
b9fbdd189a env: Move malloc/free wrappers into env
Change-Id: Ief591f5e23c4ae06cb77fab647a7afd082450a73
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:51:52 -07:00
Ben Walker
a4747c6048 env: Make the environment library configurable.
This allows users to swap out SPDK's third party
libraries for an implementation based on their own
framework.

Change-Id: Ia0b7384ce5e31acba5ad0d7002dec9e95b759c52
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:51:37 -07:00
Ben Walker
a30b5532cb memory: Rename to env
The new env library will wrap all third-party library
calls and be easily swappable with alternate implementations
at build time. For now, it's just the memory library
renamed.

Change-Id: I26a70933289f8137107208ba75f7520fd7a33da0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 09:49:35 -07:00
Ben Walker
6b1e4e732d Drop libpciaccess and switch to DPDK PCI
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.

Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.

Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-04 15:59:00 -07:00