Commit Graph

14715 Commits

Author SHA1 Message Date
Niklas Cassel
20a01a0495 nvme/fio_plugin: use calloc to allocate zone report buffer
spdk_nvme_zns_report_zones() is implemented using
nvme_allocate_request_user_copy(), which under the hood will do
a spdk_zmalloc() with the SPDK_MALLOC_DMA flag set, and will copy
over the result to our buffer.

Therefore, it is redundant for us to use spdk_dma_zmalloc(),
because it will cause us to allocate twice the amount of memory
from the precious DMA pool than needed.

Changing this zone report buffer allocation to a calloc also
has the benefit of making the code uniform with all other
spdk_nvme_zns_report_zones() call sites in the SPDK codebase.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ia354fa51c66ae07a38a9a57b07c15d145dd609f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7005
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-29 10:17:45 +00:00
Niklas Cassel
7ce5dd62f7 nvme/identify: use calloc to allocate zone report buffer
When support for printing the zone report was added,
the zone report buffer was allocated using calloc().

This was intentionally changed to a malloc in commit
5ef79a17ec ("examples/nvme/identify: add an option to dump
the full zns zone report").

While we shouldn't need to zero the buffer, since the drive
should write  the "Number of Zones" field in zone report header,
and we should never read zone descriptors beyond this value,
the ZNS spec also states that reading beyond the last zone
descriptor has undefined results.

Considering that "Number of Zones" field in the zone report
header will only represent the number of zone descriptors
in the buffer when the partial bit was set to true,
always use calloc(), to avoid the chance that someone might
copy this code and call spdk_nvme_zns_report_zones() with the
partial bit set to false.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ia39c5235aa5c62a4ec42285f53f4bc80f7ec370f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7004
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-29 10:17:45 +00:00
Niklas Cassel
f6bbec8ba9 bdev/hello_world: for zoned bdevs, reset the zone before write
If the bdev is a zoned bdev, reset zone 0 (containing offset 0),
before doing the "Hello world!" write to offset 0.

This is done to ensure that the write pointer for the first zone
is at offset 0.

If we don't do a zone reset before doing the write, the write
would fail if there already were data written to the first zone.
(E.g. if the user ran the hello_bdev example twice.)

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I52b6fc9cf6e86fef9aeb19482eafd1f857ba1478
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6943
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 10:17:45 +00:00
paul luse
346c43edd3 doc: Add accel_fw section and RPCs to jsonrpc docs
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I4f680c5fe72312224ce6722197a70a49886c9db1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4187
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-03-29 07:44:01 +00:00
Sudheer Mogilappagari
4ac5ca6558 posix: add sock to pending_recv list only if not already added
Currently there is possibility of adding a sock to pending_recv
list again if sock->pending_recv is true. Check if flag is false
before adding to the list.

Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: Ie23e1e8dbe1aa5594d9ddea30e7f235e3bf8ddad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6381
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2021-03-29 07:43:08 +00:00
Tomasz Zawadzki
83601f77bc CHANGELOG: add entry on iscsi_parse_params fix
Add CHANGELOG entry for
(f3fd56fc)lib/iscsi: return immediately from iscsi_parse_params if len is 0

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I96500e26dba231b4e5d60ae8f95011b071a78cd0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6778
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:42:52 +00:00
paul luse
30d83eb617 bdev/raid: remove deprecated parameter strip_size
For bdev_raid_create RPC, the deprecated parameter strip_size
was removed.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie9b8d133e29e82e04f92e4a9336b30d8882e37cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6742
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:41:09 +00:00
Maciej Wawryk
0872c7edbd nvmf/perf: Fix type error
TypeError: expected str, bytes or os.PathLike object, not int

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0b395035b43fd4ef5e10ae152889c737ccf66c03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7054
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:40:16 +00:00
Rafal Stefanowski
25bc25b9a0 ocf: update submodule to v20.12.2
Signed-off-by: Rafal Stefanowski <rafal.stefanowski@intel.com>
Change-Id: Iaadd551452f7dff75d507af158e70aa2899ff9d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-03-29 07:40:02 +00:00
Changpeng Liu
04cd292237 nvme/quirk: add MDTS excludes interleaved metadata quirk
The specification for Maximum Data Transfer Size (MDTS) says this field
should include the length of metadata, if metadata is interleaved with the
logical block data.  However, some drives can support MDTS without counting
the interleaved metadata, so for this case SPDK will only use data length
without interleaved metadata length.

Change-Id: I29920a25885699e2689be043b87122367be0e416
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6813
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-03-29 07:38:37 +00:00
Changpeng Liu
075f321be8 nvme/fio_plugin: exclude PRACT&&MD=8&&Extended LBA metadata size
If PRACT is enabled, and metadata size is 8 bytes, for extended
LBA format, the controller will insert/strip the metadata, so
we don't need to pass the metadata buffer, so we should exclude
this metadata buffer from host buffer.

So here add a function to calculate host buffer size.

Change-Id: I42d8d9cbfbf7ba2bc4bf64d65260c6cfe9bd4cb1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6789
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-03-29 07:38:37 +00:00
Mao Jiang
cd468b987d test/nvme_io_msg: cases for io_msg_ctrlr register and unregister
Change-Id: Icd91870712ee977c3f9d803e68e8cb4c4b2ff0e6
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6797
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:35:04 +00:00
Karol Latecki
b7423187a2 autotest.sh: call rdma_device_init only for RDMA tests
Check test flags before calling rdma_device_init if
it's really needed, so that we don't call it when doing
TCP tests.

Change-Id: I2675a008cf84c7c4084cb83b44c279d2a497c3cb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:34:38 +00:00
Krzysztof Karas
8f05269572 spdk_top: fix a bug on minimal size spdk_top window
Fixes a bug that occurs when spdk_top window is just big enough
to print empty line in data area, but unable to fill the line with
actual data, resulting in application crash.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: Id39267a9587b9e0ab74a43f5010927af6faff512
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:33:04 +00:00
Karol Latecki
de21d8f4e4 lib/nbd: fix deadStore issue in nbd_cleanup_io()
Although the value stored to 'rc' is used in the enclosing
expression, the value is never actually read from 'rc'

Fixes #1860

Change-Id: Id1001552e635968e373cad0fd27d7bda41d887cd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7082
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-03-29 07:29:57 +00:00
Ben Walker
b67aa514a4 sock/posix: No longer remove sockets from pending_recv in poll
This seems to be cleaning up the pending_recv list to account for the
missed cases in the previous patches in this series. Now that we're
correctly cleaning up the list, don't do this.

Note that if an EPOLLIN event is received but the application never does
a read/recv, the socket will remain in the pending recv list. The next
poll will get another EPOLLIN event, but the logic already handles that
case.

Additionally, left a TODO for a performance optimization.

Change-Id: I1cdde500a5c76554401a89de766d35b7a486b207
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-29 07:28:35 +00:00
Ben Walker
8ac5f9e924 sock/posix: Fix read logic to avoid double-adding socket to pending_recv
Also write some better comments

Change-Id: I81d59307c5eacc5a71879a83e5040da667909d96
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6745
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-29 07:28:35 +00:00
Ben Walker
01aa5cb385 sock/posix: Clear sock from pending_recv even if user does large read
If there was an EPOLLIN event the socket gets adding to the pending_recv
list. But if the application then does a very large read, it will bypass
the logic that clears the socket from the pending_recv list. Fix this.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ia0ba86012f7c6dfd14eb43ba6eeed94dbbce90ce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-29 07:28:35 +00:00
Ben Walker
8e7d559283 sock/posix: When a socket has no recv_pipe, reading should still clear
from pending_recv list

If the upper layer performs a read/recv, it should still remove the
socket from the pending_recv list.

Change-Id: I32ca8ecccbfe1e53ecc7d6f57343c2727e84b851
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-29 07:28:35 +00:00
Michal Berger
18020ec73b test/setup: Add tests for different block devices' setups
nvme_mount - basic test verifying if setup.sh sees mounted nvme
             devices (either partitions or the entire drive).

lvm_mount - test verifying if setup.sh sees nvme devices when
            they are part of dm setup.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id250b4081b3e04a7f5b26401d2ae1b226fdb2d8f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5311
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-03-29 07:24:14 +00:00
Michal Berger
d69b091262 test/setup: Check if correct driver is in use
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I9023a311442c01ef18d05903bed2eb1513f58a0f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5310
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-03-29 07:24:14 +00:00
Michal Berger
d60bc77142 test/setup: Add tests for hugepages allocation
These tests are meant to "predict" the right number of hugepages
given setup.sh instance will set, depending on its environment
options (HUGENODE, HUGEMEM, etc.).

Change-Id: Iaf441e69d25dd59941891e1d5c3c2b0dbd77644d
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5309
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-03-29 07:24:14 +00:00
Michal Berger
ec94874c1e test/setup: Add initital tests for setup.sh
This is first commit from the series introducing funcrional tests
for setup.sh. Issues like #1689, #1691 showed that major changes
to setup.sh may result in quite unexpected behavior. thus to avoid
any regression, we should make sure  the core of its functionality
is properly tested.

For now, only Linux systems are supported tests-wise.

The tests are explicitly run after all OCSSD are put into PCI_BLOCKED
to make sure they won't be picked up by the tests.

Change-Id: Iac036e693bdbaac476faea3d2128cf4d95434cb2
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5308
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-03-29 07:24:14 +00:00
Karol Latecki
cd94246487 test/common: remove sed commands in vm_setup.sh
Some time ago Qemu submodules links were changed to
use https:// instead of git:// so there's no need
to sed this.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If4b16e7b887af5028d208498da8d81c2fa2101d5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6847
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2021-03-29 07:20:50 +00:00
Karol Latecki
253c1785e5 test/common: install vfio-user-pci capable Qemu fork
Fork needed to run Vhost tests using vfio-user.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iedfa85083e51c08592c9f512c6650415b49deda8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6846
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2021-03-29 07:20:50 +00:00
Karol Latecki
5c3852713c test/vhost: add vfio-user option
Add option to connect Qemu VMs to vfio-user sockets.
This needs additional option to vhost_run to run nvmf_tgt
instead of vhost binary (currently the only way to create
vfio-user sockets) and an additional function to find
connected devices in the VMs.

Change-Id: Iaae234a11f4431a307e1da9c9a2eee123290f5a3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6700
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:20:50 +00:00
Karol Latecki
9f5f5da819 test/vhost: use getopts in vhost_run function
Use getopts instead of using positional args. This
will be needed in future commits when more optional
parameters will be added.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If575fd871f906de5f9d89b2f3972fa54e3208147
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:20:50 +00:00
wanghailiangx
b33da506b5 lib/iscsi: add a comment for iscsi_parse_params()
When param len > 0, param data must not be NULL.
So we add a comment to make it clearer.

Change-Id: I053c3e45ddb8fa23fb67ce899d32dadd8e286946
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-26 21:06:15 +00:00
Karol Latecki
f88416a1de scripts/ceph: load loop module explicitly
Older losetup versions loaded "loop" in background, but
with 2.36.1 this doesn't happen. We know we're going to
use this module so let's load it ourselves.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If3364e92817ae9f09e9897872bd61ca9167184a1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-26 18:46:13 +00:00
Ziye Yang
a620cd198f nvme/tcp: Fix the zero copy enablement issue.
Remove the polling group check. Because at this moment,
the qpair is not added into a polling group. If we do
not remove it, we will never enable zcopy feature for
I/O qpair.

And in sock implementmentation, we already fixed the zero copy
handling if a socket is not in a polling group. See
posix_sock_flush function. So we can fix this issue if we directly
remove this check.

Reported by: Aleksey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I969936c4b6c7f13cbfa4d6eb479010c53f3e384a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7056
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-03-26 08:22:53 +00:00
paul luse
48c057814b test/idxd: fix allocation issue with unit test
Fixes issue #1847

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I34109935b5e8187192239fa34ff43946d0e7632f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-03-26 08:22:41 +00:00
paul luse
26fdb2d8b3 lib/idxd: remove unsed config elements and add a config print
Add print to confirm how groups/queues/engines are being
programmed based on the init RPC used.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic9462c19c6899478a803433f90d9db9249dd5ca1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-03-26 08:22:33 +00:00
root
de8f3a50ee mk: Fix debug build error on ARM ThunderX2 and neoverse N1 platform
When building spdk on ARM platform like thunderx2 with --enable-debug,
there are following error:

/tmp/ccOBb4AF.s: Assembler messages:
/tmp/ccOBb4AF.s:45: Error: selected processor does not support `casp x0,x1,x2,x3,[x4]'
/tmp/ccOBb4AF.s:77: Error: selected processor does not support `caspa x0,x1,x2,x3,[x4]'
/tmp/ccOBb4AF.s:109: Error: selected processor does not support `caspl x0,x1,x2,x3,[x4]'
/tmp/ccOBb4AF.s:141: Error: selected processor does not support `caspal x0,x1,x2,x3,[x4]'

The reason is that DPDK is built with -march=armv8.1-a or -march=armv8.2-a+lse which
have these instructions while SPDK is built with -march=armv8-a+crc which does not support
them. Change spdk build machine to native can fix this.

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I759d4ce2c557ce5ff73a802d7a4b6579c4ba64f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7025
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-03-25 08:29:09 +00:00
ChengqiangMeng
844d94b8a7 unittest/nvme_rdma: increase code coverage for nvme_rdma_req
nvme_rdma_req put/get/init code comprehensive coverage.

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: I5ee5ec43410a8eef7a82f8ee304fd28393a46618
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6925
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-03-24 12:00:23 +00:00
Maciej Szwed
aaac48880d env: Register external memory with DPDK
DPDK has added APIs for registering externally allocated
memory regions. Use them instead of doing our own thing.

We have to postpone spdk_mem_unregister call in
memory_hotplug_cb() because SPDK mutex (g_spdk_mem_map_mutex)
and DPDK mutex (memory_hotplug_lock) may overlap
and cause deadlock when one thread is calling spdk_free()
(locks memory_hotplug_lock first and then tries to lock
g_spdk_mem_map_mutex) and another one is calling
vhost_session_mem_unregister() (locks g_spdk_mem_map_mutex
first and then tries to lock memory_hotplug_lock).


Change-Id: I547b4ffc3987ef088a1b659addba1456ad760a71
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3560
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-24 11:59:59 +00:00
GangCao
09a1028e0f nvme: fix double free of g_hotplug_probe_ctx
Add a check and only free g_hotplug_probe_ctx once.

This is to fix below issue:
https://github.com/spdk/spdk/issues/1849

Change-Id: I7387495890830badd06a7341d9ac4d149ca1c9f4
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7002
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-03-24 11:59:39 +00:00
Michal Berger
0968ee2fcf scripts/check_format: Run shfmt against all bash files
Currently, reference repos which are used to clone the SPDK repo in
the CI pool's systems don't provide master head refs, hence there's
no easy way to diff against it.

Instead, simply run shfmt against all the bash files in the repo.
Impact on the performance should be minimal.

Also, cleanup some related code which is not used anymore.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie0d4e35b8ad214ceed1b4ea29b01a7423b8ff73b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-03-23 11:03:21 +00:00
Ziye Yang
d515bf739d bdev/nvme: Use the accelerated_engine
This patch really uses the accelrated engine.
Currently, we only offload the crc32c caculation,
but it can be extended.

Change-Id: If0e4c6a44b6e1e10e03f7eca355bed418d67326b
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6760
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-03-23 10:09:39 +00:00
Michal Berger
9239ed33f6 mk: Don't use mime-type to determine type of the lib file
Some versions of the file (e.g. 5.33) confuse the sharedlib files with
pie-executable ones when they already have the executable bit set.
Since this particular bit is already set on the .so.* when they are
built, file may report the type as "application/x-pie-executable"
instead. As a result, lib files won't have proper permissions set
when installed.

This is a bit problematic for building the RPMs since when
find-provides kicks in to determine the dependencies, said files
are not picked up at all - package ends up with empty "provides:"
data breaking the dependency lookup during installation of other
packages which may depend on the broken one.

To mitigate, don't use mime-type, simply run file against the lib
in default mode and look for "shared object" string - this should
work for most of the file versions available on the supported
distribution.

Spotted on Fedora31.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If9974f5acbdad6eb1204b64f0ee1ac9b2e20f85b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6703
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-03-23 10:08:51 +00:00
Michal Berger
bfe83fd373 pkgdep: Add rpmdevtools package
Tools from this package are needed to handle the RPMs build.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I516341e0917998eae9c54d3194f1e091f4e5e9df
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6548
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-03-23 10:08:51 +00:00
Michal Berger
9433190fd0 scripts/vagrant: Move the proxy, tsocks and nfs setup to deploy_test_vm()
This is done in order to minimize vagrant work on the CI side. Users
may still provision given VM with full configuration includin these
pieces by running create_vbox.sh with -d argument.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7f494958e06d4e3b047938dda0bd102da824d235
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-23 10:08:37 +00:00
Michal Berger
ca31f3c512 scripts/vagrant: Run vm_setup.sh directly from the synced repo
vagrant's shell provisioner uploads files to /tmp, however,
vm_setup.sh must be run from the repo to find all the source
files. This requires the repo by synced over to the VM as well.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iaf16f066af10739589aa584f57c2b89f5c5202e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-03-23 10:08:37 +00:00
Michal Berger
07596cf183 check_so_deps: Make sure number of processed .so files is updated
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ibb5a7fd6cce53a14ce4acd63085e066db32b059b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6840
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-23 10:08:37 +00:00
Michal Berger
432eb6c21d pkgdep/git: Update location of the irdma driver
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I1074c18998fc758baa2a27449d32e7a3f9538c67
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6896
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-03-23 10:05:00 +00:00
Ziye Yang
1e27c23d09 sock: set the default placement_id to -1
Purpose: The default value of placement_id in spdk_sock
should be set to -1 in spdk_sock_connect_ext. If we still let it to 0 and call
sock_get_placement_id for the spdk socket used in the initiator side,
we will never get the correct placement_id when enable_placement_id configuration
is configured, because we will always get placement_id = 0
instead. And the same comments in spdk_sock_accept function.

And this patch also change the judgement of placement_id in other related places.

PS: Why we need to explictly set default placement_id = -1, because when use
"enable_placement_id=2" for the socket, placment_id=0 is a valid value.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I9fcc3a1c6a5007c22d11da5aeed0022577652a76
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6955
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-03-23 10:04:36 +00:00
Alexey Marchuk
f209637c9d nvmf/rpc: Don't stop listener if it was failed to remove
nvmf_subsystem_remove_listener RPC handler may fail to remove
the listener (e.g. it doesn't exist) but in eror case we
spdk_nvmf_transport_stop_listen_async and send an error
response. In a completion callback passed to
spdk_nvmf_transport_stop_listen_async we try to send a
response again but the response handler had already been
released and we dereference a NULL pointer.

The fix is to skip spdk_nvmf_transport_stop_listen_async
in error case and continue with the subsystem resuming.

Fixes github issue #1821

Change-Id: I8d96b943cca25d9f95d19e8ea600242f019e6b21
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6699
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-23 10:03:38 +00:00
Pawel Piatek
191e6ab8d2 test/nvmf: fix non-root mode
- pass LD_LIBRARY_PATH
- use $SUDO_USER instead $USER
- support --huge-dir param via $SPDK_HUGE_DIR environment variable

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: Ib42fe8a10b2c5b7d8a888605da04fee98a600515
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6171
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-03-23 10:03:11 +00:00
Liu Xiaodong
b8db1af4e2 thread: criticalmsg also indicates valid event
Change-Id: I32296be76dfdde725249f43a96f5031c9c2613ae
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-22 09:50:27 +00:00
Liu Xiaodong
1bd39bb5ee thread: remove stat update in intr msg process
Updating thread stat directly in spdk_thread_poll()
will cover the time spend in msg process in interrupt
mode.

Change-Id: I9b71790281f10fb784ef4fd4059c41438bbaabac
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6722
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2021-03-22 09:50:27 +00:00
Liu Xiaodong
3690dbbd2e thread: remove redundant set_thread in intr
thread_interrupt_msg_process is registered to thread's
fd_group, so it will be called inside spdk_thread_poll.
Since spdk_thread_poll will set/restore tls_thread,
there is no need to set or restore it again here.

Change-Id: Ida10c736ef904ff975eeb42fd0cccad9fd8317cf
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6720
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-22 09:50:27 +00:00