Commit Graph

15058 Commits

Author SHA1 Message Date
Changpeng Liu
f6d5b9cd92 bdev: introduce unmap attributes to bdev layer
The backend device module can report such capabilities
to the bdev layer, and we can split unmap request into
multiple children requests in the bdev layer.

Change-Id: I81daf7e58b797a2673ef102f2a037de20771092e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7515
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-07 11:30:55 +00:00
Changpeng Liu
518d28dba9 scsi/bdev: free bdev_io when there are split requests
Only the last bdev_io can be freed without this fix.

Change-Id: I0d05b5d89e38ef60872ebc0f23aaed0c622593c4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7571
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-05-07 11:30:55 +00:00
Ziye Yang
20698a4a8d idxd: Split the idxd library into common and user space part.
Purpose: This patch is used to prepare to add the kernel
idxd support later.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: If89665f95d622c7342ab75050664158ec6fc615a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7330
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-05-07 08:56:20 +00:00
Swapnil Ingle
6cebe9d06b nvmf: Support physical block size if exposed by bdev
(Note: this patch was previously applied as b32cfc46 and then reverted
as 63642bef.)

Today the in-guest nvme device shows physical_block_size=512 even though
the backend iSCSI bdev supports physical_block_size=4K

iSCSI targets exposes physical block size using
logical_block_per_physical_block_exponent in READ_CAPACITY_16

NPWG is one of the way to let Linux nvme driver set
physical_block_size of the nvme block device.

This patch adds spdk_bdev.phys_blocklen which is updated if the iSCSI
backend exposes physical_block_size.
Later phys_blocklen is used in nvmf to set NPWG and NAWUPF to report
back during NS identity.
Linux driver uses min(nawupf, npwg) to set physical_block_size.

Similarly in scsi_bdev fill lbppbe in READ_CAP16 response
based on spdk_bdev.phys_blocklen.

Fixes #1884

Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I0b6c81f1937e346d448f49c927eda8c79d2d75c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7739
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-06 07:14:40 +00:00
Jim Harris
fa79643725 nvmf: remove struct spdk_bdev definition dependencies
1) use spdk_bdev_get_name() accessor
2) use __SPDK_BDEV_MODULE_ONLY #define

The latter allows nvmf to just get the spdk_bdev_module
definitions and APIs that it needs for claiming bdevs
for purposes of avoiding the same namespace used in
different subsystems.

This also ensures that future changes to structures
like spdk_bdev and spdk_bdev_io will not cause
lib/nvmf so version changes.

Note: we include bdev_module.h explicitly in the
nvmf/subsystem unit tests now, before including
subsystem.c, because the unit tests do depend on
knowing the internal structure of spdk_bdev.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2f499a741d19f4749eadb402641f28137245fd23
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7738
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-06 07:14:40 +00:00
Jim Harris
2b9ec05ba7 bdev: add __SPDK_BDEV_MODULE_ONLY to bdev_module.h
We have some cases where libraries need the
spdk_bdev_module structure definition and a couple
of related APIs, but not everything else (i.e.
spdk_bdev, spdk_bdev_io), for purposes of avoiding
abidiff errors.

For example, nvmf creates a dummy spdk_bdev_module,
and then uses it with the spdk_bdev_module_claim_bdev
API to ensure multiple subsystems cannot add the same
bdev as a namespace.

But when nvmf includes bdev_module.h, it pulls in the
spdk_bdev structure definition as well.  This means
when the spdk_bdev structure is modified, it requires
a major version bump since abidiff detects the
difference in the debug info.

Alternatives considered:

* We could add a specific suppression into our abidiff
script for nvmf and struct spdk_bdev, but it would be
a risk (albeit very very small one) that we could
add a real dependency on struct spdk_bdev in the future,
and the suppression would hide the difference.

* We could also break out bdev_module.h into multiple
header files, but the ways of doing that either result
in odd file naming, or modifying every bdev module to
include a new header.

* We could add a public bdev API to expose what the
bdev library needs, but that seemed even more intrusive
than this change. nvmf is kind of abusing the bdev_module
API here, and I'd prefer to not promote that kind of
usage by adding something to the bdev API.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie8fdef8ea294d005b9ae7934dde49c62748420d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7737
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-06 07:14:40 +00:00
Jim Harris
3e54a5bb43 bdev: rearrange bdev_module.h
Move some of the bdev_module APIs immediately after
the bdev_module structure definition.  This prepares
for an upcoming patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I78d534ba047048ec27d8d3159666584a52211de3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7736
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-06 07:14:40 +00:00
Mao Jiang
bc7da85481 test/nvme_qpair: cases for getting sgl print information
Change-Id: Ic3cff09a102c4224eda3c4f6e2da0c17e50db303
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7552
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-06 07:13:09 +00:00
Mao Jiang
912dc67755 test/nvme_ctrlr_cmd: cases for command security receiving and sending
Change-Id: I07c7863ddadf333f15cfb2ebb64e7b3dcf5554d1
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7364
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-06 07:12:42 +00:00
Changpeng Liu
ed2942461d nvmf/vfio-user: ignore unmapped DMA regions
libvfio-user DMA APIs report all regions notified by the client, including those
that don't have a corresponding shared mapping.  There are several of these for
a typical VM, so just ignore this case.

Signed-off-by: John Levon <john.levon@nutanix.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I37b06f4bc6d1818a03c8742616ed142f575d3f0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7532
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-05 14:25:32 +00:00
Mao Jiang
517164a5fd test/nvme_ctrlr: cases for setting arbitration feature and ctrlr state
Change-Id: I7467049e5f5f256704f5eb8fd51a0c9c1c44a581
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7196
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-05 13:33:51 +00:00
Maciej Wawryk
33227bbcc0 test/nvmf: add remove non-existent subsystem listener test
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I3504e874d8ec1c8bacbd394e3d4d208dc155f45c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7038
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-05 13:33:11 +00:00
sunshihao520
2fc1ccae77 lib/nvme: add value check to fix value inversion
When payload_size is 0, we may get wrong cdw10 because of the calculate: 0 - 1,
add value check to fix value inversion bug.

Signed-off-by: sunshihao <sunshihao@huawei.com>
Change-Id: I3bcd38ba981c854ff917282341d32aac47d22b76
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7443
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-05 13:32:30 +00:00
Jim Harris
32999ab917 jsonrpc: use SOCK_NONBLOCK in socket() calls
This allows us to remove separate fcntl() calls to
set O_NONBLOCK.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1a590cfb3b65b3174bb5ef33e060cdc9bb7ac86c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7598
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-05-05 13:31:10 +00:00
Jim Harris
c86075607b nbd: use SOCK_NONBLOCK with socketpair() call
This simplifies the code a bit, removing the need
for the separate fcntl() calls.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4fef8f01a055d1471df87bd979c21d6198e9868a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7596
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-05 13:31:10 +00:00
Ziye Yang
988af676da nvmf/tcp: Remove the unnecessary qpair setting.
It is already set by nvmf_tcp_req_pdu_init
when we get the pdu. So we do not set it again.

Change-Id: I034bbc46e600afd802457c0b152e303f16bafba3
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7714
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-05 13:29:37 +00:00
Michal Berger
d97b106f81 pkgdep: Update liburing version to 2.0 release
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I239c6eb2e9919a4605a14d39f4462b3aa5f54883
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7496
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2021-05-05 13:27:16 +00:00
yidong0635
40549bb48e log_rpc: Unify codes in end path.
Unify the end path, and remove duplicate codes.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ie36cb2182b2b0b23f36670780b6e7af3652dc94b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7520
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-05 13:26:06 +00:00
John Kariuki
b69796f777 scripts/perf: fix json param names
Fixed parameter names in the sample json config file.

Change-Id: I4e221b17c533c0957a325e24043a627b5acb6220
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7688
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-05 13:25:48 +00:00
Ben Walker
a922a2e24a lvol: Don't include bdev_module.h in the public header
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I8590bdedd8b1c3d0e417694f07f40deb3297ef24
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7728
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-04 21:07:42 +00:00
Ben Walker
63642befd3 Revert "nvmf: Support physical block size if exposed by bdev"
This reverts commit b32cfc467b.

This commit fails the ABI checks and only got through because the checks
were disabled until 21.04 hit.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Id26b8f8ba551193d99b1ccbd31b35378b4095a20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7731
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-04 00:48:34 +00:00
Swapnil Ingle
b32cfc467b nvmf: Support physical block size if exposed by bdev
Today the in-guest nvme device shows physical_block_size=512 even though
the backend iSCSI bdev supports physical_block_size=4K

iSCSI targets exposes physical block size using
logical_block_per_physical_block_exponent in READ_CAPACITY_16

NPWG is one of the way to let Linux nvme driver set
physical_block_size of the nvme block device.

This patch adds spdk_bdev.phys_blocklen which is updated if the iSCSI
backend exposes physical_block_size.
Later phys_blocklen is used in nvmf to set NPWG and NAWUPF to report
back during NS identity.
Linux driver uses min(nawupf, npwg) to set physical_block_size.

Similarly in scsi_bdev fill lbppbe in READ_CAP16 response
based on spdk_bdev.phys_blocklen.

Fixes #1884

Signed-off-by: Swapnil Ingle <swapnil.ingle@nutanix.com>
Change-Id: I0b6c81f1937e346d448f49c927eda8c79d2d75cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7310
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-03 18:59:34 +00:00
Mao Jiang
eb76ccb04c test/nvme_tcp: cases for connecting sock and sending icreq
Change-Id: Iece99816ad67ac585b6285cbd43e415425b75239
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6821
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-30 09:49:44 +00:00
jwyka
cddfa0241a lib/nvme: allign cuse ctx data to 4k
We have seen that dptr was not alligned to 4k using cuse. Added allignment of data in cuse ctx to 4k same as it is done in nvme_allocate_request_user_copy
Signed-off-by: jwyka <jakub.wyka@intel.com>
Change-Id: Ic5c2482eae20d64ba467016eb61f5255467f70a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
2021-04-30 07:36:12 +00:00
Tomasz Zawadzki
469d8f32d0 so_ver: reenable test preventing increasing SO version
This check was disabled after 21.01 LTS to allow for
clear break of SO versions. Now that SPDK 21.04 is being
released, this test should be reenabled.

See following patch for details:
(e4070ee0)so_ver: increase all major versions

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8a299a340338e7a2ab3439b81153818778b4c93a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7616
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-04-30 07:06:12 +00:00
Liu Xiaodong
ac0c36d72a test: add functional test for reactor_set_intr
test script 'test/interrupt/reactor_set_intr.sh' will
do various reactor set intr operations on interrupt_tgt
without spdk_thread and with spdk_thread.

Change-Id: Ie5af1dc68b0272c34a91e8a66b78088c3794907c
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7348
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-29 07:10:26 +00:00
Tomasz Zawadzki
50935184c8 lib/blob: force execution of queued persists
When performing snapshot creation the I/O is frozen
during the process. The blob persists for extent page
allocation is delayed until snapshot creation is finished.

This results in multiple blob persists executing one after
the other, with only intent of writing out updated extent table
pointing to new extent pages.
Since blob->state is marked DIRTY before issuing each persist,
but a single persist completion marks state CLEAR.

Blob serialize correctly expects each persist to contain
dirtied metadata, in order to avoid unnecessary md writes.
Since all other instances of marking blob DIRTY is explicit,
assert in blob serialize is left as is.

Instead when running the queued up blob persists, the blob
state is marked DIRTY.

Side effect is that it will write out same md in some cases.

Fixes #1909

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I39f37299f3f0ebfccbdd4063781b5ecce286e993
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7640
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-29 07:10:13 +00:00
Nick Connolly
0fdd826a00 ut/nvme_ctrlr_cmd: add missing mutex init
Add missing mutex init for ctrlr ctrlr_lock.

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: Ib3d665a28e91a72d1f1f6d09c374583ff731fb6f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7613
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-29 07:10:05 +00:00
Jim Harris
5354d0c63f nvme: reset mapping_length correctly for contig SGL
spdk_vtophys() takes a mapping_length parameter, so
it can return the length for which the returned
virtual address is valid.

But spdk_vtophys() will only return the max
between the valid length and the input mapping_length
parameter.

So the nvme SGL building code for contiguous buffers
was broken, since it would only set the mapping_length
once, before the loop started.  Worst case, if a buffer
started just before (maybe 256 bytes) before a huge page
boundary, each time through the loop we would create
a new SGL for only 256 bytes at a time, very quickly
running out of SGL entries for a large buffer.

Fixes #1852.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib1000d8b130e8e4bfeacccd6e60f8109428dfc1e

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7659
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-29 07:09:43 +00:00
Jim Harris
6fd1cc3716 nvme: remove IDENTIFY_CNS quirk from normal QEMU SSDs
The IDENTIFY_CNS quirk was applied as part of QEMU
OCSSD handling in commit 6442451b.  But it was applied
not only to the OCSSD dev ID, but also the dev ID
for non-OCSSD NVMe controllers.

Starting with QEMU 5.2, QEMU will allocate a default
256 namespaces, but only some are active (associated
with the backing disks specified by the user).  QEMU
supports IDENTIFY_CNS, but since this quirk was set,
we wouldn't send a real IDENTIFY_CNS and instead
would just populate a fake list where all namespaces
were considered active.  This causes breakage in
a few places - mainly where we iterate through
the active namespaces, and then are surprised that
calling spdk_nvme_ns_is_active() returns false.

It was also breaking bdev_nvme_attach_controller RPC,
since by default we can only support returning 128
names, but since all of the namespaces were deemed
active, it was trying to return 256.

Fixes #1916.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4fdd27e0e36f0ac07a95f9f29aa83357e8505a45

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7658
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-29 07:09:43 +00:00
Mao Jiang
9ed384dab0 test/nvme_pcie: cases for building PRP and SGL request
Change-Id: I41c13f95b36308154dfa61ef8c6c54592ee7525d
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7143
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-27 08:38:12 +00:00
Karol Latecki
e1fe11875b autobuild: enable -Wno-stringop-overflow for DPDK builds only for GCC > 10
Use -Wno-stringop-overflow only for GCC > 10.
Change made to use the same condition as in
dpdkbuild/Makefile.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I6a396e250807f46720ff5dcaf21f13d802404a1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7537
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-27 08:37:12 +00:00
Karol Latecki
71b4e2a968 autobuild: build mainline DPDK with -Werror only for GCC > 5
DPDK now officially supports only GCC > 4.9.0.
Some older systems like Centos7  (gcc 4.8.5) can still
build DPDK, but sometimes fail due to -Werror enabled.
Disable this option to increase build success chance.

Fixes #1891

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0950ad01e2d8361851e9ac42c556004980e00ff4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7536
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-27 08:37:12 +00:00
Rui Chang
8af74b5ac7 test/setup: fix failure in allowed() when there is only one device
If there is only one device in ${devs[*]}, we see failure backtrace:
========== Backtrace start: ==========

in /home/jenkins/spdk_repo/spdk/test/setup/acl.sh:41 -> allowed([])
     ...
   36  	}
   37
   38  	allowed() {
   39  		PCI_ALLOWED="${devs[0]}" setup output config \
   40  			| grep "Skipping denied controller at " \
=> 41  			| grep -v "${devs[0]}"
   42  		verify "${devs[@]:1}"
   43  		setup reset
   44  	}
   45
   46  	setup reset

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I9aba6b19f6de6bea7149e0f39da8d5d8d6978a4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-27 08:36:35 +00:00
Mao Jiang
a417658803 test/nvme_ns: cases for identifying id descriptor and find descriptor
Change-Id: I20d0a798423ee08f4a9e019aa62b93981586d134
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7161
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-27 08:35:27 +00:00
Tomasz Zawadzki
5f106e26c9 version: SPDK 21.07 pre
This is first commit that should go into latest SPDK
after the code freeze for SPDK 21.04.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I515772a0c90a362bd4a23a7717281a8d7a9e4393
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7615
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-27 08:29:12 +00:00
Alexey Marchuk
2fd97e28bf sock: Deprecate enable_zerocopy_send in sock_impl_set_options RPC
This deprecated parameter will be removed in SPDK 21.07

Change-Id: I2b2fbcc798bb50fa6f9dfe35045f66e41c1ceaa9
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7608
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-04-27 08:13:32 +00:00
Alexey Marchuk
8e85b675fc sock: Add new params to configure zcopy for server, client sockets
When zcero copy send is enabled and used by initiator,
it could significantly increase latency in some payloads.
To enable more fine graing configuration of zero copy
send feature, add new parameters enable_zerocopy_send_server
and enable_zerocopy_send_client to spdk_sock_impl_opts to
enable/disable zcopy for specific type of sockets.
Exisiting enable_zerocopy_send parameter affects all types
of sockets.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I111c75608f8826980a56e210c076ab8ff16ddbdc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7457
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-04-27 08:13:32 +00:00
Niklas Cassel
0859db6b69 bdev/zone: add support for get num zones
There are three modules implementing the bdev-zone API:
bdev_nvme, bdev_ocssd, and vbdev_zone_block.

For all three modules, the number of zones can be calculated using:
block_count / zone_size.

To avoid this calculation being performed everywhere, create a helper
function in bdev_zone.h, together with the other zone APIs, such that
a user can easily get the number of zones.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I2967b15a604ab8bf4420588e7510b9820762f925
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7451
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 16:16:42 +00:00
Niklas Cassel
d69349af6b nvme/fio_plugin: fix zone_append option with multiple files per thread
Each fio thread can have multiple files that it writes to.
Which is why the per thread spdk_fio_setup() fio callback does
for_each_file() {...}.

One of these files can be e.g. a zoned namespace with append support,
another file could be a zoned namespace on another controller without
append support, and a third file could be a conventional namespace
(which never supports the zone append command).

Right now, we will return a fatal error if a thread has e.g. a zoned
namespace (with append support) together with a conventional namespace.

Instead of returning a fatal error, enable zone append only on the
namespaces that support zone append, and allow namespaces that do
not support zone append to continue as usual (using regular writes).

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ic6456d408cbe91563acd337a4b70c6e871fe34c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7611
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 16:16:42 +00:00
Niklas Cassel
09dd961b35 nvme/fio_plugin: fix report_zones
Commit f69367c788 ("fio_nvme: defer qpair allocation to file_open
callback") moved the qpair allocation from spdk_fio_setup() to
spdk_fio_open(). This broke spdk_fio_report_zones(), which needs a
qpair in order to get the initial state of the zones.

setup_files() in FIO calls td->io_ops->setup() (spdk_fio_setup()),
followed by zbd_init_files(), which calls zbd_init_zone_info(),
which calls zbd_create_zone_info(), which calls parse_zone_info(),
which calls zbd_report_zones(), which calls td->io_ops->report_zones()
(spdk_fio_report_zones()).

i.e. spdk_fio_report_zones() will always be called directly after
spdk_fio_setup(). .report_zones() is even called before the per
thread ioengine .init() callback.

Therefore, spdk_fio_report_zones() is called before the ioengine
.open_file() callback.

This is done in order to ensure that all threads will share the same
zbd_info struct, which contains the per zone locks.

Since SPDK nvme ioengine no longer initializes the qpairs in .setup(),
create a temporary qpair in .report_zones().

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: Ic376ac7844e40fceff092900ae7e4714bccf38e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7590
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 16:16:42 +00:00
Niklas Cassel
e9e97cb620 nvme/fio_plugin: fix initial_zone_reset
Commit f69367c788 ("fio_nvme: defer qpair allocation to file_open
callback") moved the qpair allocation from spdk_fio_setup() to
spdk_fio_open(). This broke --initial_zone_reset, which needs a qpair
in order to perform the initial zone reset.

While at it, move the initial zone reset from spdk_fio_setup() to
attach_cb(), as this is where all the other fio options are verified.
By placing it in attach_cb(), after the duplicated file check, we
avoid the need to loop through the whole fio_thread->fio_qpair list.

Since SPDK nvme ioengine no longer initializes the qpairs in .setup(),
create a temporary qpair, if the --initial_zone_reset option was used.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I7950304c58aef3ec783f7cd99cfb1e7d7817a197
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7589
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-26 16:16:42 +00:00
Sudheer Mogilappagari
2cbc9d4dff posix: Group connections of sock group on host side based on placement_id
On host side the connections are created and then added to thread's
poll group. Those connections could use different NIC queues underneath.
To route all connections of poll group through single queue a unique
placement id is chosen as group_placement_id and each socket of poll
group is marked with group_placment_id using getsockopt(SO_MARK) option.

The driver could use so_mark value of skb to determine the queue to use.

Change-Id: I06bda777fe07a62133b80b2491fa7772150b3b5d
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6160
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 15:33:03 +00:00
Karol Latecki
c06df11513 autobuild.sh: restore patches for older DPDK versions
Commit e9ebbdaefc updated
custom patches for mainline DPDK, but did not keep their
previous versions. We still need the old patches to build
with older DPDK versions.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibe222a444bed2fdee63c8dcfcebc8402bb991fbe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7610
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-26 14:23:15 +00:00
Ed rodriguez
b1144ce304 nvmf:Return cdw0 over tcp transport for retrieve cmd
tcp transport doesn't send a response capsule when
c2h_success is set even if cdw0 or cdw1 are non-0.

Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Signed-off-by: John Meneghini johnm@netapp.com
Change-Id: Ieba81fcc50342a2009f7931526e6f8392e26b6a5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6808
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-26 14:22:38 +00:00
Tomasz Zawadzki
a8d700419d examples/interrupt_tgt: mark reactor_set_interrupt_mode RPC private
The RPC is for testing purposes, as such does not need to provide
documentation in SPDK proper.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia153ad326e4c2ac43b511f9c2fba4aa95f1e3295
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 08:39:59 +00:00
Liu Xiaodong
74db63dcda interrupt_tgt: add RPC to set interrupt mode
Plugin RPC method 'reactor_set_interrupt_mode' is added
to set specific reactor run in interrupt mode or back
to poll mode.

Based on this RPC, scenarios later can be implemented
to verify interrupt mode functionality.

Change-Id: I9785e6942299bab7c949780faf18216e293f0b1d
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7347
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-26 06:58:36 +00:00
Liu Xiaodong
6c9b7d0896 reactor: call callback if no need to set intr
When do spdk_reactor_set_interrupt_mode, if reactor
already runs in the specific mode, directly call
callback function before return 0;

Change-Id: I1fd8b753e9881755aa128aabe6d1e2749e58b39b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7549
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 06:58:36 +00:00
Jim Harris
0e4690236b sock/posix: return error immediately if epoll_ctl fails
We do not want to do any further work on adding
the sock to the group if the epoll_ctl (or kevent)
fails.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I44b6dc86ce5676aa1b8d6c50b86f22758e4e37fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7594
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:48 +00:00
Jim Harris
b9563518c6 thread: do not ignore read() return
This fixes unused-result warning which prevents
building with -Werror.

Fixes: 924d4bf ("poller: add busy wait mechanism for intr")

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I074949b65e68405033322af34d90e96a896c8b8c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7593
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:48 +00:00