Commit Graph

1454 Commits

Author SHA1 Message Date
Jim Harris
d84a88c1a0 bdev: add base infrastructure for locked lba ranges
This adds new internal APIs bdev_lock_lba_range and
bdev_unlock_lba_range.  To start, these APIs will
manage dissemination of lock/unlock requests to all
existing channels for a given bdev.  This does not
yet interact at all with any I/O sent to the channel.

Future patches will check new I/O to see if they
are trying to write to a range that is locked.  Future
patches will also ensure we do not have overlapping
ranges active at once.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6d5b1cc84b41a7adc2a3c5791c766bb77376581f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-14 07:50:26 +00:00
Jim Harris
b87080efa2 bdev: add lba_range and overlapping checks
This will be used by upcoming patches for implementing
LBA range locks.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifa4ad8dcc0d09ccf20d35f010fcae19dcc17abc9

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478224
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2020-01-14 07:50:26 +00:00
Tomasz Zawadzki
159703c9dc ut/blob: added poll_threads() after deleting blob
For deletion to actually deliver results, poll_threads()
has to be called.

g_bserrno came from previous API call, not delete itself.

This patch fixes it so that g_bserrno is from the delete itself.

Next patch in series adds checks for number of clones.
Having such checks in first place,
would allow to notice lack of poll_threads() here before.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3477f5c5b99106b70ee3d5706906a4dc94283180
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478975
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-10 08:47:13 +00:00
Wojciech Malikowski
0637e2d9be lib/ftl: Rename ftl_dev_num_bands() to ftl_get_num_bands()
This patch is part of name refactoring associated
with moving FTL to work with zone bdev API.

Change-Id: Ia4c6ec7b245600bca895f7802fb7f68c8d0400e4
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479845
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
2c2a005695 lib/ftl: Rename ftl_dev_num_punits() to ftl_get_num_punits()
This patch is part of name refactoring associated
with moving FTL to work with zone bdev API.

Change-Id: I9e0a569e9dfd0e8b8236c0840ce10b92362c4996
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479844
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
3ff9d6e45b lib/ftl: Rename ftl_dev_lbks_in_zone() to ftl_get_num_blocks_in_zone()
This patch is part of name refactoring associated
with moving FTL to work with zone bdev API.

Change-Id: I09f671a9c6539cc259c297514b24978587c9b392
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
3725b4484a lib/ftl: Rename ftl_num_band_lbks() to ftl_get_num_blocks_in_band()
This patch is part of name refactoring associated
with moving FTL to work with zone bdev API.

Change-Id: I09781323b2ee25dbcee61eecf95ce7eca703ab93
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479672
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
1735a0acd3 lib/ftl: Remove position from ftl zone structure
There is no need of tracking position of zone inside
band.

Change-Id: I1488f23ccfd1f7eee1bd85fdb7ef8ace08ffca40
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468347
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
04f754d419 lib/ftl: Remove ftl_punit structure from device
ftl_punit is not needed anymore.

Change-Id: I90f22b980f8e0436e8c899941dddb20b13167766
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468346
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
7ef136d3f4 lib/ftl: Remove punit from ftl_zone struct
Parallel unit is not needed anymore in ftl_zone
structure.

Change-Id: Ie8d78e3ccf64db84aff19ad2ce11e662d4f3f724
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468345
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
aea8e78df2 lib/ftl: Consume whole underlying OCSSD device
Moving to zoned bdev API will not allow to setup
physical range of underlying device so we need to
remove such capabilities from ftl device.

Change-Id: Ia807a11e992a221fce906d4ab122a6c3b1391280
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467949
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
e47e16d3a4 lib/ftl: Replace ftl_ppa struct with ftl_addr
FTL working on top of zoned bdev doesn't need
physical page address (PPA) anymore.
ftl_ppa was replaced with ftl_addr which represents
zoned device addressing schema.

Change-Id: Ied5750a7ab2f4ce42067ff3e69c1f26f85f5022a
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467633
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
748785c2a9 lib/ftl: Change ftl_ppa fields to be zone specific
Remove some Open Channel dependencies from ftl_ppa struct.

Change-Id: Ic088b84c56a928906c6c01c6ef74e69be6d0107f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479549
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-10 08:42:51 +00:00
Wojciech Malikowski
7083fa3e35 lib/ftl: Remove group from ftl physical address
ftl_ppa address do not need to expose group
information when FTL will move to zones API.

Change-Id: Iece1b32f9bcd8985260668c953089eb9951f13cc
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467603
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-10 08:42:51 +00:00
dongx.yi
1044fe25e6 test/unit: Add set_host_id_cmds to unittest.
This function was ignored and a little different from others.
Add this to increase code coverage rate.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: I35b51feb8894aca54a3b21436bc21ec12e37adf7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475814
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-08 09:12:10 +00:00
Maciej Szwed
80da95481b unit/bdev: Rename stub_submit_request_aligned_buffer
Rename stub_submit_request_aligned_buffer function
to stub_submit_request_get_buf and its callback function
to stub_submit_request_get_buf_cb.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ie8cde43d30c5aa2d353bb33b023d5318948bee44

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478944
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-07 12:31:32 +00:00
Tomasz Kulasek
1c5d980300 lib/nvme: add hw support for fused operations
Change-Id: I2a184a2d60f18439947afa2d4855d51b92ed282e
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476137
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-07 12:31:32 +00:00
Seth Howell
8209c8cfa0 lib/nvmf: rdma qpairs now store device, not port.
The only reason the qpair ever uses the port is to get to
the device attributes so skip the middle man.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib14a97ceaa0c49176027d6c35c5cb2787a845cc1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-07 12:21:36 +00:00
dongx.yi
94f2def65a test/unit: Add nvme_uevent_ut.c to unittest.
Add nvme_uevent_ut.c to unittest, and test parse uevents.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: Id8bb8336790ae5e031b357dc49163d86dfa1cd57
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475080
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-07 12:20:01 +00:00
GangCao
be4a5602ce blobfs: write IO directly if it is lba aligned
UT also added for the block aligned write and verification
of the written data.

Change-Id: I8751f437c71dd7f3a4556fee420460d6dffdd4e5
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478424
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-07 12:19:14 +00:00
Shuhei Matsumoto
60eb6da8b7 lib/event: Add spdk_for_each_reactor to process event for each reactor
This API will be used in the upcoming reactor_get_stats RPC first.
This API is not public but internal in SPDK. Add necessary unit test
together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I761ffe3c8d6d8da4594c0d728b479f9f40275fc9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-01-03 08:33:25 +00:00
Shuhei Matsumoto
368be3b199 ut/event: Add test case for reactor's thread schedule
Test if spdk_reactor_schedule_thread() scheduled thread correctly
to one of the allowed cores.  Use thread library and expect
spdk_reactor_schedule_thread() is called at the end of
spdk_thread_create().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibb536d16849c29081fb63341f5dbe1a4c0db1d18
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-03 08:33:25 +00:00
Shuhei Matsumoto
c697cbfd9e ut/event: Add test case for SPDK event call
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4977d1e438b1187411d1e20c392708520780fb3f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478155
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-03 08:33:25 +00:00
Shuhei Matsumoto
5940e9cd63 ut/event: Add test case for reactor initialization
Use the newly added mocks for CPU core operations.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If90bba6fd68c29c83cdb3c201a2a9fbdb40f6755
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478154
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-03 08:33:25 +00:00
Shuhei Matsumoto
f74107fb85 test/common: Mock CPU core operations for unit tests
Mock DPDK lcore operations for unit tests. Remove duplicated stup
from unit tests for NVMe-oF FC transport. The next patch will use
them.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f6fbd8f45942ca13fcf10ba1740a9fe8e573063
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478153
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-03 08:33:25 +00:00
Jin Yu
cbc9d34362 test: add the UT of the timeout IO in mt
Extend the unit test of the timeout IO to multi-thread.

Change-Id: I6cd19b629a5f1e473f69685f495fa067a5f5da25
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476052
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-30 11:49:04 +00:00
Jin Yu
f29d20a21e vhost: add a protocol_features parameter in vdev
Add the protocol_features in vdev. There are two features
would be used in vhost device one is the virtio_features
the other is the vhost-user protocol_features. For different
vhost device, the supported features are different so we can
separate them.
Another reason is that I tested the VHOST_USER_PROTOCOL_F_
INFLIGHT_SHMFD in vhost-scsi with QEMU(version:4.0) and found
that Qemu can not boot up. After investigating found that inflight
flag is negotiated but the Qemu doesn't support this feature
and in DPDK function it is handled as an error and disconnect
with Qemu. It's a bug in DPDK and will fix it.

Change-Id: I72e418cb1885bf7dcbd0285d9cec1ad6af0665de
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478814
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-30 11:46:24 +00:00
Wojciech Malikowski
77d591e172 lib/ftl: Change ftl_chunk structure to ftl_zone
This is starting point for moving current FTL
implementation which is working on top of
Open Channel NVMe driver to work on top of
abstracted zoned bdev.

This patch is changing name of ftl_chunk structure
to ftl_zone and start using zone states from zdev
interface.

Change-Id: I5429f489cc08a1ac27f09aba3dca4b40ea95eeb3
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-24 08:14:55 +00:00
Shuhei Matsumoto
b2927bedea ut/event: Add unit test for reactor.c
Add framework and a simple test case to test creating reactor.
Subsequent patches will add more test cases.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie0eb0e49bfe40469b351b21e40558a1bf5584cce
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478151
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-12-20 10:05:38 +00:00
Changpeng Liu
ff9516bdcc nvme: call the callback for the queued requests when there is submission failure
For the requests which don't have children requests, SPDK may queue them to
the queued_req list due to limited resources, in the completion path, we
may resubmit them to the controller.  When the controller was removed
the submission path will return -ENXIO and we will free the requests directly,
so the callback will not be trigerred for these requests.  Here we added a
flag to indicate the request is from queued_req list or not, so for the failure
submission, we can triger user's callback.

Fix issue #1097

Change-Id: I901ac81733c2319e540d24baf5b8faa1c649eb35
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477754
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-12-20 10:04:57 +00:00
Changpeng Liu
10cd9e86b9 UT/nvme: use STUB macro for the empty function APIs
Change-Id: Id037a0cdc1e054dad0454e0897b04629b78e7d50
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-12-20 10:04:57 +00:00
Shuhei Matsumoto
f29c728980 lib/iscsi: Move iscsi_conn_abort_queued_datain_task() from iscsi.c to conn.c
Followin the last patch, move iscsi_conn_abort_queued_datain_task()
and iscsi_conn_abort_queued_datain_tasks() from iscsi.c to conn.c.

Refine unit tests to check more accurately. Adding more test cases
will be done later.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I30530e6871a78a58d9fb472f62168862298884a0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477417
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-12-20 10:03:00 +00:00
Konrad Sztyber
9ec598d115 lib/thread: poller pause / resume
Added spdk_poller_(pause|resume) that allow a poller to be paused and
then resumed at a later point.  These functions come in handy in cases
when a poller is known to be idle until a certain event occurs.

Change-Id: I7f21c80eb9ac4e8e1cf24d66f99da5687aafe358
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477920
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-20 10:02:15 +00:00
Konrad Sztyber
f4576dec88 bdev/ocssd: media management events
This patch adds the logic for retrieving chunk notification log and
translating it into media management events to be sent to appropriate
Open Channel bdev.

Change-Id: I7e4860eda23e61d6208fc5f5861e8fd2b75685d3
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-20 10:02:15 +00:00
Konrad Sztyber
64fe514efa bdev/ocssd: parallel unit ranges
This patch adds the ability to create bdevs on specified parallel unit
ranges on one OCSSD controller.  It allows the user to create multiple
isolated bdevs, each operating on a separate set of parallel units.

To create a bdev on a specified range, a new parameter -r|--range was
added. For example:
 rpc.py bdev_ocssd_create -c nvme0 -b nvme0n1 -r 0-4

will create a bdev on parallel units 0 through 4 (inclusive).  If the
parameter is ommited, the bdev will span across all available units.

Change-Id: Icd94cf1e22fcc72806ef1ce1efd2d7031010009f
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469898
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-20 10:02:15 +00:00
Seth Howell
4328a67ea7 lib/reduce: add array bounds checking for iovs.
We also need to make sure that the deconm_iov is large enough to handle
all of the iovs we claim to support plus 2, one for offset into the
chunk when doing writes and one for the remainder.
Plus a unittest to demonstrate the possible out of bounds error in the
library.

Change-Id: I7747ad39f76e50f25ecf5168b01e046f71fa0ea8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478125
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-19 11:04:57 +00:00
Jim Harris
1d94a0b0e1 ut/bdev: fix histogram buffer parameters
These buffers weren't actually used, but "buf" is an array,
so we should just pass buf, not &buf.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9d73a73b138d2bd9c14172c58b1825cf83532b4f

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478119
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-12-18 10:09:45 +00:00
Shuhei Matsumoto
e37fc5a32a lib/nvmf: Accept KATO for discovery controller
Some NVMe applications require SPDK NVMe-oF target to support
KATO for discovery controller.  Hence change discovery controller
to accept KATO.  Update unit tests accordingly.

Fixes the issue #1089.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib56e3b0b0faaf58276f9e692704763c1e5e5b042
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476810
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-12-18 09:38:46 +00:00
Tomasz Zawadzki
1d11ab120d ut/crypto: redirect mock rte_lcore_count
With DPDK 19.11 rte_eal_get_configuration() and rte_config
structure were made private. Those were only used in
the inline rte_lcore_count() included from DPDK.
After the update they were no longer available.

Since only rte_lcore_count() was used directly in crypto,
mock that and return 1, as was done previously.

This was tested with DPDK 19.08 and DPDK 19.11.

Change-Id: I13e4d9743b17a34ad786283f8b567d01e036d368
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477841
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-12-17 09:29:39 +00:00
Tomasz Zawadzki
2fbeb7ea52 ut/compress: remove rte_eal_get_configuration stub
This went unused in the unit tests.
Tested with DPDK 19.08 and DPDK 19.11.

Change-Id: I738919379b5751697f9533f72fbaf77993cb6fb5
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477840
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-12-17 09:29:39 +00:00
Shuhei Matsumoto
3c7890d24f lib/iscsi: Move iscsi_conn_handle_queued_datain_tasks() from iscsi.c to conn.c
Operations to queue iSCSI tasks are in iscsi.c and conn.c and cross
references due to this separation makes us difficult to create unit
tests.

This and subsequent patches will try to disentangle cross references
by moving some functions from iscsi.c to conn.c.

This patch moves spdk_iscsi_conn_handle_queued_datain_tasks() from
iscsi.c to conn.c.  For unit tests, we don't add anything new in
this patch and just create necessary simple stubs.  After code
movement, new unit tests will be added.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If5b8501a1ef7ea53682a3437c7eb2375aa52ee3b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Shuhei Matsumoto
ff7e7c83e6 ut/iscsi: Replace trace library link by stubs
Subsequent patches will change trace functions to be called in
unit tests.  Using stubs is enough as done in other libraries
and do the change in this patch.  For iscsi_ut.c, add stubs to
not iscsi_ut.c but common.c to cover other files.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia14bc932f451b14bdb0ba6e0cb949463490a879e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Shuhei Matsumoto
d4914dee06 ut/iscsi: Initialize struct to zero by not memset but "={}" in tgt_node_ut.c
Using "={}" at definition is clearer and saves lines.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I75a99583f1ecc84d4c47cacd9ce58034edb4e756
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477413
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Shuhei Matsumoto
ce3b8472ad ut/iscsi: Initialize struct to zero by not memset but "= {}" in iscsi_ut.c
Using "= {} at definition is clearer and save lines.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I410279bd3640e03aa9b6eeb488be4404d110b0bd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Konrad Sztyber
1ad438311e bdev/ocssd: request resubmission
If a spdk_bdev_io cannot be completed, because it is sent to a zone that
is currently busy, reschedule its submission.  This mechanism will be
used by appends, as mutliple append commands can be directed to the same
zone at the same time.

Change-Id: I60da2bd1835380812d22536ea275fb8fed9f8561
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477437
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
153fbcb1f8 bdev/ocssd: JSON configuration
This patch adds support for saving JSON configuration of the OCSSD
bdev module required to recreate the current state.

Change-Id: Iedbdb8b4a2b7dd02a223ce6f073553e71b9c040a
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469090
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
88406f3302 bdev/ocssd: retrieve chunks' info during initialization
OCSSD bdev needs to keep track of some portion of zone's state (write
pointer, its capacity, whether it's busy) in order to support the append
operation.  To achieve that, all the info needs to be retrieved for all
of the chunks and translated to zone format.

Change-Id: Id88282576aba337b50e7c7e1b9e9ff5aba25a3db
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468937
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
7ac802e848 bdev/ocssd: get zone information command
Added zone information command translating Open Channel's chunk
information to spdk_bdev_zone_info structure.

Change-Id: Ifdb15f2e0ea2fb8422810fa64f18942fcb6e4582
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468213
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
704830b3b7 bdev/ocssd: zone reset command
Added zone reset command.  It's very similar to a regular data set
management deallocate command with the only difference being that it
operates on zones instead of blocks.

Change-Id: Iae4d77c6877b6906231336b13fad1167de149f20
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 09:28:35 +00:00
Ben Walker
5d497f6cf5 nvmf/tcp: Use writev_async for sending data on sockets
This eliminates the flushing logic, simplifying the tcp
transport.

This also happens to greatly improve performance, especially
on random read tests. The batching done in spdk_sock_writev_async seems
to be more effectively than the previous batching logic in the tcp
transport.

Change-Id: Id980ac6073e380dc75f95df3f69cb224f50fb01b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470532
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-16 12:34:02 +00:00