4923 Commits

Author SHA1 Message Date
Darek Stojaczyk
0acb123aa4 iscsi: allocate PDU mempools with memory from any socket
Right now they can only use memory from the socket
that performed iSCSI static initialization, which
doesn't make sense. Those mempools can be used from
a different socket as well, so don't restrict them
to any specific one.

Change-Id: I0def1c554ed6227ab0f0a7be107c4c1c61f40c96
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450055
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-04 01:15:24 +00:00
Vitaliy Mysak
4fad4b86dd ocf: finish OCF reqs after put_io_channel
Relaunch queue poller on put_io_channel callback of OCF bdev
  to delay thread shutdown. New poller will finish all pending
  OCF requests as there might be some even after all SPDK IOs
  completed.

This solves the issue of OCF not being able to
  complete all its work because of queue poller getting
  unregistered in callback of put_io_channel.

This patch also changes unregister procedure:
  we call ocf_cache_stop in callback of io_device_unregister
  instead of in callback of bdev_unregister.

Change-Id: Ib7e41fc25e71029a73bb76a62e39e6bf4b8189ce
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-03 00:00:20 +00:00
Vitaliy Mysak
9977424df1 ocf: make vbdev_ocf_delete function asynchronous
vbdev_ocf_delete function accepts callback now.
RPC delete_ocf_bdev is also updated to adopt this change.

Change-Id: I1d357a5e37015268e28c07fd81dc35f48ec80ab8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449777
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-03 00:00:20 +00:00
Seth Howell
f4adbc79ce rdma: optimize and move buffers variable.
The buffers are really specific to the request and not the wr or data
object. In the case of multiple wr requests, the maximum number of
buffers per req is equal to the number of SGEs in the NVMe-oF request
*2.

Change-Id: Ic59498bfed461d180adb2fb9a481ac5b11fa9252
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449108
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 23:26:08 +00:00
Seth Howell
e590f607e6 nvmf: Report that we support more than one SGL element
Change-Id: Idf5aeb1fa3d6a3a83042bd699e0099b95e34f5b9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/428776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-02 23:26:08 +00:00
Seth Howell
62700dac2e nvmf/rdma: Add support for multiple sgl descriptors to sgl parser
Enable parsing an nvmf request that contains an inline
nvme_sgl_last_segment_descriptor element. This is the next step
towards NVMe-oF SGL support in the NVMe-oF target.

Change-Id: Ia2f1f7054e0de8a9e2bfe4dabe6af4085e3f12c4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/428745
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-02 23:26:08 +00:00
paul luse
1e6cdc190d bdev/compress: misc comment and log message cleanup
Change-Id: I65ac14f1bcb506b211d129e5e6aa1c8872f3a2f2
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449803
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-02 23:25:27 +00:00
paul luse
a827d07c2b bdev/compress: integrate with DPDK compressdev
Change-Id: I0729f688c72651790ac05f4991c04deebca2af39
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-02 23:25:27 +00:00
Ben Walker
032920f250 thread: Implement a thread scheduler
Change-Id: Ie1cad80a071f9de066fe18093a4c4c8e726e5a77
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 23:24:30 +00:00
Ben Walker
b71bebe88a blobfs: spdk_fs_thread_ctx is no longer a channel
This also means that the RocksDB threads don't need
to be SPDK threads any more.

Change-Id: Icadd6dd446958ebf470ad8ab8239f5390942eb87
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-02 23:24:30 +00:00
Ben Walker
e9d400d5b9 blobfs: Change channel type for synchronous operations
In the next step, this won't be a channel at all.

Change-Id: Ia8fe4da5b0b283e8dfc5c6477b84cfdd346d89a0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-02 23:24:30 +00:00
Jim Harris
d25f2db94a reduce: account for compressed buffers
When we decompress, pass the size of the compressed
data - not the size of the buffer itself.

We also need to remove an assert that's no longer
true since we can now handle compressed chunks.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifb7896db12ccbefdd49815b5037af11ddd82e625

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-02 21:40:21 +00:00
Jim Harris
fa0d54d431 reduce: write uncompressed data when compress returns error
Most typically, the error will be -ENOSPC, meaning the
destination buffer wasn't big enough to hold the compressed
data.  But even for other types of errors, just write the
uncompressed data rather than treating it as an I/O
failure.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id6526cab95488a1a587b183237f9980c83287761

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449511
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-02 21:40:21 +00:00
paul luse
b95d25975d bdev/compress: add unit test mocks and empty functions
Test functions to  be added later in series

Change-Id: I96286b6214b4577df334d180e7c3d641e8ce8f68
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445371
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-02 19:25:28 +00:00
Changpeng Liu
790c47d479 iscsi: fix up segment fault due to failed initialization
My iSCSI target reported segment fault after memory allocation
failure, and add the addtional check here can fix the issue.

Change-Id: Iee3e497d7028fbface6d110c78e73965ef0f178b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449717
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-02 19:25:00 +00:00
Maciej Szwed
e87e8263df lvol: Fix for lvol failed deletion flow
lvol should not only be freed but also removed
from lvols list when deletion fails.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I3a047e1cc611e71b1544ca77c256a879dd2efdae
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446619
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 19:23:22 +00:00
Jim Harris
ca44fd6955 nvmf: put \0 at end of default serial number
It's not standard to put a newline here - let's use a null
character instead.

Found while using nvme-cli - when creating a subsystem with
default serial number, the right justified callout text had
an extra newline in it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8a81dafb4f6c30f7bf2dcebfa7a5b19cfe3ab5fc

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449645
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-02 13:08:06 +00:00
Claire J. In
3be9febdf9 lib/ftl: check for null
Fixes #726.
This prevents a segmentation fault by checking the pointer
before accessing it.

Change-Id: I18a3776fbcd6242c6efd1b88044debaa88c7342f
Signed-off-by: Claire J. In <claire.in@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449591
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-02 08:22:48 +00:00
paul luse
cd4de753af bdev/compress: prepare to add some unit tests
Misc cleanup from earlier patches, some naming issues and other
minor things that are needed before adding UT.

Change-Id: Ic76f0e2fd12e48782b048fb8785902b1098cf352
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 00:06:34 +00:00
paul luse
aa3fdd01bd bdev/compress: insert vol unload into base bdev hotremove path
Change-Id: I0c23746fb210a8c209e98276fe0830185ee81407
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447372
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 00:06:34 +00:00
Wojciech Malikowski
dd54dff78b lib/ftl: Keep DMA buffer for metadata as part of ftl_md structure
This patch is starting point for metadata refactor
which is needed for efficient ANM events support.

Change-Id: I81d864605e69008d8e3922fb61adf504187447a1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-02 00:04:58 +00:00
Darek Stojaczyk
27a23a33f9 blobfs: switch to spdk_*malloc()
spdk_dma_*malloc() is about to be deprecated.

Change-Id: Ib9fb94463a69ffd9660ab2c7fbe02826a7e06741
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449796
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 00:02:58 +00:00
Pawel Wodkowski
076ebfadfa bdev: add register and unregister notification
When bdev is registered it will send "bdev_register" notification and
"bdev_unregister" when unregister is complete.

Change-Id: I3b37a4cb766a473f3da63a571036add366e78fa5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 00:02:12 +00:00
Shuhei Matsumoto
038b319dfb iscsi: Remove the prefix spdk_ from private functions missed in the previous patch
A few private functions had still the prefix spdk_ in their names.
This patch completes the effort to remove the prefix spdk_ from
private functions in iSCSI library.

Change-Id: Ic1a1e85220edb72296ca4881bdf7b550c4f6b6b8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-01 23:59:15 +00:00
cranechu
6a67d5178e nvme: remove set_state after nvme_ctrlr_identify_id_desc_namespaces
Fixes #722. The state was set in nvme_ctrlr_identify_id_desc_async

Signed-off-by: cranechu <cranechu@gmail.com>
Change-Id: I232f0035e8c45d49eca2de7174c91860a299d804
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449527
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-01 09:09:07 +00:00
Konrad Sztyber
67194e3ac7 lib/ftl: added band selection traces
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iafe7021febd12f1f9b00890c357a5d3f26611c31
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449422
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-04-01 09:05:33 +00:00
Konrad Sztyber
9ed090d23d lib/ftl: traces refactor
Replaced tpoint_ids generated based on the properties of the IO with a
static list of tpoint_ids, to fit inside the per-group maximum tpoint
limit (64).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ifb8b8637a1e6d8f0e7fd3f4363b97fdbedcdb8d5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-04-01 09:05:33 +00:00
Li Feng
80c87083ce bdev/iscsi: fix iscsi bdev wrong lun id
When you connect a iscsi lun as a bdev, and the lun id is not 0,
all IOs will fail.
We should use the correct lun id in libiscsi.

Reproduces like this:
./scripts/rpc.py construct_iscsi_bdev -b iSCSI0 -i
iqn.2016-06.io.spdk:init --url
iscsi://127.0.0.1:3261/iqn.2016-02.com.smartx:system:fl-iscsi/1

Signed-off-by: Li Feng <fengli@smartx.com>
Change-Id: I2480e866fdda17426362aca3cb30b7e90c927547
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449318
Reviewed-by: Kyle Zhang <kyle@smartx.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-01 01:16:51 +00:00
Shuhei Matsumoto
0ed66e7ef6 scsi: Remove the prefix spdk_ from names of private functions
This patch doesn't change any behavior too.

Change-Id: I67be498bfd4f1ae42e6239de4539e5e99f158279
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-01 00:52:56 +00:00
Shuhei Matsumoto
893e02a54f iscsi: Remove the prefix spdk_ from names of private functions
iSCSI library had used the prefix spdk_ for most functions regardless
of private or public. Using the prefix spdk_ only for public functions
will be helpful to distinguish private and public functions, and
will be helpful to investigate issues or do further improvement.

Besides in iscsi.c static variable spdk_arc4random_initialized had
the prefix spdk_, and change it to g_arc4random_initialized according
to the SPDK's good practice.

iSCSI library still have some issues but is more stable than before
and now will be the good time to adjust the naming rule to other
libraries.

This patch doesn't change any behavior.

Change-Id: Ia0b8585a7ce6662cabc0e6f57b7ccb8a40342297
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449396
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-01 00:52:56 +00:00
Shuhei Matsumoto
b7039655ce iscsi: Remove unnecessary function declarations
These function declarations are not used.

Change-Id: If525f4157e3fa9e5d718e6c7aa77db8584edd14c
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449395
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-01 00:52:56 +00:00
Piotr Pelplinski
de5f53220c notify: add notification library
This patch adds library that will provide ability to notify about
events. On one side each spdk library can specify what kind of
notification it can produce and on the other side libraries can fetch
those notifications.

Example is bdev module, which lib notify about added or removed bdev.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Change-Id: Ia95e564a8a43400b2745d9de8217b9cc84cd1b16
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/431920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 21:15:14 +00:00
Changpeng Liu
851829c4ea nvme: don't poll the CQ after CQ is deleted
Change-Id: I7985907298ca714df82426f240c8b32b0139be6f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-29 21:14:50 +00:00
Ziye Yang
bdb1d5713f blobfs: Add trace info for request allocation.
Our blobfs allocates fixed number of reqs on three different channels
(the default size is 512), if the req cannot be allocated, our blobfs
related function will do nothing.

With this debug info added, we can easily know the issues.
If the fixed number of requests are not enough, we need to change the
design, using malloc/free together with the fixed number of requests.

During high presure test, I think that if the reqs are not allocated,
it will cause the sync, flush related issues.

Change-Id: Ied8cd5197c4f87b861bd7dc09edd5a095fdcc802
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-29 21:14:29 +00:00
Darek Stojaczyk
4a51888953 vhost/scsi: don't hotplug targets to stopped sessions
This is just a cleanup. There's no need to hotplug
or hotremove SCSI targets from stopped sessions, because
those sessions can't access any targets anyway. When
session is started, it already inherits all SCSI targets
from the vhost device. When it's stopped, it releases
resources of all targets. Intermediate changes have
no effect whatsoever, so don't do them.

Change-Id: Ibf283bcf8260e71dec8d9ea39a9461a978031ab3
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449392
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 20:19:12 +00:00
Darek Stojaczyk
1188bdd70d vhost/scsi: forbid removing targets that are still being added
It is theoretically possible for an asynchronous
hotremove request to be finished before the hotplug
request that was started first. This is obviously
not expected and will most likely result in a resource
leak.

For SCSI target hotplug, we immediately update the
whole vhost device object and then asynchronously
ask each vhost session to poll the changes.
For hotremove, we see the device attached in the
whole vhost device object, so we immediately mark
it as "still being removed" and proceed aynchronously
asking the sessions to hotremove. When session
receives the hotremove event first, it will either
fail an assertion (when debug is on), or do nothing.
The subsequent hotplug event will attach the target
again - and that target won't be ever freed.

Change-Id: I784c979fb47127a4238038ad9fb5ed1cac3ced04
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 20:19:12 +00:00
Pawel Wodkowski
346b59bd40 vhost-scsi: separatate scsi device state from scsi session state
For session context we need only a few fields from
spdk_scsi_dev_vhost_state structure, so introduce
its stripped variant as a separate structure.

Change-Id: I1be4e77447443d156f86033450892cb7cb464cb9
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 20:19:12 +00:00
Darek Stojaczyk
3fd405c54a vhost/scsi: remove hotremoved scsi targets on device stop
In cases where initiator closes the connection as soon
as it receives a hotremove event, there is a possibility
of SPDK vhost stopping the session before finishing up
the asynchronous target hotremoval. The target would be
either hotremoved once the session is started again
(and it registers its management poller again) or it
could cause a potential memory leak if that session is
destroyed. Even though the SCSI target itself is always
freed, the hotremoval completion callback is only called
from the management poller. At least in our RPC case,
not calling that callback results in leaking the context
structure and some json data.

We fix the above by calling all hotremove callbacks just
before stopping the device.

Change-Id: Ibfd773e1ab82b63643c57d7a9d37304e3007e38b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/439445
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 20:19:12 +00:00
Seth Howell
934775db43 rdma: make semantic changes to fill_buffers func
Changing i to iovcnt in all references to the req->iov structure will be
important when we start processing multi-sgl requests.

Change-Id: I90a9b6d872b94f846ae7d29a45dd2703eafa6175
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449201
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-29 19:02:22 +00:00
Seth Howell
e70a759489 rdma: pull buffer assignment out of fill_iovs
This will be used by the multi-sgl version of this function as well.

Change-Id: Iafeba4836a77482fa2a158f86f1c17fe7fdeb510
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449104
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-29 19:02:22 +00:00
Konrad Sztyber
1d98208176 bdev/ftl: defer bdev initialization
Defer the initialization when cache bdev mentioned in the config doesn't
exist when the FTL bdev is being created.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I084502ae38c9ae5e385e5566550ec3cb8b6f9d2e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448630
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-29 04:42:48 +00:00
Konrad Sztyber
a0cb5e9d77 lib/ftl: retrieve caching bdev from configuration
Added means to configure libftl to use (optionally) another bdev as
persistent write buffer cache.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97028a681be168d9386eac8a226631ff772f803b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448629
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-29 04:42:48 +00:00
Konrad Sztyber
6a7c9763e1 bdev/ftl: reduce the number of bdev_ftl_create's params
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Id9edced3670f7de1cd08cd8e3481cc5288103458
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449405
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 04:42:48 +00:00
Ben Walker
c04332d494 rocksdb: Add an SpdkFinalizeThread function
This is just a placeholder. The real implementation comes later.

Change-Id: I657b240977ba09fd6f24e9e36e97e293e73b5801
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-29 04:42:24 +00:00
paul luse
29b446a1bc bdev/compress: insert vol unload into destruct path
Change-Id: I10c7dff267ce469c0b01db9d6a1dbf89d2fad877
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447368
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-29 04:41:32 +00:00
paul luse
8944bb469f bdev/comrpess: add/fix bdev delete
Was partially present but not complete. Fits here in the series as
it requires read/write operations.

Change-Id: I552896c934bf60256625a83951a95d58d2719646
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447358
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-29 04:41:32 +00:00
paul luse
e2a2b637c9 bdev/compress: incorporate reducelib read and write
Change-Id: I280e9d51bf23ef101b8a3ba8a68a0c1eb10cb65c
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/438274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 04:41:32 +00:00
Ziye Yang
b151999f06 blobfs: Add the return value check for calling cache_append_buffer
Reason: We need to detect this issue. Otherwise, it will
cause null pointer issue later

Change-Id: I4d9aec0570e0b46274ebf3b9642b4727049a4a3b
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-29 02:38:33 +00:00
Jim Harris
11b38a585a app, log: clarify how to enable log flags
Let's point users to use --enable-debug now instead of
CONFIG_DEBUG=y.  Also to be completely pedantic, use
"configure" instead of "build" to make it more clear
this flag needs to be passed to the configure script,
not make.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4a6f6313a4f8e87cbb1d79cb68645305abb0e106

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449339
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-29 02:35:48 +00:00
Jim Harris
df082c931e reduce: fix chunk_is_compressed calculation
Stupid bug - a chunk is compressed when the number
of io_units for the compressed data does *not* equal
the number of io_units per chunk.

Paul found this during some of his initial DPDK
framework integration testing.  I have some better
unit tests coming up to test this further, but want
to get this obvious fix in for now.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaf9c78a1c1d6045070e7625c6a54ab3d227c8ea7

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-29 02:34:21 +00:00