Commit Graph

5733 Commits

Author SHA1 Message Date
Evgeniy Kochetov
38ab383a8f nvmf/rdma: Add RDMA polling statistics
RDMA polling statistics: number of polls and number of completion
entries returned.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Iabcf2cb6f6a35f595b89b58cdfcd177a637dda13
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445289
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 20:30:00 +00:00
Evgeniy Kochetov
43bb4e6b1f rpc: Add NVMf transport statistics to nvmf_get_stats RPC method
This patch adds transport part to nvmf_get_stats RPC method and basic
infrastructure to report NVMf transport specific statistics.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ie83b34f4ed932dd5f6d6e37897cf45228114bd88
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452299
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-26 20:30:00 +00:00
paul luse
ca1a3bf18e bdev/compress: use base bdev alias as base for comp_bdev name
If available, if not use the bdev unique name. This results in
a much friendlier comp_bdev naem for the user.  For example,
now it would look something like this: COMP_lvs0/lvs as opposed
to like this COMP_0b149b31-b66b-4cf7-ab39-a55b50788cd1

Change-Id: I319a141221ed8880edcec930ca5be9f256b105e0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-26 19:47:33 +00:00
paul luse
e9211cbf45 bdev/compress: fix qpair assignment code
Was previously creating the number of qp based on the # of cores
reported via DPDK function.  Also was assigning one qp per comp_bdev.
After testing w/multiple threads in fio 2 things needed to change:

* remove use of rte core related functions as fio threads are not EAL
threads.  This required choosing an upper limit for the number of
threads that the bdev module will support
* changing the qp assignment code such that comp_bdevs share the
same device_qp if on the same thread

Fixes issue #879

Change-Id: Ib968ee04bef63013616546341974a4431d8f266a
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 19:47:33 +00:00
Jan Kryl
59339feac3 jsonrpc: tolerate null json value for parameters in json-rpc call
Rust lang has notion of empty value `()` which is translated to json null
value using serialization library like serde. Thus it means that jsonrpc
methods having no parameters translate to parameters with null value (as
opposed to a request without parameter member as it is done now in spdk).
This change handles null parameter gracefully instead of returning an
error - improving interoperability with such clients.

Signed-off-by: Jan Kryl <jan.kryl@mayadata.io>
Change-Id: I8c3cc5613582aebb10ac6eaee3ac4e6538aaa0b1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463171
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-26 19:30:57 +00:00
Wojciech Malikowski
825ae66bd0 event/subsystem/vmd: RPC support
Added enable_vmd RPC call.

Change-Id: Ib0bea12f097a46a8be537a1db8aace6cb3b9b699
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459949
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 18:27:40 +00:00
paul luse
f9cae9d20f lib/reduce: minor cleanup for consistency in recent changes
Had used a few different means to determine if there's space
remaining in a chunk after accounting for offset and host data,
just making them all consistent. Nothing functionally changes.

Change-Id: Iffd3b0b6d8ab1253f5b1bebda88d6f63d5061e19
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463023
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-26 17:44:18 +00:00
Darek Stojaczyk
f89166c089 scsi: remove bdev_scsi_mode_select_page()
The function didn't do anything.

Change-Id: Id44a7d0c129ab60751eda382911935d677730ec9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-26 04:14:59 +00:00
Konrad Sztyber
fb15434659 lib/ftl: store non-volatile cache's current write address
Remember current write position within the cache when shutting down.  It
allows for faster recovery after clean shutdown (no need to scan the
device) as well as grants a quick way to distinguish between clean and
dirty shutdowns.

Change-Id: I79c22caa0b1ca4373951ac43f747b085d331cdd0
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460796
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
2019-07-25 22:36:31 +00:00
Konrad Sztyber
3e911e17c7 lib/ftl: non-volatile cache's header validation function
Moved metadata header verification to a separate function, since the
metadata read callback got pretty large.

Change-Id: I0c8b2eb493631a6495ce2ef7d9586d2c7cb7772f
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460795
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-25 22:36:31 +00:00
Changpeng Liu
28439890e4 nvmf: always update discovery log page if the offset is zero
Global tgt->discovery_log_page may contain old hostnqn log
page, so we will update the discovery log page if the offset
is zero.

Change-Id: Iba24409b16626d157d2782c6813fe5a0c27f1082
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463123
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <shahar.salzman@kaminario.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-25 18:05:13 +00:00
Hailiang Wang
563b98223b lib/compress: fix a compilation error for low GCC version
Some low GCC(such as v4.8.5) can't recognize this style:
static struct rte_comp_xform g_comp_xform = (struct rte_comp_xform)
Fix issue #877.

Change-Id: I021c62b7b1437d67ce87cc2e13d1b90ea6a18140
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463126
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-25 15:49:24 +00:00
Tomasz Kulasek
9307ff5a54 net/vpp: move to VPP 19.04
This patch updates net/vpp implementation from version VPP 19.01 to
VPP 19.04.

1. Some binary APIs are deprecated in 19.04 and message queue is used
   to handle control events:

   - vl_api_bind_sock_reply_t_handler by SESSION_CTRL_EVT_BOUND,
   - vl_api_unbind_sock_reply_t_handler by SESSION_CTRL_EVT_UNLISTEN_REPLY,
   - vl_api_accept_session_t_handler by SESSION_CTRL_EVT_ACCEPTED,
   - vl_api_connect_session_reply_t_handler by SESSION_CTRL_EVT_CONNECTED,
   - vl_api_disconnect_session_t_handler by SESSION_CTRL_EVT_DISCONNECTED,
   - vl_api_reset_session_t_handler by SESSION_CTRL_EVT_RESET

2. Fixes for Fedora 29/30:

   - added "-Wno-address-of-packed-member" (DPDK 19.02 fails to compile
     with gcc9.1),
   - force "-maes" compile flag for gcc9.1 to compile crypto_ia32 and
     crypto_ipsecmb plugins (gcc9.1 doesn't do that for -march=silvermont)
   - some minor fixes

3. Default path for VPP instalation is changed for test scripts from
   /usr/local/src/vpp to /usr/local/src/vpp-19.04 to avoid VPP version
   conflict.

Change-Id: I1d20ad7f138f5086ba7fab41d77d86f8139d038e
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459113
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-25 14:51:21 +00:00
Tomasz Kulasek
bee1130c03 net/vpp: prepare vpp 19.01 to move to 19.04
No functional changes done, just reordering for next patch in series.

Change-Id: I32022fddffbc7b00eeecb29a2fcbaa85fbf2e99f
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462473
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-25 14:51:21 +00:00
Wojciech Malikowski
093bc3390a lib/ftl: Wait for IO retry queue drain during shutdown
In case shutdown occurs during band relocation
we can have situation that on retry queue are
some internal IOs and we need to wait to drain
this queue before we start shutdown sequence.

Change-Id: Ie5b33732568aeda0586f4de384b22dcf463bf31e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460421
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-25 14:50:58 +00:00
Mateusz Kozlowski
4d9faf7c6b lib/ftl: Add check if device is halted during reloc
It's possible there's a delay between spdk_ftl_dev_free setting
dev->halt and sending message to core thread which would set
reloc->halt. This patch should bridge this gap and prevent potential
additional/unneeded relocations from happening.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I3d69fe86900c9233fadd8e3795accb36b1c01791
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461598
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-25 14:50:58 +00:00
paul luse
7384ca015b lib/reduce: for compress path, copy data from host buffers if needed
If the data was not compressed for whatever reason, it needs to be
copied from the host buffers much in the same way as the patch before
this did.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5c92db16afbce3ca573c556ee78a51ba6511c564
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462895
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-25 14:50:25 +00:00
paul luse
907ec7268d lib/reduce: conditionally memcpy data to host follow decompression
If the decompression code path runs into a chunk that wasn't
compressed, the data needs to be copied to the host as the
compression engine was not engaged.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ibabd6d6bd2bc5db79953a0a55d7f49d556c08af7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462892
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-25 14:50:25 +00:00
paul luse
0242a80793 lib/reduce: fix problem with size of chunk map area in PMEM
This fixes a segfault found by using a very small logical volume.
The PMEM file chunk map area wasn't including the chunk map struct
itself.  In PMEM, a single entry as returned by _reduce_vol_get_chunk_map()
is made up of a 'struct spdk_reduce_chunk_map' plus one uint64_t for
each backing IO unit for the volume (# depends on vol parms). So when
an app got close to the end of the chunk map indices it would run off
the end of PMEM and segfault.

With this fix that no longer happens however data integrity issues now
show up with fio. I believe this fix is valid though and am pursuing the data
integrity issue as something new.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I69a223064c7bbd44bcd0a62d155cf409603970a3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461316
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-25 14:50:25 +00:00
Karol Latecki
5f96e92c8c bdev/pmem: add more descriptive rpc error messages
Improve error messages where possible.
Also change return code values to negative in bdev_pmem.c so that
it follows the same pattern as other bdev modules.

Change-Id: I81b205a41104e15457154d1e31540210636ef9d3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461548
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:28:46 +00:00
Karol Latecki
0fc5acea8c bdev/nvme: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: Ief04cf32f7a65ef94f3d994ca76da2637ba66ca1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:27:44 +00:00
Mateusz Kozlowski
cd115ecfcd bdev/ftl: Add defrag parameters to construct_ftl_bdev rpc
Added parsing and saving of (optional) parameters which are responsible for
relocation in FTL. Changing the parameters may affect Write
Amplification Factor and overall performance, especially during random
write workloads. If parameters are not specified, default values will be used.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic0229e39109460f3541c31e1cbf2a485efe408c6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460504
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
7a867c4fdb lib/ftl: Pass ftl conf to write_config rpc function
Changed write_config to use the ftl configuration file instead of
allow_open_bands directly. This allows for easier expansion of saved
fields.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ie1d480983636f26840dba9a3a6c93b2883903bdb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460519
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
a039d024ff lib/ftl: Remove anonymous defrag structure in config
Merged configuration fields for defrag into the main config structure.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I315c78e2843bd6a310b484af5b688aca424336b0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460518
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
5a16b8e671 lib/ftl: Segmentation fault fix after write error handling
Fixed touching NULL write pointer, which would happen after some other
I/O in the same band would receive error and issue its deletion. Any
subsequent write callbacks would then be unable to find and update the
information.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I07d9791f1b1b673a48356fae55b784c0bf6a6f7a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-24 18:20:48 +00:00
paul luse
eb1ddcb664 lib/reduce: remove dead code clearing unused backing io units
At the start of time all of values of io_unit_index[] array in
the chunk maps are FF's.  When a partial chunk is written, only
some of the io_unit_index[] vals will be assigned values from the
allocated_backing_io_units bit array.  When that chunk is written
again, it is not done so in place and at that time in when
_write_write_done() is called, the io_unit_index[] values are
set to FF again and the bit array bits are cleared for the old
chunk so that it is fresh again for use by another write.

This change removes code from an early stage of the write process
where a 'fresh' chuck in allocated.  Because of points made earlier,
it is not possible for any of the io_unit_index[] values to be
assigned or have a corresponding bit set in the bit array.

The prev version of this patch cleared the bit array because the
io_unit_index[] array was being cleared however after further
investigation, and for the reasons stated earlier, the io_unit_index[]
did not have a valid value so clearing the bit array was simply wrong.

Removing this dead code clears things up and leaves it to one place
to clear the io_unit_index[], when a chunk is overwritten.  Well, 2
places including init time.

Change-Id: I5c4e5360c9c13817d2fa7d3c7d004b44537af7f4
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458520
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:12:05 +00:00
Tomasz Zawadzki
8df52a0f4a lib/nvme_tcp: assert tcp_req->req before it is dereferenced
The value of tcp_req->req was asserted after it was already
dereferenced. This patch fixes that.

Change-Id: I5eb01e88be09d41fb8e632c49d5a7ccf2315788f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462508
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 18:09:33 +00:00
Tomasz Zawadzki
f75d1540e9 lib/ftl: check namespace for NULL during initialization
spdk_nvme_ctrlr_get_ns() can return NULL, which was
not verified, but dereferenced anyway.

Change-Id: Ib0554148cc47ea3ff2afa1862ac962f284fe6d43
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462486
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:09:29 +00:00
Tomasz Zawadzki
eb84a1133f lib/ftl: assert write pointer from band
This is just assert to assure the Klocwork
static analizer that wptr is not NULL.

Change-Id: I003620431afb8497a57c50af8480c34ec3fac6d6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462489
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:09:25 +00:00
Changpeng Liu
3fe300609e nvmf: check HOSTNQN access right for discovery service
Initiator can use `nvme discover` command to display all
the subsystem's information, because we don't check
the allowed HOSTNQN for Discovery service, so here
adding this feature so that only return the log pages
to the allowed hosts.

Fix issue #576.

Change-Id: I51e6770bd67ea0b41caf9de3a8899923377e6255
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-07-24 11:25:59 +00:00
Changpeng Liu
234eb48bf6 nvmf: save hostnqn to controller data structure
When creating a new controller in the NVMe-oF target, hostnqn is
a must parameter, so we save the hostnqn to controller data
structure, and it can be used to verify the access right of
Discovery service.

Change-Id: I86a6f50d3209d5bbb8ac85508288173d826ea216
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462439
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-07-24 11:25:59 +00:00
Alexey Marchuk
f0b7a6e7d1 rdma: fix possible double free on qpair destruction
Update rqpair->last_wqe_reached in the context of thread that owns qpair's poll group to avoid possible double free
This patch fixes #858

Change-Id: If5422944b7928c2cc05af528fbcc4482aeef22df
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:56:57 +00:00
Darek Stojaczyk
a7ff5ff558 env_dpdk/memory: silence scan-build errors
Scan-build complains that spdk_vtophys_notify() can segfault
on vaddr == 0. We know that and it's not a programming error.
In fact, SPDK with UIO pci driver can segfault upon registering
any non-mapped memory address. The user is just not supposed
to do that.

Assert vaddr != 0 to silence scan-build.

Change-Id: I9a14696361eca0aeea5ede9f9f2956fcbf59bdb5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462869
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:56:23 +00:00
Darek Stojaczyk
bb63fe6fc3 blobstore: don't realloc any memory under scan-build
Scan-build has a real issue with reallocs. The original
error from latest version of scan-build is rather complicated,
but it can be greatly simplified with the following change:

> diff --git a/lib/blob/blobstore.c b/lib/blob/blobstore.c
> index 7580c9dd2..6a594edf3 100644
> --- a/lib/blob/blobstore.c
> +++ b/lib/blob/blobstore.c
> @@ -1147,8 +1147,9 @@
> _spdk_blob_persist_clear_clusters_cpl(spdk_bs_sequence_t *seq, void *cb_arg, int
>         } else if (blob->active.num_clusters != blob->active.cluster_array_size) {
>                 tmp = realloc(blob->active.clusters, sizeof(uint64_t) * blob->active.num_clusters);
>                 assert(tmp != NULL);
> -               blob->active.clusters = tmp;
> -               blob->active.cluster_array_size = blob->active.num_clusters;
> +               ctx->blob->active.clusters = tmp;
> +               assert(ctx->blob->active.clusters[0] != 14213);
> +               ctx->blob->active.cluster_array_size = ctx->blob->active.num_clusters;
>         }
>
>         _spdk_blob_persist_complete(seq, ctx, bserrno);
> ```

Scan-build will then complain:

blobstore.c:1151:10: warning: Use of memory after it is freed
                assert(ctx->blob->active.clusters[0] != 14213);

Asserting blob == ctx->blob, blob->active.clusters == ctx->...,
or even tmp != blob->active.clusters doesn't work, so use the
last resort scan-build weapon - #ifdef __clang_analyzer__.

The realloc in this case is just down-sizing a buffer to
save some memory. For scan-build, just don't do it. This
finally silences all scan-build false positives.

Change-Id: Ib88ea145370f5035eedd2412e98ee61f96ad1915
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:56:23 +00:00
Alexey Marchuk
5282edfd7b rdma: fix double free of qpair struct in case of failed initialization
qpair structure is freed and an error code is returned to the caller in the case of failed qpair initialization in function spdk_nvmf_rdma_qpair_initialize (e.g. bad return value of rdma_create_qp).
The return code is handled by nvmf_tgt_poll_group_add function which destroys the qpair for the second time.
This patch fixes #857

Change-Id: I0773652ecccbbd634ad272106e0a93c1e591d7d2
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462011
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:55:43 +00:00
Tomasz Zawadzki
06c7ef7f6a lib/ftl: check for io not being NULL before dereferencing
ftl_io_free() handles NULL io being passed, but shouldn't
dereference it before returning.

Change-Id: Ia705d4ab9f8b3569e9704a1f97e70f7579084efb
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 19:55:30 +00:00
Tomasz Zawadzki
8500db5ee0 lib/vmd: check header_type after the vmd_adapter structure is verified
First check both vmd_pci_device for not being a NULL,
same as vmd_adapter structure being present.

Otherwise dev was dereferenced before checking for NULL.

Change-Id: I3719f33c1762f789b0eb63b0f2ddce8258843ded
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 19:55:04 +00:00
lorneli
ba323d44ca nvmf/rdma: log spdk_nvmf_rdma_destroy_defunct_qpair
Func spdk_nvmf_rdma_destroy_defunct_qpair is a "last chance option"
to destroy qp manually if some driver/hardware doesn't drain qp's
failed wr as expected.

There's a probability that ibv_poll_cq polls wr of the destoryed qp
after spdk_nvmf_rdma_destroy_defunct_qpair's execution. Although in
practice the risk of this situation is minimal(if not non-existent),
add a log here so that we could detect this situation easily.

Change-Id: Ifa9534397513bcea34c18fbb8168eef8f53599c1
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462441
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 19:35:16 +00:00
lorneli
b4d3066890 nvmf/rdma: defer qp destruction until nvmf layer closes qp
Currently rqpair will be destroyed directly in ibv_poll_cq path
if it has been drained, regardless of whether there are outstanding
I/Os issued to bdev layer. So after outstanding I/Os completing,
spdk_nvmf_rdma_close_qpair will be called from nvmf layer, accessing
a destroyed qp.

This path defers qp destruction in nvmf_rdma_destroy_drained_qpair
func until nvmf layer closes qp.

Fixes 851

Change-Id: I8bcce66f8053ddb105702ac603d5d73af54bdcfc
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461237
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 19:35:16 +00:00
Mateusz Kozlowski
a3c78e150b lib/ftl: Remove separate dma_buf allocation in band
Since 4k alignment is no longer required for I/O buffers, the
band doesn't need a separate lba map and dma buffer and can use the
same memory location.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Iea127e8c2f39e6de5d57258098b2dc6be56f439f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462042
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-23 04:00:05 +00:00
Mateusz Kozlowski
a4b3481242 lib/ftl: Remove buffer alignment requirements
Ftl should no longer require 4k alignment for I/O buffers for either
internal or external requests. The 4k alignment was needed due
to a bug in QEMU when handling internal SGL buffers in LNVM commands.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Icafc14b7811c9c0ffa13789e341c453503cf2821
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-07-23 04:00:05 +00:00
Mateusz Kozlowski
1897943987 env: define minimum alignment for dma-able memory
DPDK defines the minimum alignment as "suitable for any
kind of variable (in the same manner as malloc())", but
internally the alignment is always rounded up to the
cache line size, even if the requested alignment is 0.

We would like to start relying on this behavior in FTL,
where lba maps are allocated using DMA-able memory and
are constantly looked up or modified by different threads.
By having the lba maps unaligned, we risk having those
threads pollute each other's cache lines.

Rather than enforcing this memory alignment in FTL, we
do it in spdk_*malloc directly. In general it makes
sense to have DMA-able memory always cache-line-size
aligned for the same reason as above.

Change-Id: Ib6edda4a7bf3f4952eb1875a4e1753be96bed642
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460329
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 04:00:05 +00:00
x00267701
a50246fb64 log: optimize spdk_log() when logging is disabled
If the log level of the output log is higher than
the process's log level, the system does not output
it, so we needn't generate the formatting logs.

Change-Id: I36be0e6807ed575fcbf1d0ae01f064a6ca2c4539
Signed-off-by: Huiming Xie <xiehuiming@huawei.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462790
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 00:42:24 +00:00
yidong0635
c0d4f7145e crypto: Fix scanbuild issue on Fedora30.
Issue reports:
vbdev_crypto.c:695:12: warning: The left operand of '+' is a garbage value
                iv_ptr = rte_crypto_op_ctod_offset(crypto_ops[crypto_index], uint8_t *,

Change-Id: I1a4f89111e94a13d8daf38070aa69306783a83c5
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 00:38:27 +00:00
yidong0635
425b89a4ed compress: Fix scanbuild error about dereference of null pointer.
Issue reports:
vbdev_compress.c:468:34: warning: Dereference of null pointer
                 src_mbufs[iov_index]->userdata = reduce_cb_arg;
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
Add assert here is enough.

Change-Id: I8dccf107a89f7af46437a1f678adccf122beb4b2
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462592
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 00:17:25 +00:00
Wojciech Malikowski
4c5282bcf7 lib/vmd: Clear prefetch base and limit registers
During PCI bridge initialization not all
registers was cleaned properly.

Change-Id: Ie0752bb303ba06e2a21669908efd9f746f6b0039
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460197
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-22 20:02:54 +00:00
yidong0635
a1de2fe007 compress: Clear some simple scanbuild errors.
On Fedora30, GCC9+ reports errors:
Issues report:
vbdev_compress.c:482:17: warning: Assigned value is garbage or undefined
        comp_op->m_src = src_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:502:17: warning: Assigned value is garbage or undefined
        comp_op->m_dst = dst_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:491:3: warning: 1st function call argument is an uninitialized value
                rte_pktmbuf_attach_extbuf(dst_mbufs[iov_index],

compress_ut.c:826:2: warning: Attempt to free released memory
        free(g_mbuf_mp);
        ^~~~~~~~~~~~~~~
Should remove free(g_mbuf_mp), for pointer exchange.

Change-Id: I1a48b7c309a4e2aa0aa513f9be44ae3504d9e385
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462574
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 14:10:32 +00:00
Alexey Marchuk
0754417fa9 rdma: Use optimal ceiling integer division
This form of the celinig division allows to remove an extra condition

Change-Id: I8a2de792172ec9115563e7fb914745c476f16e8d
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462198
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 09:22:11 +00:00
Pawel Kaminski
de1d0f8fe9 jsonrpc: don't check returned value from spdk_jsonrpc_begin_result()
As spdk_jsonrpc_begin_result() is not allowed to return NULL we can
remove these checks. We didn't have any tests cases that goes this path
anyway.

Change-Id: I0894e76c0162591e550e70b172566b9060a6dd5f
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 05:13:30 +00:00
Tomasz Zawadzki
37acf6814b lib/ftl: assert that chunk is never NULL
Code on line 613 should never be reached for last
chunk on the band, since it is checked on line 606.

This assert verifies that, all chunks before last
are not NULL.

Change-Id: I78a9967f91c0873981b3308e888d85fa7c8d6641
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462488
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 04:53:05 +00:00