729 Commits

Author SHA1 Message Date
Ben Walker
a9b9f0952d nvmf/rdma: Don't trigger error recovery on IBV_EVENT_SQ_DRAINED
IBV_EVENT_SQ_DRAINED can occur during both error recovery and
normal operation. We don't want to spend time sending a message
to the correct qpair thread and then attempting to abort
all I/O in the case where this wasn't triggered by an error.

The case where this occurs during an error is very rare and
only in response to a user forcing the state to err from the
sqd state. For now, don't handle that case at all. Handle that
corner case in a later patch.

Change-Id: I16462ca52739b68f6b52a963f7344e12f7f48a55
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/420936
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2018-08-06 16:23:36 +00:00
Ben Walker
13a887f1e2 nvmf/rdma: Simplify spdk_nvmf_rdma_qp_drained
This was the only call point of two very small static functions,
so merge them into the main body.

Change-Id: Ifdd3355ffd500ac5ad4fcf69feace65b35132906
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/420935
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-06 16:23:36 +00:00
Ben Walker
c3756ae387 nvmf: Eliminate spdk_nvmf_rdma_update_ibv_qp
The update call was never used independently of the get
call, so combine them

Change-Id: Ibae622e5fd23203e79ceeae1aeccc5c7d9d1ebc0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/420934
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2018-08-06 16:23:36 +00:00
Ben Walker
1cfff49fe9 nvmf/rdma: Fix formatting of spdk_nvmf_rdma_request_set_state
Change-Id: Id6fb8a9f02a00f3a8e03f621b74f7505c549a345
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421040
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-03 06:50:41 +00:00
Ziye Yang
4c4cba9a95 nvmf: simplify the qpair_mask handling.
We should not use mutex, but use the spdk_send_msg policy,
then we can let only one thread to handle that and
eliminates the segement fault issue.

Now in the code, the qpair_mask is handled by the same
thread, e.g., the thread which owns the admin qpair of
the ctrlr.

Change-Id: I609fd4d49f5ecc85bc47bf9c23afbb507900be7c
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420827
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-03 03:38:34 +00:00
Ben Walker
6779479067 nvmf: Simplify spdk_nvmf_qpair_disconnect
Asking which thread we're currently on is more expensive
than sending a message.

Change-Id: I9d9007c9f7f30e4cdd9a97de6bf7a10b0e2a0594
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/420933
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-01 17:13:48 +00:00
Seth Howell
b0171f79c3 nvmf_tgt: delete connections accepted during shutdown
With the reordering of the nvmf_tgt states, we need to remove any
connections accepted during the shutdown pahse of the target.

Change-Id: I768484366da8273df74b8d52a3e8de6158b6995f
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/420681
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 16:14:29 +00:00
Seth Howell
e5a6540777 nvmf: disconnect qpairs before freeing i/o channel
Previously, qpair deletion was synchronous and handled by the
io_channel_destroy_cb for the target. However, with the new asynchronous
qpair deletion api, these qpairs need to be completely removed before we
free the i/o channel and the poll group.

Change-Id: I42c62391df62825d53e158306c4372523403ad27
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/420208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 16:14:29 +00:00
Seth Howell
e4c1e5f866 nvmf: destroy_poll_group uses disconnect_qpair asynch api
Change-Id: I47eff0db1ab33be23881f694d104e903706f1c28
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/417371
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 16:14:29 +00:00
Seth Howell
f2b22d68d6 subsystem: defer channel iter until pg functions return
The poll group pause, resume, remove, and add functions are only called
from the subsystem_state_change_on_pg function. Previously, they would
return immediately and the state change would move on to the next
channel. However, some of these functions (specifically remove) kick off
asynchronous APIs and we should not iterate past them until those
asynchronous operations complete.

Change-Id: I78804273b39f2d171ba26ac4478ad515356833f3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419289
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 16:14:29 +00:00
Seth Howell
d3995f6eca nvmf: remove_subsystem now uses qpair_remove asynch api
This is necessary to avoid race conditions when freeing subsystems.

Change-Id: I9b4a7d006cc42cd29e13179e940ced0cc580f548
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/417351
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-27 20:50:36 +00:00
Seth Howell
1e2c9afa95 nvmf: always call qpair_delete cb on original thread
This ensures that when we continue to iterate through channels after
deleting the qpair, we will be able to continue iterating through
channels.

Change-Id: I6fba43dc14a3e5e8faac78f8b37e9e0c6aad2687
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-27 20:50:36 +00:00
Seth Howell
4bee4e03b6 nvmf: free AER resourcess before disconnecting qpair
It is necessary to free the AER without sending a completion to ensure
that the host does not attempt to send an additional AER upon receiving
the first completion.

Change-Id: I2b3f8f286d6396019d8ace97d2376547705b8d9d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/420661
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-27 20:50:36 +00:00
Seth Howell
388e310150 nvmf: add free_req function pointer.
At times, it may be necessary to free requests without completing them.
For example, when freeing a qpair, one needs to free the AER sent from
the host before deleting the qpair. It is important not to send a
completion for the AER because:
1. According to the spec, this will trigger the host to send another AER
2. No Asynchronous Events have occured, so we should not complete the
AER.

Change-Id: I92e163f0fed0ee2bc942569a647cb3c1967edec9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419732
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-27 20:50:36 +00:00
Seth Howell
529f7f3b8e Revert "nvmf: ensure that destroying qpairs have a state_cb"
This reverts commit 238bc1de8082a20b11bd60a231cf03f34f1a4c60.

Change-Id: If9041b33bb95582406518f73b601671fae11e984
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419731
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-18 21:22:43 +00:00
Philipp Skadorov
4bfb557d80 nvmf/rdma: recover qp from fatal errors
RDMA QP is attempted to recover after IBV_EVENT_QP_FATAL event
is received from IBV asynchronous event API.

RDMA QP is put into ERROR state and is not processing any inbound
requests. The outstanding requests are only allowed to COMPLETED
and FREE states, no outbound transfers are performed.

IBV_EVENT_QP_LAST_WQE_REACHED or IBV_EVENT_SQ_DRAINED event is
expected to follow IBV_EVENT_QP_FATAL, giving a go to draining of
all outstanding requests and freeing the associated resources.

The requests executed by block layer are gracefully allowed to
complete, but no outbound transfers are made.

Note, outstanding requests can not be reliably completed through
polling the CQ, as WC's with failure status might not have all
the fields valid. The failed WC's are dropped and the outstanding
requests are fetched from the appropriate state's linked list.

QP recovery is triggered when there is no more outstanding requests.
If QP recovery is completed succesfully, the RDMA QP is put back into
ACTIVE state, the QP disconnect is triggered otherwise.

Change-Id: I45ee7feea067f80ccc6402518990014d691fbda3
Signed-off-by: Philipp Skadorov <philipp.skadorov@wdc.com>
Reviewed-on: https://review.gerrithub.io/416879
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-07-18 20:58:50 +00:00
Seth Howell
a28a5c8ff7 nvmf: ensure that destroying qpairs have a state_cb
It appears that we can get caught in a loop when shutting down a
subsystem if there is a qpair that has been moved to the deactivating
state due to the target shutting down early. These qpairs don't have a
state cb and won't ever be destroyed or removed from the global list.

Change-Id: I4f9ed774a94e0e2c7ff7bfa3af1776b38a787035
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419269
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-07-17 08:18:25 +00:00
Pawel Wodkowski
8a5a15b876 nvmf: fix coding style for astyle 2.05 in nvmf_fc.h
Remove unnecessary brackets to not confuse astyle.

Change-Id: Ieaacb5f7231bf8f77e2f8fd3429ae64bb2386626
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419230
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-17 07:53:18 +00:00
Philipp Skadorov
fdec444aa8 nvmf/rdma: track requests in any state
Requests that are being put into IBV context are lost when
IBV QP breaks and its SQ drains.

In order to track NVMf/RDMA requests, RDMA QP has been
reworked to track requests at any state with queues of
requests for each state.

This allowed to get rid of a few intermediate queues and
request counters.

A couple of states has been added to track outbound requests
with and without data. They will be used by QP recovery for
freeing resources assigned to outstanding requests.

Change-Id: Ie84207325c38e5bb2c247cd6dcddb82dfad0d503
Signed-off-by: Philipp Skadorov <philipp.skadorov@wdc.com>
Reviewed-on: https://review.gerrithub.io/416878
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-12 01:02:25 +00:00
John Barnard
6cfb803485 NVMF: Fibre Channel Transport API (follow-up)
Removed unneeded (and too generic) #define errors.

Change-Id: I26343504aaefb7e982d4dca35ffade8c70406f08
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/418881
Reviewed-by: John Meneghini <johnm@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-11 20:19:37 +00:00
Daniel Verkamp
5518a327a8 nvmf/rdma: fix error paths in spdk_nvmf_rdma_create
Most of the error paths in this function leaked resources.  Make them
all use spdk_nvmf_rdma_destroy() so all resources are consistently
freed.

The spdk_io_device_register() call is moved to the top of the function
so that the io_device is always valid when calling the destroy function.

Change-Id: Ic92f09f157ee8245fb962d8bc3330aadd87b294a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418869
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2018-07-11 16:04:43 +00:00
Daniel Verkamp
043e5edb1f nvmf/rdma: check for rdma_get_devices() failure
rdma_get_devices() may return NULL on failure; we need to check for this
before dereferencing the returned pointer.

Fixes GitHub issue #360.

Change-Id: I9628e5865365d256f4b1887bf07ce8737b55d356
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418868
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2018-07-10 23:53:57 +00:00
John Barnard
0e9f9bead9 NVMF: Fibre Channel Transport API
New header file for NVMF FC transport defining FC data, functions
and low level FC driver interface.

Change-Id: I3fd24e93cefa06647003eeb27d79166469fb4a05
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/415057
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-10 21:21:48 +00:00
Senthil Kumar V
6138d3bc72 nvmf: Allow In-Capsule data size to be 0.
Change-Id: I59f4f69ed695cc9a2b6d0b87052fdf50004ee1c7
Signed-off-by: Senthil Kumar V <senthil.kumar.veluswamy@wdc.com>
Reviewed-on: https://review.gerrithub.io/418170
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-09 22:24:40 +00:00
shahar salzman
a0246f6553 lib: validate ib_verbs context is valid before using it
Change-Id: I54793624e46a4e51b0c989ddfe933ccb5f035123
Signed-off-by: shahar salzman <shahar.salzman@kaminario.com>
Reviewed-on: https://review.gerrithub.io/417858
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-09 19:06:36 +00:00
Seth Howell
09e3f4e3db nvmf: give qpair_disconnect an asynchronous api.
qpair_disconnect has previously presented an entirely synchronous API.
However, it relies on other asynchronous operations to complete its
task. By giving it an asynchronous API, we can avoid possible race
conditions. Patch 1 of several.

Change-Id: If9e26ee70ae5d6c0273750226b4408a8e4587e19
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/417345
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 22:49:39 +00:00
Daniel Verkamp
d4ef57c072 nvme_spec: add VWC flush broadcast bits (TP 4035)
Also report that the NVMe-oF target does not support Flush with the
broadcast NSID.

Change-Id: Iad761a6f81eb099940463a67074a6dccb8871b70
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416445
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-26 22:51:37 +00:00
Daniel Verkamp
f519fbbc81 nvmf: fix SCT for fabrics command-specific errors
Some of the uses of SPDK_NVMF_FABRIC_SC_* were neglecting to set SCT to
indicate that these status codes are from the command-specific set.

Change-Id: Ief2a77e06d69e5925f8d8b1ec89ff2612250dcf0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416430
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-22 16:25:24 +00:00
Ben Walker
7d87273f60 nvmf: Reimplement abort handling
Now that the qpairs are stored on the poll groups,
reimplement abort handling to search each one. This
doesn't add new functionality compared to before - it
still only aborts AER commands.

Change-Id: I73651cf412b10f91a14be68bd9b8c03d8f22626e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/415318
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-20 22:07:24 +00:00
Ben Walker
65aa57b007 nvmf: Keep a list of qpairs per poll group
Remove the list of qpairs per controller.

Change-Id: I6aaeed43c64b38b60c5a609123b3542bb7130a90
Reviewed-on: https://review.gerrithub.io/415317
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-20 22:07:24 +00:00
Ben Walker
f80001e2c6 nvmf/rdma: Unset poll group pointer when qpair is removed
Change-Id: I2eb84490144c2e1f772c4094645e5067149d2862
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/415316
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-20 22:07:24 +00:00
Ben Walker
84523c0439 nvmf: Remove abort implementation
This will be re-added later in the series, implemented
in a different way.

Change-Id: I1f9df22025eb37f7f58ce1547862f61906ea557c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/415315
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-20 22:07:24 +00:00
Ben Walker
1a6d593345 nvmf: Only send message on qpair disconnect if on wrong thread
Change-Id: I5297c7a869402c96f69d705a13529c32296a9eaa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/415314
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-15 19:11:29 +00:00
Ben Walker
878185cf0e nvmf: Rename spdk_nvmf_ctrlr_disconnect to spdk_nvmf_qpair_disconnect
Change-Id: I0c6c410d120bec830ec17105de43ca62bf202b7b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/415313
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-15 19:11:29 +00:00
Ben Walker
a5279b6403 nvmf: Move spdk_nvmf_ctrlr_disconnect to nvmf.c
This is only code movement. Nothing changed. This is
preparing for a rewrite of the qpair disconnect path.

Change-Id: Ic782c090293a5b2e4baaba9c222479dbb2b64fec
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/415312
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-15 19:11:29 +00:00
Xiaodong Liu
5cd1b16a50 nvmf: redundant spdk_nvmf_transport assignment
Redundant with lib/nvmf/rdma.c spdk_nvmf_rdma_create,
line 1177 and 1178:
rtransport->transport.tgt = tgt;
rtransport->transport.ops = &spdk_nvmf_transport_rdma;

Change-Id: Id680c5537aa3df70d6f8c148aa8dc2726ccf8877
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/415203
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-14 17:44:27 +00:00
Pawel Wodkowski
f828b17c79 nvmf: fix hosts config dump
Change-Id: Ib15015d99e8f7a9b07cacada4f325be3f843d0ef
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/415278
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-14 17:10:24 +00:00
Pawel Wodkowski
7efc7b8302 nvmf: add JSON config dump
Change-Id: I8ed62b7da26a4d1f831dfa198c42ac12b379c92d
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/410271
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
2018-06-13 23:33:02 +00:00
Daniel Verkamp
eb387189c2 nvmf: don't change NN while ctrlrs exist
This was partially fixed in commit 1e481d0438e6 ("nvmf: Do not allow NN
to change while connections present"), but we did not handle the case
where the user asked to add a NS with a NSID outside the current NN.

This patch reworks the logic (again) to be more straightforward and
hopefully more obviously correct.

Some confusion between max_allowed_nsid and max_nsid is also clarified;
if max_allowed_nsid is set, then max_nsid == max_allowed_nsid at all
times, so we don't need the extra logic when calculating NN in
spdk_nvmf_ctrlr_identify_ctrlr().

Change-Id: If531baf1114e03441ff3e1e1be098071702d9056
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/414894
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-13 21:52:38 +00:00
Daniel Verkamp
6546fd10f8 nvmf/rpc: add max_namespaces to get_nvmf_subsystems
This is one of the construct_nvmf_subsystems parameters, so we need a
public API to retrieve it for informational purposes and for the
JSON-RPC configuration dump.

Change-Id: I8a7cafa487209311e964ea9ff1b8e09ee8b23c07
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/414864
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-13 16:43:38 +00:00
Ben Walker
6a5ae72b47 nvmf: Add trace points for the RDMA state machine
Remove the old trace points since they didn't actually
work. More trace points should be added in the future.

Change-Id: I1b658af8e309137882c31460723d7bb94d555b79
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414280
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-12 20:01:33 +00:00
Ben Walker
a83f91c29a thread: Replace #include of io_channel.h with thread.h
Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-12 15:24:07 +00:00
Ben Walker
8c9508f9b3 nvmf: Correctly unregister nvmf tgt as an I/O device
Change-Id: I9552354d620986c2a3d6b071f32b587e9e683198
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/413861
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-07 16:07:38 +00:00
Ben Walker
f4f839ecd3 nvmf: Eliminate num_qpairs counter in controller struct
The qpair mask can be used instead.

Change-Id: I9063e656c7c44663dc96adfd5ef52a5a1065d2ba
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/413860
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-06-07 16:07:38 +00:00
Ben Walker
f08cea7169 nvmf: Perform QID validation using a bit mask
Instead of scanning a list of all qpairs, use a bit
mask to determine if the requested QID is unique.

This is not for performance reasons, but because
eventually the ctrlr's list of qpairs is going to
need to go away.

Change-Id: Ic25ee60e4f9cd9d596815719760d5be892f29d0c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/413286
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-07 16:07:38 +00:00
Ben Walker
4d58c1328a nvmf: Move QID error checks to common path
QIDs need to be validated for both I/O queues and admin
queues, so move the checking to a common path.

Change-Id: Icb2009b3fe4d70f146b741c046812d9df62cc2fb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/413859
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-06 17:10:57 +00:00
Ben Walker
b95fc6fc64 nvmf: Return the correct error for out-of-range QID
This one is actually a bit tough to deduce in the specification.
The NVMe-oF spec says that QID errors detected in the RDMA
transport shall return an RDMA-specific error indicating
the problem. However, our code doesn't detect the error in
RDMA-specific code, and it isn't clear if the language is
a "must" or a "should".

The NVMe specification does clearly indicate what error
to return on invalid QID in response to a Create
I/O Queue Pair command. For now, return that while
we game plan whether we need to call into the RDMA
transport to correctly report this error.

Change-Id: I7faf37bad9b9202bc50a906214a51c17e4808fc0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/413858
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-06 17:10:57 +00:00
zkhatami88
0cdb08b0e0 env: add size parameter to spdk_mem_map_translate
Change-Id: I808101edaf4d75613baf19a950915f1d8e75b1af
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413154
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Zahra Khatami <zahra.k.khatami@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-05 18:36:00 +00:00
Philipp Skadorov
b6f90c527a nvmf/rdma: monitor asynchronous events
NVMf cnx acceptor poller is changed to check the asynchronous events
from the RDMA devices.

RDMA async events are polled together with RDMA CM events; the file
descriptors are combined into a poll fd array and processed in a single
poll syscall.

The errors handler is an empty placeholder for this patch, it just
prints the kind of event read from the IB device context.
The work for implementing event handling is left for later.

Signed-off-by: Philipp Skadorov <philipp.skadorov@wdc.com>
Change-Id: Ib167990651b585090aceef1404a88d431a910226
Reviewed-on: https://review.gerrithub.io/412540
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-04 17:28:04 +00:00
Ben Walker
72800826ec nvmf: Quiesce I/O before closing spdk_nvmf_qpairs
Change-Id: Iff58943f3f9db39b6fce1437a0cfe7380991daa6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/412078
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-04 17:20:34 +00:00