nvme_rdma_req_put() is an internal nvme_rdma.c function, and all of the
callers already have the rqpair, so pass it directly. We also already
verify that all of the callers have a valid rqpair and req before
calling nvme_rdma_req_put(), so it doesn't need to check for NULL
pointers.
This also means that spdk_nvme_rdma_req doesn't need to hold a pointer
to its rqpair anymore.
Change-Id: I893a46a9074f0a843e379d10c123f9292eb3b1a4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The only place where outstanding_reqs was checked was in
nvme_rdma_req_put(), but the error case there could only happen if some
kind of internal programming error occurred (e.g. calling
nvme_rdma_req_put() on an invalid request).
Change-Id: I71e40ce562a8720dfaf70437ffd4c6493327c091
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvme_rdma_ibv_send_wr_init() was only called in one place, so just move
its contents into nvme_rdma_qpair_submit_request() since it allows
simplification of the code:
- req was always NULL, so remove the code that used req entirely.
- wr and sg_list are never NULL, so remove the checks for those.
Change-Id: I12a4f3502219d3681607686945e343f6808c0d2f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
We currently don't handle discovery service referrals, so skip those, as
well as any other unknown subsystem type.
Change-Id: I64f889e9272fb57b5cf9bb5467b3abca3955baf5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
QEMU's virtual NVMe controller device does not support the AER Set
Feature, so ignore its failure and continue.
Change-Id: I8b5c217a3112edabb6f76ec3e5f4ef774981a1d7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Catch SIGBUS and handle it by remapping new memory into the
location where the BAR previously was.
Change-Id: Ie8d00a60a0bbe7f7ec57a5c39c0a63c5d9443206
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
These functions will attach or detach from a PCI device. Attaching
typically means mapping the BAR.
Change-Id: Iaaf59010b8a0366d32ec80bb90c1c277ada7cfe7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
spdk_nvme_probe frees ctrlr when nvme_ctrlr_process_init is failed. But
ctrlr has already been freed while calling nvme_ctrlr_destruct. So
spdk_nvme_probe doen't need to free ctrlr.
The generic NVMe library controller initialization process already
handles enabling the controller; the RDMA transport should not need to
set EN itself.
For now, the discovery controller is cheating and not using the normal
initialization process, so move the EN = 1 hack to the discovery
controller bringup until it is overhauled to use the full
nvme_ctrlr_process_init() path.
The previous code where CC.EN was set to 1 before going through the
controller init process would cause an EN = 1 to EN = 0 transition,
which triggers a controller level reset.
This change stops us from causing a reset during the controller
startup sequence, which is defined by the NVMe over Fabrics spec as
terminating the host/controller association (breaking the connection).
Our NVMe over Fabrics target does not yet implement this correctly, but
we should still do the right thing in preparation for a full reset
implementation.
This patch also reverts the NVMe over Fabrics target reset
handling hack that was added as part of the NVMe over Fabrics host
commit to its previous state of just printing an error message.
Change-Id: I0aedd73dfd2dd1168e7b13b79575cc387737d4f0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Most of the NOTICE level messages should have been TRACE.
Change-Id: Icbc4d398ab2580cf3a2349be11441b7a09603020
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Verify that qpair is not NULL before doing pointer math on it.
The NULL check after calling nvme_rdma_qpair(qpair) would not
trigger if qpair was NULL.
Fixes a crash if the Connect command failed, causing
nvme_rdma_ctrlr_create_qpair() to return NULL.
Change-Id: I158a5b1752892a7d5a72a9ac20c0c5b2cd781a81
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The status.done flag polled by nvme_ctrlr_set_keep_alive_timeout()
was never initialized.
Change-Id: I323fae5f4ce12209a9699965ce07894bc3c6205a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Use the NVMe over Fabrics spec definitions for TRTYPE rather than the
internal library transport type.
Change-Id: Idead559a8f8d95274fc580d10e82033822e6eda8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
These need to be available for the lifetime of the probe_info structure,
so they can't be pointing at e.g. temporary buffers on the stack.
Change-Id: I5aaa898acf9314aab51600dd756f966965d37fd0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
It always points to the same internal RDMA request complete function, so
just call that function directly.
Change-Id: Ic1fb6236bf43eaad62413df77d43be9ab855e5c7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
We can't transfer more than the bounce buffer in a single command, so
report that rather than some bogus value.
Change-Id: I39b147916dcc2ee478470917298763a239a6a35a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Record the user-provided asynchronous event configuration set via Set
Features, and return it in Get Features.
This value is not actually used, since AER is not implemented yet in the
virtual controller model, but it at least implements the mandatory
Set/Get Features.
This allows the hack in the NVMe host code that ignored the Set Features
failure to be reverted.
Change-Id: I2ac639eb8b069ef8e87230a21fa77225f32aedde
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Fill in the cached copy of CAP in the generic NVMe controller to match
the PCIe transport.
This is not really early enough, since CAP is used during the reset
process to determine the reset timeout, but that will have to be fixed
separately by rearranging some of the transport callbacks.
Change-Id: Ia8e20dbb8f21c2871afb9e00db56d0730e597331
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Make sure the entire NQN field is zero-padded, rather than using
strlen() on the input.
Change-Id: Icee68bd033feed057813beeb30cec102ed90840e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This fixes a compiler warning about unhandled enum cases in a switch.
Change-Id: Icecb56b47a05c13f390f03b877f8eae243b481a6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
- add SPDK_NVME_OPC_KEEP_ALIVE to admin_opcode
- add SPDK_NVME_SC_INVALID_SGL_OFFSET, SPDK_NVME_SC_INVALID_SGL_OFFSET,
SPDK_NVME_SC_HOSTID_INCONSISTENT_FORMAT, SPDK_NVME_SC_KEEP_ALIVE_EXPIRED
and SPDK_NVME_SC_KEEP_ALIVE_INVALID to generic_status
The SPDK_TRACELOG macro depends on a CONFIG setting (DEBUG), so it
should not be part of the public API.
Create a new include/spdk_internal directory for headers that should
only be used within SPDK, not exported for public use.
Change-Id: I39b90ce57da3270e735ba32210c4b3a3468c460b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Considering the process can be terminated in the cases like ctrl+c,
kill command or memory fault, the ref is tracked in the per process
structure spdk_nvme_controller_process and whenever there is other
process attaches or detaches the controller, a scan will be issued
to cleanup those unexpectedly exited processes.
Change-Id: Ib4f974f567a865748d42da4ead49edd383dfc752
Signed-off-by: GangCao <gang.cao@intel.com>
Instead of the next_sge callback returning the physical address
directly, make it return the virtual address and convert to physical
address inside the NVMe library.
This is necessary for NVMe over Fabrics host support, since the RDMA
userspace API requires virtual addresses rather than physical addresses.
It is also more consistent with the normal non-SGL NVMe functions that
already take virtual addresses.
Change-Id: I79a7af64ead987535f6bf3057b2b22aef3171c5b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Function pointers will not work for the DPDK multi-process model (they
can have different addresses in different processes), so define a
transport enum and dispatch functions that switch on the transport type
instead.
Change-Id: Ic16866786eba5e523ce533e56e7a5c92672eb2a5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The initialization of dev_addr was replaced with probe_info.pci_addr,
but its use in spdk_pci_addr_compare() wasn't replaced to match.
Fixes commit fcb00f37807a39d58bc29de3055008f7169fb91e (nvme: expand
probe information to a struct).
Change-Id: Ic4c273d2aa0bf1f9e3e1527f3ab09d3c019158cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
spdk_nvme_probe() will now provide a struct spdk_nvme_probe_info to the
probe and attach callbacks in place of the PCI device pointer.
This struct contains the useful information that could be retrieved from
the PCI device during probe.
The goal of this change is to allow expansion of the probe information
in the future when other transports (specifically, NVMe over Fabrics)
are added that do not necessarily use PCI addressing or device IDs.
Change-Id: I59a2a9e874e248ce5fa1d7f4b57c8056962ff3cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Use the new public PCI ID structure in the NVMe library to replace the
previously private struct pci_id.
Change-Id: I267d343917f60bdae949a824bc0fe67457cbbc0d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
- Split the part that gets a PCI device's address into its own function,
spdk_pci_device_get_addr(). This is useful outside of the comparison
function and is orthogonal to comparing addresses.
- Make the comparison function take two addresses instead of a device
and an address. The more general form will be useful with addresses
that are not directly associated with a device. Because of this, also
rename the function from spdk_pci_device_compare_addr() to
spdk_pci_addr_compare().
- Return a signed value similar to strcmp() so that addresses can be
ordered, not just compared for equality.
Change-Id: Idf304454af09ea57f1e1d5dc3a39b077378cecad
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
