105 Commits

Author SHA1 Message Date
Ziye Yang
320c7fb825 nvme: drop Set Features of Keep Alive Timeout
The NVMe over Fabrics transports should already be setting this in the
initial admin queue Connect command, so setting it again is not useful.

The kernel NVMe over Fabrics target additionally has a bug in the Set
Features - Keep Alive Timeout handler (it is extracting the KATO value
from the wrong offset in the command), so this works around the kernel
bug by not sending the Set Features command at all.

Change-Id: I0d7f09b71fcea116acf8810c5880157bb9315a04
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-12-07 12:34:02 -07:00
GangCao
6515577822 nvme: check mutex init return code in ctrlr_construct
Change-Id: I0561f45f96ad6e88e7cdfec44c1f20677de2cb19
Signed-off-by: GangCao <gang.cao@intel.com>
2016-12-06 12:58:24 -07:00
GangCao
bb726d516b nvme: add multi-process support
This version of multi-process support needs to have DPDK 16.11 builtin.

Change-Id: I3352944516f327800b4bd640347afc6127d82ed4
Signed-off-by: GangCao <gang.cao@intel.com>
2016-12-02 12:24:22 -07:00
Daniel Verkamp
89136259ec nvme/rdma: add hostnqn to spdk_nvme_ctrlr_opts
Allow the host NQN to be overriden when connecting to NVMe over Fabrics
controllers.

Change-Id: I8fcf2e89ae7d9722677e834f76a8fe805c52f91b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-01 08:08:19 -07:00
Cunyin Chang
e8c63cdde0 nvme: Add support for hotplug.
Change-Id: Iac504ce15e4ea3100e5afa31764fcfff7f979dbb
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-12-01 10:21:59 +08:00
Cunyin Chang
51b15b93e7 nvme: Make sure the driver not exit when aer not supported.
QEMU's virtual NVMe controller device does not support the AER Set
Feature, so ignore its failure and continue.

Change-Id: I8b5c217a3112edabb6f76ec3e5f4ef774981a1d7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-11-30 16:37:46 -07:00
Tsuyoshi Uchida
b573c3a9ef nvme: fix keep alive interval ticks (#76) 2016-11-28 10:52:44 -07:00
Daniel Verkamp
bdf982302e nvme: add trace logs for initialization process
Change-Id: Iba26bf9264dc6c72d84ecba96787efe141ba53fc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-23 16:34:13 -07:00
Daniel Verkamp
97cc39e1ef nvme: clear status.done flag in keep-alive config
The status.done flag polled by nvme_ctrlr_set_keep_alive_timeout()
was never initialized.

Change-Id: I323fae5f4ce12209a9699965ce07894bc3c6205a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-23 10:28:25 -07:00
Changpeng Liu
277ca8ddd0 nvme: free associated memory when deleting the IO queue pair
Change-Id: Ibc139c5fd2e0ef72b6dbd5a8416aec916759430f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-11-23 10:27:44 -07:00
Daniel Verkamp
fd36d11e17 nvmf_tgt: stub out Async Event Config feature
Record the user-provided asynchronous event configuration set via Set
Features, and return it in Get Features.

This value is not actually used, since AER is not implemented yet in the
virtual controller model, but it at least implements the mandatory
Set/Get Features.

This allows the hack in the NVMe host code that ignored the Set Features
failure to be reverted.

Change-Id: I2ac639eb8b069ef8e87230a21fa77225f32aedde
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-21 17:02:12 -07:00
GangCao
9ec380ba90 nvme: check whether the process is already added at the probe phase
Change-Id: I556c0fd993998a291bff227365102b9985e6d7ec
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-21 12:35:47 -07:00
Ziye Yang
246c39a7ee nvme: Add nvme over fabrics support
Change-Id: I6f6259e77baa5dc5861f31ec4a9034e15297d333
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-11-18 13:58:44 -07:00
GangCao
af9eca84d6 nvme: rename struct spdk_nvme_controller_process
Change-Id: If8f29409b86281532eae6d87d056a2f02f6609d4
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-18 12:53:52 -07:00
GangCao
ec5b6fed61 nvme: add ref to track the shared usage of ctrlr among processes
Considering the process can be terminated in the cases like ctrl+c,
kill command or memory fault, the ref is tracked in the per process
structure spdk_nvme_controller_process and whenever there is other
process attaches or detaches the controller, a scan will be issued
to cleanup those unexpectedly exited processes.

Change-Id: Ib4f974f567a865748d42da4ead49edd383dfc752
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-16 10:27:45 -07:00
GangCao
84b7670dff nvme: use spdk_zmalloc for IO qpair creation
Change-Id: I1a9b324605069b5fc1a5a7a23e87933ad3b2b3ca
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-09 15:03:45 -07:00
Daniel Verkamp
1ffec5d53a nvme: convert transport type to an enum
Function pointers will not work for the DPDK multi-process model (they
can have different addresses in different processes), so define a
transport enum and dispatch functions that switch on the transport type
instead.

Change-Id: Ic16866786eba5e523ce533e56e7a5c92672eb2a5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-07 10:42:18 -07:00
Changpeng Liu
9f860cc4b4 nvme: replace trace log with error log for error conditions
Change-Id: I634573f8d9b5ed05d850960f8c45c10d96fa3a7a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-11-02 10:31:23 -07:00
GangCao
bfc8bc87fb nvme: add the per process admin cpl queue for multi-process case
Change-Id: Ie67e3414db807160092bb10812a586b7230e0a89
Signed-off-by: GangCao <gang.cao@intel.com>
2016-11-01 16:48:21 -07:00
Daniel Verkamp
ed1e7ca2d7 env: add struct spdk_pci_id and getter function
Use the new public PCI ID structure in the NVMe library to replace the
previously private struct pci_id.

Change-Id: I267d343917f60bdae949a824bc0fe67457cbbc0d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-01 09:40:36 -07:00
Daniel Verkamp
93de96b412 nvme: add Keep Alive Timeout feature support
Add a field to struct spdk_nvme_ctrlr_opts that allows the user to
specify a keep alive timeout, and add automatic submission of Keep Alive
commands to spdk_nvme_ctrlr_process_admin_completions().

Change-Id: Ib282299a571d8edc59c7933418751bc3a6c98b40
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-31 10:28:55 -07:00
Wenbo Wang
c257e5b4ad nvme: add quirk to delay checking device readiness (#56) 2016-10-28 11:26:31 -07:00
Wenbo Wang
8bf37ee769 nvme: change nvme_intel.c to nvme_quirks.c
Make the quirks mechanism generic in preparation for quirks for devices
from other vendors.

Change-Id: Ic003b020a38f1b966021db30e3f2bce9cf6a1a0d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-24 17:01:26 -07:00
Daniel Verkamp
bee15d8be0 nvme: move PCIe-specific definitions to nvme_pcie
Add a transport function to get the max data transfer size to break the
dependency on NVME_MAX_XFER_SIZE.

Change-Id: I846d12878bdd8b80903ca1b1b49b3bb8e2be98bb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-21 09:06:30 -07:00
Daniel Verkamp
e862c5a621 nvme: move adminq creation to transport
Change-Id: Ieea86c9fdd7b4bc541eed2762e885368871ee4b5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-21 09:06:29 -07:00
Daniel Verkamp
ff3869789a nvme: add transport ctrlr_enable function
Move the PCIe-specific admin queue setup to nvme_pcie_ctrlr_enable.

Change-Id: Ic3f5625fa804f719040ba86b7fc3bf82fcc057c0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-21 09:06:29 -07:00
Daniel Verkamp
76469b2cf7 nvme: cache CAP register in spdk_nvme_ctrlr
The value of CAP should not change during the lifetime of a controller,
so read it once during ctrlr_construct and store it in the ctrlr.

Change-Id: I089d4141b4e0c9aae6c53abf9bb0ef6577dabe0b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-20 12:56:45 -07:00
Daniel Verkamp
a987bd16c2 nvme: convert adminq to a qpair pointer
Rather than embedding adminq directly in the spdk_nvme_ctrlr structure,
change it to a pointer to a spdk_nvme_qpair.  This is necessary to allow
the transport to extend the qpair structure.

Change-Id: I041685d5037088cf56d046fe99bf204edcfc57b1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-20 12:56:45 -07:00
GangCao
5ba51e5016 nvme: init the recursive and shared ctrlr_lock for multi-process case
Change-Id: Id26f1da0b59045f000b9753e6b74a33dcab69197
Signed-off-by: GangCao <gang.cao@intel.com>
2016-10-20 12:28:42 -07:00
Daniel Verkamp
c194ebd833 nvme: move I/O qpair allocation to transport
This requires a couple of related changes:
- I/O queue IDs are now allocated by using a bit array of free queue IDs
  instead of keeping an array of pre-initialized qpair structures.
- The "create I/O qpair" function has been split into two: one to create
  the queue pair at startup, and one to reinitialize an existing qpair
  structure after a reset.

Change-Id: I4ff3bf79b40130044428516f233b07c839d1b548
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-19 08:09:45 -07:00
Daniel Verkamp
823958551b nvme: move ctrlr alloction to transport
Make the transport ctrlr_construct callback responsible for allocating
its own controller.

Change-Id: I5102ee233df23e27349410ed063cde8bfdce4c67
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-18 13:35:14 -07:00
Daniel Verkamp
a5790100f2 nvme: add ctrlr construct/destruct to transport
Change-Id: I66842497a02bdb586d38ddc4a38d5b444a9d5dad
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-18 10:13:37 -07:00
Daniel Verkamp
03aead3903 nvme: add qpair operations to transport
Change-Id: Id354ba13515d54bb54a8293569ee83ea72111183
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-18 10:13:37 -07:00
Daniel Verkamp
b0e9620b4f nvme: add qpair create and delete to transport
Change-Id: Ibc057972c7eb84ada95fb173d559255e5c86c5ba
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-14 12:46:58 -07:00
Daniel Verkamp
179f122cab nvme: add register access functions to transport
Change-Id: I2e80879e69770b42ea751a8ade7110ac9f4b6d13
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-14 12:46:58 -07:00
Daniel Verkamp
a00852c1fc nvme: add PCI ID accessor to transport
Change-Id: I1776c21d7479f3ef69fe254b8dc4b6d64bbe48bc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-14 12:46:58 -07:00
GangCao
f81888b2f9 nvme: add PCI BDF in spdk_nvme_ctrlr to check whether same ctrlr
Change-Id: Ic8eb395bbfcc688e9c999a6d0026b70c24d386e3
Signed-off-by: GangCao <gang.cao@intel.com>
2016-10-13 09:50:32 -07:00
Ben Walker
bfdc02ab48 nvme: Eliminate nvme_impl.h and use the swappable env lib.
Change-Id: Ibbc557b732d5b0858a2922a7a442c4b17a0d579a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-11 13:34:09 -07:00
GangCao
a5f0d9358d nvme: use nvme_malloc for calloc to share ctrlr->ns among processes
Change-Id: I3e4f211acc0c0db7a6957dd20715da9fc298f73e
Signed-off-by: GangCao <gang.cao@intel.com>
2016-10-10 12:51:32 -07:00
Ben Walker
0dd80395f3 env: Move pci.c from util to env
This allows users to swap their PCI library from
libpciaccess/dpdk to another mechanism using the standard
method for swapping out the env library.

Change-Id: Ib2248f8b43754a540de2ec01897e571f0302b667
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:53:24 -07:00
Ben Walker
b9fbdd189a env: Move malloc/free wrappers into env
Change-Id: Ief591f5e23c4ae06cb77fab647a7afd082450a73
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:51:52 -07:00
Ben Walker
6b1e4e732d Drop libpciaccess and switch to DPDK PCI
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.

Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.

Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-04 15:59:00 -07:00
Ben Walker
b0e349a804 nvme: Use log library instead of nvme_printf
Change-Id: Ic9b2db9bff3a914b3e5021695287157f1e076f9f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-09-28 10:15:55 -07:00
GangCao
b9eedbf403 nvme: use nvme_malloc for calloc to share ctrlr->ioq among processes
Change-Id: I480fe43b8ac59c285999eedc2e4c2323c0401a8b
Signed-off-by: GangCao <gang.cao@intel.com>
2016-09-26 16:05:02 -07:00
GangCao
7faf9554dd nvme: correct the error message for mapping bar operation
Change-Id: Ie0def66b0d395937cfd1e2fec09841f02005e5ed
Signed-off-by: GangCao <gang.cao@intel.com>
2016-09-22 15:42:09 -07:00
Changpeng Liu
d7120a3ee8 nvme: Fix max I/O size enforcement when no stripe size is specified
Intel DC P3*** NVMe devices specify a desired stripe size, which was
used for splitting I/O. Not all devices, however, specify a desired
stripe size (such as the Intel DC D3*** line), and for only these
devices there was a logic mistake that overwrote the maximum I/O
size with a 2MB default. This patch corrects that error.

Change-Id: I94b72a3a3dd1dfa18bd638daf7e01a592eb6ed17
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-08-25 08:51:16 -07:00
Ben Walker
0606eaad1a No longer wrap assert()
assert is part of the C standard library and is available
on any platform we'd consider porting to. Don't put a
wrapper around it.

Change-Id: I0acfdd6a8a269d6c37df38fb7ddf4f1227630223
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-08-19 10:53:06 -07:00
Ben Walker
888014289c nvme: No longer abstract away pthread calls
pthreads are widely supported and are available on any
platform we currently foresee porting to. Use that API
instead of attempting to abstract it away to simplify
the code.

Change-Id: I822f9c10910020719e94cce6fca4e1600a2d9f2a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-08-19 10:53:06 -07:00
Changpeng Liu
dcf49aa018 nvme: disable the controller from generating INTx# interrupts
Since we bind the NVMe device to UIO driver to protect against native
NVMe driver, but for Admin queue, there are still INTx interrupts
exist, as all the completion for Admin queue will be processed in
user space, so we don't need INTx anymore.

Change-Id: Ife5b3e410ae95690ed0f3f9a2f2dfaf55a7797b5
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-08-17 09:08:33 -07:00
Daniel Verkamp
a464f1394e nvme: add API to get CAP and VS registers
This is intended to be used for examples/nvme/identify and similar
diagnostic utilities.

Change-Id: Ib2f941e9af7a3fb7555865ef253742e30ccad2b5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-07 15:32:20 -07:00