Commit Graph

1051 Commits

Author SHA1 Message Date
John Barnard
9c8435b649 env: Fix spdk_ring creation and access.
The spdk_ring creation does not support "multi-producer/multi-consumer" type
and does not honor all of the types during enqueue/dequeue operations.
  - Add SPDK_RING_TYPE_MC and update spdk_ring_create() to support it.
  - Update spdk_ring_enqueue() to call rte_ring_enqueue_bulk() instead of
    rte_ring_mp_enqueue_bulk().
  - Update spdk_ring_dequeue() to call rte_ring_dequeue_burst() instead of
    rte_ring_mp_dequeue_burst().

Change-Id: I15219513f9c45a8ec8a0af19cdc35428ba728454
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/426143
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-20 21:51:55 +00:00
Jim Harris
f6e0a1719b nvme: fix FC-related comments in spdk_nvme_transport_id
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic5a8f5d977a9483ed65d6060aa82310ce6ddf1dd

Reviewed-on: https://review.gerrithub.io/425644
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: John Barnard <john.barnard@broadcom.com>
2018-09-18 15:24:09 +00:00
John Barnard
183d81d0c6 nvmf: Move target opts to transport opts (part 2)
- Add independent functions to create transport with specific opts
  and add to target while maintaining backward compatibility with
  current apps and rpc configuration that still use the add listener
  method to create a transport.
- Add new rpc function to create transport and add to target.
  + Update json reporting to include new rpc function.
  + Update python scripts to support new rpc function.
  + New nvmf test script (cr_trprt.sh) to test new rpc function.

Change-Id: I12d0a42e34c9edff757755f18a78b722d5e1523e
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423590
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-17 20:42:16 +00:00
Seth Howell
1ee27f794d env: introduce a new mem_map_ops structure
This struct will hold the unique operations for the mem_map.

This series of changes is aimed at enabling spdk_mem_map_translate to
report back to the user the length of the valid mem_map up to the
function that requested the translation.
This will be useful when retrieving memory regions associated with I/O
buffers in NVMe-oF. For large I/O it will be possible that the buffer is
split over multiple MRs and the I/O will have to be split into multiple
SGLs.

Change-Id: Ifdd82497f238d99345033f2615c718802a591438
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425327
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-15 00:28:23 +00:00
Seth Howell
d288c41242 env_dpdk: change behavior of spdk_mem_map_translate
The function now takes a pointer as it's last argument, and copies the
size of the memory region for which the translation is validinto that
pointer.
For now, that will always be 2MB. However that behavior can change in
the future.

This series of changes is aimed at enabling spdk_mem_map_translate to
report back to the user the length of the valid mem_map up to the
function that requested the translation.
This will be useful when retrieving memory regions associated with I/O
buffers in NVMe-oF. For large I/O it will be possible that the buffer is
split over multiple MRs and the I/O will have to be split into multiple
SGLs.

Change-Id: I8686c166ec956507f5ae55cf602341281482cb89
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424888
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-15 00:28:23 +00:00
Piotr Pelplinski
c9b8909a02 blobstore: Introduce io_unit size to blobstore.
This patch just adds the call, but doesn't change behaviour of blobstore.
io unit size remains same as page size.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idcd1b7d5126fc7cacf12c996188bd41e2c9a744d

Reviewed-on: https://review.gerrithub.io/425355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:46:27 +00:00
Rami Rosen
412d89cd6f ioat: change return type of spdk_ioat_detach() to be void
The method spdk_ioat_detach() never fails. This patch changes its
return type to be void instead of int.

Change-Id: I6f5bda9ab330c02ae0535bed8b78d416c7b1e264
Signed-off-by: Rami Rosen <ramirose@gmail.com>
Reviewed-on: https://review.gerrithub.io/425252
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-11 21:16:48 +00:00
Ben Walker
f10a91ed0d nvmf: Add function to get local addr for a qpair
Change-Id: I19b9834c709bf97b1bbc1a9278b8c3b9350546e2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-11 15:23:33 +00:00
Ben Walker
311ce0e2ee nvmf: Add a function to get the listen addr for a qpair
The function returns the transport ID describing the
listen address on which the connection originated.

Change-Id: Ib11cddb8ff2ceb04a5f3ce236ba96c68b7226773
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425023
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-11 15:23:33 +00:00
Crane Chu
4e58b2544b log: fix the definition of SPDK_DEBUGLOG
Change-Id: I320dc03c5db44cfe9a029ce7179202f301be6c19
Signed-off-by: Crane Chu <cranechu@gmail.com>
Reviewed-on: https://review.gerrithub.io/425055
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-10 21:09:01 +00:00
Ben Walker
8f64db180e nvmf: Add a function to get the source address for a qpair
Change-Id: I6ae1f380aebbcf090a0ff31ff96fc4592fc29591
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421173
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-07 16:03:06 +00:00
Ben Walker
898739fbac bdev: Enforce that spdk_bdev_close() is called on same thread as open()
spdk_bdev_close() must be called on the same thread as
spdk_bdev_open(). Further, the remove callback on the
descriptor will also be run on the same thread as
spdk_bdev_open().

Change-Id: I949d6dd67de1e63d39f06944d473e4aa7134111b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424738
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-06 20:59:05 +00:00
Jim Harris
ba38785b7e bdev: remove get_bdevs_config RPC
This RPC does not work for a lot of bdev types.  For
example, NVMe namespaces and virtio scsi LUNs are not
explicitly constructed by an RPC - they are indirectly
constructed by an RPC associated with an NVMe controller
or virtio-scsi controller.

While here, remove spdk_bdev_config_json.  It was
only created to facilitate this get_bdevs_config RPC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I537166d8f91ab458bd2000859d74f7254bfc9c0a

Reviewed-on: https://review.gerrithub.io/424584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 19:46:17 +00:00
Jim Harris
a7bafb6121 bdev: clarify the config_json methods
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie2b0e24000665fd95a7c21298c86f4889a03650a

Reviewed-on: https://review.gerrithub.io/424583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 19:46:17 +00:00
Ben Walker
5bc5f86408 nbd: Encapsulate spdk_event usage in subsystem module
Events are part of the SPDK event framework and should be
isolated inside lib/event.

Change-Id: I8e9a5250bcbb9dd891f98df3177a6b5370a65fe3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-05 19:32:37 +00:00
Jim Harris
82c3c30f44 trace: remove alias concept
This was added a long time back for tracking an rte_mbuf
whose buffer was a different rte_mbuf - all related to
a userspace TCP stack that is no longer in development.
The concept isn't useful now, so remove it to reduce
the complexity of the tracing code.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I310e492eba7f55df242bb29d82fb19f6daee1f51

Reviewed-on: https://review.gerrithub.io/424565
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 18:03:43 +00:00
Ben Walker
399a38a541 thread: Fix issue where a channel would be destroyed twice
The scenario is as follows:

spdk_get_io_channel(...)
spdk_put_io_channel(...) <- sends a message to do the release
spdk_get_io_channel(...)
spdk_put_io_channel(...) <- sends a message to do the release again
_spdk_put_io_channel(...)
_spdk_put_io_channel(...) <- boom

Change-Id: I86a66c68a525964e0d2dcd9cac2c292dc8b43136
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424141
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-05 16:00:54 +00:00
Ben Walker
c94020001a thread: Add a name parameter to spdk_register_io_device
This is a string name used for debugging only.

Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 16:00:54 +00:00
Jim Harris
8dfd945b64 trace: add spdk_trace_record_tsc
This is a variant of spdk_trace_record which takes a tsc
parameter.  This allows callers who already have the
current tsc to pass it in as a parameter, saving an extra
rdtsc in the trace library.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I074f4f9fdbe649ad6717f932982a574bc61f2997

Reviewed-on: https://review.gerrithub.io/424277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-04 17:09:25 +00:00
Jim Harris
3e7356580b trace: inline tpoint_mask checking
This reduces overhead of spdk_trace_record calls when
tracing is not enabled.

While here, remove a couple of unit test stubs for
spdk_trace_record that weren't needed anymore.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2cbb91e7d3311d95444d663f466d846676d2dcce

Reviewed-on: https://review.gerrithub.io/424276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-04 17:09:25 +00:00
Ben Walker
603a6de8d5 thread: Add spdk_thread_lib_init and fini
These don't do anything yet.

Change-Id: I718c6fb19c059a39bc5cb360c3da47ec05bfa416
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423767
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-04 16:57:03 +00:00
Jim Harris
4f860d7e40 bdev: only apply split_on_optimal_io_boundary to R/W
Splitting a 1TB unmap into individual 64KB unmap commands
(for a RAID volume with 64KB strip size) would be awful -
the RAID module can be much smarter about this.

So back out the changes for splitting I/O without payload.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24fe6d911f4e3c9db4b2cb5d66c7236a5596e0d9

Reviewed-on: https://review.gerrithub.io/424103
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-31 17:53:42 +00:00
Changpeng Liu
18261f8457 blobfs: add a new API to return file's unique ID
Change-Id: I066085a0606d64a0d95ab2d28340aa35d83efdf7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-29 16:29:22 +00:00
John Barnard
8e8084903e nvmf: Move target opts to transport opts (part 1)
- Move most of the target opts from nvmf_tgt to nvmf_transport.
  - Update transport create functions to pass in transport opts.
  - When transport opts are NULL in transport create function, use
    target opts. (for backward compatiblity)
  - Part 1 of 2 patches. Part 2 (to follow after part 1 accepted)
    will allow independent creation of transport with specific opts
    while maintaining backward compatibility with current apps and
    rpc configuration that still use the add listener method to
    create a transport.

Change-Id: I0e27447c4a98e0b6a6c590541404b4e4be879b47
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423329
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-27 20:43:53 +00:00
Ben Walker
e6bbe23277 util: Move definition of SPDK_SEC_TO_USEC to util.h
This was defined in two places, so consolidate
the definitions.

Change-Id: I0bbb262b97e90d1064bcc50ee201928f6ca9518a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423182
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 18:44:51 +00:00
Jim Harris
4bd9762165 bdev: add split_on_optimal_io_boundary
A number of modules (RAID, logical volumes) have logical
"stripes" that require splitting an I/O into several
child I/O.  For example, on a RAID-0 with 128KB strip size,
an I/O that spans a 128KB boundary will require sending
one I/O for the portion that comes before the boundary to
one member disk, and another I/O for the portion that comes
after the boundary to another member disk.  Logical volumes
are similar - data is allocated in clusters, so an I/O that
spans a cluster boundary may need to be split since the
clusters may not be contiguous on disk.

Putting the splitting logic in the common bdev layer ensures
bdev module authors don't have to always do this themselves.
This is especially helpful for cases like splitting an I/O
described by many iovs - we can simplify this a lot by
handling it in the common bdev layer.

Note that currently we will only submit one child I/O
at a time.  This could be improved later to submit multiple
child I/O in parallel, but the complexity in the iov splitting
code also increases a lot.

Note: Some Intel NVMe SSDs have a similar characteristic.
We will not use this bdev stripe feature for NVMe though -
we want to primarily use the splitting functionality inside
of the NVMe driver itself to ensure it remains fully
functional.  Many SPDK users use the NVMe driver without
the bdev layer.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife804ecc56f6b2b55345a0d0ae9fda9e68632b3b

Reviewed-on: https://review.gerrithub.io/423024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-23 16:08:21 +00:00
Jim Harris
9f2dd0c4f8 bdev: save the bdev_desc specified when submitting the I/O
This will be needed for using this same descriptor when
splitting an I/O.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idec759df7ab27f8de567d3c8a4214e25dbe173f5

Reviewed-on: https://review.gerrithub.io/423022
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-22 16:29:58 +00:00
Seth Howell
34d460708e include: update SPDK_APP_GETOPT_STRING
Reconciling the getopts string with the accepted short form arguments
for spdk apps.

Change-Id: Ie36b69df8c4a0f30f718480b28dac7dcf2da3319
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/422908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-21 17:05:34 +00:00
wuzhouhui
6deac3e660 bdev/lvol: using spdk_bdev_alias_del_all() to delete all alias on destroy
So we don't need to allocate memory (maybe failed) just for free other
memory.

Change-Id: I2c83f6acc2aa6ed79455bff90f952a2e70b44d59
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/422203
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-21 00:53:31 +00:00
Jim Harris
671b77e5cd bdev: remove vbdevs and base_bdevs arrays
The intents of these arrays was to keep track in the
bdev layer of all base<->virtual bdev relationships -
i.e. which member disk bdevs make up a RAID bdev,
which logical volume bdevs are associated with a
bdev that contains an lvolstore, etc.

Currently none of this is used however.  And trying
to keep track in the bdev layer instead of asking
the bdev modules for the relationships has a number
of complications.  Early one, we tried to do this
with TAILQs - but that doesn't work since this can't
be done with a single TAILQ_ENTRY in the bdev
structures.  So we moved to arrays - that works a bit
better, but then the pointer arrays have to be
realloc'd which isn't ideal.

The biggest problem though with these arrays is that
they held bdev pointers - not bdev descriptor pointers.
It's not really valid to access bdevs without a
descriptor - the descriptors are what make sure active
references are accounted for when a bdev is hotplugged.
Of course the bdev layer knows when a bdev is getting
removed and could go and do the updates to these
arrays separately - but that just seems very convoluted.

So for now just remove these arrays completely.  If
there is a future need for the bdev layer to
understand relationships between bdevs, we can add
module APIs so that the generic layer can ask
the modules about the relationships.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99ef1068240bff1262f64f234260cf2fb44df51d
Reviewed-on: https://review.gerrithub.io/420932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
2018-08-17 00:11:03 +00:00
Jim Harris
5b900148e5 nvme: add spdk_nvme_ns_get_extended_sector_size
spdk_nvme_ns_get_sector_size returns the sector
size of the data only.  This new function adds
in the metadata size, if any.

While here, modify the bdev/nvme driver to use this
function - this is needed for ongoing extended sector
size work through the bdev layer.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8070a7f8d29b0b2ac2a2d65e0df5f4736488351

Reviewed-on: https://review.gerrithub.io/422445
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-16 04:38:10 +00:00
Jim Harris
c899854d03 bdev: add new fini_start notification callback for modules
When an SPDK application shuts down, the bdev layer will
automatically unregister all of the bdevs to ensure they
are properly quiesced and cleaned up.

Some modules may want to perform different operations when
a bdev is destructed during normal runtime vs. shutdown.
For example, for lvol, when the last lvol is cleaned up,
it should unload the lvolstore, release and close the bdev
that contains the lvolstore.  You never want to do this
during normal runtime though - it is perfectly valid to
have an lvolstore that contains no lvols.  RAID and future
bdev modules such as multipath have similar use cases.

So add a new bdev module callback named "fini_start".
If a module specifies a function pointer for this callback,
the bdev layer will call it before it starts the bdev
unregistrations.

This enables some future patches to the bdev layer such
that it will always unregister block devices that are not
claimed (i.e. logical volumes) before block devices that
are claimed (i.e. the bdev containing an lvolstore).

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6e87f5c2b27f16731ea5def858f26e882a29495a

Reviewed-on: https://review.gerrithub.io/421175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-15 23:32:04 +00:00
Ben Walker
ed60507d5e nvmf: Queue pairs can no longer be removed from poll groups
In RDMA, qpairs can't be removed from poll groups because
the poll group defines the completion queue. So don't
allow this operation anymore, even if it were theoretically
possible on other transports.

Change-Id: I69a3d1b336decd2d25e43ddea94f8b2095ef662f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-13 18:57:45 +00:00
Dariusz Stojaczyk
c2dc15a0ec app: allow full-name command line options
The following commands now have the same effect:

```
$ spdk_tgt -w -p 1 -m 3 -s 2048
$ spdk_tgt --wait-for-rpc --master-core 1 --cpumask 3 --mem-size 2048
$ spdk_tgt --wait-for-rpc --master-core=1 --cpumask=3 --mem-size=2048
```

Change-Id: I8540a377abf7e02987e9ecd957d142212e9609c4
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/420864
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-13 17:30:10 +00:00
Dariusz Stojaczyk
9b172afdb6 ioat: clear the internal channel error register on reset
I/OAT device will refuse to resume from the Halted
state until we clear both CHANERR and CHANERR_INT
registers.

We only cleared CHANERR so far. So after the I/OAT
device encountered an error, SPDK would not be able
to initialize it ever again unless it's rebound to
the ioatdma driver.

Unlike CHANERR, CHANERR_INT is only accessible through
PCI config space.

CHANERR_INT is only available on I/OAT devices with
version < 3.3.

Change-Id: Ib369ca76f58c4868fe61ff9532b2e7947e5091d3
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422026
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-13 16:59:18 +00:00
Ben Walker
f56b230063 test: Simplify the mock library
Unify the regular mocks and the pointer mocks. Simplify
several of the #defines.

Change-Id: Ica8c69dbb70a685a55b5961b73fd7872f451c305
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/418884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-09 17:18:23 +00:00
Changpeng Liu
6fac5e5b23 bdev/nvme: detect Controller Fatal Status when timeout happens
If the controller has a serious error and set the Controller
Fatal Status field to 1, host driver does not know this error,
while here, when timeout happens, try to detect the CFS and
reset the controller to recover from such fatal status.

Change-Id: I9fa5b263b34edc52d0f359d874b2920f7570d1f3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417622
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-08 17:35:18 +00:00
Jim Harris
27635c770f bdev: track io_stats from deleted channels
Currently, the bdev layer iterates over all of the
existing channels of a bdev to collect I/O
statistics.  But this ignores statistics for
channels that are deleted.

Fix that by keeping an io_stat structure in the
bdev which accumulates statistics for deleted
channels.  Use the bdev mutex to protect these
accumulations.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3103c0b8b55973c827d977765f47e5b9e7f58e5f

Reviewed-on: https://review.gerrithub.io/421029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-03 19:47:15 +00:00
Ben Walker
5a27b6b442 version: 18.10 pre
Change-Id: I2d86f5aa440d971e4852d8b175abca830f42867b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421165
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-03 00:42:36 +00:00
Ben Walker
716ad0e389 SPDK 18.07
Change-Id: I9b9c8f0c213dcdd4d233171daf41e96033324af6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421164
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-02 21:03:33 +00:00
Seth Howell
dd9cd4352d bdev: add tracking for time spent processing I/O.
Two extra fields are added to the iostat rpc.
1. io_time. The amount of time since queue depth tracking was
enabled that has been spent on I/O processing.
2. weighted_io_time. Incremented each time this bdev's queue depth is
polled by the amount of time spent processing I/O since the last polling
event times the measured queue depth.

Change-Id: Ie70489ec24dee83f3eeac8f4f813ec7074ff458f
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419031
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-31 23:26:07 +00:00
dongx.yi
6987057f97 nvmf: update spdk_nvmf_fabric_prop_get_cmd.attrib
NVMF spec lists that the property get commnad and set command
attributes bits 7:3 is reserved, bits 2:0 is used for the size
of property to return.
So update the codes for nvmf specification compliance.

Change-Id: If41cdf2b4b1b17ea6a1c44b13dba2a361d541b48
Signed-off-by: dongx.yi <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/420735
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 16:11:19 +00:00
Ben Walker
a1ec258ac7 Clean up some incorrect Copyright statements
Daisuke Aoyama originally contributed to istgt, the
iSCSI target in FreeBSD. The SPDK iSCSI code is
originally derived from that. Due to copy and paste,
some incorrect copyright attributions have been added
to other files that do  not derive from istgt, so
this patch removes those.

It is doubtful, at this point, that there is any code
whatsoever that remains from the original istgt, but
we can revisit that at a later time.

Change-Id: I207e1e6b99d271e2f12690be90a96f7d0c113af7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/420679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-30 16:29:38 +00:00
Tomasz Zawadzki
b36face57d lvol: remove destruct_req from lvol store
Since destroy_lvol_bdev was implemented, it is now more convinient
to call destroy/unload for lvol store explicitly.
Rather than using lvs struct to pass a request checked on each
lvol close/destroy.

Change-Id: I56ee626e96f8752909d1584a20fe3345c5607fdc
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/420285
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-26 21:55:36 +00:00
Ben Walker
844aedf8ad bdev: Simplify get/set/put buf functions
Also make the iov_len always set to the used length, not
the total length of the buffer.

Change-Id: I7ebb5b63c6ca7570369f4af0131a23c520c1f7b0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/419025
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-26 05:37:26 +00:00
Ziye Yang
703d1f80a8 blobfs: change the return type of spdk_file_truncate
Change-Id: I29347d7b7be0237d7d604c7cd7363fa08a0cd025
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420133
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-26 01:42:17 +00:00
Tomasz Zawadzki
4219407a93 lvol: destroy_lvol_bdev implementation
This patch fixes lvol delete behaviour.
First, we look if there are any dependencies that disallow lvol deletion.
If there are any (i.e. dependent clones) we fail.
Otherwise we delete lvol and unregister associated bdev.

destroy_bdev no longer deletes lvol.

Fixes #345

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I99e6abded2ed3ae2742103f81fc7eb937ad1cab4
Reviewed-on: https://review.gerrithub.io/407402
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-25 17:06:45 +00:00
Ziye Yang
b1da5cecaa nvme: change the type of paramter for timeout_callback set
Change the type of nvme_timeout parameter in
spdk_nvme_ctrlr_register_timeout_callback from uin32_t to uint64_t.

Reason: This will make the timeout trigger test more flexible and
will not affect the original meanings.

Also for the configuration file, still maintain the compatability
support

Change-Id: I94c90f67b2e9c57220ab82ecea11a1590d62aed4
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/419326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-19 07:18:17 +00:00
Liu Xiaodong
8aaa707981 util/base64: add base64 lib and unit tests
RPC method (nvme-passthrough-cmd) needs base64 to
transfer binary buffer of nvme-cmd through Json text.

Change-Id: I50b9d0efca97da45d6d4fd2a1b53b74811947191
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417805
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-19 00:50:54 +00:00
Shuhei Matsumoto
cefab5d647 iscsi: Support hot removal of LUN based on LUN open/close
When hot removal of a LUN is started, callback is called for each
iSCSI connection which accesses the LUN. Callback checks all transfer
tasks complete and then close the LUN.

If the connection clears all transfer tasks before getting all responses
to them from the initiator, the initiator continues to retry data write.
Hence the connection have to wait until all transfer tasks complete.

Change-Id: Iad9063673cfedbd78758890d55a4254512e4fca4
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-17 17:43:28 +00:00