Commit Graph

1040 Commits

Author SHA1 Message Date
Ben Walker
898739fbac bdev: Enforce that spdk_bdev_close() is called on same thread as open()
spdk_bdev_close() must be called on the same thread as
spdk_bdev_open(). Further, the remove callback on the
descriptor will also be run on the same thread as
spdk_bdev_open().

Change-Id: I949d6dd67de1e63d39f06944d473e4aa7134111b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424738
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-06 20:59:05 +00:00
Jim Harris
ba38785b7e bdev: remove get_bdevs_config RPC
This RPC does not work for a lot of bdev types.  For
example, NVMe namespaces and virtio scsi LUNs are not
explicitly constructed by an RPC - they are indirectly
constructed by an RPC associated with an NVMe controller
or virtio-scsi controller.

While here, remove spdk_bdev_config_json.  It was
only created to facilitate this get_bdevs_config RPC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I537166d8f91ab458bd2000859d74f7254bfc9c0a

Reviewed-on: https://review.gerrithub.io/424584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 19:46:17 +00:00
Jim Harris
a7bafb6121 bdev: clarify the config_json methods
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie2b0e24000665fd95a7c21298c86f4889a03650a

Reviewed-on: https://review.gerrithub.io/424583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 19:46:17 +00:00
Ben Walker
5bc5f86408 nbd: Encapsulate spdk_event usage in subsystem module
Events are part of the SPDK event framework and should be
isolated inside lib/event.

Change-Id: I8e9a5250bcbb9dd891f98df3177a6b5370a65fe3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-05 19:32:37 +00:00
Jim Harris
82c3c30f44 trace: remove alias concept
This was added a long time back for tracking an rte_mbuf
whose buffer was a different rte_mbuf - all related to
a userspace TCP stack that is no longer in development.
The concept isn't useful now, so remove it to reduce
the complexity of the tracing code.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I310e492eba7f55df242bb29d82fb19f6daee1f51

Reviewed-on: https://review.gerrithub.io/424565
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 18:03:43 +00:00
Ben Walker
399a38a541 thread: Fix issue where a channel would be destroyed twice
The scenario is as follows:

spdk_get_io_channel(...)
spdk_put_io_channel(...) <- sends a message to do the release
spdk_get_io_channel(...)
spdk_put_io_channel(...) <- sends a message to do the release again
_spdk_put_io_channel(...)
_spdk_put_io_channel(...) <- boom

Change-Id: I86a66c68a525964e0d2dcd9cac2c292dc8b43136
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424141
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-05 16:00:54 +00:00
Ben Walker
c94020001a thread: Add a name parameter to spdk_register_io_device
This is a string name used for debugging only.

Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 16:00:54 +00:00
Jim Harris
8dfd945b64 trace: add spdk_trace_record_tsc
This is a variant of spdk_trace_record which takes a tsc
parameter.  This allows callers who already have the
current tsc to pass it in as a parameter, saving an extra
rdtsc in the trace library.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I074f4f9fdbe649ad6717f932982a574bc61f2997

Reviewed-on: https://review.gerrithub.io/424277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-04 17:09:25 +00:00
Jim Harris
3e7356580b trace: inline tpoint_mask checking
This reduces overhead of spdk_trace_record calls when
tracing is not enabled.

While here, remove a couple of unit test stubs for
spdk_trace_record that weren't needed anymore.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2cbb91e7d3311d95444d663f466d846676d2dcce

Reviewed-on: https://review.gerrithub.io/424276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-04 17:09:25 +00:00
Ben Walker
603a6de8d5 thread: Add spdk_thread_lib_init and fini
These don't do anything yet.

Change-Id: I718c6fb19c059a39bc5cb360c3da47ec05bfa416
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423767
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-04 16:57:03 +00:00
Jim Harris
4f860d7e40 bdev: only apply split_on_optimal_io_boundary to R/W
Splitting a 1TB unmap into individual 64KB unmap commands
(for a RAID volume with 64KB strip size) would be awful -
the RAID module can be much smarter about this.

So back out the changes for splitting I/O without payload.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24fe6d911f4e3c9db4b2cb5d66c7236a5596e0d9

Reviewed-on: https://review.gerrithub.io/424103
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-31 17:53:42 +00:00
Changpeng Liu
18261f8457 blobfs: add a new API to return file's unique ID
Change-Id: I066085a0606d64a0d95ab2d28340aa35d83efdf7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-29 16:29:22 +00:00
John Barnard
8e8084903e nvmf: Move target opts to transport opts (part 1)
- Move most of the target opts from nvmf_tgt to nvmf_transport.
  - Update transport create functions to pass in transport opts.
  - When transport opts are NULL in transport create function, use
    target opts. (for backward compatiblity)
  - Part 1 of 2 patches. Part 2 (to follow after part 1 accepted)
    will allow independent creation of transport with specific opts
    while maintaining backward compatibility with current apps and
    rpc configuration that still use the add listener method to
    create a transport.

Change-Id: I0e27447c4a98e0b6a6c590541404b4e4be879b47
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423329
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-27 20:43:53 +00:00
Ben Walker
e6bbe23277 util: Move definition of SPDK_SEC_TO_USEC to util.h
This was defined in two places, so consolidate
the definitions.

Change-Id: I0bbb262b97e90d1064bcc50ee201928f6ca9518a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/423182
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 18:44:51 +00:00
Jim Harris
4bd9762165 bdev: add split_on_optimal_io_boundary
A number of modules (RAID, logical volumes) have logical
"stripes" that require splitting an I/O into several
child I/O.  For example, on a RAID-0 with 128KB strip size,
an I/O that spans a 128KB boundary will require sending
one I/O for the portion that comes before the boundary to
one member disk, and another I/O for the portion that comes
after the boundary to another member disk.  Logical volumes
are similar - data is allocated in clusters, so an I/O that
spans a cluster boundary may need to be split since the
clusters may not be contiguous on disk.

Putting the splitting logic in the common bdev layer ensures
bdev module authors don't have to always do this themselves.
This is especially helpful for cases like splitting an I/O
described by many iovs - we can simplify this a lot by
handling it in the common bdev layer.

Note that currently we will only submit one child I/O
at a time.  This could be improved later to submit multiple
child I/O in parallel, but the complexity in the iov splitting
code also increases a lot.

Note: Some Intel NVMe SSDs have a similar characteristic.
We will not use this bdev stripe feature for NVMe though -
we want to primarily use the splitting functionality inside
of the NVMe driver itself to ensure it remains fully
functional.  Many SPDK users use the NVMe driver without
the bdev layer.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife804ecc56f6b2b55345a0d0ae9fda9e68632b3b

Reviewed-on: https://review.gerrithub.io/423024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-23 16:08:21 +00:00
Jim Harris
9f2dd0c4f8 bdev: save the bdev_desc specified when submitting the I/O
This will be needed for using this same descriptor when
splitting an I/O.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idec759df7ab27f8de567d3c8a4214e25dbe173f5

Reviewed-on: https://review.gerrithub.io/423022
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-22 16:29:58 +00:00
Seth Howell
34d460708e include: update SPDK_APP_GETOPT_STRING
Reconciling the getopts string with the accepted short form arguments
for spdk apps.

Change-Id: Ie36b69df8c4a0f30f718480b28dac7dcf2da3319
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/422908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-21 17:05:34 +00:00
wuzhouhui
6deac3e660 bdev/lvol: using spdk_bdev_alias_del_all() to delete all alias on destroy
So we don't need to allocate memory (maybe failed) just for free other
memory.

Change-Id: I2c83f6acc2aa6ed79455bff90f952a2e70b44d59
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/422203
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-21 00:53:31 +00:00
Jim Harris
671b77e5cd bdev: remove vbdevs and base_bdevs arrays
The intents of these arrays was to keep track in the
bdev layer of all base<->virtual bdev relationships -
i.e. which member disk bdevs make up a RAID bdev,
which logical volume bdevs are associated with a
bdev that contains an lvolstore, etc.

Currently none of this is used however.  And trying
to keep track in the bdev layer instead of asking
the bdev modules for the relationships has a number
of complications.  Early one, we tried to do this
with TAILQs - but that doesn't work since this can't
be done with a single TAILQ_ENTRY in the bdev
structures.  So we moved to arrays - that works a bit
better, but then the pointer arrays have to be
realloc'd which isn't ideal.

The biggest problem though with these arrays is that
they held bdev pointers - not bdev descriptor pointers.
It's not really valid to access bdevs without a
descriptor - the descriptors are what make sure active
references are accounted for when a bdev is hotplugged.
Of course the bdev layer knows when a bdev is getting
removed and could go and do the updates to these
arrays separately - but that just seems very convoluted.

So for now just remove these arrays completely.  If
there is a future need for the bdev layer to
understand relationships between bdevs, we can add
module APIs so that the generic layer can ask
the modules about the relationships.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99ef1068240bff1262f64f234260cf2fb44df51d
Reviewed-on: https://review.gerrithub.io/420932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
2018-08-17 00:11:03 +00:00
Jim Harris
5b900148e5 nvme: add spdk_nvme_ns_get_extended_sector_size
spdk_nvme_ns_get_sector_size returns the sector
size of the data only.  This new function adds
in the metadata size, if any.

While here, modify the bdev/nvme driver to use this
function - this is needed for ongoing extended sector
size work through the bdev layer.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8070a7f8d29b0b2ac2a2d65e0df5f4736488351

Reviewed-on: https://review.gerrithub.io/422445
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-16 04:38:10 +00:00
Jim Harris
c899854d03 bdev: add new fini_start notification callback for modules
When an SPDK application shuts down, the bdev layer will
automatically unregister all of the bdevs to ensure they
are properly quiesced and cleaned up.

Some modules may want to perform different operations when
a bdev is destructed during normal runtime vs. shutdown.
For example, for lvol, when the last lvol is cleaned up,
it should unload the lvolstore, release and close the bdev
that contains the lvolstore.  You never want to do this
during normal runtime though - it is perfectly valid to
have an lvolstore that contains no lvols.  RAID and future
bdev modules such as multipath have similar use cases.

So add a new bdev module callback named "fini_start".
If a module specifies a function pointer for this callback,
the bdev layer will call it before it starts the bdev
unregistrations.

This enables some future patches to the bdev layer such
that it will always unregister block devices that are not
claimed (i.e. logical volumes) before block devices that
are claimed (i.e. the bdev containing an lvolstore).

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6e87f5c2b27f16731ea5def858f26e882a29495a

Reviewed-on: https://review.gerrithub.io/421175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-15 23:32:04 +00:00
Ben Walker
ed60507d5e nvmf: Queue pairs can no longer be removed from poll groups
In RDMA, qpairs can't be removed from poll groups because
the poll group defines the completion queue. So don't
allow this operation anymore, even if it were theoretically
possible on other transports.

Change-Id: I69a3d1b336decd2d25e43ddea94f8b2095ef662f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-13 18:57:45 +00:00
Dariusz Stojaczyk
c2dc15a0ec app: allow full-name command line options
The following commands now have the same effect:

```
$ spdk_tgt -w -p 1 -m 3 -s 2048
$ spdk_tgt --wait-for-rpc --master-core 1 --cpumask 3 --mem-size 2048
$ spdk_tgt --wait-for-rpc --master-core=1 --cpumask=3 --mem-size=2048
```

Change-Id: I8540a377abf7e02987e9ecd957d142212e9609c4
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/420864
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-13 17:30:10 +00:00
Dariusz Stojaczyk
9b172afdb6 ioat: clear the internal channel error register on reset
I/OAT device will refuse to resume from the Halted
state until we clear both CHANERR and CHANERR_INT
registers.

We only cleared CHANERR so far. So after the I/OAT
device encountered an error, SPDK would not be able
to initialize it ever again unless it's rebound to
the ioatdma driver.

Unlike CHANERR, CHANERR_INT is only accessible through
PCI config space.

CHANERR_INT is only available on I/OAT devices with
version < 3.3.

Change-Id: Ib369ca76f58c4868fe61ff9532b2e7947e5091d3
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/422026
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-13 16:59:18 +00:00
Ben Walker
f56b230063 test: Simplify the mock library
Unify the regular mocks and the pointer mocks. Simplify
several of the #defines.

Change-Id: Ica8c69dbb70a685a55b5961b73fd7872f451c305
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/418884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-09 17:18:23 +00:00
Changpeng Liu
6fac5e5b23 bdev/nvme: detect Controller Fatal Status when timeout happens
If the controller has a serious error and set the Controller
Fatal Status field to 1, host driver does not know this error,
while here, when timeout happens, try to detect the CFS and
reset the controller to recover from such fatal status.

Change-Id: I9fa5b263b34edc52d0f359d874b2920f7570d1f3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417622
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-08 17:35:18 +00:00
Jim Harris
27635c770f bdev: track io_stats from deleted channels
Currently, the bdev layer iterates over all of the
existing channels of a bdev to collect I/O
statistics.  But this ignores statistics for
channels that are deleted.

Fix that by keeping an io_stat structure in the
bdev which accumulates statistics for deleted
channels.  Use the bdev mutex to protect these
accumulations.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3103c0b8b55973c827d977765f47e5b9e7f58e5f

Reviewed-on: https://review.gerrithub.io/421029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-03 19:47:15 +00:00
Ben Walker
5a27b6b442 version: 18.10 pre
Change-Id: I2d86f5aa440d971e4852d8b175abca830f42867b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421165
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-03 00:42:36 +00:00
Ben Walker
716ad0e389 SPDK 18.07
Change-Id: I9b9c8f0c213dcdd4d233171daf41e96033324af6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421164
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-02 21:03:33 +00:00
Seth Howell
dd9cd4352d bdev: add tracking for time spent processing I/O.
Two extra fields are added to the iostat rpc.
1. io_time. The amount of time since queue depth tracking was
enabled that has been spent on I/O processing.
2. weighted_io_time. Incremented each time this bdev's queue depth is
polled by the amount of time spent processing I/O since the last polling
event times the measured queue depth.

Change-Id: Ie70489ec24dee83f3eeac8f4f813ec7074ff458f
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/419031
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-31 23:26:07 +00:00
dongx.yi
6987057f97 nvmf: update spdk_nvmf_fabric_prop_get_cmd.attrib
NVMF spec lists that the property get commnad and set command
attributes bits 7:3 is reserved, bits 2:0 is used for the size
of property to return.
So update the codes for nvmf specification compliance.

Change-Id: If41cdf2b4b1b17ea6a1c44b13dba2a361d541b48
Signed-off-by: dongx.yi <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/420735
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 16:11:19 +00:00
Ben Walker
a1ec258ac7 Clean up some incorrect Copyright statements
Daisuke Aoyama originally contributed to istgt, the
iSCSI target in FreeBSD. The SPDK iSCSI code is
originally derived from that. Due to copy and paste,
some incorrect copyright attributions have been added
to other files that do  not derive from istgt, so
this patch removes those.

It is doubtful, at this point, that there is any code
whatsoever that remains from the original istgt, but
we can revisit that at a later time.

Change-Id: I207e1e6b99d271e2f12690be90a96f7d0c113af7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/420679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-30 16:29:38 +00:00
Tomasz Zawadzki
b36face57d lvol: remove destruct_req from lvol store
Since destroy_lvol_bdev was implemented, it is now more convinient
to call destroy/unload for lvol store explicitly.
Rather than using lvs struct to pass a request checked on each
lvol close/destroy.

Change-Id: I56ee626e96f8752909d1584a20fe3345c5607fdc
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/420285
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-26 21:55:36 +00:00
Ben Walker
844aedf8ad bdev: Simplify get/set/put buf functions
Also make the iov_len always set to the used length, not
the total length of the buffer.

Change-Id: I7ebb5b63c6ca7570369f4af0131a23c520c1f7b0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/419025
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-26 05:37:26 +00:00
Ziye Yang
703d1f80a8 blobfs: change the return type of spdk_file_truncate
Change-Id: I29347d7b7be0237d7d604c7cd7363fa08a0cd025
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420133
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-26 01:42:17 +00:00
Tomasz Zawadzki
4219407a93 lvol: destroy_lvol_bdev implementation
This patch fixes lvol delete behaviour.
First, we look if there are any dependencies that disallow lvol deletion.
If there are any (i.e. dependent clones) we fail.
Otherwise we delete lvol and unregister associated bdev.

destroy_bdev no longer deletes lvol.

Fixes #345

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I99e6abded2ed3ae2742103f81fc7eb937ad1cab4
Reviewed-on: https://review.gerrithub.io/407402
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-25 17:06:45 +00:00
Ziye Yang
b1da5cecaa nvme: change the type of paramter for timeout_callback set
Change the type of nvme_timeout parameter in
spdk_nvme_ctrlr_register_timeout_callback from uin32_t to uint64_t.

Reason: This will make the timeout trigger test more flexible and
will not affect the original meanings.

Also for the configuration file, still maintain the compatability
support

Change-Id: I94c90f67b2e9c57220ab82ecea11a1590d62aed4
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/419326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-19 07:18:17 +00:00
Liu Xiaodong
8aaa707981 util/base64: add base64 lib and unit tests
RPC method (nvme-passthrough-cmd) needs base64 to
transfer binary buffer of nvme-cmd through Json text.

Change-Id: I50b9d0efca97da45d6d4fd2a1b53b74811947191
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417805
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-19 00:50:54 +00:00
Shuhei Matsumoto
cefab5d647 iscsi: Support hot removal of LUN based on LUN open/close
When hot removal of a LUN is started, callback is called for each
iSCSI connection which accesses the LUN. Callback checks all transfer
tasks complete and then close the LUN.

If the connection clears all transfer tasks before getting all responses
to them from the initiator, the initiator continues to retry data write.
Hence the connection have to wait until all transfer tasks complete.

Change-Id: Iad9063673cfedbd78758890d55a4254512e4fca4
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-17 17:43:28 +00:00
Shuhei Matsumoto
fec7c88e45 scsi: Enforce to open LUN and then alloc IO channel by using descriptor
Use a descriptor got by opening the LUN to allocate IO channel of the LUN.
This requires opening the LUN is done before allocating IO channel of the
LUN.

Use the descriptor to free IO channel of the LUN too.

Additionally, assert is added to the close LUN function to check if
IO channel is freed before the last close LUN function is called.

Change-Id: Iafb2f9ce790fff25801ea45b3286f3e26943807b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-17 17:43:28 +00:00
Shuhei Matsumoto
c740e56928 scsi: Open/close a LUN to remove it safely for iSCSI multi connections
To remove a LUN safely for iSCSI multiple connections, each connection
must return I/O channel after checking completion of all tasks.

Open/close mechanism provides a way to register callback to do it.

Registered callback will be called after checking completion of
outstanding tasks.

Making the close LUN function public is necessary to support hot removal.

Change-Id: I06d50d016b0b7aba0d081da226f5b2e0c911629e
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-17 17:43:28 +00:00
Changpeng Liu
01831056c5 env/app: add unlink hugepages option to app
DPDK has an option "huge-unlink" which can be used to
remove mmapped files after EAL initialization, SPDK
doesn't enable the option by default, ethier DPDK,
while here, export a new parameter which can let
user to decide enable it or not.

Fix issues #349 and #350.

Change-Id: Ic516b9f48f7b1c7c51712cc7bb7475ed904ff24b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/419156
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-17 07:06:53 +00:00
Yanbo Zhou
1f2e86283c include/nvme.h: add comments for public APIs
Change-Id: I179036912b2dd3b87ebc638d43626295b96b218a
Signed-off-by: Yanbo Zhou <yanbo.zhou@intel.com>
Reviewed-on: https://review.gerrithub.io/411128
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-16 04:56:16 +00:00
Lance Hartmann
b4ceb613e1 nvme: add chk_rdy quirk for some Samsung drives
Add some known Samsung controllers to those requiring the quirk,
NVME_QUIRK_DELAY_BEFORE_CHK_RDY.  Addresses an issue for those
who may not have later firmware that corrects the problem.
Correspondingly, extend the delay from 2 secs to 2.5 secs.

Change-Id: Iee773905a2a49711775042c061f6c347e0da85e9
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/419273
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-14 01:47:47 +00:00
Seth Howell
760b868aa8 bdev: add ability to track bdev queue depth.
This change includes a function to enable this feature on a per-bdev
basis. The new information stored in the bdev includes the following:

measured_queue_depth: The aggregate of the outstanding oparations from
each channel associated with this bdev.
period: The period at which this bdev's measured_queue_depth is being
updated.

With this information, one could calculate the average queue depth and
the disk utilization of the device

Change-Id: Ie0623ee4796e33b125504fb0965d5ef348cbff7d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418102
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-07-12 23:27:27 +00:00
Tomasz Zawadzki
f0ec7bc6e7 scsi/bdev: use spdk_bdev_queue_io_wait()
New function was added in bdev layer to allow
handling spdk_bdev_io buffer exhaustion.

This patch adds that functionality to scsi bdev.

Change-Id: Ia6a5be871ae09a4d1166991925f0a44f3b355bdd
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/417032
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-10 21:17:38 +00:00
Young Tack Jin
6d05186697 ocssd: add chunk info list as mptr of vector reset
This is verified on QEMU NVMe which merged the related patch.

Change-Id: I7c62d7a91350e0a877d1c3651796c38b55122f98
Signed-off-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/417077
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Matias Bjørling <m@bjorling.me>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-09 18:54:22 +00:00
Seth Howell
09e3f4e3db nvmf: give qpair_disconnect an asynchronous api.
qpair_disconnect has previously presented an entirely synchronous API.
However, it relies on other asynchronous operations to complete its
task. By giving it an asynchronous API, we can avoid possible race
conditions. Patch 1 of several.

Change-Id: If9e26ee70ae5d6c0273750226b4408a8e4587e19
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/417345
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 22:49:39 +00:00
John Barnard
82c64e2a41 nvmf: FC-NVMe spec. header file
Header file with data definitions from FC-NVMe specification.

Change-Id: Idbc5777d434df2451064ac72b249534e31b094e6
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/417193
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: John Meneghini <johnm@netapp.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 22:49:20 +00:00
Seth Howell
274980704e bdev: reorder items in spdk_bdev.
This fills two holes and allows the structure to fit neatly into 4
cachelines.

Change-Id: Ie175fec2656ff0b60a8c6fcbc26b1a66a8bf5162
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/417604
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 22:48:13 +00:00