Commit Graph

539 Commits

Author SHA1 Message Date
Jim Harris
a9b174e71e build: fix duplicate spdk.common.mk includes
mk/nvme.libtest.mk includes mk/spdk.common.mk, but all
of the Makefiles including mk/nvme.libtest.mk were
also including mk/spdk.common.mk unnecessarily.  So
remove the spdk.common.mk include from all of the
offending Makefiles.

This was relatively harmless, although it would cause
weird things like CFLAGS and LDFLAGS getting duplicated
when building.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie60637db3c19a2ead097562b2adf6573dbe27472

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 14:51:01 +00:00
Jim Harris
9018dd60c6 examples: rename ioat perf tool to ioat_perf
This avoids conflict with the nvme perf tool.  PGO
gets confused during building - we may have data for
nvme/perf which it then tries to use when building
ioat/perf.  Renaming the ioat perf tool fixes that
problem.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib1084d56d671e44027ea05f453075a723f067580

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455320
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-22 14:51:01 +00:00
Ben Walker
e036215fe9 thread: Add a mechanism to exit a lightweight thread
Lightweight threads may now be exited by calling
spdk_thread_exit() within the thread. The framework
polling the thread can release the resources associated
with that lightweight thread by calling
spdk_thread_destroy().

Change-Id: I6586b9d22556b3874fb113ce5402c6b1f371786e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-22 04:23:17 +00:00
Piotr Pelplinski
c4d9daeb7b Makefile: Add possibility to uninstall spdk.
Add uninstall target to makefiles to be able to perform
reverse of install target.
Fixes #464

This patch adds 'uninstall' target to makefile.
'make uninstall' will remove spdk_tgt app, headers, libraries
and shared libraries from system directories defined by $DESTDIR.
Additionaly, if there will be any empty directories left after
this operation, they will be removed as well.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I7b07fb4b81081d3914ff09165991fbe3a26b9067
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/431471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:56:18 +00:00
Chunyang Hui
6b48e743a3 Opal: Add revert tper cmd option
Reset the device to its factory defaults.

Change-Id: I43f7dc8fb7bd5226283a4762beac0e2cf016f698
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-25 18:20:42 +00:00
Tomasz Zawadzki
8bab99fb6e nvme_manage: assert ns not being NULL when displaying namespaces
Assert that spdk_nvme_ctrlr_get_ns() does not return NULL.
If display_namespace() would proceed with ns == NULL,
functions below will segfault since the input is not verified there.

Change-Id: I12c4b617aca703f07275fbc6ab593b42c25e2ccd
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451830
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-25 14:34:53 +00:00
Chunyang Hui
7250ec64db Opal: add take ownership cmd options
Also make some modification for invoking process in
nvme_manage tool.

Change-Id: Ib54db43d7336d3e839e7d9317c292b9a57b38f80
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-24 22:40:04 +00:00
Ben Walker
5d0b5e2ce9 thread: Add cpumask parameter to spdk_thread_create
Allow users to optionally specify an affinity mask when
creating a thread. This isn't currently used, but we want
the API to be in its final form for the next release.

Change-Id: I7bd05e921ece6d8d5f61775bd14286f6a58f267f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451683
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-23 20:22:55 +00:00
Liang Yan
2210eaa9ca examples/nvme/perf: use MiB/s to show bandwidth data
The perf tool's bandwidth data is MiB/s, not MB/s.
So update the data description part.

Change-Id: I770cc0d7c0f0a4d56cb4eff593e88fbed55e3ed6
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451319
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-19 20:15:55 +00:00
Seth Howell
2cc6b0dfcb rdma: set the number of wr sge_entries per I/O
This was not being properly set in the multi-sgl path.
Also add a verification step to the fio configuration file to prevent
against future regressions.

Change-Id: I510b6acd92bc2fbc9b6fbec1d59945cc53584ad3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450305
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-19 17:29:45 +00:00
Hui, Chunyang
55f5b205c5 Opal: Add Opal library
First submission. Implemented part of the Opal library
and "scan" function. Can be invoked by nvme_manage.

Change-Id: Iba86d86dd3af06a06b6805120ee5005af8183459
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/439335
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-08 17:04:25 +00:00
Seth Howell
aeb3dc9d2e hello_bdev: remove passthru section from conf file.
by removing the passthru stuff from the conf file, we create a more
dramatic effect when we add in the passthru bdev during the labs.

Change-Id: I0a0ed101dd4135d8b2774cdd0850ca3bf2e43d01
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450187
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-08 15:54:07 +00:00
Seth Howell
0a1103c41c hello_bdev: remove hardcoded config file.
I think it's simpler to just have people pass in the path of the conf
file during the lab. right now in the lab, if people try to run this
script from any othe rdirectory but hello_world, it throws errors and
can through people off during labs.

Change-Id: I74fc9bd311d4fdc9e8676cfa938d32b48fdece39
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450186
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-08 15:54:07 +00:00
Jim Harris
1a2de82456 examples/hello_blob: use io_unit_size instead of page_size
Blobstore now supports 512B reads/writes to blobs, if the
backing device is formatted for 512B LBAs.  The hello_blob
example app was never updated to account for this - so when
running against a backing device with 512B LBAs, it would
fail since it was only reading/writing 1 blob io_unit (512B)
but was comparing a page size (4KB).

Clean up a typo too while we're here.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6cfeeff1c160a24d4c10b68b9dd93717ed79f212

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450069
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-04 01:10:14 +00:00
Seth Howell
47fca7c7e8 fio_plugin: add support for mocking SGL entries
FIO is going to always present a contiguous buffer to us. But we can
fake out the nvme driver with a couple of global variables.

Change-Id: I038e70582043e1d7c1800ed065fe126aa091c290
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/439608
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-04-02 23:26:08 +00:00
Changpeng Liu
4cfd17a7ce examples/nvme: hot added drive cannot be attached
When test hotplug, the timeout callback will set the attach callback
to NULL, so it will not add the drive back to hotplug tool automatically
sometimes.

Change-Id: I91a7c29059a7f680b2a0b7b79cf0a62c5638dcda
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 00:03:40 +00:00
Darek Stojaczyk
27a23a33f9 blobfs: switch to spdk_*malloc()
spdk_dma_*malloc() is about to be deprecated.

Change-Id: Ib9fb94463a69ffd9660ab2c7fbe02826a7e06741
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449796
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-02 00:02:58 +00:00
Piotr Pelplinski
de5f53220c notify: add notification library
This patch adds library that will provide ability to notify about
events. On one side each spdk library can specify what kind of
notification it can produce and on the other side libraries can fetch
those notifications.

Example is bdev module, which lib notify about added or removed bdev.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Change-Id: Ia95e564a8a43400b2745d9de8217b9cc84cd1b16
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/431920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-29 21:15:14 +00:00
Jim Harris
a0070df9f8 nvme/perf: add -G flag to enable debug logging
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I717b738c50ea7b22eb90c6b764a4768a71208990

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449338
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-29 02:35:48 +00:00
Ben Walker
2fd1b7a7f1 perf: Add option to create unused io queue pairs
For some testing, we need queue pairs to exist but not actually
be in use.

Change-Id: I2b17ff0172c9ec002692babcf7d4d612c3062eb4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/392977
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-22 02:00:20 +00:00
Ben Walker
8165bf7125 build: Add support for linking to liburing
liburing is a wrapper around a new Linux kernel interface for
submitting I/O using user space rings. This patch simply
adds the ability to link the nvme perf tool to liburing, but
doesn't implement any of the new functionality yet.

Change-Id: Idb741c87b6d951c013af86e30eac18d3834dd4b7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-19 20:59:19 +00:00
Ben Walker
9e5075a785 perf: Allow the user to specify the number of queues
Queues will be doled out to cores as available.

Change-Id: Ib6a0fe846a9d90b659754be1c11ae022abbe38a3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/391876
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-19 18:13:31 +00:00
Ben Walker
deb8ee5c33 event: spdk_app_start now takes spdk_msg_fn
This callback type takes 1 argument instead of 2.

Change-Id: Ic3710fafb2828f08fc064f7658849b3d20521092
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446997
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-19 15:54:34 +00:00
Ben Walker
b3fc4e7b4b nvme/identify: Report all atomicity values
New per-namespace values were added.

Change-Id: I2d2bf76ea7944187e4a393111722e85c79e47240
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446323
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-19 15:51:05 +00:00
Ben Walker
f80368bbce nvme/perf: Use delay_doorbell queue pair option
This tool continually polls for completions, so it
is safe to turn this on.

Change-Id: Ice1c68cdaff070f8edd428621e19a6fb44fb8c31
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-19 07:27:44 +00:00
Ben Walker
727d820e59 fio/nvme: Enable delay_doorbell queue pair option
Since fio is continually polling for completions, this
option can be safely enabled.

Change-Id: I02ee3d2507d3b37f79e14d69fe90ee19c4b4eea2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-19 07:27:44 +00:00
Changpeng Liu
8e4d6b3d0d fio_plugin: remove metadata location limitation when doing PI
Change-Id: Ibdf833c6f6de30efa1599d35b4977149441b368a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446824
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-14 19:30:53 +00:00
Changpeng Liu
92ff5e0b12 fio_plugin: export apptag and apptag mask to users
Existing fio plugin tool uses hardcoded application
tag and application tag mask for end-to-end data
protection, we export the two options to users
now.

Change-Id: I64d89c29e99030ce8daa2947e73d941b73ac4a8e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446384
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-13 17:22:45 +00:00
Changpeng Liu
b5853777fe fio_plugin: enable separate metadata(DIX) support
Existing APIs used in the fio plugin tool already
contain the separate metadata parameter, so we just
need to allocate a separate metadata buffer for each
request, by default, each request will have 4096
metadata buffer size when PI enabled with separate
metadata, but also providing an option here to let
users can input bigger value in case one request
will need larger metadata buffer size.

Change-Id: I51679c5cb7f7b1599b81287b1fbb8d9be7959191
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446375
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-13 17:22:45 +00:00
Changpeng Liu
5a051a6c1b fio_plugin: don't submit the IO if got DIF context error
Also set the errno for submitting and verification path.

Change-Id: I97e94eb3c63167eed2f0b14fa7b79c42add834a1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447558
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-12 19:37:39 +00:00
Chunyang Hui
aa09ce098d nvme_manage: Fix a bug of uninitialized namespace
Uninitialized namespace struct will be all zeros.
Some drive might have multiple namespaces but not initialized.
nvme_manage will get to segment fault when this happens. This
patch is to fix this.

Change-Id: I8ea5bf3d7c29cf876fbea3d1bee22b95748d3b8a
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447414
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-08 20:59:51 +00:00
Changpeng Liu
030fb5e2bb fio_plugin: fix the wrong usage of queue pair data structure in fio thread
If multiple numjobs and filename were used in fio tests, one thread may
have a list of queue pairs, so we should store the queue pair when
submitting a new request.

Change-Id: I585cd40ea4295b94c8766f9adfa5a7344cb0bc3c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447272
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-08 04:34:22 +00:00
Ben Walker
362879570a event: Remove arg2 from spdk_app_start()
We never used this anywhere, and I need to move to a model where
the start up action is a thread message instead

Change-Id: I6b21ba9afb93a3245aceca2fe24713ffd16d0933
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446986
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-05 08:43:12 +00:00
Tomasz Zawadzki
154eb3399a thread: add spdk_thread_is_idle()
This function add possibility to check if there are any scheduled operations
on particular thread.

Return from spdk_thread_poll() will be used as a way to load-balance and
signify if any work was performed during the single iteration.
A poller could return 0, but still be registered.

This helps especially in fio_plugin that only checked active_pollers or
messages via spdk_thread_poll().

Change-Id: Id6237278eb3b4bd4922b2abaa3c8ebd5e434d45d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/445915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-01 21:38:02 +00:00
JinYu
885bc99596 nvme/hotplug: add ctrlr a timeout callback in hotplug example
The example would endlessly loop in unregister_dev()->spdk_nvme_
ctrlr_free_io_qpair(), if the device has been removed after leave
hotplug monitoring. Add the timeout callback to handle this corner
case. The case will not be friendly to newcomers, if we use the event
framework to update it, because the hotplug has been hidden in frame
work.

Change-Id: I33a81efd356fdf1e7921f5721e9d95936470e8b0
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439822
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-25 23:12:29 +00:00
Chunyang Hui
53d446c8a3 nvme/identify: add rpmb feature detection
Change-Id: Ib3d52108c5deb209828fe02caaa9f5f968c91d36
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/445638
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-25 22:36:00 +00:00
Shuhei Matsumoto
8696cd4288 dif: Add seed value for guard to avoid 0 in case of all zero data.
Allow user to add seed value for guard compuation to DIF context.
This will avoid the guard being zero in case of all zero data.

NVMe controller doesn't support seed value for guard computation
explicitly, and hence if we want to use such a seed value in
NVMe controller, we have to format metadata more than 8 byte,
and add seed value into the reserved metadata field.

But some popular iSCSI/FC HBAs and SAS controllers have supported
seed value for guard computation, and so supporting seed value
in the SPDK DIF library is very helpful for some use cases.

Hence this patch makes the DIF library possible to specify seed
value for those use cases.

Change-Id: I7e9e87cb441bf263e64605c7820409fdc22dd977
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/444334
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
2019-02-22 17:52:51 +00:00
Jim Harris
5ec6b08eb0 ioat/perf: allow queue depths > 256
Currently the task and buffer pools are hardcoded to
512 each.  This effectively limits the queue depth
per channel to 256 since we need both source and
destination buffers from the buffer pool.  So make
the pool sizes dynamic based on the user's queue depth
input.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I52f04308c0329fa99277746a0768eb1214e37a8e

Reviewed-on: https://review.gerrithub.io/c/445357
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-20 16:41:46 +00:00
Jim Harris
6728c1b8cb ioat/perf: use new spdk_ioat_flush interface
Use the new spdk_ioat_flush interface to batch
doorbell writes - this significantly improves
descriptor throughput.  For now, just set the
threshold for writing the descriptor to half of
the queue depth.  We can always modify this later
to allow very specific thresholds, but for now
this simple change is sufficient.

Increases 512B descriptor throughput at QD=256 from
3.7M/s to 14.0M/s on my Skylake Xeon server.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4504579e23cee5b6a1044849c49d33d58bdb51a9

Reviewed-on: https://review.gerrithub.io/c/445355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-20 16:41:46 +00:00
JinYu
c3da46d5f3 nvme/perf: Add keep alive timeout option
Change-Id: I5f2db53cb3360428d9c9c10d7c9fe8566919ee69
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-15 22:03:35 +00:00
Ben Walker
9810431431 thread: Allow for an extra region of memory allocated on each thread
Schedulers can use this region to store required information.

Change-Id: I93efb44f1a534596f6285bbe014579311fe011e7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444454
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-14 14:58:56 +00:00
Ziye Yang
a1c2b2d0a5 nvme/perf: fix the execption case for NVMe qpair polling.
If do not do that, we will continue polling. And checked
the exceptional handling of aio_check_io, it will just exit.
So we add such case, thus we shall not poll the qpair again.

This is especially useful when we use perf to test against
NVMe-oF target.

Change-Id: Ib820e0b80f80cfceb1ea5e08b359f50d9e360b30
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/444325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-13 18:23:15 +00:00
Ziye Yang
1b8cbebf1c nvme/perf: fix the core dump issue.
If spdk_nvme_probe fails, there will be no controller,
and we jump to clean up. And the thread is not created,
so we need to judge the value of thread. if it is not zero,
we will do the pthread_cancel. Otherwise, we do not
need to call that. If we call that, it will have coredump.

btw: we also fix an small issue of cleanup.

Change-Id: Iee854f6ef42ef6d4a3d3f63d57740375e293a3ba
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444147
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-12 18:40:50 +00:00
Ben Walker
8abfb06e31 thread: Optionally allow the current time to be passed to
spdk_thread_poll()

This is an optimization if the calling function already knows the
current time.

Change-Id: I1645e08e7475ba6345a44e0f9d4b297a79f6c3c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-11 22:58:45 +00:00
Sasha Kotchubievsky
bdc96b4dd6 perf: Fix integer overflow
perf application can't generate IO for NVMe namespace with
 more than 4G size.

 Example of error:
 "Attached to NVMe over Fabrics controller at 1.1.75.1:1023:
 nqn.2016-06.io.spdk.r-dcs75:rd0
 WARNING: controller SPDK bdev Controller (SPDK000DEADBEAF00   ) ns 1 has
 invalid ns size 0 / block size 4096 for I/O size 4096
 WARNING: Some requested NVMe devices were skipped
 No valid NVMe controllers or AIO devices found"

 ns_size variable is uint32_t, spdk_nvme_ns_get_size function
 returns uint64_t. Result can exceed the maximum size of
 uint32_t and ns_size remains 0.

 The issue introduced by commit: f2462909

Change-Id: Idc6dd8688d5d6268bda1a1d6b06a611643af6155
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/443996
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-11 19:30:47 +00:00
Shuhei Matsumoto
6e769bec2c fio_plugin: Use the new DIF library in FIO plugin
Apply the new DIF library to DIF generation and verification.

Condition to use DIF generation and verification are changed
only about DIF type. DIF type 3 is supported in this patch.

DIX is not supported in this patch.

The case that PI is located to the first 8 bytes of the metadata
is not supported in this patch too to because how to pass PI location
is not fixed yet. But this limitation will not be critical because
PI is located to the last 8 bytes of the metadata by default.

DIF insertion and strip will be required not to destroy data by
DIF generation. But this is still added in the TODO list even after
this patch.

Change-Id: If08bcaaaa9f4e0fb4f373ef844b88b38cfffc6b5
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/441283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-11 12:05:13 +00:00
Shuhei Matsumoto
fa92414ce4 fio_plugin: Use the prefix "g_" for global variables throughout
Change-Id: I000a3973ca624bf8d1224f910c3bf0dded9a2e36
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/441280
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-02-11 12:05:13 +00:00
Piotr Pelplinski
7a30221de7 fio_plugin: fix hang in FIO
This is fix for https://github.com/spdk/spdk/issues/523

Fio hangs on pthread_exit(NULL) from spdk thread.
This happens because, pthread_exit tries to dlopen glibc and hangs on
__lll_lock_wait. This patch prevents unmapping of glibc in fio_plugin
and phtread_exit does not need to dlopen it again.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I5078cc55e24841675d6ef4ecba43879dc3f73a4f

Reviewed-on: https://review.gerrithub.io/c/443912
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-11 09:18:26 +00:00
Ben Walker
9ca35dfc83 identify: Print out information about controller memory buffer support
Change-Id: I62c84647e3ae19b3df08c089284043ffebe9081b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442636
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-06 16:01:56 +00:00
Shuhei Matsumoto
67b720f451 examples/sock: Improve error check of input parsing by spdk_strtol
Change-Id: I2757d83344e6a3b78b18b3d3678776e84dc26614
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/441637
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-29 17:39:18 +00:00