175 Commits

Author SHA1 Message Date
Pawel Wodkowski
2f2dde6394 event: allow subsystem to write configuration in async way
Change-Id: I5a09551ff6a97ae363fa6166b284cbfc2538d736
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/404372
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-03-27 02:57:28 -04:00
Pawel Wodkowski
e7cb376aad subsystem,bdev: rework RPC JSON config dump
Changes:
- change write_config_json to return void as return value is useless.
- int spdk_bdev_config_json() -> void spdk_bdev_subsystem_config_json()
- int spdk_bdev_write_config_json -> void spdk_bdev_config_json()

Change-Id: I46e2e974abada0df67c07ba961543900f8334a8d
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/405052
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-03-27 02:57:28 -04:00
Dariusz Stojaczyk
6ccc262022 virtio: remove g_virtio_driver
g_virtio_driver contained just a list of all
Virtio-SCSI devices. It's now being removed for
two reasons:
 * it's backend-specific, doesn't fit a generic virtio lib
 * it's difficult to ensure thread-safety for it

Virtio bdev modules will now manage their Virtio
devices by themselves. Virtio-SCSI has now an internal
device list. (And VirtioBlk module maps devices to
bdevs 1:1, so doesn't need any additional work here.)

Change-Id: I3bc68d76d6904df5c56f00fca7ab387871ecf88f
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/405179
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-27 00:58:14 -04:00
Jim Harris
3fe80a337c lvol: remove num_clusters member from spdk_lvol
This is not needed - we can easily get the number
of clusters from the underlying blob.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I94a716731bd1b8c4b333fe295bd5a963928b4330

Reviewed-on: https://review.gerrithub.io/404615
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-23 04:02:13 -04:00
Pawel Wodkowski
5ffa5c003a bdev: introduce 'get_bdevs_config' RPC call
Write information needed to recreate each bdev.

Change-Id: I3d2b24fd4aaa8b98ec558d864cc28b55a899452f
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/401217
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-23 03:57:44 -04:00
Pawel Wodkowski
bd4ecea505 subsystem: add per module configuration dump
Change-Id: I5c4e51cd9cd97b05ab9a95cbe084ff2741f6ef58
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/402323
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-23 02:47:40 -04:00
Daniel Verkamp
7fd3a82561 bdev: rearrange struct spdk_bdev_io
Move the commonly-accessed fields to the front so they end up in the
same cache line where possible.

Also tweak the types of type, status, error.nvme.sct, error.nvme.sc,
error.scsi.sc, and error.scsi.sk (they can fit in 8 bits), and move the
Write Zeroes splitting variables into u.bdev.

This reduces sizeof(struct spdk_bdev_io) from 272 to 224, in addition to
the better cache line usage.

Change-Id: I4a91fd07f252e7add4a2db179df9c53268672198
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/404053
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-16 14:06:36 -04:00
Tomasz Zawadzki
9004ca4a63 net/sock: separate sock.c from posix related functions
This enables more clear division of spdk_sock_* functions
and spdk_posix_* ones.

Change-Id: I329eaf99de9390c8473b57427966dfe6af777abb
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/403700
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-03-14 13:08:09 -04:00
Daniel Verkamp
d4ef1338b0 bdev: make QoS channel management thread safe
We must hold bdev->mutex around all QoS channel manipulations, not just
channel_count; otherwise, there are race conditions.

Change-Id: I6183aef83f4d5789bded426a1832e3faaa688363
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/403367
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-13 14:06:23 -04:00
Daniel Verkamp
19100ed580 bdev: rename spdk_bdev_module_if -> spdk_bdev_module
This better matches the style in the rest of SPDK.

No functional change - this is a pure find/replace of
spdk_bdev_module_if to spdk_bdev_module.  Instances of this struct will
be renamed in another patch.

Change-Id: I3f6933c8a366e625fc3a1b6401aee26ee03ba69c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/403368
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-13 00:55:12 -04:00
GangCao
a2142f3a33 bdev/qos: add the QoS rate limiting support on bdev
This patch is to add the basic support of QoS on bdev.

Including two major functionalities:

1. The QoS rate limiting algorithm:
	a. New IO will be always queued first also under
	   the no memory condition
	b. Start the QoS IO operation based on the limit
	c. A poller started in each millisecond to reset
	   the rate limit and send new IOs down
	d. The rate limit is based on the millisecond and
	   converted from user configurable IOsPerSecond

2. The Master Thread management:
	a. Add a per bdev channel_count
	b. Whenever QoS is enabled on bdev, if QoS bdev
	   channel is not created, create the QoS bdev
	   channel and assign the QoS thread
	c. When new IOs coming from different channels
	   (threads), pass the IOs to the QoS bdev channel
	   through the thread event
	d. When the IOs are completed from the QoS bdev
	   channel, pass the IOs back to its orignal
	   channel(thread)
	e. Destroy the QoS bdev channel when it is the
	   last bdev channel for this bdev. Defer the
	   destruction if current thread is not QoS thread

Change-Id: Ie4444551d7c3c7de52f6513c9db926628796adb4
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/393136
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-09 13:48:21 -05:00
Pawel Wodkowski
4d36735401 bdev: rework bdev module registration
Currently SPDK_BDEV_MODULE_REGISTER() take many parameters. Extending it
(eg for incoming JSON configuration dump/load) is quite challenging and
error prone. As we are already here in next patches, rework this macro
to take one parameter - the pointer to struct spdk_bdev_module_if.

This patch also remove following macros:
SPDK_GET_BDEV_MODULE - this is not really needed, to find module outside
module translation unit use spdk_bdev_module_list_find()

SPDK_BDEV_MODULE_ASYNC_INIT and SPDK_BDEV_MODULE_ASYNC_FINI - replaced
by bool fields in spdk_bdev_module_if struct.

Change-Id: Ief88e023fbbaee7d5402c838dbecbdffd4dfb259
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/402883
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-09 12:07:35 -05:00
Pawel Wodkowski
4294719ed7 subsystem: make subsystem declaration macro just register subsystem
Don't do anything under the scene as adding/removing fields in struct
spdk_subsystem will require both macro change and C file changes.

Change-Id: I11bc0d5758edd7e0e0bd3f93e251d868cd9b1d2b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/401850
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-03-09 12:07:35 -05:00
Sebastian Basierski
f5e590c8f2 bdev: Added functions allowing logical volume store rename.
Change-Id: Ief1f809308fbde2e696c60d3ce79c0720cb3e2ff
Signed-off-by: Sebastian Basierski <sebastianx.basierski@intel.com>
Reviewed-on: https://review.gerrithub.io/398934
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2018-03-08 11:26:16 -05:00
Daniel Verkamp
8887697f8c bdev/lvol: add UUID to lvols
In addition to the lvolstore uuid, add a per-lvol uuid for bdev
identification.

The lvol uuid is stored as an xattr for each lvol blob; if an lvol blob
without a uuid xattr is encountered, the lvol bdev will not report a
uuid for now (it will be set to all zeroes, which will be treated as
no uuid available by the generic bdev layer).

Change-Id: If00221383a12d62234fc085d56e257dd48053103
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402973
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-08 10:49:51 -05:00
Daniel Verkamp
364d4fdfe0 bdev: add spdk_bdev_get_uuid() function
Add a generic way to get a UUID from a bdev.

For now, malloc and null bdevs generate random UUIDs, and no other bdev
types report a UUID.

Change-Id: Id9608c8c1b3ce3f1783e7f74bef96d44cd5d98a7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402177
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2018-03-08 10:49:51 -05:00
Daniel Verkamp
4c06ce9b9d util: add uuid.h to wrap libuuid
This lets us have a common place to put definitions like the length of
the UUID string, as well as abstract away some of the API warts in
libuuid (non-const values, no size checking for uuid_unparse, etc.).

Change-Id: I80607fcd21ce57fdbb8729442fbb721bc71ccb98
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402176
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-08 10:49:51 -05:00
Daniel Verkamp
8a6ba58cb4 scripts/check_format: check for spaces before tabs
Automatically detect more whitespace errors.

All existing cases are fixed; only whitespace change (verify with
diff -w) except for one comment style fixup in include/spdk/nvme.h.

Change-Id: If750e54b9c8e3421ea6feda5f20184a31431631e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402360
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-05 11:09:13 -05:00
Pawel Wodkowski
2939b715d0 bdev: rename 'dump_config_json' to 'dump_info_json'
Unfortunatly not all bdevs produce its configuration in responce to
get_bdevs RPC call (eg nvme is producing tons of additional
informations). To not breake any existing scripts rename
'dump_config_json' to 'dump_info_json' instead of reworking those
callbacks. Next patches will introduce real 'dump_config_json' handlers
and API

Change-Id: If9c1a4ab864791b24a5f7d022e970cd65990ffc0
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/401216
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-02-26 15:49:51 -05:00
Pawel Wodkowski
ba1859ed55 subsystem: add "get_subsystems" RPC call
Change-Id: I8927269134a1bab86a1dfb677bc38dc7f25c8ea4
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/400634
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-02-23 15:27:16 -05:00
Jim Harris
5665dbd91f net: add spdk_sock_group_impl
spdk_sock_group_impl represents the implementation of
a sock group for a given spdk_net_impl (posix, vpp, etc.)

This extra abstraction is required for enabling
spdk_sock_groups with sockets associated with different
spdk_net_impls.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7a380cc62c59bdac98996faf0387550c234165a0

Reviewed-on: https://review.gerrithub.io/400527
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-02-21 12:40:30 -05:00
Jim Harris
7ae63b0d5c net: add spdk_net_impl
This is the framework for providing multiple implementations
of TCP sockets in a single application.  This patch just
adds the framework - upcoming patches will convert the
existing POSIX implementation to use this framework.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I07951e71f344dfe7689c818a13d7e7113c826643

Reviewed-on: https://review.gerrithub.io/400334
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-02-21 12:40:30 -05:00
Sebastian Basierski
526cd58003 bdev: Added functions allowing logical volume rename.
Change-Id: Ice343afdc84c2a11cd9026d128a35f7a62aa7125
Signed-off-by: Sebastian Basierski <sebastianx.basierski@intel.com>
Reviewed-on: https://review.gerrithub.io/398933
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-16 16:21:20 -05:00
Daniel Verkamp
ff12a9b540 rpc: remove [Rpc] configuration file section
This section was deprecated in the previous release, and its removal
was planned for v18.04.

Users may configure the RPC listen address via the command-line '-r'
parameter.

Change-Id: Ic714417b87d422492b8742852c16983eb61793c2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/398239
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-02-06 11:15:39 -05:00
Tomasz Zawadzki
e87d3400a5 lvol: display thin_provision property during get_bdevs
Change-Id: Ib3507a9e69b210cbd173c88d166025d4579e0149
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/397602
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-31 15:39:25 -05:00
Maciej Szwed
cb5178ea40 lvol: set default cluster size to 4MiB
Blobstore default cluster size is currently 4MiB,
but default lvol cluster size overrides it to 1GiB.

Additionally add -c flag in lvol tests to set cluster
size for tests.

1GiB was fine without thin provisioning, but we do not
want to allocate and copy 1GiB of data for newly allocated
clusters on thin provisioned logical volumes.

Note that 4MiB is same as default for Linux LVM.

lvol test case 601 had to be modified to create a malloc
LUN less than size of default cluster size.  This test is
supposed to fail - without the change here, creating the
lvolstore would work, but the script still considered it a
PASS.  Defer fixing that to a separate patch - for now just
keep the same test process.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ic05defbf8e641f613f0ec74175a37bc25986e496

Reviewed-on: https://review.gerrithub.io/397562
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-31 15:37:26 -05:00
Daniel Verkamp
7d716668d6 utf.h: remove #include "spdk/json.h"
After the renaming of json_internal.h to utf.h, it doesn't make sense
for utf.h to include spdk/json.h.

Move the #include "spdk/json.h" to the JSON library implementation
files and remove it from utf.h.

Change-Id: I36092524c9b982fd2e931faf1b7c5d1d6a6c80c0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/397603
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
2018-01-31 14:37:16 -05:00
Isaac Otsiabah
a4f3920d51 bdev: Added latency to channel statistics
Modified include/spdk/bdev.h and include/spdk_internal/bdev.h
add data members to capture statistics information. Modified
lib/bdev/bdev.c to calculate read/write latency.

Change-Id: Idcd55dd2e88c4b308e016f16ced53720256c79e3
Signed-off-by: Isaac Otsiabah <iotsiabah@us.fujitsu.com>
Reviewed-on: https://review.gerrithub.io/390654
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-30 12:41:41 -05:00
Slawomir Mrozowicz
d4822a7db5 bdev: Add bdev resize function
Add api and unit test functions for
change number of blocks for provided block device.

Change-Id: I55d67c99375cb88bdaa79ce1a36d4298223beddc
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Reviewed-on: https://review.gerrithub.io/390802
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-26 14:45:08 -05:00
Daniel Verkamp
f22907768a mock: mark wrappers "used" so LTO will keep them
Fixes the build with CONFIG_LTO=y.

Change-Id: I3acbdd16c5ff44f50747644c09ca7b0228c790cc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/396449
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-25 14:21:02 -05:00
Tomasz Zawadzki
b6aaba0852 bdev: remove vbdevs during spdk_bdev_unregister()
spdk_vbdev_unregister() is part of internal bdev API,
yet bdev module that uses spdk_vbdev_register() directly
will not be removed correctly when using delete_bdev RPC.
spdk_vbdev_unregister() is now consolidated with
spdk_bdev_unregister().

This comes up when deleting lvol bdev, as it does not use
spdk_bdev_part_* functions.
base_bdev->vbdevs entry was not removed for bdev that lvs
is created on.

Additionally patch expands test to create lvol bdev,
after removing it using delete_bdev RPC.
With ASAN enabled this would report accessing
already freed memory previously.

Change-Id: I9547e83862e2daa50355d56a1c9f453aaa6cfdb8
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/395711
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 15:50:04 -05:00
Dariusz Stojaczyk
fb12bbecd2 virtio: move vdev->name allocation to generic virtio
Previously vdev->name was being allocated/freed
separately in virtio_pci and virtio_user backends.
Now it's all done in generic virtio library and
cleans up some code.

Change-Id: I810e976d09781c0c9b25c6f7fd957a83aad6c7b8
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394704
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 22:49:21 -05:00
Dariusz Stojaczyk
37fb62224a bdev/virtio: add API to attach particular PCI devices
This is just the API introduction. See subsequent
patches for its RPC usage.

Change-Id: Iadb7c9bf6a56ab4330c9f2215c6006a2935d208d
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394445
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-22 15:57:11 -05:00
Dariusz Stojaczyk
5c0c104b81 virtio: add opaque ctx param to PCI enumerate callback
In current bdev_virtio_scsi PCI enumerate callback
implementation we rely on a global variable - a global
list of virtio devices. We do not need any opaque
context data inside this callback just yet. It will
be required to add virtio devices in runtime. See the
next patch for details.

Change-Id: I116cbd3bd633f56922eedcc7c07b8c0310e51d49
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394444
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-16 19:13:32 -05:00
Jim Harris
b5f4a259bd bdev: save mgmt_ch that spdk_bdev_io was allocated from
This avoids having to dereference the spdk_bdev_io's
channel in the spdk_bdev_free_io() path.

Cleaning up after hotplug events should ensure that
all associated bdev_ios have been freed (not just
completed) before the bdev channels have been freed,
but this patch gives us some more wiggle room in this
area.

Results in a small (1-2%) performance degradation on
a bdevperf microbenchmark, but should result in no
noticeable difference on any real world workload.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8b88300fc53e8c0b83309a738a4c3bd2aeaff52

Reviewed-on: https://review.gerrithub.io/394399
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
2018-01-12 13:27:20 -05:00
Seth Howell
0a97bd140b json_internal.h: rename to utf.h and place in spdk_internal
Change-Id: I1b1f3a6c10b97c6323e52b58537293f6a6c5c56b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/394117
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-11 10:57:38 -05:00
Jim Harris
cd55b3886e bdev: change spdk_bdev_io buf_link to STAILQ
This is more efficient and buf_link users don't
need the extra flexibility that TAILQ provides.

link could also be changed to an STAILQ, but its usage
is not in the performance path so not touching that
as part of this commit.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7df2cc24a3784add8370db859003783e92cbfc21

Reviewed-on: https://review.gerrithub.io/393834
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-10 09:51:31 -05:00
Sebastian Basierski
a70a000997 bdev: Renamed lvol->old_name to lvol->unique_id
Also added vbdev_get_lvol_by_unique_id function.

Change-Id: I55a64df008c23c0fedb8a59ef67e2c356097e780
Signed-off-by: Sebastian Basierski <sebastianx.basierski@intel.com>
Reviewed-on: https://review.gerrithub.io/392658
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-04 13:15:09 -05:00
Sebastian Basierski
3f41a8e506 bdev: Return aliases list through get_bdevs
Change-Id: Ic0cdcf088ebd5053f2e69ad2e607ee825d96fcb6
Signed-off-by: Sebastian Basierski <sebastianx.basierski@intel.com>
Reviewed-on: https://review.gerrithub.io/390202
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2018-01-04 13:15:09 -05:00
Sebastian Basierski
b9afa3c732 bdev: Added bdev aliases list.
Added aliases list to bdev struct.
Added 2 API calls to add and remove aliases.
Added test for adding and removing aliases.

Change-Id: I1815aec8c02cfa398b2d1de41577197315665fdc
Signed-off-by: Sebastian Basierski <sebastianx.basierski@intel.com>
Reviewed-on: https://review.gerrithub.io/390200
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2018-01-04 13:15:09 -05:00
Dariusz Stojaczyk
8b94174a4e virtio: fix full-virtqueue handling
Distinguish between "not enough descriptors available"
and "iovcnt > queue depth".

This patch brings back I/O re-queueing for virtio bdev
module. It was temporarily disabled by 451de7e1 [1].

[1] 451de7e1: "virtio: switch to the new virtqueue API"

Change-Id: I4c4f6a6d9d91373ee647ea7cafd53ad999aa6aa2
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/393447
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-03 14:35:36 -05:00
Dariusz Stojaczyk
724c9aa7d6 virtio: generalize PCI enumerate functions
Make them usable for other Virtio drivers
(not just SCSI).

Change-Id: I7ae2c43f311fefd40e447c8b5accaf824d0e23de
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/393121
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-29 16:02:20 -05:00
Dariusz Stojaczyk
7755bed389 virtio: split device initialization to separate functions
Previously we used to manually set
vdev->max_queues and called virtio_dev_restart
to go through all virtio init states, negotiate
features and allocate virtqueues. This is,
however, insufficient for Virtio-Blk, where we
e.g. need to check against negotiated multiqueue
flag before deciding how many queues we can use
(reading num_queues field from device config is
forbidden unless VIRTIO_BLK_F_MQ is negotiated).

This patch refactors queue-num related code
and also removes various restrictions. If device
supports less queues than requested, a warning
will be printed during initialization, but
the device will now continue to init normally.

The queue-num negotiation for virtio-user should
be eventually moved to upper layers, but that is
not necessary for now.

Change-Id: I418b56fa62c17b547243422ea077f0d76555bd13
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/393087
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-27 15:13:03 -05:00
Dariusz Stojaczyk
9dde5e5c27 virtio: add new library virtio
Exported lib/bdev/rte_virtio as a separate
library not dependent on bdev.

Virtio is now accessible via spdk_internal/virtio.h.
The header is marked `internal`, as it's
not meant to be used by end users. It's
not handy to handle all backend-specific
(e.g. Virtio-SCSI) logic in a user code.

For now the Virtio interface is publicly
exposed only via bdev_virtio module. We
might want to consider adding a separate,
public Virtio-SCSI library in the future.

Note: this patch doesn't do any changes
to the virtio code. Everything is
moved 1:1.

Change-Id: I805e5d12d265d82b0bc5784c89fbadb81abdb278
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/388166
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-26 13:03:29 -05:00
Daniel Verkamp
453f5ae9f6 bdev: unregister all bdevs in spdk_bdev_finish()
Instead of requiring each bdev module to track its own bdevs and clean
them up during its fini callback, we can walk the list of registered bdevs
during spdk_bdev_finish() and call spdk_bdev_unregister() on each one of
them before cleaning up the bdev modules.

Change-Id: I01816707c9100f66f542bfd73b90bcb0e0fb0c0c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/389878
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-12-21 13:29:29 -05:00
Jim Harris
f1f14e5583 bdev: pass mgmt_channel to spdk_bdev_get_io()
This prepares for some upcoming changes which will
add a per-thread bdev_io cache.

While here, remove spdk_bdev_get_io() from the
internal bdev API.  This function is not meant
to be called outside of bdev.c.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9f764a88a079fac936931c46d615999454013732

Reviewed-on: https://review.gerrithub.io/392529
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-12-21 10:42:47 -05:00
Daniel Verkamp
ea1c15791f log: rename SPDK_TRACE_* to SPDK_LOG_*
Disambiguate the log components from the trace functionality
(include/spdk/trace.h).

The internal spdk_trace_flag structure and related functions will be
renamed in a later commit - this is just a find and replace on
SPDK_TRACE_* and SPDK_LOG_REGISTER_TRACE_FLAG().

Change-Id: I617bd5a9fbe35ffb44ae6020b292658c094a0ad6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/376421
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-07 12:23:19 -05:00
Ben Walker
2b768821bf blob: spdk_bs_destroy now only zeroes the super block
The function just needs to zero out metadata so that the
blobstore is effectively destroyed. If the user wants
to unmap the rest of the disk after the blobstore is
destroyed, they are free to do so. On future initializations
of blobstores the code will do the unmapping, so performance
is not impacted.

While here, implement the zeroing using the new
write_zeroes functionality instead of allocating a buffer
full of zeroes.

Change-Id: I7f18be0fd5e13a48b171ab3f4d5f5e12876023bc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/390307
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-12-06 17:21:46 -05:00
Daniel Verkamp
54fb60177a lvol: clean up includes in lvol.h
All SPDK public API headers should include spdk/stdinc.h first.

Additionally, the other headers included in lvol.h aren't needed;
spdk/queue.h was unused, and spdk/blob.h can be replaced with a forward
declaration of struct spdk_bs_dev.

Change-Id: I89fad764efbb4c088ce325392f8237c1a869a895
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/389895
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-01 16:36:12 -05:00
Daniel Verkamp
c6d32d39af lvol: pass UUID as a string in public API
Modify the vbdev_get_lvol_store_by_uuid() API function to take the UUID
as a string instead of a uuid_t, since this simplifies the calling code.

This also allows us to remove #include <uuid/uuid.h> from the public
API, giving us the freedom to change the underlying UUID implementation
later if necessary without breaking API.

Change-Id: Ib360281f384f95722c5ab64e75dcf3603f826e4c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/389893
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-01 16:36:12 -05:00