Commit Graph

31 Commits

Author SHA1 Message Date
Paul Luse
442ee2303e blobstore: fix issue with reading in superblobid from disk
Completion routine for reading superblock was not updating
the bs struct with the superblob id thus it would have
failed to be persistent from the application's perspective.

Change-Id: I4aa51ebe73315e9be7e08f82340b03f0e3836df7
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/373406
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-08-11 12:58:47 -04:00
Ben Walker
60c38d4022 bdev: Change unmap to use offset/len instead of descriptors
This is far simpler, although it does limit the bdev
layer to unmapped just one range per command. In practice,
all of our code reports limits of just one range per command
anyway.

Change-Id: I99247ab349fe85b9925769e965833b06708d0d70
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/370382
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-04 20:03:37 -04:00
Dariusz Stojaczyk
73358c99e1 util/nvme: added io_device unregister callback
Patch afe860ae deferred freeing the io_device. However, for nvme, the
io_device context (spdk_nvme_ctrlr) is still being destructed before
io_channels are destroyed, causing segfaults on hotremove.

This patch defers io_device context destruction and fixes nvme
hotremove.

Fixes: afe860aeb1 ("channel: Correctly defer unregisters if channels exist")
Fixes: 5533c3d208 ("util: defer put_io_channel")

Change-Id: I7af699174cac0c6c6a6faa2cc65418c47347eb9a
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/370459
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-07-20 16:07:02 -04:00
Jim Harris
be9a3b9f69 bdev: pass descriptors for I/O operations
This enables checking permissions - for example,
spdk_bdev_write will fail if the descriptor was not
created with write permissions.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I68b65a560f471f2e0f71a7f42cfa6689b911110f

Reviewed-on: https://review.gerrithub.io/369493
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-07-14 13:31:30 -04:00
Tomasz Zawadzki
c303ba2c16 blobstore: close bdev on destroy
Now spdk_bdev_create_bs_dev() opens the underlaying spdk_bdev.
Due to that spdk_bdev should be closed when bs_dev is destroyed.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0805f29abfeb52ff1db067bad7b7e0f13fc39398

Reviewed-on: https://review.gerrithub.io/368351
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-07-06 12:44:27 -04:00
Jim Harris
57d174ff67 bdev: add spdk_bdev_open/close
Retire the old claim/unclaim semantics in favor of
open/close.  Clients must now open a bdev to get
an spdk_bdev_desc, then pass this desc to get an
I/O channel.

This allows multiple clients to open a bdev,
although only one may open a bdev with write
access.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4d319f1278170124169a8a75fd791e926b3f7171

Reviewed-on: https://review.gerrithub.io/367611
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-07-05 18:04:28 -04:00
Daniel Verkamp
4671369ab2 blobstore: fix declaration after statement
Change-Id: I1439b471b101b390fcbef558039f2a543f465acd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/367121
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-06-27 13:57:06 -04:00
Ben Walker
aff7e226f7 bdev: spdk_bdev_[read|write|flush|unmap|reset] now return int
It is not actually useful to be immediately returned
a handle to the bdev_io. There isn't anything valid
that the user can do with it at that point. Instead,
return an integer error code.

Change-Id: Iffa9a8dc5b2eefab57e3cc1f68919985431d17d1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/364137
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-06-08 16:35:49 -04:00
John Meneghini
8a44220b1a env: Rename spdk_malloc/zmalloc/realloc/free to spdk_dma_(func)
- rename spdk_malloc_socket to spdk_dma_malloc_socket
  - rename spdk_malloc to spdk_dma_malloc
  - rename spdk_zmalloc to spdk_dma_zmalloc
  - rename spdk_realloc to spdk_dma_realloc
  - rename spdk_free to spdk_dma_free

Change-Id: I52a11b7a4243281f9c56f503e826fd7c4a1fd883
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/362604
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-05-31 15:30:27 -04:00
Ben Walker
f198553530 blob: Coalesce unmaps
Previously, each freed cluster was unmapped individually.
Instead, coalesce unmaps for contiguous clusters to reduce
the volume of commands sent.

Change-Id: I6ea1d2e1235e3c030cd2826c97e57aca571bd2ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/362773
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-05-26 17:59:38 -04:00
Ben Walker
d969ac445a io_channel: Remove per-channel priority
This wasn't used anywhere and we currently believe there
are superior software-only techniques for controlling
quality of service.

Change-Id: Icdadd5870ed0629b338c307d2619bbc242c3e7a3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/362065
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-26 13:42:19 -04:00
Daniel Verkamp
b9bcc3531e bdev: make enum spdk_bdev_io_status private
The user should not see the bdev_io status directly; the NVMe and SCSI
error code wrappers provide the ability to translate to the desired
format regardless of what kind of error is stored inside the bdev_io.

Replace the spdk_bdev_io_completion_cb status parameter with a bool
simply indiciating whether the I/O completed successfully.

Change-Id: Iad18c2dac4374112c41b7a656154ed3ae1a68569
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/362047
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-23 16:30:10 -04:00
Ben Walker
305cb239d2 io_channel: Remove unique flag
This is no longer used anywhere. For the places where we previously
used it, we've since found alternate solutions that do not
require it.

Change-Id: I738a80b95ef50348ce1c14969a3812b0a625b3fd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/362064
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Ben Walker
60e8fb4979 blob: Use a separate md and I/O target
This allows us to allocate different size channels and
not require the unique flag.

Change-Id: I4b1ffd244b60e9e9ab06f9ab4da8161ab57e1169
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/361668
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Ben Walker
4eda4fd245 Fix incorrect free in blob/blobfs
The channel memory isn't allocated by these
libraries, so they can't free it.

Change-Id: I30909fa4e77bc5a41b45230f04ba5fe75b172dbf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-18 14:36:11 -07:00
Ben Walker
4a3182b891 blob: Add a separate config parameter for channel ops
Separate the maximum metadata operations from the
maximum channel operations.

Change-Id: I1bbd440ab094a2a2e19c9a5b71724ac91ba88e42
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-18 14:36:11 -07:00
Ben Walker
bea2e2308f blob: Remove per-channel queue size configuration
This will need to be configured globally for all channels.

Change-Id: I773252f220373617f8d09d1f24243db8095cf8a4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-18 14:36:11 -07:00
Daniel Verkamp
76e67baf28 blob_bdev: use bdev block size getter
Change-Id: Id22f904fdb71798ef2369415d38f468b06d7ac07
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
326786a943 bdev: add getters for block size and block count
Change-Id: I6fad28da43c163ea4e2c4a04ced356b67d63652f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-12 10:34:58 -07:00
Ziye Yang
4c9645b555 blobstore: Add the size check of length field in desc_xattr
Change-Id: I522b29b3a603b1e83c866486d036137ee4fafcbd
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-05-09 08:41:53 -07:00
Ben Walker
b961d9cc12 include: Move the remainder of the code base to stdinc.h
Change-Id: I6a142feeaad3117bd3c75e7c5cb7231a1cfa78ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-08 13:20:36 -07:00
Daniel Verkamp
6b91943209 scripts/check_format.sh: check spaces in comments
Fix up all existing spacing errors in comments and add an automated
check for patterns like /*comment*/.

Change-Id: I28f61c93612dc0f8aed66bd509da78e91ea9737e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-04-24 13:37:41 -07:00
GangCao
e5a3193b80 blob: remove duplicate call to _spdk_blob_free
Change-Id: Iecb7f28474861043e6d4971c78ed5764b60181af
Signed-off-by: GangCao <gang.cao@intel.com>
2017-03-29 09:16:45 -07:00
Ben Walker
57e6e0c8be blob: Initialize new metadata pages after allocation
The descriptor type must be 0 to break out of the loop,
so we need to initialize this.

Change-Id: I5fdb24dcfece01332c487364d5694c4fb8412e1b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-28 16:51:29 -07:00
Ben Walker
9cdb51d693 blob: Free blob memory on error
Change-Id: I67d3c359b7d432fa06660175a4d533115236d640
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-28 16:20:09 -07:00
Ben Walker
97b3efa349 blob: Verify parsing does not run off end of page
Change-Id: I2663e0518705ffd23afbb69c7ae5bf5aff001c85
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-28 16:17:54 -07:00
Ben Walker
abc73f6995 blob: Handle failure to alloc xattr on parse
Or rather, at least assert that the allocation failed.
This is not a recoverable error in general.

Change-Id: I9bc325066e829fc311ce84ce83536e9933ac5473
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-27 15:15:24 -07:00
Ziye Yang
66aabffcae blob: free the blob in error case
Change-Id: I645e01123949fc4707509d0626f8a5327b7bdd3c
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-03-27 14:38:25 -07:00
Ben Walker
d827cfbde6 blob: Handle allocation failures in set xattr
Change-Id: Ida9361066e8b3117d1022379d75e290b11f86b52
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-27 13:54:54 -07:00
Jim Harris
1edd9bf3e4 blobfs: Add a lightweight filesystem built on the blobstore
This is the initial commit for "blobfs", a lightweight
filesystem built on top of the SPDK blobstore.

Also included in this patch:

1) a shim for using SPDK bdevs as the backing store for
   SPDK blobstore/blobfs
2) documentation for using blobfs as the storage engine
   with RocksDB
3) scripts for running a set of workloads and collecting
   profiling data with RocksDB and blobfs

See doc/blobfs/getting_started.md included in this commit
for more details on blobfs, including some of the current
limitations.


Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2a6d3d4b87236730051228ed62c0c04e04c42c73
2017-03-24 14:15:45 -07:00
Ben Walker
d89352a95c blob: Add a persistent, power-fail safe block allocator
This is the initial commit for the "blobstore", a lightweight,
highly parallel, persistent, power-fail safe block allocator.

Documentation will be added in future patches.

Change-Id: I20a4daf899f1215d396f7931c3ec9a2e2bb269d0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-21 14:21:21 -07:00