Before this patch reading out the extent pages during
blobstore replay was serialized. Only issuing reads for next
extent page when previous operation finished.
This was done by continously calling _spdk_bs_load_replay_extent_page_cpl()
and decreasing ctx->num_extent_pages.
This patch changes spdk_bs_sequence_* to spdk_bs_batch_*.
All the reads are submitted at once, and only when all of them
finish we proceed to next valid md chain.
Goal of this change is improving efficiency and readability.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I807cdb98166e04706fedb494363f5776e3151827
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1540
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This variable currently holds single extent page.
Further patch will utilize it to use multiple.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If86ffd57cecf5d3bfd0812a767c784d7bf503fb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This field does not hold actual pages, but just md page numbers
which hold the extent pages.
Rename as prepartation to adding new one that will hold actual
extent pages.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1fb85a58c92a93b968e1fad22e421252399e9281
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch adds more ways to back off when parts of
blob persist fails.
Otherwise the process would proceed as if nothing happened.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7cff73e1dc3066d0c822d1e3dac4bd35e27cd54a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1263
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
After opening the blob for deletion, in _spdk_bs_delete_open_cpl(),
the blob is removed from list of blobs in blobstore.
This is to prevent future _spdk_blob_lookup()s from referencing
blob while it is deleted.
In usual blob deletion path, next step is proceeding with deletion
of the blob by reducing its size to 0 and syncing the blob.
Changes from this point forward are persisted.
Meanwhile in special case of deleting snapshot which has single clone
on it, before above occurs additional steps are performed.
Each of the blobs are opened and their attributes changed.
Failures on those steps are fully recoverable on any errors,
and in such case blob should be added back to the bs list of blobs.
Original code had condition on how many references there were
to blob being deleted, which is incorrect.
Any error on that path should clean up after itself (revert attributes
and close blobs) and re-add the blob.
This change is tested with blob_delete_snapshot_power_failure() UT,
by adding error path in persist - which triggers error in aforementioned
blob delete code path.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I926e7cbf3cb86170c69f31231399535859f290dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/985
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When creating snapshot, 'original' blob will end up being a thin provisioned clone.
Before that first thin_provisioned 'newblob' is created during this process.
If the first md sync for 'newblob' fails, it means that only valid references to
clusters are still only present in 'original' blob. The 'newblob' can be safely
cleaned up.
Unfortunetly 'newblob' inherited some of 'original' blob properties before sync.
Cluster maps were already swaped in current cleanup code. But during blob close
of 'newblob' - persist blob code expects clusters to be 0 only for thin_provisioned
blobs. If original blob was thick, then it triggers an assert within persist code.
This patch makes sure to set thin_provision to 'newblob', to align with its creation.
Added asserts to verify that clusters maps are 0's, which should be the case
as I/O to origblob is frozen.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5420617792aefe8a3ef4e5989b2056504cdd1850
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1394
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will help with making the _spdk_blob_persist_write_extent_pages()
batch all writes of extent pages.
No functional change occurs with this patch, this is just refactor
for future change.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8c93b1d6473db660f7ad5e04c8ec9f3331b2055c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/986
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
super->clean value signifies if blobstore was unloaded
cleanly.
If it was not, then during bs_load the _spdk_bs_recover()
procedure if called.
Meanwhile bs->clean is always set to 1 after load, causing very first
blob_persist to also re-write super block with the super->clean
set to 0. To signify that md has changed and possibly trigger
the recovery if clean bs unload does not occur.
When the re-write of super block succeeds the bs->clean is set to 0,
because further re-writes of super block are not needed on next
blob persist.
This patch resolves issue when:
1) reading super block fails - execution should backoff, to prevent
writing an empty buffer as super block !
2) writing super->clean = 0 to the super block fails - execution
again should fail, and bs->clean should not be set to 0. It will
cause next persist to attempt re-write again.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia07cc5c6c107310059b50886edb7283c176b9169
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In general it is not possible to delete snapshot when
there are clones on top of it.
There is special case when there is just a single clone
on top that snapshot.
In such case the clone is 'merged' with snapshot.
Unallocated clusters in clone, are filled with the ones
in snapshot (if allocated there).
Similar behavior should have occurred for extent pages.
This patch adds the implementation for moving EP from
snapshot to clone along with UT.
The UT exposes the issue by allowing delete_blob
to proceed beyond just unrecoverable snapshot blob.
Fixes#1291
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib2824c5737021f8e8d9b533a4cd245c12e6fe9fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1163
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Further part of the series will hold array of md pages
in the ctx. Callers of _spdk_bs_load_replay_md_parse_page()
will make select which page to parse.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3fb70660672ba74bdb338eb1233409103903b215
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/983
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is refactoring change for future patches.
struct spdk_bs_load_ctx will contain array of pages
instead of single one. Having to change just single
line for selection of page will make it easier to
read next patches in series.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If3dc1e7da7e61c7b4866307d859e55131a32d38b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/982
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It is possible for multiple blob persists to affect one another.
Either by blob->state changes or blob mutable data.
Safe way to prevent that is to queue up the persists.
Next persist will be executed only after previous one completes.
Fixes#1170Fixes#960
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iaf95d9238510100b629050bc0d5c2c96c982a60c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
_spdk_blob_persist_check_dirty() function will be
called in subsequent patch at the end of persist
in _spdk_blob_persist_complete() to proceed
with any queued up persists.
Please see following patch for this.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ieeb334e23cde329743647f728e70dd60333c224a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
With recent changes to extent on-disk metadata format,
new format (Extent Pages) is not backwards compatible.
Meanwhile old format (Extent RLE) is backwards
compatible with older SPDK applications.
Summing up:
Blobstore created pre SPDK 20.01 can only use Extent RLE.
Blobstore created starting with SPDK 20.01 can use both,
Extent Pages and Extent RLE specified by use_extent_table opts.
When use_extent_table is set to true, invalid flag for it is set.
SPDK application pre 20.01, will not load such blob.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If14ebd03f19eb581d71dcb46191e099336655189
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483220
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This was observed after running nighly tests on previous patch.
As part of it, autopackage.sh compiles SPDK
without debug flag set. Exposing the uninitialized var here.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iedb1641f3c0d4a21f293c81cd4fcf35c6d1c7ae5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482893
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Extent table and extent page descriptors are now
set to be default way clusters are serialized on disk.
With this patch UT are ran with and without
extent table.
Changed two asserts in test, since amount is dependent on
which type of serialization is used.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica58fce6a4effd014d7dd40ee26edd0fa3196d0f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
ctx->extent_page signifies if page was allocated
for insertion.
1) It is possible for a thread to claim extent page
on its own thread, and put it in ctx->extent_page.
If conflicting thread allocates another ctx->extent_page,
then it should be freed. This does not mean failure
to insert cluster. As different threads could have
been trying to allocate different clusters,
so condition on line 6716 does not cover it.
If so then it shouldn't be an issue to release
the claimed ctx->extent_page and proceed with updating the
extent page which originally won the race.
NOTE: if clusters were conflicting, then extent_page is
freed in _spdk_blob_insert_cluster_cpl().
2) At this point of _spdk_blob_insert_cluster_msg()
we already verified that there already is
extent page allocated at "*extent_page".
In such case ctx->extent_page will be 0,
and should not be used.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id5b57c88248890eee60d2e7dbecbd984c98b561b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
sz is set to number of clusters that should be have been
in particular unallocated EP (remaining_clusters_in_et
up to SPDK_EXTENTS_PER_EP).
The cluster array should be set to 0 only in region
between original size (cluster_array_size) and new
total size (active.num_clusters).
It was incorrectly using sz in the memset.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic43e89c17d53e9529e3ed0349aeb4fb7dc6593f2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Previously part of function assumed that cluster count
1)means number of clusters in EP and another 2) that it is
following the active.num_clusters (akin to extent_rle).
This was incosistent and showed when using multiple
extent pages to serialize metadata.
This patch changes it to only go with 1), so it is clear
that it means number clusters within particular EP.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I155104cabc127ed47df04434032fb01e08948e13
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482848
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Following changes are done in this patch:
1) _spdk_blob_serialize_extent_table now persists
at least single extent page. When num_extent_pages == 0.
2) Minimum valid size of ET descriptor is even without
extent_pages. This is a case when there are no EP,
but we still want to persist num_clusters in ET.
3) Taking above points, redone the loop for serializing
extent pages.
4) Make sure to mark blob dirty if any new extent pages
were allocted.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I81dc6cf2de2722bb49927ed42f4b9f31292f78c5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482847
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Changed assert checking if cluster array is allocated
when loading extent pages. This is true only for
the first extent page being loaded, of course after that
the cluster array can be already allocated.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0f96294ede5a12ffd6bca73cbeadba8d94a35bac
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482857
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Size of a blob (thus size of clusters array in mutable data)
is known from extent table descriptor.
Extent pages were read sequentially in order they were
placed in extent table. This meant that cluster
array could have been filled up from beginning to end.
Yet reading extent pages in any other order,
would result in incorrect placement of clusters.
This patch adds first cluster index that is contained within
each extent page. This will allow to read/write
multiple extent pages in parallel, since
we will know where in clusters array to put the cluster idxs.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib6b9332111cd93f990d057dc60624152907dd87f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482701
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is more adequate name, since this value if first read from
Extent Table descriptor. Then decreased when iterating over entries in
extent table and extent pages are read.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib188c524b8488b38d4de063a9970dcfdf49c9acd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482600
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When replaying md chain for a blob, extent table
descriptor can be read. When it is present, all allocated pages
it points to are now being put into extent_pages array in ctx.
If multiple extent table descriptors are in single md chain,
the array is expanded accordingly.
After replaying single md chain is done, replay extent pages
starting from last one. Replaying extent pages, is similar to
extent_rle in that each allocated cluster is claimed and
number of free clusters in blobstore decreased.
When all extent pages are read, return to
_spdk_bs_load_replay_md_cpl() for continuing replaying
next valid md chain.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4573226aff7d7b1bcdfd188518235c8d4b68a4c3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481621
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
_spdk_bs_load_replay_md_parse_page() is only used in
replay path during blobstore load.
Next patch will expand the load ctx with array of
extent pages to be read. It is filled out when reading
in-chain metadata of extent table descriptors.
Passing the load ctx here will make it simpler to
fill out the array when processing extent table.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If96e6670560c8c4a3610f33ece14c354d7d5da39
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When EXTENT_TABLE descriptor is found when parsing metadata
that means there can be extent pages to read.
If extent page was not allocated, number of clusters can be
increased depending on the num_clusters_in_et.
Unallocated extent page contains either SPDK_EXTENTS_PER_EP
or remainder of num_clusters_in_et worth of clusters.
Depending which is less.
Added decreasing fo num_clusters_in_et to parsing
extent pages as well.
While here, remove ctx->seq = seq assignment as that is
done at beginning of blob load.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I57f54634b908ffb406f3e91e15841b7f36fd6de6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476429
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add new serialization of changed extent pages before persisting md.
Iterate over active extent pages (not array !). When they are
allocated but not yet present on disk - write them out.
All extent pages in clean mutable data are assumed to be written out
already.
So there are two cases here:
1) Active mutable array is larger than clean
All allocated extent pages should be written out.
2) Cluster allocation created new extent page
Blob has to be thin provisioned and persist was called
as part of cluster allocation. New extent page needs to be
written out and EXTENT_TABLE allocated.
Iteration is done over num_extent_pages instead of extent_pages_array_size,
to prevent writting out too many extent pages when size of blob was
made smaller. The two values come back in sync at the end of persist
either way.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I780819fd7f3c44e4cf5d71c188c642536d3cc320
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479851
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Right now output from _spdk_bs_cluster_to_extent_page()
is used to determine whether the exten_table is used at all.
If NULL pointer was returned this meant that extent table
was not allocated, even if the code might suggest just
checking if we overran the array.
To make it more obvious, the _spdk_bs_cluster_to_extent_page()
now only asserts the extent_table_id.
blob->use_extent_table is now always used to determine the
serialization path.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9d2630645213539bae5cd1d72e5f9b878f53c2bc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482599
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch add single EXTENT_PAGE updates on cluster allocations.
There are three possible outcomes after inserting a cluster:
1) blob uses EXTENT_RLE
Proceed to usual sync_md.
2) blob uses EXTENT_TABLE and extent page was not yet written out
Update the active mutable data to contain the claimed md page,
write out the EXTENT_PAGE and sync_md to update EXTENT_TABLE.
3) blob uses EXTENT_TABLE and extent page was previously written out
Only serialize that single EXTENT_PAGE and write out the updated
cluster map for it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia057b074ad1466c0e1eb9c186d09d6e944d93d03
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470015
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Similar to EXTENT_RLE, this descriptor holds LBA of clusters.
Difference is that EXTENT is kept in separate md pages,
and only single EXTENT will be updated on cluster allocation.
This patch adds the EXTENT processing, which is not used
until following patch.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifbac23db7ca3e7c8c91cee01018f20071f0d5160
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Added claiming the extent page.
Which is then followed by updates in updates
of mutable data on md thread.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If511564f812685381c48924310105a4cb6f63cd1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479850
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Functions to claim and release md pages were added.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1c8ddc13c8a5806fb874e5c34dae2a327e1ff248
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482011
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
With this patch extent pages array will change it size accordingly
to size of the blob. Similar to clusters, only resizing up
is done on blob resize. Shrinking is done on persisting the blob.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id7f7c81efbd96af414fce9fc4045cbb476cc93a6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479962
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Extent Pages claim and insertion can be asynchronous
when cluster allocation happens due to writing to a new cluster.
In such case lowest free cluster and lowest free md page
is claimed, and message is passed to md_thread.
Where inserting both into the arrays and md_sycn happens.
This patch adds parameters to pass the Extent Page offset
in such case.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I46d8ace9cd5abc0bfe48174c2f2ec218145b9c75
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Added new descriptor SPDK_MD_DESCRIPTOR_TYPE_EXTENT_TABLE.
Extent Table will hold md page offsets for new Extent Page descriptor.
Entries in Extent Table are run-length encoded 0's as unallocated
Extent Page descriptors.
Additionally total number of clusters is persisted in each Extent
Table descriptor. This is because there is no guarantee that
last Extent Page of a blob will be allocated.
Even if number of Extents per Extent Page is always the same,
Extent Page can hold less Extents than that.
This patch does not add more metadata on disk right now.
Only added descriptor parsing/serialization and applicable fields
to store it in run time.
Following patches are going to implement TODO's added in this patch.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iac5d8f00ddfc655c507bc26d69d7adf8495074e9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466920
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Lets get it removed ! :)
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I91b994a883a642d87ecc8c152c801b8a7676f33a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is an additional option that can be passed when creating
a blob.
When opts->enable_extent_pages is set to false (current default),
only EXTENT_RLE should be persisted on sync.
During blob load, when EXTENT_RLE is present in md,
blob->extent_rle_found is set to true.
When opts->enable_extent_pages is set to true,
only EXTENT_TABLE and EXTENT_PAGES should be persisted on sync.
During blob load, when EXTENT_TABLE is present in md,
blob->extent_table_found is set to true.
It is possible to find neither EXTENT_* descriptor when loading a blob.
This means that blob length is 0 and EXTENT_RLE was supposed to be used.
Yet none were persisted due to lack of clusters.
In such case blob->use_extent_table is set to true after finishing
blob load.
When parsing metadata ends, if extent_table_found is set - then
support for extent_table is enabled. All other cases disable it.
At this time path for Extent Pages is not implemented, so it should
not be used.
Later in the series, it will become the default path for serialization.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2146da6130a0645e686ab02a3b5d2d86a7d35a1f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479853
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Replaying md through _spdk_bs_load_replay_md_cpl() starts with
md page 0 in search of first valid md page starting a chain
for particular blob.
When it is found, next pages read are from the current pages
`next` page - next in chain.
After whole chain is read, it goes back to first page in chain
and starts search for next valid chain from there.
This patch adds separation between reading particular chain,
and moving to the next one.
Moving on to the next one happens in _spdk_bs_load_replay_md_chain_cpl().
Further in the series, extent pages will be added in the metadata.
Those are not within any particular blobs chain of metadata,
but spread out over the md region.
It is not enough to read all md and read extent pages.
In case of power failure, only extent pages known to be valid
are the ones which are pointed to by some valid md chain.
In futher patches, a step will be added after reading particular
valid md chain to go read extent pages pointed by it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6e7cd64af66ce5db0abd2ad5962d604ac2b30994
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481900
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Moved finishing of unloading to separate function,
which is now called on every failure and success when unloading
the blobstore.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I34539b78c5cc63a6fe5891014cba89b9eb62d4df
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482009
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Before this change it was possible to fail at
writing out some of used md pages.
bserrno output of those was not verified.
This patch adds it at every step.
With that two function don't need (and never needed)
to pass the bserrno:
_spdk_bs_load_write_used_md()
spdk_bs_load_complete()
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1a61763f03665ba1b00e5949ef0cf37eefaaf08f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482008
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is simplification of load path.
seq is save in ctx already, no need to pass it to the function.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ief0ddc1826c461adbad71ba1a3897c510ec2a971
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482007
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Assigning seq to ctx was done very late in the process.
To keep future functions lean and without the seq,
it is assigned immidietly after starting.
Only functions in load path that require separate
seq argument are those passed directly to read/write
device operations.
Rest of them can just use spdk_bs_load_ctx.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2bd610dc4c7b4a7b0c3de92391922475c514326a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481899
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
No functional change is done in this patch.
Most of the functions already translate cb_arg to ctx and
use it, but then just pass cb_arg.
This will make it clear that it is ctx that is passed around.
Along with simplifying some of changes in next patch,
where arguments of functions will be cut down just to
the ctx.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If7d8ed38dc92175d867a2231ab2ebd4f2499efcd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482006
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
This patch creates new _spdk_blob_persist_generate_new_md()
function that is responsible for generation of new metadata
from current state of blob.
Functionality so far is unchanged.
This is preparation for later in the series where new
extent pages will be written out to disk before metadata pages.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I84158cb8316a881a6170ac37e151a60aaa9d7369
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479848
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
On blob load, read in the saved clear_method option. If
BLOB_CLEAR_WITH_DEFAULT was passed in, use the setting stored
in metadata previously. If something other than the default was
specified, ignore stored value and used what was passed in. If
ignoring a stored value, print a warning.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia0c81fa0adc175dfaeb74c06e1ac91dc6b27e9ab
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Accept a clear method option on blob create by adding clear_method
to the opts structure passed in to _spdk_bs_create_blob(). Store
these 2 bits in md_ro_flags so that earlier versions without an
understanding of these bits can not alter metadata.
The new metadata values will be used later in the series.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5440645ca20b426778d13b2e544b65dc2b3b83c7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In current version, immidietly following parsing all
metadata pages an action is taken inform of loading the back_bs_dev.
Patches later in the series will add more metadata in form of
extent pages, which have to be read separetly from usual
blob metadata pages.
This patch add separation between the two steps,
so later a device read can be put between.
Additionally, _spdk_blob_load_final() when no snapshot was present
passed bserrno which was always 0. This patch just sets 0 directly
there as no errors occured at that point.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4a77527f90bb1de12f972591067b7a50926f39c9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476427
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Just to make all sizes consistent and less error prone.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id0a21bbd45954a0f2317e0eefd3725f1542ef04f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479961
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Fixed size of check in _spdk_bs_snapshot_newblob_open_cpl().
Rest are just to make all consistent and more error prone.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5a23a7795f1e598c1cfd6d17ce37b367f2f34df8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479960
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The _spdk_bs_page_to_lba() [without 'md'] is only for translating the
pages on the blobstore to lba they are at. Those pages start at
the begining of the device and cover all of it. Thus simple
math is enough to translate those.
It is used to calculate lba_count for set of pages as well.
Meanwhile there are 'md_pages' which are the same pages as for
the above, but their count start at bs->md_start.
Which is right after super_block and couple pages for bit masks.
This patch creates new _spdk_bs_md_page_to_lba() that is more
explicit in what page number is passed. Hopefully avoiding
confusion when reading which page number refers to which
'type' of page.
Exception to the that is _spdk_bs_dump_read_md_page(), where
blobstore is not actually loaded (md_start from super block
is not copied to bs structure).
Additionaly providing assert to catch errors on debug builds.
Making the check in _spdk_blob_load_cpl() for max_md_lba obsolete.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I66bbca55b5ca3d6794c462d50177e6037ddbefa6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479017
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Always when creating a snapshot, new blob is created.
That blob is explcitly set as thin provisioned with size of
the original blob in _spdk_bs_snapshot_origblob_open_cpl().
Thus it should always contain empty cluster map,
as API user has to interaction with it yet.
As sanity check for debug builds, verification if all clusters
are 0's is added.
This empty cluster map is later swapped into the original blob
in _spdk_bs_snapshot_swap_cluster_maps().
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4b935c0cf08917e9ad7b9bbedac4781890626eec
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478974
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Refactor blob loading when snapshot is present.
All paths now go through _spdk_blob_load_final().
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifc927de6800501cdf62dba8d73e950af2a46d568
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479143
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Since all error paths for blob load are now the same,
they can go through common function to handle
freeing and calling the original cb.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib3afc7e62b6f9c872bb1d5f72ef61170aee966d7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479142
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Originally the code was suposed to determine if
loading the blob succeeded, based on passing the
cb_arg.
This breaks the logic of always getting the cb_arg in
cb_fn, and basing the success on bserrno.
In order to fix this, cb_fn always gets the passed
cb_arg. Meanwhile the cb_fn (_spdk_bs_open_blob_cpl(),
now checks the bserrno to determine failure.
In addition since _spdk_bs_open_blob() was the original
caller allocating the blob structure,
the _spdk_bs_open_blob_cpl() is now responsible for freeing it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic7eb09f05e04b08dc54fc43243fd576f493cbeb2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479141
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The sequence was saved into the load context much later
into the loading, instead of right when ctx is allocated.
This will come in handy in later patches that refer to
sequence earlier (in error paths).
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibe513dbd919f36874fcde763fc96d46973b60446
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479140
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When serializing extents, run-length encoding is supposed to
1) RLE all sequential LBAs
2) RLE zero LBAs (unallocated)
There is one special case, with sequential LBAs that start
with 0 LBA. This is RLE as 1) case, but results in descriptor
matching case 2). Which causes loss of allocated clusters.
This requires following conditions to be met:
- blobstore has just a single cluster reserved for MD
- blob is thin provisioned
- first allocation occurs on cluster_num=1
For last part to be true, very first write for blob has to be
issued to LBA between cluster_size and 2*cluster_size.
Causing allocation of second cluster in blobstore and assiging
it LBA equal to number of LBAs per cluster.
To fix this, case 1) disallows to RLE zeroes.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I136282407966310c882ca97c960e9a71c442c469
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475494
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In prep for storing a clear_method in the blob metadata:
* Set the default to DEFAULT and let the switch statement choose
UNMAP
* Use switch statements to make it clearer which method we are
using and why. (ie previously we set the default to UNMAP and
then had an UNMAP || DEFAULT condition to choose UNMAP.
Later in the patch series it will become clearer why this makes
sense.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I216cb97fd8eaa772437a36c2c7a47e66618bbfbd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Originally serializing extent_rle was always done as last step.
There was no need to update the buffer pointer,
since it went unused.
Next patches in series expand serialization to new descriptors,
so here the assumption is removed and buf/buf_sz is updated.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7ccfb500d64e4276359cc98c5587c6301272d728
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468232
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch simplifies return path when returning from
serialization of extent_rle.
Both paths will share more code in upcoming patch.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibb0ebcfe4377fe09709345d580d54050b61d3c88
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468231
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In future patches new type of extents will be added,
for compatibility the current extent type will be still
handled in the code.
To signify the difference between those two types,
current type is renamed to SPDK_MD_DESCRIPTOR_TYPE_EXTENT_RLE.
Along with any variables throughout the code,
to make it clear which ones are used.
There are no functional changes in this patch.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7186ccc452d200036188abf1dcea9660dcedee72
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468230
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This change moves the code related to serializing
extents into serparate function, in order to allow
more clear changes in further patches.
There are no functional changes in this patch.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If8d7c90a5b01f1608d20fd00c3e4ff6a340ce305
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This variable went unused, since logic in
_spdk_blob_persist_write_page_chain() already dealt with
writing metadata from last to first page.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic70c47df1ea3bb01c8031244339c42e9936f28b0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
SPDK_ERRLOG lists the function name, so remove old references that
assume it doesn't and reprint the function name.
Change-Id: I69da6ca0a25bf0eda07d8dad52bcfadf964ac715
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Scan-build has a real issue with reallocs. The original
error from latest version of scan-build is rather complicated,
but it can be greatly simplified with the following change:
> diff --git a/lib/blob/blobstore.c b/lib/blob/blobstore.c
> index 7580c9dd2..6a594edf3 100644
> --- a/lib/blob/blobstore.c
> +++ b/lib/blob/blobstore.c
> @@ -1147,8 +1147,9 @@
> _spdk_blob_persist_clear_clusters_cpl(spdk_bs_sequence_t *seq, void *cb_arg, int
> } else if (blob->active.num_clusters != blob->active.cluster_array_size) {
> tmp = realloc(blob->active.clusters, sizeof(uint64_t) * blob->active.num_clusters);
> assert(tmp != NULL);
> - blob->active.clusters = tmp;
> - blob->active.cluster_array_size = blob->active.num_clusters;
> + ctx->blob->active.clusters = tmp;
> + assert(ctx->blob->active.clusters[0] != 14213);
> + ctx->blob->active.cluster_array_size = ctx->blob->active.num_clusters;
> }
>
> _spdk_blob_persist_complete(seq, ctx, bserrno);
> ```
Scan-build will then complain:
blobstore.c:1151:10: warning: Use of memory after it is freed
assert(ctx->blob->active.clusters[0] != 14213);
Asserting blob == ctx->blob, blob->active.clusters == ctx->...,
or even tmp != blob->active.clusters doesn't work, so use the
last resort scan-build weapon - #ifdef __clang_analyzer__.
The realloc in this case is just down-sizing a buffer to
save some memory. For scan-build, just don't do it. This
finally silences all scan-build false positives.
Change-Id: Ib88ea145370f5035eedd2412e98ee61f96ad1915
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
md_start and md_len are values in pages rather than lba.
Those should not be compared against lba of currently
loaded md page.
This patch changes assert to verify if the lba of current
page does not exceed max lba where md is expected to be.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id445eb9871f82f7fe367bfc396f1b495591511c1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Blob id only is matched to the very first page of md for
that particular blob.
During loading blobstore, we shouldn't verify
further pages in chain against the blobid.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifc7863ddcb403aedc264c14e6b4c3915bd30dc41
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Length of xattr descriptor is equal to length of xattr struct,
xattr name and the len of stored value.
There is no limit to how much can be stored in memory for xattr.
On disk xattr size is limited to single page and within that to
max descriptors that can fit in it.
This size is known at compile time.
Before this patch it was possible to add xattr exceeding
what was possible to be written to disk. This caused issues
when serializing the metadata during spdk_blob_sync_md()
or spdk_blob_close(). Making those fail without specific info
to the user and not actually writting such descriptor.
Since maximum length of xattr descriptor is known at compile time,
this patch compares against this value when setting the xattr.
It will immediately report back to user with error, and will
not store xattr in memory (thus not serialize it).
This patch should not affect any backward compatibility for blobs.
Too large xattrs weren't written to disk before,
API for blobstore stays the same - only reporting ENOMEM
when it should.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6f4af4d079e47f084e20d7a4969d9a78ec1f8610
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Currently we are missing cleanup routine for case when
power failure interrupts creating snapshot. This patch
add such routine.
For the case where we find blob with a parent snapshot ID
matching newly created snapshot we can finish whole process
during recovery by processing forward with setting snpashot
as read only, removing xattr and syncing. We should remove
snapshot only if there is no blob with parent pointing at
snapshot.
Fixes github issue #760
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2f0e298164e07a2b4dfa5367e8878facef640702
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In _spdk_bs_snapshot_origblob_sync_cpl function on error
path we should not close snapshot as it will be closed during
volume closing when bs_dev is being destroyed.
This issue was found in unit test (see next patch in series).
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I51c38d1f1f97b134679251b43109b1265e565a17
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455215
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.
In such case snapshot can be removed without any data
copying. This is achieved with following steps (case
with only one clone):
1. Open snapshot (Snapshot1) that shall be removed
2. Check if the Snapshot1 has no more than 1 clone (Clone1)
3. Remove Clone1 entry from Snapshot1
4. If the Snapshot1 has a parent snapshot (Snapshot2):
4a. Add Clone1 entry to the Snapshot2 clones list
4b. Remove Snapshot1 entry from Snapshot2 clones list
5. Open Clone1 blob
6. Freeze I/O operations on Clone1
7. Temporarily override md_ro flag for Snapshot1 and Clone1
for MD modification
8. Merge Snapshot1 and Clone1 clusters maps into Clone1
clusters map
9a. If Snapshot2 is present switch parent ID and backing
bs_dev on Clone1
9b. If Snapshot2 is not present set parent ID to
SPDK_BLOBID_INVALID and backing bs_dev to zeroes_dev
10. Sync MD on Clone1
11. Sync MD on Snapshot1
12. Restore MD flags for Clone1 and Snapshot1
13. Unfreeze I/O on Clone1
14. Close Clone1 blob
15. Remove Snapshot1
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I800724b981af894e01e1912d0077c5b34a2ae634
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
In some cases user may want to flag blob for removal
then do some operations (before removing it) and while
it happens there might be power failure. In such cases
we should remove this blob on next blobstore load.
Example of such usage is delete snapshot functionality
that will be introduced in upcoming patch.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I85f396b73762d2665ba8aec62528bb224acace74
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch moves _spdk_blob_set_thin_provision function
higher in the file as it will be later used during
blobstore load.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ife37ef8c69b88903646b2002b3561101c1eb5135
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455488
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When new writes come from different threads, cluster allocations can
happen many times at once. The corresponding cluster number for the map
is determined via _spdk_bs_allocate_cluster() and kept in ctx->new_cluster.
The cluster itself is inserted into the map only on md_thread.
When there is conflict of two threads allocating same cluster,
message is returned to the losting thread to release the cluster.
Before this patch, on such failure the cluster to release
was calcualted from the page. This resulted in releasing the
cluster claim for thread that actually won it.
This patch makes it so that cluster allocated and save in ctx
is used instead.
Change-Id: Id10811b887f673f9b89e41e0637d4422f1d7270d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3858c637d421b58e74fa5573d257e59fed92824a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452268
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch add new _spdk_bs_delete_blob_finish function
which will be helpful in future changes.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2d492b6102f33ad35b7b6fe408f709f54b7b2341
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452251
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds new function which is used to check
if blob can be removed when requested.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Iafa82fba9bf67ffd15cf639f4665087f054b6b7d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452242
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch creates new function that will be helpful
with further implementation of 'delete snapshot'
feature.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I66f138ba217fb4a4186f2703900a2952cdb8e438
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452240
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
At some point this parameter may have been a bool, but it's
not now - so we need to explicitly pass SPDK_BLOB_READV or
SPDK_BLOB_WRITEV.
Otherwise we end up trying to write the iovec array as the
data buffer when executing this operation later - since we
passed "false" which is treated as 0 which is
SPDK_BLOB_WRITE (not SPDK_BLOB_WRITEV).
Fixes issue #603.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I50df736216c4a88b40604722c021e204c7fac623
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452157
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Other threads are referring to this memory.
Change-Id: I9689a1b64a1f032ad479ec85095d2575dcab3a7f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452130
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
_spdk_bs_blob_list_remove function returns only 0
so it can be changed to void function.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I1827d783a7cd18caf831275554e33f398b0b962f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451783
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Block operation that should not be done simultaneously.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I3cab510377a49be4e5847ba37a6218f0025c0db6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450014
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Currently someone can create clone while blob is being
opened for deletion. This patch moves check for blob if
it is a snpashot after the blob is opened.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ia4b8b7cb956522a29784aa349d677eb87886e4db
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451682
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Previously, when creating a snapshot in blobstore the snapshot's cluster map
was copied from the "original" blob, with the original's map zeroed. These
operations are both O(num_clusters*cluster_size/page_size) while io
operations are frozen. This change replaces the linear operation with an
O(1) pointer swap at the critical moment that io is frozen, while
doing the zeroing before the freeze when preparing the snapshot to
minimize freeze time.
Change-Id: I1e468bc97623f5da161a8ddba1393c271acd3aed
Signed-off-by: Amir More <habeanf@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451486
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is a part of future changes to block blob operations
that may cause race conditions between each other.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ia728d1fc207375ddcb3b70b5081ddcffa9f99027
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
spdk_dma_*malloc() is about to be deprecated.
Change-Id: I0da485ec68329ee9f819cf6dfdb8744c5993e6f2
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Default 'unmap' option stays as it was.
'Write_zeroes' comes useful when one wants to make sure
that data presented from lvol bdevs on initial creation presents 0's.
'None' will be used for performance tests,
when whole device is preconditioned before creating lvol store.
Instead of performing preconditioning on each lvol bdev after its creation.
Change-Id: Ic5a5985e42a84f038a882bbe6f881624ae96242c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442881
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Some users require to do write zeroes operation when
erasing data on lvol. Currently the default method is
unmap. This patch adds flag to spdk_rpc_construct_lvol_bdev
call that changes default erase method. This is also a base
implementation for possible future function for erasing
data on lvol bdev.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8964f170b13c2268fe3c18104f7956c32be96040
Reviewed-on: https://review.gerrithub.io/c/441527
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Everything need to be removed to get the remove callback called.
Otherwise we will end up with dangling devices and user callbacks
possibly not called.
Fixes#567
Change-Id: I37259f6cd97268060170a6b17a0c0df4d543a224
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If IO fail e.g. during hotremove error shouldn't be ignored as this will
trigger operations (like crc checking) that shouldn't be done. Also
false error messages are printed.
Change-Id: Ie023ddcd9bdba2378e69808302ff9978497c7852
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440889
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Replaced divide_round_up() from blobstore.c, lvol.c and reduce.c with
new spdk_divide_round_up() from util.h.
Change-Id: I013383ac286ca52b5c15c7fab4fb40ad97b92656
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/437649
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When last clone of a snapshot is being deleted
we remove that snapshot from snapshots list.
We should not do that as it still works as a
snapshot and it is read-only, but it does not list
as a snpashot from get_bdevs. Instead remove snapshot
entry from the list when blob that represents that
snapshot is being removed.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8d76229567fb0d9f15d29bad3fd94b9813249604
Reviewed-on: https://review.gerrithub.io/436971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
spdk_bit_array_find_first_set() returns UINT32_MAX if no
set bits are found. But spdk_bit_array_find_first_clear()
would return the size of the bit array instead in this case.
(Note: the comments say size of the bit array + 1 which was
incorrect)
So this patch makes spdk_bit_array_find_first_clear()
consistent with spdk_bit_array_find_first_set() and returns
UINT32_MAX if no cleared bit is found.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I44c1e674149f8c2e87122800d5db45a2851f0bef
Reviewed-on: https://review.gerrithub.io/428225
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I994b5d46faffd34430cb39e66225929c4cba90ba
Reviewed-on: https://review.gerrithub.io/414935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch just adds the call, but doesn't change behaviour of blobstore.
io unit size remains same as page size.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idcd1b7d5126fc7cacf12c996188bd41e2c9a744d
Reviewed-on: https://review.gerrithub.io/425355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is a string name used for debugging only.
Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This is in response to a Scan-build error with Clang 6.0 but is a real
bug. If we don't match up our IOVs properly with the lenght we provide,
we could jump over the iov list into invalid memory in
_spdk_rw_iov_split_next.
Change-Id: I472a7aa53027af0a18ebeee8226e0b008447cce4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424248
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
