This is just assert to assure the Klocwork
static analizer that wptr is not NULL.
Change-Id: I003620431afb8497a57c50af8480c34ec3fac6d6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462489
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The cache needs to be scrubbed during the initial device creation as
well as after power loss recovery. This patch extracts the scrubbing
code into a separate function.
Change-Id: I2cb32e6993a3531470f29f466d990f0d96e45def
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
The header is being written from multiple places, so having a discrete
function serializing and writing it at the appropriate place in the
cache makes sense.
Change-Id: I7a1e6ebd05e8a4974d141f04202803f507b978e4
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459620
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
This patch adds a function that marks all active open bands to be
flushed and once all of them are closed it notifies the caller. This
needs to be done before the data from non-volatile cache can be scrubbed
to make sure its stored on bands the device can be restored from.
Change-Id: I9658554ffce90c45dabe31f294879dc17ec670b9
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459619
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added ftl_dev_has_nv_cache to check if the FTL is configured to use
non-volatile cache or not. It makes these checks a bit more readable.
Change-Id: I0140df184d89a675e40bd5056718cd64301c553e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459617
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Wait until all user writes are completed before writing band's metadata.
Otherwise in case of power loss, user data might not get written while
the metadata does, which would result in data loss.
Change-Id: I419862960c072e38265b91d0d0498ff0c6f9f29e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459615
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Moving data from one band to the other doesn't need to be stored on the
non-volatile cache. Not only does it add unnecessary traffic to the
cache (wearing it out and reducing its throughput), but it requires us
to synchronize it with user writes to the same LBAs.
To avoid all that, this patch adds the FTL_IO_BYPASS_CACHE flag to all
writes coming from the reloc module. However, to be sure that the moved
data is stored on disk and can be restored in case of power loss, we
need to make sure that each free band have all of its data moved to a
closed band before it can be erased. It's done by keeping track of the
number of outstanding IOs moving data from particular band
(num_reloc_blocks), as well as the number of open bands that contains
data from this band (num_reloc_bands). Only when both of these are at
zero and the band has zero valid blocks it can be erased.
Change-Id: I7c106011ffc9685eb8e5ff497919237a305e4478
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458101
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Some of the writes doesn't need to go through the non-volatile cache
(e.g. relocations, data recovery from the cache). This patch adds IO
flag to indicate that the write shouldn't be stored on the non-volatile
cache.
Change-Id: I3d485fe14cf25b3074832f26491ba0cb12ff0e58
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458100
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Initialize children IOs with the appropriate LBA of its parent when
allocating internal IOs.
Change-Id: I191ad741b9d88d7f18cae05982e0a06a8f371f78
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458099
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
This patch adds tracking of the phase of the writes to the non-volatile
cache. The phase is changed each time the whole buffer is filled. Along
with every block's LBA, current phase is stored in its metadata. This
allows for replaying the sequence of writes when recovering the data
from the cache after (unclean) shutdown.
Since there are only three possible phases to be stored on the device at
a time, phase is defined as a 2-bit counter cycling through 1 -> 2 -> 3
-> 1, with 0 marking blocks that were never written.
Change-Id: Id47880367934027fd102c32f183110acc9d4c62a
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458098
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
After filling whole non-volatile cache, block all further writes until
the header with metadata is written. This means that metadata stored on
the device will always be up-to-date with the most recent write
sequence.
Change-Id: I15b724b52814289622374ce77e5c3b23173a75c6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
When creating FTL device using non-volatile cache, zero out the
non-volatile cache and store metadata (device's UUID, size of the cache)
in the first block.
Change-Id: Id8f212aef756e86e8a215582ab7c32a635e18938
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
In case ANM event occurs on open band there
can be situation that reloc will try to read
block on which there is ongoing write.
This is happening because lba valid map is
updated before write submission to allow sent
consistent metadata to disk before all user
writes are completed.
Added write offset to the each chunk and
add check to reloc if particular ppa is written
on that chunk.
Change-Id: Ic95a06e69381d2152a86984b65a0975afaff955d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458056
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Band tail PPA should be initialized when
new FTL instance is created.
Change-Id: Ie2fb72aa3f29eece0b6f8912998b33af3ba6b355
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457777
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Send ANM event to core thread for further
processing. This will remove a need of locking
in relocate module when ANM event occur.
Change-Id: I0efb1f1b8c96c107cda5fe78e8ee5672cde39f11
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Send LBA along with the data block when mirroring writes to the
non-volatile cache. The metadata buffer is retrieved from the metadata
pool, so the maximum number of concurrent requests is limited to
nv_cache.max_request_cnt, while the number of blocks in a single request
is limited by nv_cache.max_requets_size.
Change-Id: If260302d16039183fb0fe073ef7419947532cfab
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458093
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added basic configuration details (transport type and address,
parallel units, cache info) as well as most important OCSSD geometry
data to get_bdevs driver_specific section.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I00c50706bd9203bcef1701be1b7d87e93c10e57f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456790
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Gives the ability to change behavior of restore after dirty shutdown
without recompiling ftl library. User can define if partial recovery
or error should be returned after such a scenario.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I6dda40df7b92d6a377957e4a70a3eab91a6ac4a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Direct wptr allows for writing to band with externally
provided PPA, rather than one assigned by the wptr. This
can be useful when padding open bands during recovery or
bdev creation, as ftl can find partially written end meta
with no way to close the band with valid data, that would
conform to an already written CRC value.
This does mean that I/O sender is responsible for calculating
and obeying per-chunk write pointer arithmetics.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ifcbc4e8ffb69c829469a571bed8e27b148d7a651
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455511
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Almost all I/O callbacks in ftl utilize data in ftl_io, which is
initialized as callback argument in ftl_io_init_internal. This patch
changes the behavior to always returning the ftl_io struct in addition
to an extra opaque buffer.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I611ab1b33575f599798a2bb65c231a724c852c7f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455831
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Keep lba map related fields in separate struct directly
in the band. Cleanup interfaces depended on ftl_md.
Lba map structure will be extended in next commit.
Change-Id: I1cfc2f2ff0c0e90bb63f39808780845673002e70
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453370
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Split the IO in case io->iov_cnt exceeds the FTL_IO_MAX_IOVEC. It allows
the user to pass any number of iovecs. Each IO request is now split if
necessary and submitted via ftl_io_call_foreach_child submitting each
child individually. Any resubmissions need to be handled on a child
level, as parent request doesn't track its children submission status.
Change-Id: If54249d54225b34191216a366c227f79215abc90
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455527
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Extended ftl_band_read_lba_map() interface to provide
range of lba map to read.
Added ftl_xfer_offset_from_ppa() function for
calculating offset of read portion of lba map
based on ppa address from IO. This offset is
used for coping read lba map portion to internal
ftl buffer.
Change-Id: I1c72a18c79eda8c33cd0b20ea36a5d9521a09d06
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449435
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This assert is not needed anymore.
Change-Id: I7bc85c980c2e120b105b35763b9bea5960564acc
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456590
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Keep an array of iovecs tied to each IO. Internal IOs can have
iov_cnt > 1 without having to allocate additional memory. User's iovec
can now be modified too (e.g. when splitting the request).
Change-Id: Iec73c6dad59acdc331a1461fd7feac085138a8de
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Replaced single IO pointer with a queue. This allows multiple requests
to be queued when the writes cannot be scheduled.
Change-Id: I0cde50e7378108a6aab4ef6fe601cb854244f5d5
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455524
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
On the user IO path, the flow always follows the same procedure:
allocate ftl_io and initialize it using ftl_io_user_init. There's no
point in separating these two actions.
Change-Id: I578347ccc7c85e5945368dd6bf76c58985af3939
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455523
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There's no point in synchronously returning an error from ftl_io_write
/ ftl_io_read as they're also reported in the IO's completion callback.
All errors are now reported through io->status.
Change-Id: I4adf4e13221b63715625276042e1172cd63b8f9b
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455521
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Select the method of choosing next PPA to read inside ftl_submit_read,
as it can be deduced from other arguments.
Change-Id: I6cbd37c2c6d7f073bf735491b4960d72e327c4e6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455520
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Some of the internal IOs split the contiguous data buffer into multiple
iovecs to separate the requests. It's unnecessary and can be replaced
by checking the position within the data buffer.
Change-Id: I9255ea0072fee6c4e6a7dca21e4008780bc45610
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Advance parent's IO position when children positions are advanced.
There's no need to call ftl_io_advance twice for the same request
anymore.
Change-Id: I1f7f04a3a83575b11e7bf0b13c0c8f3bf98b5522
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Padding requests are issued in multiple batches at the same time
when shutdown occurs. This allows for faster removal.
Change-Id: Iea40d2418bedbd7cf3c6865e5eb8f85871db13cd
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454578
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For the latest TLC NAND, one write buffer unit (rwb batch)
needs to be spread over three PUs instead of being allocated
to a single PU for better sequential read performance
since the optimal write size(ws_opt) of 3D TLC NAND is
3 times bigger than the optimal read size(rs_opt).
I added num_interleave_units in 'struct spdk_ftl_conf'
to configure the number of interleaving units per ws_opt.
If num_interleave_units is set as 1, the whole of the ws_opt
blocks are placed sequentially around each PU.
If num_interleave_units is set as N, the 1/N of the ws_opt
blocks are staggered. So consecutively numbered blocks
are separated by ws_opt / num_interleave_units.
The sequential read performance is improved from 1.9GiB/s
up to 2.97GiB/S with this patch on our system. No performance
degradation is observed on sequential writes or
4KB random reads/writes.
Please refer to the Trello card for more details.
https://trello.com/c/Osol93ZU
Change-Id: I371e72067b278ef43c3ac87a3d9ce9010d3fcb15
Signed-off-by: Claire J. In <claire.in@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450976
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Apart from writing the data to OCSSD, mirror the latest two bands of
data on the persistent write buffer cache. Currently the data is only
sent there, further patches will add metadata support, shutdown recovery
and L2P updates.
Change-Id: Ief05d0c23fa0e25bd6085e0ce3e1528d6736d174
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450266
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Removed unused 'context' parameter. Removed 'lbk' parameter from
next_ppa function signature, as it could be easily deduced from ftl_io
passed as the first argument.
Change-Id: I2b2b699ffc1b282086adcc2d4dc6c4691a87a7c5
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454228
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Replaced all direct accesses to io->lba / io->iov with getter functions,
as they both should behave differently based on the FTL_IO_VECTOR_LBA
flag and io->iov_cnt respectively.
Change-Id: I0d387a2a908cc37afa9e1b97ca5b408cef0d6b08
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Moved early read completion (due to cache hit or reading unwritten
data) traces to ftl_submit_read to keep them in one place. Also removed
the context parameter from next_ppa function, as it was unused.
Change-Id: Ic69174a6cdc0e626550cd673789e86bb891094bf
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453683
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Replaced the unnamed unions with named ones. It makes the assignments
easier (it allows for direct assignment instead of having to choose the
correct union member).
Change-Id: I130233a096070092275a040e73cbaf36106ae1ab
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This fixes incorrect LBA reported by traces (it reported the next one,
as the IO position was advanced before storing the trace).
Change-Id: I788853d729dd25e90c6c899be0616aa3a79d08b3
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453681
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are no guarantees of the order of completion
of NVMe IO submission queue so wait for write completion
on specific chunk before submitting another write to it.
To control chunk occupancy split IO to child requests
and release chunk in IO completion callback.
Change-Id: I44147a21b528a7f33fb92b9e77d7de8f5b18f8ff
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449239
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Claire Jihyun In <claire.in@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I7a7667e1c4e6321150e5bc8eb4287fb61847ccb5
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Extended the IO descriptors with the ability to group them together in a
parent / child(-ren) relationship. It allows to delay parent completion
until all children are done and enables the children to have additional
context in their completion routines (as opposed to having a single IO
describe all requests).
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6ad111dc231813e397697109a333e2a5d7f5941d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch is starting point for metadata refactor
which is needed for efficient ANM events support.
Change-Id: I81d864605e69008d8e3922fb61adf504187447a1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Added means to configure libftl to use (optionally) another bdev as
persistent write buffer cache.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97028a681be168d9386eac8a226631ff772f803b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448629
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
