When recovering the data from the non-volatile cache, the data inside
the volatile cache needs to be flushed before flushing active bands.
Otherwise, if the number of blocks in a band is smaller than the number
of blocks inside the volatile cache, part of the data may not get
flushed.
Change-Id: I4e99709c8c2a526a928578870d7fbd5fef37db02
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466883
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Moved scrubbing of nv cache to core thread. Added IO channel which is
used in user context, during shutdown of nv cache.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I88e680324e361bf7e0c0a9a9d29323f179c56e3b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Track the band under defrag inside the reloc module. This allows for
multiple bands being defragged at the same time (e.g. extra one due to
write fault) as well as makes it easier to handle cases when relocating
a band that has no valid blocks.
Change-Id: Ia54916571040f5f4dfdb8f7cdb47f28435a466d8
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465937
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Padding size calculation should take into account
already acquired write buffer entries in case
number of blocks left for closing band is less than
write buffer size.
Change-Id: I0f9c9bc94eb5a4736eede30ec424c81103e1256f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463525
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Changed initialization of io channels to be made in the core/read
thread themselves. This fixes potential hangs when using the ftl lib
directly, without bdev and initializing an ftl device from a different
thread than will be used for IO.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I7089cc46cc6de5e9008e7c22cc01d2b4103ebb14
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464891
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If an ANM event has come to a chunk/sector that has been marked as
high_prio (eg. after a write failure), there's no need to add any
sectors for relocation, as the whole band should be marked for a move
anyway.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I57824ba7e592540ea639f9f011149210fd688a5a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463289
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Remember current write position within the cache when shutting down. It
allows for faster recovery after clean shutdown (no need to scan the
device) as well as grants a quick way to distinguish between clean and
dirty shutdowns.
Change-Id: I79c22caa0b1ca4373951ac43f747b085d331cdd0
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460796
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
In case shutdown occurs during band relocation
we can have situation that on retry queue are
some internal IOs and we need to wait to drain
this queue before we start shutdown sequence.
Change-Id: Ie5b33732568aeda0586f4de384b22dcf463bf31e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460421
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Changed write_config to use the ftl configuration file instead of
allow_open_bands directly. This allows for easier expansion of saved
fields.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ie1d480983636f26840dba9a3a6c93b2883903bdb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460519
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Fixed touching NULL write pointer, which would happen after some other
I/O in the same band would receive error and issue its deletion. Any
subsequent write callbacks would then be unable to find and update the
information.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I07d9791f1b1b673a48356fae55b784c0bf6a6f7a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is just assert to assure the Klocwork
static analizer that wptr is not NULL.
Change-Id: I003620431afb8497a57c50af8480c34ec3fac6d6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462489
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The cache needs to be scrubbed during the initial device creation as
well as after power loss recovery. This patch extracts the scrubbing
code into a separate function.
Change-Id: I2cb32e6993a3531470f29f466d990f0d96e45def
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
The header is being written from multiple places, so having a discrete
function serializing and writing it at the appropriate place in the
cache makes sense.
Change-Id: I7a1e6ebd05e8a4974d141f04202803f507b978e4
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459620
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
This patch adds a function that marks all active open bands to be
flushed and once all of them are closed it notifies the caller. This
needs to be done before the data from non-volatile cache can be scrubbed
to make sure its stored on bands the device can be restored from.
Change-Id: I9658554ffce90c45dabe31f294879dc17ec670b9
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459619
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added ftl_dev_has_nv_cache to check if the FTL is configured to use
non-volatile cache or not. It makes these checks a bit more readable.
Change-Id: I0140df184d89a675e40bd5056718cd64301c553e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459617
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Wait until all user writes are completed before writing band's metadata.
Otherwise in case of power loss, user data might not get written while
the metadata does, which would result in data loss.
Change-Id: I419862960c072e38265b91d0d0498ff0c6f9f29e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459615
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Moving data from one band to the other doesn't need to be stored on the
non-volatile cache. Not only does it add unnecessary traffic to the
cache (wearing it out and reducing its throughput), but it requires us
to synchronize it with user writes to the same LBAs.
To avoid all that, this patch adds the FTL_IO_BYPASS_CACHE flag to all
writes coming from the reloc module. However, to be sure that the moved
data is stored on disk and can be restored in case of power loss, we
need to make sure that each free band have all of its data moved to a
closed band before it can be erased. It's done by keeping track of the
number of outstanding IOs moving data from particular band
(num_reloc_blocks), as well as the number of open bands that contains
data from this band (num_reloc_bands). Only when both of these are at
zero and the band has zero valid blocks it can be erased.
Change-Id: I7c106011ffc9685eb8e5ff497919237a305e4478
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458101
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Some of the writes doesn't need to go through the non-volatile cache
(e.g. relocations, data recovery from the cache). This patch adds IO
flag to indicate that the write shouldn't be stored on the non-volatile
cache.
Change-Id: I3d485fe14cf25b3074832f26491ba0cb12ff0e58
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458100
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Initialize children IOs with the appropriate LBA of its parent when
allocating internal IOs.
Change-Id: I191ad741b9d88d7f18cae05982e0a06a8f371f78
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458099
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
This patch adds tracking of the phase of the writes to the non-volatile
cache. The phase is changed each time the whole buffer is filled. Along
with every block's LBA, current phase is stored in its metadata. This
allows for replaying the sequence of writes when recovering the data
from the cache after (unclean) shutdown.
Since there are only three possible phases to be stored on the device at
a time, phase is defined as a 2-bit counter cycling through 1 -> 2 -> 3
-> 1, with 0 marking blocks that were never written.
Change-Id: Id47880367934027fd102c32f183110acc9d4c62a
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458098
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
After filling whole non-volatile cache, block all further writes until
the header with metadata is written. This means that metadata stored on
the device will always be up-to-date with the most recent write
sequence.
Change-Id: I15b724b52814289622374ce77e5c3b23173a75c6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
When creating FTL device using non-volatile cache, zero out the
non-volatile cache and store metadata (device's UUID, size of the cache)
in the first block.
Change-Id: Id8f212aef756e86e8a215582ab7c32a635e18938
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
In case ANM event occurs on open band there
can be situation that reloc will try to read
block on which there is ongoing write.
This is happening because lba valid map is
updated before write submission to allow sent
consistent metadata to disk before all user
writes are completed.
Added write offset to the each chunk and
add check to reloc if particular ppa is written
on that chunk.
Change-Id: Ic95a06e69381d2152a86984b65a0975afaff955d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458056
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Band tail PPA should be initialized when
new FTL instance is created.
Change-Id: Ie2fb72aa3f29eece0b6f8912998b33af3ba6b355
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457777
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Send ANM event to core thread for further
processing. This will remove a need of locking
in relocate module when ANM event occur.
Change-Id: I0efb1f1b8c96c107cda5fe78e8ee5672cde39f11
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Send LBA along with the data block when mirroring writes to the
non-volatile cache. The metadata buffer is retrieved from the metadata
pool, so the maximum number of concurrent requests is limited to
nv_cache.max_request_cnt, while the number of blocks in a single request
is limited by nv_cache.max_requets_size.
Change-Id: If260302d16039183fb0fe073ef7419947532cfab
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458093
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added basic configuration details (transport type and address,
parallel units, cache info) as well as most important OCSSD geometry
data to get_bdevs driver_specific section.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I00c50706bd9203bcef1701be1b7d87e93c10e57f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456790
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Gives the ability to change behavior of restore after dirty shutdown
without recompiling ftl library. User can define if partial recovery
or error should be returned after such a scenario.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I6dda40df7b92d6a377957e4a70a3eab91a6ac4a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Direct wptr allows for writing to band with externally
provided PPA, rather than one assigned by the wptr. This
can be useful when padding open bands during recovery or
bdev creation, as ftl can find partially written end meta
with no way to close the band with valid data, that would
conform to an already written CRC value.
This does mean that I/O sender is responsible for calculating
and obeying per-chunk write pointer arithmetics.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ifcbc4e8ffb69c829469a571bed8e27b148d7a651
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455511
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Almost all I/O callbacks in ftl utilize data in ftl_io, which is
initialized as callback argument in ftl_io_init_internal. This patch
changes the behavior to always returning the ftl_io struct in addition
to an extra opaque buffer.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I611ab1b33575f599798a2bb65c231a724c852c7f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455831
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Keep lba map related fields in separate struct directly
in the band. Cleanup interfaces depended on ftl_md.
Lba map structure will be extended in next commit.
Change-Id: I1cfc2f2ff0c0e90bb63f39808780845673002e70
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453370
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Split the IO in case io->iov_cnt exceeds the FTL_IO_MAX_IOVEC. It allows
the user to pass any number of iovecs. Each IO request is now split if
necessary and submitted via ftl_io_call_foreach_child submitting each
child individually. Any resubmissions need to be handled on a child
level, as parent request doesn't track its children submission status.
Change-Id: If54249d54225b34191216a366c227f79215abc90
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455527
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Extended ftl_band_read_lba_map() interface to provide
range of lba map to read.
Added ftl_xfer_offset_from_ppa() function for
calculating offset of read portion of lba map
based on ppa address from IO. This offset is
used for coping read lba map portion to internal
ftl buffer.
Change-Id: I1c72a18c79eda8c33cd0b20ea36a5d9521a09d06
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449435
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This assert is not needed anymore.
Change-Id: I7bc85c980c2e120b105b35763b9bea5960564acc
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456590
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Keep an array of iovecs tied to each IO. Internal IOs can have
iov_cnt > 1 without having to allocate additional memory. User's iovec
can now be modified too (e.g. when splitting the request).
Change-Id: Iec73c6dad59acdc331a1461fd7feac085138a8de
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Replaced single IO pointer with a queue. This allows multiple requests
to be queued when the writes cannot be scheduled.
Change-Id: I0cde50e7378108a6aab4ef6fe601cb854244f5d5
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455524
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
On the user IO path, the flow always follows the same procedure:
allocate ftl_io and initialize it using ftl_io_user_init. There's no
point in separating these two actions.
Change-Id: I578347ccc7c85e5945368dd6bf76c58985af3939
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455523
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There's no point in synchronously returning an error from ftl_io_write
/ ftl_io_read as they're also reported in the IO's completion callback.
All errors are now reported through io->status.
Change-Id: I4adf4e13221b63715625276042e1172cd63b8f9b
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455521
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Select the method of choosing next PPA to read inside ftl_submit_read,
as it can be deduced from other arguments.
Change-Id: I6cbd37c2c6d7f073bf735491b4960d72e327c4e6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455520
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Some of the internal IOs split the contiguous data buffer into multiple
iovecs to separate the requests. It's unnecessary and can be replaced
by checking the position within the data buffer.
Change-Id: I9255ea0072fee6c4e6a7dca21e4008780bc45610
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Advance parent's IO position when children positions are advanced.
There's no need to call ftl_io_advance twice for the same request
anymore.
Change-Id: I1f7f04a3a83575b11e7bf0b13c0c8f3bf98b5522
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Padding requests are issued in multiple batches at the same time
when shutdown occurs. This allows for faster removal.
Change-Id: Iea40d2418bedbd7cf3c6865e5eb8f85871db13cd
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454578
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For the latest TLC NAND, one write buffer unit (rwb batch)
needs to be spread over three PUs instead of being allocated
to a single PU for better sequential read performance
since the optimal write size(ws_opt) of 3D TLC NAND is
3 times bigger than the optimal read size(rs_opt).
I added num_interleave_units in 'struct spdk_ftl_conf'
to configure the number of interleaving units per ws_opt.
If num_interleave_units is set as 1, the whole of the ws_opt
blocks are placed sequentially around each PU.
If num_interleave_units is set as N, the 1/N of the ws_opt
blocks are staggered. So consecutively numbered blocks
are separated by ws_opt / num_interleave_units.
The sequential read performance is improved from 1.9GiB/s
up to 2.97GiB/S with this patch on our system. No performance
degradation is observed on sequential writes or
4KB random reads/writes.
Please refer to the Trello card for more details.
https://trello.com/c/Osol93ZU
Change-Id: I371e72067b278ef43c3ac87a3d9ce9010d3fcb15
Signed-off-by: Claire J. In <claire.in@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450976
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Apart from writing the data to OCSSD, mirror the latest two bands of
data on the persistent write buffer cache. Currently the data is only
sent there, further patches will add metadata support, shutdown recovery
and L2P updates.
Change-Id: Ief05d0c23fa0e25bd6085e0ce3e1528d6736d174
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450266
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Removed unused 'context' parameter. Removed 'lbk' parameter from
next_ppa function signature, as it could be easily deduced from ftl_io
passed as the first argument.
Change-Id: I2b2b699ffc1b282086adcc2d4dc6c4691a87a7c5
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454228
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
