There's one of these per core and it's a collection of other
jobs to run, so name it reactor.
In the longer term, this reactor will go away and each "job"
will map to a separate spdk_thread.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ica5b19e6edb2509b48059d829e22a734acd6b298
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1381
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Actually we can just use one API to finish the requirement.
Change-Id: Ia0d3d589755e8c92f636d3d090ec642299511401
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1280
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This will eliminate one calloc() and add one small code cleanup.
Change-Id: I3d67f2b073a46e04304ae5342974afecf532794b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The revert asynchronous API doesn't run as the *real* asynchronous
way, because the drive can only support synchronous module and only
1 session is supported. The reason why we added this API is that
RPC call has the default timeout value here, while the revert may
take over several minutes, the API itself doesn't short the revert
action, so just remove it and use the synchronous API instead.
The revert action will erase all the users data and bring the drive
back to the factory state, it should run in the synchronous mode,
so just remove the asynchronous API and we can increase the timeout
value when using RPC to call this API.
Change-Id: I08a082edea6385e378399423bbb229d05f8bc262
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1232
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The APIs even don't have a chance to be called if the drive can't
support OPAL feature, so just remove the NULL check, and we will
store level 0 discovery descriptors after initialization, so
spdk_opal_cmd_scan() isn't necessary here, remove it as well.
Change-Id: I24f12f2c352996d9ebe76dc015cd0b7502798359
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1231
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The previous erase locking range API didn't take the real
erase action, it's kind of secure erase, so rename it with
another name and add the real erase support.
This method is used to cryptographically erase user data within
a specific LBA Range and to reset the access control Locking
of that LBA Range.
The TPer SHALL reset the ReadLockEnabled, WriteLockEnabled,
ReadLocked, and WriteLocked column values to False for the
Locking object on which the method is invoked.
Change-Id: I0c83df589382b0a2f189642d8119e389aa4bc559
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1210
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are maximum 8 locking ranges per TPer, so we don't need to use
calloc() for each range.
Change-Id: Ie9d96ba28736dd7a91cb615ece99ebcefa695fc8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Both spdk_opal_key and opal_common_session are used in the same context,
so we don't need to save spdk_opal_key pointer in opal_common_session,
juse use the stack value.
Change-Id: I24b64711b45f09a60f9aca1175319e32de4c7e10
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1193
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For active key we don't need to store in the calloc() data
structure, because we'd better not to store any password
related structure, and it's only used in specific context,
so just use stack variable instead.
Change-Id: I4711e279ffe9a97d471f0db0b625b95a36b55d38
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Inform the user that something actually happened
during the test, rather than just displaying
"Test start/end".
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I69a048a5a2fd47a3f18ea2988a06aee439c06fc3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
"ls" is not needed in for loops based on filename
expansions.
Additionally "ls" was treated as first
element of for loop which cased "No corresponding
object" message to be printed which could be
mistakenly considered a problem during the build.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If5bfdb15410da36c98d9992cd04a02906599a3e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1411
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This will allow to test parts of compress bdev on systems
that do not have all types of PMDs.
Once case if for testing in VMs.
For reference on changes in execution time. Current times:
Per-patch = 8 seconds
Nightly = 84 seconds
Running the script twice in this patch, most likely doubles the time.
It is not an issue in per-patch, but if increasing nightly times is blocking
different solution could be done here.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I684569651f2290956f94fcec99cebf0fe13ba0d2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1364
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Sending large amounts of data from host to the controller with 1s
admin polling time, take a lot of time (e.g. 1M firmware file in 4k
chunks takes ~17min).
Reducing this time to 10ms whole operation takes about 3s.
Change-Id: I2dabe9f60acab57e348c34bfabc3cc7479dedec9
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1393
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In case there are no /dev/nvme* devices found, the test will continue
regardless trying to pass literal glob strings as an argument to nvme.
This, of course, fails, e.g:
ci.spdk.io/public_build/autotest-per-patch_7066.html
Avoid this by generating list of the nvme devices and abort the test
gracefully in case it's empty.
Change-Id: I71628aa78a5cfc6c5696f1fc17ed3480d4c642ee
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1392
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This will help with making the _spdk_blob_persist_write_extent_pages()
batch all writes of extent pages.
No functional change occurs with this patch, this is just refactor
for future change.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8c93b1d6473db660f7ad5e04c8ec9f3331b2055c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/986
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
super->clean value signifies if blobstore was unloaded
cleanly.
If it was not, then during bs_load the _spdk_bs_recover()
procedure if called.
Meanwhile bs->clean is always set to 1 after load, causing very first
blob_persist to also re-write super block with the super->clean
set to 0. To signify that md has changed and possibly trigger
the recovery if clean bs unload does not occur.
When the re-write of super block succeeds the bs->clean is set to 0,
because further re-writes of super block are not needed on next
blob persist.
This patch resolves issue when:
1) reading super block fails - execution should backoff, to prevent
writing an empty buffer as super block !
2) writing super->clean = 0 to the super block fails - execution
again should fail, and bs->clean should not be set to 0. It will
cause next persist to attempt re-write again.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia07cc5c6c107310059b50886edb7283c176b9169
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
time.clock() was depreceted since Python 3.3 and was finally
removed in Python 3.8.
https://docs.python.org/3.8/whatsnew/3.8.html#api-and-feature-removals
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ffe372b61c8afc0ddbbee0064b95059fa184616
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This change verifies that not only it is possible to open
the snapshot, but also the spdk_bs_create_snapshot() reported
success.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifb7fcc91802c838710f1f9be41090057268a5900
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
blob_create_snapshot_power_failure() test always attempted
snapshot creation until the snapshot did open.
Which meant that on bs reload after dirty shutdown,
the original blob might be in unrecoverable state.
UT did not verify create call beyond that point.
Next patch will stop after snapshot is possible to be opened,
AND the spdk_bs_create_snapshot() returned success.
This patch makes it easier to recreate same starting
conditions after snapshot did not open.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1901b29a8319b2203a855e7879821ed3850f4917
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In general it is not possible to delete snapshot when
there are clones on top of it.
There is special case when there is just a single clone
on top that snapshot.
In such case the clone is 'merged' with snapshot.
Unallocated clusters in clone, are filled with the ones
in snapshot (if allocated there).
Similar behavior should have occurred for extent pages.
This patch adds the implementation for moving EP from
snapshot to clone along with UT.
The UT exposes the issue by allowing delete_blob
to proceed beyond just unrecoverable snapshot blob.
Fixes#1291
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib2824c5737021f8e8d9b533a4cd245c12e6fe9fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1163
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
By default "make install" installs pmdk header files in
/usr/local/include while autotest scripts expect them
to be in /usr/include.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I906ffd8a7858a37525758c3fc8129cb22286a8f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1368
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This patch is used to add the async network I/O support in
sock layer. If code is configured with I/O uring, --with-uring,
we can use io uring in Linux (version >=5.4-rc3).
PS: We also make VPP's default priority > uring, because
for the iSCSI or sock test linked with VPP. It tests VPP with
a given address (which is not a special VPP can only open address),
so using uring can also listen those address succefully. And if we make
uring with priority > VPP, actually, VPP will not tested in those cases.
Additionally, the current CI pool is not ready for test, we need
wait for the CI system ready. And I test on my local platform, it works.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie8ee3c8ddf8d2a7264f2b382376733e002816dcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/952
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This will be used by upcoming spdk_top application.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I066567f0f14d70e6744821e8e805934a3d790882
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If nvme_rdma_qpair_submit_sends() returns -ENOMEM,
nvme_rdma_qpair_process_completions() returns immediately.
In this case, nvme_rdma_qpair_process_completions() does not
poll CQ.
However, nvme_rdma_qpair_process_completions() can poll CQ even
when there is no free slot in SQ.
Hence move nvme_rdma_qpair_submit_sends() and
nvme_rdma_qpair_submit_recvs() after the loop to poll CQ.
nvme_rdma_qpair_submit_sends() and nvme_rdma_qpair_submit_recvs()
output error log and so checking return code of them is not
necessary and is removed in this patch.
This fixes part of the github issue #1271.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icf22879c69c3f84e6b1d91dc061b6f44237eedd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1342
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is another preparation to extende spdk_thread_poll() to update
thread stats correctly on multiple SPDK threads per CPU core
configuration as an new function spdk_thread_poll_ext().
As another preparation, factor out main loops of spdk_thread_poll()
to _spdk_thread_poll(). _spdk_thread_poll() will be called in
spdk_thread_poll_ext() in the next patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie006359c00afc2e9b90e15f918f76cf1af0b7ce1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Subsequent patches will extend spdk_thread_poll() to update
thread stats correctly on multiple SPDK threads per CPU core
configuration as an new function spdk_thread_poll_ext().
Thread stats will be updated separately by spdk_thread_poll() and
spdk_thread_poll_ext(), but updating thread stats itself is
the common operation.
Hence as a preparation, factor out updating thread stats operation
from spdk_thread_poll() to _spdk_thread_update_stats().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I93c40eb0363ca34f2150299b09c49f1401583832
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1242
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The shutdown flag is only used when resubmitting the AER, and it will not
be updated when hot remove happened, so rename it to is_destructed.
Change-Id: Iafc27bd6cb23a851ed6c96470a2a45546a399c88
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1254
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch does drop in replacement of SN32_LT and SN32_GT by
spdk_sn32_lt and spdk_sn32_gt, respectively.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5df1e461d2b25a2f20222e823fb1ec68ced049ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1347
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Previously the macro SN32_LT and SN32_GT had been added to iSCSI
library. But such comparisons may be used in other libraries.
So add two inline helper functions spdk_sn32_lt() and spdk_sn32_gt()
to include/spdk/util.h.
Add unit test for these functions. These functions are located in
header file but math.c is the place if they are located in source file.
Hence add unit test as the one for math.c.
The next patch replaces the macro SN32_LT and SN32_GT by
spdk_sn32_lt() and spdk_sn32_gt() in iSCSI library, respectively.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id3e4d80fea98ad4ae1516e27b9c9e8ec6f37e7a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1346
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_jsonrpc_begin_result() was changed to always return non-NULL
a while ago. Most of the checks were removed from SPDK, but this
one must have been omitted because to its unusual structure.
Change-Id: If305d7a27d8e6bd4dae1aa8221efabba50db1982
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
pkg-config was missing for pmdk installation.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I795c4b3a71d611292beaa650960afb639e827e78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
The function is almost 20 lines long and it essentialy
appends two lines two a specified file. We don't need it.
Change-Id: I565de8e54aa71b04f15745456bec7b2b3b8fc178
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1288
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
