Commit Graph

15367 Commits

Author SHA1 Message Date
Niklas Cassel
e9e97cb620 nvme/fio_plugin: fix initial_zone_reset
Commit f69367c788 ("fio_nvme: defer qpair allocation to file_open
callback") moved the qpair allocation from spdk_fio_setup() to
spdk_fio_open(). This broke --initial_zone_reset, which needs a qpair
in order to perform the initial zone reset.

While at it, move the initial zone reset from spdk_fio_setup() to
attach_cb(), as this is where all the other fio options are verified.
By placing it in attach_cb(), after the duplicated file check, we
avoid the need to loop through the whole fio_thread->fio_qpair list.

Since SPDK nvme ioengine no longer initializes the qpairs in .setup(),
create a temporary qpair, if the --initial_zone_reset option was used.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I7950304c58aef3ec783f7cd99cfb1e7d7817a197
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7589
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-26 16:16:42 +00:00
Sudheer Mogilappagari
2cbc9d4dff posix: Group connections of sock group on host side based on placement_id
On host side the connections are created and then added to thread's
poll group. Those connections could use different NIC queues underneath.
To route all connections of poll group through single queue a unique
placement id is chosen as group_placement_id and each socket of poll
group is marked with group_placment_id using getsockopt(SO_MARK) option.

The driver could use so_mark value of skb to determine the queue to use.

Change-Id: I06bda777fe07a62133b80b2491fa7772150b3b5d
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6160
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 15:33:03 +00:00
Karol Latecki
c06df11513 autobuild.sh: restore patches for older DPDK versions
Commit e9ebbdaefc updated
custom patches for mainline DPDK, but did not keep their
previous versions. We still need the old patches to build
with older DPDK versions.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibe222a444bed2fdee63c8dcfcebc8402bb991fbe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7610
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-26 14:23:15 +00:00
Ed rodriguez
b1144ce304 nvmf:Return cdw0 over tcp transport for retrieve cmd
tcp transport doesn't send a response capsule when
c2h_success is set even if cdw0 or cdw1 are non-0.

Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Signed-off-by: John Meneghini johnm@netapp.com
Change-Id: Ieba81fcc50342a2009f7931526e6f8392e26b6a5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6808
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-26 14:22:38 +00:00
Tomasz Zawadzki
a8d700419d examples/interrupt_tgt: mark reactor_set_interrupt_mode RPC private
The RPC is for testing purposes, as such does not need to provide
documentation in SPDK proper.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia153ad326e4c2ac43b511f9c2fba4aa95f1e3295
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 08:39:59 +00:00
Liu Xiaodong
74db63dcda interrupt_tgt: add RPC to set interrupt mode
Plugin RPC method 'reactor_set_interrupt_mode' is added
to set specific reactor run in interrupt mode or back
to poll mode.

Based on this RPC, scenarios later can be implemented
to verify interrupt mode functionality.

Change-Id: I9785e6942299bab7c949780faf18216e293f0b1d
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7347
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-26 06:58:36 +00:00
Liu Xiaodong
6c9b7d0896 reactor: call callback if no need to set intr
When do spdk_reactor_set_interrupt_mode, if reactor
already runs in the specific mode, directly call
callback function before return 0;

Change-Id: I1fd8b753e9881755aa128aabe6d1e2749e58b39b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7549
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-26 06:58:36 +00:00
Jim Harris
0e4690236b sock/posix: return error immediately if epoll_ctl fails
We do not want to do any further work on adding
the sock to the group if the epoll_ctl (or kevent)
fails.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I44b6dc86ce5676aa1b8d6c50b86f22758e4e37fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7594
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:48 +00:00
Jim Harris
b9563518c6 thread: do not ignore read() return
This fixes unused-result warning which prevents
building with -Werror.

Fixes: 924d4bf ("poller: add busy wait mechanism for intr")

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I074949b65e68405033322af34d90e96a896c8b8c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7593
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:48 +00:00
Tomasz Zawadzki
9a3877213e deprecation: add a file to gather all deprecation notices
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I66e442c4d10c1d40f4d9240f4f48374fc6966f29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6591
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:03 +00:00
Tomasz Zawadzki
ea31575578 CHANGELOG: added missing entries for SPDK 21.04
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia43952bbdc5efed7bf009810ff2a5fc5c7026604
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:03 +00:00
Krzysztof Karas
02614b0db3 CHANGELOG: refactor initial changelog entries
Fix typos, add highlighting, sort entries alphabetically,
merge two nvme entries into one, improve document spacing.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I18261326fcdb6f82f8e52e29d123c19136bbb9e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7530
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:03 +00:00
Ben Walker
03a3a5c02d scripts: Add check_format test for JSON-RPC documentation
All JSON-RPCs must be documented in jsonrpc.md

Change-Id: Iec9119a01433fd0cbe4df4477ce94cc2feeb0d54
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4074
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:03 +00:00
Jim Harris
d80267cc17 net: deprecate library
The net library isn't needed - everything these RPCs
do can be done externally to the SPDK application.

This library will be removed in the 21.07 release.

As part of the deprecation, mark the net RPCs as
private.  This will prevent an upcoming patch from
complaining that these RPCs are not documented.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I61118820fd29e410dca763595c3d9fd01a57373d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7592
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:03 +00:00
Ben Walker
365db9ee48 sock/posix: Deal with hung I/O with MSG_ZEROCOPY and interrupt
suppression

When all of the following conditions are met:
- non-blocking socket
- zero copy is enabled
- interrupts are suppressed (i.e. busy polling)
- NIC tx queue is full at the time sendmsg() is called
- epoll_wait sees there is already an EPOLLIN event
then we can get into a situation where data we've sent is queued
up in the kernel network stack, but interrupts have been suppressed
because other traffic is flowing. This makes the kernel miss the
signal to flush the software tx queue. If there wasn't also already
a pending EPOLLIN event, then epoll_wait would have been sufficient
to kick the system out of this state. But when all of this aligns,
it hangs.

We deal with this by detecting the scenario and calling poll(), which
will force the kernel to issue the pending transmits.

Change-Id: Ifb247159b7de16c8fc72a90f0333f5b421c8bd07
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6750
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 18:31:07 +00:00
Konrad Sztyber
97b0c5d327 doc/nvmf: add note about the issues with RDMA on E810 NICs
Added note to the nvmf documentation about an issue [1] with destroying
qpairs on E810 cards in RoCE mode.  The note needs to be removed once
the issue is resolved.

[1] https://github.com/spdk/spdk/issues/1840

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie03a7a662391bd738848f71e582748a334a6ffd5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7577
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-23 18:10:57 +00:00
Liu Xiaodong
5a7d428d0f nbd: add set interrupt mode to poller
Change-Id: I76561561b245f42795fa9eb28967cd486163c589
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5790
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-23 18:09:54 +00:00
Liu Xiaodong
db8ac562b3 vhost-blk: add set interrupt mode to poller
Change-Id: Ie599e445aca8241cf2f71e152ea92a5f92dd4dc5
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-23 18:09:54 +00:00
Liu Xiaodong
2d36f6f11c bdev_aio: add set interrupt mode to poller
Change-Id: I089241bbb42c047e8ae4f206f2dad72c4a69f1d9
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-23 18:09:54 +00:00
Liu Xiaodong
924d4bf32d poller: add busy wait mechanism for intr
For pollers that don't natively support interrupts, using
a busy wait mechanism temporarily.
An interrupt falicity for busy wait will
be registered for non-periodic poller.
Internally, an eventfd is created to each busy wait
poller. Write the eventfd when set interrupt mode,
and only read the eventfd when set back to poll mode,
then the busy wait poller will be called repeatly
in interrupt mode.

Change-Id: Iaeae14d1ff69fd9ef7d606a0b0a70193764513e9
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6711
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 18:09:54 +00:00
Liu Xiaodong
c74421c1c0 poller: rename timerfd to interruptfd
In next patch, if poller doesn't have a period, eventfd
will be created which's always busy automatically.
This eventfd can be combined with timerfd. So rename
timerfd to interruptfd.

Change-Id: Ibffa30ecfcaa73e55f47e97fac854641b74f2dfb
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7546
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 18:09:54 +00:00
Liu Xiaodong
c7cf48ddbe poller: add register interrupt function
Defined callback for spdk_poller to adapt itself to
set interrupt or poll mode. The callback can
be registered to spdk_poller by new function
`spdk_poller_register_interrupt`

Interrupt callback operations for period poller are implemented,
so period pollers now are interruptable.

Change-Id: I2aa6ebfdd75f76b85a70af7e42530be4131ddc8a
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5752
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 18:09:54 +00:00
Liu Xiaodong
39527e93d8 thread: refine period poller intr function
These refined functions are prepared to adapt period
poller to following poller switchable API.

Change-Id: I34d2a785fa0e757b97b0dac5ccf24819d75e0184
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7156
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 18:09:54 +00:00
Liu Xiaodong
b763ebfe1d reactor: apply spdk_thread_set_interrupt_mode
The interrupt mode of spdk_thread can be operated
by reactor based on reactor's interrupt mode when
the spdk_thread is scheduled or the reactor is set
into interrupt mode.

Change-Id: Ibeef7ffb759589a7b372bd78e59e3410be061383
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6709
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-23 18:09:54 +00:00
Liu Xiaodong
2d52c6a15b thread: add spdk_thread_set_interrupt_mode func
spdk_thread_set_interrupt_mode can get spdk_thread run
between intr and poll mode. It is only valid when thread
interrupt facility is enabled by
spdk_interrupt_mode_enable(). Currently, this function
is limited that no poller is registered to the spdk_thread.

Change-Id: Iba54accd5976beb6f6e155014903928ce2858e36
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6708
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-23 18:09:54 +00:00
Konrad Sztyber
7ae061d1ea event/rpc: replace spaces with underscores in framework_get_scheduler
All the other RPCs use underscores in the names of their fields, so
`framework_get_scheduler` should also use them instead of the spaces.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0e9edd7c59a4ab61643a7b558a2359e1805ed0b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7557
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-23 18:09:23 +00:00
Karol Latecki
e9ebbdaefc autobuild: Update patches for mainline DPDK
Previous patches stopped applying cleanly at DPDK
21.05-rc1, needed to update.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If276f608c8394439064e9dfe30fed6d5531805d2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 08:16:24 +00:00
Jim Harris
6c75f0a266 env_dpdk: save hotplug timeout timestamps separate from rte_devargs
DPDK recently clarified some semantics on the rte_devargs 'data'
and 'args' fields.  This actually breaks our use of the 'data'
field to store the 2 second timeout timestamp for delaying
attach to newly inserted devices.  Investigating this further,
it does not seem our use of the 'data' field was valid - it just
happened to work until now.

We could use the 'args' field now.  But knowing whether to use
'args' or 'data' would then be dependent on the DPDK version.
We cannot use RTE_VERSION_NUM to decide, because this is a
compile time decision, and it is possible in shared library
use cases that we could actually link and execute against a
different version of DPDK than we built against.

So instead we will create our own env_devargs structure that
will store these allowed_at timestamps. Currently it's just
a linked list (which is exactly how DPDK does it) - we could
make it more optimal with a hash table down the road, but this
code only executes when we are doing PCI enumeration so it is
not performance critical.

Fixes #1904.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3ee5d65ba90635b5a96b97dd0f4ab72a093fe8f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7506
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
2021-04-23 08:16:24 +00:00
Monica Kenguva
204e6278d4 nvme/fio_plugin: fix nvme fio always reporting success
status code and type is inspected and reported
Fix issue #1893

Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I6f181d8c9464182b23c658f4c268b900398fd751
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7567
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-23 07:58:25 +00:00
Jim Harris
1f5364e821 test/blob: verify number of extent page writes
New UT verifies number of extent page writes for
thin provisioned blobs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6b52195d26c3c1414ca862b03e8a3043bbe29b42
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7017
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 07:52:25 +00:00
Tomasz Zawadzki
ae10cf633c lib/blob: do not write extent pages during md sync for thin blobs
Before this patch blob persist wrote out all allocated extent pages.

Intended design was to write out extent pages in two cases:
1) Thin provisioned blobs to write out extent pages when necessary
during cluster allocation.
2) Thick provisioned blobs to write extent pages during blob persist
when the blob was resized

This patch implements 1) by inserting extent before issuing blob persist
in cluster allocation path.
See blob_persist_extent_page_cpl() and blob_insert_new_ep_cb().
Blob persist might have to rewrite the last extent page after blob resize.
See blob_persist_start().

Meanwhile 2) was incorrecly implemented since it always re-wrote all
extent pages starting from 0. This was addressed by limiting number
of extent pages written, only to ones that were resized.
Some considerations were needed:
a) blob resize happen on cluster granularity, it might be needed to re-write
last extent page if resize was not large enough to change number of extent pages
b) first extent page to write should be based on the num_extent_pages from
active or clean, depending on resize direction
See blob_persist_start().

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibba9e3de3aadb64c1844a462eb0246e4ef65d37f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 07:52:25 +00:00
Tomasz Zawadzki
a512214517 lib/blob: update extent pages during snapshot deletion
When both clone and snapshot had already extent pages
corresponding to the same region in cluster map,
the clone extent page was replaced with one from snapshot.

This was incorrect and would result in loss of clusters
from clones extent page. It did not occur in practice
because all extent pages were rewritten anyway during
md sync. Cluster map was correct so updated extent pages
were too.

Cluster map correctness is verified in UT _blob_inflate_rw(true),
at the very end when checking data consistency of inflated blob.

This patch writes out the updated extent page explicitly.
So it would be possible to skip wirting out extent pages
during md sync later in the series.

Note 1)
At this point in series the extent page is written here,
and in blob persists. The later will be removed later in
series.

Note 2)
Errors during updating extent pages are not accounted for,
but neither does syncing them in blob persist.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7deac3c64299f33f8df49e860af1a16295c074e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 07:52:25 +00:00
Tomasz Zawadzki
bc9da1c66c lib/blob: rename blob_insert_extent to blob_write_extent_page
The blob_insert_extent() name was confusing, since the function
was actually responsible for writting out the extent page to disk.
Changed to a more fitting name.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia312b0ef152100f30d5a1bfe123e55135c8afa6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 07:52:25 +00:00
Tomasz Zawadzki
9c3895b6bc lib/blob: refactor extent page updates during snapshot deletion
This patch does not change functionality. It separates
three stages of updating clone during snapshot deletion:
- updating cluster map
- updating extent pages
- removing backing device from clone

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I44869f3be596d9d0f06db4acedfdd7e1500516ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-23 07:52:25 +00:00
Tomasz Zawadzki
0aed63f0e2 ut/blob: avoid modifying blob state in simultaneous sync test
This patch addresses couple issues:
1) Before issuing the md syncs the previous steps in test left
blob state in dirty state already. The resize never had a chance
to apply. This patch adds a proper md sync and polls for completion.

2) Changing blob state is something that should be done via API.
In order for dirty state to apply immidietly set_xattr is now used instead.

3) Verify test state in callbacks to make sure not only the number of
completions is correct, but their order.

This patch is introduced because of the test originally worked
only because of the extent pages always writing out its pages.
The second sync always was delayed because of this.
Meanwhile that should not be the case, since no MD or EP modification
was done.

Later in the series Extent Pages are fixed, but this test remained incorrect.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iac17c27f6ff83f2b79835aa6e48472d5293c44d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 07:52:25 +00:00
Monica Kenguva
0fcf6541f6 rpc: fix formating to enable the framework_get_scheduler rpc
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I50947df022301b955332aad87e41f128f31fce07
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7568
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 23:47:43 +00:00
Jim Harris
4ccc9ea3e0 configure: check if liburing is installed
When user specifies --with-uring without a directory,
check that liburing is actually installed on the system.
If it isn't, instruct user to build and install liburing
and exit with error.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iead4f30bdc3905c3b3462a32fd676a39fbd0d7d2

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7542
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-22 20:28:37 +00:00
Jim Harris
9d2f6f861d configure: wait to configure ISA-L until all deps are checked
Configuring ISA-L takes a bit of time, so it is better
to check dependencies early and bail, rather than
waiting 10-15 seconds to configure ISA-L, only to then
fail because some dependency isn't found.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I864a941df6517381761c9d4db58e2364c1776e4d

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7541
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-22 20:28:37 +00:00
Krishna Kanth Reddy
e04da24cf6 examples/pmr_persistence: Add PMR example application
Added tests to verify the PMR support.

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I63f0a18647bfb8b16b8a5fbedb902c2f28b1d36d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6562
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 20:10:21 +00:00
Krishna Kanth Reddy
c821049d2f examples/identify: Get PMR size
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: Id0dc422eefc6aa143e63d4630e6e74757df50e6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 20:10:21 +00:00
Krishna Kanth Reddy
f98b792db2 lib/nvme: Expose APIs to Access PMR
APIs to get PMR size, enable, disable, map and unmap the PMR.

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: If6ec7f3e2f19bc13b3cbb2def622a41a3d84dc4e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6560
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-22 20:10:21 +00:00
Krishna Kanth Reddy
89858bbf5d nvme/pcie: Add support for Persistent Memory Region (PMR)
Implemented functions to enable, disable, map and unmap the PMR.

Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I580e0b5060cefe1230c3db1361aee1957db457b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6559
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-22 20:10:21 +00:00
Karol Latecki
8d506e8056 autopackage.sh: clean after rpmbuild
"make clean" after finishing rpmbuild. Otherwise leftover
"./configure" options and libraries tend to fail the
next in line "--enable-lto" build.

Fixes #1905

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I6a04028bfdfeb03b089c86afe7f81489fd6976f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-22 17:52:46 +00:00
Maciej Wawryk
4503500e49 scripts/perf: fix measure_bandwidth failed to open outfile error
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9b9f8c9e407425fdab1e84050354481ad76c19c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-22 15:53:34 +00:00
Ziye Yang
467148fce5 bdev/rbd: Refactor the parameters in bdev_rbd_init.
Only passing rbd pointer is enough.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I652581879c9ba228f9af0aa3f5353915a286c6c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7548
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 15:42:53 +00:00
Krzysztof Karas
ea05f19304 spdk_top: add core frequencies to cores pop-up
Resizes cores pop-up to fit new field - core frequency.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I571ed92ea68a6c42f214caa80bbfab2b2fb3b03a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-04-22 15:40:30 +00:00
Krzysztof Karas
bc656341b7 spdk_top: add core frequency to CORES tab
Modifies functions and data structures to allow for
core frequency display in CORES tab.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: Ie3c6eadb090b8e0a6e57968b79d4852f1957b0fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6583
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-04-22 15:40:30 +00:00
Krzysztof Karas
8673a27b46 lib/event: modify RPC to get core frequencies from governor
Modifies RPC "framework_get_reactors" to get core frequency for current
core and insert it into JSON response.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: Ibb9c25e6e1d28ddb4cde42baa20a7e9808652ae8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6582
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-04-22 15:40:30 +00:00
Krzysztof Karas
84c349e1d8 dpdk_governor: refactor _get_core_curr_freqs
Currently _get_core_curr_freqs returns an index from the array
of available frequencies for given core. This change aims to
make this function execute what its name suggests.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I1143f692e7bbbf2f8f9e1cd4943f8e3ecd70ddea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 15:40:30 +00:00
ChengqiangMeng
dbef0cfd02 app/spdk_top: fix SEGV on unknown address
The index of g_thread_info is equals to the thread ID.
When the thread ID is not continuous, but index of ABC is continuous,
so some elements of g_thread_info array will be empty.

fixes #1899

Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: Ib90a26dcc2d47792a098b163746906f34043453a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-22 15:40:19 +00:00