A preparation step for enabling zero copy in NVMEoF TCP initiator.
This option will be used to disable zero copy
for admin qpair. This is needed since the admin
qpair's socket is not connected to socket poll group
and we can't receive buffer reclaim notification.
Change-Id: Ibfbb8a156aafcd7ba8975a50f790da7fbd37d96f
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4210
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When net.core.optmem_max is not set high enough, a call to sendmsg()
might fail with ENOBUFS. Currently this is treated as an error.
When we have no more buffer space left, we should continue to process
any completions and by doing so, free up the auxiliary buffers we ran out
of.
With this change I was able to run perf against the spdk target with a
purposely set to a low, value of optmem_max, where previously it would
fail.
This fixes github issue #1592
Signed-off-by: Jeffry Molanus <jeffry.molanus@gmail.com>
Change-Id: Ieeeed4fbecd827d0da815456b57fbe81495fe54d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch is used to enable placement_id getting
in sock layer and also add the rpc support.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I70de57b0ed392a0aefce9d3ff1f61ef924015a87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4146
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The type of sendmsg_idx is uint32_t, so the maximal
is 2^32 -1, so it could be overflow and get 0, so
we should fix it.
PS: I think that our code may have potential defect.
In my experiment, I try to init sendmsg_idx with 2^32 -1,
so the first req->internal.offset = 2^32 - 1.
But for the ee_info and ee_data in "struct sock_extended_err"
got from _sock_check_zcopy is all 0 in the target side.
So it means that the this req will never be completed.
With the increase of sendmsg_idx (the type is
uint32_t), sendmsg_idx will finally goto 2^32 - 1, so I
think it will still kick the issue I described.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ic9aaf629d73d5b7e2c81800a4f7f92c728adbc34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3948
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch removes implementation of VPP socket abstraction
along with ways to compile it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I089f7703cfc4fb517f8f80f4368e544bced549b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3734
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Add a helper function iscsi_parse_redirect_addr() to validate the
passed IP address-port pair.
iSCSI login redirection will support only numeric IP address and
TCP port, and add AI_NUMERICSERV and AI_NUMERICHOST.
This function is almost same as nvme_tcp_parse_addr() and
nvme_rdma_parse_addr().
Besides, update error log in posix_sock_create() to use
gai_strerror(). gai_strerror() will provide more accurate
information as done by nvme_tcp_parse_addr() and nvme_rdma_parse_addr().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I65c6de81a64dcb26551ce796172d0458e1c298a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3357
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
TCP delayed ACK can be disabled or enabled by enabling or disabling
quick ACK, respectively.
The recently added spdk_sock_impl_opts is helpful for sock library
to control quick ACK.
Hence this patch adds and uses an option enable_quickack. The option
is effective only for the POSIX sock module.
We have spdk_sock_opts now too but spdk_sock_impl_opts will be better
for this case.
This option is not supported on FreeBSD. FreeBSD users can set the
option globally via sysctl if desired.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic89620267acce5872dc8ecaf7a99bb70ae97e993
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3603
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The pipe buffer has obvious performance influence on arm64. The
following is my test result with 1core, we can also enable it
on arm64 currently like the posix socket. And later we can find
the optimal pipe size that won't cause a degradation for large
payloads.
randwrite randread
512 byte 61% 97%
4096 byte 84% 16%
16384 byte -13% -17%
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Ib4df60751c5e06ef9bd7fc7bb7efafa5ad4de211
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3329
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Zero copy send can cause performance degradation with small
payloads. This patch adds an option to disable it if required. By
default zero copy is enabled.
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I14f2b21ad375e770cb08f850360898bac675b351
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Receive pipe reduces number of system calls and gives significant
performance improvement with kernel TCP stack and relatively small IO
sizes. With user space TCP/IP implementations there are no system
calls and double buffering introduced by pipe has negative impact on
performance. Receive pipe remains enabled by default.
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ic5ddee42293df2c233ba7ffbe6662de7917ac586
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3343
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Call recv to trigger busy polling even when no socket is active. when
epoll_wait returns zero, the first socket in poll group is used to
trigger busy polling in kernel stack and potentially reap incoming data
Change-Id: I15f04cb4a2c7b382dd07391eda69678fd7919790
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
For some exceptional cases (e.g.,
https://github.com/spdk/spdk/issues/1486),
we may detect POLLERR or other events. So for those events,
we can just ingore it, but not use SPDK_UNREACHABLE.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I073575408783ff75e50b40d45ddf09388a2cab96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3262
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.
Active pollers should return BUSY status only if they
did any meangful work besides checking some conditions
(e.g. processing requests, do some complicated operations).
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id4636a0997489b129cecfe785592cc97b50992ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If the socks parameter(passed in uring_sock_group_impl_poll) is NULL,
we do not need to handle the sock_flush and prep the pollin task,
otherwise it will cause the assert issue when we reap the task when
we handle the nvmf_shutdown_tc3 issue.
Because in uring_sock_group_impl_remove_sock, we finally
set sock->group = NULL. Without this patch,
when we call uring_sock_group_impl_poll in this function,
pollin_task or write_task are prepared, then in the next round,
we will reap those tasks again.
PS: Error info can be found in
https://ci.spdk.io/results/autotest-per-patch/builds/19186/archive/nvmf-tcp-vg-autotest/build.log
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7e6deaa05e958b52e71e0bbf0ccdd20e35583685
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3031
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
According to the description of io_uring_queue_exit:
Tear down function for io_uring. Unmaps all setup shared ring buffers
and closes the low-level io_uring file descriptor returned by the kernel.
So we should remove the close operatoin on ring fd.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I910c6e8acd935925b7985c2aa750df385004eb55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2922
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Thus, we can make sure that when read data is larger than
the pipe size, it will not read the data into the pipe.
Change-Id: I87f3b03fd9b81eb693e9eae0fea9eef7d1b9eaa8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
When the initiator terminates the connection suddently, we
found the keep alive timeout issue, e.g.,
nvmf_ctrlr_keep_alive_poll: *NOTICE*:
Disconnecting host from subsystem nqn.2016-06.io.spdk:cnode1 due to keep alive timeout.
The root cause is that we did not closes the connection on our target side on time
whening using uring sock, and this patch can fix the issue.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I295f58bbdbae0ac3f5308f6eadef6a75c5ad07d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2544
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This reverts commit 79215d80c1.
Reason: Find the bugs while using batched. For example, if
we fetch 3 CQEs, A, B, C and put it in a cqes[] array.
Then we leverage io uring cqe seen to handle, Then we handle the
CQE A, then invokes the call back related with A. In A's call back,
it may also call the reap function (sock_uring_group_reap),
then the CQEs will be handled again. Thus the CQEs B and C can be already
handled. Then we will handle B or C again, then it triggers the error.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I67ece3b24e677b88d66d08722b00539693b42e1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2543
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Also, while we are here, consolidate setting SO_SUFFIX to one spot.
Previously, it was possible for a library to slip through
without an SO version.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4db5fa5839502d266c6259892e5719b05134518c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The pipe buffer gives about 19% randwrite and 56% randread
performance boost on arm64 from my test. We can also enable it on
arm64.
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Iff4c6eaffb4ec5ae8fe02b35f72aed7f6b272bb5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Purpose: To disabled the zero copy support if IP is configured on a loopback
device.
To address issue: https://github.com/spdk/spdk/issues/1340
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ia337f108851653c60d259328b6fefd4ba1e3d654
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2372
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Purpose: This can improve the performance.
Change-Id: I3f5526ab8716cb0771b5e193afa9a0dbffec5cc0
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1929
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We should add the cancel task feature in uring if
sock is removed from the group.
This issue is detected after the group polling
feature is added in NVMe tcp initiatator side. Before
the patch, SPDK NVMe initiator side will not use
group polling feature. After the group polling
feature added in the iniatiator side, we should add this
patch.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I012b403a9f57a5a8ee96c83471b775f0d99b9eb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2049
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Purpose: prepare for the next patch
Change-Id: Iae6a3d7a2c90a78c7dd1e510e718e4e1c07b3f32
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2048
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Port the similiar code from posix sock module if a sock
is removed and added to a new group.
Change-Id: I306bed3c90f79ee06a363fbab02c776e756cda2f
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2047
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Purpose: This is used to make users can specify
some options on the socket, e.g., the different priority for the socket.
While creating sockets, the priority needs to be set before connect()
and listen system calls, so better to add one parameter in spdk_sock_opts
which can contain options (e.g., priority) in spdk_sock_listen_ext and
spdk_sock_connect_ext functions.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: I406238e9da7abd69f937b7072535a19124ed0169
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1874
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Since the related feature is already contained in
spdk_sock_listen and spdk_sock_connect functions,
we no longer need this function.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I1eafff0d139fa266a355fbee2bf0fc3947db69fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1876
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Do large reads from the socket and buffer into a pipe.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ibce74917a1cd7248ff1a325204c0d338cc6a3470
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1349
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We will be create fine name for each poller but it will need large
effort. Replacing spdk_poller_register by the macro SPDK_POLLER_REGISTER
will provide better name than function address with minimum effort.
Following patches may improve function name for clarification.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If862a274c5879065c3f7cb04dcb5ca7844523e68
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1781
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Community-CI: Broadcom CI
When the socket is migrated from one polling group to another
(e.g., iSCSI target example), we should add it into pending_recv_list
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I56d34000344ae452a25b82952c831f09e0266d66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1448
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The code used to do this but it was removed when the buffering was
shifted down to the posix layer. Add a way for users of sockets
to still properly size the buffers.
This also means that by default, the receive buffering is not enabled
on sockets. That matches the behavior of the previous release.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I20ce875be2efd841fe3a900047b4655a317d7799
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1560
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The statement in line 1199 is may not be correct:
Actually, it should be following two cases
to pendinging_recv into false;
1 posck->recv_pipe == NULL
2 psock->recv_pipe != NULL & avalable_bytes == 0;
So this patch can fix this issue.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie612f781b0848e1f0ab1b48dc8c45e39862b772a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1446
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch is used to add the async network I/O support in
sock layer. If code is configured with I/O uring, --with-uring,
we can use io uring in Linux (version >=5.4-rc3).
PS: We also make VPP's default priority > uring, because
for the iSCSI or sock test linked with VPP. It tests VPP with
a given address (which is not a special VPP can only open address),
so using uring can also listen those address succefully. And if we make
uring with priority > VPP, actually, VPP will not tested in those cases.
Additionally, the current CI pool is not ready for test, we need
wait for the CI system ready. And I test on my local platform, it works.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie8ee3c8ddf8d2a7264f2b382376733e002816dcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/952
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This will allow us to keep track of compatibility issues on a
per-library basis.
Change-Id: Ib0c796adb1efe1570212a503ed660bef6f142b6e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Do large reads from the socket and buffer into a pipe.
Change-Id: I0a1bc5b356cb8c403048ab3b22a159332f085e6b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/447
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For client side connections, SO_RCVBUF needs to set
before connect call() so this patch moves those calls
accordingly.
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: Ifa8373145b3699e697e34e93132b5c006e7fbf83
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/757
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
CMSG_FIRSTHDR could theoretically return NULL. Check it for the
peace of mind.
CMSG_FIRSTHDR() returns a pointer to the first cmsghdr in the
ancillary data buffer associated with the passed msghdr.
It returns NULL if there isn't enough space for a cmsghdr
in the buffer.
Change-Id: I6c7e1eb59121b59c568d3ad7f5eda649a49026f4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/771
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Purpose: The function spdk_sock_request_put may
return an error code, and close the socket, so we should change the
return type of _sock_check_zcopy.
If the return value of _sock_check_zcopy is not zero,
we should not handle the EPOLLIN event.
Fixes#1169
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie6fbd7ebff54749da8fa48836cc631eea09c4ab8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483311
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
If available, automatically use MSG_ZEROCOPY when sending on sockets.
Storage workloads contain sufficient data transfer sizes that this is
always a performance improvement, regardless of workload.
Change-Id: I14429d78c22ad3bc036aec13c9fce6453e899c92
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Purpose: To support SPDK iSCSI target applications
when it uses spdk_sock_writev_async.
PS: for some duplicated code between posix and vpp,
we should consider it later. Since if we do it early,
if it cannot be abstracted in the common header,
it will invoke addtional work.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I30d8ee81f80ea5e74c53ff726ee44b0612867c71
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481749
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Amortize the writev syscall cost by using the writev_async socket API.
This allows the socket layer to batch writes into one system call
and also apply further optimizations such as posix's MSG_ZEROCOPY
when they are available. As part of doing so we remove the error
return in the socket layer writev_async implementation for sockets
that don't have a poll group.
Doing so eliminates the send queue processing.
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Change-Id: I5432ae322afaff7b96c22269fc06b75f9ae60b81
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475420
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Initiator drivers (e.g nvme/tcp) don't use poll groups but rather directly
poll the qpair. In this case we want to allow the polling function (e.g
_qpair_process_completions()) to flush async writes pending on the socket.
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Change-Id: Ibd8c73691213d58e287b7110d0f5a381a89a64d0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Purpose: Prepare for setting priorities for different
kernel based sock implementations.
The g_net_impls list is maintained in decreasing order
according to the priority of each sock implementation.
For examaple, if there are 3 sock implementations, i.e.,
posix (priority = 0), vpp (priority = 1), sock_ut (priority =2),
then the list will be maintained as:
sock_ut -> vpp -> posix.
Then if users use spdk_sock_open/listen with impl_name as NULL,
then the order to try is: sock_ut, vpp, then posix
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I43899de5bac14751ab060a11eb814cd7a0a83cc6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479488
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
This is preparation to add flags to the call.
Change-Id: I36109c069b42bd3ec22e023079c62d41a435f44c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471771
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Add an additional queue for requests that have been sent on the network
but aren't complete yet. As of this patch, the code
is still calling writev with no flags in the POSIX layer, so it completes
synchronously. That means requests pass through this new pending list
only very briefly inside of one function.
Change-Id: Iaab6efc118a6d5fe9589199515eb3a7293db4b8e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add spdk_sock_writev_async for performing asynchronous writes to
sockets. The user of this call is responsible for allocating their own
spdk_sock_request structures to pass to this call.
spdk_sock_writev_async will not return EAGAIN and will instead leave the
requests queued until they are fully sent or aborted due to socket
error.
Change-Id: Idf3239e65d26a3024e578122c23e4fb8f95e241b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Close can't fail. And if it did, we still want to release
the sock memory.
Change-Id: I0e4f4d23d49f32132f4526fef8587823ace0a774
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475311
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is useful for detecting sockets that have been disconnected
by the other end without reading data.
Change-Id: Ieb6529984d282d48373766d9f5555cf11720f19b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For new connections, just set the size to a good value for
storage automatically.
Change-Id: Ida2b21eac512a8a25ef70f486b43d5c47be16b63
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Instead of having two places in the code that do the
syscalls to set up the recvbuf size, just call the
function we already have.
Change-Id: I7d098fc7a39d0593c58fbe05b1f0b14f8b6a360d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Have each sock implementation free the group_impl itself.
This allows C++ based sock implementations like Seastar
to release the group_impl memory using delete rather
than free.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If40a91e8bc93a531701fc30d847ab28fa11858ab
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Everywhere we use this, we increase the size to 2MB.
Just make that the default behavior.
Change-Id: Ie174419a09df1792a0c7311eddd0c2dcefa267b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466991
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When VPP detach fails (e.g. when VPP dies before application), net
framework never finishes.
Change-Id: I2cbc7bde274e185fdf7f3cf1c7ea3ddd14dcf365
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464678
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Rounding out the module concept of SPDK libraries.
Change-Id: I2b316153809ae9f73361648fe505274a59d0bdb3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465456
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
