We have been intermittently hitting the assert where
we check sock->cb_fn != NULL in spdk_sock_group_impl_poll_count.
The only way we could be hitting this specific error is if we
wereremoving a socket from a sock group within after receiving
an event for it.
Specifically, we are seeing this error on the NVMe-oF TCP target
which relies on posix sockets using epoll.
The man page for epoll states the following:
If you use an event cache or store all the file descriptors
returned from epoll_wait(2), then make sure to provide
a way to mark its closure dynamically (i.e., caused by
a previous event's processing). Suppose you receive 100 events
from epoll_wait(2), and in event #47 a condition causes event
#13 to be closed. If you remove the structure and close(2)
the file descriptor for event #13, then your event cache might
still say there are events waiting for that file descriptor
causing confusion.
One solution for this is to call, during the processing
of event 47, epoll_ctl(EPOLL_CTL_DEL) to delete file
descriptor 13 and close(2), then mark its associated data
structure as removed and link it to a cleanup list. If
you find another event for file descriptor 13 in your batch
processing, you will discover the file descriptor had
been previously removed and there will be no confusion.
Since we do store all of the file descriptors returned from
epoll_wait, we need to implement the tracking mentioned above.
fixes issue #1294
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib592ce19e3f0b691e3a825d02ebb42d7338e3ceb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1589
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
(cherry picked from commit e71e81b631)
74c9fd40fd