d/numam-spdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

example/nvmf: Run SPDK threads by FIFO to avoid deadlock

Browse Source 
if spdk_thread_poll() calls spdk_thread_create() to create SPDK
thread dynamically, it will acquire nvmf_reactor->mutex and will
cause deadlock because the queue nvmf_reactor->threads is accessed
by any POSIX thread by holding nvmf_reactor->mutex.

If any non-owner POSIX thread manipulate nvmf_reactor->threads by
holding nvmf_reactor->threads, like general priority based scheduler,
running SPDK threads by FIFO will be better.

Change nvmf_reactor_run() to dequeue a SPDK thread from the head, run
it, and then enqueue it to the tail. Release nvmf_reactor->mutex
while spdk_thread_poll() is executed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9b4040cc4d995beaf4a1f9ed8260bb1254d7d7fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2537
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This commit is contained in:
Shuhei Matsumoto 2020-05-20 06:41:34 +09:00 committed by Tomasz Zawadzki
parent 9bea56c004
commit 8753fdb645
1 changed files with 9 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
examples/nvmf/nvmf/nvmf.c
View File

@ -186,13 +186,20 @@ nvmf_reactor_run(void *arg)
struct nvmf_lw_thread *lw_thread, *tmp;
struct spdk_thread *thread;
/* foreach all the lightweight threads in this nvmf_reactor */
/* run all the lightweight threads in this nvmf_reactor by FIFO. */
do {
pthread_mutex_lock(&nvmf_reactor->mutex);
TAILQ_FOREACH_SAFE(lw_thread, &nvmf_reactor->threads, link, tmp) {
lw_thread = TAILQ_FIRST(&nvmf_reactor->threads);
if (lw_thread != NULL) {
TAILQ_REMOVE(&nvmf_reactor->threads, lw_thread, link);
pthread_mutex_unlock(&nvmf_reactor->mutex);
thread = spdk_thread_get_from_ctx(lw_thread);
spdk_thread_poll(thread, 0, 0);
pthread_mutex_lock(&nvmf_reactor->mutex);
TAILQ_INSERT_TAIL(&nvmf_reactor->threads, lw_thread, link);
}
pthread_mutex_unlock(&nvmf_reactor->mutex);
} while (!g_reactors_exit);