d/numam-spdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

event/reactor: take into account 1 extra slot in event mempools

Browse Source 
The following assumes we use env_dpdk underneath.

rte_mempools use rte_rings internally. When creating
a mempool with capacity N, we internally try to create
a ring with least N+1 size. That's because rte_rings
need one extra empty element to differentiate between
the full and empty state. To make it worse, the ring
size has to be a power of two, so rte_mempools use
`rte_align32pow2(mp->size + 1)` for the size calculation.
And rte_ring memory has to be physically contiguous.

Allocating a mempool of capacity 262144 requires at
least (2 * 262144) * sizeof(void *) = 4MB memory. This
made us require at least 2 physically contiguous
2MB hugepages.

Change-Id: Iabc984a29a60c0b2cf5309a78cd1bcce28ac7b3d
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/408925
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This commit is contained in:
Dariusz Stojaczyk 2018-04-25 08:16:47 +02:00 committed by Ben Walker
parent d546e3d958
commit 408bb168ac
1 changed files with 8 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
lib/event/reactor.c
View File

@ -363,6 +363,12 @@ _spdk_reactor_context_switch_monitor_stop(void *arg1, void *arg2)
} }
} }
static size_t
_spdk_reactor_get_max_event_cnt(uint8_t socket_count)
{
return 262144 / socket_count - 1;
}
void void
spdk_reactor_enable_context_switch_monitor(bool enable) spdk_reactor_enable_context_switch_monitor(bool enable)
{ {
@ -663,7 +669,7 @@ spdk_reactors_init(unsigned int max_delay_us)
if ((1ULL << i) & socket_mask) { if ((1ULL << i) & socket_mask) {
snprintf(mempool_name, sizeof(mempool_name), "evtpool%d_%d", i, getpid()); snprintf(mempool_name, sizeof(mempool_name), "evtpool%d_%d", i, getpid());
g_spdk_event_mempool[i] = spdk_mempool_create(mempool_name, g_spdk_event_mempool[i] = spdk_mempool_create(mempool_name,
(262144 / socket_count), _spdk_reactor_get_max_event_cnt(socket_count),
sizeof(struct spdk_event), sizeof(struct spdk_event),
SPDK_MEMPOOL_DEFAULT_CACHE_SIZE, i); SPDK_MEMPOOL_DEFAULT_CACHE_SIZE, i);
@ -678,7 +684,7 @@ spdk_reactors_init(unsigned int max_delay_us)
*/ */
g_spdk_event_mempool[i] = spdk_mempool_create( g_spdk_event_mempool[i] = spdk_mempool_create(
mempool_name, mempool_name,
(262144 / socket_count), _spdk_reactor_get_max_event_cnt(socket_count),
sizeof(struct spdk_event), sizeof(struct spdk_event),
SPDK_MEMPOOL_DEFAULT_CACHE_SIZE, SPDK_MEMPOOL_DEFAULT_CACHE_SIZE,
SPDK_ENV_SOCKET_ID_ANY); SPDK_ENV_SOCKET_ID_ANY);