d/numam-spdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

memory: register each hotplugged page separately

Browse Source 
The problem with registering the entire hotplugged memory
region is that it won't necessarily be unregistered in one
go. Registering each hugepage separately solves that
problem.

This puts a limitation on the number of pages that can
be allocated when using RDMA. We'll hopefully lift this
limitation sometime in future - probably levereging
ibv_rereg_mr, but for now we'll have to resort to either:

 a) using 1GB hugepages
 b) preallocating memory (with [-s|--mem-size <size>] app
    param) as it will be registered as just one region no
    matter what size it is. This memory won't be returned
    to the system until the SPDK app exits.

Change-Id: I6de997fb4901b772730ba6fe995dcc0640b85749
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428716
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This commit is contained in:
Darek Stojaczyk 2018-10-10 17:52:33 +02:00 committed by Jim Harris
parent 3372a72ca9
commit f4cb6c90b4
2 changed files with 13 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
lib/env_dpdk/memory.c
View File

@ -655,7 +655,17 @@ memory_hotplug_cb(enum rte_mem_event event_type,
const void *addr, size_t len, void *arg)
{
if (event_type == RTE_MEM_EVENT_ALLOC) {
spdk_mem_register((void *)addr, len);
while (len > 0) {
struct rte_memseg *seg;
seg = rte_mem_virt2memseg(addr, NULL);
assert(seg != NULL);
assert(len >= seg->hugepage_sz);
spdk_mem_register((void *)seg->addr, seg->hugepage_sz);
addr = (void *)((uintptr_t)addr + seg->hugepage_sz);
len -= seg->hugepage_sz;
}
} else if (event_type == RTE_MEM_EVENT_FREE) {
spdk_mem_unregister((void *)addr, len);
}

2
test/env/memory/memory_ut.c vendored
View File

@ -54,6 +54,8 @@ rte_eal_get_configuration(void)
DEFINE_STUB(rte_mem_event_callback_register, int, (const char *name, rte_mem_event_callback_t clb,
void *arg), 0);
DEFINE_STUB(rte_memseg_contig_walk, int, (rte_memseg_contig_walk_t func, void *arg), 0);
DEFINE_STUB(rte_mem_virt2memseg, struct rte_memseg *, (const void *addr,
const struct rte_memseg_list *msl), NULL);
#endif
#define PAGE_ARRAY_SIZE (100)