We assumed spdk_mem_map_translate() translates only 2MB-aligned
addresses, but that's not true. Both vtophys and NVMf can use it
with any user-provided address and that breaks our contiguous memory
length calculations. Right now each buffer appeared to have the
first n * 2MB of memory always contiguous.
This is a bugfix for NVMf which does check the mapping length
internally. It will also become handy when adding the similar
functionality to spdk_vtophys().
Change-Id: I3bc8e0b2b8d203cb90320a79264effb7ea7037a7
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433076
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This allows us to remove $(ENV_LINKER_ARGS) from all of the
application Makefiles.
Modify spdk.unittest.mk to explicitly set ENV_LINKER_ARGS
to nothing, to avoid DPDK libraries from getting linked
into unit tests. This seems better than creating separate
LINK_C rules for unit tests. Provide a UNIT_TEST_LINK_ENV
flag to allow select unit tests to override this behavior -
it's needed for some of the env unit tests.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I647f2c848cf434023cf5a1f3a45869bbc4fc8102
Reviewed-on: https://review.gerrithub.io/434145
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Apparently this patch revealed an issue somewhere in SGL
handling in NVMf initiator which is causing our CI to fail.
Let's revert it while we work on a proper fix.
This reverts commit 6f5840d225.
This is not a full revert, some app.c changes to the usage text
were kept unchanged.
Change-Id: Iddea5c2b9df50bd12ef8f6226165883f6622ab33
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/432576
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: Ben Walker <benjamin.walker@intel.com>
Now that we utilize DPDK dynamic memory allocation, we
no longer need to set the default pre-reserved memory size.
It'll now be 0, which implies all memory will be allocated
at runtime.
The option to pre-reserve all hugepages on the system is
now only available on BSD, so also clarify that bit in the
app usage text.
Change-Id: I5a8a1d9bf14ad6d938532d7e6254a45e4a81bb92
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/432204
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The problem with registering the entire hotplugged memory
region is that it won't necessarily be unregistered in one
go. Registering each hugepage separately solves that
problem.
This puts a limitation on the number of pages that can
be allocated when using RDMA. We'll hopefully lift this
limitation sometime in future - probably levereging
ibv_rereg_mr, but for now we'll have to resort to either:
a) using 1GB hugepages
b) preallocating memory (with [-s|--mem-size <size>] app
param) as it will be registered as just one region no
matter what size it is. This memory won't be returned
to the system until the SPDK app exits.
Change-Id: I6de997fb4901b772730ba6fe995dcc0640b85749
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428716
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
All mem_maps will still receive separate unregister
notification for each registered region, but the
public memory unregister API is more flexible now.
This follows the VFIO_TYPE1v2_IOMMU interface, which
allows the same.
Change-Id: Ifc008afdc6bff39d9b3b4c892c379ade10c3098e
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428715
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Added sanity checks to prevent unregistering a memory
range that wasn't registered as a one, complete region.
Change-Id: I819b57560b2e48b0802113ffff9f72949d7a148a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/425556
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Removed the reference count from the registrations map.
Although technically supported, registering a single memory
region more than once had a lot of unhandled cases and could
easily lead to a segfault.
RDMA maps require all memory to be unregistered in the same
chunks the memory was registered, which is often impossible
to achieve if a region was registered more than once:
1. register region 0x0 - 0x3 -> it gets mapped to
a single ibv_mr
2. register region 0x1 - 0x2 -> nothing happens, this region
is already registered
3. unregister region 0x0 - 0x3 -> 0x0-0x1 gets unregistered as
one region. 0x2-0x3 gets
unregistered as another
(leading to segfault in the
the current RDMA implementation)
The problem is that the last two regions share the same ibv_mr,
which SPDK tries to free twice. The second free causes a segfault.
vtophys map handles this case by registering each 2MB chunk
separately, but this solution cannot be applied for RDMA, as
NICs put a limitation (~2048) on the number of regions registered.
Another option is to keep a refcount of each ibv_mr allocated,
and free it only when the entire region was unregistered from the
SPDK mem map. This is however very tricky and RDMAmojo mentions
that freeing a memory buffer before unregistering its ibv_mr
may lead to a segfault.
Change-Id: I545c56e24ffa55bda211dea22aeb8a55d9631fe5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/426085
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Prevents us from unregistering two or more separately
registered regions with a single notification.
This fixes an ibv_mr leak in RDMA. When multiple registrations
were unregistered with a single notification, only the first
ibv_mr one would be freed and the remaining memory would
possibly still remain DMA-able.
As of now, unregistering multiple complete regions with
a single unregister call is not possible. It will be
implemented later, after the rest of the code is cleaned up.
Change-Id: I7d61867fa61fd7a4a8a644ff45cab17125d63e1b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/425555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Moved them from test/env/vtophys to test/env/memory
because that's where they should land in the first
place. Our memory ut already test allocating a mem_map,
so just extend them with an extra test case now. Since we're
a unit test rather than a fully-fledged SPDK app, we can
simplify the code a lot now - we no longer have any memory
(hugepages) registered at the beginning of our test case,
hence we no longer need to alloc multiple dummy mem maps to
iterate through all registrations - we can simply hardcode
and predict which registrations are there.
Change-Id: I82cd00ea2ad2370bdc63846874885f8c55e11d53
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428714
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If during mem_map creation we managed to register only a part
of the total registered memory and then failed in the middle,
we will now undo our registrations and gracefully return
an error code leading to a spdk_mem_map_alloc() failure().
Change-Id: Id549f35109bd381318adc676392d9ee26d035359
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/423538
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This function will now check for whether or not a memory region is
contiguous accross 2MB map entries and return the total length of that
contiguous buffer up to the size specified by the user.
Also includes unittests
This series of changes is aimed at enabling spdk_mem_map_translate to
report back to the user the length of the valid mem_map up to the
function that requested the translation.
This will be useful when retrieving memory regions associated with I/O
buffers in NVMe-oF. For large I/O it will be possible that the buffer is
split over multiple MRs and the I/O will have to be split into multiple
SGLs.
Change-Id: I2ce582427d451be5a317808d0825c770e12e9a69
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425329
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This series of changes is aimed at enabling spdk_mem_map_translate to
report back to the user the length of the valid mem_map up to the
function that requested the translation.
This will be useful when retrieving memory regions associated with I/O
buffers in NVMe-oF. For large I/O it will be possible that the buffer is
split over multiple MRs and the I/O will have to be split into multiple
SGLs.
Change-Id: I90da6d4d31c669a3bf046f7721923dd743c5ef21
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add dependency $(ENV_LIBS) for test/env/*
as part of new variable $(ADDITIONAL_LIBS) which is direct dependency
of $(APP) in mk/spdk.unittest.mk
This solves the problem of
make not triggering LINKing for test/env/*
after changes made to lib/env_dpdk/*.c
Issue was that Makefiles in test/env/*/ classified tests as unittests
by including mk/spdk.unittest.mk
But unittests are not using env_dpdk which means that
unittests don't relink on env_dpdk update
Change-Id: I91b6690dc92d68bc69bbf11ac5336d3a7f56e4e4
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/425665
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The function now takes a pointer as it's last argument, and copies the
size of the memory region for which the translation is validinto that
pointer.
For now, that will always be 2MB. However that behavior can change in
the future.
This series of changes is aimed at enabling spdk_mem_map_translate to
report back to the user the length of the valid mem_map up to the
function that requested the translation.
This will be useful when retrieving memory regions associated with I/O
buffers in NVMe-oF. For large I/O it will be possible that the buffer is
split over multiple MRs and the I/O will have to be split into multiple
SGLs.
Change-Id: I8686c166ec956507f5ae55cf602341281482cb89
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424888
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There is no reason for this test to consume all of
the huge pages on the system. 256MB is sufficient.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id1813f4ff6a1fdd1d65fdcc76f6919f6e73b988f
Reviewed-on: https://review.gerrithub.io/420309
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Freebsd complains:
`error: redefinition of typedef 'rte_mem_event_callback_t' is
a C11 feature [-Werror,-Wtypedef-redefinition]`
Those typedefs are already defined in rte_memory.h, we can use
them directly.
Change-Id: If87dd0ad93578bcb22a1d07e1c3433801e2bd5bc
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/420276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a check to prevent spdk_mem_map_set_translation() or
spdk_mem_map_clear_translation() calls that start within the valid
address range but specify a size that would access parts of the mem map
outside of the valid region.
spdk_mem_map_translate() is safe without any extra checks since it only
accesses the first entry regardless of size, and the MASK_256TB check
catches out-of-range accesses to that entry.
Change-Id: Ie1437e57b5158363bb98a6b42a26fb41a089bbad
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418106
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
The arrays for both the map_256tb and map_1gb structures were twice
as large as necessary; fix the sizes and add unit tests for the boundary
conditions to verify that the fix works.
Change-Id: I66bce463f234f54e69cf2a697db9f806d398ca1e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418105
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
This brings DPDK 18.05 support and introduces
dynamic hugepage memory allocation.
The following is now possible:
./spdk_tgt -s 32
rpc.py construct_malloc_bdev 128 512
or even:
./spdk_tgt -s 0
Note that if no -s param is given, DPDK will still
allocate all available hugepage memory.
This has been tested with DPDK 18.05-rc6.
Fixes#281
Change-Id: Ic9521484c2871eb5b2a56445f1177f305b147707
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/410540
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Change-Id: Icc8aeb642d31b5c031c90855b33913bf886cefe8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/412406
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
