The hotplug lib can be used for pcie devices
such as nvme, virtio_blk and virtio scsi.
For the sigbus handler, there is only one in a
process and it should handle all the devices.
And align nvme to the hotplug lib
Add the ADD uevent support for allowing the
device hotplug.
Change-Id: I82cd3b4af38ca24cee8b041a215a85c4a69e60f7
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5653
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The dbdf format is xxxx:xx:xx.x and with the wrong
format the rte_devargs_parse always fails.
Change-Id: Ia34bc5e68f6401bb25907d5d07c65636b4f491b5
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7140
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
DPDK has added APIs for registering externally allocated
memory regions. Use them instead of doing our own thing.
We have to postpone spdk_mem_unregister call in
memory_hotplug_cb() because SPDK mutex (g_spdk_mem_map_mutex)
and DPDK mutex (memory_hotplug_lock) may overlap
and cause deadlock when one thread is calling spdk_free()
(locks memory_hotplug_lock first and then tries to lock
g_spdk_mem_map_mutex) and another one is calling
vhost_session_mem_unregister() (locks g_spdk_mem_map_mutex
first and then tries to lock memory_hotplug_lock).
Change-Id: I547b4ffc3987ef088a1b659addba1456ad760a71
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3560
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I191ad5e3b153fb563256eba1aa695716f66db788
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6377
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Can not remove device in the remove event
callback as we can not unregister the remove
callback. So use the alarm_set to fix this issue.
Fixes#1809
Change-Id: Ib86bc4eeecc0fe2bc51538e28684d015405e8835
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6553
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
instead appending to output file (which occurs on each make execution)
sed is used to modify `Requires` section of the *.pc file
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I1a8cb1ec35bf583293c7174a413302191bbbd735
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6460
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
bus_pci depends on pci so it shall be listed before, otherwise it can
result in linking issue e.g.
/usr/bin/ld: /home/jkalwas/spdk/dpdk/build/lib/librte_bus_pci.a(bus_pci_pci_common.c.o): in function `pci_parse':
pci_common.c:(.text+0x6e): undefined reference to `rte_pci_addr_parse'
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Idff446df82c37844edc122d5171e8ffa684b296f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6404
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Provide a default stub definition for spdk_pci_device_claim/unclaim
for non-linux platforms, rather than just for FreeBSD.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: Ica45d967878582d9a58e37b088eba4bf0d94104e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Vhost is enabled by default, so rte_net was always included.
When disabled, rte_power failed as it depends on rte_ethdev and rte_net.
rte_vhost was only possible to enable on Linux, so there
is no conflict with adding it next to rte_power under this condition.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2e183004d6457e404471740a0540dcb08aa738d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6398
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Nick Connolly <nick.connolly@mayadata.io>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
The env layer has a pci_allowed list, which specifies
that only a subset of PCI devices may be attached
by the associated process.
But that doesn't cover PCI devices that are hot-inserted
after the application starts, which is common for
storage/NVMe.
So add a new spdk_pci_device_allow() API which allows
an application to add new devices to the allowed list.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7bd5ff428d84480d46bc236698daadd019b20b8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6183
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.
Increasing major so version after LTS, allows the quarterly releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.
Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 21.04
release.
This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I44d01154430a074103bd21c7084f44932e81fe72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6167
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In dpdk 19.11, rte_kernel_driver is the old version, add version check before use the members.
Signed-off-by: sunshihao <sunshihao@huawei.com>
Change-Id: Ic1db37cc0760c7d03692fd2cdcbb6ff1e41f872d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6252
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
As of DPDK main commit 682a6454 ("power: add ethdev
power management"), rte_power depends on rte_ethdev.
So add rte_ethdev library, but only on Linux since
that is the only place where we use rte_power.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibc3473d6ec75c14a8c0c5517616950017cb2ea96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6210
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
The VFIO container must have at least one device present to
be able to perform DMA mapping operations. Instead of using
the count of SPDK devices, use the count of DPDK devices. This
allows for the user of SPDK's memory management APIs even if
only a DPDK device is attached.
Change-Id: Ie7e21f09bdf1cdf1a85424c35212f64f24ae4e26
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Users can now generate the necessary linker args for their
own applications using something like:
PKG_CONFIG_PATH=build/lib/pkgconfig pkg-config --libs spdk_nvme
Dependencies between libraries are included in the generated
.pc files, so the user only needs to pass the top-level subsystems
or individual SPDK libraries they are using in their application.
Modules will automatically be added to the output if the associated
library is specified. For example, specifying "spdk_bdev" will include
the libraries not only for spdk_bdev, but also all of the bdev modules.
Users still need to supply the -Wl,--no-as-needed or -Wl,--whole-archive
flags. They cannot be added to the .pc files without increasing the length
of the argument string by a factor of 15x to 20x.
Modify the test/external_code/hello_world Makefile to use pkg-config to
ensure this gets tested at some level in our autotest environment.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie48a75f11969d5d775d514cf10bcb82d197eabfd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4371
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
SPDK supports two latest DPDK LTS releases and DPDK 20.11
is out now.
Remove all functionality and checks for versions <= DPDK 19.11.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib4c750c49352fe4ad4b50c7a719867387808f0df
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5343
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
__rte_experimental attribute in DPDK ends up being changed to
'deprecated'. See rte_compat.h in DPDK.
Goal of patch below was to allow SPDK to use experimental API
"(b6fce19) env/dpdk: add support for DPDK 18.05 dynamic memory allocation"
Yet it disabled warnings for both. This patch makes it so that
__rte_experimental will not produce warnings, but deprecations in DPDK
will.
It will help identify incoming changes when testing newer versions
of DPDK in the future.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9a75e37c976d613c7ec6ee32e05fc6da79a60a1f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5342
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This helps us remove master/slave terminology from
SPDK and is aligned with similar changes made recently
in DPDK.
While updating nvme/identify to use the new member
name, also replace g_master_core there with g_main_core.
Other nvme utility usage of "master_core" will be updated
in a later patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0ec4e3e9b644bec21b3729809bf5c4d35b10837f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5351
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The old pci_whitelist/pci_blacklist are now deprecated.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9fddec0c90691dd385eb21d13be849247f144889
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5279
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
pci-whitelist and pci-blacklist are deprecated, so
use 'allow' and 'block' instead for DPDK versions that
support it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I430f73fc402ca9fc07d004e719435e5b3dc963f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5278
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
POSIX defines PRId64/PRIu64/PRIx64 for printing 64-bit values in a
portable way. Replace a few references to %ld to remove the assumption
about the size of a long. Similarly, use %z with size_t arguments.
Where the value being printed is an unsigned 64-bit value, use PRIu64
instead of %ld.
Explicitly test for not __linux__ where that is the intent, rather
than testing for __FreeBSD__.
Cast pointer to uintptr_t before aligning it, rather than using
a specific integer size which may not be large enough to store a
pointer.
Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: Icfe219e1bbb2d06b3ef05710fac5b7091d340251
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5142
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
RTE_DEV_ALLOWED is an enum and has no associated define, hence checking
for its presence will always be false.
We could test for RTE_DEV_WHITELISTED define, but this macro added for
deprecation warning will be dropped in the future.
Switch to a check on DPDK version.
Fixes: 10ed0eb755 ("env_dpdk/pci: adapt to 20.11 EAL changes")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Change-Id: I75270977b580065b36c753266cbaa5fb73f99eb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5165
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
DPDK 20.11 renamed device and bus control enums [1].
This is a simple renaming, no change in semantics.
1: https://git.dpdk.org/dpdk/commit/?id=a65a34a85ebf
Signed-off-by: David Marchand <david.marchand@redhat.com>
Change-Id: Ia40bae750ad74f405eb700b47514fca021ffd052
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5116
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Move the lock files from '/tmp' to '/var/tmp' cause user maybe delete files in /tmp
or remount /tmp by mistake, And the JSON-RPC domain socket located in '/var/tmp' also.
Signed-off-by: Weifeng Su <suweifeng1@huawei.com>
Signed-off-by: Shihao Sun <sunshihao@huawei.com>
Change-Id: I18d52f42462e8477fb35aeea9e38efc51610d17c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5096
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
rte_hash depends on rte_rcu starting in upcoming
DPDK 20.11 release. rte_rcu was only added in
DPDK 19.05 release, so we need to check if it
exists before linking it.
Fixes issue #1661.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7e343c6f964b03cc62484b57803a3bad00f80288
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4947
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
When enable Werror compile option with new kernel(v5.8), there is
following error reported due to the <linux/vfio.h> data structure
change(added a uint8_t data[] variable in new kernel), we can just
put the 'unmap' at the end of the data structure to fix the issue,
I think it's better to just use a stack variable instead.
CC lib/env_dpdk/memory.o
memory.c:63:36: error: field 'unmap' with variable sized type 'struct vfio_iommu_type1_dma_unmap' not
at the end of a struct or class is a GNU extension [-Werror,-Wgnu-variable-sized-type-not-at-end]
struct vfio_iommu_type1_dma_unmap unmap;
^
1 error generated.
Change-Id: Icf73a3c48a301e74b92b9ae2e2d8715262b2d056
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4925
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We can verify readdir(dir) directly.
And this fixes issue :
memory.c:1233:23: warning: Although the value stored to 'd'
is used in the enclosing expression, the value is never
actually read from 'd'
while (count < 3 && (d = readdir(dir)) != NULL) {
^ ~~~~~~~~~~~~
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I34b191c9f89d8ae67697ec8e331f9c7c78c16ab2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4819
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ide0ac38497bc500886e26fd219cb120113896966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4152
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Post DPDK 20.08 crypto, isa-l and qat shared object
libraries had their names changed. Detect which library
names are available and use them.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I8c9b9b24f9232f303b99c959754a7f7550ffdf90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4780
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For virtio-user library and coming vfio-user feature, the client needs
to send the memory file descriptors to target so that the two processes
can setup shared memory region to do data processing without memory copy.
Currently virtio-user will read /proc/self/maps to get memory file descriptor,
since DPDK already provides this such APIs, so here we can just use it,
for existing virtio-user library we may replace it with the new added
API.
Change-Id: Icfeae465d53826d0c8d1b335287634b03cd174aa
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The internal vhost library is used when DPDK's version
is older than 19.05 and the experiemntal vhost nvme
target.
For the CONFIG_INTERNAL_VHOST_LIB option, SPDK doesn't
enable this option by default over one year and CI
doesn't cover it either, so we may remove it with
this release.
As for the vhost-nvme target, since we are developing
a new vfio-user target solution, it's OK for us to remove
it now.
Change-Id: Ib2cce1db99cd09754307c2828b3187f2d4550304
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
DPDK patch (7ba49d39) included in DPDK v19.11
renamed phys_addr to iova in rte_memseg structure.
Meanwhile latest DPDK from (72f82c43) removed the
phys_addr altogether.
This patch replaces the reference for DPDK v19.11 and up.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I24b28d32ec8758d730b14bc54fd0cf1a4cc5a333
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Here we add some new variables which we will be able
to use in a later patch to generate pkg-config files
for this env_dpdk library and our DPDK library
dependencies.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8a256096ea08f97eba5d4460405f419624e6f0bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4468
Reviewed-by: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
DPDK patch (4143b122) included in DPDK v18.05 replaced
MEMPOOL_F_NO_PHYS_CONTIG with MEMPOOL_F_NO_IOVA_CONTIG.
Meanwhile latest DPDK from (28e3c8b2) removed the
MEMPOOL_F_NO_PHYS_CONTIG.
This patch simply replaced the define, since it will
work for any DPDK v18.05+.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I43ada50df31be18c724b2f5078d3f29f3d1c0c71
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4418
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This library has existed since DPDK 17.05, so there are
no supported versions of DPDK that do not contain this
library.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I84f2b77046d093989dfa9533f3d1c76e8c243c3f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4417
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
These libraries have existed since DPDK 17.11. We
do not support any DPDK versions older than that, so
there is no need to conditionally handle cases where
those libraries do not exist.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3906db4d07ae04344b4c3bfaac02da58f248bf75
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4392
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This library was removed in DPDK 2.1 which SPDK has
not supported for a very long time now.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I01cf47078e69b9d396a80f5680a4f1c1c3a9be46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4391
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
And modify test/env/vtophys to resolve linking errors.
SPDK_PRINTF() and SPDK_ERRLOG() use spdk_log() procedure which is
customizable and redirectable, so it is preffered over printf()
In case of test/env/vtophys/ program,
we have to make it an app first to avoid linking errors.
Change-Id: Id806ec3bb235745316063bbdf6b5a15a9d5dc2d9
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1944
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Registration macro now generates function based on driver's name.
It allows to have multiple registration within single source file.
Similar pattern is used e.g. by SPDK_NVMF_TRANSPORT_REGISTER.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ied0887e8dae7fe9ca1517313be5eff8f218b7e98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3895
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Give this a more generic name. We're going to be using these
events for more than just hotremove coming up.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ia5356e9ab809807ba4d85ecc212a496e96012bce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3559
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This reverts commit 6194cb2e15.
It's unclear whether we need to add a new API for the env layer
for upcoming work. Nothing currently uses it. When we have a clear
need, we can add this back in.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I174276799d650a1365b37a737271a54a796cd455
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3561
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Makefile support in DPDK was deprecated and will be removed soon,
so switch to the officially supported way of building DPDK -
with meson and ninja. Two new tools. Basically, our Makefiles
will invoke meson+ninja for DPDK, no other SPDK components are
affected.
Apparently DPDK wanted to move away from an octopus-like config
system and the ideology behind meson configuration is simple now:
build everything by default. Some PMDs can be explicitly disabled
with meson command line, but all libraries (both static and shared
versions) and test apps are built unconditionally.
How long does it take to build minimal DPDK with meson? Too much.
On my machine half of the total build time is spent on libraries
we don't need at all. (I have some hacks up my sleeve to disable
building those libraries - see the subsequent patch.) As for the
official way of building a minimal DPDK, there was a patch [1]
on dpdk mailing list to introduce more specific configuration,
but it was rejected:
> We talked about this a few times in the past, and it was actually one
> of the design goals to _avoid_ replicating the octopus-like config
> system of the makefiles. That's because it makes the test matrix
> insanely complicated, not to mention the harm to user friendliness,
> among other things.
>
> If someone doesn't want to use a PMD, they can just avoid installing it
> - it's simple enough.
>
> Sorry, but from me it's a very strong NACK.
Let's not follow that direction, hack the DPDK build system instead.
As for advantages of meson+ninja over Makefiles? I can't find any.
It's another build system that does a lot for you with some functions,
magic options, and a built-in dependency system. It seems nice if you know
the syntax, but it's another component that you need to learn, debug,
and possibly find bugs in (there's a lot of github issues open for meson).
I would compare it to CMake.
As for changes in this patch: rather that explicitly disabling
PMDs we don't need, specify a list of PMDs we do need and disable
everything else found in ./dpdk/drivers/*. This way we won't have
to disable the new PMDs as they're added to DPDK.
Meson configuration also sets RTE_EAL_PMD_PATH #define to a valid directory
with built PMD shared libs. When it's set, DPDK dynamically loads all shared
libraries inside. The drivers there depend on DPDK shared libs and fail to
load in static SPDK builds, so we disable them altogether by unsetting
RTE_EAL_PMD_PATH in the meson-generated config file - just like
DPDK Makefiles did. EAL checks for RTE_EAL_PMD_PATH being empty and skips
loading any external PMDs then. We do it for both static and shared libs.
We specify all PMDs at build time for now, so there's just no need to load
them dynamically.
We have three more hacks in our submodule:
* disable building dpdk apps by commenting-out a line in dpdk/meson.build
* disable building unnecessary libs (build everything that spdk *may*
need)
* build isa-l compress pmd with `-L[...] -lisal`. DPDK expects to find
libisal with pkg-config. We don't want to prepare a pkg-config file,
so comment-out a failing check in another meson.build file and provide
isa-l through CFLAGS and LDFLAGS.
We also need to make some changes to our test/external_code. First of
all, -ldpdk is no more. Meson build generates a pkg-config file with all
libs, but we'll switch to it in a separate patch - for now just specify
all -lrte_ libs one by one. -Wl,--no-as-needed has to be added to some
test cases, otherwise rte_mempool_ring isn't loaded. We don't use any
APIs from this library, it only has a static constructor that provides
a few callbacks used by rte_mempool_create(). Also, since DPDK now builds
both static and shared libraries, we need to add -Wl,-Bstatic to force
using static libswhere required. It's only needed for DPDK libs, but we
use it for SPDK libs as well since there's no harm.
As for performance:
$ ./configure --enable-debug --with-crypto --with-reduce
$ time make -j40 -C dpdkbuild all
with meson:
real 0m8.287s
user 1m7.983s
sys 0m10.548s
before, with the old DPDK makefiles:
real 0m20.232s
user 0m55.921s
sys 0m16.491s
The subsequent builds are much faster too:
$ time make -j40 -C dpdkbuild all
meson:
real 0m0.876s
user 0m0.663s
sys 0m0.217s
makefiles:
real 0m10.150s
user 0m11.740s
sys 0m6.772s
[1] http://inbox.dpdk.org/dev/1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org/
Change-Id: Ic65db563014100bafb12e61ee0530cc2ae64401d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1440
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Now that drivers can be registered from upper layers there's
no need to keep them centralized inside env.
(check_format.sh complains that spdk_pci_nvme_get_driver() shouldn't
start with the spdk_ prefix - to workaround that we move the function
declaration from one place in env.h to another - that's enough to
convince check_format it really is a public function)
Change-Id: If86aebd6c997349569c71430ec815b413eb44ef8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3187
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This allows SPDK apps to register new PCI drivers outside of
the env layer, enabling SPDK as a whole with new use cases.
Change-Id: I0c998a9ec249c3ca610b7b3b8b6caf616b16f64c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3185
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
An alternative to spdk_pci_*_get_driver();
Change-Id: I20a80b3c655a37fb1c76da21c2b70d5678041fab
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3186
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We want to make struct spdk_pci_driver public, so add env-agnostic
fields that define a driver directly to that struct.
PCI driver registration (potentially in upper layers) will only use
spdk_pci_id-s and spdk pci drv_flags, then those will be translated
to DPDK equivalents inside env_dpdk.
Change-Id: Ia24ecfc99ebf0f54f096eaf27bca5ed9c0dfe01d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3183
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
