Commit Graph

151 Commits

Author SHA1 Message Date
1edb07ac50 fix build system 2022-03-08 04:08:39 +08:00
Jim Harris
4ccc9ea3e0 configure: check if liburing is installed
When user specifies --with-uring without a directory,
check that liburing is actually installed on the system.
If it isn't, instruct user to build and install liburing
and exit with error.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iead4f30bdc3905c3b3462a32fd676a39fbd0d7d2

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7542
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-22 20:28:37 +00:00
Jim Harris
9d2f6f861d configure: wait to configure ISA-L until all deps are checked
Configuring ISA-L takes a bit of time, so it is better
to check dependencies early and bail, rather than
waiting 10-15 seconds to configure ISA-L, only to then
fail because some dependency isn't found.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I864a941df6517381761c9d4db58e2364c1776e4d

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7541
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-22 20:28:37 +00:00
Nick Connolly
9854c138f7 mk: add support for mingw builds
MinGW builds require a thin layer above the standard libraries to
provide POSIX functionality that is missing on Windows. Add support
for building this.

MinGW cross builds are experimental and work is ongoing to integrate
them into the CI and test environment. Including the changes at this
stage is being done to facilitate that process.

The layer has been arranged in the same way as DPDK and is
accessed as an external build using ./configure --with-wpdk=<dir>.
Support has also been added for using a default ./wpdk in
preparation for reaching the required level of stability.

The help text for ./configure indicates that support for --with-wpdk
is experimental.

Further details and instructions can be found at https://wpdk.github.io.

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: Iff0f705789f19fb193dcb3c9090c3e90613a8d9a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6589
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-03-15 08:55:11 +00:00
Changpeng Liu
d0b2448b63 mk/libvfio-user: support --with-vfio-user=/usr/path
Users may not use SPDK libvfio-user submodule, so add a
library path for this case, users can still just use
'--with-vfio-user' without adding the path, for this case
a submodule default path will be used.

Change-Id: Ib0dd82ab6910056ff21b5b2d373c0d16916162c6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
2021-02-26 12:27:06 +00:00
Michal Berger
bb0f78a278 configure: Make code compliant with shellcheck
Make sure to at least cover directives check_format.sh is looking
for.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I51750c9c8ee627de4dbab8e0e4933eb39e84c527
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6025
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-02-04 16:11:20 +00:00
Michal Berger
5ccabc8f27 configure: Fix Bash style syntax to be compliant with shfmt
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7858a32bb3186df89844407cfc55c8c76b1dddd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6024
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-02-04 16:11:20 +00:00
Nick Connolly
2a0c2a2dcc configure: Adjust platform tests for portability
Explicitly test sys_host against "Linux" which is clearly the intent
and matches the behaviour described in the comments.

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Change-Id: Ic79d32b5d58bf556847849817e6bd72f70363335
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6055
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-02-04 08:44:13 +00:00
Changpeng Liu
3b673b8431 nvmf/vfio_user: disable vfio_user by default
Users can use "--with-vfio-user" to enable it when testing it.
For CI configuration, we add a new test flag SPDK_TEST_VFIOUSER
to enable CI tests.

Change-Id: Id284df721171d01cc52491ebf4088bcc17eee147
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6139
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-01-29 15:15:23 +00:00
Rui Chang
229a67ad6a configure: fix 'break' error in configure file
Break is not supported in if statement which cause error messages

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I671510ac936e2ac43a6570c9c3d0a6458e28f0d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Nick Connolly <nick.connolly@mayadata.io>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-01-25 04:42:04 +00:00
Tomasz Zawadzki
aa22321aca configure: remove --with-igb-uio-driver option
DPDK 20.11 moved the kernel modules to separate
dpdk-kmod repository. It has to be built separate
from DPDK.

If needed for testing vm_setup.sh script now contains option
to build this driver.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I98a5eb956eb0cc60ec402d88fcdbd66d4854f19a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6033
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-01-22 08:47:48 +00:00
Jim Harris
91e422572a configure: add --enable-cet to build with CET support
Intel Control-flow Enforcement Technology (CET) is a
processor feature that blocks return/jump-oriented
programming (ROP) attacks.

It is currently only supported on Tiger Lake client
processors, but will be available on other processors
in the future.

CET requires toolchain support.  gcc8 does support it.
For now, this will opt-in only at least until the
technology is available on server processors.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8c7f882eeeaed26484c31dc0d67d5cc42baeaa2d

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5921
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-01-21 19:24:10 +00:00
Changpeng
3ff9c13614 NVMe/vfio-user: add initial version vfio-user transport to NVMe driver
When NVMf target linked with vfio-user library, we can use
vfio-user client library to connect to the target.

Here is the three examples that can work with target:

identify -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g
perf -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g -q 1 -o 4096 -w read -t 10
reconnect -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g -q 32 -o 4096 -w randrw \
-M 50 -t 10 -c 0xE

You can run the following test script test/nvmf/target/nvmf_vfio_user.sh to have a quick test,
currently enabled with NVMe Identify,Perf,Reconnect tools.

Change-Id: Ieb9842b2f372184fffbf7f23e4aad26feb47c350
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3839
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-01-21 05:00:18 +00:00
Changpeng Liu
b30d57cdad libvfio-user: include libvfio-user as a submodule with SPDK
Also add a CONFIG_VFIO_USER config flag, it's enabled by
default.

Change-Id: I18b44c024a264516a60f743d5c366a4c7f7c6785
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5000
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-01-21 05:00:18 +00:00
Tomasz Zawadzki
b4916f9f69 configure: do not check for > DPDK 19.05 to enable vhost
SPDK supports two latest DPDK LTS releases and DPDK 20.11
is out now.

Remove all functionality and checks for versions <= DPDK 19.11.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Icc3e2ca15f536effc865c82637bf7f2960d5c11b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5346
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-12-04 09:51:00 +00:00
Tomasz Zawadzki
38e26cf55f build/dpdk: detect DPDK presence using meson.build file
This check in configure serves as sanity check only.
Since starting with DPDK 20.11 makefile were removed,
so was the common_base file.

Instead check for meson.build file which exists since
DPDK 18.02.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie62738c5a580e6e9bdfde2dfc4e85b376fc01056
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5341
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-03 09:41:44 +00:00
Michal Berger
65ed603449 configure: Fix check against NASM version
Currently, NASM in version 2.14 or newer is required. However, the
old code assumed that nasm -v returns version in the form of x.y.z,
however, some builds return only the x.y. In such a case, check would
always fail to detect proper version since the following would be
be always true:

NASM version 2.14

2.14 -> 214 -> [[ 214 -lt "21400" ]]

Change-Id: I63bfc7aa84383bf2d71e936ae534536ecf25723a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4809
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-22 15:01:09 +00:00
Changpeng Liu
6bd7bd0da2 vhost: deprecate internal vhost library support
The internal vhost library is used when DPDK's version
is older than 19.05 and the experiemntal vhost nvme
target.

For the CONFIG_INTERNAL_VHOST_LIB option, SPDK doesn't
enable this option by default over one year and CI
doesn't cover it either, so we may remove it with
this release.

As for the vhost-nvme target, since we are developing
a new vfio-user target solution, it's OK for us to remove
it now.

Change-Id: Ib2cce1db99cd09754307c2828b3187f2d4550304
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
35e54647fb intel-ipsec-mb: update submodule to v0.54
Updated required NASM version to 2.14,
since intel-ipsec-mb v0.54 now requires it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I10cee6aea941593b828a6a171297d7e997d0f30c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-22 11:40:50 +00:00
WANGHAILIANG
fa729fd3a7 configure: add a tips for installing libubsan
In practice, libubsan has been installed but there will still be such alarms sometimes.
In fact, this has something to do with the GCC version.
Most users may not think of it immediately, so they will be confused and waste time here.
So I think we should add a hint here.

Change-Id: Ib01cf4b35be5b75a3620aea556f4c097a1ee7a01
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3832
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-20 09:35:41 +00:00
Tomasz Zawadzki
30a31a16eb sock/vpp: remove VPP implementation
This patch removes implementation of VPP socket abstraction
along with ways to compile it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I089f7703cfc4fb517f8f80f4368e544bced549b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3734
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-08-17 08:19:46 +00:00
Darek Stojaczyk
922d90c806 log: remove backtrace printing
It was broken since long ago. --enable-log-bt doesn't change
anything. log.c expects SPDK_LOG_BACKTRACE_LVL to be defined
for backtrace to work, but it's not defined anywhere.

Apparently nobody needs this, so remove it.

Change-Id: I2313fd24198b0bf718663f2eafee9b5c6efa0a7f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2194
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-03 07:39:08 +00:00
Alexey Marchuk
daee62a05b rdma: Add mlx5_dv RDMA provider
The new RDMA provider can be enabled by passing
--with-rdma=mlx5_dv parameter to configure script
This provider uses "externally created qpair"
functionality of rdma cm - it must move a qpair
to RTS state manually

Change-Id: I72484f6edd1f4dad15430e2c8d36b65d1975e8a2
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1658
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-20 12:03:50 +00:00
Michal Berger
6c23a6f4d9 configure: Add extra dir for header|lib files lookup
These paths are not being looked up by clang (cc) by default, hence
currently it's not possible for vm_setup.sh to compile ref spdk on
FreeBSD as the compiler is not able to find libunwind.* files.

The reason why it looks for libunwind is because get_config_params(),
which is passed to ./configure, uses hardcoded gcc (not $CC) to check
if libunwind is supported.

This fixes the particular issue with libunwind (and potentially other
files for different options), inconsistency mentioned above should be
addressed separately (Issue: #1397).

Change-Id: I2afe11a669ba248ddb7854aa2a36a36372e944f4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2434
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-05-18 10:17:07 +00:00
Jim Harris
cfb65ba611 test: add generic unlink wrapper
reduce library uses unlink, but the unit tests need to
override it in a specific way.

But linking unit tests with LTO requires the wrapper
definitions be in objects/libraries listed *after*
the object/library that refers to it.  So we need to
make the unlink wrapper somewhat generic.  We do this
by exporting a string and callback function that the
user can set to enable a user-defined function to be
called when unlink() is called with a specific file
name.

Also revert 3ef6d06 as part of this patch, since we
no longer require the workaround that it implemented.

Fixes issue #1357.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ee4c424ad31fe7d91d7b524ed47aedd279e5b5c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1948
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-06 12:43:57 +00:00
paul luse
2755fbdf35 scripts: fix configure & autotest scripts that check for Intel proc
Used to determine whether IDXD isto be  configured, previous check
did not work on FreeBSD, only Linux.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I70148ee2f0fffc83c2b89d2de6e81193b9357d3f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2060
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-30 17:51:19 +00:00
Jonathan Richardson
51d69a7f54 configure: Fix parsing of DEFAULT_CC from cc.mk
Commit 649367850 (make: Allow CC and CXX to contain spaces) requires a
change to configure script also so that when it reads DEFAULT_CC from
cc.mk it retains all characters after DEFAULT_CC=. Currently only the
first two fields are retained.

Change-Id: If28493f7f2fb8e96fdf5a653ec30700abc8ce55e
Signed-off-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-29 06:42:09 +00:00
Seth Howell
3ef6d0613d autopackage: Don't build unit tests with lto.
Some of the unittests require symbols that don't work well
with the lto option. For example when you build reduce_ut.c
with the lto option, the __wrap_unlink symbol cannot be found.

Also, while we are here, add a check to configure for unit tests
built with lto.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iaf76cd050565ce2ff13b4230d7a3b735fd5384a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2055
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-04-29 06:32:12 +00:00
paul luse
e58e9fbda8 lib/idxd: add low level idxd library
Module, etc., will follow. Notes:

* IDXD is an Intel silicon feature available in future Intel CPUs.
Initial development is being done on a simulator. Once HW is
available and the code fully tested the experimental label will be
lifted. Spec can be found here: https://software.intel.com/en-us/download/intel-data-streaming-accelerator-preliminary-architecture-specification

* The current implementation will only work with VFIO.

* DSA has a number of engines that can be grouped based on application
need such as type of memory being served or QoS. Engines are processing
units and are assigned to groups. Work queues are on device structures
that act as front-end groups for queueing descriptors. Full details on
what is configurable & how will come in later doc patches.

* There is a finite number of work queue slots that are divided amongst
the number of desired work queues in some fashion (ie evenly).

* SW (outside of the idxd lib) is required to manage flow control, to not
over-run the work queues.This is provided in the accel plug-in module.
The upper layers use public API to manage this.

* Work queue submissions are done with a 64 byte atomic instruction

* The design here creates a set of descriptor rings per channel that match
the size of the work queues. Then, an spdk_bit_array is used to make sure
we don't overrun a queue.  If there are not slots available, the operation
is put on a linked list to be retried later from the poller.

* As we need to support any number of channels (we can't limit ourselves
to the number of work queues) we need to dynamically size/resize our
per channel descriptor rings based on the number of current channels. This
is done from upper layers via public API into the lib.

* As channels are created, the total number of work queue slots is divided
across the channels evenly. Same thing when they are destroyed, remaining
channels with see the ring sizes increase. This is done from upper layers
via public API into the lib.

* The sim has 64 total work queue entries (WQE) that get dolled out to the
work queues (WQ) evenly.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I899bbeda3cef3db05bea4197b8757e89dddb579d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1809
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-23 15:48:32 +00:00
Tomasz Zawadzki
4d2e26d888 configure: fix LTO and CROSS_PREFIX
Starting with patch 873c874, it was not possible to configure SPDK
with LTO enabled or setting a cross compilation prefix.

detect_cc.sh was moved before the configuration options
were read from command line.
This patch fixes this behavior by reading passed options
before invoking detect_cc.sh.

Next loop that reads all options has empty handling for those,
as otherwise "Unrecognized option" condition would trigger.

Added LTO to NIGHTLY release build tests.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I21802e14c0565cb7f323bcbd805967a3c4781b46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1714
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-14 11:36:17 +00:00
Ziye Yang
465f4c1254 sock/uring: Add the async network I/O support for socket
This patch is used to add the async network I/O support in
sock layer. If code is configured with I/O uring, --with-uring,
we can use io uring in Linux (version >=5.4-rc3).

PS: We also make VPP's default priority > uring, because
for the iSCSI or sock test linked with VPP. It tests VPP with
a given address (which is not a special VPP can only open address),
so using uring can also listen those address succefully. And if we make
uring with priority > VPP, actually, VPP will not tested in those cases.

Additionally, the current CI pool is not ready for test, we need
wait for the CI system ready. And I test on my local platform, it works.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie8ee3c8ddf8d2a7264f2b382376733e002816dcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/952
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-20 08:41:28 +00:00
Michal Berger
db3ebfe69a configure: Dump build CONFIG to sourceable file
Change-Id: I33d03d1850c92673f28107d52c886df5bfb685f1
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 14:44:20 +00:00
Michal Berger
c3329b52a0 configure: Use /usr/src/fio as a default location for the fio repo
This allows to run ./configure with plain --with-fio argument
depending on its default CONFIG value.

Change-Id: Icb3facc5a07d38030cda836ec49a54fe9535de71
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1105
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 14:44:20 +00:00
Michal Berger
5a7649559b configure: Use uname for system name check
On FreeBSD, the value that $OSTYPE returns may differ, depending on
which user owns the shell.

To be consistent, use uname call instead as it should always return
the same value ("FreeBSD" in this case).

Change-Id: I2edd221b63ac4bff0ee722cbce6ee6e8a5823e7a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1230
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-12 09:03:38 +00:00
Richael Zhuang
325010bf91 configure: check nasm only on x86
For other architectures like arm64 don't depend on nasm package
when using isa-l or crypto. Limit the check to x86.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I8ee1a711fc76c25a9173853dbb075dffe7df056b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-11 11:35:16 +00:00
Tomasz Zawadzki
9fcb1cb0e1 build: add option to disable building unit tests
If unit tests are not required, add option to disable them.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I387ec043fd47d3033726a51ab673752a521d45c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-10 09:12:21 +00:00
Jacek Kalwas
9b2c645294 build: add option to disable building examples
If examples are not needed using that option can reduce build time and
output logs so it is easier to spot most important issues/warnings.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Id279cba96ddf25d50a8748555d511d21f243bd7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1047
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-04 10:05:10 +00:00
Artur Paszkiewicz
73763d40ec module/raid: raid5 module
Add raid5 module and unit tests. Use './configure' with option
'--with-raid5' to enable it.

Change-Id: I9f07da8c3567fa65499444899c899adaa2e29550
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-04 10:03:45 +00:00
Alexey Marchuk
94966468ae nvme/rdma: Introduce transport_ack_timeout
Add transport_ack_timeout parameter to nvme controller opts.
This parameter allows to configure RDMA ACK timeout according
to the formula 4.096 * 2^(transport_ack_timeout) usec.
The parameter should be in range 0..31 where 0 means use
driver-specific default value.

Change-Id: I0c8a5a636aa9d816bda5c1ba58f56a00a585b060
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/502
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-27 10:16:00 +00:00
Tomasz Kulasek
873c8748eb config: fix fail if explicit dependency not met
This patch modifies default configuration disabling features
with known issues or incompatibilities.

If user tries to explicitly turn on feature which dependences are
not met it exits with error immediately.

	Fixes issue #1193

Change-Id: If7363506ee812c5135f5a3a023ab9e8d26268528
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-27 10:15:02 +00:00
Alexey Marchuk
157b1a339a configure: Check nasm - redirect stderr to null device
configure prints an error if nasm is not installed on the system:
./configure: line 554: nasm: command not found
Notice: ISA-L, compression & crypto auto-disabled due to nasm dependency.
These features require NASM version 2.13.03 or newer.  Please install
or upgrade then re-run this script.
Redirect stderr to null device to disable error printing

Change-Id: Ifa596fba19ecaf5270614ab22f09f0a66af08475
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-21 09:39:41 +00:00
Seth Howell
cbc1e069ee configure: allow cuncurrent build of RBD and reduce
This was disabled in configure because at one time there was an ISA-L
dependency break between the RBD libraries and our own submodule.
It seems that the dependency break was fixed a while ago, but the check
to not allow one to build with the other was never removed.
Unfortunately, I don't have the details on what that fix was.

Either way, this compilation works now.

Change-Id: Ic249791549dae36c7279114d67e77be94e842ddf
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479726
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Richael Zhuang
8cad9cc10a spdk: enable isa-l by default aarch64
In addition to x86, enable isa-l by default on aarch64.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I2db3f248bdc857326ec9b6f4b040a95d2a2ca1ca
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:14:41 +00:00
Alexey Marchuk
e4aee795c0 configure/ISA-L: Move log file to submodule directory
Running configure script by multiple users on the same machine
leads to access to the same ISA-L log file which is created
with read access for user's group. As the result only the
user who created this file can modify it and other users will
receive "permission denied" error. Move this log file to the
local ISA-L directory to prevent access by multiple users

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I1862738bc0ba242c98816a22dc39bd11ef5d02dd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-14 03:46:17 +00:00
Tomasz Kulasek
805d994281 lib/nvme: add NVMe character device
NVMe character device implementation. This patch adds implementation
of IO producer using CUSE library. It allows to create nvme device
nodes in linux kernel for controller as well as for namespace and
process ioctl requests as usual from linux environment.

Both devices (controller and namespaces) are exposed as character
devices.

To compile NVMe CUSE module use "./configure --with-nvme-cuse".

Names for created CUSE devices can be retrieved using
spdk_nvme_cuse_get_ctrlr_name() and spdk_nvme_cuse_get_ns_name().


Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I0fc9a9a1ef3c9c2b3112d07c2b4b1f8d49665ee1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-10-24 23:43:59 +00:00
Xiaodong Liu
7fa15e28e4 configure: add --with-fuse option
blobfs_bdev module may contain functions related to
FUSE which will utilize libfuse3.
By default, it is diabled to compile blobfs_bdev module
with FUSE related functions. Running './configure' with
option '--with-fuse' can enable it.

Change-Id: I6552a6c04cc3412c739691630a7a481e0ae6b59c
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-10-10 16:20:46 +00:00
Alexey Marchuk
131c9aad05 configure: Use -march=native for x86 architecture only.
Exclude -march=native parameter from build command for other architectures since it causes a compilation error in ARM.

Change-Id: Ida22ea4b3afba9759102d2e2203fc9e6e312c3ba
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469390
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-10-01 14:02:40 +00:00
Tomasz Zawadzki
d43ee98c36 configure: disable internal rte_vhost on FreeBSD
On master DPDK (d03d8622d), building SPDK on FreeBSD
fails due to attempt to build internal rte_vhost.

This started occuring with SPDK patch:
28099e0 "make: add dependencies for the rte_vhost build."

This patch disables VHOST_INTERNAL_LIB on FreeBSD,
along with vhost/virtio.

Errors seen:
./configure --enable-debug --enable-werror --with-fio=/usr/src/fio --without-isal --with-dpdk=/var/jenkins/workspace/Other_systems/freebsd_autotest/dpdk/build
Using default SPDK env in /var/jenkins/workspace/Other_systems/freebsd_autotest/spdk/lib/env_dpdk
Notice: DPDK's rte_vhost not found or version < 19.05, using internal, legacy rte_vhost library.
Vhost is only supported on Linux. Disabling it.
Virtio is only supported on Linux. Disabling it.
...
gmake -j4
...
  CC lib/rte_vhost/socket.o
In file included from socket.c:52:
./vhost.h:41:10: fatal error: 'linux/vhost.h' file not found
#include <linux/vhost.h>
         ^~~~~~~~~~~~~~~
....
  CC lib/rte_vhost/vhost_user.o
vhost_user.c:41:10: fatal error: 'asm/mman.h' file not found
#include <asm/mman.h>
         ^~~~~~~~~~~~

Change-Id: Ia2f5fd98a82412a5691bc0f2201f7259a45d2b4d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468043
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-09-20 20:33:14 +00:00
Seth Howell
00dd9e8519 configure: add -march-native to dpdk build check.
Some older versions of gcc fail with errors when -march=native is not
supplied to the build cmd for checking DPDK rte_vhost support.

Change-Id: I88e12f6823b2a143dc8406c7c01041c00c25011c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-09-20 20:22:05 +00:00
Seth Howell
a988124db9 make: override scan-build for unit tests.
The unit test mock macro function doesn't mesh well with the scan-build
static analysis tool. So we need to reset cc to the user provided value
or default before running the unit tests.

Change-Id: Ib636f6e396b09f9bf1cb8f930e6befa56ce99848
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466816
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-09-11 16:30:38 +00:00