Commit Graph

81 Commits

Author SHA1 Message Date
Darek Stojaczyk
ce71172650 configure: fail --with-pmdk if libpmemblk is not available
Change-Id: Ibbd84374dac6c2ac2eba30dd2664b8a4b56277a6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453707
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
eed0c6be16 configure: fail --with-rdma if it's not available
While here, also refactor IBV_WR_SEND_WITH_INV check
to use the new BUILD_CMD command.

Change-Id: Ieefe77d769f2b1369d204ec54b976fa11441af46
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453706
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
32b356cd04 configure: check IBV_WR_SEND_WITH_INV not only for Linux
If the user wants to build RDMA on BSD and IBV_WR_SEND_WITH_INV
is unavailable there he'll now get a warning message.

Change-Id: Ie6c6ce55d21de519bb371ceed44fb1e01d6dc878
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
c4a3b31278 configure: don't assume /usr/include/ is the header location
It doesn't have to be. In order to check if the system
header is available we'll now to try to compile a C program
that includes it.

Change-Id: Ie46c20dab5e96da746650bba1d5dbd0fc32ecec9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
00a6c491d2 configure: try harder to parse nasm version
Nasm doesn't always include the patch version in its
version string and we currently detect 2.14 as older
than 2.13.03. Fix it by checking the major and minor
version numbers separately.

Change-Id: I37d1129775b17d252b84a911ee1c849f4f0e2ec7
Reported-by: James Bergsten <jamesx.bergsten@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452533
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-05-03 20:54:14 +00:00
Jim Harris
af25204404 bdev/nvme: always enable FTL
FTL doesn't have any kind of special package requirements.
It is getting pretty good traction in the community, so
let's enable it by default.

Note that we will disable FTL on FreeBSD.  FTL uses
CIRCLEQ which is not available on FreeBSD.  Let's not
spend time trying to get FTL to work on FreeBSD until
there's a demand to do so.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15525b6c4e6ee52f49adf74d55f9484fe08a6dcc

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-02 08:41:56 +00:00
Darek Stojaczyk
5316b12e5e configure: bump the required nasm version to 2.13.03
ipsec-intel-mb bumped the requirement in its v0.52.

Change-Id: I41d40e03720dca6146035bf261a4bc684a1bb52f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452045
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-29 15:48:01 +00:00
Ben Walker
8165bf7125 build: Add support for linking to liburing
liburing is a wrapper around a new Linux kernel interface for
submitting I/O using user space rings. This patch simply
adds the ability to link the nvme perf tool to liburing, but
doesn't implement any of the new functionality yet.

Change-Id: Idb741c87b6d951c013af86e30eac18d3834dd4b7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-19 20:59:19 +00:00
Ben Walker
5ffeed2582 configure: Make indentation consistenly use tabs
We don't have enforcement for bash scripts, so of course
this ended up with mixed tabs and spaces.

Change-Id: Ideba6cdffa0687a40851a5cda6216a13f870a9ba
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447242
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-13 23:19:37 +00:00
Darek Stojaczyk
2b846acc49 configure: add option not to use the internal rte_vhost copy
It's disabled by default, so no functionality is changed yet.
The intention is to use the upstream rte_vhost from DPDK,
which - starting from DPDK 19.05 - is finally capable of
running with storage device backends.

SPDK still requires a lot of changes in order to support
that upstream version, but the most fundamental change is
dropping vhost-nvme support. It'll remain usable only with
the internal rte_vhost copy and with the upstream rte_vhost
it simply won't be compiled. This allows us at least to
compile with that upstream rte_vhost, where we can pursue
adding the full integration.

Change-Id: Ic8bc5497c4d77bfef77c57f3d5a1f8681ffb6d1f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446082
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-13 14:26:20 +00:00
Pawel Niedzwiecki
4a5e97c9d0 configure: Add "ftl" dependency co configure's help message.
--with-ftl dependeny is missing in spdk configuere's script
help message.

Change-Id: I505a2914073d392809d243d8df8b42c0f5d79dab
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447320
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-08 21:00:06 +00:00
Piotr Pelplinski
d6c4518c69 ocf: Move ocf configuration to Makefile
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I116b53319152b229900e56afc6277340a9daa988
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-03-08 19:16:28 +00:00
James Bergsten
d2bbef66dc lib/util: Add ISA-L and faster table-based options for T10 CRC-16 generation
This patch extends existing ISA-L support for NVMe/SCSI T10 CRC-16
protection check (PI). The choice of which algorithm to use is set
via an existing configure parameter.  The default for x86 systems with
appropriate CPU instructions is to us ISA-L.

Testing shows that PI checking/generation will run at device speed using
ISA-L.  The table-based algorithm is much faster than the original algorithm
and is provided to speed up calculations on non-x86 systems.

This patch also fixes one issue in a false configure error if nasm
is at a higher version than currently checked (i.e. 2.13.03)

Local performance testing yeilds these results:

Original -    15 mbit/sec. (code removed)
Table    -   400 mbit/sec. (27 times faster than original)
ISA-L    - 3,400 mbit/sec. (227 times faster than original)

Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Change-Id: Idc7f1d97252644d839621ffe9fa995edd922da52
Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/443646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-08 15:39:32 +00:00
Piotr Pelplinski
98d60de60f ocf: add ocf submodule
This patch adds OCF as spdk submodule. By default ocf submodule is unused.
Type ./configure --with-ocf to enable ocf compliation.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I7df228481c4228e6dbdea2ef0dc1d5513069ee08

Reviewed-on: https://review.gerrithub.io/c/444256
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-27 17:26:51 +00:00
Ben Walker
954728e9df build: Support PGO on clang.
Change-Id: I837c6c35c7ab34f5610d782647e333db2aff1f63
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-19 06:39:10 +00:00
Ben Walker
06cc80459f configure: Make --with-env and --with-dpdk mutually exclusive
Also put the DPDK/env options next to each other in configure.

Change-Id: Ic167c37046300e69212ec0b97b6831949f4d60ad
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444111
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-13 09:36:28 +00:00
Ben Walker
995d60f493 build: Add support for profile guided optimization
./configure --enable-pgo-capture
make

<Run sample workload>

./configure --enable-pgo-use
make

Output has used the captured profile from the sample workload to improve
performance.

Change-Id: Ie5690b873b05d11b4ea6c7d44021e564d4e4d170
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444110
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-13 09:36:28 +00:00
paul luse
d9d4e40dd2 bdev/compress: Add configure option and build dependencies
Includes the required DPDK dependencies for SPDK block Reduce aka
Compression.

Change-Id: Ic1ea3cbeb9373a7700f6f0c2a3194d65d6a34a41
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/429523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-11 19:23:17 +00:00
Pawel Wodkowski
c796c54794 configure: detect IBV_WR_SEND_WITH_INV instead checking version
Checking version of libibverbs is error prone as custom version might be
installed that implements needed features but version number is not
incremented. Instead test if we can compile with needed features.

Fixes #524

Change-Id: I18e9ca923eea92b124e95a5f660955a01afad5c4
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-07 23:46:16 +00:00
gila
8629d8da66 configure: update how CPU arch is determined
The -i option for uname is not portable, -m is a better choice.
Fixes #648

Signed-off-by: gila <jeffry.molanus@gmail.com>
Change-Id: I2287e652e8d3243df2bf101c1cfbdc6aedf643f1
Reviewed-on: https://review.gerrithub.io/c/443315
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-05 22:26:46 +00:00
Ben Walker
7b74567eb1 configure: Correctly hyphenate ISA-L
Change-Id: If5d40ed4784fecd50f5d675512a8f2f9f9967492
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442624
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-02-04 07:04:35 +00:00
Ben Walker
a89dbbffa2 configure: Reword message printed when ISA-L not found
Change-Id: Iecd649dce5dcd7a3a5169e4d0e3233a26cb05578
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442614
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-02-04 07:04:35 +00:00
Darek Stojaczyk
d555d6c2d0 configure: remove deprecated RAID options
Change-Id: Ibbb53fefad5fa117c9b630301fb50bc04c87c3ea
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442728
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-30 17:58:10 +00:00
Darek Stojaczyk
385f42eb97 configure: fix building without isa-l submodule
The isa-l directory is generated automatically by
git on checkout, although it's empty. We check for
that directory existence in order to build ISA-L
and without `git submodule update --init` the build
just fails.

To fix, instead of checking the dir existence, we
check for the autogen.sh file inside that dir.

Change-Id: I7b62513afcd7e9f3953f825c41b265efe3484d57
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-29 17:42:49 +00:00
Ziye Yang
552e21cce6 spdk: Add ISA-L support with related crc32 function
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.

Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.

Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
2019-01-29 08:31:00 +00:00
Vitaliy Mysak
d1fee489a7 OCF: add OCF module
Add OCF module based on OCF meta-library
Open CAS Framwework (OCF) is high performance block storage
  caching meta-library
It is open-source, published at https://github.com/Open-CAS/ocf

With this patch OCF-enabled device is represented in SPDK
  as virtual bdev having core and caching devices as its base devices

This patch includes implementation of:
  * OCF top adapter          (vbdev_ocf.c)
  * OCF bottom adapter       (dobj.c, data.c)
  * Adaptation layer for OCF (env/)
  * OCF context abstractions (ctx.c)

Adaptation layer and context abstractions are not dependent on SPDK bdev

OCF bdev supports reads and writes, configured at startup
Other features will be added with separate patches

Change-Id: Ic2dcab378c8238d16f1e4b64d4374bdf257565bc
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435708
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-28 09:41:52 +00:00
Jim Harris
f823b7f5c1 ipsec_mb: use EXTRA_CFLAGS to point DPDK to intel-ipsec-mb submodule
This allows us to remove the requirement to install intel-ipsec-mb to
system directories.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I579655a98b515cf148b7cd17823a9bb541ea6ad7

Reviewed-on: https://review.gerrithub.io/c/440785
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 17:21:02 +00:00
Wojciech Malikowski
ef2e614cef ftl: Initial implementation
This patch adds core FTL (flash translation layer) submodules.

On regular SSDs the FTL is part of firmware, whereas Open Channel
enables moving it to the host and allows for having full controll
of data placement on the device.

Main functionalities added:
 * logical to physical address map
 * read / write buffer cache
 * wear-leveling
 * bad block management

Change-Id: I5c28aa277b212734bd4b1f71ae386b3d6f8c3715
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-11 09:15:39 +00:00
Pawel Wodkowski
a5380c752b configure: detect libibverbs using ldcofnig
This should be most robust way of detecting it.

Fixes #524

Change-Id: Ic1e50500ca5eb51177ac5197c59005b133cd8367
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/435996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-06 22:42:06 +00:00
Pawel Wodkowski
6f2e275aab configure: save used environment variables in separate file
currently they are saved in mk/config.mk but this break config.h

Change-Id: Iecb037f1aea91469f4093724cdf100dc122d21a5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427222
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-03 20:25:44 +00:00
Ed rodriguez
9ba446a25c test: Add support to configure ipsec git repo
Also make ipsec configuration and install optional
  in pkgdep.sh.

Change-Id: I0c973398cebce0f50c704b21f6397e8d996ba9f9
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/432424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-12 22:27:35 +00:00
Jim Harris
970228038f reduce: add configure option
--with-reduce required to build reduce.  This depends on
libpmem being installed.

We still need to work out details in pkgdep.sh and
vm_setup.sh.  Some distributions like Ubuntu still
require configuring extra package repositories to
get libpmem packages.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4e056ce1da9a1fecb4458f8f5e7ff5d61c422533

Reviewed-on: https://review.gerrithub.io/430646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-08 00:01:47 +00:00
Darek Stojaczyk
9e6558746f dpdk: update the dpdk submodule to v18.08
Also use the DPDK's default T=x86_64-native-* configuration
target instead of T=spdk-*, as the T=spdk-* is no longer
available in spdk-18.08 branch of our DPDK fork.

Change-Id: If12e7ae1acabece129b63ba2f1bec6b0b65678ca
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/423053
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-22 18:01:45 +00:00
Seth Howell
9abf7e8f00 make: build the igb_uio kernel module with crypto
Also, update the qat_setup script to point to this module.

Dependent upon:https://review.gerrithub.io/c/spdk/dpdk/+/428718

Change-Id: Ie8694a1996621165f52a513d9b19c7232aaed9ce
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428719
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-19 14:27:47 +00:00
Pawel Wodkowski
ddd38a5e9d configure: remove CONFIG.local symlink
nvme-cli is updated to use mk/config.mk so we can now remove
CONFIG.local symlink.

Change-Id: I7d39c2559e43198b9e0bc4519edaa1cb3180de07
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/428373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-19 14:26:14 +00:00
paul luse
2975822ffe misc: Remove experimental comments regarding crypto
Change-Id: I1635a81f6b6417b5b632f6fc244f815a30df45b7
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/429514
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-18 16:18:00 +00:00
Pawel Wodkowski
85bc2bbe7d configure: use mk/config.mk instead of CONFIG.local
We have hairy configuration method. First configure script uses CONFIG,
environment and arguments to produce CONFIG.local and config.h then
makefile uses CONFIG, environment and CONFIG.local to produce config.h
again. First time config.h can't be generated reliably as config.h
target in Makefile is passing MAKEFLAGS and they are unknown at
configuration phase.

Lets remove this mess by producing final mk/config.mk file and get rid
of CONFIG.local. This will also generate full build configuration in one
file instead scattering them in CONFIG and CONFIG.local.
In next patch scripts/genconfig.py can be removed.

Change-Id: I5696813312edff7e493207f7aa8569234a365ec0
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/426364
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-10-16 12:40:43 +00:00
Seth Howell
2b6d9ce21f configure: Move checks for nasm and ipsec
The way they were previously being checked was triggering the trap and
printing out a "Configuration failed" message even though the
configuration was successful.

Change-Id: I9de4f390c603631ebf5af5555ea7164aae2b6213
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428663
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 18:50:27 +00:00
Pawel Wodkowski
48f70f48cf configure: add concistency check of CONFIG file vs CONFIG[@] array
If someone adds configure option in configure script but forgot to
update the CONFIG file following error will be printed:

$ ./configure
Using default SPDK env in /home/pwodkowx/Private/spdk/lib/env_dpdk
Using default DPDK in /home/pwodkowx/Private/spdk/dpdk/build

BUG: Some configuration option are not present in CONFIG file. Please
update this file.
Missing options in CONFIG (+) file and in current config (-):
--- CONFIG file
+++ CONFIG[@]
@@ -9,6 +9,7 @@
 CONFIG_ISCSI_INITIATOR
 CONFIG_LOG_BACKTRACE
 CONFIG_LTO
+CONFIG_MISSING_OPTION_IN_CONFIG_FILE
 CONFIG_PMDK
 CONFIG_PMDK_DIR
 CONFIG_PREFIX

Change-Id: If313a95caf9dc657fc81e78baf85ac86f20565e1
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/428086
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-04 23:00:22 +00:00
Pawel Wodkowski
14bbde7bd4 configure: add message showing if something failed
Change-Id: I9ec15e8ec081786cea11a50abbd3c05a0d5a0ce6
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-04 21:32:21 +00:00
Pawel Wodkowski
577df16ab9 configure: switch to arrays when generating CONFIG.local
This patch is preparation for including build configuration in fedora
devel package.

Change-Id: I174e2a9392a2be2efa9d8032cb8c4d8360208781
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-04 21:31:42 +00:00
Pawel Wodkowski
461cfcecc6 CONFIG: add missing CONFIG_ options
Also fix options that doesn't start with CONFIG_ prefix.

Change-Id: I32a24373328bab8f513d56c0c09b5a86589f690c
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427767
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-04 21:31:42 +00:00
Pawel Wodkowski
ae2b2f64ea log: add a way to turn on and off stack unwinding in logs
As a side effect, if SPDK_LOG_DISABLED is used in spdk_log() as log
level then no message will be printed.

Change-Id: I2d57b60a5a310a9ef2a1187a81088d0acf828742
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425105
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-02 22:09:27 +00:00
Pawel Wodkowski
3b9baaf0f4 configure: don't generate config.h during configure
This script don't respect MAKEFLAGS here. Also config.h is dependency
during build so no need to invoke it multpiple times.

Change-Id: I2062819cb5d3f1ab87d8127a1a41c209515b199c
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/426363
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-26 20:56:50 +00:00
Jim Harris
1d64773c2a raid: enable by default
We kept this disabled by default originally because it lacked
multiple iov support needed for vhost and nvmf in-capsule data.
Now that support has been added, there is no reason to not
enable it automatically - it has no extra package dependencies,
so requiring users to specify --with-raid to configure is just
an extra unneeded step.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia9796370e12a6f93993e70083de36114a37851bd

Reviewed-on: https://review.gerrithub.io/426003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-19 17:22:31 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
Paul Luse
b324f73cb6 configure: fix typo and add exit on error
When wrong nasm was detected we marched on anyway. As seen in CI, the build
of ipsec just fails sorta ugly with an older version.

Change-Id: I03d1e5774386e6aed354e8cb6877160f507a5c58
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/423754
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-28 23:01:28 +00:00
Paul Luse
0fb1e1c524 ipsec: updates to require ipsec installation
Although a pre-req to crypto only, the latest ipsec library now
requires root to install.  This patch takes 2 approaches to making
sure the ipsec module is installed properly.  First, in configure,
if crypto is enabled it will check to see if ipsec is already installed
and if so do nothing.  If it is not it will exit and instruct the user
to manually install and re-run the configure script.

Secondly, to be proactive, this patch adds the installation, if not
already installed, to the pkg dependency script because it is already
executed as root.

It also does the same for nasm, checking the required versions in both
places.

Change-Id: I037629f21be474b88c12451733ac12ff0b1651f5
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/422746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-27 16:33:16 +00:00
Jim Harris
17a819f882 configure: use echo to print RDMA invalidate warning
Using cat << EOF seems to cause problems on some tests
(like iscsi ext4test.sh) where we copy the SPDK repository
into a filesystem on an iSCSI LUN, and do a (cd $dir &&
./configure).  So use echo instead to work around this
problem.

Fixes GitHub issue #412.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8863b5060f41a48c2ace177f7440b85dc43f187d

Reviewed-on: https://review.gerrithub.io/423163
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-23 16:07:42 +00:00
Seth Howell
7671073bed config: add define for rdma send with inval
On newer versions of the kernel, we see a significant performance impact
if send with invalidate is not enabled. However, depending on the user's
version of IB verbs, it may not be supported. We need to warn them of
the performance issues.

Change-Id: I93a5f5375af90ca3102a6bd084df430b1153419c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/422460
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-17 20:56:02 +00:00