Commit Graph

366 Commits

Author SHA1 Message Date
Seth Howell
a988124db9 make: override scan-build for unit tests.
The unit test mock macro function doesn't mesh well with the scan-build
static analysis tool. So we need to reset cc to the user provided value
or default before running the unit tests.

Change-Id: Ib636f6e396b09f9bf1cb8f930e6befa56ce99848
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466816
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-09-11 16:30:38 +00:00
Ben Walker
52ccf4b761 sock/posix: Increase SO_RECVBUF size by default
Everywhere we use this, we increase the size to 2MB.
Just make that the default behavior.

Change-Id: Ie174419a09df1792a0c7311eddd0c2dcefa267b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466991
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-09-10 17:48:49 +00:00
Seth Howell
28099e0ed9 make: add dependencies for the rte_vhost build.
The dependencies between vhost and rte_vhost were not added during
earlier changes. This change moves the rte_vhost directory up to the
level of the other libraries and adds the proper dependencies for when
it is linked.

Change-Id: I089de1cd945062b64975a0011887700c0e38bb0f
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467700
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-09-09 22:15:24 +00:00
Seth Howell
5c62618f26 module/event: add subsystem interdependencies
The SPDK application framework defines a list of event subsystem
dependencies. When linking against individual shared libraries, it is
useful for this dependency structure to be codified in the shared
libraries themselves.

For example, when linking a bdev based application against
libspdk_bdev.so, one might wish to only specify this shared object at
link time. However, when you actually run the application, it will fail
to start because it is not linked to the copy and vmd subsystems.
However, once thesedependencies are added, one can effectively link
against only the exact subsystems they need and any dependent subsystems
will be linked in automatically.

Change-Id: Ic986281a162ac20b523486e9f8cccf4a0787afd7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-08-29 20:16:18 +00:00
Seth Howell
407e88fd2a lib/mk: update OCF build.
The OCF build was broken by some of the recent changes
to the Makefiles. This change aims to fix that by separating out the ocf
environment from the ocf bdev.

Change-Id: Id445340033898e9ae70a4bcfc799951110762d55
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465808
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-27 18:49:56 +00:00
Seth Howell
7e1881cef9 module: add shared lib deps for all modules.
This will allow us to link individually against dpdk module libraries
without having to define those libraries dependencies.

Change-Id: Ief2140ec0fadd970aba990dab333d603dfb46317
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465704
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-08-27 18:49:56 +00:00
Seth Howell
f66caa1f80 make: start adding inter-lib dependencies.
This will make it much easier to link against some subset of spdk
libraries when building SPDK applications.

Change-Id: I8ad95a001965288a8b5e38eb252391fef68d7138
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-27 18:49:56 +00:00
Seth Howell
9d14641dd5 mk: add a lib deps file.
By adding this file and modifying the way we define a couple of
variables in the makefiles, we can actually avoid having to redefine
the library dependencies on a per file basis.

Change-Id: Ieab4aa1021b0341fc21e3b65677a9ad7f70559c2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466063
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-08-23 17:48:35 +00:00
Seth Howell
07fe6a43d6 mk: move the bdev modules under module directory.
This is more accurate to what they are, and will make defining library
dependencies much simpler. This change in directory does not affect the
final placement of naming of libraries at the end of time.

Change-Id: Ic48a9233dff564e39ce357a9ea0a111ea2b6414b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-08-22 16:29:49 +00:00
Seth Howell
2a3b6b4179 mk/lib: Build subdirs before lib dir
The subdirectories of libraries are currently listed as dependencies of
the library. If you look at the actual symbols they contain, the
opposite is true. For all of the bdevs, event subsystems and other
library directories, the subdirectories actually rely on symbols
contained in the parent directory.
If we want to create the proper inter-library dependencies in SPDK at
compile/link time, then we need to respect the order of these symbol
dependencies when executing make (e.g. make libspdk_event.so before
libspdk_event_bdev.so). Otherwise we create a loop of dependencies in
the make system.

Change-Id: Idcbb050b2709f575d0cec46b481824d6313612b4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465191
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-15 20:32:28 +00:00
Seth Howell
e29c6fe133 mk: force --no-as-needed for spdk libs.
When building shared libs, we specifically don't set the --as-needed
flag so that we still link against libraries that have constructor
functions. LD reports on its man page that the default behavior is
equivalent to --no-as-needed.

For RHEL based distros like fedora and centos this works fine. While
the LD man page shipped with ubuntu distributions also states that
--no-as-needed is the default, they don't respect that behavior and do
--as-needed linking unless you force them with --no-as-needed.

Change-Id: I914ab849323de198af5c5e53fffb1f57fcaff5fe
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-12 21:12:16 +00:00
Darek Stojaczyk
35a4f3a4bd mk: clean up BLOCKDEV_MODULES_LIST in spdk.modules.mk
Reorder BLOCKDEV_MODULES_LIST entries - put dependant
libraries first. Also break lines after 80 characters.

Change-Id: Ib09b28938253435f335c4e1ebe83928ab551194c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-08 22:41:30 +00:00
Ben Walker
73439e6f46 build: Detect compiler toolchain in configure
We need to know this up front.

Change-Id: I3a9ceb90cf62eacbf3fdf518a9ccb4c4978b3a05
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463014
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-26 16:22:27 +00:00
Ben Walker
0e6ddfd66a build: Allow the user to set target architecture in configure
Change-Id: I7f2a1464c0c8872e5e7856a830aaf7ea8cdf4e81
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 22:55:06 +00:00
Ben Walker
ebd32197ba build: Allow TARGET_ARCHITECTURE to be specified on ARM
Previously it always forced armv8-a+crc. Now it only does that
if not otherwise specified.

Change-Id: I7c1d73fca8e89779cc3e8780a38f52975488d97a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 22:55:06 +00:00
Ben Walker
7dbdf87f63 make: Allow users to specify the target build architecture
make TARGET_ARCHICTECTURE=broadwell

Change-Id: I868c2f5c203dfc1a4e6cf48c39b391e174d5d33b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 17:01:17 +00:00
Tomasz Kulasek
14a62d67ac net/vpp: allow to compile with vpp
This patch prepares environment to allow compile applications and
libraries with VPP.

Change-Id: Icc067c17b57932062afa5e6a67791bd58de3a2cc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-06-27 08:23:08 +00:00
Wojciech Malikowski
9d92b9d420 examples/nvme/perf: Optional VMD enumeration
New flag -V was added for VMD enabling.

Change-Id: Ie0feae1bdfce39ad246f57411d00587c6417d21d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450926
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 03:59:36 +00:00
Seth Howell
3fb4a66d7d bdev: Create a new delay vbdev module
This module simply sits on another virtual bdev
and adds a simulated average and p99 latency to that drive.

Change-Id: Ie9fc91e27585fd0636cb7dc845cb41744bf24625
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-31 14:37:26 +00:00
orden smith
68eda44030 vmd: Initial SPDK VMD baseline code
Signed-off-by: Orden Smith <orden.e.smith@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Change-Id: I4a3c4d1ca8d0b18201edf99a67a3d75ca7ab8153
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449499
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-30 17:32:43 +00:00
Piotr Pelplinski
c4d9daeb7b Makefile: Add possibility to uninstall spdk.
Add uninstall target to makefiles to be able to perform
reverse of install target.
Fixes #464

This patch adds 'uninstall' target to makefile.
'make uninstall' will remove spdk_tgt app, headers, libraries
and shared libraries from system directories defined by $DESTDIR.
Additionaly, if there will be any empty directories left after
this operation, they will be removed as well.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I7b07fb4b81081d3914ff09165991fbe3a26b9067
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/431471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-16 20:56:18 +00:00
Ben Walker
49bff95f09 test: Add test_sock.c, a mocked up socket library
Instead of linking to the real sock library in the unit tests,
link to a mocked up version like we do with the env.

Change-Id: I9607f135c6af86e09fa52badf44897f90ce94521
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454492
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-15 18:45:46 +00:00
Richael Zhuang
5327875c74 lib/ftl: fix PAGE_SIZE for AArch64
Not only 4KB page size is supported on AArch64. The most common is 4KB
and 64KB. 16KB is supported too. We can get it by "getconf PAGESIZE".

Change-Id: I6f09741372d35e868228528d976fe3b5a65ff96c
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454447
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-14 22:48:56 +00:00
Alexey Marchuk
744adf39ee build: Fix ARM compilation
ARM system doesn't provide definition of PAGE_SIZE. The patch adds the definition of that symbol to the compiler flags for ARM system

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Change-Id: Iae0a528569aedaea7df95d9f0d9256cb23b9798b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453657
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-08 21:24:39 +00:00
Jim Harris
cb0c0509f4 build: add full version to SONAME for shared libraries
Fixes issue #781.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic0793721cb996ec01286ff21ed10fe2076b706ec

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452895
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-06 16:27:51 +00:00
Jim Harris
af25204404 bdev/nvme: always enable FTL
FTL doesn't have any kind of special package requirements.
It is getting pretty good traction in the community, so
let's enable it by default.

Note that we will disable FTL on FreeBSD.  FTL uses
CIRCLEQ which is not available on FreeBSD.  Let's not
spend time trying to get FTL to work on FreeBSD until
there's a demand to do so.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I15525b6c4e6ee52f49adf74d55f9484fe08a6dcc

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-02 08:41:56 +00:00
Paul Luse
1717b5d580 bdev/compress: add reduce integration
Basic integration, init and load sequences

Change-Id: I12595a251f38168aad15e95275651cb4ce40ea7d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435764
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-27 19:17:46 +00:00
Paul Luse
db541f8eb3 bdev/compress: add compression vbdev module
Just the starting point, base functions and structs.

Change-Id: I2ff2b672aa675a19b89e4449f1cc4aa664007f6f
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-27 14:51:22 +00:00
Vishal Verma
dfb60590c9 bdev: Add io_uring bdev module
This adds bdev io_uring module.

Change-Id: I9a8171d7c871673b189bff59a89d06679da4e191
Signed-off-by: Vishal Verma <vishal4.verma@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445334
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-22 18:51:48 +00:00
Jonathan Richardson
1c96c421eb build: Don't pass -fuse-ld to compiler if LD_TYPE not set
Don't assume detection of LD by the detect_cc script suceeded. If it
wasn't detected then LD_TYPE won't be set and the gcc command line
option shouldn't be set either. Compilation will fail if -fuse-ld is
specified with no value.

Change-Id: If4a0ede8cdc5a71cf92de2beb0f24e4650f58f3a
Signed-off-by: Jonathan Richardson <jonathan.richardson@broadcom.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447376
Reviewed-by: Scott Branden <sbranden@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-12 21:38:59 +00:00
Maciej Szwed
25d26b910f ftl: Make FTL bdev a part of NVMe bdev
This is a beginning of series that will provide
common code for all NVMe based bdevs.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I097b4a03ef79277659f45d217c7e4e58236c98bf

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445893
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-04 19:00:23 +00:00
Jim Harris
14e443bdfb build: fix duplicated clean target in shared_lib/Makefile
Add a CLEAN_FILES macro that shared_lib/Makefile can use
to add to the list of files to be cleaned.

Fixes #663.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I12982e0989e02a69aaea4e470777301280090096

Reviewed-on: https://review.gerrithub.io/c/444427
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-14 05:15:40 +00:00
Ben Walker
995d60f493 build: Add support for profile guided optimization
./configure --enable-pgo-capture
make

<Run sample workload>

./configure --enable-pgo-use
make

Output has used the captured profile from the sample workload to improve
performance.

Change-Id: Ie5690b873b05d11b4ea6c7d44021e564d4e4d170
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444110
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-13 09:36:28 +00:00
Ziye Yang
552e21cce6 spdk: Add ISA-L support with related crc32 function
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.

Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.

Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
2019-01-29 08:31:00 +00:00
Vitaliy Mysak
d1fee489a7 OCF: add OCF module
Add OCF module based on OCF meta-library
Open CAS Framwework (OCF) is high performance block storage
  caching meta-library
It is open-source, published at https://github.com/Open-CAS/ocf

With this patch OCF-enabled device is represented in SPDK
  as virtual bdev having core and caching devices as its base devices

This patch includes implementation of:
  * OCF top adapter          (vbdev_ocf.c)
  * OCF bottom adapter       (dobj.c, data.c)
  * Adaptation layer for OCF (env/)
  * OCF context abstractions (ctx.c)

Adaptation layer and context abstractions are not dependent on SPDK bdev

OCF bdev supports reads and writes, configured at startup
Other features will be added with separate patches

Change-Id: Ic2dcab378c8238d16f1e4b64d4374bdf257565bc
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435708
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-28 09:41:52 +00:00
Wojciech Malikowski
7c233adeff bdev_ftl: Added FTL bdev module
This patch adds FTL bdev. RPC scripts have been updated to allow for
creation and removal of FTL bdevs.

Change-Id: I82a5c5033b65bbeb67c238cae969a68cff767dcc
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431329
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 18:55:56 +00:00
Jim Harris
f823b7f5c1 ipsec_mb: use EXTRA_CFLAGS to point DPDK to intel-ipsec-mb submodule
This allows us to remove the requirement to install intel-ipsec-mb to
system directories.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I579655a98b515cf148b7cd17823a9bb541ea6ad7

Reviewed-on: https://review.gerrithub.io/c/440785
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 17:21:02 +00:00
Richael Zhuang
e3910413b2 Update spdk crc32c.c with ARM CRC32 intrinsics
Implement spdk_crc32c_update() with ARM CRC32 intrinsics.

Change-Id: I1daf7f21012aab02290f88a65bbae619eedf5087
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/437218
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-16 20:39:18 +00:00
Ben Walker
5977aad8f7 thread: Eliminate use of pthread_self and thread_ids
SPDK threads are no longer mapped 1:1 to system threads. They are
instead identified by the memory address of the spdk_thread object.

Change-Id: I417f8376cebd2ee94f624f4436e6394b51486063
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437999
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-15 16:53:12 +00:00
Konrad Sztyber
3816dbf89b test/unit: remove unnecessary mock definitions
Added -ffunction-sections / --gc-sections to allow the linker to remove
unused functions when linking test executables. This reduces the number
of required mock definitions to those actually required by tested code.

Change-Id: I22725ce82e22675a7c2d28c8227e586f2cd65023
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/436319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-13 00:48:20 +00:00
Wojciech Malikowski
ef2e614cef ftl: Initial implementation
This patch adds core FTL (flash translation layer) submodules.

On regular SSDs the FTL is part of firmware, whereas Open Channel
enables moving it to the host and allows for having full controll
of data placement on the device.

Main functionalities added:
 * logical to physical address map
 * read / write buffer cache
 * wear-leveling
 * bad block management

Change-Id: I5c28aa277b212734bd4b1f71ae386b3d6f8c3715
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-11 09:15:39 +00:00
Konrad Sztyber
ea002f5068 build: use linker specified by the LD env variable
Modifed scripts/detect_cc.sh to take additional parameter specifing
the linker to use. Default to LLD on FreeBSD systems.

Change-Id: Idf97e9676a144028c0803d272ae6f0e903b0dd1f
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-07 19:49:11 +00:00
Jim Harris
5b16c18d17 build: move app rules to spdk.app.mk
This removes a ton of duplicate from a lot of
Makefiles.  Note that spdk.unittest.mk still keeps
its own app rules, and now just includes
spdk.app_vars.mk instead.  Fixing Makefiles that
include this .mk file requires more work that is
deferred for now.  We do need to add the
"install: all" rule explicitly to this .mk files.

Note that Makefiles need to include spdk.app.mk
now at the end of the file, so that things like
APP and SPDK_LIB_LIST are defined before including
it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If6823cda135180c37690ffc96c91660b7f0718e3

Reviewed-on: https://review.gerrithub.io/434283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-12 22:35:21 +00:00
Jim Harris
40f3800480 build: remove ADDITIONAL_LIBS
This can just be replaced with ENV_LIBS - similar
to other application Makefiles.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icad14e6f74264389972c7798aac493e584f7c8c3

Reviewed-on: https://review.gerrithub.io/434280
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 18:07:53 +00:00
Jim Harris
cceaeedb1f build: add mk/spdk.app_cxx.mk
This is just a copy of spdk.app.mk for now.  But later
patches will modify spdk.app.mk in a "C-specific" way,
so add this cxx variant to use now in the C++ apps.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d4d28c6119364d4c3cc7fa5274598b69f4ee6b1

Reviewed-on: https://review.gerrithub.io/434279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 18:07:53 +00:00
Jim Harris
b6c6ef2958 build: add mk/spdk.app_vars.mk
This is a step towards moving a lot of the boilerplate
rules in various application Makefiles into mk/spdk.app.mk.
We will need to add a mk/spdk.app_cxx.mk for our C++
applications, but that will need to share these same
variables.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaee73a4fa47ff2ee54f1a1ea3c0d2097900df218

Reviewed-on: https://review.gerrithub.io/434278
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 18:07:53 +00:00
Pawel Wodkowski
6f2e275aab configure: save used environment variables in separate file
currently they are saved in mk/config.mk but this break config.h

Change-Id: Iecb037f1aea91469f4093724cdf100dc122d21a5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427222
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-03 20:25:44 +00:00
Lance Hartmann
f0bd658fd5 build: Install targets for nvme perf and identify
Introduces new macro INSTALL_EXAMPLE and two example apps,
examples/nvme/perf and example/nvme/identify that make
use of it to install them while at the same time
renaming them in the target directory based on the
source directory path relative to examples.

Change-Id: I2d850458bb2589f80e0af6fb7a9d00aa3bbc6907
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/429963
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-29 19:30:08 +00:00
Jim Harris
5704be8987 build: remove xx_MODULES_LINKER_ARGS and xx_MODULES_FILES
Application Makefiles can now just add xx_MODULES_LIB_LIST
to SPDK_LIB_LIST.  This is possible now since all
SPDK libraries are linked with --whole-archive, so there
is no need to differentiate between "modules" libraries
and other SPDK libraries.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iabf81a388b72d3b2a2f48287a8491ddc977722ac

Reviewed-on: https://review.gerrithub.io/434277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 16:43:01 +00:00
Jim Harris
42b6771f40 build: add ALL_MODULES_* variables to spdk.modules.mk
This allows a lot of simplification to SPDK application
makefiles.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5fa463f6369834b84a8d92e79fa7768082209d7a

Reviewed-on: https://review.gerrithub.io/434274
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
9a47106188 build: use SYS_LIBS for modules system lib dependencies
This is a step towards simplifying the Makefiles for
SPDK applications.  After this change, all of the
xxx_MODULES_LINKER_ARGS variables are just a list
of SPDK libraries wrapped in --whole-archive.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0cf069c2f74e8998d42dea9598db55f333c861b7

Reviewed-on: https://review.gerrithub.io/434146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
3da5847023 build: add ENV_LINKER_ARGS to LINK_C and LINK_CXX
This allows us to remove $(ENV_LINKER_ARGS) from all of the
application Makefiles.

Modify spdk.unittest.mk to explicitly set ENV_LINKER_ARGS
to nothing, to avoid DPDK libraries from getting linked
into unit tests.  This seems better than creating separate
LINK_C rules for unit tests.  Provide a UNIT_TEST_LINK_ENV
flag to allow select unit tests to override this behavior -
it's needed for some of the env unit tests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I647f2c848cf434023cf5a1f3a45869bbc4fc8102
Reviewed-on: https://review.gerrithub.io/434145
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
2ac1cbbf07 build: link all static libs with --whole-archive
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc.  Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.

SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary.  But there
are still a lot of exceptions that we need to work around.  The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.

Note that DPDK also links all libraries with --whole-archive by
default.

This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53f0
Reviewed-on: https://review.gerrithub.io/434107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Jim Harris
0b5fa082db build: remove sock from SOCK_MODULES_LIST
The various *_MODULES_LIST are intended for the modules
that implement the specific interface - not for the
associated core library itself.

For example, BLOCKDEV_MODULES_LIST doesn't contain bdev,
it contains all of the bdev modules.  Similar for
COPY_MODULES_LIST and copy.

So make SOCK_MODULES_LIST consistent.  This is a real
pain because we have to add sock explicitly to a bunch
of individual Makefiles, but I have plans to clean that
up in some future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d293372cfb38bfa46c8478a8d9ca9ce23924747

Reviewed-on: https://review.gerrithub.io/434125
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Jim Harris
880a1426f2 Revert "build: link all static libs with --whole-archive"
There was an implicit merge conflict between this patch
and e956be96e which added NVMe/TCP support including adding
"sock" as an nvme.libtest.mk depdendency.

This reverts commit 7962ce312d.

Change-Id: Ie7e58e89128f28080a986e12cd014de7df4752e5
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/434103
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-20 20:33:27 +00:00
Jim Harris
f425b98513 ut_mock: rename library from spdk_mock to ut_mock
This matches the lib/ut_mock directory name, and also
avoids the final library being named spdk_spdk_mock.a.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I80ec09568d0455d1f131e45694b6e2d536e6ca91

Reviewed-on: https://review.gerrithub.io/433909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-20 14:57:57 +00:00
Jim Harris
7962ce312d build: link all static libs with --whole-archive
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc.  Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.

SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary.  But there
are still a lot of exceptions that we need to work around.  The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.

Note that DPDK also links all libraries with --whole-archive by
default.

This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53fe

Reviewed-on: https://review.gerrithub.io/432917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 14:57:57 +00:00
Ziye Yang
e956be96eb nvme: Add the NVMe over fabrics TCP/IP transport support
It is the first patch to follow the NVMe over fabrics
spec and implmenent the NVMe/TCP transport. It can be
divided into work in the host and target sides:

Host side: Add the TCP/IP transport in nvme lib (lib/nvme).
Target side: Add the TCP/IP transport in nvmf lib (lib/nvmf).

Change-Id: Idc4f93750df676354f6c2ea8ecdb234e3638fd44
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-19 20:36:05 +00:00
Jim Harris
bf7f87cecd build: remove blobfs, blob and blob_bdev from app SPDK_LIB_LISTs
blob and blob_bdev are already included via the lvol bdev module
dependencies in mk/spdk.modules.mk.  This patch adds blobfs there
too.

These changes are needed for some upcoming patches which will
apply --whole-archive to all libraries linked into the application.
That patch will require that we never include the same library
twice.

Shared library builds *require* that blobfs be listed before
blob on the link line, hence adding blobfs here even though it's
not technically associated with a bdev module.

We can also simplify spdk.modules.mk while we're here - just
filter out bdev_lvol to disable lvol for applications like
blob_cli that need to disable it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I01e5e6b91d947463749f942f54c12f48229bbe5a

Reviewed-on: https://review.gerrithub.io/432916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-16 08:57:16 +00:00
Pawel Wodkowski
535466d7cd mk: set executable bit only for real libraries
Change-Id: I9f68f83472844cabe6a36f70a00f2d2cd319fc62
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/431169
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-02 17:57:49 +00:00
Jim Harris
fd1e204647 bdev: remove "v" prefix from vbdev library names
Let's just make them consistent and all with a
bdev_* format.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia3dec7f4be2ecc765b4aa095e6b675f13d326063

Reviewed-on: https://review.gerrithub.io/429289
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
2018-11-01 21:25:55 +00:00
Jim Harris
d0a7baa63c mk: add framework for defining inter-lib dependencies
Currently only the iscsi => scsi dependency is defined to
demonstrate how the framework behaves.

Eventually, lib/Makefile will be populated with more lines
of the form:

DEPDIRS-iscsi : scsi log conf event

This line will ensure that the iscsi subdirectory is built
after the specified dependency directories.

Then the lib/iscsi/Makefile also defines:

DEP_LIBNAMES = scsi log conf event

This line adds dependencies from the iscsi.so to the specified
libraries.

Ideally we could avoid this duplication by passing the DEPDIRS
value to the lower level Makefile, but that breaks down when we
start considering event/subsystems/* libraries.  So for now we'll
need to duplicate it.  Incidentally, DPDK also duplicates the
dependency information like this.

While here, remove use of $(MAKESUBDIRFLAGS) in spdk.subdirs.mk -
it was no longer used anywhere in the tree.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If63cbc10ffa18ac6361e4210bd6196ffe88bec43

Reviewed-on: https://review.gerrithub.io/429287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-01 21:25:55 +00:00
Pawel Wodkowski
85bc2bbe7d configure: use mk/config.mk instead of CONFIG.local
We have hairy configuration method. First configure script uses CONFIG,
environment and arguments to produce CONFIG.local and config.h then
makefile uses CONFIG, environment and CONFIG.local to produce config.h
again. First time config.h can't be generated reliably as config.h
target in Makefile is passing MAKEFLAGS and they are unknown at
configuration phase.

Lets remove this mess by producing final mk/config.mk file and get rid
of CONFIG.local. This will also generate full build configuration in one
file instead scattering them in CONFIG and CONFIG.local.
In next patch scripts/genconfig.py can be removed.

Change-Id: I5696813312edff7e493207f7aa8569234a365ec0
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/426364
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-10-16 12:40:43 +00:00
Jim Harris
2c5b956ee5 mk: add spdk_lib_list_to_shared_libs
While here, change spdk_lib_list_to_files to
spdk_lib_list_to_static_libs to differentiate it from
the new spdk_lib_list_to_shared_libs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6e5913addfbdd556fae2451d4e2b2c43feaf33ab

Reviewed-on: https://review.gerrithub.io/429286
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-15 16:44:36 +00:00
Seth Howell
89ef21520d Make: Add deps for each shared obj individually
This prevents the SPDK build from requirng unnecessary dependencies when
not compiling certain features. Also, fixes github issue #434

Change-Id: I7d0520474f3656ae32670313f2290e6b741c5ca8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426131
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-09 21:45:26 +00:00
Seth Howell
c779298802 Make: Build shared libs while building static libs
This will allow us to specify more granular system dependencies for each
.so file.

Change-Id: I759876e09ffb36380a503962b4be8fc9d7c71b63
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-09 21:45:26 +00:00
Pawel Wodkowski
461cfcecc6 CONFIG: add missing CONFIG_ options
Also fix options that doesn't start with CONFIG_ prefix.

Change-Id: I32a24373328bab8f513d56c0c09b5a86589f690c
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427767
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-04 21:31:42 +00:00
Pawel Wodkowski
c4fee1e970 mk: don't use '-include spdk/config.h'
Each file that need to check SPDK_CONFIG_* options need to include
spdk/config.h explicitly.

Change-Id: If9f2a91ac4c2b1a300dcf88ec3e2a12714ad344a
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/427221
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-02 23:13:32 +00:00
Pawel Wodkowski
c5467556f0 mk: move config.h to include/spdk directory
We need to include it in devel package.


Change-Id: I823200632e8bcb9fdb86c8cb5fbf3a651a710b78
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/426839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-02 23:13:32 +00:00
Pawel Wodkowski
ae2b2f64ea log: add a way to turn on and off stack unwinding in logs
As a side effect, if SPDK_LOG_DISABLED is used in spdk_log() as log
level then no message will be printed.

Change-Id: I2d57b60a5a310a9ef2a1187a81088d0acf828742
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425105
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-02 22:09:27 +00:00
Jim Harris
1d64773c2a raid: enable by default
We kept this disabled by default originally because it lacked
multiple iov support needed for vhost and nvmf in-capsule data.
Now that support has been added, there is no reason to not
enable it automatically - it has no extra package dependencies,
so requiring users to specify --with-raid to configure is just
an extra unneeded step.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia9796370e12a6f93993e70083de36114a37851bd

Reviewed-on: https://review.gerrithub.io/426003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-19 17:22:31 +00:00
Vitaliy Mysak
4a7240a7f5 make: Fix test app Makefile dependencies
Add dependency $(ENV_LIBS) for test/env/*
 as part of new variable $(ADDITIONAL_LIBS) which is direct dependency
 of $(APP) in mk/spdk.unittest.mk

This solves the problem of
 make not triggering LINKing for test/env/*
 after changes made to lib/env_dpdk/*.c

Issue was that Makefiles in test/env/*/ classified tests as unittests
 by including mk/spdk.unittest.mk
But unittests are not using env_dpdk which means that
 unittests don't relink on env_dpdk update

Change-Id: I91b6690dc92d68bc69bbf11ac5336d3a7f56e4e4
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/425665
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-19 15:53:29 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
Seth Howell
407915eb6d make: move FreeBSD linker change to SYS_LIBS
This linker flag is really more important when dealing with external SO
files such as libuuid. It shoudl follow those linked options around
instead of all of our internal dependencies.

This series is aimed at fixing github issue 434

Change-Id: Iee65470632b5ab0adeb361005edc894e05c43ea4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:56:46 +00:00
Seth Howell
6c4f29bb01 make: simplify rdma dependency linking
Instead of adding the linker flags just before application compilation
on linux, move that linking to earlier in the process. This will also be
useful for the shared library build.
This series is aimed at fixing github issue 434
Change-Id: I84bcc6bd922ddb6e48adb2dfe0b154700773cf27
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425435
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:56:46 +00:00
Seth Howell
c58d3c445b mk: Add all external libs to sys_libs
This allows us to better distinguish between external and internal libs.
This series is aimed at fixing github issue 434
Change-Id: I2ed141f909e7c4a800df02061007b0d23da25380
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425434
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:56:46 +00:00
Jim Harris
5e181ba84f build: explicitly link unit tests with static libraries
Unit tests define mock functions in the unit test code
itself.  This links fine with static libraries, but
doesn't work if trying to link with shared libraries.

So change the unit tests to always explicitly link
with the static libraries.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3a8a60162fcab6c73a216c4bb364175767fdc62c

Reviewed-on: https://review.gerrithub.io/422441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-16 14:27:18 +00:00
Jim Harris
ffc1d37c53 spdk.app.mk: fix SPDK_FILTER_LIB_LIST
When we build the SPDK_FILTER_LIB_LIST, we should
maintain the dependencies between libraries.  Primary
culprit is vhost depending on scsi.  So add vhost
to SPDK_FILTER_LIB_LIST before scsi, and then stop
sorting the list unnecessarily before using it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib22a5016aa492c30a389a498ae0360fbcec7c12b

Reviewed-on: https://review.gerrithub.io/422439
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-16 14:27:18 +00:00
Lance Hartmann
3f6f569854 shared_lib: Update shared lib build and install
Updates build and install of shared lib to conform with
standard naming conventions for shared libraries, including
the setting of the lib's soname.

Change-Id: Ib46d298ab5f77fd8c32b7c04022b546446189428
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/421936
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-16 00:27:53 +00:00
Pawel Wodkowski
848daf274b log: add backtrace option
Add '--enable-log-bt=lvl' option to show simple backtrace in following
form:

thread.c: 346:spdk_io_device_register: *ERROR*: io_device 0xf2ef80
already registered
*ERROR*: === BACKTRACE START ===
*ERROR*:   1: spdk_io_device_register() at 0x6d64df
*ERROR*:   2:  spdk_copy_engine_initialize() at 0x71059c
*ERROR*:   3:   spdk_copy_engine_subsystem_initialize() at 0x572ed0
*ERROR*:   4:    spdk_subsystem_init_next() at 0x6ca756
*ERROR*:   5:     spdk_subsystem_verify() at 0x6caba7
*ERROR*:   6:      _spdk_event_queue_run_batch() at 0x6c1ffa
*ERROR*:   7:       _spdk_reactor_run() at 0x6c5349
*ERROR*:   8:        spdk_reactors_start() at 0x6c784f
*ERROR*:   9:         spdk_app_start() at 0x6bf18e
*ERROR*: === BACKTRACE END ===

This adds additional libunwind dependency so don't enable by default.

Change-Id: Ice93d7571a000d8a57d2fedda7670c9a0b6ff7b7
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-25 15:48:25 +00:00
Ben Walker
b207af9a4b build: Only link virtio bdev module if CONFIG_VIRTIO is 'y'
Change-Id: I842ab24d350ec63cce57f3f1aaabc564197cc91f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/419915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-20 20:22:01 +00:00
Kunal Sablok
41586b0f1d bdev: add raid bdev module
Raid module:
============
- SPDK raid bdev module is a new bdev module which is
  responsible for striping various NVMe devices and expose the raid bdev
  to bdev layer which would enhance the performance and capacity.
- It can support theoretically 256 base devices (currently it is being
  tested max upto 8 base devices)
- Multiple strip sizes like 32KB, 64KB, 128KB, 256KB, 512KB etc is
  supported. Most of the current testing is focused on 64KB strip size.
- New RPC commands like "create raid bdev", "destroy raid bdev" and "get raid bdevs"
  are introduced to configure raid bdev dynamically in a running
SPDK system.
- Currently raid bdev configuration parameters are persisted in the
  current SPDK configuration file for across reboot support. DDF will be
introduced later.

High level testing done:
=======================
- Raid bdev is created with 8 base NVMe devices via configuration
  file and is exposed to initiator via existing methods. Initiator is
able to see a single NVMe namespace with capacity equal to sum of the
minimum capacities of 8 devices. Initiator was able to run raw
read/write workload, file system workload etc (tested with XFS file
system workload).
- Multiple raid bdevs are also created and exposed to initiator and
  tested with file system and other workloads for read/write IO.
- LVS / LVOL are created over raid bdev and exposed to initiator.
  Testing was done for raw read/write workloads and XFS file system
workloads.
- RPC testing is done where on the running SPDK system raid bdevs
  are created out of NVMe base devices. These raid bdevs (and LVOLs
over raid bdevs) are then exposed to initiator and IO workload was
tested for raw read/write and XFS file system workload.
- RPC testing is done for delete raid bdevs where all raid bdevs
  are deleted in running SPDK system.
- RPC testing is done for get raid bdevs where existing list of
  raid bdev names is printed (it can be all raid bdevs or only
online or only configuring or only offline).
- RPC testing is done where raid bdevs and underlying NVMe devices
  relationship was returned in JSON RPC commands

Change-Id: I10ae1266f8f2cca3c106e4df8c1c0993ddf435d8
Signed-off-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-on: https://review.gerrithub.io/410484
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-16 20:50:40 +00:00
Daniel Verkamp
9539edca02 build: add sock dependency for all NVMe lib users
This will be necessary when the NVMe/TCP transport is added, which will
use the sock abstraction library.

Change-Id: I6561ccdad1ff575db2afa418eab2f2cd24806cb9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416477
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-22 17:09:57 +00:00
Daniel Verkamp
64823f28a8 modules: rename NET_MODULES_* -> SOCK_MODULES_*
Also remove libspdk_net from the list and add it to each app that
originally linked it via NET_MODULES.

This will allow the use of SOCK_MODULES in application Makefiles without
pulling in the RPC libraries required by spdk_net.

Change-Id: Iad68ddd25bea24bba53a681c78d41e6e703f866d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416476
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-22 17:09:57 +00:00
Daniel Verkamp
ea3c58ea77 net: split sock abstraction into lib/sock
This separates the network interface management code (interface.c and
net_rpc.c) from the socket abstraction layer, which allows users that
only want the socket abstraction to avoid pulling in the JSON RPC
libraries.

Change-Id: I9b00285a70bac0c74c73353cfa900d4f3b2e465f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416475
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-22 17:09:57 +00:00
Daniel Verkamp
fdf5c1d7d2 build: add -lcrypto (OpenSSL) to SYS_LIBS
This is already required to build the shared library and iSCSI
components; simplify these Makefiles by adding -lcrypto to the central
SYS_LIBS in mk/spdk.common.mk.

Change-Id: I8fdaffbccc1294e24e32559387b0ed99d1deb0ce
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/415351
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-20 20:16:31 +00:00
Daniel Verkamp
6f46e272c3 build: add combined shared library (libspdk.so)
Change-Id: If48df087c145a3fd278c0ae04558d8352b01cc1a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/414687
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-14 03:36:03 +00:00
Ben Walker
02d75f6237 thread: Move threading abstraction code out of util
This makes more sense as a first class library.

Change-Id: Ibd5c578f8708bd8c7d83fe1629e97c9a3316b56b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414698
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
2018-06-12 15:24:07 +00:00
Sai Tallamraju
27a2d1d04d nvmf:Add RNIC HBA userspace library for FreeBSD
Change-Id: I7d929e7520007f29511299442f791038f7a4ce1d
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/409539
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-07 16:53:43 +00:00
Tomasz Zawadzki
5d94d6d3a5 vpp: add support for vpp comm library
This patch integrates VPP userspace TCP/IP stack,
as optional network framework to be used in SPDK.

Support is done via VPP Communications Library.

Change-Id: I4c2945c76878dbf24ff2f8612e0a21615f7d87e6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/389566
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-17 20:33:43 -04:00
Tomasz Zawadzki
6e9293eaca blobcli: exclude vbdev_lvol from blobcli compilation
At this time blobcli makes use of all the bdev modules
during compilation. Yet vbdev_lvol does examine,
that already creates bs_dev on a bdev and loads
blobstore for all devices where lvol store exists.
Since bs_dev cannot be created on device twice,
same as blobstore cannot be loaded twice.
It prevented blobcli from displaying information
for blobstore underlying lvol store.

This patch remove vbdev_lvol from list of bdev modules
for blobcli.

Change-Id: I144ba22ec28f7f315e768745f5dd0cbbe3bbad86
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/404832
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-12 12:40:01 -04:00
Jim Harris
218dcd840a bdev/pmem: change all NVML strings to PMDK
NVML (Non Volatile Memory Library) changed its name to
PMDK (Persistent Memory Development Kit), so make the
necessary changes to the SPDK repository.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id07a87eafb4e9a3099603030c58c0e927bafa608

Reviewed-on: https://review.gerrithub.io/406256
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-07 00:07:57 -04:00
Jim Harris
2af15344ab bdev: add iSCSI initiator bdev module
This uses libiscsi to implement an iSCSI initiator bdev
module for SPDK.  Still a lots of work to do on this - posting
it in case anyone is interested in working on this further.

A number of todo items are listed in a README in the lib/bdev/iscsi
directory.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Change-Id: I060e33de0cd6796246789bf0e1bb4f2df59d8f71
Reviewed-on: https://review.gerrithub.io/390313
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-21 20:35:42 -04:00
paul luse
2191985125 bdev: Vbdev Passthru Example
Example showing the basics of a vbdev via a module that implements
the main functions, and a few extras, needed for a vbdev. The tests
also demonstrate how to configure an application to use a vbdev and
an underlying bdev

Change-Id: I67bf542143eb6532465cdb6af63c0f2ab7084c0e
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/399768
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-20 19:10:08 -04:00
Tomasz Zawadzki
e793fad5e9 vpp: dynamic compilation with VPP
Enable dynamic compilation of SPDK with VPP.

Change-Id: I9f5f7223625cbebec0d6209b5a2ce6bdcc1a04bf
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/401825
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-15 17:54:29 -04:00
Tomasz Zawadzki
609fb9a16b net: create NET_MODULE for compiling different net_implementations
Change-Id: Id370631770cb2e090c3efbe95cd86db4f6168c5e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/403702
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-15 17:54:29 -04:00
Daniel Verkamp
245e2e9855 app/spdk_tgt: add 'make install' rule
Change-Id: Ib76031972824231ed824e53a327c7d648842ad6a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/403224
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-09 09:56:44 -05:00
Daniel Verkamp
0e0a489013 build: link mock lib with --whole-archive
The unit test mock library contains wrappers that aren't directly
referenced but still need to be kept to allow the final link to work
when they are used via --wrap.  Add the spdk_mock library to the
whole-archive list to prevent it from getting removed.

Fixes build with LTO enabled.

Change-Id: I2c70be7cb517a373178d34026f9980396727f8df
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/401074
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2018-02-23 10:15:23 -05:00
Seth Howell
d712c310e8 Make: remove -e from subdirectory command
in gmake, -e gives precedence to variables defined in the environment.
this argument disables appending to CFLAGS or CXXFLAGS when they are set
in the environment and breaks our make configuration.

Change-Id: Id395e7edc4b243bd3805d7f5fdb3456d94893952
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/400863
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-02-21 13:10:29 -05:00
Jim Harris
bb24ce82a6 build: add @: to remaining 'all' targets
It is not ideal to have to add this in a bunch of
different Makefiles, but further consolidation of
Makefiles is going to be a more arduous process.

With this change, rebuilding SPDK after no changes
will result in no output - all of the "Nothing to
be done for 'all'" messages have been removed.  Note
that DPDK build output still remains - this can be
suppressed by either using an out-of-tree DPDK, or
using SKIP_DPDK_BUILD=1 when using the in-tree DPDK
submodule.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib1d45432f2baaa1d9c1b9509906ee1e77966d3ca

Reviewed-on: https://review.gerrithub.io/399918
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-14 17:38:25 -05:00
Jim Harris
36b00b7f61 build: add dummy @: to some "all" targets
This suppresses:

make[3]: Nothing to be done for 'all'.

when nothing needs to be rebuilt.

Still more work will be needed, specifically in
examples and apps directories.  But this will cover
most of lib/ and anything in unit/test that is using
the new spdk.unittest.mk.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1c0c2b88e70ec37629740aa47f612c9d46fc88fb

Reviewed-on: https://review.gerrithub.io/399725
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-14 10:40:40 -05:00
Jim Harris
29eb956793 test: use spdk.unittest.mk for nvmf unit tests
This includes removing the nvmf.unittest.mk that these
unit tests were using previously.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b53db0a565aee910587826b37e8b43d6a324e75

Reviewed-on: https://review.gerrithub.io/399724
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-14 10:40:40 -05:00
Jim Harris
f8ad4dca9d test: use spdk.unittest.mk for json unit tests
This includes removing the json.unittest.mk that these
unit tests were using previously.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icfa2e5a5554b686002407d1ef5328a0aecb442f1

Reviewed-on: https://review.gerrithub.io/399723
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-14 10:40:40 -05:00
Jim Harris
3465da9302 test: use spdk.unittest.mk for ioat unit tests
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I53a04187638a8ea4569b18bb4959b0bcf7f3d860

Reviewed-on: https://review.gerrithub.io/399717
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-14 10:40:40 -05:00
Jim Harris
435990003e build: make nvme.unittest.mk the common unittest .mk include
This is the first of a long line of patches which will
reduce Makefile rule duplication in all of our SPDK
unit tests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1cd6e9e1e9cabab0a15dbcb901e4db782d73b1a4

Reviewed-on: https://review.gerrithub.io/399716
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-02-14 10:40:40 -05:00
Jim Harris
0db2c2ef47 test: remove explicit build rule from nvme/nvmf mk file
These aren't needed - the common %.o rule is sufficient.

As part of this patch, remove OTHER_FILES references from
nvme and nvmf unit tests, and just include these .c files
directly.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I76ab20da2f560c7997e978fcbfe28c2a4907f759

Reviewed-on: https://review.gerrithub.io/399715
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-02-14 10:40:40 -05:00
Jim Harris
5b9b47fae5 mk: remove -fno-omit-frame-pointer for release builds
This can make profiling more challenging, but should not
affect debugging since we still have unwind tables.

Keep the C_OPT in COMMON_CFLAGS to allow a way to
still add compile flags, even though C_OPT is now
not set by default.

This results in a 11% performance improvement on
a bdevperf microbenchmark using null bdevs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If16fef6e94a73dfca6c10d3b7f158a12dfc70b53

Reviewed-on: https://review.gerrithub.io/393823
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-09 12:46:35 -05:00
Daniel Verkamp
543cb17248 build: remove $(ENV_CFLAGS) where not necessary
Only Makefiles for libraries that directly depend on DPDK (rather than
the SPDK env abstraction) should add $(ENV_CFLAGS).

Change-Id: Ifdf44d3ef8c42bbf7f20edd524b330d00658235b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/392818
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-03 12:05:59 -05:00
Dariusz Stojaczyk
9dde5e5c27 virtio: add new library virtio
Exported lib/bdev/rte_virtio as a separate
library not dependent on bdev.

Virtio is now accessible via spdk_internal/virtio.h.
The header is marked `internal`, as it's
not meant to be used by end users. It's
not handy to handle all backend-specific
(e.g. Virtio-SCSI) logic in a user code.

For now the Virtio interface is publicly
exposed only via bdev_virtio module. We
might want to consider adding a separate,
public Virtio-SCSI library in the future.

Note: this patch doesn't do any changes
to the virtio code. Everything is
moved 1:1.

Change-Id: I805e5d12d265d82b0bc5784c89fbadb81abdb278
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/388166
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-12-26 13:03:29 -05:00
Xiaodong Liu
b93d767fe8 nbd: add nbd stop rpc
Use soft-disconnection to stop nbd disk
Examople RPC request:
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "stop_nbd_disk",
  "params": {
    "nbd_device": "/dev/nbd0"
  }
}

Change-Id: I3d9161635b4422bbb9895914a03483dbaa6b6386
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/389781
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-12-07 16:23:19 -05:00
Daniel Verkamp
2e7fe8e888 build: add 'make install' rule
For now, this only installs libraries and headers; we will need to
consider which binaries should be installed and what they should be
named before we add them to the install rule.

Change-Id: I78dc8631f793d0df88cd884b0ac66406df9e4427
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/387637
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-11-27 17:58:02 -05:00
Daniel Verkamp
b17d4d9135 build: only touch output if build is successful
Running the 'touch' command unconditionally causes the compile step to
return 0 even if the compilation failed.  Instead, only run it if the
compiler succeeded.

Fixes: 4f16344356 ("make: Update build artifact timestamps for clang
support")

Change-Id: I46907851fde8bb9e3f73549276e489b6c30ed1fd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/388324
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <ed.rodriguez@netapp.com>
2017-11-21 18:39:44 -05:00
Daniel Verkamp
322cae6af4 build: remove tab from spdk_lib_list_to_files
The tab inside the define ends up in the output when called, which leads
to problems if the output is quoted (the tab will be treated as part of
the filename).

Change-Id: I1038e1650a74861660fa4164dbde065217bc7b99
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/387631
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-11-15 17:57:07 -05:00
Ed Rodriguez
4f16344356 make: Update build artifact timestamps for clang support
Change-Id: Ibc165386b074d82969610467f4da73c73d8f4ea0
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Reviewed-on: https://review.gerrithub.io/386164
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-11-10 12:50:09 -05:00
Daniel Verkamp
b1864503cd build: only use -march=native on x86_64
Check the target machine type explicitly before turning on -march=native
rather than assuming any non-PowerPC machine is x86.

Change-Id: I651b1852dc5ac6a3d42c60d639ec48f399f6b3bd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/386348
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-11-09 12:33:14 -05:00
Daniel Verkamp
826c9ef64a build: detect architecture via cc -dumpmachine
Similar to commit 98f84e6255 ("build: detect OS via cc -dumpmachine"),
we can use the C compiler's -dumpmachine output to detect the target
CPU architecture rather than relying on the build machine's uname
output.  This helps enable correct CPU detection when $(CC) is a cross
compiler.

Change-Id: I72c34294a1ff7dd9df3aa45dfc319e5b81d51f85
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/385709
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-11-08 18:32:57 -05:00
Daniel Verkamp
98f84e6255 build: detect OS via cc -dumpmachine
Rather than running 'uname', which returns information about the build
system, use the C compiler's -dumpmachine option, which outputs the
target triplet, which looks something like "x86_64-unknown-linux-gnu".

This enables us to detect the correct target OS when
the configured $(CC) is a cross compiler.

Change-Id: I5659dd5b091094cc8e408a6c6f4e7f64a65e9070
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/385166
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-11-03 18:53:15 -04:00
PepperJo
cccd5ad557 build: fix compiler options for ppc64le
ppc64le does not support compiler option march=native
but instead uses mcpu

Change-Id: I2dd14b1acf003e8229e9c9392c4c0606e2c15af8
Signed-off-by: Jonas Pfefferle <jpf@zurich.ibm.com>
Reviewed-on: https://review.gerrithub.io/383725
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-11-03 18:53:04 -04:00
Piotr Pelplinski
fca11f1519 bdev_pmem: initial implementation of persistent memory bdev
Use SPDK bdev subsystem to allow basic IO to pmem pool
using pmemblk_* calls.

New pmem bdev can be constructed using RPC call
construct_pmem_bdev PMEM_FILE

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If5ca94161fe53644b8fccd3b77de5479da1b2e55
Reviewed-on: https://review.gerrithub.io/376973
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-09-29 14:04:32 -04:00
Jim Harris
19de08066d test/mock: add pthread_self
This will be used to simulate multi-threading for
bdev unit tests.

While here, alphabetize the existing calls - calloc
had been inserted out of order in a couple of places.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I559cd1fd79e78f03ebdac313e0bbedbcdde4a8c1

Reviewed-on: https://review.gerrithub.io/377968
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2017-09-19 17:15:15 -04:00
Daniel Verkamp
2453352025 build: remove explicit -m64 compiler option
This should be the default already on x86-64 compilers.

Note that the code may still not necessarily compile or work properly on
other architectures, but this is at least a start toward portability.

Change-Id: I9c842aa329dd761277bf90669f2c905e436e0af4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/378365
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-09-18 12:57:29 -04:00
Tomasz Zawadzki
27f44662ac lvol: Logical volume implementation
Change-Id: Ia96ae78ff9530d953181ac5f7255a38f3c8ec430
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Reviewed-on: https://review.gerrithub.io/375392
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-09-14 19:30:54 -04:00
Jim Harris
179ed697b3 blob: add readv/writev support
Most of the work here revolves around having to split
an I/O that spans a cluster boundary.  In this case
we need to allocate a separate iov array, and then
issue each sub-I/O serially, copying the relevant
subset of the original iov array.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d46b3f832245900d109ee6c78cc6d49cf96428b

Reviewed-on: https://review.gerrithub.io/374880
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-22 18:19:00 -04:00
Jim Harris
c2175d2c51 virtio: add initial rev of virtio-scsi initiator bdev module
Supports both PCI mode (for usage in guest VMs) and
vhost-user mode (for usage in host processes).  The rte_virtio
subdirectory contains a lot of code lifted from the DPDK
virtio-net driver.  Most of the PCI and vhost-user code is
reused almost exactly as-is, but the virtio code is drastically
rewritten as the DPDK code was very network specific.

Has been lightly tested with both the bdevio and bdevperf
applications in both PCI and vhost-user modes.

Still quite a bit of work needed - a list of todo
items is included in a README in the module's directory.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I85989d3de9ea89a87b719ececdb6d2ac16b77f53
Reviewed-on: https://review.gerrithub.io/374519
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-22 11:54:15 -04:00
Ziye Yang
26d6770f1c GPT: add GPT bdev support
Change-Id: I6a138e1c1d775e8c8bfeede9600f8b0f799ecdad
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/362445
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-07-06 13:44:37 -04:00
Ben Walker
4c0eaac47a bdev: Create a separate bdev subsystem library
Much like bdev modules inside the bdev directory,
add a subsystems directory inside of event. The subsystem
specific code for the bdev library is placed in to
a separate library in that directory, breaking the
strict dependency of the bdev library on the event subsystem
code.

Change-Id: I255941b823a9ec3e2d62f22a586414949d8ff5ad
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/365055
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-06-16 13:10:26 -04:00
Paul Luse
be74ecf79a test: add one unit test showing example of ut_mock wrap
This patch shows how to use the spdk_mock library which now only
consists of --wrap capability for a few functions.

Next will be a patch that provides a generic mock capability to
use either the --wrap feature or a combination of globals and
stubs to make adding more UT easier wrt mocking. After that is
in place nvme_ut.c will be updated to use the new library for
all mocking.

Change-Id: I1a6ffb722da043bb70bd4cfe1afa7c5e39a0fccb
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/365074
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-06-15 13:46:19 -04:00
Paul Luse
95a2dcb321 test: add wrap capability
This patch doesn't use it yet but sets up the ability to so that
upcoming unit test pathces can. The next patch will include use
of wrap in one nvme UT and then following that the wrap and
today's use of globally controlled stubs will be morphed into a
simple mock type capability for spdk unit tests.

Change-Id: I252160266f0b5c76e50fd213cb3e1686d48b9d0e
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/363441
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-06-08 19:24:54 -04:00
Ben Walker
830912e28a Add thread sanitizer to configure options
Change-Id: Ibeab0bc70bb59da4ebb8bd175642f5f549f1d273
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/362425
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-05-26 13:17:14 -04:00
Cunyin Chang
32b4ab71a2 bdev/error: Add new bdev/error to inject errors in bdev layer.
Change-Id: I3c68cb5911b3662dbb35f551dc291d5ebdd84bf6
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2017-05-18 07:56:14 -07:00
Jim Harris
4eafea0360 bdev: add a null bdev module
Also change the discovery/nvmf.sh test to use it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I56bce9a84bd46f13b6d4f34da81abf23413f2598
2017-05-15 14:00:37 -07:00
Daniel Verkamp
6d6d1161c7 build: add CONFIG_LTO to enable link-time optimization
Change-Id: I9546d715b8d2ca3ebf46183bdbaa58e8aa921d95
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-10 13:42:11 -07:00
Ben Walker
9dd998cdc0 Add a configure script to generate the CONFIG file
Add a configure script in the root of the repository
that looks and feels like one that would be generated
by autotools. This script simply generates a CONFIG
file for you, which used to be done by hand.

Now to build SPDK you can do the following:

./configure --with-dpdk=path
make

Change-Id: I44ebb91f0cb1468b86da4c0033ac1406595d4967
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-04-03 13:30:12 -07:00
Daniel Verkamp
d535f9b7bb nvmf: split discovery service into its own file
No functional change, just rearranging code.

Change-Id: I28328dfefd7de269d326834c484f2c2fca4e6c1f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-03-17 15:55:36 -07:00
Piotr Pelplinski
1dbf53eebf vhost: add a library and app for userspace vhost-scsi processing
This patch adds a library, application and test scripts for extending
SPDK to present virtio-scsi controllers to QEMU-based VMs and
process I/O submitted to devices attached to those controllers.
This functionality is dependent on QEMU patches to enable
vhost-scsi in userspace - those patches are currently working their
way through the QEMU mailing list, but temporary patches to enable
this functionality in QEMU will be made available shortly through the
SPDK github repository.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Krzysztof Jakimiak <krzysztof.jakimiak@intel.com>
Signed-off-by: Michal Kosciowski <michal.kosciowski@intel.com>
Signed-off-by: Karol Latecki <karolx.latecki@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

Signed-off-by: Krzysztof Jakimiak <krzysztof.jakimiak@intel.com>
Change-Id: I138e4021f0ac4b1cd9a6e4041783cdf06e6f0efb
2017-03-06 12:44:35 -07:00
Daniel Verkamp
1d3edefbcf build: remove -Wunused-result flag
This flag is on by default for compilers that support it, and it
prevents compilation on older compilers that do not recognize it.

Change-Id: I862cff6297ecd4fc40f2f8761bf398d524771921
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-02-01 09:51:49 -07:00
Daniel Verkamp
2c0bc38754 bdev: add "split" virtual blockdev example
This virtual block device takes an underlying block device and splits it
into several smaller equal-sized block devices.

Change-Id: I6f6e686c1177b2e4885f7e88809ad329caae55bd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-30 13:51:18 -07:00
Fan Lu
fb85072562 test/nvmf:add virtual.c unit test
Change-Id: I48aab13b93f3b231c799a82e0044fa3eae8d5df1
Signed-off-by: Fan Lu <lux.fan@intel.com>
2017-01-27 10:54:28 -07:00
Daniel Verkamp
53bf2d0f91 build: enable -fno-common
This flag prevents the compiler from merging multiple tenative global
variable declarations (e.g. 'int i;' without an assignment) into a
single copy later.  In the default (-fcommon) mode, this allows multiple
definitions of an uninitialized global variable to be linked into a
single program without any warnings.

Change-Id: I0339012a7164dccd311ab1f62d11111d24529fbd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-20 15:10:44 -07:00
Daniel Verkamp
72a45a5a4e test/nvme: don't link unit tests against spdk_nvme
The unit test Makefiles should already declare exactly which files to
include.

Change-Id: Icae8a2c862db164d72d8e9ed4064d777de43742b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-08 14:19:25 -07:00
Daniel Verkamp
1b68a8e705 config: add option to turn on UBSan
Change-Id: Ifb6b86db530d78bffdc28112f5fe1bdfcba6249c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-06 11:17:33 -07:00
Daniel Verkamp
6b1e22bd86 config: rename CONFIG_ADDRESS_SANITIZER to CONFIG_ASAN
This is shorter and matches the more commonly-used name.

Change-Id: I6af862de90e5cf8a7c27161f1ee9bb5a1164bec6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-06 11:17:15 -07:00
Daniel Verkamp
f19876b6c5 build: remove static libraries before running ar
If a library rule is being built, one of the files it contains was
rebuilt, so we need to regenerate the whole library anyway.

Remove the previous copy of the library to make sure nothing is
left from the previous build.

Change-Id: I835f721d5bf7cc908c343f47de78bd1f44be58b8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-02 15:00:20 -07:00
Jim Harris
c61bd850f0 build: add copyright header where missing
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7908bbd58038d0a79087a03641f671e3f18e2a82
2016-11-22 16:41:47 -07:00
Daniel Verkamp
1cb7e396a7 json: add printf-style spdk_json_write_string_fmt()
Change-Id: I9ea18072d4e54344f145a0b2d16aa6ab7f4d5e03
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-22 16:32:35 -07:00
Daniel Verkamp
b4bd76bc63 util: add vsprintf version of spdk_sprintf_alloc
-Wformat-nonliteral needs to be disabled since clang triggers it on the
call to vsnprintf() now that it is nested two calls deep.

Change-Id: I228b9d099cfc2b65181941cbb4798b7f8eae3baa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-22 11:52:00 -07:00
Jim Harris
407b550ff4 build: define SPDK_LIB_LIST to simplify building dependencies and linker args
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia969f9e5ed01f4b58cec2559133347d836118131
2016-11-22 11:45:48 -07:00
Jim Harris
2655503ff7 build: make a common mk file for nvmf unit tests
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7adcb863f991a8a9939430a7f15009b971ec937b
2016-11-22 11:45:48 -07:00
Jim Harris
5616e0880d build: create a common mk file for nvme library tests and examples
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ief0f99b7f014cf905d08e2c0cd9dc99653bd0bb8
2016-11-22 11:45:47 -07:00
Daniel Verkamp
44e60dc47d build: consolidate library outputs in build/lib
Make it easier to use SPDK libraries by putting them all in a single
directory that can be added with -L rather than scattered around the
source tree.

Change-Id: I5c0f5dd6e7058b5f92fa9bc41548190ffc064761
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-17 13:15:09 -07:00
Daniel Verkamp
c194ebd833 nvme: move I/O qpair allocation to transport
This requires a couple of related changes:
- I/O queue IDs are now allocated by using a bit array of free queue IDs
  instead of keeping an array of pre-initialized qpair structures.
- The "create I/O qpair" function has been split into two: one to create
  the queue pair at startup, and one to reinitialize an existing qpair
  structure after a reset.

Change-Id: I4ff3bf79b40130044428516f233b07c839d1b548
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-19 08:09:45 -07:00
Ben Walker
bfdc02ab48 nvme: Eliminate nvme_impl.h and use the swappable env lib.
Change-Id: Ibbc557b732d5b0858a2922a7a442c4b17a0d579a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-11 13:34:09 -07:00
Ben Walker
e0196e8124 ioat: Remove ioat_impl, use swappable env lib instead.
Change-Id: I377e41503b349b34c9ffc911840cb8fb9b13f322
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-11 13:34:08 -07:00
Ben Walker
a4747c6048 env: Make the environment library configurable.
This allows users to swap out SPDK's third party
libraries for an implementation based on their own
framework.

Change-Id: Ia0b7384ce5e31acba5ad0d7002dec9e95b759c52
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:51:37 -07:00
Ben Walker
6b1e4e732d Drop libpciaccess and switch to DPDK PCI
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.

Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.

Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-04 15:59:00 -07:00
Ziye Yang
c1341b0718 bdev: Add RBD support in lib/bdev
This patch will add a new bdev module, rbd.
It can make ceph rbd as the backend of iSCSI
target.

Change-Id: Id5eb3b159ee607052e3c33a2e59d721739fd9977
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-09-30 15:26:28 -07:00
Ben Walker
29004b6738 ioat: Use log library instead of ioat_printf
Change-Id: I10a71b5c83f60d3ef1bbf83b464813405edd1938
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-09-28 10:15:56 -07:00
Ben Walker
b0e349a804 nvme: Use log library instead of nvme_printf
Change-Id: Ic9b2db9bff3a914b3e5021695287157f1e076f9f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-09-28 10:15:55 -07:00
Daniel Verkamp
c85360da36 build: warn about old-style C function definitions
Fix the existing cases (all missing void in parameter lists) and enable
the warning to prevent new ones from being introduced.

Change-Id: Ieaf00b3dfd5daf1e21fcbefb124514882e8996c9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-09-27 16:23:07 -07:00
Daniel Verkamp
db8b791f9b build: check for warn_unused_result violations
Change-Id: I217d23b31e67752a621d6f66d601f1457fb2bc94
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-09-27 15:55:01 -07:00
Daniel Verkamp
021be6df16 bdev: add Linux AIO (libaio) backend
Change-Id: I6e7de4330f0c792514df21557a3010adef08c670
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-08-05 09:08:23 -07:00
Daniel Verkamp
861e78bf48 bdev: add block device abstraction layer
Change-Id: I235cf146a52714756c9782c03b118f518c5f5182
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-21 10:35:03 -07:00
Daniel Verkamp
07697ff1d2 build: let 'make V=1' be equivalent to 'make Q='
The V= syntax is more familiar from other build systems, so allow both.

Change-Id: I024cf4a338c2fe10bef482311a1703bf5fb2a38e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-20 09:53:58 -07:00
Daniel Verkamp
336ba0d09c build: replace USE_PCIACCESS with config.h define
Drop the special-case preprocessor definition for PCI access library now
that config.h is available with an equivalent SPDK_CONFIG_PCIACCESS
define.

Change-Id: I4891d0f2fd7d3eea51b767df9e594555b36265ea
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-20 09:53:47 -07:00
Daniel Verkamp
20c11032ae build: link DPDK libraries with --whole-archive
The upcoming DPDK version adds objects that have no externally
referenced symbols but that must be linked in (specifically, the
default mempool handlers are now assigned by a constructor function).

Link all of the DPDK libraries with -Wl,--whole-archive to fix linking
against a static library build of DPDK.

Change-Id: Ia54b100d81aa7be30a8f4bcad65cc182e91dcc65
Reported-by: Jay Sternberg <jay.e.sternberg@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 10:02:10 -07:00
Daniel Verkamp
3f80d55199 CONFIG: rename CONFIG_NVMF to CONFIG_RDMA
The NVMf target is being refactored to split the RDMA transport-specific
code into its own file.  Once this is complete, we should be able to
plug in other transports and build the NVMf target without any RDMA
dependency if desired.

To enable this, change the CONFIG option to RDMA; it still controls
whether the whole NVMf target is built for now, but once the RDMA
dependency is actually made optional, we will be able to build the
generic NVMf target code without libibverbs installed.

Change-Id: I8cd90a9aaa85dcefcc9b0f8f2e7b6af21958b2a8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 12:58:17 -07:00
Ben Walker
44d131105e log: Add a lib for integrating the log lib with event framework.
Linking this library into an application makes the log
configurable via RPC. This only works if the rpc subsystem
is also loaded.

Change-Id: If1340cf2a845ef159290232c26f341150c98fb9a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-13 10:13:48 -07:00
Daniel Verkamp
ce60297bda build: move -lrt to a new SYS_LIBS variable
On some platforms, librt may be a static library, so linker command line
order matters.  Create a new Makefile variable, SYS_LIBS, which is added
to the end of all link commands and move -lrt there.

Change-Id: Ie28ebd91a255097bf33cbcf9f0f74f7177a0b142
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 09:29:22 -07:00
Daniel Verkamp
0faa2948b5 build: replace -lpthread with -pthread
This is the standard way to add multithreading support and handles
platform-specific requirements for pthread library name, #defines, etc.

Also include -pthread in CFLAGS, since it affects preprocessor
definitions.

Change-Id: I94b436c7ef482ae3320b7937f4f64b51e5d57032
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 09:04:11 -07:00
Daniel Verkamp
39cc59635a build: include -lrt and -lpthread in LDFLAGS
Simplify the build rules so that common libraries are always linked.

Also fix up a couple of -lpciaccess instances that should have been
changed to $(PCIACCESS_LIB).

Change-Id: I4c50fa3aa59cae013d3385e38fbb830794299f6e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 09:04:02 -07:00
Daniel Verkamp
92a02f580c build: generate config.h and implicitly include it
All source files will now depend on config.h, which is generated based
on the defaults in CONFIG and the command line arguments passed to make.

This ensures that any configuration changes, including on the command
line, cause a full rebuild.

Change-Id: I6b6fa3290941200dbcf32297c66df8dc5ee18e94
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-08 10:26:50 -07:00
Daniel Verkamp
d237632434 nvmf: add unit tests
Change-Id: I22c57c3989f4b69cb35ef4e17507864d0f07f084
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-08 08:49:38 -07:00
Daniel Verkamp
eeeac6676d Add event-driven application framework
Change-Id: Iba90db6d8853dde972b4eec2c35eb44eeddae780
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-31 09:58:05 -07:00
Daniel Verkamp
376d117c90 jsonrpc: add JSON-RPC 2.0 library
Change-Id: I4f58792c3af1f85f55144717478f868ebe5b1700
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-23 10:28:58 -07:00
Daniel Verkamp
bdece622f5 trace: add tracepoint library and app
Change-Id: I472fb7e7a82e1337c6c06b1d3bb4e8a2a13d884a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-11 10:43:09 -07:00
Daniel Verkamp
cd48a01fcb build: wrap $(CURDIR) relative paths in $(abspath)
Resolve relative paths before using them to clean up command lines.

This should also help shorten the overall command line length that gets
embedded in the binary and used when locating the executable from a
coredump.

Change-Id: Ibff9849ede198bb04313496c8b7131485ffaf14f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-09 13:56:07 -07:00
Daniel Verkamp
f9193f4ce7 json: add JSON parser and encoder libraries
Change-Id: Id73fb7e300d66d31a7c3986c0334b6f56e284905
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-09 13:30:02 -07:00
Daniel Verkamp
3a94688d8e build: add spdk.lib.mk for common library rules
Change-Id: I0547554c010b0eaa6e98c843e7d9eb78c90f76d5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-09 11:24:58 -07:00
Jim Harris
d05669cef4 build: prepare spdk.common.mk for building C++ files
1) -Wstrict-prototypes applies to C code only
2) add CXXFLAGS definition

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I440702d989c4e89bd297956a58725f62d2d72831
2016-03-25 04:31:32 -07:00
Daniel Verkamp
55d9ff6a66 config: make -Werror optional and off by default
Allow end users to build the code even if, for example, warnings are
added with new compilers versions.

-Werror is still enforced for automated builds.

Change-Id: I567009d55cf5b941c452bd41d09f75d03b037209
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-18 10:42:28 -07:00
Daniel Verkamp
7198a4701a build: don't print subdir banners
Since the individual build steps include the path now, the subdirectory
printouts are less useful (especially with parallel make, where output
from multiple subdirs gets mixed together).

Change-Id: Ied71d39b360b847ca32ccd3547def751ddd9434c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-16 08:15:42 -07:00
Daniel Verkamp
d6540ae907 build: don't print top-level directory
For example, if the source code is located in a directory called "spdk",
don't print the "spdk/" prefix on all of the directories.

Change-Id: I9adc650b13e013773450c721c8434c6e5afeecc2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-08 10:59:56 -07:00
Daniel Verkamp
bafbc89d3a build: print path in CC, LINK, and LIB messages
This makes it somewhat clearer what is happening during parallel
builds (make -j).

Change-Id: Ibbe8862da7600e784cba64cd333086f1473455ac
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-08 10:43:29 -07:00
Daniel Verkamp
4cb05719bd build: add config option to use Address Sanitizer
Change-Id: I98bf4ab1c58a90341c2a8529de4537fb0250f00d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-23 16:41:31 -07:00
Ziye Yang
ff6125195c SPDK: Support configuring libpciaccess library
With CONFIG_PCIACCESS=y in CONFIG file, we can
use libpciaccess library; With CONFIG_PCIACCESS=n
in CONFIG file, we use pciaccess functions provided
in DPDK.

Change-Id: I786c5589b8e7909ba2e59d222938dd5ba45bf92d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-04 17:30:09 -07:00
Daniel Verkamp
c02b179490 Remove year from copyright headers.
Also add a space between Copyright and (c).

The copyright year can be determined using git metadata.

Also remove the duplicated "All rights reserved." - every instance of
this line already has a corresponding "All rights reserved" immediately
below it, except for examples/ioat/kperf/kmod/dma_perf.c, where I have
added it manually.

Performed using this command:

git ls-files | xargs sed -i -e 's/Copyright(c) \(.*\) Intel Corporation. All rights reserved./Copyright (c) Intel Corporation./'

Change-Id: I3779f404966800709024eb1eb66a50068af2716c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-01-28 08:54:18 -07:00
Daniel Verkamp
5f844a0477 nvme/utest: add SPDK_CU_ASSERT_FATAL wrapper
The normal CU_ASSERT_FATAL macro calls a function that is not marked as
noreturn, so static analyzers (e.g. scan-buid) can't figure out that
fatal asserts are really fatal.

Add a wrapper macro that calls abort(), which the analyzer can determine
does not return.

Change-Id: I0c087bf9c8d3c272bf88120caa70e87dab6d9546
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-01-08 09:31:10 -07:00
Daniel Verkamp
7ab7984885 build: undefine _FORTIFY_SOURCE before setting it
Some compilers automatically define _FORTIFY_SOURCE to a different
value, so undefine it before setting it on the compiler command line.

Change-Id: Id662953bd4466b99d0fe9bd5bb96f9fc335ff9a8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-12-22 10:15:06 -07:00
Daniel Verkamp
d4ab30ba33 ioat: add user-mode Intel I/OAT driver
The ioat driver supports DMA engine copy offload hardware available on
Intel Xeon platforms.

Change-Id: Ida0b17b25816576948ddb1b0443587e0f09574d4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-12-09 10:14:15 -07:00
Daniel Verkamp
01b48b7c90 build: define a common Makefile clean command
Use $(CLEAN_C) throughout the Makefiles to clean up a consistent set of
generated files.

This also adds coverage files to the list of cleaned files.

Change-Id: Iceb922935a45c9eecbf2f3443bd0ee4f5c966825
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-12-07 15:14:36 -07:00
Ben Walker
af2a731de3 build: Allow relative path for DPDK_DIR
This is very convenient. It also works from subdirectories, as
long as you adjust the relative path you pass appropriately.

Change-Id: I8cfeac380ccb5fad4389ee1feb838f986a837f11
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2015-12-07 14:08:28 -07:00
Jim Harris
38ea764f6d Conditionally add -lrte_malloc.
librte_malloc has been deprecated by DPDK, but to allow for backwards
compability, check for its existence and link with it if it exists.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3e05a3295b6768bd6804a8ec86c301e22040df37
2015-11-30 08:54:39 -07:00
Daniel Verkamp
1c3b08c63c build: move $(OBJS) definition to spdk.common.mk
Change-Id: I2dd592269ef644cfe73921151b9b174dc255323d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-11 14:59:55 -07:00
Daniel Verkamp
3d988d287a build: use $(APP) target consistently
Replace $(UT_APP) in nvme.unittest.mk with $(APP); there is no actual
difference in the way they are used.

Change-Id: I039c2488ee998ee8b1a67af129045b66a97bc29c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-11 14:59:54 -07:00
Daniel Verkamp
dba4829471 build: factor out common compile and link commands
Add common $(COMPILE_C), $(LINK_C), and $(LIB_C) variables that contain
the commands to build a .o from a .c, an app from objects and libraries,
and a library from objects, respectively.

Change-Id: Ie2eaa13156b8bf3db7a4ffa66161382d829aef07
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-11 14:59:54 -07:00
Daniel Verkamp
9ea5194634 build: simplify dependency generation
Modern GCC and Clang versions can generate .d files without needing the
complex sequence of sed scripts that we currently use.

Also adjust the .o rule so the .d file is regenerated if it gets
deleted.

Also make the .o depend on all of the Makefile fragments in
$(MAKEFILE_LIST) so that it is rebuilt if the Makefile is changed.

Change-Id: I44e186c0f34dabfa0cf35db1ffa66b84d9dae87d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-10 14:40:22 -07:00
Daniel Verkamp
7cb4463dc5 build: use -std=gnu99 rather than -std=gnu11
We don't use any C11-specific features, and switching to C99 will allow
us to support older versions of GCC.

In particular, the version of GCC used by Travis CI is too old to
support -std=gnu11.

Change-Id: Id5bd8351fdd83e7f4b6703048a921c78c3f06ec6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-04 14:39:38 -07:00
Daniel Verkamp
afed5ba9da build: disable missing field initializer warning
This partially reverts commit fd93c1b1ef.

Older versions of GCC don't consider the empty initializer to be
acceptable when this warning is turned on.

Change-Id: I5c1c22ca988f464990c68eb56e4dd4d51cb99410
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 19:47:34 -07:00
Daniel Verkamp
2e03f6fe1c build: enable strict prototypes warning
This catches functions with empty parameter lists, which are not the
same as void parameter lists in C, as well as other (less-likely)
K&R-style parameter lists.

Change-Id: I18b09e01a60b3669d1cc5d8d96eec20cd872c497
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 16:05:06 -07:00
Daniel Verkamp
8307eb5f55 build: enable missing function declaration warning
This helps weed out functions that should be static, functions that are
not declared in public header files, and .c files that don't include
their .h interface headers.

Change-Id: Ie39f83ad4b320847e4a938bd1d4d0b4fa21c2ffa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 14:40:23 -07:00