zipf is a power law probability distribution. When
applied to performance testing of block devices, it
will select blocks over the full range of LBAs, but
will more frequently select lower-numbered LBAs.
The theta parameter governs the distribution - higher
values of theta will concentrate the distribution on
a smaller number of LBAs.
Note that fio supports zipf, so adding it to SPDK
will enable our perf tools (bdevperf, nvme-perf) to
provide similar functionality.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7df129c9d61996a2070188c6cd9f1fde631ac208
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7779
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.
Increasing major so version after LTS, allows the quarterly releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.
Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 21.04
release.
This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I44d01154430a074103bd21c7084f44932e81fe72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6167
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Interrupt is aiming to support interrupt
mode in SPDK application framework.
fd_group is implemented by epoll on linux.
fd_group can be inserted into thread and reactor
to do epoll_wait internally in order to avoid polling
Change-Id: I9077ad648a97fbd68fd46f43de2e16440bedab0b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4268
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_bit_pool is a wrapper around spdk_bit_array with the
intentions of providing much better performance for allocating
from a fragmented bit array. The cost of searching a large bit
array for a cleared bit can become expensive so the spdk_bit_pool
will provide an ability to track extents of recently cleared
bits.
This initial commit does not adding the tracking yet - it is strictly
a wrapper around spdk_bit_array with enough functionality to replace
the use of spdk_bit_pool in SPDK blobstore with equivalent performance.
This will allow us to switch blobstore to use this minimal
wrapper first, and then iteratively improve spdk_bit_pool to provide
the better performance.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I95d0d12db47eac73e0641eb7f94fa5df43d42e45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3974
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Also, while we are here, consolidate setting SO_SUFFIX to one spot.
Previously, it was possible for a library to slip through
without an SO version.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4db5fa5839502d266c6259892e5719b05134518c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This will allow us to keep track of compatibility issues on a
per-library basis.
Change-Id: Ib0c796adb1efe1570212a503ed660bef6f142b6e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Like memcpy, but works on two iovecs.
Change-Id: Ia1cf462a95690286f0c19325fc10937b9ba6baf3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When dealing with sockets most code in SPDK buffers
data into large chunks to minimize the number of
syscalls made. The pipe utility is designed to make
that easy.
Change-Id: Ie29966712bbfb43fb49457e042903cf45864e6c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465707
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Function multiversioning conflicts with LTO when applied
to a function defined in a header file included from
multiple compilation units.
Change-Id: I65bed3903a717b7e982ab185c314d2118ae0e795
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465972
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
The new added API can load file content into a data buffer, which
can be used to read configuration file as well as JSON file, and
we can add WRITE API in future which can be used to implement the
persistent reservation feature.
Change-Id: I9aaca7571648e1ab6dbfdd7cfd6ca34083cbeec2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
DIF and DIX will be supported in SPDK throughout, e.g., NVMe
driver, NVMe-oF initiator and target, NVMe block device, malloc
block device, SCSI, iSCSI target, FIO plugin, and Perf.
Generic and common APIs to generate and verify DIF and inject
bit flip error to any field will be helpful for them.
This patch is the first in the patch series.
This patch adds APIs to generate and verify DIF for SGL extended
LBA payload as byte alignement and granularity.
Change-Id: Ie6588d960113761f10efbf2d2a3cae004af37ce8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/432261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This prevents the SPDK build from requirng unnecessary dependencies when
not compiling certain features. Also, fixes github issue #434
Change-Id: I7d0520474f3656ae32670313f2290e6b741c5ca8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426131
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
RPC method (nvme-passthrough-cmd) needs base64 to
transfer binary buffer of nvme-cmd through Json text.
Change-Id: I50b9d0efca97da45d6d4fd2a1b53b74811947191
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417805
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This makes more sense as a first class library.
Change-Id: Ibd5c578f8708bd8c7d83fe1629e97c9a3316b56b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414698
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
This lets us have a common place to put definitions like the length of
the UUID string, as well as abstract away some of the API warts in
libuuid (non-const values, no size checking for uuid_unparse, etc.).
Change-Id: I80607fcd21ce57fdbb8729442fbb721bc71ccb98
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402176
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Fixes github issue #218.
This patch introduces spdk_cpuset object to store and manipulate
the set of individual CPUs. The main objective of this object is
to replace cpumask declared as uint64_t and extend the limitation
of supported CPUs (lcores) above 64 CPUs.
spdk_cpuset is always allocated dynamically and accessed by opaque
pointer, what makes it easier to extend in the future without
breaking API/ABI.
This patch also extends parsing function allowing to set cpumask
using a list of cpus e.g. "[0-4,10,12]" sets mask of 0,1,2,3,4,10,12
as well as hexadecimal string with and without "0x" prefix.
Change-Id: I475c3ba7fab629021a22e03176e57e400dd24a49
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/390794
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch remove need for additional buffer when translating error code
to string.
Change-Id: Iaa60088b5c450581d3cdddbb425119b17d55a44b
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/386114
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Only Makefiles for libraries that directly depend on DPDK (rather than
the SPDK env abstraction) should add $(ENV_CFLAGS).
Change-Id: Ifdf44d3ef8c42bbf7f20edd524b330d00658235b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/392818
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7174f1799361b8337ff5590b90ad6a0564ca8e9b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/391899
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Factor out the iSCSI and GPT CRC32 functions into generic library
functions.
Change-Id: I1f1a5f3968a983b663a51bd984500492eeb12605
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/370765
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This allows users to swap their PCI library from
libpciaccess/dpdk to another mechanism using the standard
method for swapping out the env library.
Change-Id: Ib2248f8b43754a540de2ec01897e571f0302b667
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This allows users to swap out SPDK's third party
libraries for an implementation based on their own
framework.
Change-Id: Ia0b7384ce5e31acba5ad0d7002dec9e95b759c52
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This patch adds a basic framework for creating I/O channels
for I/O devices. An spdk_io_channel represents a one-to-one
mapping between a calling thread (represented by spdk_thread)
and an I/O device that the thread will perform I/O operations
on.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I658ab7f995cc962f4e2a204e058cdd3ad3fd735d
Explicitly include spdk.common.mk at the top of all lib Makefiles so
that CONFIG options and other predefined variables are set.
Change-Id: I1e560c294fe8242602e45191a280f4295533ae44
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Resolve relative paths before using them to clean up command lines.
This should also help shorten the overall command line length that gets
embedded in the binary and used when locating the executable from a
coredump.
Change-Id: Ibff9849ede198bb04313496c8b7131485ffaf14f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Also add a space between Copyright and (c).
The copyright year can be determined using git metadata.
Also remove the duplicated "All rights reserved." - every instance of
this line already has a corresponding "All rights reserved" immediately
below it, except for examples/ioat/kperf/kmod/dma_perf.c, where I have
added it manually.
Performed using this command:
git ls-files | xargs sed -i -e 's/Copyright(c) \(.*\) Intel Corporation. All rights reserved./Copyright (c) Intel Corporation./'
Change-Id: I3779f404966800709024eb1eb66a50068af2716c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Use $(CLEAN_C) throughout the Makefiles to clean up a consistent set of
generated files.
This also adds coverage files to the list of cleaned files.
Change-Id: Iceb922935a45c9eecbf2f3443bd0ee4f5c966825
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
lib/memory was already using this pattern; extend it to lib/util and
lib/nvme.
Change-Id: I84a633d7805522fc94d8fc11ad5486ce552702e5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The .o files are always kept anyway, so there is no need for an explicit
rule.
Change-Id: Id1687ba89daabfda5802e4328deb127403277928
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Add common $(COMPILE_C), $(LINK_C), and $(LIB_C) variables that contain
the commands to build a .o from a .c, an app from objects and libraries,
and a library from objects, respectively.
Change-Id: Ie2eaa13156b8bf3db7a4ffa66161382d829aef07
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Set SPDK_ROOT_DIR explicitly in each Makefile so that make from a
subdirectory will work (assuming all dependencies from the upper
directory have already been built). This allows partial rebuilds of the
source tree, as well as building the unit tests without requiring DPDK.
Change-Id: I3f65b805d490b40ff5ec53cceb61df542ce814f1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Like sprintf() with automatic buffer allocation.
This should help to avoid fixed-size buffers in
non-performance-sensitive code that formats strings.
Change-Id: I35209ae84014ed5daf41baa5b03af8a5f6b02b8e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Move dependency includes into a new spdk.deps.mk file,
then include it at the end of Makefiles that build
source files.
Also add a test to autobuild.sh to confirm that
binaries are regenerated if we make after touching a
header file.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If6a1905706a840f92cbdf3ace7fbdb27fe2de213
