numam-spdk/examples/nvme/fio_plugin
Ziye Yang 908ecf9cc4 fio_plugin: add 1 io_queue_depth
Checked the code and found that our nvme perf example
added 1 io_queue_depth. Since if you set queue_depth=n,
the real entry is n - 1.

Change-Id: Ie204fc061b98e853e9b810eff21f79565c2ad8d0
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/367931
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-07-05 15:40:34 -04:00
..
.gitignore nvme: Add an fio plugin 2016-05-18 13:51:36 -07:00
example_config.fio fio_plugin: switch to LD_PRELOAD instead of dynamically loading 2017-06-14 11:12:29 -04:00
fio_plugin.c fio_plugin: add 1 io_queue_depth 2017-07-05 15:40:34 -04:00
full_bench.fio fio_plugin: switch to LD_PRELOAD instead of dynamically loading 2017-06-14 11:12:29 -04:00
Makefile build: create a common mk file for nvme library tests and examples 2016-11-22 11:45:47 -07:00
README.md fio_plugin: fix README 2017-06-28 13:06:37 -04:00

Compiling

First, clone the fio source repository from https://github.com/axboe/fio

git clone https://github.com/axboe/fio

Then check out the fio 2.18 tag

cd fio && git checkout fio-2.18

Finally, compile the code with

./configure && make

Next, edit the CONFIG file located in the root of the SPDK repository and set CONFIG_FIO_PLUGIN to y and FIO_SOURCE_DIR to the location of the fio repository that was just created.

Further, you'll need to build DPDK with -fPIC set. You can do this by modifying your DPDK config file (i.e. config/defconfig_x86_64-native-linuxapp-gcc) to include the line

EXTRA_CFLAGS=-fPIC

At this point, build SPDK as per normal. The fio plugin will be placed in the same directory as this README.

Usage

To use the SPDK fio plugin with fio, specify the plugin binary using LD_PRELOAD when running fio and set ioengine=spdk in the fio configuration file (see example_config.fio in the same directory as this README).

LD_PRELOAD=<path to fio_plugin binary> fio

To select NVMe devices, you simply pass an identifier as the filename in the format

filename=key=value [key=value] ... ns=value

Do not have any ':' in filename, otherwise it will be spilt into several file names. Also the NVMe namespaces start at 1, not 0! The namespace must also be specified at the end.

Local PCIe NVMe devices will be specified using a PCI domain.bus.slot.func traddr format. For example:

filename=trtype=PCIe traddr=0000.04.00.0 ns=1

Remote devices accessed via NVMe over Fabrics will be specified based on the following example:

filename=trtype=RDMA adrfam=IPv4 traddr=192.168.100.8 trsvcid=4420 ns=1

Currently the SPDK fio plugin is limited to thread usage model, so fio jobs must also specify thread=1 when using the SPDK fio plugin.

fio also currently has a race condition on shutdown if dynamically loading the ioengine by specifying the engine's full path via the ioengine parameter - LD_PRELOAD is recommended to avoid this race condition.

When testing random workloads, it is recommended to set norandommap=1. fio's random map processing consumes extra CPU cycles which will degrade performance over time with the fio_plugin since all I/O are submitted and completed on a single CPU core.