Commit Graph

98 Commits

Author SHA1 Message Date
Daniel Verkamp
555a3d85c6 build: add Travis CI integration
Use the container-based travis-ci.org infrastructure to run only the unit
tests.

Change-Id: I4f7d011edb07341a668fa6fea495f6eb2a9c029b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-04 11:05:59 -07:00
Daniel Verkamp
3677f46af8 build: allow make to work from any directory
Set SPDK_ROOT_DIR explicitly in each Makefile so that make from a
subdirectory will work (assuming all dependencies from the upper
directory have already been built).  This allows partial rebuilds of the
source tree, as well as building the unit tests without requiring DPDK.

Change-Id: I3f65b805d490b40ff5ec53cceb61df542ce814f1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-04 10:19:08 -07:00
Daniel Verkamp
35e56a9603 nvme/perf: let all threads complete on error
Previously, as soon as a worker thread failed, the program would exit
before printing results.

Also add a message at exit time if any errors occurred during test
execution.

Change-Id: I7b3920f0acb8ce364e2bc5cbb78bbe88f3fa7146
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-03 15:57:33 -07:00
Daniel Verkamp
55f3f20e3b nvme/perf: increase task pool size
This matches the size of the request pool and enables running with a
higher concurrency level.

Ideally, these limits should be calculated from the requested queue
depth and number of workers, but for now, just increase the hardcoded
limit.

Change-Id: I6e890efc78a1336dddc0ab61db20c68004b30f54
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-03 15:57:33 -07:00
Daniel Verkamp
cfc88a45d0 nvme/perf: capture master lcore return code
Previously, if the work function on the master lcore failed, the perf
program would still return with a successful exit code.

Change-Id: Iec91c1f60824759ae62476ef6dce670fd402ddc5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-03 15:57:33 -07:00
Daniel Verkamp
a900f9ac9a nvme/perf: check nvme_register_io_thread() status
Do not continue running the thread work function if that thread could
not get an I/O queue.

Change-Id: I89033250bde0663f073ff35c76d1558d55b72ece
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-03 15:57:33 -07:00
Daniel Verkamp
c04ba5e952 nvme/perf: check memory allocations
Change-Id: I1ca8203a4f803e04afececc8f382cd088c53d63e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-03 15:57:33 -07:00
Daniel Verkamp
860c3ed1ba autotest: use hostname as lcov test name
This will allow us to figure out which test machines actually executed
each path when the results are combined.

Change-Id: I965fed0153f5d28aa8a0ff5e0d02fed19b49003d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-03 15:54:11 -07:00
Daniel Verkamp
f40e98d200 autobuild: add sleep to fix dependency test
With a reasonably fast machine, the full build can take less than a
second.

Make sure that at least one second has passed so the stat output will
differ when testing Makefile dependencies.

Change-Id: I462944d00c9c2ace2f6a1a7d6dadec4f77c25784
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-03 15:19:36 -07:00
Ben Walker
70db0e1c08 nvme/perf: Add support for multiple cores per device.
Intelligently allocate cores and devices to handle
the following cases:

1) Equal cores and devices
2) More cores than devices by using multiple cores per device
3) More devices than cores by using multiple devices from a single core

Change-Id: I3703f5c523268539bd00d399fe104c474a8e8c99
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2015-11-03 13:07:37 -07:00
Daniel Verkamp
afed5ba9da build: disable missing field initializer warning
This partially reverts commit fd93c1b1ef.

Older versions of GCC don't consider the empty initializer to be
acceptable when this warning is turned on.

Change-Id: I5c1c22ca988f464990c68eb56e4dd4d51cb99410
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 19:47:34 -07:00
Daniel Verkamp
2e03f6fe1c build: enable strict prototypes warning
This catches functions with empty parameter lists, which are not the
same as void parameter lists in C, as well as other (less-likely)
K&R-style parameter lists.

Change-Id: I18b09e01a60b3669d1cc5d8d96eec20cd872c497
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 16:05:06 -07:00
Daniel Verkamp
8307eb5f55 build: enable missing function declaration warning
This helps weed out functions that should be static, functions that are
not declared in public header files, and .c files that don't include
their .h interface headers.

Change-Id: Ie39f83ad4b320847e4a938bd1d4d0b4fa21c2ffa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 14:40:23 -07:00
Daniel Verkamp
e9fa656027 build: enable -Wextra except -Wunused-parameter
-Wold-style-declaration, -Wsign-compare, and
-Wmissing-field-initializers are included in -Wextra.

-Wunused-parameter is not very useful, so disable that explicitly, but
enable the rest of -Wextra.

Change-Id: I034b368d16306544e0eadafab9ec1ccf27f21645
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 14:40:22 -07:00
Daniel Verkamp
fd93c1b1ef build: enable missing field initializer warning
Replace {0} initializers with {}, since it is equivalent and simpler
(and avoids the warning).

Change-Id: Id2014a352e2c627598cd3b4f1ca04dba35f2dc3a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 14:40:22 -07:00
Daniel Verkamp
a945f60c79 build: enable signed-vs-unsigned compare warning
Change-Id: I93f069241cb74b3ec7d272bc390998372c376b16
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 14:40:22 -07:00
Daniel Verkamp
f79a334e33 build: fix old-style declaration warnings
Fix all of the uses of __thread so they are at the beginning (similar to
e.g. static).

Don't actually enable -Wold-style-declaration, since clang doesn't
understand that.

Change-Id: I0dcbb758143eab90fc978334c8f256c6602cc4cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 14:40:12 -07:00
Daniel Verkamp
497d40b19b build: add CONFIG_COVERAGE code coverage option
Collect coverage information for use with gcov.

Coverage is disabled on FreeBSD because the current version of clang
provided by FreeBSD can't successfully link with -ftest-coverage enabled
(the compiler-rt support libs are too old).

Change-Id: Icc444936caa852bfb9a02b37223209319a27a770
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 14:40:49 -07:00
Daniel Verkamp
ae80d88bcb util: fix return code checks in dev_get_blocklen
The ioctl() calls in dev_get_blocklen() were checking for != 0 instead
of == 0, so the default path (512) was always being taken.

Change-Id: Ib0b016b1d453fb94d408063417b7485ff24ed220
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-11-02 08:48:58 -07:00
Daniel Verkamp
c9cc869a3e nvme/perf: add Linux libaio benchmarking support
This allows comparing the Linux kernel driver's performance to the SPDK
user-mode NVMe driver.

Change-Id: I71c70163a4133c2f237c8c57b3c698ec261455f5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-26 11:16:57 -07:00
Daniel Verkamp
aaf0555e03 util: add file size and block size functions
Change-Id: I0d3269805470768ce75e81955db601c48ae0e737
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-26 11:16:56 -07:00
Daniel Verkamp
d5ced366c0 CONFIG: allow overriding options in make command
Change-Id: Iba75d4b4be0175e8ddd5d5905eece1e2c9f6db51
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-22 12:24:57 -07:00
Daniel Verkamp
4f677a1d4c nvme: only invoke request free macro in one place
Rename the nvme_free_request macro to nvme_dealloc_request to match
nvme_alloc_request and add a wrapper function to nvme.c so that the
macro contents are only expanded once.

The DPDK nvme_impl.h uses rte_mempool_put(), which generates a large
amount of code inline.  Moving this macro expansion to a wrapper
function avoids inlining it in the multiple places nvme_free_request()
gets called, most of which are error handling cases that are not in the
hot I/O path.

Change-Id: I64ea9c39ba47e26672eee8d5058f1489e07eee5b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-20 07:43:41 -07:00
Daniel Verkamp
38997df85d nvme/test: fix memory leaks in nvme_ns_cmd_ut
The I/O splitting tests don't pass the I/Os through the normal
completion path, so we need to free the children ourselves.

Fixes all Valgrind warnings for nvme_ns_cmd_ut.

Change-Id: Iaf7d9f7f4cab71428a0123ee30d0f6042001e423
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-19 10:57:34 -07:00
Daniel Verkamp
3adea82331 build: always build with -fPIC
This allows the same objects to be linked into static and shared
libraries and allows the creation of position-independent executables
with the static libraries.

Change-Id: I119949c3644c02a83e414227615dcc2d8f896286
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-13 10:12:54 -07:00
Daniel Verkamp
557f3bd6c2 build: enable -fstack-protector
Change-Id: I2ba23c48ee8b9048022cff6cc861e990fd1ff570
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-13 10:04:04 -07:00
Daniel Verkamp
3a2b871a41 build: enable printf format warnings
Some of these are covered under -Wall, but add them all just to be clear.

Change-Id: I63104be4bf58becd19a4c30a4d275d2c24c6761d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-12 16:41:55 -07:00
Daniel Verkamp
28b4b01136 build: enable noexecstack
Modern toolchains already enable noexecstack by default, but set it
explicitly just to be sure.

Change-Id: I2866220db5a5ed0bb6e78a4e83763839ea3b93b0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-12 16:41:54 -07:00
Daniel Verkamp
ed3b72fba9 build: enable _FORTIFY_SOURCE=2 for release builds
Change-Id: I61a0700c2ee6e22cdde6e6cc8c491a1e52043136
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-12 16:41:54 -07:00
Daniel Verkamp
92a7dba668 build: enable full RELRO in LDFLAGS
This forces the ELF loader to resolve all relocations immediately at
load time rather than allowing lazy binding.

Therefore the loader can make the GOT (global offset table) read-only
once the relocations are complete during the loading process.

Change-Id: I89a878d1060ce79d58c5be7700d8b122a33a8ceb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-12 16:41:54 -07:00
Daniel Verkamp
2a3d4b18ca build: add CUnit-Memory-Dump.xml to .gitignore
This is generated by the CUnit library when built with
--enable-memtrace.

In particular, the CUnit package on FreeBSD is configured this way.

Change-Id: I616b7d91ed560be2706089287510a772a5c47e11
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-12 15:11:57 -07:00
Daniel Verkamp
3e5fe6f602 autobuild: unbind NVMe devices on FreeBSD
Change-Id: I5e6a6ca6751d77aed48bedc2e59f97139dd8ef00
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-09 10:03:25 -07:00
Daniel Verkamp
17f255e208 autotest: use inherited output_dir if it's set
Change-Id: I182ebe8dbbd650b6093ac34e52a2ff595811e047
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-08 17:25:58 -07:00
Daniel Verkamp
84b8f0cc8b autotest: output timing in flamegraph format
Change-Id: I9f571ce635431613fc6a84e106e5052cce2dbf54
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-08 16:07:47 -07:00
Liang Yan
ae482f07c1 nvme: Add get_status_string test for nvme_qpair
Change-Id: Ib3f9d342549de52869ec984d0bc61ccd25b71a14
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2015-10-08 13:44:54 +08:00
Daniel Verkamp
92da744700 util: add sprintf_alloc() function
Like sprintf() with automatic buffer allocation.

This should help to avoid fixed-size buffers in
non-performance-sensitive code that formats strings.

Change-Id: I35209ae84014ed5daf41baa5b03af8a5f6b02b8e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-06 10:40:51 -07:00
Daniel Verkamp
5fc71d7e40 nvme: disable valgrind for nvme_ut
nvme_ut is testing multi-threaded operations, and Valgrind's thread
behavior is different than running the program natively.  Under
Valgrind, the unit test essentially hangs waiting for the global
variable to be updated.

Change-Id: Id3665002c16ac3e695c50325375305a76f72cee4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-06 10:21:23 -07:00
Daniel Verkamp
f0b3db3f1e autobuild: FreeBSD portability fixes
Change-Id: Id6db5d80ee7fa602da8763372f92555e7e299445
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-06 09:09:00 -07:00
Daniel Verkamp
8780063f2c nvme: run unit tests under Valgrind if available
Change-Id: Iad8403903d420418692793d9940e33e2d3772e6b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-05 11:00:29 -07:00
Daniel Verkamp
1c96eff6a2 nvme: fix uninitialized memory in nvme_qpair_ut
nvme_alloc_request() does not zero out the request (this is
intentional, since the real implementation uses a mempool where requests
get reused).  Add nvme_allocate_request() wrapper that initializes the
request correctly.  This fixes all uses of uninitialized memory caught
by Valgrind in nvme_qpair_ut.  This test was also failing in practice on
FreeBSD due to non-zero buffers returned from malloc().

Change-Id: I2d6ea29289bd4887aaa9120fba6bce10088e6917
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-10-05 10:33:22 -07:00
Daniel Verkamp
cac5caec8b nvme: sync up unit test nvme_impl.h mutex wrapper
The default version of nvme_impl.h was cleaned up to release the pthread
mutex attr on failure cases, but the unit test version did not receive
this update.

Change-Id: I899b7dc809393dc8e6fd24ed98e1d0a61ecf1c95
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-09-30 16:18:18 -07:00
Liang Yan
2bec440057 nvme: Change some strange alignment in nvme_ctrlr.c
Change-Id: I295d1864eaf8233df0678c3df948ac353613b27d
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2015-09-29 14:01:44 -07:00
Ben Walker
c45dfec4f6 nvme: Add concurrency to nvme perf example.
Change-Id: Ic565b70517bb2958b64fe7f2cf59a31e4b6250ef
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2015-09-29 14:01:33 -07:00
Daniel Verkamp
364331fd94 nvme: add I/O split test with stripe size enabled
Change-Id: I49166c2a07274cf66a8a9519805308c0db5ab13b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-09-28 14:20:03 -07:00
Daniel Verkamp
ac0c37924c nvme: validate child I/O in splitting test
Check the LBA and block count fields of the I/Os generated by the
splitting function to ensure they were split correctly.

Change-Id: I84abb1ac462fb7423d51a1be384fd1be68dfecae
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-09-28 14:04:51 -07:00
Daniel Verkamp
d4ee014f21 nvme: use CU_ASSERT_FATAL to abort tests on errors
Instead of repeating the check for conditions that must be fulfilled to
continue the tests, just use CU_ASSERT_FATAL, which will abort the test
and return.

Change-Id: If617b286a587d9efb1ce57b90061634ed5bc7ae8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-09-28 14:04:11 -07:00
Daniel Verkamp
d2e10e88ec nvme: move nvme_request_add_child to nvme_ns_cmd.c
This is the only place nvme_request_add_child() is used, so move it
nearby and make it static to allow the compiler to inline it.

Change-Id: If4a7e17fde0b0272e1d4432c1dcedbec27c25371
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-09-28 14:04:11 -07:00
Daniel Verkamp
21b37d4ee6 nvme: make the unit test assert actually assert
The current implementation of nvme_assert in the unit test nvme_impl.h
just prints the message and continues.

We should not be triggering assert conditions, even in the unit test
code, so make nvme_assert actually call assert().  This lets us catch
mistakes in the unit tests more easily.

Also fix the two unit tests that currently trigger an assert:
- The I/O splitting test in nvme_ns_cmd_ut was passing an invalid
  combination of NULL payload with non-zero lba_count.
- The ctrlr_cmd test was passing an invalid number of entries to
  nvme_ctrlr_cmd_get_error_page().  This case should probably not be an
  assert but rather an error code.  However, the function does not
  return a status code currently, so it is not trivial to make that
  change.  For now, just drop the asserting test case and the code added
  to the test to work around it.

While we're here, fix the macros in the unit test nvme_impl.h so they
are usable in single-line conditionals without braces - that is the
whole point of the do { ... } while (0) pattern, so there should be no
trailing semicolon.

Change-Id: Iad503c5c5d19a426d48c80d9a7d6da12ff2c982a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-09-28 14:04:07 -07:00
Daniel Verkamp
29fb756232 vtophys: avoid NULL dereference if malloc fails
Change-Id: I5fd865d158716966de6baf5d9d45fbeb47c44e71
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2015-09-28 12:52:05 -07:00
Liang Yan
fdd17ae3ee nvme: Update nvme unit test
Cover more functions and conditions for nvme_ctrlr_cmd.
Remove unnecessary lines in nvme_ctrlr_ut.
Update nvme_qpair_ut.
Remove unnecessary header file.

Change-Id: I8c5a75573b26210ca57711b366acd55ab96614c0
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2015-09-28 10:02:41 -07:00