This also requires us to change the create_test_list function to rely on
the run_test function for creating the canonical test list.
Change-Id: Ib35e7752935a3ac83de2702b6dfbd42539027f6a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476962
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We will be able to distinguish case and suite in the post process
scripts based on if they have any internal tests.
Change-Id: Iff2aa5caa251924d8a842085d87eb2a17aca45d4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
And clean out the corresponding calls to timing_enter and timing_exit
from the various test scripts.
Change-Id: I0759417b5a529e4c3649ce04cca1799c089da278
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476804
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
It's always set in autotest_common.sh, there's no need
to set it again in each test script.
Change-Id: Ib14c4189c553dad54a3065c1a1d413a5fc5a5347
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.
Change-Id: Id093b6c82d1f766ba6af13bed720977eceaa7ffc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Further patches in series will modify the suppression file,
so this patch now creates such file in /var/tmp/.
Meanwhile adding the known false positive to the list.
Any known false positives or leaks in external libs/execs
should be added by adding further entries to the suppression
file.
Change-Id: I7b78d900a86c6eca0b41240fb34dc4f7ad597079
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456622
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Remove SpdkFinalizeThread. It is no longer needed - the
thread-local destructor handles the cleanup.
Keep SpdkInitializeThread but make it static. We still
need this function for the main thread, so that it can
be explicitly called after the filesystem has been
loaded.
Part of this patch moves the rocksdb_commit_id to a
version that doesn't make calls to these functions.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib9aefdfede7e5636085079d2226ec1c4d2cae406
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452687
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This will help collect evidence for the intermittent
RocksDB failures seen in the test pool.
The method for getting the name of the /dev/shm trace
file is pretty kludgy (grepping the log) but it's the
best I could come up with and is very simple. Good
enough for now.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7b3acbcb54033555ff42b3b21a09c798a87c481b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452263
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
When debugging issues or running local tests, it is
nice to skip rebuilding all of the db_bench tool or
running mkfs on the underlying SPDK device. So add
SKIP_GIT_CLEAN and SKIP_MKFS checks which allow the
test caller to skip these steps.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4d3f739aaa908a816aacf89d3fed0985493b7f59
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This makes the test script easier to use outside of
the test pool, since it will just reuse the rocksdb
results directory rather than failing because the
directory already exists.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I754489c3321323bce8c8a5c352d4947d6270d489
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This was still pointing at the wrong place.
Change-Id: I793030432e9b4e7a4c284aa6c47273e9434cd4dc
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452245
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We are switching to a method where we clone the db bench dir in
each build before running tests.
Also update the commit ID to point at the proper place.
Change-Id: I0eeba7bf7118e7da367b1405ae7322668eea3221
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451984
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This signals which RocksDB commit should be checked
out for the SPDK RocksDB tests.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ba0be00747a2642b359b1e0e0c8c2c6d99cc4f1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451772
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We can still use 4GB for nightly testing, but cutting
the cache size to 2GB reduces the setup time for each
of the 5 db_bench test runs.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic946cfcd2ecac209e0e06b05777c18ccdec50d14
Reviewed-on: https://review.gerrithub.io/436564
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
remove from the list test cases from rocksdb.sh
add vhost live migration tc1 and tc2
Change-Id: I2e7beff62e9c54ea86481832c29c613357e19c99
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/429480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
This it to prepare for RPM package. Also lower number of dependencies
needed by SPDK tools.
Update changelog to deprecate Python 2 and explicit interpeter invoking
in scripts.
Change-Id: I2497cca721cbcbadc1c99c675f8b8b7f682d5efa
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Since IOAT is disabled by default, that option can be removed from
any config file used in tests.
Change-Id: I681bd1eb6a8c81328ba580be75b418994ccd85e3
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/423357
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We especially want to do this when a test fails, but
running it in passing cases too ensures spdk_bs_dump at
least gets run on every patch through the test pool.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2d696dbf21d14681a73cb990933cba4e57a5f8a5
Reviewed-on: https://review.gerrithub.io/414482
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
rocksdb has been frequently failing in nightly tests:
https://ci.spdk.io/spdk/nightly_status.html
Change-Id: I53d812dfb6146bd973fff3c40fe3490c0fba9fc8
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/406975
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will allow us to reduce core assignments for build pool vms to 8.
Change-Id: I405a12f2c1041f77f90add0b5d27abd75f7b948d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/403559
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This is the first step in reorganizing the spdk test heirarchy.
Change-Id: I83467653af2da2a53251649092bed6902d6cb86e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/401707
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I138362d4ebe157926228384ab032c057c03c822e
Reviewed-on: https://review.gerrithub.io/391891
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
move the main portion of run_tests.sh into rocksdb.sh to simplify
rocksdb testing and allow for the provision of more granular timing
coverage.
Change-Id: I60649394c496a347f4dca948d1d499ad8578c4a0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/378827
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
This adds support for cleaning page caches between each of the
db_bench test within run_test.sh script.
Change-Id: Ibcf37945d530a7a940d4a6365e3f1a9eeffb26c0
Signed-off-by: Vishal Verma <vishal4.verma@intel.com>
Reviewed-on: https://review.gerrithub.io/373631
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Remove RESULTS_DIR PREFIX since we already change the folder
Change-Id: I6be0b4dda481377a96d8fdbb69d5d60a6616c7d0
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/370829
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Temporarily disable stub usage in the RocksDB test until we diagnose the
cause of the intermittent test failures (no hugepages available).
This reverts commit 6bd7c5b42c.
Change-Id: Ie0eca36c3a15708d2483bae32ce2e217036d2b1d
Reviewed-on: https://review.gerrithub.io/367263
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: Daniel Verkamp <daniel.verkamp@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba5bb17a30b7d7ec39fe64df93860dd8500f2ad1
Reviewed-on: https://review.gerrithub.io/362607
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
RocksDB's build system does not correctly generate dependencies - if
headers have been removed in the SPDK repository since the previous
build, the .d files will prevent the build from continuing.
Avoid this by cleaning up any left-over build outputs from the previous
run.
Change-Id: Ib8669273eb38c241f83e14714683693430469462
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The RocksDB SPDK environment no longer depends directly on DPDK.
Change-Id: Ifcabcd137292f86724a1e3fed93abb94fa5a2344
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Core dumps stop working if the path is longer than 64 bytes.
Use readlink to shorten some of the very long relative paths.
Change-Id: I5e7eb6580ca581c5ac3a71afb7b62953836e2660
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Automated test scripts now set up 4096 huge pages by default,
so we can now safely allocate the default 4GB for blobfs cache.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8aea322068a5df79ca75737c43a4e0e80e7d2be4
For the automated tests, use this to directly write output
files to the output_dir.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I403f51a4f4d0cac0fca7ec610527bf616f97b20d
Change NO_PERF to USE_PERF which defaults to 1. This
will be overridden to 0 if set to 0 in the environment
or perf is not available on the system.
While here, disable perf in the automated test suite.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia182f6a3068d0da631973d4010dce24bd0f98eb5
This is the initial commit for "blobfs", a lightweight
filesystem built on top of the SPDK blobstore.
Also included in this patch:
1) a shim for using SPDK bdevs as the backing store for
SPDK blobstore/blobfs
2) documentation for using blobfs as the storage engine
with RocksDB
3) scripts for running a set of workloads and collecting
profiling data with RocksDB and blobfs
See doc/blobfs/getting_started.md included in this commit
for more details on blobfs, including some of the current
limitations.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2a6d3d4b87236730051228ed62c0c04e04c42c73
