Commit Graph

22 Commits

Author SHA1 Message Date
Pawel Niedzwiecki
cb2d8466dc test/rocksdb: move nightly test case to RUN_NIGHTLY_FAILING.
rocksdb has been frequently failing in nightly tests:
https://ci.spdk.io/spdk/nightly_status.html

Change-Id: I53d812dfb6146bd973fff3c40fe3490c0fba9fc8
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/406975
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-20 17:24:45 -04:00
Jim Harris
bb5f94f838 blobfs/fuse: account for leading slash in filenames
The RocksDB plugin no longer strips the leading slash
from filenames - so update the fuse plugin accordingly.

If and when BlobFS supports a real directory structure,
we'll need to avoid the leading slashes, but for now let's
just make the fuse plugin usable.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5609dd4d5fd3db85f266963172f35ac004869d66

Reviewed-on: https://review.gerrithub.io/407231
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-11 16:03:14 -04:00
Seth Howell
656c4a7abe test: move test/lib/blobfs under test/blobfs
Change-Id: Icaac9df0623148e6c785d2b1a560b165a1a03408
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/404966
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-23 19:45:09 -04:00
Seth Howell
f1bf28de41 test: set all coremasks to use 8 or fewer cores.
This will allow us to reduce core assignments for build pool vms to 8.

Change-Id: I405a12f2c1041f77f90add0b5d27abd75f7b948d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/403559
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-13 00:51:37 -04:00
Seth Howell
a562812dcf autotest: add test completion tracking
Change-Id: I1ca0578a010db2ff8535505bfd981cd1c368e403
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/392240
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-01 17:13:41 -05:00
Seth Howell
c9e5d2a6d0 autotest_common.sh: move to test/common
This is the first step in reorganizing the spdk test heirarchy.

Change-Id: I83467653af2da2a53251649092bed6902d6cb86e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/401707
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-02-27 20:37:27 -05:00
Jim Harris
7c2c5b947f test/rocksdb: reduce test time for per-patch tests
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I138362d4ebe157926228384ab032c057c03c822e

Reviewed-on: https://review.gerrithub.io/391891
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-12-15 15:48:06 -05:00
Seth Howell
70684937b7 test/blobfs: add timing for rocksdb benchmarks
Change-Id: Ibcbb75671e3526076d31f6dc53da490f3c0e6d3e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/378830
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-09-20 19:33:50 -04:00
Seth Howell
9290e0bcf7 test/blobfs: consolidate run_tests.sh into rocksdb.sh
move the main portion of run_tests.sh into rocksdb.sh to simplify
rocksdb testing and allow for the provision of more granular timing
coverage.
Change-Id: I60649394c496a347f4dca948d1d499ad8578c4a0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/378827
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-09-19 17:16:05 -04:00
Vishal Verma
94262269de test/blobfs: add 'drop_cache' in run_tests script
This adds support for cleaning page caches between each of the
db_bench test within run_test.sh script.

Change-Id: Ibcf37945d530a7a940d4a6365e3f1a9eeffb26c0
Signed-off-by: Vishal Verma <vishal4.verma@intel.com>
Reviewed-on: https://review.gerrithub.io/373631
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-09 13:39:27 -04:00
Ziye Yang
3a289a6eea test, blofs: remove the RESULTS_DIR prefix
Remove RESULTS_DIR PREFIX since we already change the folder

Change-Id: I6be0b4dda481377a96d8fdbb69d5d60a6616c7d0
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/370829
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-07-25 15:56:57 -04:00
Daniel Verkamp
16fb6ba93b Revert "test/blobfs: use test stub"
Temporarily disable stub usage in the RocksDB test until we diagnose the
cause of the intermittent test failures (no hugepages available).

This reverts commit 6bd7c5b42c.

Change-Id: Ie0eca36c3a15708d2483bae32ce2e217036d2b1d
Reviewed-on: https://review.gerrithub.io/367263
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-06-27 13:52:15 -04:00
Jim Harris
6bd7c5b42c test/blobfs: use test stub
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba5bb17a30b7d7ec39fe64df93860dd8500f2ad1

Reviewed-on: https://review.gerrithub.io/362607
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-05-25 16:55:03 -04:00
Daniel Verkamp
bf2795c65d test/blobfs/rocksdb: clean before building
RocksDB's build system does not correctly generate dependencies - if
headers have been removed in the SPDK repository since the previous
build, the .d files will prevent the build from continuing.

Avoid this by cleaning up any left-over build outputs from the previous
run.

Change-Id: Ib8669273eb38c241f83e14714683693430469462
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-02 16:49:50 -07:00
Daniel Verkamp
f50ea14403 rocksdb: stop passing $DPDK_DIR to db_bench build
The RocksDB SPDK environment no longer depends directly on DPDK.

Change-Id: Ifcabcd137292f86724a1e3fed93abb94fa5a2344
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-01 11:11:03 -07:00
Daniel Verkamp
8375124f24 test/blobfs/rocksdb: check for errors
Change-Id: I8ab76beff22b803ac1ff22f1c91826870dbe9e7b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-01 11:11:03 -07:00
xiajingx
a005708d7d blobfs: Add code to create results folder.
Change-Id: I28a4a1bcce3de6df62a1d4ee2171447c7e26dc9e
Signed-off-by: xiajingx <jingx.y.xia@intel.com>
2017-04-11 09:51:26 -07:00
Ben Walker
10f9391a22 test: Shorten some relative paths using readlink
Core dumps stop working if the path is longer than 64 bytes.
Use readlink to shorten some of the very long relative paths.

Change-Id: I5e7eb6580ca581c5ac3a71afb7b62953836e2660
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-04-07 08:14:34 -07:00
Jim Harris
6b1046f780 test/blobfs: use default cache size
Automated test scripts now set up 4096 huge pages by default,
so we can now safely allocate the default 4GB for blobfs cache.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8aea322068a5df79ca75737c43a4e0e80e7d2be4
2017-03-29 14:13:22 -07:00
Jim Harris
031937d0ff test/blobfs/rocksdb: allow alternate location for output
For the automated tests, use this to directly write output
files to the output_dir.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I403f51a4f4d0cac0fca7ec610527bf616f97b20d
2017-03-28 07:55:40 -07:00
Jim Harris
3f7f19cb36 test/blobfs/run_tests: rework perf checks
Change NO_PERF to USE_PERF which defaults to 1.  This
will be overridden to 0 if set to 0 in the environment
or perf is not available on the system.

While here, disable perf in the automated test suite.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia182f6a3068d0da631973d4010dce24bd0f98eb5
2017-03-27 14:23:27 -07:00
Jim Harris
1edd9bf3e4 blobfs: Add a lightweight filesystem built on the blobstore
This is the initial commit for "blobfs", a lightweight
filesystem built on top of the SPDK blobstore.

Also included in this patch:

1) a shim for using SPDK bdevs as the backing store for
   SPDK blobstore/blobfs
2) documentation for using blobfs as the storage engine
   with RocksDB
3) scripts for running a set of workloads and collecting
   profiling data with RocksDB and blobfs

See doc/blobfs/getting_started.md included in this commit
for more details on blobfs, including some of the current
limitations.


Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2a6d3d4b87236730051228ed62c0c04e04c42c73
2017-03-24 14:15:45 -07:00