1122 Commits

Author SHA1 Message Date
Karol Latecki
5277a5324e scripts/vagrant: change vagrant box sources
Some time ago I created a patch which unified all boxes
URLs to generic/* which come from a single author/company
and seemed to be quite universal.
Unfortunately some of the boxes are not that generic as
the name suggest, so I am reverting my change and will
use boxes from vaious sources.

Ubuntu 1804 - had heavy DNS issues due to hardcoded
addresses in resolved.conf. Very hard if not impossible
to fix if working with synced folders like we do.

Centos 7 and 8 - sshfs not available by default. Also
had some other minor quirks. Official boxes available
so let's use them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I514d3b92c6ac5844b1929fdadfd35d6079933247
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1622
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
a34c17027b scripts/vagrant: fix permissions for backend files
Owner and group were not set properly on Ubuntu systems,
as they use different names for that.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ie7fc49516072ce4bcd506c4c39bbd26786004c49
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1580
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
3d3e8096b7 scripts/vagrant: replace hardcoded ocssd metadata file paths
These can't be hardcoded to a single location. It makes
running multiple VMs with OCSSD emulation impossible.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4ce0412fc9fc4542670a9e70c08219003e41828b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1488
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
53dbdf6dce scripts/vagrant: update help message
Just moving stuff around and updating some information.
I just felt like it's needed.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ia41f99603fb60111924cd5a3daf3c44091245c89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1157
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
e16128cece scripts/vagrant: remove vhost test configuration steps
We are not using VMs to run Vhost tests (at least at
the moment) and these steps are outdated. These steps
will need a re-write if we decide to put them back in,
so we can remove them for now.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I61d09bdc481814d8bcf6fe86ed4ccd84b36ad8d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1155
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
2f96a090de scripts/vagrant: remove Fedora 29
Remove Fedora 29 from the OS list. It reached EOL
some time ago and is not supported.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ie1bd704a39ab4c83d5e652741d21f6f0ac260540
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1156
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Maciej Wawryk
051d3fe016 test/vm_setup: Fix permision error in Centos7
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic704e9ae97bb68d96401818fb52940e86eaa36a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1409
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-02 14:51:24 +00:00
Michal Berger
576602acc4 qat_setup: Fix logic check against service's exit status
service is a plain Bash wrapper which either executes systemctl or
rc sysvinit scripts directly, hence its exit status should follow
standard logic - 0 for success != 0 for failure.

Reverse the check to not complain about service failure when it
was actually successfully started.

Change-Id: I29dd22c884157182549ebcc2ab4097eb098d7d3b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1603
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-02 07:03:47 +00:00
Tomasz Zawadzki
99cff8350b test: remove SPDK_TEST_EVENT flag
Test in event.sh take 5 seconds on CI at this time.
For the tests to run both SPDK_RUN_FUNCTIONAL_TEST
and SPDK_TEST_EVENT had to be enabled.

This served as very minor time saver, so removing
this flag shouldn't affter test times more that the
5 seconds.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie3eaa587daf28b2efbc3cfdff5ad648b7529e7f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:40 +00:00
Michal Berger
6b8f9a0f81 test/bdev: Convert setup_rbd_conf() to RPC calls
Change-Id: Icfbb571d1cd2a2539cd9026f1437c75450b77512
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-31 07:55:58 +00:00
Karol Latecki
051a6902d8 rpc/client.py: replace use process_time() instead of time()
time.clock() was depreceted since Python 3.3 and was finally
removed in Python 3.8.
https://docs.python.org/3.8/whatsnew/3.8.html#api-and-feature-removals

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ffe372b61c8afc0ddbbee0064b95059fa184616
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-23 09:37:44 +00:00
Karol Latecki
4f24f2ca4a scripts/pkgdep: change archlinux pmdk installation dir
By default "make install" installs pmdk header files in
/usr/local/include while autotest scripts expect them
to be in /usr/include.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I906ffd8a7858a37525758c3fc8129cb22286a8f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1368
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-23 09:37:17 +00:00
Maciej Szwed
3524dd0153 pkgdep: Add ncurses to pkgdep
ncurses library will be required by upcoming spdk_top app.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I16268c2ac772a45abdbd12d8a761d46041ce6e20

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1247
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-03-20 08:41:11 +00:00
Karol Latecki
2da5c24d8e scripts/pkgdep: change keyserver for archlinux rdma-core installation
Previously used keyserver keeps rejecting the gpg key.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ifed1d5c4db61f1b6d1e985fa048cc17801046f35
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1329
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-19 08:51:13 +00:00
Karol Latecki
9bf3be9f95 scripts/pkgdep: add archlinux dependency for pmdk
pkg-config was missing for pmdk installation.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I795c4b3a71d611292beaa650960afb639e827e78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-03-19 08:51:13 +00:00
Maciej Wawryk
b5700d4d7c scripts/vagrant: Fix unsupported machine type error
Vagrant fails during creation virtual machine using custom qemu emulator.
Setting machine type to "pc" resolved this problem.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1822439bb2235b5881d3c331c378017928953ac4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-19 08:50:53 +00:00
Karol Latecki
db43b387ba test: do not use iter_pci_class_code in tests
Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.

This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests.

Fixes #1235

Change-Id: I82b9935fc88605b636c2096be6c71d4880a567c8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1309
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:21 +00:00
Maciej Wawryk
0172f27974 perf/nvmf: Add more latency measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1ff5e6440ebef50d0488f020edfd8d5b7e274162
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1281
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:02 +00:00
Maciej Wawryk
3509fc12df Add cpus_allowed_policy param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I82a1632d4bb8742bd93f33a5a094a9cb4aa2e927
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-17 16:38:16 +00:00
Maciej Wawryk
0ebd0dce19 Add subsystem division in Kernel initiator
This will allow us to use num_cores and cpus_allowed option also
in Kernel initiator side.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iad4e8bf7cd8a614ada8f443a8b1036dc2c4803e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
0eab351528 Add cpu_frequency param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id594da091d7bcd5d23a752ccfab9bfa5b7726b49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1113
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
e068db83f1 Add cpus_allowed param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0d113e39c1ae9ff284f408998f5bba82352a2891
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/993
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Liu Xiaodong
f2bfad5908 scripts/common.sh: print error msg to stderr
Error msg in common.sh should be printed into stderr
in order to avoid improper usage.

For example:
  If env has no lspci and pciconf, then function
iter_all_pci_class_code will return the error msg back,
every word in error msg will be used by caller wrongly.

Change-Id: I6a875e49527539ba82bd331c8878e972e26cbc39
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-12 09:03:33 +00:00
Darek Stojaczyk
8b98cdb64a scripts/rpc.py: add daemon mode
Add rpc_cmd() bash command that sends rpc command to an
rpc.py instance permanently running in background.
This makes sending RPC commands even 17 times faster.

We make use of bash coprocesses - a builtin bash feature
that allow starting background processes with stdin and
stdout connected to pipes. rpc.py will block trying to
read stdin, effectively being always "ready" to read
an RPC command.

The background rpc.py is started with a new --server flag
that's described as:

> Start listening on stdin, parse each line as a regular
> rpc.py execution and create a separate connection for each command.
> Each command's output ends with either **STATUS=0 if the
> command succeeded or **STATUS=1 if it failed.
> --server is meant to be used in conjunction with bash
> coproc, where stdin and stdout are named pipes and can be
> used as a faster way to send RPC commands.

As a part of this patch I'm attaching a sample test
that runs the following rpc commands first with the regular
rpc.py, then the new rpc_cmd() function.

```
time {
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]

        malloc=$($rpc bdev_malloc_create 8 512)
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "1" ]

        $rpc bdev_passthru_create -b "$malloc" -p Passthru0
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "2" ]

        $rpc bdev_passthru_delete Passthru0
        $rpc bdev_malloc_delete $malloc
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]
}
```

Regular rpc.py:
```
real    0m1.477s
user    0m1.289s
sys     0m0.139s
```

rpc_cmd():
```
real    0m0.085s
user    0m0.025s
sys     0m0.006s
```

autotest_common.sh will now spawn an rpc.py daemon if
it's not running yet, and it will offer rpc_cmd() function
to quickly send RPC commands. If the command is invalid or
SPDK returns with error, the bash function will return
a non-zero code and may trigger ERR trap just like a regular
rpc.py instance.

Pipes have major advantage over e.g. unix domain sockets - the pipes
will be automatically closed once the owner process exits.
This means we can create a named pipe in autotest_common.sh,
open it, then start rpc.py in background and never worry
about it again - it will be closed automatically once the
test exits. It doesn't even matter if the test is executed
manually in isolation, or as a part of the entire autotest.

(check_so_deps.sh needs to be modified not to wait for *all*
background processes to finish, but just the ones it started)

Change-Id: If0ded961b7fef3af3837b44532300dee8b5b4663
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-11 11:35:03 +00:00
Shuhei Matsumoto
49cffc1e68 rpc: Add thread_get_io_channels RPC
Add an new JSON RPC thread_get_io_channels to retrieve IO channels
of the threads. IO channel don't have its own name and so get the
name of the corresponding IO device instead.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id8fbb328ffba09e1eace35994cdbf36b24832b82
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/887
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-10 09:13:02 +00:00
Shuhei Matsumoto
0b0f0c5415 rpc: Add thread_get_pollers RPC
Add an new JSON RPC thread_get_pollers to retrieve pollers of all
the threads. By adding a helper function spdk_poller_state_str(),
output poller state as string to improve readability. Most of the
code of thread_get_stats and thread_get_pollers are common and so
unify these two RPCs as possible as we can.

Sample output of thread_get_stats RPC in doc/jsonrpc.md was wrong
because thread_get_stats doesn't output ticks. Fix this together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I003ffe569d3c0651ae65c5858eff8287f7e9031d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-06 10:29:56 +00:00
Maciej Szczepaniak
12642633bf bdev/ftl: Add ftl_path parameter to rpc parsing
Signed-off-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Change-Id: I51cc68536f162b0d920d135ba491b1657e699618
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/880
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:28:21 +00:00
Shuhei Matsumoto
95b35daa5f rpc: Add thread_set_cpumask RPC
Add an new JSON RPC thread_set_cpumask to control CPU affinity of
the thread dynamically from outside of SPDK application.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6b6b794405f612200a2c3cb27dc4fc1ad2e88bec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/501
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-04 10:03:30 +00:00
Maciej Wawryk
23f78f22cf scripts/perf: extract SKT section to new file
PCM cpu measure give as result 1306 columns, we want check only ones which contains
SKTdata and SKTtraffic values.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I52b6781a7ff99e0d894f7f1c2b25b655fa274eb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/946
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-02 10:33:48 +00:00
Artur Paszkiewicz
339fb7cd15 genconfig.py: allow numbers in config option names
To make it possible to add CONFIG_RAID5.

Change-Id: I1712bde5bae552811343fb1865f1224d3adb1c7e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/854
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-20 14:40:19 +00:00
Maciej Wawryk
f7d520e864 scripts/vagrant: Fix missing line break
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iaace103027e4261e01b5e13233afc4094abf2386
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/939
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-19 09:47:56 +00:00
Maciej Wawryk
af694c3de8 scripts/pkgdep.sh: Add dependencies for Centos8
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I5921d5f7f70f2df634ccd008eb27a02496a47bee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/603
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
e5a5eaa660 scripts/pkgdep.sh: Add flag for building docs dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: If2c64eaa688fdc067479a357cbfd407570937419
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
fe5954c67f scripts/pkgdep.sh: Add flag for RDMA dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id38a3c5066c2c71d681d61f7ea8be748ddadaaf0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
2fc1ed76af scripts/pkgdep.sh: Add flag for FUSE and NVMe-CUSE dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I2cfdd33b7d0e655d80e28958aa0f12c465592cb7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
f035937dd5 scripts/pkgdep.sh: Add flag for pmem dependencies
Add flag for pmem dependencies needed by reduce and pmdk components.
Also add missing libpmem in Ubuntu section.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I596a1e6660f8b6f92bdab81046fa3533ae78fca3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
c79a950fe4 scripts/pkgdep.sh: Fix NASM dependencies Ubuntu
Add information about missing dependencies for Ubuntu versions smaller than 19.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I776d93e156c07c618df9cb5eeb99c0ca8f78e388
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/598
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
0c50f8311e scripts/pkgdep.sh: Fix libfuse3-dev dependencies for Ubuntu
Add information about missing dependencies for Ubuntu versions smaller than 19.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id48b462be35a761fd9fad4bdc48c1757cf5a7580
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
9ebbf3acac scripts/pkgdep.sh: Add flag for developer tools installation
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8876fd7c60198c597d931c5f498644c415c2a868
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
paul luse
f070b78cd1 module/crypto: add RPC and plumbing to support QAT AES_XTS cipher
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iadd03f8da305e97e3f1d57ef4aeaece400e8d959
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 08:05:42 +00:00
paul luse
e753402aa1 lib/accel: add RPC to select which accel module should be used
Startup RPC only to avoid issues trying to switch while in use.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7f1801905de0927f3610c4065182dd8fa88cdfdd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/660
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-18 08:05:34 +00:00
paul luse
aa7a13afc7 global: rename copy to accel
The copy engine library, modules and public APIs have been renamed.
Use of the word `copy` has been replaced with the word `accel`
short for accelerator in preparation for adding new capabilities
in the future. Additionally, APIs for what was previously called
the `memcpy` engine have been renamed to identify the engine as a
software accelerator.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia607aa718416146fbba1e6792b8de0f66bd8a5de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-18 08:05:34 +00:00
Karol Latecki
93be26a51d Fix Markdown MD022 linter warnings - headers blank lines
MD022 Headers should be surrounded by blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I768324b00fc684c254aff6a85b93d9aed7a0cee5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Maciej Wawryk
344c318464 scripts/vagrant: Create prebuild box and allow using spdk boxes
Build local vagrant box and add it to local vagrant box list.
Check if spdk box exist and use it instead of generic boxes.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib3b87c9783354680657f96b5b5adea1706712c5e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/848
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
dd9a88cc7f scripts/vagrant: Unified distribution names in Vagrantfile and create_vbox.sh
Remove unnecessary case statement in Vagrantfile

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6aa5d0650d268851ee5af73ffc2aa54d9396fec5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/834
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
22310f6176 scripts/vagrant: Make the Box as Small as possible
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I23b92fb6ef7d1169712f81d6d842db3981b8f764
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/677
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
58e6ef44f8 scripts/vagrant: add option to build box for spdk
New option --package-box will install all dependencies needed to build SPDK

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iedc21812b754692a5eba87127186e11ec44d805d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-17 10:06:09 +00:00
Seth Howell
b4aa3074bc setup.sh: make it easier to override with igb_uio
When overriding with igb_uio you used to have to manually probe uio
before calling setup.sh. I feel this is a common enough use case that we
should check it in setup.sh directly.

Change-Id: I8a31f90bab6960eade468816ffe960820f83d2fe
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-13 09:51:54 +00:00
Seth Howell
615b6849b4 setup.sh: fix DRIVER_OVERRIDE behavior.
This wasn't done properly for passing the whole path to a .ko file
previously.

Change-Id: Iabfcda95537003a68f952e4e8e6bcd141d2936f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/793
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-13 09:51:54 +00:00
Maciej Wawryk
db65d58359 Add pcm measurement to nvmf tests
PCM - Processor Counter Monitor is an application to monitor performance
and energy metrics of Intel® Core™, Xeon®, Atom™ and Xeon Phi™ processors.

Using this measure tool will help us to capture memory bandwidth consumed
on Intel processors.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie580d2aad6ddd98c5256df65f65e478dd78317e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-11 08:37:42 +00:00