numam-spdk/scripts
Darek Stojaczyk 8b98cdb64a scripts/rpc.py: add daemon mode
Add rpc_cmd() bash command that sends rpc command to an
rpc.py instance permanently running in background.
This makes sending RPC commands even 17 times faster.

We make use of bash coprocesses - a builtin bash feature
that allow starting background processes with stdin and
stdout connected to pipes. rpc.py will block trying to
read stdin, effectively being always "ready" to read
an RPC command.

The background rpc.py is started with a new --server flag
that's described as:

> Start listening on stdin, parse each line as a regular
> rpc.py execution and create a separate connection for each command.
> Each command's output ends with either **STATUS=0 if the
> command succeeded or **STATUS=1 if it failed.
> --server is meant to be used in conjunction with bash
> coproc, where stdin and stdout are named pipes and can be
> used as a faster way to send RPC commands.

As a part of this patch I'm attaching a sample test
that runs the following rpc commands first with the regular
rpc.py, then the new rpc_cmd() function.

```
time {
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]

        malloc=$($rpc bdev_malloc_create 8 512)
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "1" ]

        $rpc bdev_passthru_create -b "$malloc" -p Passthru0
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "2" ]

        $rpc bdev_passthru_delete Passthru0
        $rpc bdev_malloc_delete $malloc
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]
}
```

Regular rpc.py:
```
real    0m1.477s
user    0m1.289s
sys     0m0.139s
```

rpc_cmd():
```
real    0m0.085s
user    0m0.025s
sys     0m0.006s
```

autotest_common.sh will now spawn an rpc.py daemon if
it's not running yet, and it will offer rpc_cmd() function
to quickly send RPC commands. If the command is invalid or
SPDK returns with error, the bash function will return
a non-zero code and may trigger ERR trap just like a regular
rpc.py instance.

Pipes have major advantage over e.g. unix domain sockets - the pipes
will be automatically closed once the owner process exits.
This means we can create a named pipe in autotest_common.sh,
open it, then start rpc.py in background and never worry
about it again - it will be closed automatically once the
test exits. It doesn't even matter if the test is executed
manually in isolation, or as a part of the entire autotest.

(check_so_deps.sh needs to be modified not to wait for *all*
background processes to finish, but just the ones it started)

Change-Id: If0ded961b7fef3af3837b44532300dee8b5b4663
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-11 11:35:03 +00:00
..
ceph test: Shellcheck - correct rule: Consider using { cmd1; cmd2; } 2019-11-22 13:51:15 +00:00
perf scripts/perf: extract SKT section to new file 2020-03-02 10:33:48 +00:00
rpc rpc: Add thread_get_io_channels RPC 2020-03-10 09:13:02 +00:00
spdkcli module/raid: make raid_level an enum instead of plain integer 2019-11-01 14:34:32 +00:00
vagrant scripts/vagrant: Fix missing line break 2020-02-19 09:47:56 +00:00
check_format.sh test: shellcheck - ignore SC2010 2020-01-17 14:07:39 +00:00
common.sh test: Shellcheck - apply rule SC2155 2019-11-12 18:14:59 +00:00
config_converter.py global: rename copy to accel 2020-02-18 08:05:34 +00:00
detect_cc.sh make: override scan-build for unit tests. 2019-09-11 16:30:38 +00:00
dpdk_mem_info.py scripts: add a parser for the dpdk memory stats. 2020-01-24 05:46:43 +00:00
eofnl scripts/check_format: check trailing whitespace 2017-10-24 17:35:23 -04:00
fio.py scripts/fio.py: add argparse for command line parameters 2019-06-03 04:34:10 +00:00
gdb_macros.py scripts: Use range instead of xrange 2019-02-13 22:54:15 +00:00
gen_ftl.sh lib/ftl: Functional tests 2020-01-15 12:24:44 +00:00
gen_nvme.sh rpc: Rename construct_nvme_bdev to bdev_nvme_attach_controller 2019-09-18 21:53:25 +00:00
gen_rbd.sh test/bdev: add Ceph RBD bdev to bdev tests 2018-02-26 11:12:35 -05:00
genconfig.py genconfig.py: allow numbers in config option names 2020-02-20 14:40:19 +00:00
histogram.py histograms: add script for parsing histogram data 2019-01-24 14:14:32 +00:00
iostat.py script: add iostat.py to check bdev iostat 2020-01-22 13:51:41 +00:00
pkgdep.sh scripts/pkgdep.sh: Add dependencies for Centos8 2020-02-18 14:42:16 +00:00
posix.txt build: Automatically detect use of POSIX includes 2017-05-08 13:20:37 -07:00
prep_benchmarks.sh test: Shellcheck - correct rule: $/${} is unnecessary on arithmetic variables. 2019-11-07 23:04:10 +00:00
qat_setup.sh test: Shellcheck - correct rule: Double quote array 2019-11-27 07:08:57 +00:00
rpc_http_proxy.py scripts, test: fix python overindent issues 2019-04-04 03:59:12 +00:00
rpc.py scripts/rpc.py: add daemon mode 2020-03-11 11:35:03 +00:00
setup.sh setup.sh: make it easier to override with igb_uio 2020-02-13 09:51:54 +00:00
spdkcli.py spdkcli: fix pylint warnings 2019-10-22 17:23:00 +00:00