Commit Graph

1115 Commits

Author SHA1 Message Date
Maciej Wawryk
e96d3280a0 scripts/pkgdep: Add ninja and meson packages to minimal install option
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I36cba747313349f9dbe9d98e6e6297e1a449af3f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2517
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-05-20 14:14:29 +00:00
Michal Berger
3425fc1fb8 check_format: Don't attempt to review perms of the symlinks
Change-Id: I3cd20ad9f1cadb5ca5e6de78d9b44d5df95f51d8
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2502
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-19 09:43:37 +00:00
Dayu Liu
5cdab5e2e6 doc: fix some typos and align indent when setup.sh --help
Signed-off-by: Dayu Liu <liu.dayu@zte.com.cn>
Change-Id: Id0a2a3a577adfd5026b662d97d74c9ec6aebeb8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2396
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-18 10:14:56 +00:00
Darek Stojaczyk
e5f021c10e scripts/check_format: auto-detect shfmt binary
A newer version of shfmt was released, but the script still
looked explicitly for the older one.

Make it execute the first binary '^shfmt.*' in PATH whose --version
returns at least v3.1.0.

Change-Id: If621abfa4d995a9dc985e6bab8c33aa36c327fae
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2428
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-18 10:14:52 +00:00
Tomasz Zawadzki
2e55b97d96 setup.sh: execute checking ulimit as a user
When using vfio a warning for memlock limit is printed.
TARGET_USER is one set by caller when executing the script,
or set to user that called the script (even when using sudo).

Yet ulimit was printed for the one executing the script,
most likely root.

This patch verifies ulimit for a particular TARGET_USER.
Along with more explicit information which user the
information pertains to.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I20ad62109bc316295208e21f959ecfc263307e1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1964
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-13 07:46:51 +00:00
Maciej Szwed
b0bac20ad6 rpc: Add possibility to load module with additional RPC commands
This patch adds '--plugin <module_name>' parameter
to rpc.py script which is used to provide module
name with additional RPC commands that should be
included in rpc.py script. User must export path
to rpc module with PYTHONPATH before calling rpc.py
with this new parameter.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I971429a33cf4ea943d2b7f3abd5a6cb82398d2a4

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2263
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-13 07:46:17 +00:00
Michal Berger
871214e2c1 rpc.py: Allow loading json config from different sources
This is relevant in context of rpc_cmd() which was introduced with the
following commit:

     "scripts/rpc.py: add daemon mode" 8b98cdb64a

When rpc.py runs in a server mode, its stdin is already attached to a
pipe connected to its parent Bash process. In such a setup, it's not
possible to use some of the rpc methods, e.g. load_config().

To make use of said methods possible, allow them to load the config
from different sources - a regular file or a string.

Change-Id: I6fa7d13fa1957b6449ce5d5c5a810bd58d0a5703
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2106
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-05-13 07:46:04 +00:00
Michal Berger
15f52aec2f setup.sh: Watch for NUMA_NO_NODE value
The -1 is valid value indicating that the node id for given device
has not been specified. Detect it and map it to "unknown".

Change-Id: Id97d9046be0e4d5ff257bca66ae3e73e906253fe
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2229
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
5853749ba0 setup.sh: Check status of all numa nodes
Use * instead of ? to match all the potential nodes not only those
with a single digit id. The actual node limit is dependent on
kernel's CONFIG_NODES_SHIFT hence in theory, the actual number of
numa nodes may be far greater than 10.

Change-Id: I1d587735b433ed23594725b9d77a44f7cc729494
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2228
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
c8c4f71efe gen_nvme: Be FreeBSD friendly
Since sysfs is not used by the FreeBSD the following error can be
seen each time gen_nvme.sh is run:

  grep: /sys/bus/pci/devices/$bdf/uevent: No such file or directory

Avoid that by looking up the pci address in a way specific to
FreeBSD.

Change-Id: If81e71cece52c2f27dcf68f7a7eba3dd7d8ce10f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2227
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
d51345c0b9 setup.sh: Add basic function for handling status option on FreeBSD
Change-Id: I64bcc9f06be7e5cca18195bab58f87a4ba74c903
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2226
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
18c0288730 setup.sh: exit from the script if not supported platform is detected
Also, store kernel name in the separate var and use it throughout the
entire script.

Change-Id: Iaa1c4c4aa52fb4aa708fb476478a81e9c6067b58
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2340
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-12 08:12:09 +00:00
Michal Berger
5a436f0c61 scripts/common: Cache vendor|device ids for reverse lookup
Change-Id: I69959547d1c71cf3b65f8eb0b786197650a6ece0
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2225
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Karol Latecki
4cfc2e2641 scripts/pkgdep: install yum-utils for CentOS
This will probably be not installed by default in CentOS
installations. If we're going to use "config-manager" then
it requires yum-utils.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I7d310e376430b065742b671d983d11dd87bb5fac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2221
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-11 12:35:00 +00:00
Karol Latecki
4d7936227a scripts/pkgdep: add missing "assume yes" to install command
Change-Id: I2a1a22e9d665a5b806b42db504062780f4f5ce53
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2220
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-11 12:35:00 +00:00
Michal Berger
d596ba87b2 check_format: Introduce shfmt tooling for .sh style enforcement
Add new dev tool for enforcing proper formatting of the Bash code
across the entire repo. This is done in order of defining a common
set of good practices to follow when writing .sh|Bash code.

As powerful as shfmt may be, it allows only for some specific rules
to be enforced, hence it still needs to work side by side with
shellcheck syntax-wise. If it comes to style, following rules are
being enforced:

  * indent_style = tab - Lines must be indented with tabs. The exception
			 from this rule is the use of heredocs with
			 <<BASH redirect operator. Spaces can be used to
			 format the line only if it's already preceded
			 with a tab.

  * binary_next_line = true - Lines can start with logical operators. E.g:

			      if [[ -v foo ]] \
			      	&& [[ -v bar ]]; then
				 ...
			      fi
  * switch_case_indent = true - case|esac patterns are indented with tabs.

  * space_redirects = true - redirect operators are followed with a space.
			     E.g: > foo over >foo.

In addition, shfmt will enforce its own Bash-style for different parts
of the code as well. Examples and more details can be found here:

  https://github.com/mvdan/sh

Change-Id: I6e5c8d79e6dba9c6471010f3d0f563dd34e62fd6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1418
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-07 20:52:21 +00:00
Michal Berger
844c8ec383 check_format: Reformat the Bash code in compliance with shfmt
Change-Id: I93e7b9d355870b0528a0ac3382fba1a10a558d45
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1718
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-07 20:52:21 +00:00
paul luse
3044bd27d0 lib/accel: remove RPC for setting the module
This was added before the usage of having a SW engine and 2 HW
engines was fully thought out. The current rules are:

* if no HW engine specific enable RPC is sent, use SW
* if a HW engine specific enable RPC is sent, use it
* If a 2nd HW engine specific enable RPC is sent, ignore

In this scheme there's no need for an RPC that lets the user
choose which engine to use because they already do so when
they enable an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I006ffb3b417f1e93bb061b29535d157ba66f03b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2033
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-05-07 10:41:44 +00:00
Michal Berger
ae7300425b autobuild: Use mktemp() for spdk workspace
With issues similar to https://github.com/spdk/spdk/issues/1293 it
would be better to use temporary directories for spdk workspace
instead of depending on the hardcoded path (/tmp/spdk). This will
mitigate potential scenario where permissions of the target dir
are suddenly changed in-between autobuild runs.

Change-Id: If44a2dcce712a185287186f33e7361223dc6d451
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-07 10:41:08 +00:00
Tomasz Zawadzki
519d720902 scripts/rpc: do not list positional arguments for the subparser
Main subparser for rpc lists all positional arguments. Like so:
positional arguments:
  {framework_start_init,start_subsystem_init,framework_wait_init,....

This again is duplicated with better help message.

This patch just removes this list, leaving the most detailed one.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5e6b9d0dd6ccf68bfe1cd5959ed509032738c037
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2126
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-05-07 08:26:54 +00:00
Tomasz Zawadzki
0d105d4f1f scripts/rpc: do not list commands for the main parser
The main parser will show usaged like so:
usage: rpc.py [-h] [-s SERVER_ADDR] [-p PORT] [-t TIMEOUT] [-v]
              [--verbose {DEBUG,INFO,ERROR}] [--dry_run] [--server]
              {framework_start_init,start_subsystem_init,............

The above contains very long list of all RPCs.
This is duplicates help from the subparsers, which contain
more descriptive help message.

This patch replaces the usage with:
usage: rpc.py [options]

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibf400987103f5571f79c57d8a9588d97f463d773
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2125
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-05-07 08:26:54 +00:00
Tomasz Zawadzki
17c2f4ee8f script/rpc: do not send rpc when no arguments provided
Any time script is called an RPC is called and socket
connection is attempted.

This should not be required when no arguments were provided,
such as calling help to explore possible options.

Example of the error:

[tzawadzk@PowerEdgeFour spdk]$ sudo ./scripts/rpc.py
Traceback (most recent call last):
  File "/home/tzawadzk/spdk/scripts/rpc/client.py", line 40, in __init__
    self.sock.connect(addr)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./scripts/rpc.py", line 2438, in <module>
    args.client = rpc.client.JSONRPCClient(args.server_addr, args.port, args.timeout, log_level=getattr(logging, args.verbose.upper()))
  File "/home/tzawadzk/spdk/scripts/rpc/client.py", line 56, in __init__
    "Error details: %s" % (addr, ex))
rpc.client.JSONRPCException: Error while connecting to /var/tmp/spdk.sock
Error details: [Errno 111] Connection refused

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3734e1603fa7151fd53b99b8af4ea79bf82fe0f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2124
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-05-07 08:26:54 +00:00
Michal Berger
36e573fc6b scripts/common: Introduce cache for the pci devices
Expose a cache of pci devices in form of an assoc array that could be
looked up during the runtime of a script like setup.sh.

In case of setup.sh, caching speeds up execution quite visibly:

config run, no caching:
real    0m4.488s
user    0m1.440s
sys     0m1.260s

config run, caching in use:
real    0m2.876s
user    0m0.365s
sys     0m0.420s

Note that for initial config runs, binding controllers to proper
drivers is the actual bottleneck.

status run, no caching:
real    0m1.877s
user    0m1.252s
sys     0m0.984s

status run, caching in use:
real    0m0.371s
user    0m0.242s
sys     0m0.204s

reset run, no caching:
real    0m2.559s
user    0m1.409s
sys     0m1.322s

reset run, caching in use:
real    0m0.960s
user    0m0.432s
sys     0m0.419s

Additionally, in case common tools, e.g. lspci, are missing, fallback to
sysfs to pick all needed devices from the pci bus. Targeted for Linux
systems only.

Change-Id: Ib69ef724b9f09eca0cbb9b88f1c363edc1efd5dc
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1845
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-07 08:23:52 +00:00
Maciej Szwed
2bd001620f scripts: Add SSL certificate option in rpc_http_proxy
This is required for SSL verification when running on https.
Added argparse to handle multiple optional arguments.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ib8731bafaaf38264ed86ed84ca7eeca5ff4935b3

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2122
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-05 08:04:00 +00:00
Vitaliy Mysak
20bc88d284 scripts/vagrant: fix exporting array in bash script
In create_vhost_vm we export SDPK_VAGRANT_DISTRO
variable to vagrant subprocess that is called
during the script.
But if SPDK_VAGRANT_DISTRO is an array variable,
then Vagrant will not see it
and always default to ubuntu16.
This small patch fixes that.

Change-Id: I45e9ed4f847e4fa10b90650a613f9cfe17ee38e3
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1676
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-28 13:47:46 +00:00
Ziye Yang
94345a0a1a nvme: Add the priority field in struct spdk_nvme_transport_id
Purpose: To set the priority of the NVMe-oF connection especially
for TCP connection.

For example, the previous example can be:

trtype:TCP adrfam:IPv4 traddr:10.67.110.181 trsvcid:4420

With the change, it could be:

trtype:TCP adrfam:IPv4 traddr:10.67.110.181 trsvcid:4420 priority:2

The priority is optional. We try to change
spdk_nvme_transport_id but not in spdk_nvme_ctrlr_opts since
the opts in spdk_nvme_ctrlr_opts will reflect in every nvme ctrlr,
this is short of flexibility.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: I1ba364c714a95f2dbeab2b3fcc832b0222b48a15
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1875
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-24 15:53:34 +00:00
paul luse
42eb233e48 scripts/rpc: addd RPC support to enable idxd
The new RPC also includes and option for which
idxd config is to be used. Config options will be
explained in detail in a later patch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I1322bab4d6ca9cc45ec3a56d7267dc7dc09dbea3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1726
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-23 15:48:32 +00:00
Liang Yan
6a29c6a906 bdev/rbd: add ceph rbd resize function.
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Change-Id: Id7f24f1f5e0789ee6d4c07ae7f59cc2d98983e98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1766
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-23 07:28:10 +00:00
Maciej Wawryk
6ec892634e scripts/perf: Enable percentile 999 measurements in fio config
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I32c534196108c4795bcd6cd8392e3a1b0a5dd6e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1842
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-04-17 07:31:55 +00:00
paul luse
3429f97a89 scripts/setup: add support for IDXD (aka DSA)
Add bin/unbind support between the kernel idxd driver
and VFIO. UIO not tested but tested this extensively
on a system with valid idxd kernel driver and VFIO.

Future patches in this series add the actual idxd
low level library and bdev accel plug-in module. See
those patches for more details on idxd.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I09dd00b1aedcf88577335bde79d31cfd50ac4cde
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-16 08:10:33 +00:00
Maciej Wawryk
36b7098a84 scripts/pkgdep: Fix failing symlink creation during python instalation
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ida6bfaf9ed21c5349a3107e033391f8d30a9ff93
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-04-16 08:08:40 +00:00
Karol Latecki
485d615d7e scripts/vagrant: disable default synced folder
Disable default synced folder. We have our own set of
directories to synchronize so this is not needed.

Change-Id: Ie52e38af94fc2c403e3d3f095071833b9df09c9c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1158
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:08:02 +00:00
Karol Latecki
3a06abc9d9 scripts/vagrant: enable hugepages backing for VM memory
Change-Id: I66c1c965d9de8b4e9625c1f1850cbb28d11138ad
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1684
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-04-16 08:08:02 +00:00
Karol Latecki
7fe1b36a36 scripts/vagrant: select tool for synced folders
In order to have all the boxes fully functional we
need to select the proper tool for synced folders.

FreeBSD boxes fail to set proxy properly via vagrant-
proxyconf plugin, so they fail to install rsync and sshfs
clients. NFS is pre-installed so we can use it.

Other boxes, like generic/fedora*, have issues using NFS
so we need to use sshfs/rsync instead.

Change-Id: Ic191b5a80dbd43479943c1739971abdff39baf41
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1581
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:08:02 +00:00
Maciej Wawryk
451c9e7b69 scripts/perf: Add frequency-info after changing cpu frequency
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Icf45fbdff547fcf45c361b1d9c95b352756ad126
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:07:14 +00:00
Pawel Kaminski
fbedb897bd test/openstack: Change http to https for cirros server.
Extend build timeout, enable image volume cache and
change max spdk queue depth on local.conf.

Change-Id: I1dcacd8fbfbfabcba1d017d950dd09ce8ee68076
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-15 07:37:25 +00:00
Jin Yu
2eabc71518 vhost: add packed ring support
Update vhost blk contruct rpc, make it enable to
support packed ring feature.

Change-Id: Ia1f75e72e8441e8d82fad89073e4875f89e5b9cd
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1567
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-15 03:55:59 +00:00
Maciej Wawryk
282b1505e5 scripts/perf: Fix cpu_frequency parametr passing
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9617cd92d20b07c32761fb67ce6a1b30e0985831
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1697
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-09 08:08:38 +00:00
Maciej Wawryk
a61af6cfc9 scripts/perf: replace dots by floor sign in percentile variable
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I34e912dd207f16b1926edb37df41128a0524bc4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1711
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-09 08:08:17 +00:00
Maciej Wawryk
7c1e69b3fa scripts/nvmf: Fix subsystem divide and format error
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I874c3a28c453737d0ba6ab42ec0d0a9fbc4719f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:43:16 +00:00
Ben Walker
649367850a make: Allow CC and CXX to contain spaces
On some systems, CC is set to a value with arguments already included.
Make sure not to lose those.

Change-Id: I93b354dc21f045f3cacff1f0d1214442ea893aa5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-07 08:37:57 +00:00
Maciej Wawryk
a2eeec65f8 scripts/vagrant: synchronize artifacts from vm to host
Sync "output" directory (containing all build results,
like doc, coverage, test completions, etc.)
from guest to host.

Change-Id: I9a57a72eb610aacde011fc39bb8d39c74486cef1
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1044
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
2a75be1a56 scripts/vagrant: cleanup packaging routine
Add "halt" command before packaging, it's needed for this
process. Destroy the VM once it's packaged as we have no
further use for it.

Move provision-related commands to update.sh.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ib8041df023b80b4cd5fbad734481eb6bd4ffc21c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1154
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
5277a5324e scripts/vagrant: change vagrant box sources
Some time ago I created a patch which unified all boxes
URLs to generic/* which come from a single author/company
and seemed to be quite universal.
Unfortunately some of the boxes are not that generic as
the name suggest, so I am reverting my change and will
use boxes from vaious sources.

Ubuntu 1804 - had heavy DNS issues due to hardcoded
addresses in resolved.conf. Very hard if not impossible
to fix if working with synced folders like we do.

Centos 7 and 8 - sshfs not available by default. Also
had some other minor quirks. Official boxes available
so let's use them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I514d3b92c6ac5844b1929fdadfd35d6079933247
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1622
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
a34c17027b scripts/vagrant: fix permissions for backend files
Owner and group were not set properly on Ubuntu systems,
as they use different names for that.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ie7fc49516072ce4bcd506c4c39bbd26786004c49
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1580
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
3d3e8096b7 scripts/vagrant: replace hardcoded ocssd metadata file paths
These can't be hardcoded to a single location. It makes
running multiple VMs with OCSSD emulation impossible.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4ce0412fc9fc4542670a9e70c08219003e41828b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1488
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
53dbdf6dce scripts/vagrant: update help message
Just moving stuff around and updating some information.
I just felt like it's needed.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ia41f99603fb60111924cd5a3daf3c44091245c89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1157
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
e16128cece scripts/vagrant: remove vhost test configuration steps
We are not using VMs to run Vhost tests (at least at
the moment) and these steps are outdated. These steps
will need a re-write if we decide to put them back in,
so we can remove them for now.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I61d09bdc481814d8bcf6fe86ed4ccd84b36ad8d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1155
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
2f96a090de scripts/vagrant: remove Fedora 29
Remove Fedora 29 from the OS list. It reached EOL
some time ago and is not supported.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ie1bd704a39ab4c83d5e652741d21f6f0ac260540
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1156
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Maciej Wawryk
051d3fe016 test/vm_setup: Fix permision error in Centos7
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic704e9ae97bb68d96401818fb52940e86eaa36a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1409
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-02 14:51:24 +00:00