Commit Graph

112 Commits

Author SHA1 Message Date
Karol Latecki
03beec4466 autobuild.sh: export CC when building with custom DPDK
When buliding with build_native_dpdk() CC is not set
which causes "compiler" to be set to gcc and gcc-specific
cflags options are used.
Meson build however is not aware of this (as CC variable
is not set) and uses whatever compiler is default in the OS.
In case of doing the build of freebsd this results in
trying to do a clang build with gcc options, which of course
fails.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I72a5919410502786c5d44fe31ce231c023df9acd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11182
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-20 20:46:46 +00:00
Tomasz Zawadzki
e3efd781be dpdkbuild: use enable_drivers rather than disable_drivers
Since DPDK 21.05 -Denable_drivers option is present,
which only enables selected drivers in DPDK.
As name would suggest it is exact oposite of -Ddisable_drivers.

When building DPDK submodule all unused drivers were disabled
anyway.

Simplify this process by using the new option.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic8ac84b1d6dc88dd7c0b8c4aca391f31bfc93404
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10505
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-07 08:43:16 +00:00
Michal Berger
5673325a6f autobuild: Move DPDK patches into the repo
Fetching these patches from external forks of DPDK repo is a bit
cumbersome and hard to maintain so instead have them in one place
for easy apply.

All of the patches were fetched "as-is" from the used forks, with
a commit section trimmed to provide just the relevant info.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6bf538ea15ef87fe04d5a77944de36c797cdf284
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10360
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-02 07:59:42 +00:00
Michal Berger
c1831303e0 autobuild: Stop patching DPDK versions not tested under CI
Currently CI tests 21.08, 20.11 and the main branch. Maintaining
patches for other versions feels extraneous and adds only extra
work under autobuild.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I2dfac60fe3745b09fbe7159b170739680c94a8c5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10359
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-11-30 09:07:21 +00:00
Michal Berger
d2d55a0359 autobuild: Use patch instead of git am
It's more forgiving in case of small changes that force some fuzzing
like formatting changes. For instance, this change is currently breaking
our vs-dpdk builds:

f88b0b8922

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I8816ad59f930fa61ec081ac13e35129060f6cb86
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10135
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-11-17 16:43:45 +00:00
Michal Berger
089cbda8bf autobuild: Adjusting crypto build for latest DPDK (21.11)
Details of the changes here:
918fd2f146.

They mainly target the aesni_mb driver which was moved to ipsec_mb and
bump the minimal supported version of the ipsec to v1.0.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ica3b4fd66684751939159511845eb6ac6f7d5205
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10003
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2021-10-28 07:08:26 +00:00
Michal Berger
71a8e5658c autobuild: Consider use of clang while building DPDK
Our CI has a separate job dedicated for clang builds. Consider that
while specifying arguments to make sure they are compatible with the
actual compiler that is being used during the test.

Also, be strict as to what compilers can be used for the test -
gcc and clang only.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia1a4e9fea9a46e37c07fffe5077c4dacbe13102a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9614
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-07 09:24:16 +00:00
Michal Berger
bf988beba5 autobuild: Enable mlx5 drivers for supported DPDK versions
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I224fa7d01286a68afecaca6e4afddcf7e966fed5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9559
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-09-24 07:42:05 +00:00
Jim Harris
9ef1a55d0a build: don't put out-of-tree ocf build in build/ dir
When we are testing out-of-tree ocf builds, don't
put the resulting binary in the build directory.
An upcoming patch will enable 'make clean' to just
remove the entire build/ directory, which would
then include this ocf.a binary.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I74040fc414a5009faffaeada7fd76842b8cad38e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9170
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-08-17 08:54:30 +00:00
Karol Latecki
95a59652f9 autobuild.sh: Update DPDK 21.05 patches
93c4adda4d did not take
DPDK 21.05 release into consideration.

Change-Id: I9cc6a0fb7cb600d6ad08fc9eca1cbcd5e3c25d14
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8835
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-27 15:47:20 +00:00
Pawel Piatek
93c4adda4d autobuild: Update patches for mainline DPDK
Patch f95e331be3a1f856b816948990dd2afc67ea4020 is no longer needed,
since it was solved at upstream level.

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: Icf064b121351566988c26390f45286e7a456ca39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8698
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-07-16 00:33:37 +00:00
Michal Berger
860a61f6eb autobuild: Export EXTERNAL_MAKE_HUGEMEM if set
EXTERNAL_MAKE_HUGEMEM can be currently used from the external_code
suite.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I4359edc62ebdee91b261133877523c52175cb248
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-05-31 07:14:28 +00:00
Michal Berger
3a53213fb3 autobuild: Run scan-build under a dedicated test flag
Also, move ocf build outside the autobuild suite to make sure
scanbuild also picks up all the pieces.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5f0de407cc55b1dc153cc856f68e053a54fc0682
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6841
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-31 07:14:28 +00:00
Michal Berger
274f52b527 autobuild: Separate unit tests from autobuild
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If6ce4d3f7bc58855ba61b352be71dff404c87720
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6819
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-21 12:56:48 +00:00
Karol Latecki
7908736c22 autobuild: update patches for mainline DPDK
Patches stopped applying cleanly because of dpdk/dpdk changes:
7d5cfaa7508de0fd248b05effbf421a98317006a
4ad4b20a79052d9c8062b64eaf0170c16a333ff8
Needed to rebase custom patches.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1006f7f6ba21a3cee5b607cfc44adedb4c1d5830
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7903
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-21 08:04:49 +00:00
Karol Latecki
e1fe11875b autobuild: enable -Wno-stringop-overflow for DPDK builds only for GCC > 10
Use -Wno-stringop-overflow only for GCC > 10.
Change made to use the same condition as in
dpdkbuild/Makefile.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I6a396e250807f46720ff5dcaf21f13d802404a1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7537
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-27 08:37:12 +00:00
Karol Latecki
71b4e2a968 autobuild: build mainline DPDK with -Werror only for GCC > 5
DPDK now officially supports only GCC > 4.9.0.
Some older systems like Centos7  (gcc 4.8.5) can still
build DPDK, but sometimes fail due to -Werror enabled.
Disable this option to increase build success chance.

Fixes #1891

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0950ad01e2d8361851e9ac42c556004980e00ff4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7536
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-27 08:37:12 +00:00
Karol Latecki
c06df11513 autobuild.sh: restore patches for older DPDK versions
Commit e9ebbdaefc updated
custom patches for mainline DPDK, but did not keep their
previous versions. We still need the old patches to build
with older DPDK versions.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibe222a444bed2fdee63c8dcfcebc8402bb991fbe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7610
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-26 14:23:15 +00:00
Karol Latecki
e9ebbdaefc autobuild: Update patches for mainline DPDK
Previous patches stopped applying cleanly at DPDK
21.05-rc1, needed to update.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If276f608c8394439064e9dfe30fed6d5531805d2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-23 08:16:24 +00:00
Michal Berger
63330f68b4 autobuild: Ignore warnings comming from doxygen 1.8.20
This particular version reports plenty of false positives causing the
test to fail. Example:

nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ctrlr_alloc_io_qpair()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ctrlr_get_ns()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_read()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_readv()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_read_with_md()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_write()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_writev()\ilinebr' not found

This is potentially a regression in caused by the following issue:
https://github.com/doxygen/doxygen/issues/7948

The above seems to be resolved in latest releases, however, until
they are shipped as part of distros we support, similar mitigation is
in order.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I1e39c94f9f388de99e1106bcf15750e4b39384e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-01 11:18:53 +00:00
Karol Latecki
6b6ae3a214 autobuild.sh: enable i40 net driver for external DPDK build
RTE_NET_I40E has to be enabled, otherwise test-pmd app
compilation fails due to -Werror=unused-function.

This is caused by dpdk/app/test-pmd/cmdline.c:
    (cmdline_parse_inst_t *)&cmd_add_del_raw_flow_director,

Change-Id: I1add570732dedde082aec158dd3d09ab0051f7d3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-12-03 09:38:29 +00:00
Karol Latecki
cc5ade1bfc autobuild.sh: print git log when builiding external dpdk
Having a brief gitlog of build external DPDK makes it
easier to track what has changed in case of failures
detected in CI.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I549405b017c38bb5706d64a00022a2521d950a3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-11-16 12:22:31 +00:00
Karol Latecki
17f3add0a4 autobuild.sh: exclude custom dpdk from scanbuild path
Change-Id: Icc668defa2670561f1160a78f68e76452e0862fc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4774
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:43 +00:00
Karol Latecki
b7e4207b29 autobuild.sh: cherry pick dpdk fix for disabling qat_asym driver
Use the same fix that we already have in spdk/dpdk
repository:
c2c273d5c8

Change-Id: I136d3e90a048c039cb101a190f10dabd043875ab
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4828
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:43 +00:00
Karol Latecki
c98d7a4adf autobuild.sh: cherry pick dpdk fix for freeing pci devices
Change-Id: I57e380508da69b9c4d7d5219c81a550816966b4b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4732
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
7ba6712604 autobuild.sh: fix ipsec installation for custom dpdk
- Enable shared objects
- Lininkg to libs for meson done with -Dc_link_args
- append lib path to LD_LIBRARY_PATH in case ipsec
  is needed in later build or test steps
- Save LD_LIBRARY_PATH from autobuild.sh and pass it
  to autotest.sh. This is needed for crypto tests
  using ipsec-intel-mb library.

Change-Id: I047c6d3187ea42de8790181428d18596b195ff2e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4730
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
b7b4348233 autobuild.sh: preserve LD_LIBRARY_PATH in sudo call
Apparently LD_LIBRARY_PATH is not preserved even with
"-E" option. Pleaes see sudoers(5) manual as a reference.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I7d213abc4322eed3e01199416d5c7880965bdbe2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
d9b311e7fd autobuild: fix isa-l installation for custom dpdk
pkgconfig path must be set for meson/ninja installation
as described in:
https://doc.dpdk.org/guides/compressdevs/isal.html

Change-Id: I9dd01b576c4468c42241ede65781e5d50bd52bbd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4700
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
26811624b8 autobuild/external: preserve PATH env
sudo -E is not enough to preserve PATH, which after
sudo does not contain /usr/local/bin where "meson"
bin needed for DPDK is located.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ied3126dac66fc667a278aa3a973565eab02c1f6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4549
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-08 13:11:22 +00:00
Darek Stojaczyk
30116833fa autobuild: add SPDK_TEST_NATIVE_DPDK to test again pristine DPDK master
This was previously done in a Jenkins script. Move it
to autobuild.sh inside the repo where we can all see it.
Use new meson/ninja build system while at it.

To test, use autorun.sh with the following cfg options:
SPDK_RUN_EXTERNAL_DPDK=/tmp/spdk/dpdk/build
SPDK_TEST_NATIVE_DPDK=1

SPDK_RUN_EXTERNAL_DPDK can point to a different path,
but needs to have correct access permissions for current
user,

I had to reorder some code in autobuild.sh. Since
SPDK_RUN_EXTERNAL_DPDK needs to specified the ./configure
script can't be run without existing dpdk directory,
so the dpdk build needs to happen before the initial
./configure as well as the ocf pre-build.

Change-Id: Ibc57094806b361dc3c6acf55942f04a938e5194f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-22 11:40:25 +00:00
Tomasz Zawadzki
bc803d6bcc autobuild: disable scan-build on DPDK
There is no need to perform scan-build on DPDK,
as changes there are outside of SPDK repository.

This prepares for next change in series that
updates submodule to DPDK 20.05, with it own set
of failures to address.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iedb143c81339f0d33fec9a5cb58b48e2651b0a4f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3522
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-24 22:06:11 +00:00
Seth Howell
40d20acb72 autotest: always test against SPDK shared objects
We aren't going to run into any runtime issues when
linking against static libs, but we will when linking
against shared objects, so always test against shared
objects in autotest.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2d3d6e90f5fa4c38b3f44cf35e24b4f7e43b60ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2980
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-01 07:50:27 +00:00
Seth Howell
05d92eaff7 autobuild.sh: configure without shared libraries for unittests
This will allow us to run the env tests with the other tests.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1716ff25f088c49a010005ae883c57bf09b27bb5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2982
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-30 07:56:40 +00:00
Ben Walker
6b9b448e53 build: Output executables from the app directory to build/bin
Automatically place binaries produced from the app directory
into build/bin. This matches with the output in build/lib
that already exists.

Change-Id: I13cd2da71d2f88592e22308fe8a907bf458458b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2379
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Seth Howell
46b614d174 test/check_so_deps: pass the conffile to this test.
Apparently the environment wasn't getting passed to this test so
the configuration wasn't being preserved.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I65126e94c82d92fe33e81d2d0a6ef578aa506a34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2251
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-11 12:29:07 +00:00
Michal Berger
844c8ec383 check_format: Reformat the Bash code in compliance with shfmt
Change-Id: I93e7b9d355870b0528a0ac3382fba1a10a558d45
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1718
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-07 20:52:21 +00:00
Michal Berger
ae7300425b autobuild: Use mktemp() for spdk workspace
With issues similar to https://github.com/spdk/spdk/issues/1293 it
would be better to use temporary directories for spdk workspace
instead of depending on the hardcoded path (/tmp/spdk). This will
mitigate potential scenario where permissions of the target dir
are suddenly changed in-between autobuild runs.

Change-Id: If44a2dcce712a185287186f33e7361223dc6d451
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-07 10:41:08 +00:00
Seth Howell
3509378604 test/autobuild: verify external code can link to libs
This test helps ensure that nothing about the way we build our shared
objects changes in a way that breaks external applications or bdev
modules trying to link to SPDK.

Change-Id: I45ac36a2afc873b04eb237f4e633a9d84066737c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/435
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-04-15 22:10:08 +00:00
Seth Howell
ae1dc68f39 test/autobuild: move ocf_precompile to new location.
This is needed so that we maintain proper configuration when
I add a new test that confirms our ability to run an application
that links dynamically to our shared libraries.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I9a8796c75280a5265579dc3c4a58955e1f82d049
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1813
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-15 22:10:08 +00:00
Seth Howell
4f7f0f0986 autotest_common: move config_params to a function.
This accomplishes a couple of things:

1. Now we don't perform these checks every single time
we source autotest_common.sh (some 140 times)

2. We have some flexibility in changing parameters either
during a test or when trying to do a specific compilation
and calling this function to get the updated parameters.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ibbe2dc9113a56f651d41216e8557708824652442
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1784
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-15 07:39:00 +00:00
Karol Latecki
6d1bb438bf autobuild.sh: delete scanbuild output files before the build
"all_c_files.txt" had output always appended, not overwritten.
This caused autobuild.sh to fail if next builds had different
configuration.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I9ba52d7f5a71d79a269c962e6c38c2a7d48f294c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1652
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-10 07:07:23 +00:00
Seth Howell
9cb8c2387a autobuild: optimizations around compiling tests.
This commit combines a couple of small optimizations to
help make the unittest scan-build checks more comprehensive
while not taking extra time to compile examples when we are
doing functional tests that don't require them.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1d3af0026b4be6e799516ef52e9414790eab85e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:44:15 +00:00
Karol Latecki
0bf189a39d autobuild.sh: put artifacts to output_dir
Use output_dir path provided by autotest_common.sh
to store all build artifacts in one place.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ie5d3bb4dcb3fc623b6719859ce25235e1aae6bcb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1135
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-31 08:08:16 +00:00
Seth Howell
e1f6fd0f09 Revert "test/build: add test directory to scanbuild verification"
This reverts commit 606ba0c321.

Change-Id: Ibdc797e4c7e4ea9c60e070487027a069007dcf98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:56:15 +00:00
Tomasz Zawadzki
606ba0c321 test/build: add test directory to scanbuild verification
scanbuild_make verifies if all files in repository were
scanned by scanbuild. 'test' directory was not part of this
check, even though it contains C files with applications
used during tests.

This change does not affect what gets, and what does not
scanned by scanbuild. Only what we verify that was.
It will help to spot any files accidentally skipped.

This patch adds 'test' directory to the check.
Along with skipping directories when not compiled with:
- Functional tests disabled - don't check 'test', but still check 'test/unit'
- Unit tests disabled - don't check 'test/unit'

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I01e16de83827022c412a57df379280bfed671797
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1212
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
79c28aa698 test/build: disable compiling unused components in tests
This patch decreases the compile time when particular test
category is not executed by tests.

Added skipping the 'examples' dir during verification
of scanbuild scanned files. Only when examples were not
compiled.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I41e7b9d18913fd02e2b6cbe44f933ab2e0371d28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
0b7dddc793 autobuild: change binary for dependency check
Previously identify tool was always compiled into
SPDK, but now there is a flag to disable compilation of
examples. Next patch in series will disable compilation of
examples on some jobs to speed up the test time.

Switch to binary that is always compiled, like spdk_tgt.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I84be821fbeaff67a1249b427ee5c67d3a4813d8c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1195
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-10 09:12:21 +00:00
Darek Stojaczyk
15a3067b78 autobuild: remove a lot of unnecessary scanbuild output
The loop was generating huge xtrace output that was
hardly helpful.

Change-Id: Ida683c01ec3a05674965bec972cc9f762a1f8fa0
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1053
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-06 10:28:15 +00:00
Darek Stojaczyk
242a899573 autobuild: don't silence errtrap with leading !
Apparently some shellcheck versions on our CI complain about it
and suggest to use && exit 1 instead, we'll try to use || true
instead.

Change-Id: Idb505770a80929fbb9957f4ba039e009aae98f02
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/835
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-12 14:32:39 +00:00
Darek Stojaczyk
7a5b2019e8 autobuild.sh: don't rely on make output in listing not-built files
To get the list of not-built files, we used to concatenate
the list of files expected to be built and a specific chunk of the
`make` output, then we cut out any repeated lines. Selecting that
specific chunk of make output used to rely on 'Leaving directory'
string from make, which apparently doesn't appear under certain
circumstances, causing the build to fail with false-positives.

Rewrite this code not to rely (this much) on make output:
  1. get the list of files expected to be built
  2. get the list of all built files
  3. check if any file from #1 is not in #2

^ comm utility does exactly that

Change-Id: I6c6c1267c738f23d6804fe5c108cd80141329d10
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/550
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-12 12:07:52 +00:00