Using another flag for the conditional will let us
run release build test in selected job configurations
for per-patch testing on CI side, not just for nightly
tests.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ib9de1c98bf6d7380670f22321f2df7784f2517de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Using "get_config_params" and appending --disable-
debug results in contradictory options on the same
./configure line. Sed out "--enable-debug" instead.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I465f38b4c63ab7a6ae824c7932b2548fdb362ff3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
reduce library uses unlink, but the unit tests need to
override it in a specific way.
But linking unit tests with LTO requires the wrapper
definitions be in objects/libraries listed *after*
the object/library that refers to it. So we need to
make the unlink wrapper somewhat generic. We do this
by exporting a string and callback function that the
user can set to enable a user-defined function to be
called when unlink() is called with a specific file
name.
Also revert 3ef6d06 as part of this patch, since we
no longer require the workaround that it implemented.
Fixes issue #1357.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ee4c424ad31fe7d91d7b524ed47aedd279e5b5c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1948
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Some of the unittests require symbols that don't work well
with the lto option. For example when you build reduce_ut.c
with the lto option, the __wrap_unlink symbol cannot be found.
Also, while we are here, add a check to configure for unit tests
built with lto.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iaf76cd050565ce2ff13b4230d7a3b735fd5384a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2055
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This accomplishes a couple of things:
1. Now we don't perform these checks every single time
we source autotest_common.sh (some 140 times)
2. We have some flexibility in changing parameters either
during a test or when trying to do a specific compilation
and calling this function to get the updated parameters.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ibbe2dc9113a56f651d41216e8557708824652442
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1784
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Starting with patch 873c874, it was not possible to configure SPDK
with LTO enabled or setting a cross compilation prefix.
detect_cc.sh was moved before the configuration options
were read from command line.
This patch fixes this behavior by reading passed options
before invoking detect_cc.sh.
Next loop that reads all options has empty handling for those,
as otherwise "Unrecognized option" condition would trigger.
Added LTO to NIGHTLY release build tests.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I21802e14c0565cb7f323bcbd805967a3c4781b46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1714
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Remove SPDK_BUILD_PACKAGE flag, since it was not set
on any of the CI jobs. The default value "0" was always used.
Only indicator whether second part of the script executed,
was handled by RUN_NIGHTLY flag.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1c5a660ed86bb520972c630118f8499966bdb6d5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Removed part of the script was only creating tarball from
sources and built from them.
Since SPDK is not distributed in this way usually,
this test does not add much value.
On the other hand this was the only palce where
release build (--disable-debug) was tested.
This patch replaces original tarballing,
with release build test as that was the only
part of the script adding value.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6a054e15e6caf827653b671543c08d4a0986f353
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1606
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Shellcheck version 0.6.0 reports all instances where we cd .. or cd - as
instances of SC2103. The version we are using on the test pool must be
newer, because we don't get the same errors, but for people running
check_format locally, it can make it difficult to parse the output.
Change-Id: I09f81a83c6f37480f13c36eb622e500364a1c437
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Until now the autopackage.sh execution was tied to
RUN_NIGHTLY flag. This was useful to check for
errors on all jobs/systems during a nightly test.
By their nature nightly does not run on per-patch
basis.
This allows for issues to slip in and be noticed only
by the nightly job after merge occurred.
Adding separate SPDK_BUILD_PACKAGE flag, allows to
explicitly test packaging on single job on per-patch.
RUN_NIGHTLY was kept as trigger for autopackage.sh
to still test it on all systems by default.
Change-Id: I29925fd6256b218e24c24ebcc4974c65a9bd986c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
This reverts commit 0b9931516d.
"cd $dir && ..." in single line does switch directory for
the following commands.
The patch being reverted, broke the nightly (autopackage.sh) tests.
It was only fixing couple of instances of SC2164,
which is being excluded either way.
https://github.com/koalaman/shellcheck/wiki/SC2164
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id4f134ad98953b92fdc69cd0d0b80c02660cf7a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473220
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Don't change the directory of the script if at all possible.
Change-Id: I5e20784694cb61fac2cf148628de566361e2673e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472650
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.
Also update two scripts from spdk/test which we previously missed.
Change-Id: I429f9bc158076462b419fae597f716c329f9b7aa
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Most of our bash test scripts source autotest_common.sh
to be able to use some autotest-specific functions like
timing_enter(). The same test scripts allow specifying
custom command line parameters without actually realizing
that those parameters can be potentially picked up by
autotest_common.sh as well.
For example, if particular nvmf tests are run in "isolation"
mode by being executed with the first param set to "iso",
and there is a file named "iso" in the current dir, that
file will be sourced. This could be bad.
In this patch we stop sourcing or even processing $1 in
autotest_common.sh. Instead, the test configuration will
be sourced just once from autobuild.sh, autopackage.sh
and autotest.sh. If the user wants to run particular test
scripts manually, he should source an SPDK test configuration
by himself - manually as well. In most cases he won't even
have two, as only a few test scripts depend on SPDK_*
variables.
Note that we still have to setup the default values for
SPDK_* variables in autotest_common.sh because some of
our test scripts actually depend on them:
> if [ $SPDK_TEST_RBD -eq 1 ]; then ...
Because it lacks any type of quotes around SPDK_TEST_RBD,
it will print the following message when that variable
is unset:
> /bin/bash: line 0: [: -eq: unary operator expected
It doesn't trigger any error ($? == 0), but can be still
a bit misleading in the script output.
Change-Id: I350045d8582d66fe1ed7697d4bcbba324cb541ad
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453876
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There's not a lot of value in building from the
tarballs on every patch. The git status --porcelain
check is still good on every patch though. So
return early if it's not the nightly test - this may
cut up 90 seconds on some tests.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I351a43b802061fe7d7bc4556b3b003a9ec73833f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452810
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.
Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.
Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
In a couple of cases, we do specify --ignore-submodules
for the actual check, but when dumping the results
to the console, we omit the --ignore-submodules. Fix that.
Same for check_format.sh - don't consider submodules for
files changes that suggest a CHANGELOG.md update.
This is in preparation for adding isa-l as an SPDK
submodule. isa-l doesn't use any .gitignore files, so I
want to make sure we're just ignoring anything related to
isa-l build artifacts. We could probably remove our
local DPDK submodule patch to its .gitignore after this
patch but will leave that for another time.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id9be9ce87569004c426d02c6cd44d645f3ff859f
Reviewed-on: https://review.gerrithub.io/c/440808
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
The way autopackage builds spdk, the absence of ipsec
in the tarball will result in missing dpdk crypto options
if crypto is enabled. This patch adds the ipsec submodule
to the tarball.
Change-Id: Ia21f3e90c4753569cf2aa1887e4938740fd47113
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/424142
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
This is a somewhat temporary workaround for avoiding git status
errors related to symlinked files in the DPDK submodule. The
build pools will be fixed to not change symlinks to regular
files when copying repositories between systems, but until
then apply this fix so we can move forward with switching
to DPDK 18.05 (which now has symlinked files in its
repository).
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8a6df76fb77a6f3ee3817bd975d198ae4cc3014a
Reviewed-on: https://review.gerrithub.io/420444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is the first step in reorganizing the spdk test heirarchy.
Change-Id: I83467653af2da2a53251649092bed6902d6cb86e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/401707
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use the "HEAD^{tree}" syntax to indicate a tree-ish object instead of a
commit-ish one; this makes git archive use the current time instead of
the timestamps from the commit, avoiding issues when the build machine
timezone doesn't match the committer's timezone.
Change-Id: Ia5033b220b9b86166aa18db5b6850ac97abb5dbc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/363292
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This will allow a git clone to default to an SPDK-specific
version of the DPDK repository. Users can still override
to use a separate DPDK repository/installation with the
--with-dpdk configure script options.
While here, remove gzip option for the git-archive operations
in autopackage.sh. We need to add a git-archive for the DPDK
submodule if we are using it, and compressing at -9 adds a lot
of unnecessary time. Since we are not archiving these packages,
there is no need to compress them. Also explicitly disable
coverage and ubsan for the autopackage build, since this build
is only to test compilation and is not actually used for any
test execution.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3cf8a2ed984003a175cdece6542636ede8cb2479
Add a configure script in the root of the repository
that looks and feels like one that would be generated
by autotools. This script simply generates a CONFIG
file for you, which used to be done by hand.
Now to build SPDK you can do the following:
./configure --with-dpdk=path
make
Change-Id: I44ebb91f0cb1468b86da4c0033ac1406595d4967
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
The tarball can be recreated with 'git archive'; it is not useful to
keep it as part of the build output.
Change-Id: I585aaaac765d9a52e444bcd878d3826bb7b96a45
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Exercise both the DEBUG=y path (during autobuild) and DEBUG=n (during
autopackage).
Change-Id: Ie2f0fea42ac675cd18f49d0b3274aa2c3413b88a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
