Commit Graph

22 Commits

Author SHA1 Message Date
Seth Howell
d5eb583584 post_process: clearly delineate the beginning os script output.
This is a companion change to
https://review.spdk.io/gerrit/c/build_pool/jenkins_build_pool/+/1810
and the two should be merged together to make sure the entire output
of the post process script is included in one file when output from
the CI.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ia6506208881f5d745ff63a356d3965426449ecc7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1811
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-17 07:21:44 +00:00
Seth Howell
237ef4c600 test/post_process.py: Don't print coverage info to separate file.
We already have the post process log readily available. Put the
info there to avoid hiding the coverage errors in an obscure second
file.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I2fe9f79403ca1accccc207155f8d0dafd4c24f8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1790
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-14 11:32:50 +00:00
Karol Latecki
ac26fec9c6 autorun_post: skip confirming executed tests
Allow to skip confirmPerPatchTests if needed.

Change-Id: I8741d80de5cac9954e3429b951a71dc065c40bb5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:36:29 +00:00
Seth Howell
bde783d460 autorun_post: fail if not all per-patch tests ran.
This will keep us honest with our testing strategy and help prevent us
from unintentionally skipping tests.

Change-Id: Ie9dda265b48814c5e8aafe0504205310b886eea9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477981
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
df254d5a25 autorun_post: add a file for tests we don't run.
Change-Id: I8beac9959d08026ee984833d0e90ec17381a66e5
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478240
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
ea781d6dbb test: report test domains for tests.
This will allow us to run test cases under multiple suites without
clogging up the logs too much, but it will also preserve information
about which test suites were run or (more importantly) not run.

Change-Id: I2434a54a0877ae36b9f84bfab8a62653ac1172f8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477367
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
4e3104d0e9 autorun_post: rewrite the aggregateCompletedTests function.
This whole function was just written poorly. Variables weren't named
correctly, little effort was made to use proper python programming
constructs, and there was a lot of redundant code in the function that
didn't do any good.

And I can say that because I wrote it.

Change-Id: I08576698d99c35c43348b61762791415f79eebf2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
9e118adeca test/autorun_post: fix ubsan reporting.
Change-Id: I7bfe785b355ec925d28d8c306395733945a7b5d0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477366
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-14 07:49:08 +00:00
Ben Walker
d705ab93e1 Check file permissions in the check_format script
Change-Id: I572b24b2f4f149333f2e4b04f6f4961de9c23373
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/427539
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-04 23:08:12 +00:00
Seth Howell
1791ccfd4f post_process: print coverage tables in a new directory
Change-Id: I2d9a53456f5ca1da5607a4f7bd47017026e8f996
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/422850
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-21 22:11:11 +00:00
Seth Howell
df531e7faf test: add output tables to autotest_post.py
These output tables will help show graphically which tests are being run
where, and which ones are being tested with ASAN and UBSAN enabled.

Change-Id: Ia11657d4468f1ed41ab13b3241893ce09e949668
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/421556
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-17 22:15:50 +00:00
Karol Latecki
d798ac57a7 autorun_post: save aggregated test completion list to file
First save the list of run / not run tests to file for
easier re-use later and then print to screen.

Change-Id: I1cb4ea057a75f08cac43853299f509d03b299709
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/405222
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-03 15:12:50 -04:00
Daniel Verkamp
be69f0ffc8 autorun_post: clean up cov_total.info files
These aren't needed after the combined coverage information is
generated.

Change-Id: I4769cdf471b2b7a6502c3466ffb317c8435f1e3d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/403748
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-16 16:29:32 -04:00
Daniel Verkamp
6cf871d49c autorun_post: collect one instance of ut_coverage
Change-Id: I10c8b17cb160d8ae70a6f1042218aa24f009ceab
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/403740
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-16 16:29:32 -04:00
Daniel Verkamp
bba169c1f4 autorun_post: refactor doc collection function
This will be useful for other directories where we only want to keep one
copy.

Change-Id: Iac3cf964936e03c1164ffd961cf8c35a24db5f31
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/403739
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-16 16:29:32 -04:00
Karol Latecki
7f3e956c73 autorun_post: use absolute paths for lcov
Ubuntu LTS 16.04 has lcov 1.12 which is bugged: when
using relative paths for -a and -o lcov parameters command the
result is "error: cannot write / read file X".
Using absolute paths solves this issue.

Change-Id: If73a62335b75381ce944969723cfdca84aaff7bf
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/403679
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-13 13:59:07 -04:00
Daniel Verkamp
83b019a533 test/autorun_post: list tests that ran
In addition to listing the tests that did not run, add a list of the
tests that did run.

Change-Id: Idf8b3583214a5e05c2d7f1a89a2ea60a2ef68432
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402389
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-09 12:31:27 -05:00
Seth Howell
a562812dcf autotest: add test completion tracking
Change-Id: I1ca0578a010db2ff8535505bfd981cd1c368e403
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/392240
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-01 17:13:41 -05:00
Daniel Verkamp
dc6f9571c1 pep8: re-enable E302 (2 blank lines between functions)
This isn't a significant effort to fix, so let's just enable it to move
closer to the official Python PEP 8 guidelines.

Fix up the existing instances where we didn't already have two lines
between functions, and re-enable the pep8 warning.

Change-Id: I9cf5e7e32b9c6dfedbd1b0f9bc92951cbb2c8a0d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/399949
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-02-15 11:10:41 -05:00
Daniel Verkamp
9aa4bc7010 autorun_post: collect a single instance of doc
Put a copy of the generated Doxygen documentation output into the
top-level build directory so it can be linked from the build result
template.

Change-Id: Ied8ac9f6298b28ed4d991774b1396960498bd04c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/382890
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2017-10-18 12:44:29 -04:00
Seth Howell
8a9cfbb688 autorun_post: add missing library.
Change-Id: I58b2f85b78bca7c24bfae8335e26acf6a29c97ce
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/376033
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-08-28 19:12:47 -04:00
Seth Howell
a64e37677b CI: add autorun_post.py script
This script will be called by the build pool test scripts to run any
build post processing operations. In the initial state, it
aggegates the lcov output for each machine in the automated test pool.
Change-Id: I3395a5563763cd155c92b16a16076c18bea05b98
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/375816
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-08-28 14:22:24 -04:00