d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Import the kyua testing framework for infrastructure software

Browse Source 
Imported at 0.13 plus assumulated changes to git hash a685f91.

Obtained from:	https://github.com/jmmv/kyua
Sponsored by:	DARPA
This commit is contained in:
Brooks Davis 2020-03-17 16:56:50 +00:00
commit 08334c51db
542 changed files with 96704 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
.gitignore vendored Normal file
View File

@ -0,0 +1,23 @@
*.a
*.o
*_helpers
*_inttest
*_test
*~
.deps
.dirstamp
Doxyfile
Makefile
Makefile.in
aclocal.m4
api-docs
autom4te.cache
config.h
config.h.in
config.log
config.status
configure
kyua
local-kyua
stamp-h1

49
.travis.yml Normal file
View File

@ -0,0 +1,49 @@
language: cpp
sudo: required
before_install:
- ./admin/travis-install-deps.sh
matrix:
include:
- os: linux
dist: xenial
compiler: clang
env: ARCH=amd64 DO=distcheck AS_ROOT=no
- os: linux
dist: xenial
compiler: gcc
env: ARCH=amd64 DO=distcheck AS_ROOT=no
- os: linux
dist: xenial
compiler: clang
env: ARCH=amd64 DO=apidocs
- os: linux
dist: xenial
compiler: clang
env: ARCH=amd64 DO=style
- os: linux
dist: xenial
compiler: clang
env: ARCH=amd64 DO=distcheck AS_ROOT=yes UNPRIVILEGED_USER=no
- os: linux
dist: xenial
compiler: clang
env: ARCH=amd64 DO=distcheck AS_ROOT=yes UNPRIVILEGED_USER=yes
# TODO(ngie): reenable i386; the libraries were not available in the
# Ubuntu Xenial x86_64 docker image.
#- os: linux
# dist: xenial
# compiler: clang
# env: ARCH=i386 DO=distcheck AS_ROOT=no
#- os: linux
# dist: xenial
# compiler: gcc
# env: ARCH=i386 DO=distcheck AS_ROOT=no
script:
- ./admin/travis-build.sh
notifications:
email:
- kyua-log@googlegroups.com

11
AUTHORS Normal file
View File

@ -0,0 +1,11 @@
# This is the official list of Kyua authors for copyright purposes.
#
# This file is distinct from the CONTRIBUTORS files; see the latter for
# an explanation.
#
# Names are sorted alphabetically and should be added to this file as:
#
# * Name <email address>
# * Organization <optional email address>
* Google Inc.

173
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,173 @@
Contributing code to Kyua
=========================
Want to contribute? Great! But first, please take a few minutes to read this
document in full. Doing so upfront will minimize the turnaround time required
to get your changes incorporated.
Legal notes
-----------
* Before we can use your code, you must sign the
[Google Individual Contributor License
Agreement](https://developers.google.com/open-source/cla/individual),
also known as the CLA, which you can easily do online. The CLA is necessary
mainly because you own the copyright to your changes, even after your
contribution becomes part of our codebase, so we need your permission to use
and distribute your code. We also need to be sure of various other
things--for instance that you will tell us if you know that your code
infringes on other people's patents. You do not have to sign the CLA until
after you have submitted your code for review and a member has approved it,
but you must do it before we can put your code into our codebase.
* Contributions made by corporations are covered by a different agreement than
the one above: the
[Google Software Grant and Corporate Contributor License
Agreement](https://developers.google.com/open-source/cla/corporate).
Please get your company to sign this agreement instead if your contribution is
on their behalf.
* Unless you have a strong reason not to, please assign copyright of your
changes to Google Inc. and use the 3-clause BSD license text included
throughout the codebase (see [LICENSE](LICENSE)). Keeping the whole project
owned by a single entity is important, particularly to avoid the problem of
having to replicate potentially hundreds of different copyright notes in
documentation materials, etc.
Communication
-------------
* Before you start working on a larger contribution, you should get in touch
with us first through the
[kyua-discuss mailing
list](https://groups.google.com/forum/#!forum/kyua-discuss)
with your idea so that we can help out and possibly guide you. Coordinating
upfront makes it much easier to avoid frustration later on.
* Subscribe to the
[kyua-log mailing list](https://groups.google.com/forum/#!forum/kyua-log) to
get notifications on new commits, Travis CI results, or changes to bugs.
Git workflow
------------
* Always work on a non-master branch.
* Make sure the history of your branch is clean. (Ab)use `git rebase -i master`
to ensure the sequence of commits you want pulled is easy to follow and that
every commit does one (and only one) thing. In particular, commits of the
form `Fix previous` or `Fix build` should never ever exist; merge those fixes
into the relevant commits so that the history is clean at pull time.
* Always trigger Travis CI builds for your changes (hence why working on a
branch is important). Push your branch to GitHub so that Travis CI picks it
up and performs a build. If you have forked the repository, you may need to
enable Travis CI builds on your end. Wait for a green result.
* It is OK and expected for you to `git push --force` on **non-master**
branches. This is required if you need to go through the commit/test cycle
more than once for any given branch after you have "fixed-up" commits to
correct problems spotted in earlier builds.
* Do not send pull requests that subsume other/older pull requests. Each major
change being submitted belongs in a different pull request, which is trivial
to achieve if you use one branch per change as requested in this workflow.
Code reviews
------------
* All changes will be subject to code reviews pre-merge time. In other words:
all pull requests will be carefully inspected before being accepted and they
will be returned to you with comments if there are issues to be fixed.
* Be careful of stylistic errors in your code (see below for style guidelines).
Style violations hinder the review process and distract from the actual code.
By keeping your code clean of style issues upfront, you will speed up the
review process and avoid frustration along the way.
* Whenever you are ready to submit a pull request, review the *combined diff*
you are requesting to be pulled and look for issues. This is the diff that
will be subject to review, not necessarily the individual commits. You can
view this diff in GitHub at the bottom of the `Open a pull request` form that
appears when you click the button to file a pull request, or you can see the
diff by typing `git diff <your-branch> master`.
Commit messages
---------------
* Follow standard Git commit message guidelines. The first line has a maximum
length of 50 characters, does not terminate in a period, and has to summarize
the whole commit. Then a blank line comes, and then multiple plain-text
paragraphs provide details on the commit if necessary with a maximum length of
72-75 characters per line. Vim has syntax highlighting for Git commit
messages and will let you know when you go above the maximum line lengths.
* Use the imperative tense. Say `Add foo-bar` or `Fix baz` instead of `Adding
blah`, `Adds bleh`, or `Added bloh`.
Handling bug tracker issues
---------------------------
* All changes pushed to `master` should cross-reference one or more issues in
the bug tracker. This is particularly important for bug fixes, but also
applies to major feature improvements.
* Unless you have a good reason to do otherwise, name your branch `issue-N`
where `N` is the number of the issue being fixed.
* If the fix to the issue can be done *in a single commit*, terminate the commit
message with `Fixes #N.` where `N` is the number of the issue being fixed and
include a note in `NEWS` about the issue in the same commit. Such fixes can
be merged onto master using fast-forward (the default behavior of `git
merge`).
* If the fix to the issue requires *more than one commit*, do **not** include
`Fixes #N.` in any of the individual commit messages of the branch nor include
any changes to the `NEWS` file in those commits. These "announcement" changes
belong in the merge commit onto `master`, which is done by `git merge --no-ff
--no-commit your-branch`, followed by an edit of `NEWS`, and terminated with a
`git commit -a` with the proper note on the bug being fixed.
Style guide
-----------
These notes are generic and certainly *non-exhaustive*:
* Respect formatting of existing files. Note where braces are placed, number of
blank lines between code chunks, how continuation lines are indented, how
docstrings are typed, etc.
* Indentation is *always* done using spaces, not tabs. The only exception is in
`Makefile`s, where any continuation line within a target must be prefixed by a
*single tab*.
* [Be mindful of spelling and
grammar.](http://julipedia.meroh.net/2013/06/readability-mind-your-typos-and-grammar.html)
Mistakes of this kind are enough of a reason to return a pull request.
* Use proper punctuation for all sentences. Always start with a capital letter
and terminate with a period.
* Respect lexicographical sorting wherever possible.
* Lines must not be over 80 characters.
* No trailing whitespace.
* Two spaces after end-of-sentence periods.
* Two blank lines between functions. If there are two blank lines among code
blocks, they usually exist for a reason: keep them.
* In C++ code, prefix all C identifiers (those coming from `extern "C"`
includes) with `::`.
* Getter functions/methods only need to be documented via `\return`. A
redundant summary is not necessary.

20
CONTRIBUTORS Normal file
View File

@ -0,0 +1,20 @@
# This is the list of people who have agreed to one of the CLAs and can
# contribute patches to the Kyua project.
#
# The AUTHORS file lists the copyright holders; this file lists people.
# For example: Google employees are listed here but not in AUTHORS
# because Google holds the copyright.
#
# See the following links for details on the CLA:
#
# https://developers.google.com/open-source/cla/individual
# https://developers.google.com/open-source/cla/corporate
#
# Names are sorted by last name and should be added as:
#
# * Name <email address>
* Sergey Bronnikov <sergeyb@openvz.org>
* Enji Cooper <yaneurabeya@gmail.com>
* Julio Merino <jmmv@google.com>
* Craig Rodrigues <rodrigc@crodrigues.org>

59
Doxyfile.in Normal file
View File

@ -0,0 +1,59 @@
# Copyright 2010 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
BUILTIN_STL_SUPPORT = YES
ENABLE_PREPROCESSING = YES
EXCLUDE_SYMBOLS = "ATF_TC*"
EXTRACT_ANON_NSPACES = YES
EXTRACT_LOCAL_CLASSES = YES
EXTRACT_PRIVATE = YES
EXTRACT_STATIC = YES
EXPAND_ONLY_PREDEF = YES
EXTENSION_MAPPING = ipp = C++
FILE_PATTERNS = *.c *.h *.cpp *.hpp *.ipp
GENERATE_LATEX = NO
GENERATE_TAGFILE = @top_builddir@/api-docs/api-docs.tag
HIDE_FRIEND_COMPOUNDS = YES
INPUT = @top_srcdir@
INPUT_ENCODING = ISO-8859-1
JAVADOC_AUTOBRIEF = YES
MACRO_EXPANSION = YES
OUTPUT_DIRECTORY = @top_builddir@/api-docs
OUTPUT_LANGUAGE = English
PREDEFINED += "KYUA_DEFS_NORETURN="
PREDEFINED += "KYUA_DEFS_FORMAT_PRINTF(x, y)="
PROJECT_NAME = "@PACKAGE_NAME@"
PROJECT_NUMBER = @VERSION@
QUIET = YES
RECURSIVE = YES
SORT_BY_SCOPE_NAME = YES
SORT_MEMBERS_CTORS_1ST = YES
WARN_IF_DOC_ERROR = YES
WARN_IF_UNDOCUMENTED = YES
WARN_NO_PARAMDOC = YES
WARNINGS = YES

268
INSTALL.md Normal file
View File

@ -0,0 +1,268 @@
Installation instructions
=========================
Kyua uses the GNU Automake, GNU Autoconf and GNU Libtool utilities as
its build system. These are used only when compiling the application
from the source code package. If you want to install Kyua from a binary
package, you do not need to read this document.
For the impatient:
$ ./configure
$ make
$ make check
Gain root privileges
# make install
Drop root privileges
$ make installcheck
Or alternatively, install as a regular user into your home directory:
$ ./configure --prefix ~/local
$ make
$ make check
$ make install
$ make installcheck
Dependencies
------------
To build and use Kyua successfully you need:
* A standards-compliant C and C++ complier.
* Lutok 0.4.
* pkg-config.
* SQLite 3.6.22.
To build the Kyua tests, you optionally need:
* The Automated Testing Framework (ATF), version 0.15 or greater. This
is required if you want to create a distribution file.
If you are building Kyua from the code on the repository, you will also
need the following tools:
* GNU Autoconf.
* GNU Automake.
* GNU Libtool.
Regenerating the build system
-----------------------------
This is not necessary if you are building from a formal release
distribution file.
On the other hand, if you are building Kyua from code extracted from the
repository, you must first regenerate the files used by the build
system. You will also need to do this if you modify `configure.ac`,
`Makefile.am` or any of the other build system files. To do this, simply
run:
$ autoreconf -i -s
If ATF is installed in a different prefix than Autoconf, you will also
need to tell autoreconf where the ATF M4 macros are located. Otherwise,
the configure script will be incomplete and will show confusing syntax
errors mentioning, for example, `ATF_CHECK_SH`. To fix this, you have
to run autoreconf in the following manner, replacing `<atf-prefix>` with
the appropriate path:
$ autoreconf -i -s -I <atf-prefix>/share/aclocal
General build procedure
-----------------------
To build and install the source package, you must follow these steps:
1. Configure the sources to adapt to your operating system. This is
done using the `configure` script located on the sources' top
directory, and it is usually invoked without arguments unless you
want to change the installation prefix. More details on this
procedure are given on a later section.
2. Build the sources to generate the binaries and scripts. Simply run
`make` on the sources' top directory after configuring them. No
problems should arise.
3. Check that the built programs work by running `make check`. You do
not need to be root to do this, but if you are not, some checks will
be skipped.
4. Install the program by running `make install`. You may need to
become root to issue this step.
5. Issue any manual installation steps that may be required. These are
described later in their own section.
6. Check that the installed programs work by running `make
installcheck`. You do not need to be root to do this, but if you are
not, some checks will be skipped.
Configuration flags
-------------------
The most common, standard flags given to `configure` are:
* `--prefix=directory`:
**Possible values:** Any path.
**Default:** `/usr/local`.
Specifies where the program (binaries and all associated files) will
be installed.
* `--sysconfdir=directory`:
**Possible values:** Any path.
**Default:** `/usr/local/etc`.
Specifies where the installed programs will look for configuration
files. `/kyua` will be appended to the given path unless
`KYUA_CONFSUBDIR` is redefined as explained later on.
* `--help`:
Shows information about all available flags and exits immediately,
without running any configuration tasks.
The following environment variables are specific to Kyua's `configure`
script:
* `GDB`:
**Possible values:** empty, absolute path to GNU GDB.
**Default:** empty.
Specifies the path to the GNU GDB binary that Kyua will use to gather a
stack trace of a crashing test program. If empty, the configure script
will try to find a suitable binary for you and, if not found, Kyua will
attempt to do the search at run time.
* `KYUA_ARCHITECTURE`:
**Possible values:** name of a CPU architecture (e.g. `x86_64`, `powerpc`).
**Default:** autodetected; typically the output of `uname -p`.
Specifies the name of the CPU architecture on which Kyua will run.
This value is used at run-time to determine tests that are not
applicable to the host system.
* `KYUA_CONFSUBDIR`:
**Possible values:** empty, a relative path.
**Default:** `kyua`.
Specifies the subdirectory of the configuration directory (given by
the `--sysconfdir` argument) under which Kyua will search for its
configuration files.
* `KYUA_CONFIG_FILE_FOR_CHECK`:
**Possible values:** none, an absolute path to an existing file.
**Default:** none.
Specifies the `kyua.conf` configuration file to use when running any
of the `check`, `installcheck` or `distcheck` targets on this source
tree. This setting is exclusively used to customize the test runs of
Kyua itself and has no effect whatsoever on the built product.
* `KYUA_MACHINE`:
**Possible values:** name of a machine type (e.g. `amd64`, `macppc`).
**Default:** autodetected; typically the output of `uname -m`.
Specifies the name of the machine type on which Kyua will run. This
value is used at run-time to determine tests that are not applicable
to the host system.
* `KYUA_TMPDIR`:
**Possible values:** an absolute path to a temporary directory.
**Default:** `/tmp`.
Specifies the path that Kyua will use to create temporary directories
in by default.
The following flags are specific to Kyua's `configure` script:
* `--enable-developer`:
**Possible values:** `yes`, `no`.
**Default:** `yes` in Git `HEAD` builds; `no` in formal releases.
Enables several features useful for development, such as the inclusion
of debugging symbols in all objects or the enforcement of compilation
warnings.
The compiler will be executed with an exhaustive collection of warning
detection features regardless of the value of this flag. However, such
warnings are only fatal when `--enable-developer` is `yes`.
* `--with-atf`:
**Possible values:** `yes`, `no`, `auto`.
**Default:** `auto`.
Enables usage of ATF to build (and later install) the tests.
Setting this to `yes` causes the configure script to look for ATF
unconditionally and abort if not found. Setting this to `auto` lets
configure perform the best decision based on availability of ATF.
Setting this to `no` explicitly disables ATF usage.
When support for tests is enabled, the build process will generate the
test programs and will later install them into the tests tree.
Running `make check` or `make installcheck` from within the source
directory will cause these tests to be run with Kyua.
* `--with-doxygen`:
**Possible values:** `yes`, `no`, `auto` or a path.
**Default:** `auto`.
Enables usage of Doxygen to generate documentation for internal APIs.
This documentation is *not* installed and is only provided to help the
developer of this package. Therefore, enabling or disabling Doxygen
causes absolutely no differences on the files installed by this
package.
Setting this to `yes` causes the configure script to look for Doxygen
unconditionally and abort if not found. Setting this to `auto` lets
configure perform the best decision based on availability of Doxygen.
Setting this to `no` explicitly disables Doxygen usage. And, lastly,
setting this to a path forces configure to use a specific Doxygen
binary, which must exist.
Post-installation steps
-----------------------
Copy the `Kyuafile.top` file installed in the examples directory to the
root of your tests hierarchy and name it `Kyuafile`. For example:
# cp /usr/local/share/kyua/examples/Kyuafile.top \
/usr/local/tests/Kyuafile
This will allow you to simply go into `/usr/tests` and run the tests
from there.
Run the tests!
--------------
Lastly, after a successful installation, you should periodically run the
tests from the final location to ensure things remain stable. Do so as
follows:
$ cd /usr/local/kyua && kyua test
The following configuration variables are specific to the 'kyua' test
suite and can be given to Kyua with arguments of the form
`-v test_suites.kyua.<variable_name>=<value>`:
* `run_coredump_tests`:
**Possible values:** `true` or `false`.
**Default:** `true`.
Avoids running tests that crash subprocesses on purpose to make them
dump core. Such tests are particularly slow on macOS, and it is
sometimes handy to disable them for quicker development iteration.
If you see any tests fail, do not hesitate to report them in:
https://github.com/jmmv/kyua/issues/
Thank you!

18
Kyuafile Normal file
View File

@ -0,0 +1,18 @@
syntax(2)
test_suite("kyua")
include("bootstrap/Kyuafile")
include("cli/Kyuafile")
if fs.exists("doc/Kyuafile") then
-- The tests for the docs are not installed because they only cover the
-- build-time process of the manual pages.
include("doc/Kyuafile")
end
include("drivers/Kyuafile")
include("engine/Kyuafile")
include("examples/Kyuafile")
include("integration/Kyuafile")
include("model/Kyuafile")
include("store/Kyuafile")
include("utils/Kyuafile")

27
LICENSE Normal file
View File

@ -0,0 +1,27 @@
Copyright 2010-2015 The Kyua Authors.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Google Inc. nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

186
Makefile.am Normal file
View File

@ -0,0 +1,186 @@
# Copyright 2010 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
ACLOCAL_AMFLAGS = -I m4
CHECK_BOOTSTRAP_DEPS =
CHECK_KYUA_DEPS =
CHECK_LOCAL =
CLEAN_TARGETS =
DIST_HOOKS =
PHONY_TARGETS =
CLEANFILES =
EXTRA_DIST =
noinst_DATA =
noinst_LIBRARIES =
noinst_SCRIPTS =
doc_DATA = AUTHORS CONTRIBUTING.md CONTRIBUTORS LICENSE NEWS.md
noinst_DATA += INSTALL.md README.md
EXTRA_DIST += $(doc_DATA) INSTALL.md README.md
if WITH_ATF
tests_topdir = $(pkgtestsdir)
tests_top_DATA = Kyuafile
EXTRA_DIST += $(tests_top_DATA)
endif
include admin/Makefile.am.inc
include bootstrap/Makefile.am.inc
include cli/Makefile.am.inc
include doc/Makefile.am.inc
include drivers/Makefile.am.inc
include engine/Makefile.am.inc
include examples/Makefile.am.inc
include integration/Makefile.am.inc
include misc/Makefile.am.inc
include model/Makefile.am.inc
include store/Makefile.am.inc
include utils/Makefile.am.inc
bin_PROGRAMS = kyua
kyua_SOURCES = main.cpp
kyua_CXXFLAGS = $(CLI_CFLAGS) $(ENGINE_CFLAGS) $(UTILS_CFLAGS)
kyua_LDADD = $(CLI_LIBS) $(ENGINE_LIBS) $(UTILS_LIBS)
CHECK_ENVIRONMENT = KYUA_CONFDIR="/non-existent" \
KYUA_DOCDIR="$(abs_top_srcdir)" \
KYUA_EXAMPLESDIR="$(abs_top_srcdir)/examples" \
KYUA_MISCDIR="$(abs_top_srcdir)/misc" \
KYUA_STOREDIR="$(abs_top_srcdir)/store" \
KYUA_STORETESTDATADIR="$(abs_top_srcdir)/store" \
PATH="$(abs_top_builddir):$${PATH}"
INSTALLCHECK_ENVIRONMENT = KYUA_CONFDIR="/non-existent" \
PATH="$(prefix)/bin:$${PATH}"
# Generate local-kyua, a wrapper shell script to run the just-built 'kyua'
# binary by pointing it to the possibly not-yet-installed data files in the
# build tree.
noinst_SCRIPTS += local-kyua
CLEANFILES += local-kyua local-kyua.tmp
local-kyua: Makefile
$(AM_V_GEN)echo '#!/bin/sh' >local-kyua.tmp; \
echo 'env $(CHECK_ENVIRONMENT) $(TESTS_ENVIRONMENT)' \
'"$(abs_top_builddir)/kyua" \
--config='$(KYUA_CONFIG_FILE_FOR_CHECK)' \
"$${@}"' >>local-kyua.tmp; \
chmod +x local-kyua.tmp; \
mv -f local-kyua.tmp local-kyua
if WITH_ATF
CHECK_LOCAL += dump-ulimits check-kyua
PHONY_TARGETS += check-kyua
check-kyua: $(CHECK_KYUA_DEPS)
@failed=no; \
./local-kyua test \
--kyuafile='$(top_srcdir)/Kyuafile' --build-root='$(top_builddir)' \
|| failed=yes; \
if [ "$${failed}" = yes ]; then \
./local-kyua report --results-file='$(abs_top_srcdir)' \
--verbose --results-filter=broken,failed; \
exit 1; \
fi
installcheck-local: dump-ulimits installcheck-kyua
PHONY_TARGETS += installcheck-kyua
installcheck-kyua:
@failed=no; \
cd $(pkgtestsdir) && $(INSTALLCHECK_ENVIRONMENT) $(TESTS_ENVIRONMENT) \
kyua --config='$(KYUA_CONFIG_FILE_FOR_CHECK)' test \
|| failed=yes; \
if [ "$${failed}" = yes ]; then \
cd $(pkgtestsdir) && $(INSTALLCHECK_ENVIRONMENT) \
$(TESTS_ENVIRONMENT) \
kyua --config='$(KYUA_CONFIG_FILE_FOR_CHECK)' report \
--verbose --results-filter=broken,failed; \
exit 1; \
fi
# TODO(jmmv): kyua should probably be recording this information itself as part
# of the execution context, just as we record environment variables.
PHONY_TARGETS += dump-ulimits
dump-ulimits:
@echo "Resource limits:"
@{ \
ulimit -a | sed -e 's,$$, (soft),'; \
ulimit -a -H | sed -e 's,$$, (hard),'; \
} | sort | sed -e 's,^, ,'
@echo
else
DIST_HOOKS += forbid-dist
PHONY_TARGETS += forbid-dist
forbid-dist:
@echo "Sorry; cannot make dist without atf."
@false
endif
check-local: $(CHECK_LOCAL)
if WITH_DOXYGEN
# Runs doxygen on the source tree and validates the contents of the docstrings.
# We do not do this by default, even if doxygen has been enabled, because this
# step takes a long time. Instead, we just rely on a Travis CI build to catch
# inconsistencies.
PHONY_TARGETS += check-api-docs
check-api-docs: api-docs/api-docs.tag
@$(AWK) -f $(srcdir)/admin/check-api-docs.awk api-docs/doxygen.out
api-docs/api-docs.tag: $(builddir)/Doxyfile $(SOURCES)
@$(MKDIR_P) api-docs
@rm -f api-docs/doxygen.out api-docs/doxygen.out.tmp
$(AM_V_GEN)$(DOXYGEN) $(builddir)/Doxyfile \
>api-docs/doxygen.out.tmp 2>&1 && \
mv api-docs/doxygen.out.tmp api-docs/doxygen.out
CLEAN_TARGETS += clean-api-docs
clean-api-docs:
rm -rf api-docs
endif
# Replace Automake's builtin check-news functionality so that we can validate
# the NEWS.md file instead of NEWS.
DIST_HOOKS += check-news
PHONY_TARGETS += check-news
check-news:
@case "$$(sed 15q "$(srcdir)/NEWS.md")" in \
*"$(VERSION)"*) : ;; \
*) \
echo "NEWS.md not updated; not releasing" 1>&2; \
exit 1 \
;; \
esac
clean-local: $(CLEAN_TARGETS)
dist-hook: $(DIST_HOOKS)
PHONY_TARGETS += clean-all
clean-all:
GIT="$(GIT)" $(SH) $(srcdir)/admin/clean-all.sh
.PHONY: $(PHONY_TARGETS)

622
NEWS.md Normal file
View File

@ -0,0 +1,622 @@
Major changes between releases
==============================
Changes in version 0.14
-----------------------
**NOT RELEASED YET; STILL UNDER DEVELOPMENT.**
* Explicitly require C++11 language features when compiling Kyua.
Changes in version 0.13
-----------------------
**Released on August 26th, 2016.**
* Fixed execution of test cases as an unprivileged user, at least under
NetBSD 7.0. Kyua-level failures were probably a regression introduced
in Kyua 0.12, but the underlying may have existed for much longer:
test cases might have previously failed for mysterious reasons when
running under an unprivileged user.
* Issue #134: Fixed metadata test broken on 32-bit platforms.
* Issue #139: Added per-test case start/end timestamps to all reports.
* Issue #156: Fixed crashes due to the invalid handling of cleanup
routine data and triggered by the reuse of PIDs in long-running Kyua
instances.
* Issue #159: Fixed TAP parser to ignore case while matching `TODO` and
`SKIP` directives, and to also recognize `Skipped`.
* Fixed potential crash due to a race condition in the unprogramming of
timers to control test deadlines.
Changes in version 0.12
-----------------------
**Released on November 22nd, 2015.**
This is a huge release and marks a major milestone for Kyua as it finally
implements a long-standing feature request: the ability to execute test
cases in parallel. This is a big deal because test cases are rarely
CPU-bound: running them in parallel yields much faster execution times for
large test suites, allowing faster iteration of changes during development.
As an example: the FreeBSD test suite as of this date contains 3285 test
cases. With sequential execution, a full test suite run takes around 12
minutes to complete, whereas on a 4-core machine with a high level of
parallelism it takes a little over 1 minute.
Implementing parallel execution required rewriting most of Kyua's core and
partly explains explains why there has not been a new release for over a
year. The current implementation is purely subprocess-based, which works
but has some limitations and has resulted in a core that is really complex
and difficult to understand. Future versions will investigate the use of
threads instead for a simplified programming model and additional
parallelization possibilities.
* Issue #2: Implemented support to execute test cases in parallel when
invoking `kyua test`. Parallel execution is *only* enabled when the new
`parallelism` configuration variable is set to a value greater than `1`.
The default behavior is still to run tests sequentially because some test
suites contain test cases with side-effects that might fail when run in
parallel. To resolve this, the new metadata property `is_exclusive` can
be set to `true` on a test basis to indicate that the test must be run on
its own.
* Known regression: Running `kyua debug` on a TAP-based test program does
not currently report the output in real time. The output will only be
displayed once the test program completes. This is a shortcoming of
the new parallel execution engine and will be resolved.
* Removed the external C-based testers code in favor of the new built-in
implementations. The new approach feels significantly faster than the
previous one.
* Fixed the handling of relative paths in the `fs.*` functions available
in `Kyuafile`s. All paths are now resolved relative to the location of
the caller `Kyuafile`. `Kyuafile.top` has been updated with these
changes and you should update custom copies of this file with the new
version.
* Changed temporary directory creation to always grant search
permissions on temporary directories. This is to prevent potential
problems when running Kyua as root and executing test cases that require
dropping privileges (as they may later be unable to use absolute paths
that point inside their work directory).
* The cleanup of work directories does not longer attempt to deal with
mount points. If a test case mounts a file system and forgets to unmount
it, the mount point will be left behind. It is now the responsibility of
the test case to clean after itself. The reasons for this change are
simplicity and clarity: there are many more things that a test case can
do that have side-effects on the system and Kyua cannot protect against
them all, so it is better to just have the test undo anything it might
have done.
* Improved `kyua report --verbose` to properly handle environment
variables with continuation lines in them, and fixed the integration
tests for this command to avoid false negatives.
* Changed the configuration file format to accept the definition of
unknown variables without declaring them local. The syntax version
number remains at 2. This is to allow configuration files for newer Kyua
versions to work on older Kyua versions, as there is no reason to forbid
this.
* Fixed stacktrace gathering with FreeBSD's ancient version of GDB.
GDB 6.1.1 (circa 2004) does not have the `-ex` flag so we need to
generate a temporary GDB script and feed it to GDB with `-x` instead.
* Issue #136: Fixed the XML escaping in the JUnit output so that
non-printable characters are properly handled when they appear in the
process's stdout or stderr.
* Issue #141: Improved reporting of errors triggered by sqlite3. In
particular, all error messages are now tagged with their corresponding
database filename and, if they are API-level errors, the name of the
sqlite3 function that caused them.
* Issue #144: Improved documentation on the support for custom properties
in the test metadata.
* Converted the `INSTALL`, `NEWS`, and `README` distribution documents to
Markdown for better formatting online.
Changes in version 0.11
-----------------------
**Released on October 23rd, 2014.**
* Added support to print the details of all test cases (metadata and
their output) to `report`. This is via a new `--verbose` flag which
replaces the previous `--show-context`.
* Added support to specify the amount of physical disk space required
by a test case. This is in the form of a new `required_disk_space`
metadata property, which can also be provided by ATF test cases as
`require.diskspace`.
* Assimilated the contents of all the `kyua-*-tester(1)` and
`kyua-*-interface(7)` manual pages into more relevant places. In
particular, added more details on test program registration and their
metadata to `kyuafile(5)`, and added `kyua-test-isolation(7)`
describing the isolation features of the test execution.
* Assimilated the contents of all auxiliary manual pages, including
`kyua-build-root(7)`, `kyua-results-files(7)`, `kyua-test-filters(7)`
and `kyua-test-isolation(7)`, into the relevant command-specific
manual pages. This is for easier discoverability of relevant
information when reading how specific Kyua commands work.
* Issue #30: Plumbed through support to query configuration variables
from ATF's test case heads. This resolves the confusing situation
where test cases could only do this from their body and cleanup
routines.
* Issue #49: Extended `report` to support test case filters as
command-line arguments. Combined with `--verbose`, this allows
inspecting the details of a test case failure after execution.
* Issue #55: Deprecated support for specifying `test_suite` overrides on
a test program basis. This idiom should not be used but support for
it remains in place.
* Issue #72: Added caching support to the `getcwd(3)` test in configure
so that the result can be overriden for cross-compilation purposes.
* Issue #83: Changed manual page headings to include a `kyua` prefix in
their name. This prevents some possible confusion when displaying,
for example, the `kyua-test` manual page with a plain name of `test`.
* Issue #84: Started passing test-suite configuration variables to plain
and TAP test programs via the environment. The name of the
environment variables set this way is prefixed by `TEST_ENV_`, so a
configuration variable of the form
`test_suites.some_name.allow_unsafe_ops=yes` in `kyua.conf` becomes
`TEST_ENV_allow_unsafe_ops=YES` in the environment.
* Issues #97 and #116: Fixed the build on Illumos.
* Issue #102: Set `TMPDIR` to the test case's work directory when running
the test case. If the test case happens to use the `mktemp(3)` family
of functions (due to misunderstandings on how Kyua works or due to
the reuse of legacy test code), we don't want it to easily escape the
automanaged work directory.
* Issue #103: Started being more liberal in the parsing of TAP test
results by treating the number in `ok` and `not ok` lines as optional.
* Issue #105: Started using tmpfs instead of md as a temporary file
system for tests in FreeBSD so that we do not leak `md(4)` devices.
* Issue #109: Changed the privilege dropping code to start properly
dropping group privileges when `unprivileged_user` is set. Also fixes
`testers/run_test:fork_wait__unprivileged_group`.
* Issue #110: Changed `help` to display version information and clarified
the purpose of the `about` command in its documentation.
* Issue #111: Fixed crash when defining a test program in a `Kyuafile`
that has not yet specified the test suite name.
* Issue #114: Improved the `kyuafile(5)` manual page by clarifying the
restrictions of the `include()` directive and by adding abundant
examples.
Changes in version 0.10
-----------------------
**Experimental version released on August 14th, 2014.**
* Merged `kyua-cli` and `kyua-testers` into a single `kyua` package.
* Dropped the `kyua-atf-compat` package.
* Issue #100: Do not try to drop privileges to `unprivileged_user` when we
are already running as an unprivileged user. Doing so is not possible
and thus causes spurious test failures when the current user is not
root and the current user and `unprivileged_user` do not match.
* Issue #79: Mention `kyua.conf(5)` in the *See also* section of `kyua(1)`.
* Issue #75: Change the `rewrite__expected_signal__bad_arg` test in
`testers/atf_result_test` to use a different signal value. This is to
prevent triggering a core dump that made the test fail in some platforms.
Changes in kyua-cli version 0.9
-------------------------------
**Experimental version released on August 8th, 2014.**
Major changes:
The internal architecture of Kyua to record the results of test suite
runs has completely changed in this release. Kyua no longer stores all
the different test suite run results as different "actions" within the
single `store.db` database. Instead, Kyua now generates a separate
results file inside `~/.kyua/store/` for every test suite run.
Due to the complexity involved in the migration process and the little
need for it, this is probably going to be the only release where the
`db-migrate` command is able to convert an old `store.db` file to the
new scheme.
Changes in more detail:
* Added the `report-junit` command to generate JUnit XML result files.
The output has been verified to work within Jenkins.
* Switched to results files specific to their corresponding test suite
run. The unified `store.db` file is now gone: `kyua test` creates a
new results file for every invocation under `~/.kyua/store/` and the
`kyua report*` commands are able to locate the latest file for a
corresponding test suite automatically.
* The `db-migrate` command takes an old `store.db` file and generates
one results file for every previously-recorded action, later deleting
the `store.db` file.
* The `--action` flag has been removed from all commands that accepted
it. This has been superseded by the tests results files.
* The `--store` flag that many commands took has been renamed to
`--results-file` in line with the semantical changes.
* The `db-exec` command no longer creates an empty database when none
is found. This command is now intended to run only over existing
files.
Changes in kyua-testers version 0.3
-----------------------------------
**Experimental version released on August 8th, 2014.**
* Made the testers set a "sanitized" value for the `HOME` environment
variable where, for example, consecutive and trailing slashes have
been cleared. Mac OS X has a tendency to append a trailing slash to
the value of `TMPDIR`, which can cause third-party tests to fail if
they compare `${HOME}` with `$(pwd)`.
* Issues #85, #86, #90 and #92: Made the TAP parser more complete: mark
test cases reported as `TODO` or `SKIP` as passed; handle skip plans;
ignore lines that look like `ok` and `not ok` but aren't results; and
handle test programs that report a pass but exit with a non-zero code.
Changes in kyua-cli version 0.8
-------------------------------
**Experimental version released on December 7th, 2013.**
* Added support for Lutok 0.4.
* Issue #24: Plug the bootstrap tests back into the test suite. Fixes
in `kyua-testers` 0.2 to isolate test cases into their own sessions
should allow these to run fine.
* Issue #74: Changed the `kyuafile(5)` parser to automatically discover
existing tester interfaces. The various `*_test_program()` functions
will now exist (or not) based on tester availability, which simplifies
the addition of new testers or the selective installation of them.
Changes in kyua-testers version 0.2
-----------------------------------
**Experimental version released on December 7th, 2013.**
* Issue #74: Added the `kyua-tap-tester`, a new backend to interact with
test programs that comply with the Test Anything Protocol.
* Issue #69: Cope with the lack of `AM_PROG_AR` in `configure.ac`, which
first appeared in Automake 1.11.2. Fixes a problem in Ubuntu 10.04
LTS, which appears stuck in 1.11.1.
* Issue #24: Improve test case isolation by confining the tests to their
own session instead of just to their own process group.
Changes in kyua-cli version 0.7
-------------------------------
**Experimental version released on October 18th, 2013.**
* Made failures from testers more resilent. If a tester fails, the
corresponding test case will be marked as broken instead of causing
kyua to exit.
* Added the `--results-filter` option to the `report-html` command and
set its default value to skip passed results from HTML reports. This
is to keep these reports more succint and to avoid generating tons of
detail files that will be, in general, useless.
* Switched to use Lutok 0.3 to gain compatibility with Lua 5.2.
* Issue #69: Cope with the lack of `AM_PROG_AR` in `configure.ac`, which
first appeared in Automake 1.11.2. Fixes a problem in Ubuntu 10.04
LTS, which appears stuck in 1.11.1.
Changes in kyua-cli version 0.6
-------------------------------
**Experimental version released on February 22nd, 2013.**
* Issue #36: Changed `kyua help` to not fail when the configuration file
is bogus. Help should always work.
* Issue #37: Simplified the `syntax()` calls in configuration and
`Kyuafile` files to only specify the requested version instead of also
the format name. The format name is implied by the file being loaded, so
there is no use in the caller having to specify it. The version number
of these file formats has been bumped to 2.
* Issue #39: Added per-test-case metadata values to the HTML reports.
* Issue #40: Rewrote the documentation as manual pages and removed the
previous GNU Info document.
* Issue #47: Started using the independent testers in the `kyua-testers`
package to run the test cases. Kyua does not implement the logic to
invoke test cases any more, which provides for better modularity,
extensibility and robustness.
* Issue #57: Added support to specify arbitrary metadata properties for
test programs right from the `Kyuafile`. This is to make plain test
programs more versatile, by allowing them to specify any of the
requirements (allowed architectures, required files, etc.) supported
by Kyua.
* Reduced automatic screen line wrapping of messages to the `help`
command and the output of tables by `db-exec`. Wrapping any other
messages (specially anything going to stderr) was very annoying
because it prevented natural copy/pasting of text.
* Increased the granularity of the error codes returned by `kyua(1)` to
denote different error conditions. This avoids the overload of `1` to
indicate both "expected" errors from specific subcommands and
unexpected errors caused by the internals of the code. The manual now
correctly explain how the exit codes behave on a command basis.
* Optimized the database schema to make report generation almost
instantaneous.
* Bumped the database schema to 2. The database now records the
metadata of both test programs and test cases generically, without
knowledge of their interface.
* Added the `db-migrate` command to provide a mechanism to upgrade a
database with an old schema to the current schema.
* Removed the GDB build-time configuration variable. This is now part
of the `kyua-testers` package.
* Issue #31: Rewrote the `Kyuafile` parsing code in C++, which results in
a much simpler implementation. As a side-effect, this gets rid of the
external Lua files required by `kyua`, which in turn make the tool
self-contained.
* Added caching of various configure test results (particularly in those
tests that need to execute a test program) so that cross-compilers can
predefine the results of the tests without having to run the
executables.
Changes in kyua-testers version 0.1
-----------------------------------
**Experimental version released on February 19th, 2013.**
This is the first public release of the `kyua-testers` package.
The goal of this first release is to adopt all the test case execution
code of `kyua-cli` 0.5 and ship it as a collection of independent tester
binaries. The `kyua-cli` package will rely on these binaries to run the
tests, which provides better modularity and simplicity to the
architecture of Kyua.
The code in this package is all C as opposed to the current C++ codebase
of `kyua-cli`, which means that the overall build times of Kyua are now
reduced.
Changes in kyua-cli version 0.5
-------------------------------
**Experimental version released on July 10th, 2012.**
* Issue #15: Added automatic stacktrace gathering of crashing test cases.
This relies on GDB and is a best-effort operation.
* Issue #32: Added the `--build-root` option to the debug, list and test
commands. This allows executing test programs from a different
directory than where the `Kyuafile` scripts live. See the *Build roots*
section in the manual for more details.
* Issue #33: Removed the `kyuaify.sh` script. This has been renamed to
atf2kyua and moved to the `kyua-atf-compat` module, where it ships as a
first-class utility (with a manual page and tests).
* Issue #34: Changed the HTML reports to include the stdout and stderr of
every test case.
* Fixed the build when using a "build directory" and a clean source tree
from the repository.
Changes in kyua-cli version 0.4
-------------------------------
**Experimental version released on June 6th, 2012.**
* Added the `report-html` command to generate HTML reports of the
execution of any recorded action.
* Changed the `--output` flag of the `report` command to only take a
path to the target file, not its format. Different formats are better
supported by implementing different subcommands, as the options they
may receive will vary from format to format.
* Added a `--with-atf` flag to the configure script to control whether
the ATF tests get built or not. May be useful for packaging systems
that do not have ATF in them yet. Disabling ATF also cuts down the
build time of Kyua significantly, but with the obvious drawbacks.
* Grouped `kyua` subcommands by topic both in the output of `help` and
in the documentation. In general, the user needs to be aware of
commands that rely on a current project and those commands that rely
purely on the database to generate reports.
* Made `help` print the descriptions of options and commands properly
tabulated.
* Changed most informational messages to automatically wrap on screen
boundaries.
* Rewrote the configuration file parsing module for extensibility. This
will allow future versions of Kyua to provide additional user-facing
options in the configuration file.
No syntax changes have been made, so existing configuration files
(version 1) will continue to be parsed without problems. There is one
little exception though: all variables under the top-level
`test_suites` tree must be declared as strings.
Similarly, the `-v` and `--variable` flags to the command line must
now carry a `test_suites.` prefix when referencing any variables under
such tree.
Changes in kyua-cli version 0.3
-------------------------------
**Experimental version released on February 24th, 2012.**
* Made the `test` command record the results of the executed test
cases into a SQLite database. As a side effect, `test` now supports a
`--store` option to indicate where the database lives.
* Added the `report` command to generate plain-text reports of the
test results stored in the database. The interface of this command is
certainly subject to change at this point.
* Added the `db-exec` command to directly interact with the store
database.
* Issue #28: Added support for the `require.memory` test case property
introduced in ATF 0.15.
* Renamed the user-specific configuration file from `~/.kyuarc` to
`~/.kyua/kyua.conf` for consistency with other files stored in the
`~/.kyua/` subdirectory.
* Switched to use Lutok instead of our own wrappers over the Lua C
library. Lutok is just what used to be our own utils::lua module, but
is now distributed separately.
* Removed the `Atffile`s from the source tree. Kyua is stable enough
to generate trustworthy reports, and we do not want to give the
impression that atf-run / atf-report are still supported.
* Enabled logging to stderr for our own test programs. This makes it
slightly easier to debug problems in our own code when we get a
failing test.
Changes in kyua-cli version 0.2
-------------------------------
**Experimental version released on August 24th, 2011.**
The biggest change in this release is the ability for Kyua to run test
programs implemented using different frameworks. What this means is
that, now, a Kyua test suite can include not only ATF-based test
programs, but also "legacy" (aka plain) test programs that do not use
any framework. I.e. if you have tests that are simple programs that
exit with 0 on success and 1 on failure, you can plug them in into a
Kyua test suite.
Other than this, there have been several user-visible changes. The most
important are the addition of the new `config` and `debug` subcommands
to the `kyua` binary. The former can be used to inspect the runtime
configuration of Kyua after parsing, and the latter is useful to
interact with failing tests cases in order to get more data about the
failure itself.
Without further ado, here comes the itemized list of changes:
* Generalized the run-time engine to support executing test programs
that implement different interfaces. Test programs that use the ATF
libraries are just a special case of this. (Issue #18.)
* Added support to the engine to run `plain` test programs: i.e. test
programs that do not use any framework and report their pass/fail
status as an exit code. This is to simplify the integration of legacy
test programs into a test suite, and also to demonstrate that the
run-time engine is generic enough to support different test
interfaces. (Issue #18.)
* Added the `debug` subcommand. This command allows end users to tweak
the execution of a specific test case and to poke into the behavior of
its execution. At the moment, all this command allows is to view the
stdout and stderr of the command in real time (which the `test`
command currently completely hides).
* Added the `config` subcommand. This command allows the end user to
inspect the current configuration variables after evaluation, without
having to read through configuration files. (Issue #11.)
* Removed the `test_suites_var` function from configuration files. This
was used to set the value of test-suite-sepecific variables, but it
was ugly-looking. It is now possible to use the more natural syntax
`test_suites.<test-suite-name>.<variable> = <value>`. (Issue #11.)
* Added a mechanism to disable the loading of configuration files
altogether. Needed for testing purposes and for scriptability.
Available by passing the `--config=none` flag.
* Enabled detection of unused parameters and variables in the code and
fixed all warnings. (Issue #23.)
* Changed the behavior of "developer mode". Compiler warnings are now
enabled unconditionally regardless of whether we are in developer mode
or not; developer mode is now only used to perform strict warning
checks and to enable assertions. Additionally, developer mode is now
only automatically enabled when building from the repository, not for
formal releases. (Issue #22.)
* Fixed many build and portability problems to Debian sid with GCC 4.6.3
and Ubuntu 10.04.1 LTS. (Issues #20, #21, #26.)
Changes in kyua-cli version 0.1
-------------------------------
**Experimental version released on June 23rd, 2011.**
This is the first public release of the `kyua-cli` package.
The scope of this release is to provide functional replacement for the
`atf-run` utility included in the atf package. At this point, `kyua`
can reliably run the NetBSD 5.99.53 test suite delivering the same
results as `atf-run`.
The reporting facilities of this release are quite limited. There is
no replacement for `atf-report` yet, and there is no easy way of
debugging failing test programs other than running them by hand. These
features will mark future milestones and therefore be part of other
releases.
Be aware that this release has suffered very limited field testing.
The test suite for `kyua-cli` is quite comprehensive, but some bugs may
be left in any place.

84
README.md Normal file
View File

@ -0,0 +1,84 @@
Welcome to the Kyua project!
============================
Kyua is a **testing framework** for infrastructure software, originally
designed to equip BSD-based operating systems with a test suite. This
means that Kyua is lightweight and simple, and that Kyua integrates well
with various build systems and continuous integration frameworks.
Kyua features an **expressive test suite definition language**, a **safe
runtime engine** for test suites and a **powerful report generation
engine**.
Kyua is for **both developers *and* users**, from the developer applying a
simple fix to a library to the system administrator deploying a new release
on a production machine.
Kyua is **able to execute test programs written with a plethora of testing
libraries and languages**. The library of choice is
[ATF](https://github.com/jmmv/atf/), for which Kyua was originally
designed, but simple, framework-less test programs and TAP-compliant test
programs can also be executed through Kyua.
Kyua is licensed under a **[liberal BSD 3-clause license](LICENSE)**.
This is not an official Google product.
[Read more about Kyua in the About wiki page.](../../wiki/About)
Download
--------
The latest version of Kyua is 0.13 and was released on August 26th, 2016.
Download: [kyua-0.13](../../releases/tag/kyua-0.13).
See the [release notes](NEWS.md) for information about the changes in this
and all previous releases.
Installation
------------
You are encouraged to install binary packages for your operating system
wherever available:
* Fedora 20 and above: install the `kyua-cli` package with `yum install
kyua-cli`.
* FreeBSD 10.0 and above: install the `kyua` package with `pkg install kyua`.
* NetBSD with pkgsrc: install the `pkgsrc/devel/kyua` package.
* OpenBSD with packages: install the `kyua` package with `pkg_add kyua`.
* OS X (with Homebrew): install the `kyua` package with `brew install kyua`.
Should you want to build and install Kyua from the source tree provided
here, follow the instructions in the
[INSTALL.md file](INSTALL.md).
You should also install the ATF libraries to assist in the development of
test programs. To that end, see the
[ATF project page](https://github.com/jmmv/atf/).
Contributing
------------
Want to contribute? Great! But please first read the guidelines provided
in [CONTRIBUTING.md](CONTRIBUTING.md).
If you are curious about who made this project possible, you can check out
the [list of copyright holders](AUTHORS) and the [list of
individuals](CONTRIBUTORS).
Support
-------
Please use the [kyua-discuss mailing
list](https://groups.google.com/forum/#!forum/kyua-discuss) for any support
inquiries.
*Homepage:* https://github.com/jmmv/kyua/

6
admin/.gitignore vendored Normal file
View File

@ -0,0 +1,6 @@
ar-lib
compile
depcomp
install-sh
mdate-sh
missing

41
admin/Makefile.am.inc Normal file
View File

@ -0,0 +1,41 @@
# Copyright 2015 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
PHONY_TARGETS += check-style
check-style:
@$(srcdir)/admin/check-style.sh \
-b "$(abs_top_builddir)" \
-s "$(abs_top_srcdir)" \
-t "$(PACKAGE_TARNAME)"
EXTRA_DIST += admin/check-style-common.awk \
admin/check-style-cpp.awk \
admin/check-style-make.awk \
admin/check-style-man.awk \
admin/check-style-shell.awk \
admin/check-style.sh

131
admin/build-bintray-dist.sh Executable file
View File

@ -0,0 +1,131 @@
#! /bin/sh
# Copyright 2017 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# \file admin/build-bintray-dist.sh
# Builds a full Kyua installation under /usr/local for Ubuntu.
#
# This script is used to create the bintray distribution packages in lieu
# of real Debian packages for Kyua. The result of this script is a
# tarball that provides the contents of /usr/local for Kyua.
set -e -x
err() {
echo "${@}" 1>&2
exit 1
}
install_deps() {
sudo apt-get update -qq
local pkgsuffix=
local packages=
packages="${packages} autoconf"
packages="${packages} automake"
packages="${packages} clang"
packages="${packages} g++"
packages="${packages} gdb"
packages="${packages} git"
packages="${packages} libtool"
packages="${packages} make"
if [ "${ARCH?}" = i386 ]; then
pkgsuffix=:i386
packages="${packages} gcc-multilib"
packages="${packages} g++-multilib"
fi
packages="${packages} liblua5.2-0${pkgsuffix}"
packages="${packages} liblua5.2-dev${pkgsuffix}"
packages="${packages} libsqlite3-0${pkgsuffix}"
packages="${packages} libsqlite3-dev${pkgsuffix}"
packages="${packages} pkg-config${pkgsuffix}"
packages="${packages} sqlite3"
sudo apt-get install -y ${packages}
}
install_from_github() {
local name="${1}"; shift
local release="${1}"; shift
local distname="${name}-${release}"
local baseurl="https://github.com/jmmv/${name}"
wget --no-check-certificate \
"${baseurl}/releases/download/${distname}/${distname}.tar.gz"
tar -xzvf "${distname}.tar.gz"
local archflags=
[ "${ARCH?}" != i386 ] || archflags=-m32
cd "${distname}"
./configure \
--disable-developer \
--without-atf \
--without-doxygen \
CC="${CC?}" \
CFLAGS="${archflags}" \
CPPFLAGS="-I/usr/local/include" \
CXX="${CXX?}" \
CXXFLAGS="${archflags}" \
LDFLAGS="-L/usr/local/lib -Wl,-R/usr/local/lib" \
PKG_CONFIG_PATH="/usr/local/lib/pkgconfig"
make
sudo make install
cd -
rm -rf "${distname}" "${distname}.tar.gz"
}
main() {
[ "${ARCH+set}" = set ] || err "ARCH must be set in the environment"
[ "${CC+set}" = set ] || err "CC must be set in the environment"
[ "${CXX+set}" = set ] || err "CXX must be set in the environment"
[ ! -f /root/local.tgz ] || err "/root/local.tgz already exists"
tar -czf /root/local.tgz /usr/local
restore() {
rm -rf /usr/local
tar -xz -C / -f /root/local.tgz
rm /root/local.tgz
}
trap restore EXIT
rm -rf /usr/local
mkdir /usr/local
install_deps
install_from_github atf 0.21
install_from_github lutok 0.4
install_from_github kyua 0.13
local version="$(lsb_release -rs | cut -d . -f 1-2 | tr . -)"
local name="$(date +%Y%m%d)-usr-local-kyua"
name="${name}-ubuntu-${version}-${ARCH?}-${CC?}.tar.gz"
tar -czf "${name}" /usr/local
}
main "${@}"

72
admin/check-api-docs.awk Normal file
View File

@ -0,0 +1,72 @@
#! /bin/sh
# Copyright 2015 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
BEGIN {
failed = 0
}
# Skip empty lines.
/^$/ {next}
# Skip lines that do not directly reference a file.
/^[^\/]/ {next}
# Ignore known problems. As far as I can tell, all the cases listed here are
# well-documented in the code but Doxygen fails, for some reason or another, to
# properly locate the docstrings.
/engine\/kyuafile\.cpp.*no matching class member/ {next}
/engine\/scheduler\.hpp.*Member setup\(void\).*friend/ {next}
/engine\/scheduler\.hpp.*Member wait_any\(void\)/ {next}
/utils\/optional\.ipp.*no matching file member/ {next}
/utils\/optional\.hpp.*Member make_optional\(const T &\)/ {next}
/utils\/config\/nodes\.hpp.*Member set_lua\(lutok::state &, const int\)/ {next}
/utils\/config\/nodes\.hpp.*Member push_lua\(lutok::state &\)/ {next}
/utils\/config\/nodes\.hpp.*Member set_string\(const std::string &\)/ {next}
/utils\/config\/nodes\.hpp.*Member to_string\(void\)/ {next}
/utils\/config\/nodes\.hpp.*Member is_set\(void\)/ {next}
/utils\/process\/executor\.hpp.*Member spawn\(Hook.*\)/ {next}
/utils\/process\/executor\.hpp.*Member spawn_followup\(Hook.*\)/ {next}
/utils\/process\/executor\.hpp.*Member setup\(void\).*friend/ {next}
/utils\/signals\/timer\.hpp.*Member detail::invoke_do_fired.*friend/ {next}
/utils\/stacktrace_test\.cpp.*no matching class member/ {next}
# Dump any other problems and account for the failure.
{
failed = 1
print
}
END {
if (failed) {
print "ERROR: Unexpected docstring problems encountered"
exit 1
} else {
exit 0
}
}

79
admin/check-style-common.awk Normal file
View File

@ -0,0 +1,79 @@
# Copyright 2015 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
function warn(msg) {
print FILENAME "[" FNR "]: " msg > "/dev/stderr"
error = 1
}
BEGIN {
skip = 0
error = 0
}
/CHECK_STYLE_DISABLE/ {
skip = 1
next
}
/CHECK_STYLE_ENABLE/ {
skip = 0
next
}
/CHECK_STYLE_(ENABLE|DISABLE)/ {
next
}
{
if (skip)
next
if (length > 80 && NF > 1)
warn("Line too long to fit on screen")
}
/^ *\t+/ {
if (! match(FILENAME, "Makefile"))
warn("Tab character used for indentation");
}
/[ \t]+$/ {
warn("Trailing spaces or tabs");
}
/^#![^ ]/ {
warn("Missing space after #!");
}
END {
if (skip)
warn("Missing CHECK_STYLE_ENABLE");
if (error)
exit 1
}

87
admin/check-style-cpp.awk Normal file
View File

@ -0,0 +1,87 @@
# Copyright 2015 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
function warn(msg) {
print FILENAME "[" FNR "]: " msg > "/dev/stderr"
error = 1
}
BEGIN {
skip = 0
error = 0
}
/CHECK_STYLE_DISABLE/ {
skip = 1
next
}
/CHECK_STYLE_ENABLE/ {
skip = 0
next
}
/CHECK_STYLE_(ENABLE|DISABLE)/ {
next
}
{
if (skip)
next
}
/#ifdef/ {
warn("Undesired usage of #ifdef; use #if defined()")
}
/#ifndef/ {
warn("Undesired usage of #ifndef; use #if !defined()")
}
/assert[ \t]*\(/ {
warn("Use the macros in sanity.hpp instead of assert");
}
/#.*include.*assert/ {
warn("Do not include assert.h nor cassert");
}
/std::endl/ {
warn("Use \\n instead of std::endl");
}
/\/\*/ && ! /\*\// {
warn("Do not use multi-line C-style comments");
}
END {
if (skip)
warn("Missing CHECK_STYLE_ENABLE");
if (error)
exit 1
}

71
admin/check-style-make.awk Normal file
View File

@ -0,0 +1,71 @@
# Copyright 2015 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
function warn(msg) {
print FILENAME "[" FNR "]: " msg > "/dev/stderr"
error = 1
}
BEGIN {
skip = 0
error = 0
}
/CHECK_STYLE_DISABLE/ {
skip = 1
next
}
/CHECK_STYLE_ENABLE/ {
skip = 0
next
}
/CHECK_STYLE_(ENABLE|DISABLE)/ {
next
}
{
if (skip)
next
}
/^\t *\t/ {
warn("Continuation lines must use a single tab");
}
/mkdir.*-p/ {
warn("Use $(MKDIR_P) instead of mkdir -p");
}
END {
if (skip)
warn("Missing CHECK_STYLE_ENABLE");
if (error)
exit 1
}

71
admin/check-style-man.awk Normal file
View File

@ -0,0 +1,71 @@
# Copyright 2015 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
function warn(msg) {
print FILENAME "[" FNR "]: " msg > "/dev/stderr"
error = 1
}
BEGIN {
skip = 0
error = 0
}
/CHECK_STYLE_DISABLE|^\.Bd/ {
skip = 1
next
}
/CHECK_STYLE_ENABLE|^\.Ed/ {
skip = 0
next
}
/CHECK_STYLE_(ENABLE|DISABLE)/ {
next
}
/^\.\\"/ {
next
}
{
if (skip)
next
}
/\.\.|e\.g\.|i\.e\./ {
next
}
END {
if (skip)
warn("Missing CHECK_STYLE_ENABLE");
if (error)
exit 1
}

95
admin/check-style-shell.awk Normal file
View File

@ -0,0 +1,95 @@
# Copyright 2015 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
function warn(msg) {
print FILENAME "[" FNR "]: " msg > "/dev/stderr"
error = 1
}
BEGIN {
skip = 0
error = 0
}
/CHECK_STYLE_DISABLE/ {
skip = 1
next
}
/CHECK_STYLE_ENABLE/ {
skip = 0
next
}
/CHECK_STYLE_(ENABLE|DISABLE)/ {
next
}
{
if (skip)
next
}
/^[ \t]*#/ {
next
}
/[$ \t]+_[a-zA-Z0-9]+=/ {
warn("Variable should not start with an underline")
}
/[^\\]\$[^0-9!'"$?@#*{}(|\/,]+/ {
warn("Missing braces around variable name")
}
/=(""|'')/ {
warn("Assignment to the empty string does not need quotes");
}
/basename[ \t]+/ {
warn("Use parameter expansion instead of basename");
}
/if[ \t]+(test|![ \t]+test)/ {
warn("Use [ instead of test");
}
/[ \t]+(test|\[).*==/ {
warn("test(1)'s == operator is not portable");
}
/if.*;[ \t]*fi$/ {
warn("Avoid using a single-line if conditional");
}
END {
if (skip)
warn("Missing CHECK_STYLE_ENABLE");
if (error)
exit 1
}

170
admin/check-style.sh Executable file
View File

@ -0,0 +1,170 @@
#! /bin/sh
# Copyright 2011 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# \file admin/check-style.sh
#
# Sanity checks the coding style of all source files in the project tree.
ProgName="${0##*/}"
# Prints an error message and exits.
#
# \param ... Parts of the error message; concatenated using a space as the
# separator.
err() {
echo "${ProgName}:" "${@}" 1>&2
exit 1
}
# Locates all source files within the project directory.
#
# We require the project to have been configured in a directory that is separate
# from the source tree. This is to allow us to easily filter out build
# artifacts from our search.
#
# \param srcdir Absolute path to the source directory.
# \param builddir Absolute path to the build directory.
# \param tarname Basename of the project's tar file, to skip possible distfile
# directories.
find_sources() {
local srcdir="${1}"; shift
local builddir="${1}"; shift
local tarname="${1}"; shift
(
cd "${srcdir}"
find . -type f -a \
\! -path "*/.git/*" \
\! -path "*/.deps/*" \
\! -path "*/autom4te.cache/*" \
\! -path "*/${tarname}-[0-9]*/*" \
\! -path "*/${builddir##*/}/*" \
\! -name "Makefile.in" \
\! -name "aclocal.m4" \
\! -name "config.h.in" \
\! -name "configure" \
\! -name "testsuite"
)
}
# Prints the style rules applicable to a given file.
#
# \param file Path to the source file.
guess_rules() {
local file="${1}"; shift
case "${file}" in
*/ax_cxx_compile_stdcxx.m4) ;;
*/ltmain.sh) ;;
*Makefile*) echo common make ;;
*.[0-9]) echo common man ;;
*.cpp|*.hpp) echo common cpp ;;
*.sh) echo common shell ;;
*) echo common ;;
esac
}
# Validates a given file against the rules that apply to it.
#
# \param srcdir Absolute path to the source directory.
# \param file Name of the file to validate relative to srcdir.
#
# \return 0 if the file is valid; 1 otherwise, in which case the style
# violations are printed to the output.
check_file() {
local srcdir="${1}"; shift
local file="${1}"; shift
local err=0
for rule in $(guess_rules "${file}"); do
awk -f "${srcdir}/admin/check-style-${rule}.awk" \
"${srcdir}/${file}" || err=1
done
return ${err}
}
# Entry point.
main() {
local builddir=.
local srcdir=.
local tarname=UNKNOWN
local arg
while getopts :b:s:t: arg; do
case "${arg}" in
b)
builddir="${OPTARG}"
;;
s)
srcdir="${OPTARG}"
;;
t)
tarname="${OPTARG}"
;;
\?)
err "Unknown option -${OPTARG}"
;;
esac
done
shift $(expr ${OPTIND} - 1)
srcdir="$(cd "${srcdir}" && pwd -P)"
builddir="$(cd "${builddir}" && pwd -P)"
[ "${srcdir}" != "${builddir}" ] || \
err "srcdir and builddir cannot match; reconfigure the package" \
"in a separate directory"
local sources
if [ ${#} -gt 0 ]; then
sources="${@}"
else
sources="$(find_sources "${srcdir}" "${builddir}" "${tarname}")"
fi
local ok=0
for file in ${sources}; do
local file="$(echo ${file} | sed -e "s,\\./,,")"
check_file "${srcdir}" "${file}" || ok=1
done
return "${ok}"
}
main "${@}"

90
admin/clean-all.sh Executable file
View File

@ -0,0 +1,90 @@
#! /bin/sh
# Copyright 2010 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Prog_Name=${0##*/}
if [ ! -f ./main.cpp ]; then
echo "${Prog_Name}: must be run from the source top directory" 1>&2
exit 1
fi
if [ ! -f configure ]; then
echo "${Prog_Name}: configure not found; nothing to clean?" 1>&2
exit 1
fi
[ -f Makefile ] || ./configure
make distclean
# Top-level directory.
rm -f Makefile.in
rm -f aclocal.m4
rm -rf autom4te.cache
rm -f config.h.in
rm -f configure
rm -f mkinstalldirs
rm -f kyua-*.tar.gz
# admin directory.
rm -f admin/ar-lib
rm -f admin/compile
rm -f admin/config.guess
rm -f admin/config.sub
rm -f admin/depcomp
rm -f admin/install-sh
rm -f admin/ltmain.sh
rm -f admin/mdate-sh
rm -f admin/missing
# bootstrap directory.
rm -f bootstrap/package.m4
rm -f bootstrap/testsuite
# doc directory.
rm -f doc/*.info
rm -f doc/stamp-vti
rm -f doc/version.texi
# m4 directory.
rm -f m4/libtool.m4
rm -f m4/lt*.m4
# Files and directories spread all around the tree.
find . -name '#*' | xargs rm -rf
find . -name '*~' | xargs rm -rf
find . -name .deps | xargs rm -rf
find . -name .gdb_history | xargs rm -rf
find . -name .libs | xargs rm -rf
find . -name .tmp | xargs rm -rf
# Show remaining files.
if [ -n "${GIT}" ]; then
echo ">>> untracked and ignored files"
"${GIT}" status --porcelain --ignored | grep -E '^(\?\?|!!)' || true
fi

98
admin/travis-build.sh Executable file
View File

@ -0,0 +1,98 @@
#! /bin/sh
# Copyright 2014 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
set -e -x
run_autoreconf() {
if [ -d /usr/local/share/aclocal ]; then
autoreconf -isv -I/usr/local/share/aclocal
else
autoreconf -isv
fi
}
do_apidocs() {
run_autoreconf || return 1
./configure --with-doxygen || return 1
make check-api-docs
}
do_distcheck() {
run_autoreconf || return 1
./configure || return 1
sudo sysctl -w "kernel.core_pattern=core.%p"
local archflags=
[ "${ARCH?}" != i386 ] || archflags=-m32
cat >kyua.conf <<EOF
syntax(2)
-- We do not know how many CPUs the test machine has. However, parallelizing
-- the execution of our tests to _any_ degree speeds up the time it takes to
-- complete a test run because many of our tests are blocking.
parallelism = 4
EOF
[ "${UNPRIVILEGED_USER:-no}" = no ] || \
echo "unprivileged_user = 'travis'" >>kyua.conf
local f=
f="${f} CFLAGS='${archflags}'"
f="${f} CPPFLAGS='-I/usr/local/include'"
f="${f} CXXFLAGS='${archflags}'"
f="${f} LDFLAGS='-L/usr/local/lib -Wl,-R/usr/local/lib'"
f="${f} PKG_CONFIG_PATH='/usr/local/lib/pkgconfig'"
f="${f} KYUA_CONFIG_FILE_FOR_CHECK=$(pwd)/kyua.conf"
if [ "${AS_ROOT:-no}" = yes ]; then
sudo -H PATH="${PATH}" make distcheck DISTCHECK_CONFIGURE_FLAGS="${f}"
else
make distcheck DISTCHECK_CONFIGURE_FLAGS="${f}"
fi
}
do_style() {
run_autoreconf || return 1
mkdir build
cd build
../configure || return 1
make check-style
}
main() {
if [ -z "${DO}" ]; then
echo "DO must be defined" 1>&2
exit 1
fi
for step in ${DO}; do
"do_${DO}" || exit 1
done
}
main "${@}"

83
admin/travis-install-deps.sh Executable file
View File

@ -0,0 +1,83 @@
#! /bin/sh
# Copyright 2014 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
set -e -x
install_deps() {
local pkgsuffix=
local packages=
if [ "${ARCH?}" = i386 ]; then
pkgsuffix=:i386
packages="${packages} gcc-multilib"
packages="${packages} g++-multilib"
sudo dpkg --add-architecture i386
fi
packages="${packages} gdb"
packages="${packages} liblua5.2-0${pkgsuffix}"
packages="${packages} liblua5.2-dev${pkgsuffix}"
packages="${packages} libsqlite3-0${pkgsuffix}"
packages="${packages} libsqlite3-dev${pkgsuffix}"
packages="${packages} pkg-config${pkgsuffix}"
packages="${packages} sqlite3"
sudo apt-get update -qq
sudo apt-get install -y ${packages}
}
install_kyua() {
local name="20190321-usr-local-kyua-ubuntu-16-04-${ARCH?}-${CC?}.tar.gz"
wget -O "${name}" "http://dl.bintray.com/ngie-eign/kyua/${name}" || return 1
sudo tar -xzvp -C / -f "${name}"
rm -f "${name}"
}
do_apidocs() {
sudo apt-get install -y doxygen
}
do_distcheck() {
:
}
do_style() {
:
}
main() {
if [ -z "${DO}" ]; then
echo "DO must be defined" 1>&2
exit 1
fi
install_deps
install_kyua
for step in ${DO}; do
"do_${DO}" || exit 1
done
}
main "${@}"

4
bootstrap/.gitignore vendored Normal file
View File

@ -0,0 +1,4 @@
atconfig
package.m4
testsuite
testsuite.log

5
bootstrap/Kyuafile Normal file
View File

@ -0,0 +1,5 @@
syntax(2)
test_suite("kyua")
plain_test_program{name="testsuite"}

90
bootstrap/Makefile.am.inc Normal file
View File

@ -0,0 +1,90 @@
# Copyright 2010 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
if WITH_ATF
tests_bootstrapdir = $(pkgtestsdir)/bootstrap
tests_bootstrap_DATA = bootstrap/Kyuafile
EXTRA_DIST += $(tests_bootstrap_DATA)
DISTCLEANFILES = bootstrap/atconfig \
bootstrap/testsuite.lineno \
bootstrap/testsuite.log
distclean-local: distclean-testsuite
distclean-testsuite:
-rm -rf bootstrap/testsuite.dir
EXTRA_DIST += bootstrap/Kyuafile \
bootstrap/testsuite \
bootstrap/package.m4 \
bootstrap/testsuite.at
tests_bootstrap_PROGRAMS = bootstrap/atf_helpers
bootstrap_atf_helpers_SOURCES = bootstrap/atf_helpers.cpp
bootstrap_atf_helpers_CXXFLAGS = $(ATF_CXX_CFLAGS)
bootstrap_atf_helpers_LDADD = $(ATF_CXX_LIBS)
tests_bootstrap_PROGRAMS += bootstrap/plain_helpers
bootstrap_plain_helpers_SOURCES = bootstrap/plain_helpers.cpp
bootstrap_plain_helpers_CXXFLAGS = $(UTILS_CFLAGS)
tests_bootstrap_SCRIPTS = bootstrap/testsuite
@target_srcdir@bootstrap/package.m4: $(top_srcdir)/configure.ac
$(AM_V_GEN){ \
echo '# Signature of the current package.'; \
echo 'm4_define(AT_PACKAGE_NAME, @PACKAGE_NAME@)'; \
echo 'm4_define(AT_PACKAGE_TARNAME, @PACKAGE_TARNAME@)'; \
echo 'm4_define(AT_PACKAGE_VERSION, @PACKAGE_VERSION@)'; \
echo 'm4_define(AT_PACKAGE_STRING, @PACKAGE_STRING@)'; \
echo 'm4_define(AT_PACKAGE_BUGREPORT, @PACKAGE_BUGREPORT@)'; \
} >$(srcdir)/bootstrap/package.m4
@target_srcdir@bootstrap/testsuite: $(srcdir)/bootstrap/testsuite.at \
@target_srcdir@bootstrap/package.m4
$(AM_V_GEN)autom4te --language=Autotest -I $(srcdir) \
-I $(srcdir)/bootstrap \
$(srcdir)/bootstrap/testsuite.at -o $@.tmp; \
mv $@.tmp $@
CHECK_LOCAL += check-bootstrap
PHONY_TARGETS += check-bootstrap
check-bootstrap: @target_srcdir@bootstrap/testsuite $(check_PROGRAMS) \
$(CHECK_BOOTSTRAP_DEPS)
cd bootstrap && $(CHECK_ENVIRONMENT) $(TESTS_ENVIRONMENT) \
./testsuite
if !TARGET_SRCDIR_EMPTY
CHECK_BOOTSTRAP_DEPS += copy-bootstrap-testsuite
CHECK_KYUA_DEPS += copy-bootstrap-testsuite
PHONY_TARGETS += copy-bootstrap-testsuite
copy-bootstrap-testsuite:
cp -f @target_srcdir@bootstrap/testsuite bootstrap/testsuite
CLEANFILES += bootstrap/testsuite
endif
endif

71
bootstrap/atf_helpers.cpp Normal file
View File

@ -0,0 +1,71 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include <cstdlib>
#include <string>
#include <atf-c++.hpp>
ATF_TEST_CASE_WITHOUT_HEAD(fails);
ATF_TEST_CASE_BODY(fails)
{
fail("Failed on purpose");
}
ATF_TEST_CASE_WITHOUT_HEAD(passes);
ATF_TEST_CASE_BODY(passes)
{
}
ATF_TEST_CASE_WITHOUT_HEAD(skips);
ATF_TEST_CASE_BODY(skips)
{
skip("Skipped on purpose");
}
ATF_INIT_TEST_CASES(tcs)
{
std::string enabled;
const char* tests = std::getenv("TESTS");
if (tests == NULL)
enabled = "fails passes skips";
else
enabled = tests;
if (enabled.find("fails") != std::string::npos)
ATF_ADD_TEST_CASE(tcs, fails);
if (enabled.find("passes") != std::string::npos)
ATF_ADD_TEST_CASE(tcs, passes);
if (enabled.find("skips") != std::string::npos)
ATF_ADD_TEST_CASE(tcs, skips);
}

141
bootstrap/plain_helpers.cpp Normal file
View File

@ -0,0 +1,141 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include <cstdlib>
#include <cstring>
#include <iostream>
#include "utils/defs.hpp"
#include "utils/test_utils.ipp"
namespace {
/// Prints a fake but valid test case list and then aborts.
///
/// \param argv The original arguments of the program.
///
/// \return Nothing because this dies before returning.
static int
helper_abort_test_cases_list(int /* argc */, char** argv)
{
for (const char* const* arg = argv; *arg != NULL; arg++) {
if (std::strcmp(*arg, "-l") == 0) {
std::cout << "Content-Type: application/X-atf-tp; "
"version=\"1\"\n\n";
std::cout << "ident: foo\n";
}
}
utils::abort_without_coredump();
}
/// Just returns without printing anything as the test case list.
///
/// \return Always 0, as required for test programs.
static int
helper_empty_test_cases_list(int /* argc */, char** /* argv */)
{
return EXIT_SUCCESS;
}
/// Prints a correctly-formatted test case list but empty.
///
/// \param argv The original arguments of the program.
///
/// \return Always 0, as required for test programs.
static int
helper_zero_test_cases(int /* argc */, char** argv)
{
for (const char* const* arg = argv; *arg != NULL; arg++) {
if (std::strcmp(*arg, "-l") == 0)
std::cout << "Content-Type: application/X-atf-tp; "
"version=\"1\"\n\n";
}
return EXIT_SUCCESS;
}
/// Mapping of the name of a helper to its implementation.
struct helper {
/// The name of the helper, as will be provided by the user on the CLI.
const char* name;
/// A pointer to the function implementing the helper.
int (*hook)(int, char**);
};
/// NULL-terminated table mapping helper names to their implementations.
static const helper helpers[] = {
{ "abort_test_cases_list", helper_abort_test_cases_list, },
{ "empty_test_cases_list", helper_empty_test_cases_list, },
{ "zero_test_cases", helper_zero_test_cases, },
{ NULL, NULL, },
};
} // anonymous namespace
/// Entry point to the ATF-less helpers.
///
/// The caller must select a helper to execute by defining the HELPER
/// environment variable to the name of the desired helper. Think of this main
/// method as a subprogram dispatcher, to avoid having many individual helper
/// binaries.
///
/// \todo Maybe we should really have individual helper binaries. It would
/// avoid a significant amount of complexity here and in the tests, at the
/// expense of some extra files and extra build logic.
///
/// \param argc The user argument count; delegated to the helper.
/// \param argv The user arguments; delegated to the helper.
///
/// \return The exit code of the helper, which depends on the requested helper.
int
main(int argc, char** argv)
{
const char* command = std::getenv("HELPER");
if (command == NULL) {
std::cerr << "Usage error: HELPER must be set to a helper name\n";
std::exit(EXIT_FAILURE);
}
const struct helper* iter = helpers;
for (; iter->name != NULL && std::strcmp(iter->name, command) != 0; iter++)
;
if (iter->name == NULL) {
std::cerr << "Usage error: unknown command " << command << "\n";
std::exit(EXIT_FAILURE);
}
return iter->hook(argc, argv);
}

200
bootstrap/testsuite.at Normal file
View File

@ -0,0 +1,200 @@
dnl Copyright 2010 The Kyua Authors.
dnl All rights reserved.
dnl
dnl Redistribution and use in source and binary forms, with or without
dnl modification, are permitted provided that the following conditions are
dnl met:
dnl
dnl * Redistributions of source code must retain the above copyright
dnl notice, this list of conditions and the following disclaimer.
dnl * Redistributions in binary form must reproduce the above copyright
dnl notice, this list of conditions and the following disclaimer in the
dnl documentation and/or other materials provided with the distribution.
dnl * Neither the name of Google Inc. nor the names of its contributors
dnl may be used to endorse or promote products derived from this software
dnl without specific prior written permission.
dnl
dnl THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
dnl "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
dnl LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
dnl A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
dnl OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
dnl SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
dnl LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
dnl DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
dnl THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
dnl (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
dnl OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
AT_INIT([bootstrapping tests])
m4_define([GUESS_TOPDIR], {
old=$(pwd)
cd "$(dirname ${as_myself})"
# We need to locate a build product, not a source file, because the
# test suite may be run outside of the source tree (think distcheck).
while test $(pwd) != '/' -a ! -e bootstrap/plain_helpers; do
cd ..
done
topdir=$(pwd)
cd ${old}
echo ${topdir}
})
m4_define([CREATE_ATF_HELPERS], [
AT_DATA([Kyuafile], [
syntax(2)
test_suite("bootstrap")
atf_test_program{name="atf_helpers"}
])
ln -s $(GUESS_TOPDIR)/bootstrap/atf_helpers atf_helpers
])
m4_define([RUN_ATF_HELPERS],
[HOME=$(pwd) TESTS="$1" kyua --config=none \
test --results-file=bootstrap.db $2])
m4_define([CREATE_PLAIN_HELPERS], [
AT_DATA([Kyuafile], [
syntax(2)
test_suite("bootstrap")
atf_test_program{name="plain_helpers"}
])
ln -s $(GUESS_TOPDIR)/bootstrap/plain_helpers plain_helpers
])
m4_define([RUN_PLAIN_HELPER],
[HOME=$(pwd) HELPER="$1" kyua --config=none \
test --results-file=bootstrap.db])
AT_SETUP([test program crashes in test list])
AT_TESTED([kyua])
CREATE_PLAIN_HELPERS
AT_CHECK([RUN_PLAIN_HELPER([abort_test_cases_list])], [1], [stdout], [])
re='plain_helpers:__test_cases_list__.*broken.*Test program received signal'
AT_CHECK([grep "${re}" stdout], [0], [ignore], [])
AT_CLEANUP
AT_SETUP([test program prints an empty test list])
AT_TESTED([kyua])
CREATE_PLAIN_HELPERS
AT_CHECK([RUN_PLAIN_HELPER([empty_test_cases_list])], [1], [stdout], [])
re="plain_helpers:__test_cases_list__.*broken.*Invalid header.*got ''"
AT_CHECK([grep "${re}" stdout], [0], [ignore], [])
AT_CLEANUP
AT_SETUP([test program with zero test cases])
AT_TESTED([kyua])
CREATE_PLAIN_HELPERS
AT_CHECK([RUN_PLAIN_HELPER([zero_test_cases])], [1], [stdout], [])
re='plain_helpers:__test_cases_list__.*broken.*No test cases'
AT_CHECK([grep "${re}" stdout], [0], [ignore], [])
AT_CLEANUP
AT_SETUP([run test case that passes])
AT_TESTED([kyua])
CREATE_ATF_HELPERS
AT_CHECK([RUN_ATF_HELPERS([passes])], [0], [stdout], [])
AT_CHECK([grep "atf_helpers:fails" stdout], [1], [], [])
AT_CHECK([grep "atf_helpers:passes.*passed" stdout], [0], [ignore], [])
AT_CHECK([grep "atf_helpers:skips" stdout], [1], [], [])
AT_CLEANUP
AT_SETUP([run test case that fails])
AT_TESTED([kyua])
CREATE_ATF_HELPERS
AT_CHECK([RUN_ATF_HELPERS([fails])], [1], [stdout], [])
AT_CHECK([grep "atf_helpers:fails.*failed.*Failed on purpose" stdout],
[0], [ignore], [])
AT_CHECK([grep "atf_helpers:passes" stdout], [1], [], [])
AT_CHECK([grep "atf_helpers:skips" stdout], [1], [], [])
AT_CLEANUP
AT_SETUP([run test case that skips])
AT_TESTED([kyua])
CREATE_ATF_HELPERS
AT_CHECK([RUN_ATF_HELPERS([skips])], [0], [stdout], [])
AT_CHECK([grep "atf_helpers:fails" stdout], [1], [], [])
AT_CHECK([grep "atf_helpers:passes" stdout], [1], [], [])
AT_CHECK([grep "atf_helpers:skips.*skipped.*Skipped on purpose" stdout],
[0], [ignore], [])
AT_CLEANUP
AT_SETUP([run two test cases, success])
AT_TESTED([kyua])
CREATE_ATF_HELPERS
AT_CHECK([RUN_ATF_HELPERS([passes skips])], [0], [stdout], [])
AT_CHECK([grep "atf_helpers:fails" stdout], [1], [], [])
AT_CHECK([grep "atf_helpers:passes.*passed" stdout], [0], [ignore], [])
AT_CHECK([grep "atf_helpers:skips.*skipped.*Skipped on purpose" stdout],
[0], [ignore], [])
AT_CLEANUP
AT_SETUP([run two test cases, failure])
AT_TESTED([kyua])
CREATE_ATF_HELPERS
AT_CHECK([RUN_ATF_HELPERS([fails passes])], [1], [stdout], [])
AT_CHECK([grep "atf_helpers:fails.*failure.*Failed on purpose" stdout],
[1], [], [])
AT_CHECK([grep "atf_helpers:passes.*passed" stdout], [0], [ignore], [])
AT_CHECK([grep "atf_helpers:skips" stdout], [1], [], [])
AT_CLEANUP
AT_SETUP([run mixed test cases])
AT_TESTED([kyua])
CREATE_ATF_HELPERS
AT_CHECK([RUN_ATF_HELPERS([fails passes skips])], [1], [stdout], [])
AT_CHECK([grep "atf_helpers:fails.*failure.*Failed on purpose" stdout],
[1], [], [])
AT_CHECK([grep "atf_helpers:passes.*passed" stdout], [0], [ignore], [])
AT_CHECK([grep "atf_helpers:skips.*skipped.*Skipped on purpose" stdout],
[0], [ignore], [])
AT_CLEANUP
AT_SETUP([run tests from build directories])
AT_TESTED([kyua])
CREATE_ATF_HELPERS
AT_CHECK([mkdir src], [0], [], [])
AT_CHECK([mv Kyuafile src], [0], [], [])
AT_CHECK([mkdir obj], [0], [], [])
AT_CHECK([mv atf_helpers obj], [0], [], [])
AT_CHECK([RUN_ATF_HELPERS([fails passes skips],
[--kyuafile=src/Kyuafile --build-root=obj])],
[1], [stdout], [])
AT_CHECK([grep "atf_helpers:fails.*failure.*Failed on purpose" stdout],
[1], [], [])
AT_CHECK([grep "atf_helpers:passes.*passed" stdout], [0], [ignore], [])
AT_CHECK([grep "atf_helpers:skips.*skipped.*Skipped on purpose" stdout],
[0], [ignore], [])
AT_CLEANUP

14
cli/Kyuafile Normal file
View File

@ -0,0 +1,14 @@
syntax(2)
test_suite("kyua")
atf_test_program{name="cmd_about_test"}
atf_test_program{name="cmd_config_test"}
atf_test_program{name="cmd_db_exec_test"}
atf_test_program{name="cmd_debug_test"}
atf_test_program{name="cmd_help_test"}
atf_test_program{name="cmd_list_test"}
atf_test_program{name="cmd_test_test"}
atf_test_program{name="common_test"}
atf_test_program{name="config_test"}
atf_test_program{name="main_test"}

123
cli/Makefile.am.inc Normal file
View File

@ -0,0 +1,123 @@
# Copyright 2010 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
CLI_CFLAGS = $(DRIVERS_CFLAGS)
CLI_LIBS = libcli.a $(DRIVERS_LIBS)
noinst_LIBRARIES += libcli.a
libcli_a_SOURCES = cli/cmd_about.cpp
libcli_a_SOURCES += cli/cmd_about.hpp
libcli_a_SOURCES += cli/cmd_config.cpp
libcli_a_SOURCES += cli/cmd_config.hpp
libcli_a_SOURCES += cli/cmd_db_exec.cpp
libcli_a_SOURCES += cli/cmd_db_exec.hpp
libcli_a_SOURCES += cli/cmd_db_migrate.cpp
libcli_a_SOURCES += cli/cmd_db_migrate.hpp
libcli_a_SOURCES += cli/cmd_debug.cpp
libcli_a_SOURCES += cli/cmd_debug.hpp
libcli_a_SOURCES += cli/cmd_help.cpp
libcli_a_SOURCES += cli/cmd_help.hpp
libcli_a_SOURCES += cli/cmd_list.cpp
libcli_a_SOURCES += cli/cmd_list.hpp
libcli_a_SOURCES += cli/cmd_report.cpp
libcli_a_SOURCES += cli/cmd_report.hpp
libcli_a_SOURCES += cli/cmd_report_html.cpp
libcli_a_SOURCES += cli/cmd_report_html.hpp
libcli_a_SOURCES += cli/cmd_report_junit.cpp
libcli_a_SOURCES += cli/cmd_report_junit.hpp
libcli_a_SOURCES += cli/cmd_test.cpp
libcli_a_SOURCES += cli/cmd_test.hpp
libcli_a_SOURCES += cli/common.cpp
libcli_a_SOURCES += cli/common.hpp
libcli_a_SOURCES += cli/common.ipp
libcli_a_SOURCES += cli/config.cpp
libcli_a_SOURCES += cli/config.hpp
libcli_a_SOURCES += cli/main.cpp
libcli_a_SOURCES += cli/main.hpp
libcli_a_CPPFLAGS = -DKYUA_CONFDIR="\"$(kyua_confdir)\""
libcli_a_CPPFLAGS += -DKYUA_DOCDIR="\"$(docdir)\""
libcli_a_CPPFLAGS += -DKYUA_MISCDIR="\"$(miscdir)\""
libcli_a_CPPFLAGS += $(DRIVERS_CFLAGS)
libcli_a_LIBADD = libutils.a
if WITH_ATF
tests_clidir = $(pkgtestsdir)/cli
tests_cli_DATA = cli/Kyuafile
EXTRA_DIST += $(tests_cli_DATA)
tests_cli_PROGRAMS = cli/cmd_about_test
cli_cmd_about_test_SOURCES = cli/cmd_about_test.cpp
cli_cmd_about_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_cmd_about_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/cmd_config_test
cli_cmd_config_test_SOURCES = cli/cmd_config_test.cpp
cli_cmd_config_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_cmd_config_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/cmd_db_exec_test
cli_cmd_db_exec_test_SOURCES = cli/cmd_db_exec_test.cpp
cli_cmd_db_exec_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_cmd_db_exec_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/cmd_debug_test
cli_cmd_debug_test_SOURCES = cli/cmd_debug_test.cpp
cli_cmd_debug_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_cmd_debug_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/cmd_help_test
cli_cmd_help_test_SOURCES = cli/cmd_help_test.cpp
cli_cmd_help_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_cmd_help_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/cmd_list_test
cli_cmd_list_test_SOURCES = cli/cmd_list_test.cpp
cli_cmd_list_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_cmd_list_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/cmd_test_test
cli_cmd_test_test_SOURCES = cli/cmd_test_test.cpp
cli_cmd_test_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_cmd_test_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/common_test
cli_common_test_SOURCES = cli/common_test.cpp
cli_common_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_common_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/config_test
cli_config_test_SOURCES = cli/config_test.cpp
cli_config_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_config_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
tests_cli_PROGRAMS += cli/main_test
cli_main_test_SOURCES = cli/main_test.cpp
cli_main_test_CXXFLAGS = $(CLI_CFLAGS) $(ATF_CXX_CFLAGS)
cli_main_test_LDADD = $(CLI_LIBS) $(ATF_CXX_LIBS)
endif

160
cli/cmd_about.cpp Normal file
View File

@ -0,0 +1,160 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_about.hpp"
#include <cstdlib>
#include <fstream>
#include <utility>
#include <vector>
#include "cli/common.ipp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/defs.hpp"
#include "utils/env.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/path.hpp"
#include "utils/sanity.hpp"
#include "utils/text/regex.hpp"
#if defined(HAVE_CONFIG_H)
# include "config.h"
#endif
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace text = utils::text;
using cli::cmd_about;
namespace {
/// Print the contents of a document.
///
/// If the file cannot be opened for whatever reason, an error message is
/// printed to the output of the program instead of the contents of the file.
///
/// \param ui Object to interact with the I/O of the program.
/// \param file The file to print.
/// \param filter_re Regular expression to match the lines to print. If empty,
/// no filtering is applied.
///
/// \return True if the file was printed, false otherwise.
static bool
cat_file(cmdline::ui* ui, const fs::path& file,
const std::string& filter_re = "")
{
std::ifstream input(file.c_str());
if (!input) {
ui->err(F("Failed to open %s") % file);
return false;
}
std::string line;
if (filter_re.empty()) {
while (std::getline(input, line).good()) {
ui->out(line);
}
} else {
const text::regex filter = text::regex::compile(filter_re, 0);
while (std::getline(input, line).good()) {
if (filter.match(line)) {
ui->out(line);
}
}
}
input.close();
return true;
}
} // anonymous namespace
/// Default constructor for cmd_about.
cmd_about::cmd_about(void) : cli_command(
"about", "[authors|license|version]", 0, 1,
"Shows detailed authors and contributors; license; and version information")
{
}
/// Entry point for the "about" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
///
/// \return 0 if everything is OK, 1 if any of the necessary documents cannot be
/// opened.
int
cmd_about::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& /* user_config */)
{
const fs::path docdir(utils::getenv_with_default(
"KYUA_DOCDIR", KYUA_DOCDIR));
bool success = true;
static const char* list_re = "^\\* ";
if (cmdline.arguments().empty()) {
ui->out(PACKAGE " (" PACKAGE_NAME ") " PACKAGE_VERSION);
ui->out("");
ui->out("License terms:");
ui->out("");
success &= cat_file(ui, docdir / "LICENSE");
ui->out("");
ui->out("Brought to you by:");
ui->out("");
success &= cat_file(ui, docdir / "AUTHORS", list_re);
ui->out("");
success &= cat_file(ui, docdir / "CONTRIBUTORS", list_re);
ui->out("");
ui->out(F("Homepage: %s") % PACKAGE_URL);
} else {
const std::string& topic = cmdline.arguments()[0];
if (topic == "authors") {
success &= cat_file(ui, docdir / "AUTHORS", list_re);
success &= cat_file(ui, docdir / "CONTRIBUTORS", list_re);
} else if (topic == "license") {
success &= cat_file(ui, docdir / "LICENSE");
} else if (topic == "version") {
write_version_header(ui);
} else {
throw cmdline::usage_error(F("Invalid about topic '%s'") % topic);
}
}
return success ? EXIT_SUCCESS : EXIT_FAILURE;
}

57
cli/cmd_about.hpp Normal file
View File

@ -0,0 +1,57 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_about.hpp
/// Provides the cmd_about class.
#if !defined(CLI_CMD_ABOUT_HPP)
#define CLI_CMD_ABOUT_HPP
#include "cli/common.hpp"
namespace cli {
/// Implementation of the "about" subcommand.
class cmd_about : public cli_command
{
/// Path to the directory containing the distribution documents.
const std::string _docdir;
public:
cmd_about(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_ABOUT_HPP)

306
cli/cmd_about_test.cpp Normal file
View File

@ -0,0 +1,306 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_about.hpp"
#if defined(HAVE_CONFIG_H)
# include "config.h"
#endif
#include <cstdlib>
#include <atf-c++.hpp>
#include "cli/common.ipp"
#include "engine/config.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/config/tree.ipp"
#include "utils/env.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
namespace cmdline = utils::cmdline;
namespace fs = utils::fs;
using cli::cmd_about;
ATF_TEST_CASE_WITHOUT_HEAD(all_topics__ok);
ATF_TEST_CASE_BODY(all_topics__ok)
{
cmdline::args_vector args;
args.push_back("about");
fs::mkdir(fs::path("fake-docs"), 0755);
atf::utils::create_file("fake-docs/AUTHORS",
"Content of AUTHORS\n"
"* First author\n"
" * garbage\n"
"* Second author\n");
atf::utils::create_file("fake-docs/CONTRIBUTORS",
"Content of CONTRIBUTORS\n"
"* First contributor\n"
" * garbage\n"
"* Second contributor\n");
atf::utils::create_file("fake-docs/LICENSE", "Content of LICENSE\n");
utils::setenv("KYUA_DOCDIR", "fake-docs");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(atf::utils::grep_string(PACKAGE_NAME, ui.out_log()[0]));
ATF_REQUIRE(atf::utils::grep_string(PACKAGE_VERSION, ui.out_log()[0]));
ATF_REQUIRE(!atf::utils::grep_collection("Content of AUTHORS",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* First author", ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("garbage", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* Second author", ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("Content of CONTRIBUTORS",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* First contributor",
ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("garbage", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* Second contributor",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("Content of LICENSE",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("Homepage", ui.out_log()));
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(all_topics__missing_docs);
ATF_TEST_CASE_BODY(all_topics__missing_docs)
{
cmdline::args_vector args;
args.push_back("about");
utils::setenv("KYUA_DOCDIR", "fake-docs");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_FAILURE, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(atf::utils::grep_string(PACKAGE_NAME, ui.out_log()[0]));
ATF_REQUIRE(atf::utils::grep_string(PACKAGE_VERSION, ui.out_log()[0]));
ATF_REQUIRE(atf::utils::grep_collection("Homepage", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("Failed to open.*AUTHORS",
ui.err_log()));
ATF_REQUIRE(atf::utils::grep_collection("Failed to open.*CONTRIBUTORS",
ui.err_log()));
ATF_REQUIRE(atf::utils::grep_collection("Failed to open.*LICENSE",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(topic_authors__ok);
ATF_TEST_CASE_BODY(topic_authors__ok)
{
cmdline::args_vector args;
args.push_back("about");
args.push_back("authors");
fs::mkdir(fs::path("fake-docs"), 0755);
atf::utils::create_file("fake-docs/AUTHORS",
"Content of AUTHORS\n"
"* First author\n"
" * garbage\n"
"* Second author\n");
atf::utils::create_file("fake-docs/CONTRIBUTORS",
"Content of CONTRIBUTORS\n"
"* First contributor\n"
" * garbage\n"
"* Second contributor\n");
utils::setenv("KYUA_DOCDIR", "fake-docs");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(!atf::utils::grep_string(PACKAGE_NAME, ui.out_log()[0]));
ATF_REQUIRE(!atf::utils::grep_collection("Content of AUTHORS",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* First author", ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("garbage", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* Second author", ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("Content of CONTRIBUTORS",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* First contributor",
ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("garbage", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("\\* Second contributor",
ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("LICENSE", ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("Homepage", ui.out_log()));
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(topic_authors__missing_doc);
ATF_TEST_CASE_BODY(topic_authors__missing_doc)
{
cmdline::args_vector args;
args.push_back("about");
args.push_back("authors");
utils::setenv("KYUA_DOCDIR", "fake-docs");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_FAILURE, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE_EQ(0, ui.out_log().size());
ATF_REQUIRE(atf::utils::grep_collection("Failed to open.*AUTHORS",
ui.err_log()));
ATF_REQUIRE(atf::utils::grep_collection("Failed to open.*CONTRIBUTORS",
ui.err_log()));
ATF_REQUIRE(!atf::utils::grep_collection("Failed to open.*LICENSE",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(topic_license__ok);
ATF_TEST_CASE_BODY(topic_license__ok)
{
cmdline::args_vector args;
args.push_back("about");
args.push_back("license");
fs::mkdir(fs::path("fake-docs"), 0755);
atf::utils::create_file("fake-docs/LICENSE", "Content of LICENSE\n");
utils::setenv("KYUA_DOCDIR", "fake-docs");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(!atf::utils::grep_string(PACKAGE_NAME, ui.out_log()[0]));
ATF_REQUIRE(!atf::utils::grep_collection("AUTHORS", ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("CONTRIBUTORS", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("Content of LICENSE",
ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection("Homepage", ui.out_log()));
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(topic_license__missing_doc);
ATF_TEST_CASE_BODY(topic_license__missing_doc)
{
cmdline::args_vector args;
args.push_back("about");
args.push_back("license");
utils::setenv("KYUA_DOCDIR", "fake-docs");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_FAILURE, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE_EQ(0, ui.out_log().size());
ATF_REQUIRE(!atf::utils::grep_collection("Failed to open.*AUTHORS",
ui.err_log()));
ATF_REQUIRE(!atf::utils::grep_collection("Failed to open.*CONTRIBUTORS",
ui.err_log()));
ATF_REQUIRE(atf::utils::grep_collection("Failed to open.*LICENSE",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(topic_version__ok);
ATF_TEST_CASE_BODY(topic_version__ok)
{
cmdline::args_vector args;
args.push_back("about");
args.push_back("version");
utils::setenv("KYUA_DOCDIR", "fake-docs");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE_EQ(1, ui.out_log().size());
ATF_REQUIRE(atf::utils::grep_string(PACKAGE_NAME, ui.out_log()[0]));
ATF_REQUIRE(atf::utils::grep_string(PACKAGE_VERSION, ui.out_log()[0]));
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(invalid_args);
ATF_TEST_CASE_BODY(invalid_args)
{
cmdline::args_vector args;
args.push_back("about");
args.push_back("first");
args.push_back("second");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_THROW_RE(cmdline::usage_error, "Too many arguments",
cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(invalid_topic);
ATF_TEST_CASE_BODY(invalid_topic)
{
cmdline::args_vector args;
args.push_back("about");
args.push_back("foo");
cmd_about cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_THROW_RE(cmdline::usage_error, "Invalid about topic 'foo'",
cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, all_topics__ok);
ATF_ADD_TEST_CASE(tcs, all_topics__missing_docs);
ATF_ADD_TEST_CASE(tcs, topic_authors__ok);
ATF_ADD_TEST_CASE(tcs, topic_authors__missing_doc);
ATF_ADD_TEST_CASE(tcs, topic_license__ok);
ATF_ADD_TEST_CASE(tcs, topic_license__missing_doc);
ATF_ADD_TEST_CASE(tcs, topic_version__ok);
ATF_ADD_TEST_CASE(tcs, invalid_args);
ATF_ADD_TEST_CASE(tcs, invalid_topic);
}

122
cli/cmd_config.cpp Normal file
View File

@ -0,0 +1,122 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_config.hpp"
#include <cstdlib>
#include "cli/common.ipp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/config/tree.ipp"
#include "utils/format/macros.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
using cli::cmd_config;
namespace {
/// Prints all configuration variables.
///
/// \param ui Object to interact with the I/O of the program.
/// \param properties The key/value map representing all the configuration
/// variables.
///
/// \return 0 for success.
static int
print_all(cmdline::ui* ui, const config::properties_map& properties)
{
for (config::properties_map::const_iterator iter = properties.begin();
iter != properties.end(); iter++)
ui->out(F("%s = %s") % (*iter).first % (*iter).second);
return EXIT_SUCCESS;
}
/// Prints the configuration variables that the user requests.
///
/// \param ui Object to interact with the I/O of the program.
/// \param properties The key/value map representing all the configuration
/// variables.
/// \param filters The names of the configuration variables to print.
///
/// \return 0 if all specified filters are valid; 1 otherwise.
static int
print_some(cmdline::ui* ui, const config::properties_map& properties,
const cmdline::args_vector& filters)
{
bool ok = true;
for (cmdline::args_vector::const_iterator iter = filters.begin();
iter != filters.end(); iter++) {
const config::properties_map::const_iterator match =
properties.find(*iter);
if (match == properties.end()) {
cmdline::print_warning(ui, F("'%s' is not defined.") % *iter);
ok = false;
} else
ui->out(F("%s = %s") % (*match).first % (*match).second);
}
return ok ? EXIT_SUCCESS : EXIT_FAILURE;
}
} // anonymous namespace
/// Default constructor for cmd_config.
cmd_config::cmd_config(void) : cli_command(
"config", "[variable1 .. variableN]", 0, -1,
"Inspects the values of configuration variables")
{
}
/// Entry point for the "config" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
/// \param user_config The runtime configuration of the program.
///
/// \return 0 if everything is OK, 1 if any of the necessary documents cannot be
/// opened.
int
cmd_config::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& user_config)
{
const config::properties_map properties = user_config.all_properties();
if (cmdline.arguments().empty())
return print_all(ui, properties);
else
return print_some(ui, properties, cmdline.arguments());
}

54
cli/cmd_config.hpp Normal file
View File

@ -0,0 +1,54 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_config.hpp
/// Provides the cmd_config class.
#if !defined(CLI_CMD_CONFIG_HPP)
#define CLI_CMD_CONFIG_HPP
#include "cli/common.hpp"
namespace cli {
/// Implementation of the "config" subcommand.
class cmd_config : public cli_command
{
public:
cmd_config(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_CONFIG_HPP)

144
cli/cmd_config_test.cpp Normal file
View File

@ -0,0 +1,144 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_config.hpp"
#include <cstdlib>
#include <atf-c++.hpp>
#include "cli/common.ipp"
#include "engine/config.hpp"
#include "utils/cmdline/globals.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/config/tree.ipp"
#include "utils/optional.ipp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
using cli::cmd_config;
using utils::none;
namespace {
/// Instantiates a fake user configuration for testing purposes.
///
/// The user configuration is populated with a collection of test-suite
/// properties and some hardcoded values for the generic configuration options.
///
/// \return A new user configuration object.
static config::tree
fake_config(void)
{
config::tree user_config = engine::default_config();
user_config.set_string("architecture", "the-architecture");
user_config.set_string("parallelism", "128");
user_config.set_string("platform", "the-platform");
//user_config.set_string("unprivileged_user", "");
user_config.set_string("test_suites.foo.bar", "first");
user_config.set_string("test_suites.foo.baz", "second");
return user_config;
}
} // anonymous namespace
ATF_TEST_CASE_WITHOUT_HEAD(all);
ATF_TEST_CASE_BODY(all)
{
cmdline::args_vector args;
args.push_back("config");
cmd_config cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, fake_config()));
ATF_REQUIRE_EQ(5, ui.out_log().size());
ATF_REQUIRE_EQ("architecture = the-architecture", ui.out_log()[0]);
ATF_REQUIRE_EQ("parallelism = 128", ui.out_log()[1]);
ATF_REQUIRE_EQ("platform = the-platform", ui.out_log()[2]);
ATF_REQUIRE_EQ("test_suites.foo.bar = first", ui.out_log()[3]);
ATF_REQUIRE_EQ("test_suites.foo.baz = second", ui.out_log()[4]);
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(some__ok);
ATF_TEST_CASE_BODY(some__ok)
{
cmdline::args_vector args;
args.push_back("config");
args.push_back("platform");
args.push_back("test_suites.foo.baz");
cmd_config cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, fake_config()));
ATF_REQUIRE_EQ(2, ui.out_log().size());
ATF_REQUIRE_EQ("platform = the-platform", ui.out_log()[0]);
ATF_REQUIRE_EQ("test_suites.foo.baz = second", ui.out_log()[1]);
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(some__fail);
ATF_TEST_CASE_BODY(some__fail)
{
cmdline::args_vector args;
args.push_back("config");
args.push_back("platform");
args.push_back("unknown");
args.push_back("test_suites.foo.baz");
cmdline::init("progname");
cmd_config cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_FAILURE, cmd.main(&ui, args, fake_config()));
ATF_REQUIRE_EQ(2, ui.out_log().size());
ATF_REQUIRE_EQ("platform = the-platform", ui.out_log()[0]);
ATF_REQUIRE_EQ("test_suites.foo.baz = second", ui.out_log()[1]);
ATF_REQUIRE_EQ(1, ui.err_log().size());
ATF_REQUIRE(atf::utils::grep_string("unknown.*not defined",
ui.err_log()[0]));
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, all);
ATF_ADD_TEST_CASE(tcs, some__ok);
ATF_ADD_TEST_CASE(tcs, some__fail);
}

200
cli/cmd_db_exec.cpp Normal file
View File

@ -0,0 +1,200 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_db_exec.hpp"
#include <algorithm>
#include <cstdlib>
#include <iterator>
#include <sstream>
#include <string>
#include "cli/common.ipp"
#include "store/exceptions.hpp"
#include "store/layout.hpp"
#include "store/read_backend.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/defs.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/path.hpp"
#include "utils/sanity.hpp"
#include "utils/sqlite/database.hpp"
#include "utils/sqlite/exceptions.hpp"
#include "utils/sqlite/statement.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace layout = store::layout;
namespace sqlite = utils::sqlite;
using cli::cmd_db_exec;
namespace {
/// Concatenates a vector into a string using ' ' as a separator.
///
/// \param args The objects to join. This cannot be empty.
///
/// \return The concatenation of all the objects in the set.
static std::string
flatten_args(const cmdline::args_vector& args)
{
std::ostringstream output;
std::copy(args.begin(), args.end(),
std::ostream_iterator< std::string >(output, " "));
std::string result = output.str();
result.erase(result.end() - 1);
return result;
}
} // anonymous namespace
/// Formats a particular cell of a statement result.
///
/// \param stmt The statement whose cell to format.
/// \param index The index of the cell to format.
///
/// \return A textual representation of the cell.
std::string
cli::format_cell(sqlite::statement& stmt, const int index)
{
switch (stmt.column_type(index)) {
case sqlite::type_blob: {
const sqlite::blob blob = stmt.column_blob(index);
return F("BLOB of %s bytes") % blob.size;
}
case sqlite::type_float:
return F("%s") % stmt.column_double(index);
case sqlite::type_integer:
return F("%s") % stmt.column_int64(index);
case sqlite::type_null:
return "NULL";
case sqlite::type_text:
return stmt.column_text(index);
}
UNREACHABLE;
}
/// Formats the column names of a statement for output as CSV.
///
/// \param stmt The statement whose columns to format.
///
/// \return A comma-separated list of column names.
std::string
cli::format_headers(sqlite::statement& stmt)
{
std::string output;
int i = 0;
for (; i < stmt.column_count() - 1; ++i)
output += stmt.column_name(i) + ',';
output += stmt.column_name(i);
return output;
}
/// Formats a row of a statement for output as CSV.
///
/// \param stmt The statement whose current row to format.
///
/// \return A comma-separated list of values.
std::string
cli::format_row(sqlite::statement& stmt)
{
std::string output;
int i = 0;
for (; i < stmt.column_count() - 1; ++i)
output += cli::format_cell(stmt, i) + ',';
output += cli::format_cell(stmt, i);
return output;
}
/// Default constructor for cmd_db_exec.
cmd_db_exec::cmd_db_exec(void) : cli_command(
"db-exec", "sql_statement", 1, -1,
"Executes an arbitrary SQL statement in a results file and prints "
"the resulting table")
{
add_option(results_file_open_option);
add_option(cmdline::bool_option("no-headers", "Do not show headers in the "
"output table"));
}
/// Entry point for the "db-exec" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
///
/// \return 0 if everything is OK, 1 if the statement is invalid or if there is
/// any other problem.
int
cmd_db_exec::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& /* user_config */)
{
try {
const fs::path results_file = layout::find_results(
results_file_open(cmdline));
// TODO(jmmv): Shouldn't be using store::detail here...
sqlite::database db = store::detail::open_and_setup(
results_file, sqlite::open_readwrite);
sqlite::statement stmt = db.create_statement(
flatten_args(cmdline.arguments()));
if (stmt.step()) {
if (!cmdline.has_option("no-headers"))
ui->out(cli::format_headers(stmt));
do
ui->out(cli::format_row(stmt));
while (stmt.step());
}
return EXIT_SUCCESS;
} catch (const sqlite::error& e) {
cmdline::print_error(ui, F("SQLite error: %s.") % e.what());
return EXIT_FAILURE;
} catch (const store::error& e) {
cmdline::print_error(ui, F("%s.") % e.what());
return EXIT_FAILURE;
}
}

61
cli/cmd_db_exec.hpp Normal file
View File

@ -0,0 +1,61 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_db_exec.hpp
/// Provides the cmd_db_exec class.
#if !defined(CLI_CMD_DB_EXEC_HPP)
#define CLI_CMD_DB_EXEC_HPP
#include <string>
#include "cli/common.hpp"
#include "utils/sqlite/statement_fwd.hpp"
namespace cli {
std::string format_cell(utils::sqlite::statement&, const int);
std::string format_headers(utils::sqlite::statement&);
std::string format_row(utils::sqlite::statement&);
/// Implementation of the "db-exec" subcommand.
class cmd_db_exec : public cli_command
{
public:
cmd_db_exec(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_DB_EXEC_HPP)

165
cli/cmd_db_exec_test.cpp Normal file
View File

@ -0,0 +1,165 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_db_exec.hpp"
#include <cstring>
#include <atf-c++.hpp>
#include "utils/format/macros.hpp"
#include "utils/sqlite/database.hpp"
#include "utils/sqlite/statement.ipp"
namespace sqlite = utils::sqlite;
namespace {
/// Performs a test for the cli::format_cell() function.
///
/// \tparam Cell The type of the value to insert into the test column.
/// \param column_type The SQL type of the test column.
/// \param value The value to insert into the test column.
/// \param exp_value The expected return value of cli::format_cell().
template< class Cell >
static void
do_format_cell_test(const std::string column_type,
const Cell& value, const std::string& exp_value)
{
sqlite::database db = sqlite::database::in_memory();
sqlite::statement create = db.create_statement(
F("CREATE TABLE test (column %s)") % column_type);
create.step_without_results();
sqlite::statement insert = db.create_statement(
"INSERT INTO test (column) VALUES (:column)");
insert.bind(":column", value);
insert.step_without_results();
sqlite::statement query = db.create_statement("SELECT * FROM test");
ATF_REQUIRE(query.step());
ATF_REQUIRE_EQ(exp_value, cli::format_cell(query, 0));
ATF_REQUIRE(!query.step());
}
} // anonymous namespace
ATF_TEST_CASE_WITHOUT_HEAD(format_cell__blob);
ATF_TEST_CASE_BODY(format_cell__blob)
{
const char* contents = "Some random contents";
do_format_cell_test(
"BLOB", sqlite::blob(contents, std::strlen(contents)),
F("BLOB of %s bytes") % strlen(contents));
}
ATF_TEST_CASE_WITHOUT_HEAD(format_cell__float);
ATF_TEST_CASE_BODY(format_cell__float)
{
do_format_cell_test("FLOAT", 3.5, "3.5");
}
ATF_TEST_CASE_WITHOUT_HEAD(format_cell__integer);
ATF_TEST_CASE_BODY(format_cell__integer)
{
do_format_cell_test("INTEGER", 123456, "123456");
}
ATF_TEST_CASE_WITHOUT_HEAD(format_cell__null);
ATF_TEST_CASE_BODY(format_cell__null)
{
do_format_cell_test("TEXT", sqlite::null(), "NULL");
}
ATF_TEST_CASE_WITHOUT_HEAD(format_cell__text);
ATF_TEST_CASE_BODY(format_cell__text)
{
do_format_cell_test("TEXT", "Hello, world", "Hello, world");
}
ATF_TEST_CASE_WITHOUT_HEAD(format_headers);
ATF_TEST_CASE_BODY(format_headers)
{
sqlite::database db = sqlite::database::in_memory();
sqlite::statement create = db.create_statement(
"CREATE TABLE test (c1 TEXT, c2 TEXT, c3 TEXT)");
create.step_without_results();
sqlite::statement query = db.create_statement(
"SELECT c1, c2, c3 AS c3bis FROM test");
ATF_REQUIRE_EQ("c1,c2,c3bis", cli::format_headers(query));
}
ATF_TEST_CASE_WITHOUT_HEAD(format_row);
ATF_TEST_CASE_BODY(format_row)
{
sqlite::database db = sqlite::database::in_memory();
sqlite::statement create = db.create_statement(
"CREATE TABLE test (c1 TEXT, c2 BLOB)");
create.step_without_results();
const char* memory = "BLOB contents";
sqlite::statement insert = db.create_statement(
"INSERT INTO test VALUES (:v1, :v2)");
insert.bind(":v1", "A string");
insert.bind(":v2", sqlite::blob(memory, std::strlen(memory)));
insert.step_without_results();
sqlite::statement query = db.create_statement("SELECT * FROM test");
query.step();
ATF_REQUIRE_EQ(
(F("A string,BLOB of %s bytes") % std::strlen(memory)).str(),
cli::format_row(query));
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, format_cell__blob);
ATF_ADD_TEST_CASE(tcs, format_cell__float);
ATF_ADD_TEST_CASE(tcs, format_cell__integer);
ATF_ADD_TEST_CASE(tcs, format_cell__null);
ATF_ADD_TEST_CASE(tcs, format_cell__text);
ATF_ADD_TEST_CASE(tcs, format_headers);
ATF_ADD_TEST_CASE(tcs, format_row);
}

82
cli/cmd_db_migrate.cpp Normal file
View File

@ -0,0 +1,82 @@
// Copyright 2013 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_db_migrate.hpp"
#include <cstdlib>
#include "cli/common.ipp"
#include "store/exceptions.hpp"
#include "store/layout.hpp"
#include "store/migrate.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/ui.hpp"
#include "utils/defs.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/path.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace layout = store::layout;
using cli::cmd_db_migrate;
/// Default constructor for cmd_db_migrate.
cmd_db_migrate::cmd_db_migrate(void) : cli_command(
"db-migrate", "", 0, 0,
"Upgrades the schema of an existing results file to the currently "
"implemented version. A backup of the results file is created, but "
"this operation is not reversible")
{
add_option(results_file_open_option);
}
/// Entry point for the "db-migrate" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
///
/// \return 0 if everything is OK, 1 if the statement is invalid or if there is
/// any other problem.
int
cmd_db_migrate::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& /* user_config */)
{
try {
const fs::path results_file = layout::find_results(
results_file_open(cmdline));
store::migrate_schema(results_file);
return EXIT_SUCCESS;
} catch (const store::error& e) {
cmdline::print_error(ui, F("Migration failed: %s.") % e.what());
return EXIT_FAILURE;
}
}

54
cli/cmd_db_migrate.hpp Normal file
View File

@ -0,0 +1,54 @@
// Copyright 2013 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_db_migrate.hpp
/// Provides the cmd_db_migrate class.
#if !defined(CLI_CMD_DB_MIGRATE_HPP)
#define CLI_CMD_DB_MIGRATE_HPP
#include "cli/common.hpp"
namespace cli {
/// Implementation of the "db-migrate" subcommand.
class cmd_db_migrate : public cli_command
{
public:
cmd_db_migrate(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_DB_MIGRATE_HPP)

94
cli/cmd_debug.cpp Normal file
View File

@ -0,0 +1,94 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_debug.hpp"
#include <cstdlib>
#include "cli/common.ipp"
#include "drivers/debug_test.hpp"
#include "engine/filters.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/format/macros.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
using cli::cmd_debug;
/// Default constructor for cmd_debug.
cmd_debug::cmd_debug(void) : cli_command(
"debug", "test_case", 1, 1,
"Executes a single test case providing facilities for debugging")
{
add_option(build_root_option);
add_option(kyuafile_option);
add_option(cmdline::path_option(
"stdout", "Where to direct the standard output of the test case",
"path", "/dev/stdout"));
add_option(cmdline::path_option(
"stderr", "Where to direct the standard error of the test case",
"path", "/dev/stderr"));
}
/// Entry point for the "debug" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
/// \param user_config The runtime debuguration of the program.
///
/// \return 0 if everything is OK, 1 if any of the necessary documents cannot be
/// opened.
int
cmd_debug::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& user_config)
{
const std::string& test_case_name = cmdline.arguments()[0];
if (test_case_name.find(':') == std::string::npos)
throw cmdline::usage_error(F("'%s' is not a test case identifier "
"(missing ':'?)") % test_case_name);
const engine::test_filter filter = engine::test_filter::parse(
test_case_name);
const drivers::debug_test::result result = drivers::debug_test::drive(
kyuafile_path(cmdline), build_root_path(cmdline), filter, user_config,
cmdline.get_option< cmdline::path_option >("stdout"),
cmdline.get_option< cmdline::path_option >("stderr"));
ui->out(F("%s -> %s") % cli::format_test_case_id(result.test_case) %
cli::format_result(result.test_result));
return result.test_result.good() ? EXIT_SUCCESS : EXIT_FAILURE;
}

54
cli/cmd_debug.hpp Normal file
View File

@ -0,0 +1,54 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_debug.hpp
/// Provides the cmd_debug class.
#if !defined(CLI_CMD_DEBUG_HPP)
#define CLI_CMD_DEBUG_HPP
#include "cli/common.hpp"
namespace cli {
/// Implementation of the "debug" subcommand.
class cmd_debug : public cli_command
{
public:
cmd_debug(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_DEBUG_HPP)

82
cli/cmd_debug_test.cpp Normal file
View File

@ -0,0 +1,82 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_debug.hpp"
#include <stdexcept>
#include <atf-c++.hpp>
#include "cli/common.ipp"
#include "engine/config.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/config/tree.ipp"
namespace cmdline = utils::cmdline;
ATF_TEST_CASE_WITHOUT_HEAD(invalid_filter);
ATF_TEST_CASE_BODY(invalid_filter)
{
cmdline::args_vector args;
args.push_back("debug");
args.push_back("incorrect:");
cli::cmd_debug cmd;
cmdline::ui_mock ui;
// TODO(jmmv): This error should really be cmdline::usage_error.
ATF_REQUIRE_THROW_RE(std::runtime_error, "Test case.*'incorrect:'.*empty",
cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(filter_without_test_case);
ATF_TEST_CASE_BODY(filter_without_test_case)
{
cmdline::args_vector args;
args.push_back("debug");
args.push_back("program");
cli::cmd_debug cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_THROW_RE(cmdline::error, "'program'.*not a test case",
cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, invalid_filter);
ATF_ADD_TEST_CASE(tcs, filter_without_test_case);
}

250
cli/cmd_help.cpp Normal file
View File

@ -0,0 +1,250 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_help.hpp"
#include <algorithm>
#include <cstdlib>
#include "cli/common.ipp"
#include "utils/cmdline/commands_map.ipp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/globals.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui.hpp"
#include "utils/defs.hpp"
#include "utils/format/macros.hpp"
#include "utils/sanity.hpp"
#include "utils/text/table.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace text = utils::text;
using cli::cmd_help;
namespace {
/// Creates a table with the help of a set of options.
///
/// \param options The set of options to describe. May be empty.
///
/// \return A 2-column wide table with the description of the options.
static text::table
options_help(const cmdline::options_vector& options)
{
text::table table(2);
for (cmdline::options_vector::const_iterator iter = options.begin();
iter != options.end(); iter++) {
const cmdline::base_option* option = *iter;
std::string description = option->description();
if (option->needs_arg() && option->has_default_value())
description += F(" (default: %s)") % option->default_value();
text::table_row row;
if (option->has_short_name())
row.push_back(F("%s, %s") % option->format_short_name() %
option->format_long_name());
else
row.push_back(F("%s") % option->format_long_name());
row.push_back(F("%s.") % description);
table.add_row(row);
}
return table;
}
/// Prints the summary of commands and generic options.
///
/// \param ui Object to interact with the I/O of the program.
/// \param options The set of program-wide options for which to print help.
/// \param commands The set of commands for which to print help.
static void
general_help(cmdline::ui* ui, const cmdline::options_vector* options,
const cmdline::commands_map< cli::cli_command >* commands)
{
PRE(!commands->empty());
cli::write_version_header(ui);
ui->out("");
ui->out_tag_wrap(
"Usage: ",
F("%s [general_options] command [command_options] [args]") %
cmdline::progname(), false);
const text::table options_table = options_help(*options);
text::widths_vector::value_type first_width =
options_table.column_width(0);
std::map< std::string, text::table > command_tables;
for (cmdline::commands_map< cli::cli_command >::const_iterator
iter = commands->begin(); iter != commands->end(); iter++) {
const std::string& category = (*iter).first;
const std::set< std::string >& command_names = (*iter).second;
command_tables.insert(std::map< std::string, text::table >::value_type(
category, text::table(2)));
text::table& table = command_tables.find(category)->second;
for (std::set< std::string >::const_iterator i2 = command_names.begin();
i2 != command_names.end(); i2++) {
const cli::cli_command* command = commands->find(*i2);
text::table_row row;
row.push_back(command->name());
row.push_back(F("%s.") % command->short_description());
table.add_row(row);
}
if (table.column_width(0) > first_width)
first_width = table.column_width(0);
}
text::table_formatter formatter;
formatter.set_column_width(0, first_width);
formatter.set_column_width(1, text::table_formatter::width_refill);
formatter.set_separator(" ");
if (!options_table.empty()) {
ui->out_wrap("");
ui->out_wrap("Available general options:");
ui->out_table(options_table, formatter, " ");
}
// Iterate using the same loop as above to preserve ordering.
for (cmdline::commands_map< cli::cli_command >::const_iterator
iter = commands->begin(); iter != commands->end(); iter++) {
const std::string& category = (*iter).first;
ui->out_wrap("");
ui->out_wrap(F("%s commands:") %
(category.empty() ? "Generic" : category));
ui->out_table(command_tables.find(category)->second, formatter, " ");
}
ui->out_wrap("");
ui->out_wrap("See kyua(1) for more details.");
}
/// Prints help for a particular subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param general_options The options that apply to all commands.
/// \param command Pointer to the command to describe.
static void
subcommand_help(cmdline::ui* ui,
const utils::cmdline::options_vector* general_options,
const cli::cli_command* command)
{
cli::write_version_header(ui);
ui->out("");
ui->out_tag_wrap(
"Usage: ", F("%s [general_options] %s%s%s") %
cmdline::progname() % command->name() %
(command->options().empty() ? "" : " [command_options]") %
(command->arg_list().empty() ? "" : (" " + command->arg_list())),
false);
ui->out_wrap("");
ui->out_wrap(F("%s.") % command->short_description());
const text::table general_table = options_help(*general_options);
const text::table command_table = options_help(command->options());
const text::widths_vector::value_type first_width =
std::max(general_table.column_width(0), command_table.column_width(0));
text::table_formatter formatter;
formatter.set_column_width(0, first_width);
formatter.set_column_width(1, text::table_formatter::width_refill);
formatter.set_separator(" ");
if (!general_table.empty()) {
ui->out_wrap("");
ui->out_wrap("Available general options:");
ui->out_table(general_table, formatter, " ");
}
if (!command_table.empty()) {
ui->out_wrap("");
ui->out_wrap("Available command options:");
ui->out_table(command_table, formatter, " ");
}
ui->out_wrap("");
ui->out_wrap(F("See kyua-%s(1) for more details.") % command->name());
}
} // anonymous namespace
/// Default constructor for cmd_help.
///
/// \param options_ The set of program-wide options for which to provide help.
/// \param commands_ The set of commands for which to provide help.
cmd_help::cmd_help(const cmdline::options_vector* options_,
const cmdline::commands_map< cli_command >* commands_) :
cli_command("help", "[subcommand]", 0, 1, "Shows usage information"),
_options(options_),
_commands(commands_)
{
}
/// Entry point for the "help" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
///
/// \return 0 to indicate success.
int
cmd_help::run(utils::cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& /* user_config */)
{
if (cmdline.arguments().empty()) {
general_help(ui, _options, _commands);
} else {
INV(cmdline.arguments().size() == 1);
const std::string& cmdname = cmdline.arguments()[0];
const cli::cli_command* command = _commands->find(cmdname);
if (command == NULL)
throw cmdline::usage_error(F("The command %s does not exist") %
cmdname);
else
subcommand_help(ui, _options, command);
}
return EXIT_SUCCESS;
}

62
cli/cmd_help.hpp Normal file
View File

@ -0,0 +1,62 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_help.hpp
/// Provides the cmd_help class.
#if !defined(CLI_CMD_HELP_HPP)
#define CLI_CMD_HELP_HPP
#include "cli/common.hpp"
#include "utils/cmdline/commands_map_fwd.hpp"
namespace cli {
/// Implementation of the "help" subcommand.
class cmd_help : public cli_command
{
/// The set of program-wide options for which to provide help.
const utils::cmdline::options_vector* _options;
/// The set of commands for which to provide help.
const utils::cmdline::commands_map< cli_command >* _commands;
public:
cmd_help(const utils::cmdline::options_vector*,
const utils::cmdline::commands_map< cli_command >*);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_HELP_HPP)

347
cli/cmd_help_test.cpp Normal file
View File

@ -0,0 +1,347 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_help.hpp"
#include <algorithm>
#include <cstdlib>
#include <iterator>
#include <atf-c++.hpp>
#include "cli/common.ipp"
#include "engine/config.hpp"
#include "utils/cmdline/commands_map.ipp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/globals.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/config/tree.ipp"
#include "utils/defs.hpp"
#include "utils/sanity.hpp"
#if defined(HAVE_CONFIG_H)
# include "config.h"
#endif
namespace cmdline = utils::cmdline;
namespace config = utils::config;
using cli::cmd_help;
namespace {
/// Mock command with a simple definition (no options, no arguments).
///
/// Attempting to run this command will result in a crash. It is only provided
/// to validate the generation of interactive help.
class cmd_mock_simple : public cli::cli_command {
public:
/// Constructs a new mock command.
///
/// \param name_ The name of the command to create.
cmd_mock_simple(const char* name_) : cli::cli_command(
name_, "", 0, 0, "Simple command")
{
}
/// Runs the mock command.
///
/// \return Nothing because this function is never called.
int
run(cmdline::ui* /* ui */,
const cmdline::parsed_cmdline& /* cmdline */,
const config::tree& /* user_config */)
{
UNREACHABLE;
}
};
/// Mock command with a complex definition (some options, some arguments).
///
/// Attempting to run this command will result in a crash. It is only provided
/// to validate the generation of interactive help.
class cmd_mock_complex : public cli::cli_command {
public:
/// Constructs a new mock command.
///
/// \param name_ The name of the command to create.
cmd_mock_complex(const char* name_) : cli::cli_command(
name_, "[arg1 .. argN]", 0, 2, "Complex command")
{
add_option(cmdline::bool_option("flag_a", "Flag A"));
add_option(cmdline::bool_option('b', "flag_b", "Flag B"));
add_option(cmdline::string_option('c', "flag_c", "Flag C", "c_arg"));
add_option(cmdline::string_option("flag_d", "Flag D", "d_arg", "foo"));
}
/// Runs the mock command.
///
/// \return Nothing because this function is never called.
int
run(cmdline::ui* /* ui */,
const cmdline::parsed_cmdline& /* cmdline */,
const config::tree& /* user_config */)
{
UNREACHABLE;
}
};
/// Initializes the cmdline library and generates the set of test commands.
///
/// \param [out] commands A mapping that is updated to contain the commands to
/// use for testing.
static void
setup(cmdline::commands_map< cli::cli_command >& commands)
{
cmdline::init("progname");
commands.insert(new cmd_mock_simple("mock_simple"));
commands.insert(new cmd_mock_complex("mock_complex"));
commands.insert(new cmd_mock_simple("mock_simple_2"), "First");
commands.insert(new cmd_mock_complex("mock_complex_2"), "First");
commands.insert(new cmd_mock_simple("mock_simple_3"), "Second");
}
/// Performs a test on the global help (not that of a subcommand).
///
/// \param general_options The genral options supported by the tool, if any.
/// \param expected_options Expected lines of help output documenting the
/// options in general_options.
/// \param ui The cmdline::mock_ui object to which to write the output.
static void
global_test(const cmdline::options_vector& general_options,
const std::vector< std::string >& expected_options,
cmdline::ui_mock& ui)
{
cmdline::commands_map< cli::cli_command > mock_commands;
setup(mock_commands);
cmdline::args_vector args;
args.push_back("help");
cmd_help cmd(&general_options, &mock_commands);
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, engine::default_config()));
std::vector< std::string > expected;
expected.push_back(PACKAGE " (" PACKAGE_NAME ") " PACKAGE_VERSION);
expected.push_back("");
expected.push_back("Usage: progname [general_options] command "
"[command_options] [args]");
if (!general_options.empty()) {
expected.push_back("");
expected.push_back("Available general options:");
std::copy(expected_options.begin(), expected_options.end(),
std::back_inserter(expected));
}
expected.push_back("");
expected.push_back("Generic commands:");
expected.push_back(" mock_complex Complex command.");
expected.push_back(" mock_simple Simple command.");
expected.push_back("");
expected.push_back("First commands:");
expected.push_back(" mock_complex_2 Complex command.");
expected.push_back(" mock_simple_2 Simple command.");
expected.push_back("");
expected.push_back("Second commands:");
expected.push_back(" mock_simple_3 Simple command.");
expected.push_back("");
expected.push_back("See kyua(1) for more details.");
ATF_REQUIRE(expected == ui.out_log());
ATF_REQUIRE(ui.err_log().empty());
}
} // anonymous namespace
ATF_TEST_CASE_WITHOUT_HEAD(global__no_options);
ATF_TEST_CASE_BODY(global__no_options)
{
cmdline::ui_mock ui;
cmdline::options_vector general_options;
global_test(general_options, std::vector< std::string >(), ui);
}
ATF_TEST_CASE_WITHOUT_HEAD(global__some_options);
ATF_TEST_CASE_BODY(global__some_options)
{
cmdline::ui_mock ui;
cmdline::options_vector general_options;
const cmdline::bool_option flag_a("flag_a", "Flag A");
general_options.push_back(&flag_a);
const cmdline::string_option flag_c('c', "lc", "Flag C", "X");
general_options.push_back(&flag_c);
std::vector< std::string > expected;
expected.push_back(" --flag_a Flag A.");
expected.push_back(" -c X, --lc=X Flag C.");
global_test(general_options, expected, ui);
}
ATF_TEST_CASE_WITHOUT_HEAD(subcommand__simple);
ATF_TEST_CASE_BODY(subcommand__simple)
{
cmdline::options_vector general_options;
cmdline::commands_map< cli::cli_command > mock_commands;
setup(mock_commands);
cmdline::args_vector args;
args.push_back("help");
args.push_back("mock_simple");
cmd_help cmd(&general_options, &mock_commands);
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(atf::utils::grep_collection(
"^kyua.*" PACKAGE_VERSION, ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection(
"^Usage: progname \\[general_options\\] mock_simple$", ui.out_log()));
ATF_REQUIRE(!atf::utils::grep_collection(
"Available.*options", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection(
"^See kyua-mock_simple\\(1\\) for more details.", ui.out_log()));
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(subcommand__complex);
ATF_TEST_CASE_BODY(subcommand__complex)
{
cmdline::options_vector general_options;
const cmdline::bool_option global_a("global_a", "Global A");
general_options.push_back(&global_a);
const cmdline::string_option global_c('c', "global_c", "Global C",
"c_global");
general_options.push_back(&global_c);
cmdline::commands_map< cli::cli_command > mock_commands;
setup(mock_commands);
cmdline::args_vector args;
args.push_back("help");
args.push_back("mock_complex");
cmd_help cmd(&general_options, &mock_commands);
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(atf::utils::grep_collection(
"^kyua.*" PACKAGE_VERSION, ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection(
"^Usage: progname \\[general_options\\] mock_complex "
"\\[command_options\\] \\[arg1 .. argN\\]$", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("Available general options",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("--global_a", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("--global_c=c_global",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("Available command options",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("--flag_a *Flag A",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection("-b.*--flag_b *Flag B",
ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection(
"-c c_arg.*--flag_c=c_arg *Flag C", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection(
"--flag_d=d_arg *Flag D.*default.*foo", ui.out_log()));
ATF_REQUIRE(atf::utils::grep_collection(
"^See kyua-mock_complex\\(1\\) for more details.", ui.out_log()));
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(subcommand__unknown);
ATF_TEST_CASE_BODY(subcommand__unknown)
{
cmdline::options_vector general_options;
cmdline::commands_map< cli::cli_command > mock_commands;
setup(mock_commands);
cmdline::args_vector args;
args.push_back("help");
args.push_back("foobar");
cmd_help cmd(&general_options, &mock_commands);
cmdline::ui_mock ui;
ATF_REQUIRE_THROW_RE(cmdline::usage_error, "command foobar.*not exist",
cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(invalid_args);
ATF_TEST_CASE_BODY(invalid_args)
{
cmdline::options_vector general_options;
cmdline::commands_map< cli::cli_command > mock_commands;
setup(mock_commands);
cmdline::args_vector args;
args.push_back("help");
args.push_back("mock_simple");
args.push_back("mock_complex");
cmd_help cmd(&general_options, &mock_commands);
cmdline::ui_mock ui;
ATF_REQUIRE_THROW_RE(cmdline::usage_error, "Too many arguments",
cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, global__no_options);
ATF_ADD_TEST_CASE(tcs, global__some_options);
ATF_ADD_TEST_CASE(tcs, subcommand__simple);
ATF_ADD_TEST_CASE(tcs, subcommand__complex);
ATF_ADD_TEST_CASE(tcs, subcommand__unknown);
ATF_ADD_TEST_CASE(tcs, invalid_args);
}

161
cli/cmd_list.cpp Normal file
View File

@ -0,0 +1,161 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_list.hpp"
#include <cstdlib>
#include <utility>
#include <vector>
#include "cli/common.ipp"
#include "drivers/list_tests.hpp"
#include "engine/filters.hpp"
#include "model/metadata.hpp"
#include "model/test_case.hpp"
#include "model/test_program.hpp"
#include "model/types.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/defs.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/path.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace {
/// Hooks for list_tests to print test cases as they come.
class progress_hooks : public drivers::list_tests::base_hooks {
/// The ui object to which to print the test cases.
cmdline::ui* _ui;
/// Whether to print test case details or just their names.
bool _verbose;
public:
/// Initializes the hooks.
///
/// \param ui_ The ui object to which to print the test cases.
/// \param verbose_ Whether to print test case details or just their names.
progress_hooks(cmdline::ui* ui_, const bool verbose_) :
_ui(ui_),
_verbose(verbose_)
{
}
/// Reports a test case as soon as it is found.
///
/// \param test_program The test program containing the test case.
/// \param test_case_name The name of the located test case.
void
got_test_case(const model::test_program& test_program,
const std::string& test_case_name)
{
cli::detail::list_test_case(_ui, _verbose, test_program,
test_case_name);
}
};
} // anonymous namespace
/// Lists a single test case.
///
/// \param [out] ui Object to interact with the I/O of the program.
/// \param verbose Whether to be verbose or not.
/// \param test_program The test program containing the test case to print.
/// \param test_case_name The name of the test case to print.
void
cli::detail::list_test_case(cmdline::ui* ui, const bool verbose,
const model::test_program& test_program,
const std::string& test_case_name)
{
const model::test_case& test_case = test_program.find(test_case_name);
const std::string id = format_test_case_id(test_program, test_case_name);
if (!verbose) {
ui->out(id);
} else {
ui->out(F("%s (%s)") % id % test_program.test_suite_name());
// TODO(jmmv): Running these for every test case is probably not the
// fastest thing to do.
const model::metadata default_md = model::metadata_builder().build();
const model::properties_map default_props = default_md.to_properties();
const model::metadata& test_md = test_case.get_metadata();
const model::properties_map test_props = test_md.to_properties();
for (model::properties_map::const_iterator iter = test_props.begin();
iter != test_props.end(); iter++) {
const model::properties_map::const_iterator default_iter =
default_props.find((*iter).first);
if (default_iter == default_props.end() ||
(*iter).second != (*default_iter).second)
ui->out(F(" %s = %s") % (*iter).first % (*iter).second);
}
}
}
/// Default constructor for cmd_list.
cli::cmd_list::cmd_list(void) :
cli_command("list", "[test-program ...]", 0, -1,
"Lists test cases and their meta-data")
{
add_option(build_root_option);
add_option(kyuafile_option);
add_option(cmdline::bool_option('v', "verbose", "Show properties"));
}
/// Entry point for the "list" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
/// \param user_config The runtime configuration of the program.
///
/// \return 0 to indicate success.
int
cli::cmd_list::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& user_config)
{
progress_hooks hooks(ui, cmdline.has_option("verbose"));
const drivers::list_tests::result result = drivers::list_tests::drive(
kyuafile_path(cmdline), build_root_path(cmdline),
parse_filters(cmdline.arguments()), user_config, hooks);
return report_unused_filters(result.unused_filters, ui) ?
EXIT_FAILURE : EXIT_SUCCESS;
}

65
cli/cmd_list.hpp Normal file
View File

@ -0,0 +1,65 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_list.hpp
/// Provides the cmd_list class.
#if !defined(CLI_CMD_LIST_HPP)
#define CLI_CMD_LIST_HPP
#include <string>
#include "cli/common.hpp"
#include "model/test_program_fwd.hpp"
#include "utils/fs/path_fwd.hpp"
namespace cli {
namespace detail {
void list_test_case(utils::cmdline::ui*, const bool, const model::test_program&,
const std::string&);
} // namespace detail
/// Implementation of the "list" subcommand.
class cmd_list : public cli_command
{
public:
cmd_list(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_LIST_HPP)

112
cli/cmd_list_test.cpp Normal file
View File

@ -0,0 +1,112 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_list.hpp"
#include <atf-c++.hpp>
#include "model/metadata.hpp"
#include "model/test_program.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/fs/path.hpp"
namespace cmdline = utils::cmdline;
namespace fs = utils::fs;
ATF_TEST_CASE_WITHOUT_HEAD(list_test_case__no_verbose);
ATF_TEST_CASE_BODY(list_test_case__no_verbose)
{
const model::metadata md = model::metadata_builder()
.set_description("This should not be shown")
.build();
const model::test_program test_program = model::test_program_builder(
"mock", fs::path("the/test-program"), fs::path("root"), "suite")
.add_test_case("abc", md)
.set_metadata(md)
.build();
cmdline::ui_mock ui;
cli::detail::list_test_case(&ui, false, test_program, "abc");
ATF_REQUIRE_EQ(1, ui.out_log().size());
ATF_REQUIRE_EQ("the/test-program:abc", ui.out_log()[0]);
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(list_test_case__verbose__no_properties);
ATF_TEST_CASE_BODY(list_test_case__verbose__no_properties)
{
const model::test_program test_program = model::test_program_builder(
"mock", fs::path("hello/world"), fs::path("root"), "the-suite")
.add_test_case("my_name")
.build();
cmdline::ui_mock ui;
cli::detail::list_test_case(&ui, true, test_program, "my_name");
ATF_REQUIRE_EQ(1, ui.out_log().size());
ATF_REQUIRE_EQ("hello/world:my_name (the-suite)", ui.out_log()[0]);
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(list_test_case__verbose__some_properties);
ATF_TEST_CASE_BODY(list_test_case__verbose__some_properties)
{
const model::metadata md = model::metadata_builder()
.add_custom("my-property", "value")
.set_description("Some description")
.set_has_cleanup(true)
.build();
const model::test_program test_program = model::test_program_builder(
"mock", fs::path("hello/world"), fs::path("root"), "the-suite")
.add_test_case("my_name", md)
.set_metadata(md)
.build();
cmdline::ui_mock ui;
cli::detail::list_test_case(&ui, true, test_program, "my_name");
ATF_REQUIRE_EQ(4, ui.out_log().size());
ATF_REQUIRE_EQ("hello/world:my_name (the-suite)", ui.out_log()[0]);
ATF_REQUIRE_EQ(" custom.my-property = value", ui.out_log()[1]);
ATF_REQUIRE_EQ(" description = Some description", ui.out_log()[2]);
ATF_REQUIRE_EQ(" has_cleanup = true", ui.out_log()[3]);
ATF_REQUIRE(ui.err_log().empty());
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, list_test_case__no_verbose);
ATF_ADD_TEST_CASE(tcs, list_test_case__verbose__no_properties);
ATF_ADD_TEST_CASE(tcs, list_test_case__verbose__some_properties);
// Tests for cmd_list::run are located in integration/cmd_list_test.
}

421
cli/cmd_report.cpp Normal file
View File

@ -0,0 +1,421 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_report.hpp"
#include <algorithm>
#include <cstddef>
#include <cstdlib>
#include <map>
#include <ostream>
#include <string>
#include <vector>
#include "cli/common.ipp"
#include "drivers/scan_results.hpp"
#include "model/context.hpp"
#include "model/metadata.hpp"
#include "model/test_case.hpp"
#include "model/test_program.hpp"
#include "model/test_result.hpp"
#include "model/types.hpp"
#include "store/layout.hpp"
#include "store/read_transaction.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/datetime.hpp"
#include "utils/defs.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/path.hpp"
#include "utils/optional.ipp"
#include "utils/sanity.hpp"
#include "utils/stream.hpp"
#include "utils/text/operations.ipp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace datetime = utils::datetime;
namespace fs = utils::fs;
namespace layout = store::layout;
namespace text = utils::text;
using cli::cmd_report;
using utils::optional;
namespace {
/// Generates a plain-text report intended to be printed to the console.
class report_console_hooks : public drivers::scan_results::base_hooks {
/// Stream to which to write the report.
std::ostream& _output;
/// Whether to include details in the report or not.
const bool _verbose;
/// Collection of result types to include in the report.
const cli::result_types& _results_filters;
/// Path to the results file being read.
const fs::path& _results_file;
/// The start time of the first test.
optional< utils::datetime::timestamp > _start_time;
/// The end time of the last test.
optional< utils::datetime::timestamp > _end_time;
/// The total run time of the tests. Note that we cannot subtract _end_time
/// from _start_time to compute this due to parallel execution.
utils::datetime::delta _runtime;
/// Representation of a single result.
struct result_data {
/// The relative path to the test program.
utils::fs::path binary_path;
/// The name of the test case.
std::string test_case_name;
/// The result of the test case.
model::test_result result;
/// The duration of the test case execution.
utils::datetime::delta duration;
/// Constructs a new results data.
///
/// \param binary_path_ The relative path to the test program.
/// \param test_case_name_ The name of the test case.
/// \param result_ The result of the test case.
/// \param duration_ The duration of the test case execution.
result_data(const utils::fs::path& binary_path_,
const std::string& test_case_name_,
const model::test_result& result_,
const utils::datetime::delta& duration_) :
binary_path(binary_path_), test_case_name(test_case_name_),
result(result_), duration(duration_)
{
}
};
/// Results received, broken down by their type.
///
/// Note that this may not include all results, as keeping the whole list in
/// memory may be too much.
std::map< model::test_result_type, std::vector< result_data > > _results;
/// Pretty-prints the value of an environment variable.
///
/// \param indent Prefix for the lines to print. Continuation lines
/// use this indentation twice.
/// \param name Name of the variable.
/// \param value Value of the variable. Can have newlines.
void
print_env_var(const char* indent, const std::string& name,
const std::string& value)
{
const std::vector< std::string > lines = text::split(value, '\n');
if (lines.size() == 0) {
_output << F("%s%s=\n") % indent % name;;
} else {
_output << F("%s%s=%s\n") % indent % name % lines[0];
for (std::vector< std::string >::size_type i = 1;
i < lines.size(); ++i) {
_output << F("%s%s%s\n") % indent % indent % lines[i];
}
}
}
/// Prints the execution context to the output.
///
/// \param context The context to dump.
void
print_context(const model::context& context)
{
_output << "===> Execution context\n";
_output << F("Current directory: %s\n") % context.cwd();
const std::map< std::string, std::string >& env = context.env();
if (env.empty())
_output << "No environment variables recorded\n";
else {
_output << "Environment variables:\n";
for (std::map< std::string, std::string >::const_iterator
iter = env.begin(); iter != env.end(); iter++) {
print_env_var(" ", (*iter).first, (*iter).second);
}
}
}
/// Dumps a detailed view of the test case.
///
/// \param result_iter Results iterator pointing at the test case to be
/// dumped.
void
print_test_case_and_result(const store::results_iterator& result_iter)
{
const model::test_case& test_case =
result_iter.test_program()->find(result_iter.test_case_name());
const model::properties_map props =
test_case.get_metadata().to_properties();
_output << F("===> %s:%s\n") %
result_iter.test_program()->relative_path() %
result_iter.test_case_name();
_output << F("Result: %s\n") %
cli::format_result(result_iter.result());
_output << F("Start time: %s\n") %
result_iter.start_time().to_iso8601_in_utc();
_output << F("End time: %s\n") %
result_iter.end_time().to_iso8601_in_utc();
_output << F("Duration: %s\n") %
cli::format_delta(result_iter.end_time() -
result_iter.start_time());
_output << "\n";
_output << "Metadata:\n";
for (model::properties_map::const_iterator iter = props.begin();
iter != props.end(); ++iter) {
if ((*iter).second.empty()) {
_output << F(" %s is empty\n") % (*iter).first;
} else {
_output << F(" %s = %s\n") % (*iter).first % (*iter).second;
}
}
const std::string stdout_contents = result_iter.stdout_contents();
if (!stdout_contents.empty()) {
_output << "\n"
<< "Standard output:\n"
<< stdout_contents;
}
const std::string stderr_contents = result_iter.stderr_contents();
if (!stderr_contents.empty()) {
_output << "\n"
<< "Standard error:\n"
<< stderr_contents;
}
}
/// Counts how many results of a given type have been received.
///
/// \param type Test result type to count results for.
///
/// \return The number of test results with \p type.
std::size_t
count_results(const model::test_result_type type)
{
const std::map< model::test_result_type,
std::vector< result_data > >::const_iterator iter =
_results.find(type);
if (iter == _results.end())
return 0;
else
return (*iter).second.size();
}
/// Prints a set of results.
///
/// \param type Test result type to print results for.
/// \param title Title used when printing results.
void
print_results(const model::test_result_type type,
const char* title)
{
const std::map< model::test_result_type,
std::vector< result_data > >::const_iterator iter2 =
_results.find(type);
if (iter2 == _results.end())
return;
const std::vector< result_data >& all = (*iter2).second;
_output << F("===> %s\n") % title;
for (std::vector< result_data >::const_iterator iter = all.begin();
iter != all.end(); iter++) {
_output << F("%s:%s -> %s [%s]\n") % (*iter).binary_path %
(*iter).test_case_name %
cli::format_result((*iter).result) %
cli::format_delta((*iter).duration);
}
}
public:
/// Constructor for the hooks.
///
/// \param [out] output_ Stream to which to write the report.
/// \param verbose_ Whether to include details in the output or not.
/// \param results_filters_ The result types to include in the report.
/// Cannot be empty.
/// \param results_file_ Path to the results file being read.
report_console_hooks(std::ostream& output_, const bool verbose_,
const cli::result_types& results_filters_,
const fs::path& results_file_) :
_output(output_),
_verbose(verbose_),
_results_filters(results_filters_),
_results_file(results_file_)
{
PRE(!results_filters_.empty());
}
/// Callback executed when the context is loaded.
///
/// \param context The context loaded from the database.
void
got_context(const model::context& context)
{
if (_verbose)
print_context(context);
}
/// Callback executed when a test results is found.
///
/// \param iter Container for the test result's data.
void
got_result(store::results_iterator& iter)
{
if (!_start_time || _start_time.get() > iter.start_time())
_start_time = iter.start_time();
if (!_end_time || _end_time.get() < iter.end_time())
_end_time = iter.end_time();
const datetime::delta duration = iter.end_time() - iter.start_time();
_runtime += duration;
const model::test_result result = iter.result();
_results[result.type()].push_back(
result_data(iter.test_program()->relative_path(),
iter.test_case_name(), iter.result(), duration));
if (_verbose) {
// TODO(jmmv): _results_filters is a list and is small enough for
// std::find to not be an expensive operation here (probably). But
// we should be using a std::set instead.
if (std::find(_results_filters.begin(), _results_filters.end(),
iter.result().type()) != _results_filters.end()) {
print_test_case_and_result(iter);
}
}
}
/// Prints the tests summary.
void
end(const drivers::scan_results::result& /* r */)
{
typedef std::map< model::test_result_type, const char* > types_map;
types_map titles;
titles[model::test_result_broken] = "Broken tests";
titles[model::test_result_expected_failure] = "Expected failures";
titles[model::test_result_failed] = "Failed tests";
titles[model::test_result_passed] = "Passed tests";
titles[model::test_result_skipped] = "Skipped tests";
for (cli::result_types::const_iterator iter = _results_filters.begin();
iter != _results_filters.end(); ++iter) {
const types_map::const_iterator match = titles.find(*iter);
INV_MSG(match != titles.end(), "Conditional does not match user "
"input validation in parse_types()");
print_results((*match).first, (*match).second);
}
const std::size_t broken = count_results(model::test_result_broken);
const std::size_t failed = count_results(model::test_result_failed);
const std::size_t passed = count_results(model::test_result_passed);
const std::size_t skipped = count_results(model::test_result_skipped);
const std::size_t xfail = count_results(
model::test_result_expected_failure);
const std::size_t total = broken + failed + passed + skipped + xfail;
_output << "===> Summary\n";
_output << F("Results read from %s\n") % _results_file;
_output << F("Test cases: %s total, %s skipped, %s expected failures, "
"%s broken, %s failed\n") %
total % skipped % xfail % broken % failed;
if (_verbose && _start_time) {
INV(_end_time);
_output << F("Start time: %s\n") %
_start_time.get().to_iso8601_in_utc();
_output << F("End time: %s\n") %
_end_time.get().to_iso8601_in_utc();
}
_output << F("Total time: %s\n") % cli::format_delta(_runtime);
}
};
} // anonymous namespace
/// Default constructor for cmd_report.
cmd_report::cmd_report(void) : cli_command(
"report", "", 0, -1,
"Generates a report with the results of a test suite run")
{
add_option(results_file_open_option);
add_option(cmdline::bool_option(
"verbose", "Include the execution context and the details of each test "
"case in the report"));
add_option(cmdline::path_option("output", "Path to the output file", "path",
"/dev/stdout"));
add_option(results_filter_option);
}
/// Entry point for the "report" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
///
/// \return 0 if everything is OK, 1 if the statement is invalid or if there is
/// any other problem.
int
cmd_report::run(cmdline::ui* ui,
const cmdline::parsed_cmdline& cmdline,
const config::tree& /* user_config */)
{
std::auto_ptr< std::ostream > output = utils::open_ostream(
cmdline.get_option< cmdline::path_option >("output"));
const fs::path results_file = layout::find_results(
results_file_open(cmdline));
const result_types types = get_result_types(cmdline);
report_console_hooks hooks(*output.get(), cmdline.has_option("verbose"),
types, results_file);
const drivers::scan_results::result result = drivers::scan_results::drive(
results_file, parse_filters(cmdline.arguments()), hooks);
return report_unused_filters(result.unused_filters, ui) ?
EXIT_FAILURE : EXIT_SUCCESS;
}

54
cli/cmd_report.hpp Normal file
View File

@ -0,0 +1,54 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_report.hpp
/// Provides the cmd_report class.
#if !defined(CLI_CMD_REPORT_HPP)
#define CLI_CMD_REPORT_HPP
#include "cli/common.hpp"
namespace cli {
/// Implementation of the "report" subcommand.
class cmd_report : public cli_command
{
public:
cmd_report(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_REPORT_HPP)

474
cli/cmd_report_html.cpp Normal file
View File

@ -0,0 +1,474 @@
// Copyright 2012 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_report_html.hpp"
#include <algorithm>
#include <cerrno>
#include <cstdlib>
#include <set>
#include <stdexcept>
#include "cli/common.ipp"
#include "drivers/scan_results.hpp"
#include "engine/filters.hpp"
#include "model/context.hpp"
#include "model/metadata.hpp"
#include "model/test_case.hpp"
#include "model/test_program.hpp"
#include "model/test_result.hpp"
#include "store/layout.hpp"
#include "store/read_transaction.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/datetime.hpp"
#include "utils/env.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/exceptions.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
#include "utils/optional.ipp"
#include "utils/text/templates.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace datetime = utils::datetime;
namespace fs = utils::fs;
namespace layout = store::layout;
namespace text = utils::text;
using utils::optional;
namespace {
/// Creates the report's top directory and fails if it exists.
///
/// \param directory The directory to create.
/// \param force Whether to wipe an existing directory or not.
///
/// \throw std::runtime_error If the directory already exists; this is a user
/// error that the user must correct.
/// \throw fs::error If the directory creation fails for any other reason.
static void
create_top_directory(const fs::path& directory, const bool force)
{
if (force) {
if (fs::exists(directory))
fs::rm_r(directory);
}
try {
fs::mkdir(directory, 0755);
} catch (const fs::system_error& e) {
if (e.original_errno() == EEXIST)
throw std::runtime_error(F("Output directory '%s' already exists; "
"maybe use --force?") %
directory);
else
throw e;
}
}
/// Generates a flat unique filename for a given test case.
///
/// \param test_program The test program for which to genereate the name.
/// \param test_case_name The test case name.
///
/// \return A filename unique within a directory with a trailing HTML extension.
static std::string
test_case_filename(const model::test_program& test_program,
const std::string& test_case_name)
{
static const char* special_characters = "/:";
std::string name = cli::format_test_case_id(test_program, test_case_name);
std::string::size_type pos = name.find_first_of(special_characters);
while (pos != std::string::npos) {
name.replace(pos, 1, "_");
pos = name.find_first_of(special_characters, pos + 1);
}
return name + ".html";
}
/// Adds a string to string map to the templates.
///
/// \param [in,out] templates The templates to add the map to.
/// \param props The map to add to the templates.
/// \param key_vector Name of the template vector that holds the keys.
/// \param value_vector Name of the template vector that holds the values.
static void
add_map(text::templates_def& templates, const config::properties_map& props,
const std::string& key_vector, const std::string& value_vector)
{
templates.add_vector(key_vector);
templates.add_vector(value_vector);
for (config::properties_map::const_iterator iter = props.begin();
iter != props.end(); ++iter) {
templates.add_to_vector(key_vector, (*iter).first);
templates.add_to_vector(value_vector, (*iter).second);
}
}
/// Generates an HTML report.
class html_hooks : public drivers::scan_results::base_hooks {
/// User interface object where to report progress.
cmdline::ui* _ui;
/// The top directory in which to create the HTML files.
fs::path _directory;
/// Collection of result types to include in the report.
const cli::result_types& _results_filters;
/// The start time of the first test.
optional< utils::datetime::timestamp > _start_time;
/// The end time of the last test.
optional< utils::datetime::timestamp > _end_time;
/// The total run time of the tests. Note that we cannot subtract _end_time
/// from _start_time to compute this due to parallel execution.
utils::datetime::delta _runtime;
/// Templates accumulator to generate the index.html file.
text::templates_def _summary_templates;
/// Mapping of result types to the amount of tests with such result.
std::map< model::test_result_type, std::size_t > _types_count;
/// Generates a common set of templates for all of our files.
///
/// \return A new templates object with common parameters.
static text::templates_def
common_templates(void)
{
text::templates_def templates;
templates.add_variable("css", "report.css");
return templates;
}
/// Adds a test case result to the summary.
///
/// \param test_program The test program with the test case to be added.
/// \param test_case_name Name of the test case.
/// \param result The result of the test case.
/// \param has_detail If true, the result of the test case has not been
/// filtered and therefore there exists a separate file for the test
/// with all of its information.
void
add_to_summary(const model::test_program& test_program,
const std::string& test_case_name,
const model::test_result& result,
const bool has_detail)
{
++_types_count[result.type()];
if (!has_detail)
return;
std::string test_cases_vector;
std::string test_cases_file_vector;
switch (result.type()) {
case model::test_result_broken:
test_cases_vector = "broken_test_cases";
test_cases_file_vector = "broken_test_cases_file";
break;
case model::test_result_expected_failure:
test_cases_vector = "xfail_test_cases";
test_cases_file_vector = "xfail_test_cases_file";
break;
case model::test_result_failed:
test_cases_vector = "failed_test_cases";
test_cases_file_vector = "failed_test_cases_file";
break;
case model::test_result_passed:
test_cases_vector = "passed_test_cases";
test_cases_file_vector = "passed_test_cases_file";
break;
case model::test_result_skipped:
test_cases_vector = "skipped_test_cases";
test_cases_file_vector = "skipped_test_cases_file";
break;
}
INV(!test_cases_vector.empty());
INV(!test_cases_file_vector.empty());
_summary_templates.add_to_vector(
test_cases_vector,
cli::format_test_case_id(test_program, test_case_name));
_summary_templates.add_to_vector(
test_cases_file_vector,
test_case_filename(test_program, test_case_name));
}
/// Instantiate a template to generate an HTML file in the output directory.
///
/// \param templates The templates to use.
/// \param template_name The name of the template. This is automatically
/// searched for in the installed directory, so do not provide a path.
/// \param output_name The name of the output file. This is a basename to
/// be created within the output directory.
///
/// \throw text::error If there is any problem applying the templates.
void
generate(const text::templates_def& templates,
const std::string& template_name,
const std::string& output_name) const
{
const fs::path miscdir(utils::getenv_with_default(
"KYUA_MISCDIR", KYUA_MISCDIR));
const fs::path template_file = miscdir / template_name;
const fs::path output_path(_directory / output_name);
_ui->out(F("Generating %s") % output_path);
text::instantiate(templates, template_file, output_path);
}
/// Gets the number of tests with a given result type.
///
/// \param type The type to be queried.
///
/// \return The number of tests of the given type, or 0 if none have yet
/// been registered by add_to_summary().
std::size_t
get_count(const model::test_result_type type) const
{
const std::map< model::test_result_type, std::size_t >::const_iterator
iter = _types_count.find(type);
if (iter == _types_count.end())
return 0;
else
return (*iter).second;
}
public:
/// Constructor for the hooks.
///
/// \param ui_ User interface object where to report progress.
/// \param directory_ The directory in which to create the HTML files.
/// \param results_filters_ The result types to include in the report.
/// Cannot be empty.
html_hooks(cmdline::ui* ui_, const fs::path& directory_,
const cli::result_types& results_filters_) :
_ui(ui_),
_directory(directory_),
_results_filters(results_filters_),
_summary_templates(common_templates())
{
PRE(!results_filters_.empty());
// Keep in sync with add_to_summary().
_summary_templates.add_vector("broken_test_cases");
_summary_templates.add_vector("broken_test_cases_file");
_summary_templates.add_vector("xfail_test_cases");
_summary_templates.add_vector("xfail_test_cases_file");
_summary_templates.add_vector("failed_test_cases");
_summary_templates.add_vector("failed_test_cases_file");
_summary_templates.add_vector("passed_test_cases");
_summary_templates.add_vector("passed_test_cases_file");
_summary_templates.add_vector("skipped_test_cases");
_summary_templates.add_vector("skipped_test_cases_file");
}
/// Callback executed when the context is loaded.
///
/// \param context The context loaded from the database.
void
got_context(const model::context& context)
{
text::templates_def templates = common_templates();
templates.add_variable("cwd", context.cwd().str());
add_map(templates, context.env(), "env_var", "env_var_value");
generate(templates, "context.html", "context.html");
}
/// Callback executed when a test results is found.
///
/// \param iter Container for the test result's data.
void
got_result(store::results_iterator& iter)
{
const model::test_program_ptr test_program = iter.test_program();
const std::string& test_case_name = iter.test_case_name();
const model::test_result result = iter.result();
if (std::find(_results_filters.begin(), _results_filters.end(),
result.type()) == _results_filters.end()) {
add_to_summary(*test_program, test_case_name, result, false);
return;
}
add_to_summary(*test_program, test_case_name, result, true);
if (!_start_time || _start_time.get() > iter.start_time())
_start_time = iter.start_time();
if (!_end_time || _end_time.get() < iter.end_time())
_end_time = iter.end_time();
const datetime::delta duration = iter.end_time() - iter.start_time();
_runtime += duration;
text::templates_def templates = common_templates();
templates.add_variable("test_case",
cli::format_test_case_id(*test_program,
test_case_name));
templates.add_variable("test_program",
test_program->absolute_path().str());
templates.add_variable("result", cli::format_result(result));
templates.add_variable("start_time",
iter.start_time().to_iso8601_in_utc());
templates.add_variable("end_time",
iter.end_time().to_iso8601_in_utc());
templates.add_variable("duration", cli::format_delta(duration));
const model::test_case& test_case = test_program->find(test_case_name);
add_map(templates, test_case.get_metadata().to_properties(),
"metadata_var", "metadata_value");
{
const std::string stdout_text = iter.stdout_contents();
if (!stdout_text.empty())
templates.add_variable("stdout", stdout_text);
}
{
const std::string stderr_text = iter.stderr_contents();
if (!stderr_text.empty())
templates.add_variable("stderr", stderr_text);
}
generate(templates, "test_result.html",
test_case_filename(*test_program, test_case_name));
}
/// Writes the index.html file in the output directory.
///
/// This should only be called once all the processing has been done;
/// i.e. when the scan_results driver returns.
void
write_summary(void)
{
const std::size_t n_passed = get_count(model::test_result_passed);
const std::size_t n_failed = get_count(model::test_result_failed);
const std::size_t n_skipped = get_count(model::test_result_skipped);
const std::size_t n_xfail = get_count(
model::test_result_expected_failure);
const std::size_t n_broken = get_count(model::test_result_broken);
const std::size_t n_bad = n_broken + n_failed;
if (_start_time) {
INV(_end_time);
_summary_templates.add_variable(
"start_time", _start_time.get().to_iso8601_in_utc());
_summary_templates.add_variable(
"end_time", _end_time.get().to_iso8601_in_utc());
} else {
_summary_templates.add_variable("start_time", "No tests run");
_summary_templates.add_variable("end_time", "No tests run");
}
_summary_templates.add_variable("duration",
cli::format_delta(_runtime));
_summary_templates.add_variable("passed_tests_count",
F("%s") % n_passed);
_summary_templates.add_variable("failed_tests_count",
F("%s") % n_failed);
_summary_templates.add_variable("skipped_tests_count",
F("%s") % n_skipped);
_summary_templates.add_variable("xfail_tests_count",
F("%s") % n_xfail);
_summary_templates.add_variable("broken_tests_count",
F("%s") % n_broken);
_summary_templates.add_variable("bad_tests_count", F("%s") % n_bad);
generate(text::templates_def(), "report.css", "report.css");
generate(_summary_templates, "index.html", "index.html");
}
};
} // anonymous namespace
/// Default constructor for cmd_report_html.
cli::cmd_report_html::cmd_report_html(void) : cli_command(
"report-html", "", 0, 0,
"Generates an HTML report with the result of a test suite run")
{
add_option(results_file_open_option);
add_option(cmdline::bool_option(
"force", "Wipe the output directory before generating the new report; "
"use care"));
add_option(cmdline::path_option(
"output", "The directory in which to store the HTML files",
"path", "html"));
add_option(cmdline::list_option(
"results-filter", "Comma-separated list of result types to include in "
"the report", "types", "skipped,xfail,broken,failed"));
}
/// Entry point for the "report-html" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
///
/// \return 0 if everything is OK, 1 if the statement is invalid or if there is
/// any other problem.
int
cli::cmd_report_html::run(cmdline::ui* ui,
const cmdline::parsed_cmdline& cmdline,
const config::tree& /* user_config */)
{
const result_types types = get_result_types(cmdline);
const fs::path results_file = layout::find_results(
results_file_open(cmdline));
const fs::path directory =
cmdline.get_option< cmdline::path_option >("output");
create_top_directory(directory, cmdline.has_option("force"));
html_hooks hooks(ui, directory, types);
drivers::scan_results::drive(results_file,
std::set< engine::test_filter >(),
hooks);
hooks.write_summary();
return EXIT_SUCCESS;
}

55
cli/cmd_report_html.hpp Normal file
View File

@ -0,0 +1,55 @@
// Copyright 2012 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_report_html.hpp
/// Provides the cmd_report_html class.
#if !defined(CLI_CMD_REPORT_HTML_HPP)
#define CLI_CMD_REPORT_HTML_HPP
#include "cli/common.hpp"
#include "utils/cmdline/ui_fwd.hpp"
namespace cli {
/// Implementation of the "report-html" subcommand.
class cmd_report_html : public cli_command
{
public:
cmd_report_html(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_REPORT_HTML_HPP)

89
cli/cmd_report_junit.cpp Normal file
View File

@ -0,0 +1,89 @@
// Copyright 2014 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_report_junit.hpp"
#include <cstddef>
#include <cstdlib>
#include <set>
#include "cli/common.ipp"
#include "drivers/report_junit.hpp"
#include "drivers/scan_results.hpp"
#include "engine/filters.hpp"
#include "store/layout.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/defs.hpp"
#include "utils/optional.ipp"
#include "utils/stream.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace layout = store::layout;
using cli::cmd_report_junit;
using utils::optional;
/// Default constructor for cmd_report.
cmd_report_junit::cmd_report_junit(void) : cli_command(
"report-junit", "", 0, 0,
"Generates a JUnit report with the result of a test suite run")
{
add_option(results_file_open_option);
add_option(cmdline::path_option("output", "Path to the output file", "path",
"/dev/stdout"));
}
/// Entry point for the "report" subcommand.
///
/// \param cmdline Representation of the command line to the subcommand.
///
/// \return 0 if everything is OK, 1 if the statement is invalid or if there is
/// any other problem.
int
cmd_report_junit::run(cmdline::ui* /* ui */,
const cmdline::parsed_cmdline& cmdline,
const config::tree& /* user_config */)
{
const fs::path results_file = layout::find_results(
results_file_open(cmdline));
std::auto_ptr< std::ostream > output = utils::open_ostream(
cmdline.get_option< cmdline::path_option >("output"));
drivers::report_junit_hooks hooks(*output.get());
drivers::scan_results::drive(results_file,
std::set< engine::test_filter >(),
hooks);
return EXIT_SUCCESS;
}

54
cli/cmd_report_junit.hpp Normal file
View File

@ -0,0 +1,54 @@
// Copyright 2014 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_report_junit.hpp
/// Provides the cmd_report_junit class.
#if !defined(CLI_CMD_REPORT_JUNIT_HPP)
#define CLI_CMD_REPORT_JUNIT_HPP
#include "cli/common.hpp"
namespace cli {
/// Implementation of the "report-junit" subcommand.
class cmd_report_junit : public cli_command
{
public:
cmd_report_junit(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_REPORT_JUNIT_HPP)

186
cli/cmd_test.cpp Normal file
View File

@ -0,0 +1,186 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_test.hpp"
#include <cstdlib>
#include "cli/common.ipp"
#include "drivers/run_tests.hpp"
#include "model/test_program.hpp"
#include "model/test_result.hpp"
#include "store/layout.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/config/tree.ipp"
#include "utils/datetime.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/path.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace datetime = utils::datetime;
namespace fs = utils::fs;
namespace layout = store::layout;
using cli::cmd_test;
namespace {
/// Hooks to print a progress report of the execution of the tests.
class print_hooks : public drivers::run_tests::base_hooks {
/// Object to interact with the I/O of the program.
cmdline::ui* _ui;
/// Whether the tests are executed in parallel or not.
bool _parallel;
public:
/// The amount of positive test results found so far.
unsigned long good_count;
/// The amount of negative test results found so far.
unsigned long bad_count;
/// Constructor for the hooks.
///
/// \param ui_ Object to interact with the I/O of the program.
/// \param parallel_ True if we are executing more than one test at once.
print_hooks(cmdline::ui* ui_, const bool parallel_) :
_ui(ui_),
_parallel(parallel_),
good_count(0),
bad_count(0)
{
}
/// Called when the processing of a test case begins.
///
/// \param test_program The test program containing the test case.
/// \param test_case_name The name of the test case being executed.
virtual void
got_test_case(const model::test_program& test_program,
const std::string& test_case_name)
{
if (!_parallel) {
_ui->out(F("%s -> ") %
cli::format_test_case_id(test_program, test_case_name),
false);
}
}
/// Called when a result of a test case becomes available.
///
/// \param test_program The test program containing the test case.
/// \param test_case_name The name of the test case being executed.
/// \param result The result of the execution of the test case.
/// \param duration The time it took to run the test.
virtual void
got_result(const model::test_program& test_program,
const std::string& test_case_name,
const model::test_result& result,
const datetime::delta& duration)
{
if (_parallel) {
_ui->out(F("%s -> ") %
cli::format_test_case_id(test_program, test_case_name),
false);
}
_ui->out(F("%s [%s]") % cli::format_result(result) %
cli::format_delta(duration));
if (result.good())
good_count++;
else
bad_count++;
}
};
} // anonymous namespace
/// Default constructor for cmd_test.
cmd_test::cmd_test(void) : cli_command(
"test", "[test-program ...]", 0, -1, "Run tests")
{
add_option(build_root_option);
add_option(kyuafile_option);
add_option(results_file_create_option);
}
/// Entry point for the "test" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
/// \param user_config The runtime configuration of the program.
///
/// \return 0 if all tests passed, 1 otherwise.
int
cmd_test::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& user_config)
{
const layout::results_id_file_pair results = layout::new_db(
results_file_create(cmdline), kyuafile_path(cmdline).branch_path());
const bool parallel = (user_config.lookup< config::positive_int_node >(
"parallelism") > 1);
print_hooks hooks(ui, parallel);
const drivers::run_tests::result result = drivers::run_tests::drive(
kyuafile_path(cmdline), build_root_path(cmdline), results.second,
parse_filters(cmdline.arguments()), user_config, hooks);
int exit_code;
if (hooks.good_count > 0 || hooks.bad_count > 0) {
ui->out("");
if (!results.first.empty()) {
ui->out(F("Results file id is %s") % results.first);
}
ui->out(F("Results saved to %s") % results.second);
ui->out("");
ui->out(F("%s/%s passed (%s failed)") % hooks.good_count %
(hooks.good_count + hooks.bad_count) % hooks.bad_count);
exit_code = (hooks.bad_count == 0 ? EXIT_SUCCESS : EXIT_FAILURE);
} else {
// TODO(jmmv): Delete created empty file; it's useless!
if (!results.first.empty()) {
ui->out(F("Results file id is %s") % results.first);
}
ui->out(F("Results saved to %s") % results.second);
exit_code = EXIT_SUCCESS;
}
return report_unused_filters(result.unused_filters, ui) ?
EXIT_FAILURE : exit_code;
}

54
cli/cmd_test.hpp Normal file
View File

@ -0,0 +1,54 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/cmd_test.hpp
/// Provides the cmd_test class.
#if !defined(CLI_CMD_TEST_HPP)
#define CLI_CMD_TEST_HPP
#include "cli/common.hpp"
namespace cli {
/// Implementation of the "test" subcommand.
class cmd_test : public cli_command
{
public:
cmd_test(void);
int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};
} // namespace cli
#endif // !defined(CLI_CMD_TEST_HPP)

63
cli/cmd_test_test.cpp Normal file
View File

@ -0,0 +1,63 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/cmd_test.hpp"
#include <atf-c++.hpp>
#include "cli/common.ipp"
#include "engine/config.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/config/tree.ipp"
namespace cmdline = utils::cmdline;
ATF_TEST_CASE_WITHOUT_HEAD(invalid_filter);
ATF_TEST_CASE_BODY(invalid_filter)
{
cmdline::args_vector args;
args.push_back("test");
args.push_back("correct");
args.push_back("incorrect:");
cli::cmd_test cmd;
cmdline::ui_mock ui;
ATF_REQUIRE_THROW_RE(cmdline::error, "Test case.*'incorrect:'.*empty",
cmd.main(&ui, args, engine::default_config()));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, invalid_filter);
}

411
cli/common.cpp Normal file
View File

@ -0,0 +1,411 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/common.hpp"
#include <algorithm>
#include <fstream>
#include <iostream>
#include <stdexcept>
#include "engine/filters.hpp"
#include "model/test_program.hpp"
#include "model/test_result.hpp"
#include "store/layout.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/datetime.hpp"
#include "utils/env.hpp"
#include "utils/format/macros.hpp"
#include "utils/logging/macros.hpp"
#include "utils/fs/exceptions.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
#include "utils/optional.ipp"
#include "utils/sanity.hpp"
#if defined(HAVE_CONFIG_H)
# include "config.h"
#endif
namespace cmdline = utils::cmdline;
namespace datetime = utils::datetime;
namespace fs = utils::fs;
namespace layout = store::layout;
using utils::none;
using utils::optional;
/// Standard definition of the option to specify the build root.
const cmdline::path_option cli::build_root_option(
"build-root",
"Path to the built test programs, if different from the location of the "
"Kyuafile scripts",
"path");
/// Standard definition of the option to specify a Kyuafile.
const cmdline::path_option cli::kyuafile_option(
'k', "kyuafile",
"Path to the test suite definition",
"file", "Kyuafile");
/// Standard definition of the option to specify filters on test results.
const cmdline::list_option cli::results_filter_option(
"results-filter", "Comma-separated list of result types to include in "
"the report", "types", "skipped,xfail,broken,failed");
/// Standard definition of the option to specify the results file.
///
/// TODO(jmmv): Should support a git-like syntax to go back in time, like
/// --results-file=LATEST^N where N indicates how many runs to go back to.
const cmdline::string_option cli::results_file_create_option(
'r', "results-file",
"Path to the results file to create; if left to the default value, the "
"name of the file is automatically computed for the current test suite",
"file", layout::results_auto_create_name);
/// Standard definition of the option to specify the results file.
///
/// TODO(jmmv): Should support a git-like syntax to go back in time, like
/// --results-file=LATEST^N where N indicates how many runs to go back to.
const cmdline::string_option cli::results_file_open_option(
'r', "results-file",
"Path to the results file to open or the identifier of the current test "
"suite or a previous results file for automatic lookup; if left to the "
"default value, uses the current directory as the test suite name",
"file", layout::results_auto_open_name);
namespace {
/// Gets the path to the historical database if it exists.
///
/// TODO(jmmv): This function should go away. It only exists as a temporary
/// transitional path to force the use of the stale ~/.kyua/store.db if it
/// exists.
///
/// \return A path if the file is found; none otherwise.
static optional< fs::path >
get_historical_db(void)
{
optional< fs::path > home = utils::get_home();
if (home) {
const fs::path old_db = home.get() / ".kyua/store.db";
if (fs::exists(old_db)) {
if (old_db.is_absolute())
return utils::make_optional(old_db);
else
return utils::make_optional(old_db.to_absolute());
} else {
return none;
}
} else {
return none;
}
}
/// Converts a set of result type names to identifiers.
///
/// \param names The collection of names to process; may be empty.
///
/// \return The result type identifiers corresponding to the input names.
///
/// \throw std::runtime_error If any name in the input names is invalid.
static cli::result_types
parse_types(const std::vector< std::string >& names)
{
typedef std::map< std::string, model::test_result_type > types_map;
types_map valid_types;
valid_types["broken"] = model::test_result_broken;
valid_types["failed"] = model::test_result_failed;
valid_types["passed"] = model::test_result_passed;
valid_types["skipped"] = model::test_result_skipped;
valid_types["xfail"] = model::test_result_expected_failure;
cli::result_types types;
for (std::vector< std::string >::const_iterator iter = names.begin();
iter != names.end(); ++iter) {
const types_map::const_iterator match = valid_types.find(*iter);
if (match == valid_types.end())
throw std::runtime_error(F("Unknown result type '%s'") % *iter);
else
types.push_back((*match).second);
}
return types;
}
} // anonymous namespace
/// Gets the path to the build root, if any.
///
/// This is just syntactic sugar to simplify quierying the 'build_root_option'.
///
/// \param cmdline The parsed command line.
///
/// \return The path to the build root, if specified; none otherwise.
optional< fs::path >
cli::build_root_path(const cmdline::parsed_cmdline& cmdline)
{
optional< fs::path > build_root;
if (cmdline.has_option(build_root_option.long_name()))
build_root = cmdline.get_option< cmdline::path_option >(
build_root_option.long_name());
return build_root;
}
/// Gets the path to the Kyuafile to be loaded.
///
/// This is just syntactic sugar to simplify quierying the 'kyuafile_option'.
///
/// \param cmdline The parsed command line.
///
/// \return The path to the Kyuafile to be loaded.
fs::path
cli::kyuafile_path(const cmdline::parsed_cmdline& cmdline)
{
return cmdline.get_option< cmdline::path_option >(
kyuafile_option.long_name());
}
/// Gets the value of the results-file flag for the creation of a new file.
///
/// \param cmdline The parsed command line from which to extract any possible
/// override for the location of the database via the --results-file flag.
///
/// \return The path to the database to be used.
///
/// \throw cmdline::error If the value passed to the flag is invalid.
std::string
cli::results_file_create(const cmdline::parsed_cmdline& cmdline)
{
std::string results_file = cmdline.get_option< cmdline::string_option >(
results_file_create_option.long_name());
if (results_file == results_file_create_option.default_value()) {
const optional< fs::path > historical_db = get_historical_db();
if (historical_db)
results_file = historical_db.get().str();
} else {
try {
(void)fs::path(results_file);
} catch (const fs::error& e) {
throw cmdline::usage_error(F("Invalid value passed to --%s") %
results_file_create_option.long_name());
}
}
return results_file;
}
/// Gets the value of the results-file flag for the lookup of the file.
///
/// \param cmdline The parsed command line from which to extract any possible
/// override for the location of the database via the --results-file flag.
///
/// \return The path to the database to be used.
///
/// \throw cmdline::error If the value passed to the flag is invalid.
std::string
cli::results_file_open(const cmdline::parsed_cmdline& cmdline)
{
std::string results_file = cmdline.get_option< cmdline::string_option >(
results_file_open_option.long_name());
if (results_file == results_file_open_option.default_value()) {
const optional< fs::path > historical_db = get_historical_db();
if (historical_db)
results_file = historical_db.get().str();
} else {
try {
(void)fs::path(results_file);
} catch (const fs::error& e) {
throw cmdline::usage_error(F("Invalid value passed to --%s") %
results_file_open_option.long_name());
}
}
return results_file;
}
/// Gets the filters for the result types.
///
/// \param cmdline The parsed command line.
///
/// \return A collection of result types to be used for filtering.
///
/// \throw std::runtime_error If any of the user-provided filters is invalid.
cli::result_types
cli::get_result_types(const utils::cmdline::parsed_cmdline& cmdline)
{
result_types types = parse_types(
cmdline.get_option< cmdline::list_option >("results-filter"));
if (types.empty()) {
types.push_back(model::test_result_passed);
types.push_back(model::test_result_skipped);
types.push_back(model::test_result_expected_failure);
types.push_back(model::test_result_broken);
types.push_back(model::test_result_failed);
}
return types;
}
/// Parses a set of command-line arguments to construct test filters.
///
/// \param args The command-line arguments representing test filters.
///
/// \return A set of test filters.
///
/// \throw cmdline:error If any of the arguments is invalid, or if they
/// represent a non-disjoint collection of filters.
std::set< engine::test_filter >
cli::parse_filters(const cmdline::args_vector& args)
{
std::set< engine::test_filter > filters;
try {
for (cmdline::args_vector::const_iterator iter = args.begin();
iter != args.end(); iter++) {
const engine::test_filter filter(engine::test_filter::parse(*iter));
if (filters.find(filter) != filters.end())
throw cmdline::error(F("Duplicate filter '%s'") % filter.str());
filters.insert(filter);
}
check_disjoint_filters(filters);
} catch (const std::runtime_error& e) {
throw cmdline::error(e.what());
}
return filters;
}
/// Reports the filters that have not matched any tests as errors.
///
/// \param unused The collection of unused filters to report.
/// \param ui The user interface object through which errors are to be reported.
///
/// \return True if there are any unused filters. The caller should report this
/// as an error to the user by means of a non-successful exit code.
bool
cli::report_unused_filters(const std::set< engine::test_filter >& unused,
cmdline::ui* ui)
{
for (std::set< engine::test_filter >::const_iterator iter = unused.begin();
iter != unused.end(); iter++) {
cmdline::print_warning(ui, F("No test cases matched by the filter "
"'%s'.") % (*iter).str());
}
return !unused.empty();
}
/// Formats a time delta for user presentation.
///
/// \param delta The time delta to format.
///
/// \return A user-friendly representation of the time delta.
std::string
cli::format_delta(const datetime::delta& delta)
{
return F("%.3ss") % (delta.seconds + (delta.useconds / 1000000.0));
}
/// Formats a test case result for user presentation.
///
/// \param result The result to format.
///
/// \return A user-friendly representation of the result.
std::string
cli::format_result(const model::test_result& result)
{
std::string text;
switch (result.type()) {
case model::test_result_broken: text = "broken"; break;
case model::test_result_expected_failure: text = "expected_failure"; break;
case model::test_result_failed: text = "failed"; break;
case model::test_result_passed: text = "passed"; break;
case model::test_result_skipped: text = "skipped"; break;
}
INV(!text.empty());
if (!result.reason().empty())
text += ": " + result.reason();
return text;
}
/// Formats the identifier of a test case for user presentation.
///
/// \param test_program The test program containing the test case.
/// \param test_case_name The name of the test case.
///
/// \return A string representing the test case uniquely within a test suite.
std::string
cli::format_test_case_id(const model::test_program& test_program,
const std::string& test_case_name)
{
return F("%s:%s") % test_program.relative_path() % test_case_name;
}
/// Formats a filter using the same syntax of a test case.
///
/// \param test_filter The filter to format.
///
/// \return A string representing the test filter.
std::string
cli::format_test_case_id(const engine::test_filter& test_filter)
{
return F("%s:%s") % test_filter.test_program % test_filter.test_case;
}
/// Prints the version header information to the interface output.
///
/// \param ui Interface to which to write the version details.
void
cli::write_version_header(utils::cmdline::ui* ui)
{
ui->out(PACKAGE " (" PACKAGE_NAME ") " PACKAGE_VERSION);
}

104
cli/common.hpp Normal file
View File

@ -0,0 +1,104 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/common.hpp
/// Utility functions to implement CLI subcommands.
#if !defined(CLI_COMMON_HPP)
#define CLI_COMMON_HPP
#include <memory>
#include <set>
#include <vector>
#include "engine/filters_fwd.hpp"
#include "model/test_program_fwd.hpp"
#include "model/test_result.hpp"
#include "utils/cmdline/base_command.hpp"
#include "utils/cmdline/options_fwd.hpp"
#include "utils/cmdline/parser_fwd.hpp"
#include "utils/cmdline/ui_fwd.hpp"
#include "utils/config/tree_fwd.hpp"
#include "utils/datetime_fwd.hpp"
#include "utils/fs/path_fwd.hpp"
#include "utils/optional_fwd.hpp"
namespace cli {
extern const utils::cmdline::path_option build_root_option;
extern const utils::cmdline::path_option kyuafile_option;
extern const utils::cmdline::string_option results_file_create_option;
extern const utils::cmdline::string_option results_file_open_option;
extern const utils::cmdline::list_option results_filter_option;
extern const utils::cmdline::property_option variable_option;
/// Base type for commands defined in the cli module.
///
/// All commands in Kyua receive a configuration object as their runtime
/// data parameter because the configuration file applies to all the
/// commands.
typedef utils::cmdline::base_command< utils::config::tree > cli_command;
/// Scoped, strictly owned pointer to a cli_command.
typedef std::auto_ptr< cli_command > cli_command_ptr;
/// Collection of result types.
///
/// This is a vector rather than a set because we want to respect the order in
/// which the user provided the types.
typedef std::vector< model::test_result_type > result_types;
utils::optional< utils::fs::path > build_root_path(
const utils::cmdline::parsed_cmdline&);
utils::fs::path kyuafile_path(const utils::cmdline::parsed_cmdline&);
std::string results_file_create(const utils::cmdline::parsed_cmdline&);
std::string results_file_open(const utils::cmdline::parsed_cmdline&);
result_types get_result_types(const utils::cmdline::parsed_cmdline&);
std::set< engine::test_filter > parse_filters(
const utils::cmdline::args_vector&);
bool report_unused_filters(const std::set< engine::test_filter >&,
utils::cmdline::ui*);
std::string format_delta(const utils::datetime::delta&);
std::string format_result(const model::test_result&);
std::string format_test_case_id(const model::test_program&, const std::string&);
std::string format_test_case_id(const engine::test_filter&);
void write_version_header(utils::cmdline::ui*);
} // namespace cli
#endif // !defined(CLI_COMMON_HPP)

30
cli/common.ipp Normal file
View File

@ -0,0 +1,30 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/common.hpp"
#include "utils/cmdline/base_command.ipp"

488
cli/common_test.cpp Normal file
View File

@ -0,0 +1,488 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/common.hpp"
#include <fstream>
#include <atf-c++.hpp>
#include "engine/exceptions.hpp"
#include "engine/filters.hpp"
#include "model/metadata.hpp"
#include "model/test_program.hpp"
#include "model/test_result.hpp"
#include "store/layout.hpp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/globals.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/datetime.hpp"
#include "utils/env.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/exceptions.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
#include "utils/optional.ipp"
#include "utils/sanity.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace datetime = utils::datetime;
namespace fs = utils::fs;
namespace layout = store::layout;
using utils::optional;
namespace {
/// Syntactic sugar to instantiate engine::test_filter objects.
///
/// \param test_program Test program.
/// \param test_case Test case.
///
/// \return A \code test_filter \endcode object, based on \p test_program and
/// \p test_case.
inline engine::test_filter
mkfilter(const char* test_program, const char* test_case)
{
return engine::test_filter(fs::path(test_program), test_case);
}
} // anonymous namespace
ATF_TEST_CASE_WITHOUT_HEAD(build_root_path__default);
ATF_TEST_CASE_BODY(build_root_path__default)
{
std::map< std::string, std::vector< std::string > > options;
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE(!cli::build_root_path(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(build_root_path__explicit);
ATF_TEST_CASE_BODY(build_root_path__explicit)
{
std::map< std::string, std::vector< std::string > > options;
options["build-root"].push_back("/my//path");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE(cli::build_root_path(mock_cmdline));
ATF_REQUIRE_EQ("/my/path", cli::build_root_path(mock_cmdline).get().str());
}
ATF_TEST_CASE_WITHOUT_HEAD(kyuafile_path__default);
ATF_TEST_CASE_BODY(kyuafile_path__default)
{
std::map< std::string, std::vector< std::string > > options;
options["kyuafile"].push_back(cli::kyuafile_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_EQ(cli::kyuafile_option.default_value(),
cli::kyuafile_path(mock_cmdline).str());
}
ATF_TEST_CASE_WITHOUT_HEAD(kyuafile_path__explicit);
ATF_TEST_CASE_BODY(kyuafile_path__explicit)
{
std::map< std::string, std::vector< std::string > > options;
options["kyuafile"].push_back("/my//path");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_EQ("/my/path", cli::kyuafile_path(mock_cmdline).str());
}
ATF_TEST_CASE_WITHOUT_HEAD(result_types__default);
ATF_TEST_CASE_BODY(result_types__default)
{
std::map< std::string, std::vector< std::string > > options;
options["results-filter"].push_back(
cli::results_filter_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
cli::result_types exp_types;
exp_types.push_back(model::test_result_skipped);
exp_types.push_back(model::test_result_expected_failure);
exp_types.push_back(model::test_result_broken);
exp_types.push_back(model::test_result_failed);
ATF_REQUIRE(exp_types == cli::get_result_types(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(result_types__empty);
ATF_TEST_CASE_BODY(result_types__empty)
{
std::map< std::string, std::vector< std::string > > options;
options["results-filter"].push_back("");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
cli::result_types exp_types;
exp_types.push_back(model::test_result_passed);
exp_types.push_back(model::test_result_skipped);
exp_types.push_back(model::test_result_expected_failure);
exp_types.push_back(model::test_result_broken);
exp_types.push_back(model::test_result_failed);
ATF_REQUIRE(exp_types == cli::get_result_types(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(result_types__explicit__all);
ATF_TEST_CASE_BODY(result_types__explicit__all)
{
std::map< std::string, std::vector< std::string > > options;
options["results-filter"].push_back("passed,skipped,xfail,broken,failed");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
cli::result_types exp_types;
exp_types.push_back(model::test_result_passed);
exp_types.push_back(model::test_result_skipped);
exp_types.push_back(model::test_result_expected_failure);
exp_types.push_back(model::test_result_broken);
exp_types.push_back(model::test_result_failed);
ATF_REQUIRE(exp_types == cli::get_result_types(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(result_types__explicit__some);
ATF_TEST_CASE_BODY(result_types__explicit__some)
{
std::map< std::string, std::vector< std::string > > options;
options["results-filter"].push_back("skipped,broken");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
cli::result_types exp_types;
exp_types.push_back(model::test_result_skipped);
exp_types.push_back(model::test_result_broken);
ATF_REQUIRE(exp_types == cli::get_result_types(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(result_types__explicit__invalid);
ATF_TEST_CASE_BODY(result_types__explicit__invalid)
{
std::map< std::string, std::vector< std::string > > options;
options["results-filter"].push_back("skipped,foo,broken");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_THROW_RE(std::runtime_error, "Unknown result type 'foo'",
cli::get_result_types(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(results_file_create__default__new);
ATF_TEST_CASE_BODY(results_file_create__default__new)
{
std::map< std::string, std::vector< std::string > > options;
options["results-file"].push_back(
cli::results_file_create_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const fs::path home("homedir");
utils::setenv("HOME", home.str());
ATF_REQUIRE_EQ(cli::results_file_create_option.default_value(),
cli::results_file_create(mock_cmdline));
ATF_REQUIRE(!fs::exists(home / ".kyua"));
}
ATF_TEST_CASE_WITHOUT_HEAD(results_file_create__default__historical);
ATF_TEST_CASE_BODY(results_file_create__default__historical)
{
std::map< std::string, std::vector< std::string > > options;
options["results-file"].push_back(
cli::results_file_create_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const fs::path home("homedir");
utils::setenv("HOME", home.str());
fs::mkdir_p(fs::path("homedir/.kyua"), 0755);
atf::utils::create_file("homedir/.kyua/store.db", "fake store");
ATF_REQUIRE_EQ(fs::path("homedir/.kyua/store.db").to_absolute(),
fs::path(cli::results_file_create(mock_cmdline)));
}
ATF_TEST_CASE_WITHOUT_HEAD(results_file_create__explicit);
ATF_TEST_CASE_BODY(results_file_create__explicit)
{
std::map< std::string, std::vector< std::string > > options;
options["results-file"].push_back("/my//path/f.db");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_EQ("/my//path/f.db",
cli::results_file_create(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(results_file_open__default__latest);
ATF_TEST_CASE_BODY(results_file_open__default__latest)
{
std::map< std::string, std::vector< std::string > > options;
options["results-file"].push_back(
cli::results_file_open_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const fs::path home("homedir");
utils::setenv("HOME", home.str());
ATF_REQUIRE_EQ(cli::results_file_open_option.default_value(),
cli::results_file_open(mock_cmdline));
ATF_REQUIRE(!fs::exists(home / ".kyua"));
}
ATF_TEST_CASE_WITHOUT_HEAD(results_file_open__default__historical);
ATF_TEST_CASE_BODY(results_file_open__default__historical)
{
std::map< std::string, std::vector< std::string > > options;
options["results-file"].push_back(
cli::results_file_open_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const fs::path home("homedir");
utils::setenv("HOME", home.str());
fs::mkdir_p(fs::path("homedir/.kyua"), 0755);
atf::utils::create_file("homedir/.kyua/store.db", "fake store");
ATF_REQUIRE_EQ(fs::path("homedir/.kyua/store.db").to_absolute(),
fs::path(cli::results_file_open(mock_cmdline)));
}
ATF_TEST_CASE_WITHOUT_HEAD(results_file_open__explicit);
ATF_TEST_CASE_BODY(results_file_open__explicit)
{
std::map< std::string, std::vector< std::string > > options;
options["results-file"].push_back("/my//path/f.db");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_EQ("/my//path/f.db", cli::results_file_open(mock_cmdline));
}
ATF_TEST_CASE_WITHOUT_HEAD(parse_filters__none);
ATF_TEST_CASE_BODY(parse_filters__none)
{
const cmdline::args_vector args;
const std::set< engine::test_filter > filters = cli::parse_filters(args);
ATF_REQUIRE(filters.empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(parse_filters__ok);
ATF_TEST_CASE_BODY(parse_filters__ok)
{
cmdline::args_vector args;
args.push_back("foo");
args.push_back("bar/baz");
args.push_back("other:abc");
args.push_back("other:bcd");
const std::set< engine::test_filter > filters = cli::parse_filters(args);
std::set< engine::test_filter > exp_filters;
exp_filters.insert(mkfilter("foo", ""));
exp_filters.insert(mkfilter("bar/baz", ""));
exp_filters.insert(mkfilter("other", "abc"));
exp_filters.insert(mkfilter("other", "bcd"));
ATF_REQUIRE(exp_filters == filters);
}
ATF_TEST_CASE_WITHOUT_HEAD(parse_filters__duplicate);
ATF_TEST_CASE_BODY(parse_filters__duplicate)
{
cmdline::args_vector args;
args.push_back("foo/bar//baz");
args.push_back("hello/world:yes");
args.push_back("foo//bar/baz");
ATF_REQUIRE_THROW_RE(cmdline::error, "Duplicate.*'foo/bar/baz'",
cli::parse_filters(args));
}
ATF_TEST_CASE_WITHOUT_HEAD(parse_filters__nondisjoint);
ATF_TEST_CASE_BODY(parse_filters__nondisjoint)
{
cmdline::args_vector args;
args.push_back("foo/bar");
args.push_back("hello/world:yes");
args.push_back("foo/bar:baz");
ATF_REQUIRE_THROW_RE(cmdline::error, "'foo/bar'.*'foo/bar:baz'.*disjoint",
cli::parse_filters(args));
}
ATF_TEST_CASE_WITHOUT_HEAD(report_unused_filters__none);
ATF_TEST_CASE_BODY(report_unused_filters__none)
{
std::set< engine::test_filter > unused;
cmdline::ui_mock ui;
ATF_REQUIRE(!cli::report_unused_filters(unused, &ui));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(ui.err_log().empty());
}
ATF_TEST_CASE_WITHOUT_HEAD(report_unused_filters__some);
ATF_TEST_CASE_BODY(report_unused_filters__some)
{
std::set< engine::test_filter > unused;
unused.insert(mkfilter("a/b", ""));
unused.insert(mkfilter("hey/d", "yes"));
cmdline::ui_mock ui;
cmdline::init("progname");
ATF_REQUIRE(cli::report_unused_filters(unused, &ui));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE_EQ(2, ui.err_log().size());
ATF_REQUIRE( atf::utils::grep_collection("No.*matched.*'a/b'",
ui.err_log()));
ATF_REQUIRE( atf::utils::grep_collection("No.*matched.*'hey/d:yes'",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(format_delta);
ATF_TEST_CASE_BODY(format_delta)
{
ATF_REQUIRE_EQ("0.000s", cli::format_delta(datetime::delta()));
ATF_REQUIRE_EQ("0.012s", cli::format_delta(datetime::delta(0, 12300)));
ATF_REQUIRE_EQ("0.999s", cli::format_delta(datetime::delta(0, 999000)));
ATF_REQUIRE_EQ("51.321s", cli::format_delta(datetime::delta(51, 321000)));
}
ATF_TEST_CASE_WITHOUT_HEAD(format_result__no_reason);
ATF_TEST_CASE_BODY(format_result__no_reason)
{
ATF_REQUIRE_EQ("passed", cli::format_result(
model::test_result(model::test_result_passed)));
ATF_REQUIRE_EQ("failed", cli::format_result(
model::test_result(model::test_result_failed)));
}
ATF_TEST_CASE_WITHOUT_HEAD(format_result__with_reason);
ATF_TEST_CASE_BODY(format_result__with_reason)
{
ATF_REQUIRE_EQ("broken: Something", cli::format_result(
model::test_result(model::test_result_broken, "Something")));
ATF_REQUIRE_EQ("expected_failure: A B C", cli::format_result(
model::test_result(model::test_result_expected_failure, "A B C")));
ATF_REQUIRE_EQ("failed: More text", cli::format_result(
model::test_result(model::test_result_failed, "More text")));
ATF_REQUIRE_EQ("skipped: Bye", cli::format_result(
model::test_result(model::test_result_skipped, "Bye")));
}
ATF_TEST_CASE_WITHOUT_HEAD(format_test_case_id__test_case);
ATF_TEST_CASE_BODY(format_test_case_id__test_case)
{
const model::test_program test_program = model::test_program_builder(
"mock", fs::path("foo/bar/baz"), fs::path("unused-root"),
"unused-suite-name")
.add_test_case("abc")
.build();
ATF_REQUIRE_EQ("foo/bar/baz:abc",
cli::format_test_case_id(test_program, "abc"));
}
ATF_TEST_CASE_WITHOUT_HEAD(format_test_case_id__test_filter);
ATF_TEST_CASE_BODY(format_test_case_id__test_filter)
{
const engine::test_filter filter(fs::path("foo/bar"), "baz");
ATF_REQUIRE_EQ("foo/bar:baz", cli::format_test_case_id(filter));
}
ATF_TEST_CASE_WITHOUT_HEAD(write_version_header);
ATF_TEST_CASE_BODY(write_version_header)
{
cmdline::ui_mock ui;
cli::write_version_header(&ui);
ATF_REQUIRE_EQ(1, ui.out_log().size());
ATF_REQUIRE_MATCH("^kyua .*[0-9]+\\.[0-9]+$", ui.out_log()[0]);
ATF_REQUIRE(ui.err_log().empty());
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, build_root_path__default);
ATF_ADD_TEST_CASE(tcs, build_root_path__explicit);
ATF_ADD_TEST_CASE(tcs, kyuafile_path__default);
ATF_ADD_TEST_CASE(tcs, kyuafile_path__explicit);
ATF_ADD_TEST_CASE(tcs, result_types__default);
ATF_ADD_TEST_CASE(tcs, result_types__empty);
ATF_ADD_TEST_CASE(tcs, result_types__explicit__all);
ATF_ADD_TEST_CASE(tcs, result_types__explicit__some);
ATF_ADD_TEST_CASE(tcs, result_types__explicit__invalid);
ATF_ADD_TEST_CASE(tcs, results_file_create__default__new);
ATF_ADD_TEST_CASE(tcs, results_file_create__default__historical);
ATF_ADD_TEST_CASE(tcs, results_file_create__explicit);
ATF_ADD_TEST_CASE(tcs, results_file_open__default__latest);
ATF_ADD_TEST_CASE(tcs, results_file_open__default__historical);
ATF_ADD_TEST_CASE(tcs, results_file_open__explicit);
ATF_ADD_TEST_CASE(tcs, parse_filters__none);
ATF_ADD_TEST_CASE(tcs, parse_filters__ok);
ATF_ADD_TEST_CASE(tcs, parse_filters__duplicate);
ATF_ADD_TEST_CASE(tcs, parse_filters__nondisjoint);
ATF_ADD_TEST_CASE(tcs, report_unused_filters__none);
ATF_ADD_TEST_CASE(tcs, report_unused_filters__some);
ATF_ADD_TEST_CASE(tcs, format_delta);
ATF_ADD_TEST_CASE(tcs, format_result__no_reason);
ATF_ADD_TEST_CASE(tcs, format_result__with_reason);
ATF_ADD_TEST_CASE(tcs, format_test_case_id__test_case);
ATF_ADD_TEST_CASE(tcs, format_test_case_id__test_filter);
ATF_ADD_TEST_CASE(tcs, write_version_header);
}

223
cli/config.cpp Normal file
View File

@ -0,0 +1,223 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/config.hpp"
#include "cli/common.hpp"
#include "engine/config.hpp"
#include "engine/exceptions.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/config/tree.ipp"
#include "utils/format/macros.hpp"
#include "utils/fs/exceptions.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
#include "utils/env.hpp"
#include "utils/logging/macros.hpp"
#include "utils/optional.ipp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
using utils::optional;
namespace {
/// Basename of the configuration file.
static const char* config_basename = "kyua.conf";
/// Magic string to disable loading of configuration files.
static const char* none_config = "none";
/// Textual description of the default configuration files.
///
/// This is just an auxiliary string required to define the option below, which
/// requires a pointer to a static C string.
///
/// \todo If the user overrides the KYUA_CONFDIR environment variable, we don't
/// reflect this fact here. We don't want to query the variable during program
/// initialization due to the side-effects it may have. Therefore, fixing this
/// is tricky as it may require a whole rethink of this module.
static const std::string config_lookup_names =
(fs::path("~/.kyua") / config_basename).str() + " or " +
(fs::path(KYUA_CONFDIR) / config_basename).str();
/// Loads the configuration file for this session, if any.
///
/// This is a helper function that does not apply user-specified overrides. See
/// the documentation for cli::load_config() for more details.
///
/// \param cmdline The parsed command line.
///
/// \return The loaded configuration file, or the configuration defaults if the
/// loading is disabled.
///
/// \throw engine::error If the parsing of the configuration file fails.
/// TODO(jmmv): I'm not sure if this is the raised exception. And even if
/// it is, we should make it more accurate.
config::tree
load_config_file(const cmdline::parsed_cmdline& cmdline)
{
// TODO(jmmv): We should really be able to use cmdline.has_option here to
// detect whether the option was provided or not instead of checking against
// the default value.
const fs::path filename = cmdline.get_option< cmdline::path_option >(
cli::config_option.long_name());
if (filename.str() == none_config) {
LD("Configuration loading disabled; using defaults");
return engine::default_config();
} else if (filename.str() != cli::config_option.default_value())
return engine::load_config(filename);
const optional< fs::path > home = utils::get_home();
if (home) {
const fs::path path = home.get() / ".kyua" / config_basename;
try {
if (fs::exists(path))
return engine::load_config(path);
} catch (const fs::error& e) {
// Fall through. If we fail to load the user-specific configuration
// file because it cannot be openend, we try to load the system-wide
// one.
LW(F("Failed to load user-specific configuration file '%s': %s") %
path % e.what());
}
}
const fs::path confdir(utils::getenv_with_default(
"KYUA_CONFDIR", KYUA_CONFDIR));
const fs::path path = confdir / config_basename;
if (fs::exists(path)) {
return engine::load_config(path);
} else {
return engine::default_config();
}
}
/// Loads the configuration file for this session, if any.
///
/// This is a helper function for cli::load_config() that attempts to load the
/// configuration unconditionally.
///
/// \param cmdline The parsed command line.
///
/// \return The loaded configuration file data.
///
/// \throw engine::error If the parsing of the configuration file fails.
static config::tree
load_required_config(const cmdline::parsed_cmdline& cmdline)
{
config::tree user_config = load_config_file(cmdline);
if (cmdline.has_option(cli::variable_option.long_name())) {
typedef std::pair< std::string, std::string > override_pair;
const std::vector< override_pair >& overrides =
cmdline.get_multi_option< cmdline::property_option >(
cli::variable_option.long_name());
for (std::vector< override_pair >::const_iterator
iter = overrides.begin(); iter != overrides.end(); iter++) {
try {
user_config.set_string((*iter).first, (*iter).second);
} catch (const config::error& e) {
// TODO(jmmv): Raising this type from here is obviously the
// wrong thing to do.
throw engine::error(e.what());
}
}
}
return user_config;
}
} // anonymous namespace
/// Standard definition of the option to specify a configuration file.
///
/// You must use load_config() to load a configuration file while honoring the
/// value of this flag.
const cmdline::path_option cli::config_option(
'c', "config",
(std::string("Path to the configuration file; '") + none_config +
"' to disable loading").c_str(),
"file", config_lookup_names.c_str());
/// Standard definition of the option to specify a configuration variable.
const cmdline::property_option cli::variable_option(
'v', "variable",
"Overrides a particular configuration variable",
"K=V");
/// Loads the configuration file for this session, if any.
///
/// The algorithm implemented here is as follows:
/// 1) If ~/.kyua/kyua.conf exists, load it.
/// 2) Otherwise, if sysconfdir/kyua.conf exists, load it.
/// 3) Otherwise, use the built-in settings.
/// 4) Lastly, apply any user-provided overrides.
///
/// \param cmdline The parsed command line.
/// \param required Whether the loading of the configuration file must succeed.
/// Some commands should run regardless, and therefore we need to set this
/// to false for those commands.
///
/// \return The loaded configuration file data. If required was set to false,
/// this might be the default configuration data if the requested file could not
/// be properly loaded.
///
/// \throw engine::error If the parsing of the configuration file fails.
config::tree
cli::load_config(const cmdline::parsed_cmdline& cmdline,
const bool required)
{
try {
return load_required_config(cmdline);
} catch (const engine::error& e) {
if (required) {
throw;
} else {
LW(F("Ignoring failure to load configuration because the requested "
"command should not fail: %s") % e.what());
return engine::default_config();
}
}
}

55
cli/config.hpp Normal file
View File

@ -0,0 +1,55 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/config.hpp
/// Utility functions to load configuration files.
///
/// \todo All this should probably just be merged into the main module
/// as nothing else should have access to this.
#if !defined(CLI_CONFIG_HPP)
#define CLI_CONFIG_HPP
#include "utils/cmdline/options_fwd.hpp"
#include "utils/cmdline/parser_fwd.hpp"
#include "utils/config/tree_fwd.hpp"
namespace cli {
extern const utils::cmdline::path_option config_option;
extern const utils::cmdline::property_option variable_option;
utils::config::tree load_config(const utils::cmdline::parsed_cmdline&,
const bool);
} // namespace cli
#endif // !defined(CLI_CONFIG_HPP)

351
cli/config_test.cpp Normal file
View File

@ -0,0 +1,351 @@
// Copyright 2011 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/config.hpp"
#include <atf-c++.hpp>
#include "engine/config.hpp"
#include "engine/exceptions.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/config/tree.ipp"
#include "utils/env.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace {
/// Creates a configuration file for testing purposes.
///
/// To ensure that the loaded file is the one created by this function, use
/// validate_mock_config().
///
/// \param name The name of the configuration file to create.
/// \param cookie The magic value to set in the configuration file, or NULL if a
/// broken configuration file is desired.
static void
create_mock_config(const char* name, const char* cookie)
{
if (cookie != NULL) {
atf::utils::create_file(
name,
F("syntax(2)\n"
"test_suites.suite.magic_value = '%s'\n") % cookie);
} else {
atf::utils::create_file(name, "syntax(200)\n");
}
}
/// Creates an invalid system configuration.
///
/// \param cookie The magic value to set in the configuration file, or NULL if a
/// broken configuration file is desired.
static void
mock_system_config(const char* cookie)
{
fs::mkdir(fs::path("system-dir"), 0755);
utils::setenv("KYUA_CONFDIR", (fs::current_path() / "system-dir").str());
create_mock_config("system-dir/kyua.conf", cookie);
}
/// Creates an invalid user configuration.
///
/// \param cookie The magic value to set in the configuration file, or NULL if a
/// broken configuration file is desired.
static void
mock_user_config(const char* cookie)
{
fs::mkdir(fs::path("user-dir"), 0755);
fs::mkdir(fs::path("user-dir/.kyua"), 0755);
utils::setenv("HOME", (fs::current_path() / "user-dir").str());
create_mock_config("user-dir/.kyua/kyua.conf", cookie);
}
/// Ensures that a loaded configuration was created with create_mock_config().
///
/// \param user_config The configuration to validate.
/// \param cookie The magic value to expect in the configuration file.
static void
validate_mock_config(const config::tree& user_config, const char* cookie)
{
const config::properties_map& properties = user_config.all_properties(
"test_suites.suite", true);
const config::properties_map::const_iterator iter =
properties.find("magic_value");
ATF_REQUIRE(iter != properties.end());
ATF_REQUIRE_EQ(cookie, (*iter).second);
}
/// Ensures that two configuration trees are equal.
///
/// \param exp_tree The expected configuration tree.
/// \param actual_tree The configuration tree being validated against exp_tree.
static void
require_eq(const config::tree& exp_tree, const config::tree& actual_tree)
{
ATF_REQUIRE(exp_tree.all_properties() == actual_tree.all_properties());
}
} // anonymous namespace
ATF_TEST_CASE_WITHOUT_HEAD(load_config__none);
ATF_TEST_CASE_BODY(load_config__none)
{
utils::setenv("KYUA_CONFDIR", "/the/system/does/not/exist");
utils::setenv("HOME", "/the/user/does/not/exist");
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
require_eq(engine::default_config(),
cli::load_config(mock_cmdline, true));
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__explicit__ok);
ATF_TEST_CASE_BODY(load_config__explicit__ok)
{
mock_system_config(NULL);
mock_user_config(NULL);
create_mock_config("test-file", "hello");
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back("test-file");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const config::tree user_config = cli::load_config(mock_cmdline, true);
validate_mock_config(user_config, "hello");
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__explicit__disable);
ATF_TEST_CASE_BODY(load_config__explicit__disable)
{
mock_system_config(NULL);
mock_user_config(NULL);
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back("none");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
require_eq(engine::default_config(),
cli::load_config(mock_cmdline, true));
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__explicit__fail);
ATF_TEST_CASE_BODY(load_config__explicit__fail)
{
mock_system_config("ok1");
mock_user_config("ok2");
create_mock_config("test-file", NULL);
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back("test-file");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_THROW_RE(engine::error, "200",
cli::load_config(mock_cmdline, true));
const config::tree config = cli::load_config(mock_cmdline, false);
require_eq(engine::default_config(), config);
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__user__ok);
ATF_TEST_CASE_BODY(load_config__user__ok)
{
mock_system_config(NULL);
mock_user_config("I am the user config");
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const config::tree user_config = cli::load_config(mock_cmdline, true);
validate_mock_config(user_config, "I am the user config");
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__user__fail);
ATF_TEST_CASE_BODY(load_config__user__fail)
{
mock_system_config("valid");
mock_user_config(NULL);
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_THROW_RE(engine::error, "200",
cli::load_config(mock_cmdline, true));
const config::tree config = cli::load_config(mock_cmdline, false);
require_eq(engine::default_config(), config);
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__user__bad_home);
ATF_TEST_CASE_BODY(load_config__user__bad_home)
{
mock_system_config("Fallback system config");
utils::setenv("HOME", "");
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const config::tree user_config = cli::load_config(mock_cmdline, true);
validate_mock_config(user_config, "Fallback system config");
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__system__ok);
ATF_TEST_CASE_BODY(load_config__system__ok)
{
mock_system_config("I am the system config");
utils::setenv("HOME", "/the/user/does/not/exist");
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const config::tree user_config = cli::load_config(mock_cmdline, true);
validate_mock_config(user_config, "I am the system config");
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__system__fail);
ATF_TEST_CASE_BODY(load_config__system__fail)
{
mock_system_config(NULL);
utils::setenv("HOME", "/the/user/does/not/exist");
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_THROW_RE(engine::error, "200",
cli::load_config(mock_cmdline, true));
const config::tree config = cli::load_config(mock_cmdline, false);
require_eq(engine::default_config(), config);
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__overrides__no);
ATF_TEST_CASE_BODY(load_config__overrides__no)
{
utils::setenv("KYUA_CONFDIR", fs::current_path().str());
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
options["variable"].push_back("architecture=1");
options["variable"].push_back("platform=2");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const config::tree user_config = cli::load_config(mock_cmdline, true);
ATF_REQUIRE_EQ("1",
user_config.lookup< config::string_node >("architecture"));
ATF_REQUIRE_EQ("2",
user_config.lookup< config::string_node >("platform"));
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__overrides__yes);
ATF_TEST_CASE_BODY(load_config__overrides__yes)
{
atf::utils::create_file(
"config",
"syntax(2)\n"
"architecture = 'do not see me'\n"
"platform = 'see me'\n");
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back("config");
options["variable"].push_back("architecture=overriden");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
const config::tree user_config = cli::load_config(mock_cmdline, true);
ATF_REQUIRE_EQ("overriden",
user_config.lookup< config::string_node >("architecture"));
ATF_REQUIRE_EQ("see me",
user_config.lookup< config::string_node >("platform"));
}
ATF_TEST_CASE_WITHOUT_HEAD(load_config__overrides__fail);
ATF_TEST_CASE_BODY(load_config__overrides__fail)
{
utils::setenv("KYUA_CONFDIR", fs::current_path().str());
std::map< std::string, std::vector< std::string > > options;
options["config"].push_back(cli::config_option.default_value());
options["variable"].push_back(".a=d");
const cmdline::parsed_cmdline mock_cmdline(options, cmdline::args_vector());
ATF_REQUIRE_THROW_RE(engine::error, "Empty component in key.*'\\.a'",
cli::load_config(mock_cmdline, true));
const config::tree config = cli::load_config(mock_cmdline, false);
require_eq(engine::default_config(), config);
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, load_config__none);
ATF_ADD_TEST_CASE(tcs, load_config__explicit__ok);
ATF_ADD_TEST_CASE(tcs, load_config__explicit__disable);
ATF_ADD_TEST_CASE(tcs, load_config__explicit__fail);
ATF_ADD_TEST_CASE(tcs, load_config__user__ok);
ATF_ADD_TEST_CASE(tcs, load_config__user__fail);
ATF_ADD_TEST_CASE(tcs, load_config__user__bad_home);
ATF_ADD_TEST_CASE(tcs, load_config__system__ok);
ATF_ADD_TEST_CASE(tcs, load_config__system__fail);
ATF_ADD_TEST_CASE(tcs, load_config__overrides__no);
ATF_ADD_TEST_CASE(tcs, load_config__overrides__yes);
ATF_ADD_TEST_CASE(tcs, load_config__overrides__fail);
}

356
cli/main.cpp Normal file
View File

@ -0,0 +1,356 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/main.hpp"
#if defined(HAVE_CONFIG_H)
# include "config.h"
#endif
extern "C" {
#include <signal.h>
#include <unistd.h>
}
#include <cstdlib>
#include <iostream>
#include <string>
#include <utility>
#include "cli/cmd_about.hpp"
#include "cli/cmd_config.hpp"
#include "cli/cmd_db_exec.hpp"
#include "cli/cmd_db_migrate.hpp"
#include "cli/cmd_debug.hpp"
#include "cli/cmd_help.hpp"
#include "cli/cmd_list.hpp"
#include "cli/cmd_report.hpp"
#include "cli/cmd_report_html.hpp"
#include "cli/cmd_report_junit.hpp"
#include "cli/cmd_test.hpp"
#include "cli/common.ipp"
#include "cli/config.hpp"
#include "engine/atf.hpp"
#include "engine/plain.hpp"
#include "engine/scheduler.hpp"
#include "engine/tap.hpp"
#include "store/exceptions.hpp"
#include "utils/cmdline/commands_map.ipp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/globals.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/cmdline/ui.hpp"
#include "utils/config/tree.ipp"
#include "utils/env.hpp"
#include "utils/format/macros.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
#include "utils/logging/macros.hpp"
#include "utils/logging/operations.hpp"
#include "utils/optional.ipp"
#include "utils/sanity.hpp"
#include "utils/signals/exceptions.hpp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace logging = utils::logging;
namespace signals = utils::signals;
namespace scheduler = engine::scheduler;
using utils::none;
using utils::optional;
namespace {
/// Registers all valid scheduler interfaces.
///
/// This is part of Kyua's setup but it is a bit strange to find it here. I am
/// not sure what a better location would be though, so for now this is good
/// enough.
static void
register_scheduler_interfaces(void)
{
scheduler::register_interface(
"atf", std::shared_ptr< scheduler::interface >(
new engine::atf_interface()));
scheduler::register_interface(
"plain", std::shared_ptr< scheduler::interface >(
new engine::plain_interface()));
scheduler::register_interface(
"tap", std::shared_ptr< scheduler::interface >(
new engine::tap_interface()));
}
/// Executes the given subcommand with proper usage_error reporting.
///
/// \param ui Object to interact with the I/O of the program.
/// \param command The subcommand to execute.
/// \param args The part of the command line passed to the subcommand. The
/// first item of this collection must match the command name.
/// \param user_config The runtime configuration to pass to the subcommand.
///
/// \return The exit code of the command. Typically 0 on success, some other
/// integer otherwise.
///
/// \throw cmdline::usage_error If the user input to the subcommand is invalid.
/// This error does not encode the command name within it, so this function
/// extends the message in the error to specify which subcommand was
/// affected.
/// \throw std::exception This propagates any uncaught exception. Such
/// exceptions are bugs, but we let them propagate so that the runtime will
/// abort and dump core.
static int
run_subcommand(cmdline::ui* ui, cli::cli_command* command,
const cmdline::args_vector& args,
const config::tree& user_config)
{
try {
PRE(command->name() == args[0]);
return command->main(ui, args, user_config);
} catch (const cmdline::usage_error& e) {
throw std::pair< std::string, cmdline::usage_error >(
command->name(), e);
}
}
/// Exception-safe version of main.
///
/// This function provides the real meat of the entry point of the program. It
/// is allowed to throw some known exceptions which are parsed by the caller.
/// Doing so keeps this function simpler and allow tests to actually validate
/// that the errors reported are accurate.
///
/// \return The exit code of the program. Should be EXIT_SUCCESS on success and
/// EXIT_FAILURE on failure. The caller extends this to additional integers for
/// errors reported through exceptions.
///
/// \param ui Object to interact with the I/O of the program.
/// \param argc The number of arguments passed on the command line.
/// \param argv NULL-terminated array containing the command line arguments.
/// \param mock_command An extra command provided for testing purposes; should
/// just be NULL other than for tests.
///
/// \throw cmdline::usage_error If the user ran the program with invalid
/// arguments.
/// \throw std::exception This propagates any uncaught exception. Such
/// exceptions are bugs, but we let them propagate so that the runtime will
/// abort and dump core.
static int
safe_main(cmdline::ui* ui, int argc, const char* const argv[],
cli::cli_command_ptr mock_command)
{
cmdline::options_vector options;
options.push_back(&cli::config_option);
options.push_back(&cli::variable_option);
const cmdline::string_option loglevel_option(
"loglevel", "Level of the messages to log", "level", "info");
options.push_back(&loglevel_option);
const cmdline::path_option logfile_option(
"logfile", "Path to the log file", "file",
cli::detail::default_log_name().c_str());
options.push_back(&logfile_option);
cmdline::commands_map< cli::cli_command > commands;
commands.insert(new cli::cmd_about());
commands.insert(new cli::cmd_config());
commands.insert(new cli::cmd_db_exec());
commands.insert(new cli::cmd_db_migrate());
commands.insert(new cli::cmd_help(&options, &commands));
commands.insert(new cli::cmd_debug(), "Workspace");
commands.insert(new cli::cmd_list(), "Workspace");
commands.insert(new cli::cmd_test(), "Workspace");
commands.insert(new cli::cmd_report(), "Reporting");
commands.insert(new cli::cmd_report_html(), "Reporting");
commands.insert(new cli::cmd_report_junit(), "Reporting");
if (mock_command.get() != NULL)
commands.insert(mock_command);
const cmdline::parsed_cmdline cmdline = cmdline::parse(argc, argv, options);
const fs::path logfile(cmdline.get_option< cmdline::path_option >(
"logfile"));
fs::mkdir_p(logfile.branch_path(), 0755);
LD(F("Log file is %s") % logfile);
utils::install_crash_handlers(logfile.str());
try {
logging::set_persistency(cmdline.get_option< cmdline::string_option >(
"loglevel"), logfile);
} catch (const std::range_error& e) {
throw cmdline::usage_error(e.what());
}
if (cmdline.arguments().empty())
throw cmdline::usage_error("No command provided");
const std::string cmdname = cmdline.arguments()[0];
const config::tree user_config = cli::load_config(cmdline,
cmdname != "help");
cli::cli_command* command = commands.find(cmdname);
if (command == NULL)
throw cmdline::usage_error(F("Unknown command '%s'") % cmdname);
register_scheduler_interfaces();
return run_subcommand(ui, command, cmdline.arguments(), user_config);
}
} // anonymous namespace
/// Gets the name of the default log file.
///
/// \return The path to the log file.
fs::path
cli::detail::default_log_name(void)
{
// Update doc/troubleshooting.texi if you change this algorithm.
const optional< std::string > home(utils::getenv("HOME"));
if (home) {
return logging::generate_log_name(fs::path(home.get()) / ".kyua" /
"logs", cmdline::progname());
} else {
const optional< std::string > tmpdir(utils::getenv("TMPDIR"));
if (tmpdir) {
return logging::generate_log_name(fs::path(tmpdir.get()),
cmdline::progname());
} else {
return logging::generate_log_name(fs::path("/tmp"),
cmdline::progname());
}
}
}
/// Testable entry point, with catch-all exception handlers.
///
/// This entry point does not perform any initialization of global state; it is
/// provided to allow unit-testing of the utility's entry point.
///
/// \param ui Object to interact with the I/O of the program.
/// \param argc The number of arguments passed on the command line.
/// \param argv NULL-terminated array containing the command line arguments.
/// \param mock_command An extra command provided for testing purposes; should
/// just be NULL other than for tests.
///
/// \return 0 on success, some other integer on error.
///
/// \throw std::exception This propagates any uncaught exception. Such
/// exceptions are bugs, but we let them propagate so that the runtime will
/// abort and dump core.
int
cli::main(cmdline::ui* ui, const int argc, const char* const* const argv,
cli_command_ptr mock_command)
{
try {
const int exit_code = safe_main(ui, argc, argv, mock_command);
// Codes above 1 are reserved to report conditions captured as
// exceptions below.
INV(exit_code == EXIT_SUCCESS || exit_code == EXIT_FAILURE);
return exit_code;
} catch (const signals::interrupted_error& e) {
cmdline::print_error(ui, F("%s.") % e.what());
// Re-deliver the interruption signal to self so that we terminate with
// the right status. At this point we should NOT have any custom signal
// handlers in place.
::kill(getpid(), e.signo());
LD("Interrupt signal re-delivery did not terminate program");
// If we reach this, something went wrong because we did not exit as
// intended. Return an internal error instead. (Would be nicer to
// abort in principle, but it wouldn't be a nice experience if it ever
// happened.)
return 2;
} catch (const std::pair< std::string, cmdline::usage_error >& e) {
const std::string message = F("Usage error for command %s: %s.") %
e.first % e.second.what();
LE(message);
ui->err(message);
ui->err(F("Type '%s help %s' for usage information.") %
cmdline::progname() % e.first);
return 3;
} catch (const cmdline::usage_error& e) {
const std::string message = F("Usage error: %s.") % e.what();
LE(message);
ui->err(message);
ui->err(F("Type '%s help' for usage information.") %
cmdline::progname());
return 3;
} catch (const store::old_schema_error& e) {
const std::string message = F("The database has schema version %s, "
"which is too old; please use db-migrate "
"to upgrade it.") % e.old_version();
cmdline::print_error(ui, message);
return 2;
} catch (const std::runtime_error& e) {
cmdline::print_error(ui, F("%s.") % e.what());
return 2;
}
}
/// Delegate for ::main().
///
/// This function is supposed to be called directly from the top-level ::main()
/// function. It takes care of initializing internal libraries and then calls
/// main(ui, argc, argv).
///
/// \pre This function can only be called once.
///
/// \throw std::exception This propagates any uncaught exception. Such
/// exceptions are bugs, but we let them propagate so that the runtime will
/// abort and dump core.
int
cli::main(const int argc, const char* const* const argv)
{
logging::set_inmemory();
LI(F("%s %s") % PACKAGE % VERSION);
std::string plain_args;
for (const char* const* arg = argv; *arg != NULL; arg++)
plain_args += F(" %s") % *arg;
LI(F("Command line:%s") % plain_args);
cmdline::init(argv[0]);
cmdline::ui ui;
const int exit_code = main(&ui, argc, argv);
LI(F("Clean exit with code %s") % exit_code);
return exit_code;
}

61
cli/main.hpp Normal file
View File

@ -0,0 +1,61 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
/// \file cli/main.hpp
/// Entry point for the program.
///
/// These entry points are separate from the top-level ::main() function to
/// allow unit-testing of the main code.
#if !defined(CLI_MAIN_HPP)
#define CLI_MAIN_HPP
#include "cli/common.hpp"
#include "utils/cmdline/ui_fwd.hpp"
#include "utils/fs/path_fwd.hpp"
namespace cli {
namespace detail {
utils::fs::path default_log_name(void);
} // namespace detail
int main(utils::cmdline::ui*, const int, const char* const* const,
cli_command_ptr = cli_command_ptr());
int main(const int, const char* const* const);
} // namespace cli
#endif // !defined(CLI_MAIN_HPP)

489
cli/main_test.cpp Normal file
View File

@ -0,0 +1,489 @@
// Copyright 2010 The Kyua Authors.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "cli/main.hpp"
extern "C" {
#include <signal.h>
}
#include <cstdlib>
#include <atf-c++.hpp>
#include "utils/cmdline/base_command.ipp"
#include "utils/cmdline/exceptions.hpp"
#include "utils/cmdline/globals.hpp"
#include "utils/cmdline/options.hpp"
#include "utils/cmdline/parser.hpp"
#include "utils/cmdline/ui_mock.hpp"
#include "utils/datetime.hpp"
#include "utils/defs.hpp"
#include "utils/env.hpp"
#include "utils/fs/operations.hpp"
#include "utils/fs/path.hpp"
#include "utils/logging/macros.hpp"
#include "utils/logging/operations.hpp"
#include "utils/process/child.ipp"
#include "utils/process/status.hpp"
#include "utils/test_utils.ipp"
namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace datetime = utils::datetime;
namespace fs = utils::fs;
namespace logging = utils::logging;
namespace process = utils::process;
namespace {
/// Fake command implementation that crashes during its execution.
class cmd_mock_crash : public cli::cli_command {
public:
/// Constructs a new mock command.
///
/// All command parameters are set to irrelevant values.
cmd_mock_crash(void) :
cli::cli_command("mock_error", "", 0, 0, "Mock command that crashes")
{
}
/// Runs the mock command.
///
/// \return Nothing because this function always aborts.
int
run(cmdline::ui* /* ui */,
const cmdline::parsed_cmdline& /* cmdline */,
const config::tree& /* user_config */)
{
utils::abort_without_coredump();
}
};
/// Fake command implementation that throws an exception during its execution.
class cmd_mock_error : public cli::cli_command {
/// Whether the command raises an exception captured by the parent or not.
///
/// If this is true, the command will raise a std::runtime_error exception
/// or a subclass of it. The main program is in charge of capturing these
/// and reporting them appropriately. If false, this raises another
/// exception that does not inherit from std::runtime_error.
bool _unhandled;
public:
/// Constructs a new mock command.
///
/// \param unhandled If true, make run raise an exception not catched by the
/// main program.
cmd_mock_error(const bool unhandled) :
cli::cli_command("mock_error", "", 0, 0,
"Mock command that raises an error"),
_unhandled(unhandled)
{
}
/// Runs the mock command.
///
/// \return Nothing because this function always aborts.
///
/// \throw std::logic_error If _unhandled is true.
/// \throw std::runtime_error If _unhandled is false.
int
run(cmdline::ui* /* ui */,
const cmdline::parsed_cmdline& /* cmdline */,
const config::tree& /* user_config */)
{
if (_unhandled)
throw std::logic_error("This is unhandled");
else
throw std::runtime_error("Runtime error");
}
};
/// Fake command implementation that prints messages during its execution.
class cmd_mock_write : public cli::cli_command {
public:
/// Constructs a new mock command.
///
/// All command parameters are set to irrelevant values.
cmd_mock_write(void) : cli::cli_command(
"mock_write", "", 0, 0, "Mock command that prints output")
{
}
/// Runs the mock command.
///
/// \param ui Object to interact with the I/O of the program.
///
/// \return Nothing because this function always aborts.
int
run(cmdline::ui* ui,
const cmdline::parsed_cmdline& /* cmdline */,
const config::tree& /* user_config */)
{
ui->out("stdout message from subcommand");
ui->err("stderr message from subcommand");
return EXIT_FAILURE;
}
};
} // anonymous namespace
ATF_TEST_CASE_WITHOUT_HEAD(detail__default_log_name__home);
ATF_TEST_CASE_BODY(detail__default_log_name__home)
{
datetime::set_mock_now(2011, 2, 21, 21, 10, 30, 0);
cmdline::init("progname1");
utils::setenv("HOME", "/home//fake");
utils::setenv("TMPDIR", "/do/not/use/this");
ATF_REQUIRE_EQ(
fs::path("/home/fake/.kyua/logs/progname1.20110221-211030.log"),
cli::detail::default_log_name());
}
ATF_TEST_CASE_WITHOUT_HEAD(detail__default_log_name__tmpdir);
ATF_TEST_CASE_BODY(detail__default_log_name__tmpdir)
{
datetime::set_mock_now(2011, 2, 21, 21, 10, 50, 987);
cmdline::init("progname2");
utils::unsetenv("HOME");
utils::setenv("TMPDIR", "/a/b//c");
ATF_REQUIRE_EQ(fs::path("/a/b/c/progname2.20110221-211050.log"),
cli::detail::default_log_name());
}
ATF_TEST_CASE_WITHOUT_HEAD(detail__default_log_name__hardcoded);
ATF_TEST_CASE_BODY(detail__default_log_name__hardcoded)
{
datetime::set_mock_now(2011, 2, 21, 21, 15, 00, 123456);
cmdline::init("progname3");
utils::unsetenv("HOME");
utils::unsetenv("TMPDIR");
ATF_REQUIRE_EQ(fs::path("/tmp/progname3.20110221-211500.log"),
cli::detail::default_log_name());
}
ATF_TEST_CASE_WITHOUT_HEAD(main__no_args);
ATF_TEST_CASE_BODY(main__no_args)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 1;
const char* const argv[] = {"progname", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(atf::utils::grep_collection("Usage error: No command provided",
ui.err_log()));
ATF_REQUIRE(atf::utils::grep_collection("Type.*progname help",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__unknown_command);
ATF_TEST_CASE_BODY(main__unknown_command)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 2;
const char* const argv[] = {"progname", "foo", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(atf::utils::grep_collection("Usage error: Unknown command.*foo",
ui.err_log()));
ATF_REQUIRE(atf::utils::grep_collection("Type.*progname help",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__logfile__default);
ATF_TEST_CASE_BODY(main__logfile__default)
{
logging::set_inmemory();
datetime::set_mock_now(2011, 2, 21, 21, 30, 00, 0);
cmdline::init("progname");
const int argc = 1;
const char* const argv[] = {"progname", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE(!fs::exists(fs::path(
".kyua/logs/progname.20110221-213000.log")));
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(fs::exists(fs::path(
".kyua/logs/progname.20110221-213000.log")));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__logfile__override);
ATF_TEST_CASE_BODY(main__logfile__override)
{
logging::set_inmemory();
datetime::set_mock_now(2011, 2, 21, 21, 30, 00, 321);
cmdline::init("progname");
const int argc = 2;
const char* const argv[] = {"progname", "--logfile=test.log", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE(!fs::exists(fs::path("test.log")));
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(!fs::exists(fs::path(
".kyua/logs/progname.20110221-213000.log")));
ATF_REQUIRE(fs::exists(fs::path("test.log")));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__loglevel__default);
ATF_TEST_CASE_BODY(main__loglevel__default)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 2;
const char* const argv[] = {"progname", "--logfile=test.log", NULL};
LD("Mock debug message");
LE("Mock error message");
LI("Mock info message");
LW("Mock warning message");
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(!atf::utils::grep_file("Mock debug message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock error message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock info message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock warning message", "test.log"));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__loglevel__higher);
ATF_TEST_CASE_BODY(main__loglevel__higher)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 3;
const char* const argv[] = {"progname", "--logfile=test.log",
"--loglevel=debug", NULL};
LD("Mock debug message");
LE("Mock error message");
LI("Mock info message");
LW("Mock warning message");
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(atf::utils::grep_file("Mock debug message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock error message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock info message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock warning message", "test.log"));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__loglevel__lower);
ATF_TEST_CASE_BODY(main__loglevel__lower)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 3;
const char* const argv[] = {"progname", "--logfile=test.log",
"--loglevel=warning", NULL};
LD("Mock debug message");
LE("Mock error message");
LI("Mock info message");
LW("Mock warning message");
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(!atf::utils::grep_file("Mock debug message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock error message", "test.log"));
ATF_REQUIRE(!atf::utils::grep_file("Mock info message", "test.log"));
ATF_REQUIRE(atf::utils::grep_file("Mock warning message", "test.log"));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__loglevel__error);
ATF_TEST_CASE_BODY(main__loglevel__error)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 3;
const char* const argv[] = {"progname", "--logfile=test.log",
"--loglevel=i-am-invalid", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(3, cli::main(&ui, argc, argv));
ATF_REQUIRE(atf::utils::grep_collection("Usage error.*i-am-invalid",
ui.err_log()));
ATF_REQUIRE(!fs::exists(fs::path("test.log")));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__subcommand__ok);
ATF_TEST_CASE_BODY(main__subcommand__ok)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 2;
const char* const argv[] = {"progname", "mock_write", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(EXIT_FAILURE,
cli::main(&ui, argc, argv,
cli::cli_command_ptr(new cmd_mock_write())));
ATF_REQUIRE_EQ(1, ui.out_log().size());
ATF_REQUIRE_EQ("stdout message from subcommand", ui.out_log()[0]);
ATF_REQUIRE_EQ(1, ui.err_log().size());
ATF_REQUIRE_EQ("stderr message from subcommand", ui.err_log()[0]);
}
ATF_TEST_CASE_WITHOUT_HEAD(main__subcommand__invalid_args);
ATF_TEST_CASE_BODY(main__subcommand__invalid_args)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 3;
const char* const argv[] = {"progname", "mock_write", "bar", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(3,
cli::main(&ui, argc, argv,
cli::cli_command_ptr(new cmd_mock_write())));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(atf::utils::grep_collection(
"Usage error for command mock_write: Too many arguments.",
ui.err_log()));
ATF_REQUIRE(atf::utils::grep_collection("Type.*progname help",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__subcommand__runtime_error);
ATF_TEST_CASE_BODY(main__subcommand__runtime_error)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 2;
const char* const argv[] = {"progname", "mock_error", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE_EQ(2, cli::main(&ui, argc, argv,
cli::cli_command_ptr(new cmd_mock_error(false))));
ATF_REQUIRE(ui.out_log().empty());
ATF_REQUIRE(atf::utils::grep_collection("progname: E: Runtime error.",
ui.err_log()));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__subcommand__unhandled_exception);
ATF_TEST_CASE_BODY(main__subcommand__unhandled_exception)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 2;
const char* const argv[] = {"progname", "mock_error", NULL};
cmdline::ui_mock ui;
ATF_REQUIRE_THROW(std::logic_error, cli::main(&ui, argc, argv,
cli::cli_command_ptr(new cmd_mock_error(true))));
}
static void
do_subcommand_crash(void)
{
logging::set_inmemory();
cmdline::init("progname");
const int argc = 2;
const char* const argv[] = {"progname", "mock_error", NULL};
cmdline::ui_mock ui;
cli::main(&ui, argc, argv,
cli::cli_command_ptr(new cmd_mock_crash()));
}
ATF_TEST_CASE_WITHOUT_HEAD(main__subcommand__crash);
ATF_TEST_CASE_BODY(main__subcommand__crash)
{
const process::status status = process::child::fork_files(
do_subcommand_crash, fs::path("stdout.txt"),
fs::path("stderr.txt"))->wait();
ATF_REQUIRE(status.signaled());
ATF_REQUIRE_EQ(SIGABRT, status.termsig());
ATF_REQUIRE(atf::utils::grep_file("Fatal signal", "stderr.txt"));
}
ATF_INIT_TEST_CASES(tcs)
{
ATF_ADD_TEST_CASE(tcs, detail__default_log_name__home);
ATF_ADD_TEST_CASE(tcs, detail__default_log_name__tmpdir);
ATF_ADD_TEST_CASE(tcs, detail__default_log_name__hardcoded);
ATF_ADD_TEST_CASE(tcs, main__no_args);
ATF_ADD_TEST_CASE(tcs, main__unknown_command);
ATF_ADD_TEST_CASE(tcs, main__logfile__default);
ATF_ADD_TEST_CASE(tcs, main__logfile__override);
ATF_ADD_TEST_CASE(tcs, main__loglevel__default);
ATF_ADD_TEST_CASE(tcs, main__loglevel__higher);
ATF_ADD_TEST_CASE(tcs, main__loglevel__lower);
ATF_ADD_TEST_CASE(tcs, main__loglevel__error);
ATF_ADD_TEST_CASE(tcs, main__subcommand__ok);
ATF_ADD_TEST_CASE(tcs, main__subcommand__invalid_args);
ATF_ADD_TEST_CASE(tcs, main__subcommand__runtime_error);
ATF_ADD_TEST_CASE(tcs, main__subcommand__unhandled_exception);
ATF_ADD_TEST_CASE(tcs, main__subcommand__crash);
}

173
configure.ac Normal file
View File

@ -0,0 +1,173 @@
dnl Copyright 2010 The Kyua Authors.
dnl All rights reserved.
dnl
dnl Redistribution and use in source and binary forms, with or without
dnl modification, are permitted provided that the following conditions are
dnl met:
dnl
dnl * Redistributions of source code must retain the above copyright
dnl notice, this list of conditions and the following disclaimer.
dnl * Redistributions in binary form must reproduce the above copyright
dnl notice, this list of conditions and the following disclaimer in the
dnl documentation and/or other materials provided with the distribution.
dnl * Neither the name of Google Inc. nor the names of its contributors
dnl may be used to endorse or promote products derived from this software
dnl without specific prior written permission.
dnl
dnl THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
dnl "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
dnl LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
dnl A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
dnl OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
dnl SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
dnl LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
dnl DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
dnl THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
dnl (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
dnl OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
AC_INIT([Kyua], [0.14], [kyua-discuss@googlegroups.com], [kyua],
[https://github.com/jmmv/kyua/])
AC_PREREQ([2.65])
AC_COPYRIGHT([Copyright 2010 The Kyua Authors.])
AC_CONFIG_AUX_DIR([admin])
AC_CONFIG_FILES([Doxyfile Makefile utils/defs.hpp])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_SRCDIR([main.cpp])
AC_CONFIG_TESTDIR([bootstrap])
AM_INIT_AUTOMAKE([1.9 foreign subdir-objects -Wall])
AC_LANG([C++])
AC_PROG_CXX
AX_CXX_COMPILE_STDCXX([11], [noext], [mandatory])
m4_ifdef([AM_PROG_AR], [AM_PROG_AR])
KYUA_DEVELOPER_MODE([C++])
KYUA_ATTRIBUTE_NORETURN
KYUA_ATTRIBUTE_PURE
KYUA_ATTRIBUTE_UNUSED
KYUA_FS_MODULE
KYUA_GETOPT
KYUA_LAST_SIGNO
KYUA_MEMORY
AC_CHECK_FUNCS([putenv setenv unsetenv])
AC_CHECK_HEADERS([termios.h])
AC_PROG_RANLIB
m4_ifndef([PKG_CHECK_MODULES],
[m4_fatal([Cannot find pkg.m4; see the INSTALL document for help])])
m4_ifndef([ATF_CHECK_CXX],
[m4_fatal([Cannot find atf-c++.m4; see the INSTALL document for help])])
ATF_CHECK_CXX([>= 0.17])
m4_ifndef([ATF_CHECK_SH],
[m4_fatal([Cannot find atf-sh.m4; see the INSTALL document for help])])
ATF_CHECK_SH([>= 0.15])
m4_ifndef([ATF_ARG_WITH],
[m4_fatal([Cannot find atf-common.m4; see the INSTALL document for help])])
ATF_ARG_WITH
PKG_CHECK_MODULES([LUTOK], [lutok >= 0.4],
[],
AC_MSG_ERROR([lutok (0.4 or newer) is required]))
PKG_CHECK_MODULES([SQLITE3], [sqlite3 >= 3.6.22],
[],
AC_MSG_ERROR([sqlite3 (3.6.22 or newer) is required]))
KYUA_DOXYGEN
AC_PATH_PROG([GDB], [gdb])
test -n "${GDB}" || GDB=gdb
AC_PATH_PROG([GIT], [git])
KYUA_UNAME_ARCHITECTURE
KYUA_UNAME_PLATFORM
AC_ARG_VAR([KYUA_CONFSUBDIR],
[Subdirectory of sysconfdir under which to look for files])
if test x"${KYUA_CONFSUBDIR-unset}" = x"unset"; then
KYUA_CONFSUBDIR=kyua
else
case ${KYUA_CONFSUBDIR} in
/*)
AC_MSG_ERROR([KYUA_CONFSUBDIR must hold a relative path])
;;
*)
;;
esac
fi
if test x"${KYUA_CONFSUBDIR}" = x""; then
AC_SUBST(kyua_confdir, \${sysconfdir})
else
AC_SUBST(kyua_confdir, \${sysconfdir}/${KYUA_CONFSUBDIR})
fi
dnl Allow the caller of 'make check', 'make installcheck' and 'make distcheck'
dnl on the Kyua source tree to override the configuration file passed to our
dnl own test runs. This is for the development of Kyua only and the value of
dnl this setting has no effect on the built product in any way. If we go
dnl through great extents in validating the value of this setting, it is to
dnl minimize the chance of false test run negatives later on.
AC_ARG_VAR([KYUA_CONFIG_FILE_FOR_CHECK],
[kyua.conf file to use at 'make (|dist|install)check' time])
case "${KYUA_CONFIG_FILE_FOR_CHECK-none}" in
none)
KYUA_CONFIG_FILE_FOR_CHECK=none
;;
/*)
if test -f "${KYUA_CONFIG_FILE_FOR_CHECK}"; then
: # All good!
else
AC_MSG_ERROR([KYUA_CONFIG_FILE_FOR_CHECK file does not exist])
fi
;;
*)
AC_MSG_ERROR([KYUA_CONFIG_FILE_FOR_CHECK must hold an absolute path])
;;
esac
AC_ARG_VAR([KYUA_TMPDIR],
[Path to the directory in which to place work directories])
case "${KYUA_TMPDIR:-unset}" in
unset)
KYUA_TMPDIR=/tmp
;;
/*)
;;
*)
AC_MSG_ERROR([KYUA_TMPDIR must be an absolute path])
;;
esac
AC_SUBST(examplesdir, \${pkgdatadir}/examples)
AC_SUBST(luadir, \${pkgdatadir}/lua)
AC_SUBST(miscdir, \${pkgdatadir}/misc)
AC_SUBST(pkgtestsdir, \${testsdir}/kyua)
AC_SUBST(storedir, \${pkgdatadir}/store)
AC_SUBST(testsdir, \${exec_prefix}/tests)
dnl BSD make(1) doesn't deal with targets specified as './foo' well: they
dnl need to be specified as 'foo'. The following hack is to workaround this
dnl issue.
if test "${srcdir}" = .; then
target_srcdir=
else
target_srcdir="${srcdir}/"
fi
AM_CONDITIONAL(TARGET_SRCDIR_EMPTY, [test -z "${target_srcdir}"])
AC_SUBST([target_srcdir])
AC_OUTPUT

14
doc/.gitignore vendored Normal file
View File

@ -0,0 +1,14 @@
kyua-about.1
kyua-config.1
kyua-db-exec.1
kyua-db-migrate.1
kyua-debug.1
kyua-help.1
kyua-list.1
kyua-report-html.1
kyua-report-junit.1
kyua-report.1
kyua-test.1
kyua.1
kyua.conf.5
kyuafile.5

5
doc/Kyuafile Normal file
View File

@ -0,0 +1,5 @@
syntax(2)
test_suite("kyua")
atf_test_program{name="manbuild_test"}

152
doc/Makefile.am.inc Normal file
View File

@ -0,0 +1,152 @@
# Copyright 2011 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
BUILD_MANPAGE = \
$(MKDIR_P) doc; \
$(SHELL) $(srcdir)/doc/manbuild.sh \
-v "CONFDIR=$(kyua_confdir)" \
-v "DOCDIR=$(docdir)" \
-v "EGDIR=$(examplesdir)" \
-v "MISCDIR=$(miscdir)" \
-v "PACKAGE=$(PACKAGE_TARNAME)" \
-v "STOREDIR=$(storedir)" \
-v "TESTSDIR=$(testsdir)" \
-v "VERSION=$(PACKAGE_VERSION)" \
"$(srcdir)/doc/$${name}.in" "doc/$${name}"
DIST_MAN_DEPS = doc/manbuild.sh \
doc/build-root.mdoc \
doc/results-file-flag-read.mdoc \
doc/results-file-flag-write.mdoc \
doc/results-files.mdoc \
doc/results-files-report-example.mdoc \
doc/test-filters.mdoc \
doc/test-isolation.mdoc
MAN_DEPS = $(DIST_MAN_DEPS) Makefile
EXTRA_DIST += $(DIST_MAN_DEPS)
man_MANS = doc/kyua-about.1
CLEANFILES += doc/kyua-about.1
EXTRA_DIST += doc/kyua-about.1.in
doc/kyua-about.1: $(srcdir)/doc/kyua-about.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-about.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-config.1
CLEANFILES += doc/kyua-config.1
EXTRA_DIST += doc/kyua-config.1.in
doc/kyua-config.1: $(srcdir)/doc/kyua-config.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-config.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-db-exec.1
CLEANFILES += doc/kyua-db-exec.1
EXTRA_DIST += doc/kyua-db-exec.1.in
doc/kyua-db-exec.1: $(srcdir)/doc/kyua-db-exec.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-db-exec.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-db-migrate.1
CLEANFILES += doc/kyua-db-migrate.1
EXTRA_DIST += doc/kyua-db-migrate.1.in
doc/kyua-db-migrate.1: $(srcdir)/doc/kyua-db-migrate.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-db-migrate.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-debug.1
CLEANFILES += doc/kyua-debug.1
EXTRA_DIST += doc/kyua-debug.1.in
doc/kyua-debug.1: $(srcdir)/doc/kyua-debug.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-debug.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-help.1
CLEANFILES += doc/kyua-help.1
EXTRA_DIST += doc/kyua-help.1.in
doc/kyua-help.1: $(srcdir)/doc/kyua-help.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-help.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-list.1
CLEANFILES += doc/kyua-list.1
EXTRA_DIST += doc/kyua-list.1.in
doc/kyua-list.1: $(srcdir)/doc/kyua-list.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-list.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-report-html.1
CLEANFILES += doc/kyua-report-html.1
EXTRA_DIST += doc/kyua-report-html.1.in
doc/kyua-report-html.1: $(srcdir)/doc/kyua-report-html.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-report-html.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-report-junit.1
CLEANFILES += doc/kyua-report-junit.1
EXTRA_DIST += doc/kyua-report-junit.1.in
doc/kyua-report-junit.1: $(srcdir)/doc/kyua-report-junit.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-report-junit.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-report.1
CLEANFILES += doc/kyua-report.1
EXTRA_DIST += doc/kyua-report.1.in
doc/kyua-report.1: $(srcdir)/doc/kyua-report.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-report.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua-test.1
CLEANFILES += doc/kyua-test.1
EXTRA_DIST += doc/kyua-test.1.in
doc/kyua-test.1: $(srcdir)/doc/kyua-test.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua-test.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua.1
CLEANFILES += doc/kyua.1
EXTRA_DIST += doc/kyua.1.in
doc/kyua.1: $(srcdir)/doc/kyua.1.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua.1; $(BUILD_MANPAGE)
man_MANS += doc/kyua.conf.5
CLEANFILES += doc/kyua.conf.5
EXTRA_DIST += doc/kyua.conf.5.in
doc/kyua.conf.5: $(srcdir)/doc/kyua.conf.5.in $(MAN_DEPS)
$(AM_V_GEN)name=kyua.conf.5; $(BUILD_MANPAGE)
man_MANS += doc/kyuafile.5
CLEANFILES += doc/kyuafile.5
EXTRA_DIST += doc/kyuafile.5.in
doc/kyuafile.5: $(srcdir)/doc/kyuafile.5.in $(MAN_DEPS)
$(AM_V_GEN)name=kyuafile.5; $(BUILD_MANPAGE)
if WITH_ATF
EXTRA_DIST += doc/Kyuafile
noinst_SCRIPTS += doc/manbuild_test
CLEANFILES += doc/manbuild_test
EXTRA_DIST += doc/manbuild_test.sh
doc/manbuild_test: $(srcdir)/doc/manbuild_test.sh Makefile
$(AM_V_GEN)$(MKDIR_P) doc; \
echo "#! $(ATF_SH)" >doc/manbuild_test.tmp; \
echo "# AUTOMATICALLY GENERATED FROM Makefile" \
>>doc/manbuild_test.tmp; \
sed -e 's,__MANBUILD__,$(abs_srcdir)/doc/manbuild.sh,g' \
<$(srcdir)/doc/manbuild_test.sh >>doc/manbuild_test.tmp; \
mv doc/manbuild_test.tmp doc/manbuild_test; \
chmod +x doc/manbuild_test
endif

104
doc/build-root.mdoc Normal file
View File

@ -0,0 +1,104 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Em Build directories
(or object directories, target directories, product directories, etc.) is
the concept that allows a developer to keep the source tree clean from
build products by asking the build system to place such build products
under a separate subtree.
.Pp
Most build systems today support build directories.
For example, the GNU Automake/Autoconf build system exposes such concept when
invoked as follows:
.Bd -literal -offset indent
$ cd my-project-1.0
$ mkdir build
$ cd build
$ ../configure
$ make
.Ed
.Pp
Under such invocation, all the results of the build are left in the
.Pa my-project-1.0/build/
subdirectory while maintaining the contents of
.Pa my-project-1.0/
intact.
.Pp
Because build directories are an integral part of most build systems, and
because they are a tool that developers use frequently,
.Nm
supports build directories too.
This manifests in the form of
.Nm
being able to run tests from build directories while reading the (often
immutable) test suite definition from the source tree.
.Pp
One important property of build directories is that they follow (or need to
follow) the exact same layout as the source tree.
For example, consider the following directory listings:
.Bd -literal -offset indent
src/Kyuafile
src/bin/ls/
src/bin/ls/Kyuafile
src/bin/ls/ls.c
src/bin/ls/ls_test.c
src/sbin/su/
src/sbin/su/Kyuafile
src/sbin/su/su.c
src/sbin/su/su_test.c
obj/bin/ls/
obj/bin/ls/ls*
obj/bin/ls/ls_test*
obj/sbin/su/
obj/sbin/su/su*
obj/sbin/su/su_test*
.Ed
.Pp
Note how the directory layout within
.Pa src/
matches that of
.Pa obj/ .
The
.Pa src/
directory contains only source files and the definition of the test suite
(the Kyuafiles), while the
.Pa obj/
directory contains only the binaries generated during a build.
.Pp
All commands that deal with the workspace support the
.Fl -build-root Ar path
option.
When this option is provided, the directory specified by the
option is considered to be the root of the build directory.
For example, considering our previous fake tree layout, we could invoke
.Nm
as any of the following:
.Bd -literal -offset indent
$ kyua __COMMAND__ --kyuafile=src/Kyuafile --build-root=obj
$ cd src && kyua __COMMAND__ --build-root=../obj
.Ed

95
doc/kyua-about.1.in Normal file
View File

@ -0,0 +1,95 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd May 20, 2015
.Dt KYUA-ABOUT 1
.Os
.Sh NAME
.Nm "kyua about"
.Nd Shows detailed authors, license, and version information
.Sh SYNOPSIS
.Nm
.Op Ar authors | license | version
.Sh DESCRIPTION
The
.Sq about
command provides generic information about the
.Xr kyua 1
tool.
In the default synopsis form (no arguments), the information printed
includes:
.Bl -enum
.It
The name of the package, which is
.Sq __PACKAGE__ .
.It
The version number, which is
.Sq __VERSION__ .
.It
License information.
.It
Authors information.
.It
A link to the project web site.
.El
.Pp
You can customize the information printed by this command by specifying
the desired topic as the single argument to the command.
This can be one of:
.Bl -tag -width authorsXX
.It Ar authors
Displays the list of authors and contributors only.
.It Ar license
Displays the license information and the list of copyrights.
.It Ar version
Displays the package name and the version number in a format that is
compatible with the output of GNU tools that support a
.Fl -version
flag.
Use this whenever you have to query the version number of the package.
.El
.Sh FILES
The following files are read by the
.Nm
command:
.Bl -tag -width XX
.It Pa __DOCDIR__/AUTHORS
List of authors (aka copyright holders).
.It Pa __DOCDIR__/CONTRIBUTORS
List of contributors (aka individuals that have contributed to the project).
.It Pa __DOCDIR__/LICENSE
License information.
.El
.Sh EXIT STATUS
The
.Nm
command always returns 0.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1

59
doc/kyua-config.1.in Normal file
View File

@ -0,0 +1,59 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd September 9, 2012
.Dt KYUA-CONFIG 1
.Os
.Sh NAME
.Nm "kyua config"
.Nd Inspects the values of the loaded configuration
.Sh SYNOPSIS
.Nm
.Op Ar variable1 .. variableN
.Sh DESCRIPTION
The
.Nm
command provides a way to list all defined configuration variables and
their current values.
.Pp
This command is intended to help you in resolving the values of the
configuration variables without having to scan over configuration files.
.Pp
In the default synopsis form (no arguments), the command prints all
configuration variables.
If any arguments are provided, the command will only print the
requested variables.
.Sh EXIT STATUS
The
.Nm
command returns 0 on success or 1 if any of the specified configuration
variables does not exist.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1

80
doc/kyua-db-exec.1.in Normal file
View File

@ -0,0 +1,80 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-DB-EXEC 1
.Os
.Sh NAME
.Nm "kyua db-exec"
.Nd Executes a SQL statement in a results file
.Sh SYNOPSIS
.Nm
.Op Fl -no-headers
.Op Fl -results-file Ar file
.Ar statement
.Sh DESCRIPTION
The
.Nm
command provides a way to execute an arbitrary SQL statement within the
database.
This command is mostly intended to aid in debugging, but can also be used to
extract information from the database when the current interfaces do not
provide the desired functionality.
.Pp
The input database must exist.
It makes no sense to use
.Nm
on a nonexistent or empty database.
.Pp
The
.Nm
command takes one or more arguments, all of which are concatenated to form
a single SQL statement.
Once the statement is executed,
.Nm
prints the resulting table on the screen, if any.
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -no-headers
Avoids printing the headers of the table in the output of the command.
.It Fl -results-file Ar path , Fl s Ar path
__include__ results-file-flag-read.mdoc
.El
.Ss Results files
__include__ results-files.mdoc
.Sh EXIT STATUS
The
.Nm
command returns 0 on success or 1 if the SQL statement is invalid or fails
to run.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyua-test 1

63
doc/kyua-db-migrate.1.in Normal file
View File

@ -0,0 +1,63 @@
.\" Copyright 2013 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-DB-MIGRATE 1
.Os
.Sh NAME
.Nm "kyua db-migrate"
.Nd Upgrades the schema of an existing results file
.Sh SYNOPSIS
.Nm
.Op Fl -results-file Ar file
.Sh DESCRIPTION
The
.Nm
command migrates the schema of an existing database to the latest
version implemented in
.Xr kyua 1 .
.Pp
This operation is not reversible.
However, a backup of the database is created in the same directory where the
database lives.
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -results-file Ar path , Fl s Ar path
__include__ results-file-flag-read.mdoc
.El
.Ss Results files
__include__ results-files.mdoc
.Sh EXIT STATUS
The
.Nm
command returns 0 on success or 1 if the migration fails.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1

145
doc/kyua-debug.1.in Normal file
View File

@ -0,0 +1,145 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-DEBUG 1
.Os
.Sh NAME
.Nm "kyua debug"
.Nd Executes a single test case with facilities for debugging
.Sh SYNOPSIS
.Nm
.Op Fl -build-root Ar path
.Op Fl -kyuafile Ar file
.Op Fl -stdout Ar path
.Op Fl -stderr Ar path
.Ar test_case
.Sh DESCRIPTION
The
.Nm
command provides a mechanism to execute a single test case bypassing some
of the Kyua infrastructure and allowing the user to poke into the execution
behavior of the test.
.Pp
The test case to run is selected by providing a test filter, described below in
.Sx Test filters ,
that matches a single test case.
The test case is executed and its result is printed as the last line of the
output of the tool.
.Pp
The test executed by
.Nm
is run under a controlled environment as described in
.Sx Test isolation .
.Pp
At the moment, the
.Nm
command allows the following aspects of a test case execution to be
tweaked:
.Bl -bullet
.It
Redirection of the test case's stdout and stderr to the console (the
default) or to arbitrary files.
See the
.Fl -stdout
and
.Fl -stderr
options below.
.El
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -build-root Ar path
Specifies the build root in which to find the test programs referenced
by the Kyuafile, if different from the Kyuafile's directory.
See
.Sx Build directories
below for more information.
.It Fl -kyuafile Ar file , Fl k Ar file
Specifies the Kyuafile to process.
Defaults to
.Pa Kyuafile
file in the current directory.
.It Fl -stderr Ar path
Specifies the file to which to send the standard error of the test
program's body.
The default is
.Pa /dev/stderr ,
which is a special character device that redirects the output to
standard error on the console.
.It Fl -stdout Ar path
Specifies the file to which to send the standard output of the test
program's body.
The default is
.Pa /dev/stdout ,
which is a special character device that redirects the output to
standard output on the console.
.El
.Pp
For example, consider the following Kyua session:
.Bd -literal -offset indent
$ kyua test
kernel/fs:mkdir -> passed
kernel/fs:rmdir -> failed: Invalid argument
1/2 passed (1 failed)
.Ed
.Pp
At this point, we do not have a lot of information regarding the
failure of the
.Sq kernel/fs:rmdir
test.
We can run this test through the
.Nm
command to inspect its output a bit closer, hoping that the test case is
kind enough to log its progress:
.Bd -literal -offset indent
$ kyua debug kernel/fs:rmdir
Trying rmdir('foo')
Trying rmdir(NULL)
kernel/fs:rmdir -> failed: Invalid argument
.Ed
.Pp
Luckily, the offending test case was printing status lines as it
progressed, so we could see the last attempted call and we can know match
the failure message to the problem.
.Ss Build directories
__include__ build-root.mdoc COMMAND=debug
.Ss Test filters
__include__ test-filters.mdoc
.Ss Test isolation
__include__ test-isolation.mdoc
.Sh EXIT STATUS
The
.Nm
command returns 0 if the test case passes or 1 if the test case fails.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyuafile 5

64
doc/kyua-help.1.in Normal file
View File

@ -0,0 +1,64 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd September 9, 2012
.Dt KYUA-HELP 1
.Os
.Sh NAME
.Nm "kyua help"
.Nd Shows usage information
.Sh SYNOPSIS
.Nm
.Op Ar command
.Sh DESCRIPTION
The
.Nm
command provides interactive help on all supported commands and options.
If, for some reason, you happen to spot a discrepancy in the output of this
command and this document, the command is the authoritative source of
information.
.Pp
If no arguments are provided, the command prints the list of common options
and the list of supported subcommands.
.Pp
If the
.Ar command
argument is provided to, this single argument is the name of a valid
subcommand.
In that case,
.Nm
prints a textual description of the command, the list of common options and
the list of subcommand-specific options.
.Sh EXIT STATUS
The
.Nm
command always returns 0.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1

90
doc/kyua-list.1.in Normal file
View File

@ -0,0 +1,90 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-LIST 1
.Os
.Sh NAME
.Nm "kyua list"
.Nd Lists test cases and their metadata
.Sh SYNOPSIS
.Nm
.Op Fl -build-root Ar path
.Op Fl -kyuafile Ar file
.Op Fl -verbose
.Ar test_case1 Op Ar .. test_caseN
.Sh DESCRIPTION
The
.Nm
command scans all the test programs and test cases in a test suite (as
defined by a
.Xr kyuafile 5 )
and prints a list of all their names, optionally accompanied by any metadata
properties they have.
.Pp
The optional arguments to
.Nm
are used to select which test programs or test cases to run.
These are filters and are described below in
.Sx Test filters .
.Pp
This command must be run within a test suite or a test suite must be
provided with the
.Fl -kyuafile
flag.
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -build-root Ar path
Specifies the build root in which to find the test programs referenced
by the Kyuafile, if different from the Kyuafile's directory.
See
.Sx Build directories
below for more information.
.It Fl -kyuafile Ar path , Fl k Ar path
Specifies the Kyuafile to process.
Defaults to a
.Pa Kyuafile
file in the current directory.
.It Fl -verbose , Fl v
Prints metadata properties for every test case.
.El
.Ss Build directories
__include__ build-root.mdoc COMMAND=list
.Ss Test filters
__include__ test-filters.mdoc
.Sh EXIT STATUS
The
.Nm
command returns 0 on success or 1 if any of the given test case filters
does not match any test case.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyuafile 5

103
doc/kyua-report-html.1.in Normal file
View File

@ -0,0 +1,103 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-REPORT-HTML 1
.Os
.Sh NAME
.Nm "kyua report-html"
.Nd Generates an HTML report with the results of a test suite run
.Sh SYNOPSIS
.Nm
.Op Fl -force
.Op Fl -output Ar path
.Op Fl -results-file Ar file
.Op Fl -results-filter Ar types
.Sh DESCRIPTION
The
.Nm
command provides a simple mechanism to generate HTML reports of the
execution of a test suite.
The command processes a results file and then populates a directory with
multiple HTML and supporting files to describe the results recorded in that
results file.
.Pp
The HTML output is static and self-contained, so it can easily be served by
any simple web server.
The command expects the target directory to not exist, because it would
overwrite any contents if not careful.
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -force
Forces the deletion of the output directory if it exists.
Use care, as this effectively means a
.Sq rm -rf .
.It Fl -output Ar directory
Specifies the target directory into which to generate the HTML files.
The directory must not exist unless the
.Fl -force
option is provided.
The default is
.Pa ./html .
.It Fl -results-file Ar path , Fl s Ar path
__include__ results-file-flag-read.mdoc
.It Fl -results-filter Ar types
Comma-separated list of the test result types to include in the report.
The ordering of the values is respected so that you can determine how you
want the list of tests to be shown.
.Pp
The valid values are:
.Sq broken ,
.Sq failed ,
.Sq passed ,
.Sq skipped
and
.Sq xfail .
If the parameter supplied to the option is empty, filtering is suppressed
and all result types are shown in the report.
.Pp
The default value for this flag includes all the test results except the
passed tests.
Showing the passed tests by default clutters the report with too much
information, so only abnormal conditions are included.
.El
.Ss Results files
__include__ results-files.mdoc
.Sh EXIT STATUS
The
.Nm
command always returns 0.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh EXAMPLES
__include__ results-files-report-example.mdoc REPORT_COMMAND=report-html
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyua-report 1 ,
.Xr kyua-report-junit 1

87
doc/kyua-report-junit.1.in Normal file
View File

@ -0,0 +1,87 @@
.\" Copyright 2014 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-REPORT-JUNIT 1
.Os
.Sh NAME
.Nm "kyua report-junit"
.Nd Generates a JUnit report with the results of a test suite run
.Sh SYNOPSIS
.Nm
.Op Fl -output Ar path
.Op Fl -results-file Ar file
.Sh DESCRIPTION
The
.Nm
command provides a simple mechanism to generate JUnit reports of the
execution of a test suite.
The command processes a results file and then generates a single XML file
that complies with the JUnit XSchema.
.Pp
The JUnit output is static and self-contained, so it can easily be plugged
into any continuous integration system, like Jenkins.
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -output Ar directory
Specifies the file into which to store the JUnit report.
.It Fl -results-file Ar path , Fl s Ar path
__include__ results-file-flag-read.mdoc
.El
.Ss Caveats
Because of limitations in the JUnit XML schema, not all the data collected by
Kyua can be properly represented in JUnit reports.
However, because test data are extremely useful for debugging purposes, the
.Nm
command shovels these data into the JUnit output.
In particular:
.Bl -bullet
.It
The test case metadata values are prepended to the test case's standard error
output.
.It
Test cases that report expected failures as their results are recorded as
passed.
The fact that they failed as expected is recorded in the test case's standard
error output along with the corresponding reason.
.El
.Ss Results files
__include__ results-files.mdoc
.Sh EXIT STATUS
The
.Nm
command always returns 0.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh EXAMPLES
__include__ results-files-report-example.mdoc REPORT_COMMAND=report-junit
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyua-report 1 ,
.Xr kyua-report-html 1

118
doc/kyua-report.1.in Normal file
View File

@ -0,0 +1,118 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-REPORT 1
.Os
.Sh NAME
.Nm "kyua report"
.Nd Generates reports with the results of a test suite run
.Sh SYNOPSIS
.Nm
.Op Fl -output Ar path
.Op Fl -results-file Ar file
.Op Fl -results-filter Ar types
.Op Fl -verbose
.Op Ar test_filter1 .. test_filterN
.Sh DESCRIPTION
The
.Nm
command parses a results file and generates a user-friendly, plaintext
report for user consumption on the terminal.
By default, these reports only display a summary of the execution of the full
test suite to highlight where problems may lie.
.Pp
The output of
.Nm
can be customized to display full details on all executed test cases.
Additionally, the optional arguments to
.Nm
are used to select which test programs or test cases to display.
These are filters and are described below in
.Sx Test filters .
.Pp
Reports generated by
.Nm
are
.Em not intended to be machine-parseable .
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -output Ar path
Specifies the path to which the report should be written to.
The special values
.Pa /dev/stdout
and
.Pa /dev/stderr
can be used to specify the standard output and the standard error,
respectively.
.It Fl -results-file Ar path , Fl s Ar path
__include__ results-file-flag-read.mdoc
.It Fl -results-filter Ar types
Comma-separated list of the test result types to include in the report.
The ordering of the values is respected so that you can determine how you
want the list of tests to be shown.
.Pp
The valid values are:
.Sq broken ,
.Sq failed ,
.Sq passed ,
.Sq skipped
and
.Sq xfail .
If the parameter supplied to the option is empty, filtering is suppressed
and all result types are shown in the report.
.Pp
The default value for this flag includes all the test results except the
passed tests.
Showing the passed tests by default clutters the report with too much
information, so only abnormal conditions are included.
.It Fl -verbose
Prints a detailed report of the execution.
In addition to all the information printed by default, verbose reports
include the runtime context of the test suite run, the metadata of each
test case, and the verbatim output of the test cases.
.El
.Ss Results files
__include__ results-files.mdoc
.Ss Test filters
__include__ test-filters.mdoc
.Sh EXIT STATUS
The
.Nm
command returns 0 if no filters were specified or if all filters match one
or more test cases.
If any filter fails to match any test case, the command returns 1.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh EXAMPLES
__include__ results-files-report-example.mdoc REPORT_COMMAND=report
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyua-report-html 1 ,
.Xr kyua-report-junit 1

102
doc/kyua-test.1.in Normal file
View File

@ -0,0 +1,102 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-TEST 1
.Os
.Sh NAME
.Nm "kyua test"
.Nd Runs tests
.Sh SYNOPSIS
.Nm
.Op Fl -build-root Ar path
.Op Fl -kyuafile Ar file
.Op Fl -results-file Ar file
.Op Ar test_filter1 .. test_filterN
.Sh DESCRIPTION
The
.Nm
command loads a test suite definition from a
.Xr kyuafile 5 ,
runs the tests defined in it, and records the results into a new results
file.
By default, all tests in the test suite are executed but the optional
arguments to
.Nm
can be used to select which test programs or test cases to run.
These are filters and are described below in
.Sx Test filters .
.Pp
Every test executed by
.Nm
is run under a controlled environment as described in
.Sx Test isolation .
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -build-root Ar path
Specifies the build root in which to find the test programs referenced by
the Kyuafile, if different from the Kyuafile's directory.
See
.Sx Build directories
below for more information.
.It Fl -kyuafile Ar path , Fl k Ar path
Specifies the Kyuafile to process.
Defaults to a
.Pa Kyuafile
file in the current directory.
.It Fl -results-file Ar path , Fl s Ar path
__include__ results-file-flag-write.mdoc
.El
.Pp
You can later inspect the results of the test run in more detail by using
.Xr kyua-report 1
or you can execute a single test case with debugging functionality by using
.Xr kyua-debug 1 .
.Ss Build directories
__include__ build-root.mdoc COMMAND=test
.Ss Results files
__include__ results-files.mdoc
.Ss Test filters
__include__ test-filters.mdoc
.Ss Test isolation
__include__ test-isolation.mdoc
.Sh EXIT STATUS
The
.Nm
command returns 0 if all executed test cases pass or 1 if any of the
executed test cases fails or if any of the given test case filters does not
match any test case.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh EXAMPLES
__include__ results-files-report-example.mdoc REPORT_COMMAND=report
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyua-report 1 ,
.Xr kyuafile 5

400
doc/kyua.1.in Normal file
View File

@ -0,0 +1,400 @@
.\" Copyright 2011 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd May 12, 2015
.Dt KYUA 1
.Os
.Sh NAME
.Nm kyua
.Nd Testing framework for infrastructure software
.Sh SYNOPSIS
.Nm
.Op Fl -config Ar file
.Op Fl -logfile Ar file
.Op Fl -loglevel Ar level
.Op Fl -variable Ar name=value
.Ar command
.Op Ar command_options
.Op Ar command_arguments
.Sh DESCRIPTION
.Em If you are here looking for details on how to run the test suite in
.Pa /usr/tests
.Em ( or
.Pa __TESTSDIR__ ) ,
.Em please start by reading the
.Xr tests 7
.Em manual page that should be supplied by your system .
.Pp
Kyua is a testing framework for infrastructure software, originally
designed to equip BSD-based operating systems with a test suite.
This means that Kyua is lightweight and simple, and that Kyua integrates well
with various build systems and continuous integration frameworks.
.Pp
Kyua features an expressive test suite definition language, a safe
runtime engine for test suites and a powerful report generation engine.
.Pp
Kyua is for both developers and users, from the developer applying a
simple fix to a library to the system administrator deploying a new
release on a production machine.
.Pp
Kyua is able to execute test programs written with a plethora of testing
libraries and languages.
The test program library of choice is ATF, which
.Nm Ns 's
design originated from.
However, framework-less test programs and TAP-compliant test programs can also
be executed through
.Nm
.Ss Overview
As can be observed in the synopsis, the interface of
.Nm
implements a common subcommand-based interface.
The arguments to the tool specify, in this order: a set of common options
that all the commands accept, a required
.Ar command
name that specifies what
.Nm
should do, and
a set of possibly-optional
.Ar command_options
and
.Ar command_arguments
that are specific to the chosen command.
.Pp
The following options are recognized by all the commands.
Keep in mind that these must always be specified before the command name.
.Bl -tag -width XX
.It Fl -config Ar path , Fl c Ar path
Specifies the configuration file to process, which must be in the format
described in
.Xr kyua.conf 5 .
The special value
.Sq none
explicitly disables the loading of any configuration file.
.Pp
Defaults to
.Pa ~/.kyua/kyua.conf
if it exists, otherwise to
.Pa __CONFDIR__/kyua.conf
if it exists,
or else to
.Sq none .
.It Fl -logfile Ar path
Specifies the location of the file to which
.Nm
will log run time events useful for postmortem debugging.
.Pp
The default depends on different environment variables as described in
.Sx Logging ,
but typically the file will be stored within the user's home directory.
.It Fl -loglevel Ar level
Specifies the maximum logging level to record in the log file.
See
.Sx Logging
for more details.
.Pp
The default is
.Sq info .
.It Fl -variable Ar name=value , Fl v Ar name=value
Sets the
.Ar name
configuration variable to
.Ar value .
The values set through this option have preference over the values set in the
configuration file.
.Pp
The specified variable can either be a builtin variable or a test-suite
specific variable.
See
.Xr kyua.conf 5
for more details.
.El
.Pp
The following commands are generic and do not have any relation to the execution
of tests or the inspection of their results:
.Bl -tag -width reportXjunitXX -offset indent
.It Ar about
Shows general program information.
See
.Xr kyua-about 1 .
.It Ar config
Inspects the values of the configuration variables.
See
.Xr kyua-config 1 .
.It Ar db-exec
Executes an arbitrary SQL statement on a results file and prints the
resulting table.
See
.Xr kyua-db-exec 1 .
.It Ar help
Shows usage information.
See
.Xr kyua-help 1 .
.El
.Pp
The following commands are used to generate reports based on the data previously
recorded in a results file:
.Bl -tag -width reportXjunitXX -offset indent
.It Ar report
Generates a plaintext report.
Combined with its
.Fl -verbose
flag and the ability to only display specific test cases, this command can also
be used to debug test failures post-facto on the console.
See
.Xr kyua-report 1 .
.It Ar report-html
Generates an HTML report.
See
.Xr kyua-report-html 1 .
.It Ar report-junit
Generates a JUnit report.
See
.Xr kyua-report-junit 1 .
.El
.Pp
The following commands are used to interact with a test suite:
.Bl -tag -width reportXjunitXX -offset indent
.It Ar debug
Executes a single test case in a controlled environment for debugging purposes.
See
.Xr kyua-debug 1 .
.It Ar list
Lists test cases defined in a test suite by a
.Xr kyuafile 5
and, optionally, displays their metadata.
See
.Xr kyua-list 1 .
.It Ar test
Runs tests defined in a test suite by a
.Xr kyuafile 5 .
See
.Xr kyua-test 1 .
.El
.Ss Logging
.Nm
has a logging facility that collects all kinds of events at run time.
These events are always logged to a file so that the log is available when
it is most needed: right after a non-reproducible problem happens.
The only way to disable logging is by sending the log to
.Pa /dev/null .
.Pp
The location of the log file can be manually specified with the
.Fl -logfile
option, which applies to all commands.
If no file is explicitly specified, the location of the log files is chosen in
this order:
.Bl -enum -offset indent
.It
.Pa ${HOME}/.kyua/logs/
if
.Va HOME
is defined.
.It
.Pa ${TMPDIR}/
if
.Va TMPDIR
is defined.
.It
.Pa /tmp/ .
.El
.Pp
And the default naming scheme of the log files is:
.Sq <progname>.<timestamp>.log .
.Pp
The messages stored in the log file have a level (or severity) attached to
them.
These are:
.Bl -tag -width warningXX -offset indent
.It error
Fatal error messages.
The program generally terminates after these, either in a clean manner or by
crashing.
.It warning
Non-fatal error messages.
These generally report a condition that must be addressed but the application
can continue to run.
.It info
Informational messages.
These tell the user what the program was doing at a general level of
operation.
.It debug
Detailed informational messages.
These are often useful when debugging problems in the application, as they
contain lots of internal details.
.El
.Pp
The default log level is
.Sq info
unless explicitly overridden with
.Fl -loglevel .
.Pp
The log file is a plain text file containing one line per log record.
The format of each line is as follows:
.Bd -literal -offset indent
timestamp entry_type pid file:line: message
.Ed
.Pp
.Ar entry_type
can be one of:
.Sq E
for an error,
.Sq W
for a warning,
.Sq I
for an informational message and
.Sq D
for a debug message.
.Ss Bug reporting
If you think you have encountered a bug in
.Nm ,
please take the time to let the developers know about it.
This will ensure that the bug is addressed and potentially fixed in the next
Kyua release.
.Pp
The first step in reporting a bug is to check if there already is a similar
bug in the database.
You can check what issues are currently in the database by going to:
.Bd -literal -offset indent
https://github.com/jmmv/kyua/issues/
.Ed
.Pp
If there is no existing issue that describes an issue similar to the
one you are experiencing, you can open a new one by visiting:
.Bd -literal -offset indent
https://github.com/jmmv/kyua/issues/new/
.Ed
.Pp
When doing so, please include as much detail as possible.
Among other things, explain what operating system and platform you are running
.Nm
on, what were you trying to do, what exact messages you saw on the screen,
how did you expect the program to behave, and any other details that you
may find relevant.
.Pp
Also, please include a copy of the log file corresponding to the problem
you are experiencing.
Unless you have changed the location of the log files, you can most likely
find them in
.Pa ~/.kyua/logs/ .
If the problem is reproducible, it is good idea to regenerate the log file
with an increased log level so as to provide more information.
For example:
.Bd -literal -offset indent
$ kyua --logfile=problem.log --loglevel=debug \\
[rest of the command line]
.Ed
.Sh ENVIRONMENT
The following variables are recognized and can be freely tuned by the end user:
.Bl -tag -width COLUMNSXX
.It Va COLUMNS
The width of the screen, in number of characters.
.Nm
uses this to wrap long lines.
If not present, the width of the screen is determined from the terminal
stdout is connected to, and, if the guessing fails, this defaults to infinity.
.It Va HOME
Path to the user's home directory.
.Nm
uses this location to determine paths to configuration files and default log
files.
.It Va TMPDIR
Path to the system-wide temporary directory.
.Nm
uses this location to place the work directory of test cases, among other
things.
.Pp
The default value of this variable depends on the operating system.
In general, it is
.Pa /tmp .
.El
.Pp
The following variables are also recognized, but you should not need to set them
during normal operation.
They are only provided to override the value of built-in values, which is useful
when testing
.Nm
itself:
.Bl -tag -width KYUAXCONFDIRXX
.It Va KYUA_CONFDIR
Path to the system-wide configuration files for
.Nm .
.Pp
Defaults to
.Pa __CONFDIR__ .
.It Va KYUA_DOCDIR
Path to the location of installed documentation.
.Pp
Defaults to
.Pa __DOCDIR__ .
.It Va KYUA_MISCDIR
Path to the location of the installed miscellaneous scripts and data
files provided by
.Nm .
.Pp
Defaults to
.Pa __MISCDIR__ .
.It Va KYUA_STOREDIR
Path to the location of the installed store support files; e.g., the
directory containing the SQL database schema.
.Pp
Defaults to
.Pa __STOREDIR__ .
.El
.Sh FILES
.Bl -tag -width XXXX
.It Pa ~/.kyua/store/
Default location for the results files.
.It Pa ~/.kyua/kyua.conf
User-specific configuration file.
.It Pa ~/.kyua/logs/
Default location for the collected log files.
.It Pa __CONFDIR__/kyua.conf
System-wide configuration file.
.El
.Sh EXIT STATUS
.Nm
returns 0 on success, 1 on a controlled error condition in the given
subcommand, 2 on a general unexpected error and 3 on a usage error.
.Pp
The documentation of the subcommands in the corresponding manual pages only
details the difference between a successful exit (0) and the detection of a
controlled error (1).
Even though when those manual pages do not describe any other exit statuses,
codes above 1 can be returned.
.Sh SEE ALSO
.Xr kyua.conf 5 ,
.Xr kyuafile 5 ,
.Xr atf 7 ,
.Xr tests 7
.Sh AUTHORS
For more details on the people that made
.Nm
possible and the license terms, run:
.Bd -literal -offset indent
$ kyua about
.Ed

141
doc/kyua.conf.5.in Normal file
View File

@ -0,0 +1,141 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd February 20, 2015
.Dt KYUA.CONF 5
.Os
.Sh NAME
.Nm kyua.conf
.Nd Configuration file for the kyua tool
.Sh SYNOPSIS
.Fn syntax "int version"
.Pp
Variables:
.Va architecture ,
.Va platform ,
.Va test_suites ,
.Va unprivileged_user .
.Sh DESCRIPTION
The configuration of Kyua is a simple collection of key/value pairs called
configuration variables.
There are configuration variables that have a special meaning to the runtime
engine implemented by
.Xr kyua 1 ,
and there are variables that only have meaning in the context of particular
test suites.
.Pp
Configuration files are Lua scripts.
In their most basic form, their whole purpose is to assign values to
variables, but the user has the freedom to implement any logic he desires
to compute such values.
.Ss File versioning
Every
.Nm
file starts with a call to
.Fn syntax "int version" .
This call determines the specific schema used by the file so that future
backwards-incompatible modifications to the file can be introduced.
.Pp
Any new
.Nm
file should set
.Fa version
to
.Sq 2 .
.Ss Runtime configuration variables
The following variables are internally recognized by
.Xr kyua 1 :
.Bl -tag -width XX -offset indent
.It Va architecture
Name of the system architecture (aka processor type).
.It Va parallelism
Maximum number of test cases to execute concurrently.
.It Va platform
Name of the system platform (aka machine type).
.It Va unprivileged_user
Name or UID of the unprivileged user.
.Pp
If set, the given user must exist in the system and his privileges will be
used to run test cases that need regular privileges when
.Xr kyua 1
is executed as root.
.El
.Ss Test-suite configuration variables
Each test suite is able to recognize arbitrary configuration variables, and
their type and meaning is specific to the test suite.
Because the existence and naming of these variables depends on every test
suite, this manual page cannot detail them; please refer to the documentation
of the test suite you are working with for more details on this topic.
.Pp
Test-suite specific configuration variables are defined inside the
.Va test_suites
dictionary.
The general syntax is:
.Bd -literal -offset indent
test_suites.<test_suite_name>.<variable_name> = <value>
.Ed
.Pp
where
.Va test_suite_name
is the name of the test suite,
.Va variable_name
is the name of the variable to set, and
.Va value
is a value.
The value can be a string, an integer or a boolean.
.Sh FILES
.Bl -tag -width XX
.It __EGDIR__/kyua.conf
Sample configuration file.
.El
.Sh EXAMPLES
The following
.Nm
shows a simple configuration file that overrides a bunch of the built-in
.Xr kyua 1
configuration variables:
.Bd -literal -offset indent
syntax(2)
architecture = 'x86_64'
platform = 'amd64'
.Ed
.Pp
The following is a more complex example that introduces the definition of
per-test suite configuration variables:
.Bd -literal -offset indent
syntax(2)
-- Assign built-in variables.
unprivileged_user = '_tests'
-- Assign test-suite variables. All of these must be strings.
test_suites.NetBSD.file_systems = 'ffs ext2fs'
test_suites.X11.graphics_driver = 'vesa'
.Ed
.Sh SEE ALSO
.Xr kyua 1

407
doc/kyuafile.5.in Normal file
View File

@ -0,0 +1,407 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd July 3, 2015
.Dt KYUAFILE 5
.Os
.Sh NAME
.Nm Kyuafile
.Nd Test suite description files
.Sh SYNOPSIS
.Fn atf_test_program "string name" "[string metadata]"
.Fn current_kyuafile
.Fn fs.basename "string path"
.Fn fs.dirname "string path"
.Fn fs.exists "string path"
.Fn fs.files "string path"
.Fn fs.is_absolute "string path"
.Fn fs.join "string path" "string path"
.Fn include "string path"
.Fn plain_test_program "string name" "[string metadata]"
.Fn syntax "int version"
.Fn tap_test_program "string name" "[string metadata]"
.Fn test_suite "string name"
.Sh DESCRIPTION
A test suite is a collection of test programs and is represented by a
hierarchical layout of test binaries on the file system.
Any subtree of the file system can represent a test suite, provided that it
includes one or more
.Nm Ns s ,
which are the test suite definition files.
.Pp
A
.Nm
is a Lua script whose purpose is to describe the structure of the test
suite it belongs to.
To do so, the script has access to a collection of special functions provided
by
.Xr kyua 1
as described in
.Sx Helper functions .
.Ss File versioning
Every
.Nm
file starts with a call to
.Fn syntax "int version" .
This call determines the specific schema used by the file so that future
backwards-incompatible modifications to the file can be introduced.
.Pp
Any new
.Nm
file should set
.Fa version
to
.Sq 2 .
.Ss Test suite definition
If the
.Nm
registers any test programs,
the
.Nm
must define the name of the test suite the test programs belong to by using the
.Fn test_suite
function at the very beginning of the file.
.Pp
The test suite name provided in the
.Fn test_suite
call tells
.Xr kyua 1
which set of configuration variables from
.Xr kyua.conf 5
to pass to the test programs at run time.
.Ss Test program registration
A
.Nm
can register test programs by means of a variety of
.Fn *_test_program
functions, all of which take the name of a test program and a set of
optional metadata properties that describe such test program.
.Pp
The test programs to be registered must live in the current directory; in
other words, the various
.Fn *_test_program
calls cannot reference test programs in other directories.
The rationale for this is to force all
.Nm
files to be self-contained, and to simplify their internal representation.
.Pp
.Em ATF test programs
are those that use the
.Xr atf 7
libraries.
They can be registered with the
.Fn atf_test_program
table constructor.
This function takes the
.Fa name
of the test program and a collection of optional metadata settings for all
the test cases in the test program.
Any metadata properties defined by the test cases themselves override the
metadata values defined here.
.Pp
.Em Plain test programs
are those that return 0 on success and non-0 on failure; in general, most test
programs (even those that use fancy unit-testing libraries) behave this way and
thus also qualify as plain test programs.
They can be registered with the
.Fn plain_test_program
table constructor.
This function takes the
.Fa name
of the test program, an optional
.Fa test_suite
name that overrides the global test suite name, and a collection of optional
metadata settings for the test program.
.Pp
.Em TAP test programs
are those that implement the Test Anything Protocol.
They can be registered with the
.Fn tap_test_program
table constructor.
This function takes the
.Fa name
of the test program and a collection of optional metadata settings for the
test program.
.Pp
The following metadata properties can be passed to any test program definition:
.Bl -tag -width XX -offset indent
.It Va allowed_architectures
Whitespace-separated list of machine architecture names allowed by the test.
If empty or not defined, the test is allowed to run on any machine
architecture.
.It Va allowed_platforms
Whitespace-separated list of machine platform names allowed by the test.
If empty or not defined, the test is allowed to run on any machine
platform.
.It Va custom.NAME
Custom variable defined by the test where
.Sq NAME
denotes the name of the variable.
These variables are useful to tag your tests with information specific to
your project.
The values of such variables are propagated all the way from the tests to the
results files and later to any generated reports.
.Pp
Note that if the name happens to have dashes or any other special characters
in it, you will have to use a special Lua syntax to define the property.
Refer to the
.Sx EXAMPLES
section below for clarification.
.It Va description
Textual description of the test.
.It Va is_exclusive
If true, indicates that this test program cannot be executed along any other
programs at the same time.
Test programs that affect global system state, such as those that modify the
value of a
.Xr sysctl 8
setting, must set themselves as exclusive to prevent failures due to race
conditions.
Defaults to false.
.It Va required_configs
Whitespace-separated list of configuration variables that the test requires
to be defined before it can run.
.It Va required_disk_space
Amount of available disk space that the test needs to run successfully.
.It Va required_files
Whitespace-separated list of paths that the test requires to exist before
it can run.
.It Va required_memory
Amount of physical memory that the test needs to run successfully.
.It Va required_programs
Whitespace-separated list of basenames or absolute paths pointing to executable
binaries that the test requires to exist before it can run.
.It Va required_user
If empty, the test has no restrictions on the calling user for it to run.
If set to
.Sq unprivileged ,
the test needs to not run as root.
If set to
.Sq root ,
the test must run as root.
.It Va timeout
Amount of seconds that the test is allowed to execute before being killed.
.El
.Ss Recursion
To reference test programs in another subdirectory, a different
.Nm
must be created in that directory and it must be included into the original
.Nm
by means of the
.Fn include
function.
.Pp
.Fn include
may only be called with a relative path and with at most one directory
component.
This is by design: Kyua uses the file system structure as the layout of the
test suite definition.
Therefore, each subdirectory in a test suite must include its own
.Nm
and each
.Nm
can only descend into the
.Nm Ns s
of immediate subdirectories.
.Pp
If you need to source a
.Nm
located in disjoint parts of your file system namespace, you will have to
create a
.Sq shadow tree
using symbolic links and possibly helper
.Nm Ns s
to plug the various subdirectories together.
See the
.Sx EXAMPLES
section below for details.
.Pp
Note that each file is processed in its own Lua environment: there is no
mechanism to pass state from one file to the other.
The reason for this is that there is no such thing as a
.Dq top-level
.Nm
in a test suite: the user has to be able to run the test suite from any
directory in a given hierarchy, and this execution must not depend on files
that live in parent directories.
.Ss Top-level Kyuafile
Every system has a top directory into which test suites get installed.
The default is
.Pa __TESTSDIR__ .
Within this directory live test suites, each of which is in an independent
subdirectory.
Each subdirectory can be provided separately by independent third-party
packages.
.Pp
Kyua allows running all the installed test suites at once in order to
provide comprehensive cross-component reports.
In order to do this, there is a special file in the top directory that knows
how to inspect the subdirectories in search for other Kyuafiles and include
them.
.Pp
The
.Sx FILES
section includes more details on where this file lives.
.Ss Helper functions
The
.Sq base ,
.Sq string ,
and
.Sq table
Lua modules are fully available in the context of a
.Nm .
.Pp
The following extra functions are provided by Kyua:
.Bl -tag -width XX -offset indent
.It Ft string Fn current_kyuafile
Returns the absolute path to the current
.Nm .
.It Ft string Fn fs.basename "string path"
Returns the last component of the given path.
.It Ft string Fn fs.dirname "string path"
Returns the given path without its last component or a dot if the path has
a single component.
.It Ft bool Fn fs.exists "string path"
Checks if the given path exists.
If the path is not absolute, it is relative to the directory containing the
.Nm
in which the call to this function occurs.
.It Ft iterator Fn fs.files "string path"
Opens a directory for scanning of its entries.
The returned iterator yields an entry on each call, and the entry is simply
the filename.
If the path is not absolute, it is relative to the directory containing the
.Nm
in which the call to this function occurs.
.It Ft is_absolute Fn fs.is_absolute "string path"
Returns true if the given path is absolute; false otherwise.
.It Ft join Fn fs.join "string path" "string path"
Concatenates the two paths.
The second path cannot be absolute.
.El
.Sh FILES
.Bl -tag -width XX
.It Pa __TESTSDIR__/Kyuafile .
Top-level
.Nm
for the current system.
.It Pa __EGDIR__/Kyuafile.top .
Sample file to serve as a top-level
.Nm .
.El
.Sh EXAMPLES
The following
.Nm
is the simplest you can define.
It provides a test suite definition and registers a couple of different test
programs using different interfaces:
.Bd -literal -offset indent
syntax(2)
test_suite('first')
atf_test_program{name='integration_test'}
plain_test_program{name='legacy_test'}
.Ed
.Pp
The following example is a bit more elaborate.
It introduces some metadata properties to the test program definitions and
recurses into a couple of subdirectories:
.Bd -literal -offset indent
syntax(2)
test_suite('second')
plain_test_program{name='legacy_test',
allowed_architectures='amd64 i386',
required_files='/bin/ls',
timeout=30}
tap_test_program{name='privileged_test',
required_user='root'}
include('module-1/Kyuafile')
include('module-2/Kyuafile')
.Ed
.Pp
The syntax to define custom properties may be not obvious if their names
have any characters that make the property name not be a valid Lua identifier.
Dashes are just one example.
To set such properties, do something like this:
.Bd -literal -offset indent
syntax(2)
test_suite('FreeBSD')
plain_test_program{name='the_test',
['custom.FreeBSD-Bug-Id']='category/12345'}
.Ed
.Ss Connecting disjoint test suites
Now suppose you had various test suites on your file system and you would
like to connect them together so that they could be executed and treated as
a single unit.
The test suites we would like to connect live under
.Pa /usr/tests ,
.Pa /usr/local/tests
and
.Pa ~/local/tests .
.Pp
We cannot create a
.Nm
that references these because the
.Fn include
directive does not support absolute paths.
Instead, what we can do is create a shadow tree using symbolic links:
.Bd -literal -offset indent
$ mkdir ~/everything
$ ln -s /usr/tests ~/everything/system-tests
$ ln -s /usr/local/tests ~/everything/local-tests
$ ln -s ~/local/tests ~/everything/home-tests
.Ed
.Pp
And then we create an
.Pa ~/everything/Kyuafile
file to drive the execution of the integrated test suite:
.Bd -literal -offset indent
syntax(2)
test_suite('test-all-the-things')
include('system-tests/Kyuafile')
include('local-tests/Kyuafile')
include('home-tests/Kyuafile')
.Ed
.Pp
Or, simply, you could reuse the sample top-level
.Nm
to avoid having to manually craft the list of directories into which to
recurse:
.Bd -literal -offset indent
$ cp __EGDIR__/Kyuafile.top ~/everything/Kyuafile
.Ed
.Sh SEE ALSO
.Xr kyua 1

171
doc/manbuild.sh Executable file
View File

@ -0,0 +1,171 @@
#! /bin/sh
# Copyright 2014 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# \file doc/manbuild.sh
# Generates a manual page from a source file.
#
# Input files can have __VAR__-style patterns in them that are replaced
# with the values provided by the caller via the -v VAR=VALUE flag.
#
# Input files can also include other files using the __include__ directive,
# which takes a relative path to the file to include plus an optional
# collection of additional variables to replace in the included file.
# Name of the running program for error reporting purposes.
Prog_Name="${0##*/}"
# Prints an error message and exits.
#
# Args:
# ...: The error message to print. Multiple arguments are joined with a
# single space separator.
err() {
echo "${Prog_Name}: ${*}" 1>&2
exit 1
}
# Invokes sed(1) translating input variables to expressions.
#
# Args:
# ...: List of var=value pairs to replace.
#
# Returns:
# True if the operation succeeds; false otherwise.
sed_with_vars() {
local vars="${*}"
set --
for pair in ${vars}; do
local var="$(echo "${pair}" | cut -d = -f 1)"
local value="$(echo "${pair}" | cut -d = -f 2-)"
set -- "${@}" -e"s&__${var}__&${value}&g"
done
if [ "${#}" -gt 0 ]; then
sed "${@}"
else
cat
fi
}
# Generates the manual page reading from stdin and dumping to stdout.
#
# Args:
# include_dir: Path to the directory containing the include files.
# ...: List of var=value pairs to replace in the manpage.
#
# Returns:
# True if the generation succeeds; false otherwise.
generate() {
local include_dir="${1}"; shift
while :; do
local read_ok=yes
local oldifs="${IFS}"
IFS=
read -r line || read_ok=no
IFS="${oldifs}"
[ "${read_ok}" = yes ] || break
case "${line}" in
__include__*)
local file="$(echo "${line}" | cut -d ' ' -f 2)"
local extra_vars="$(echo "${line}" | cut -d ' ' -f 3-)"
# If we fail to output the included file, just leave the line as
# is. validate_file() will later error out.
[ -f "${include_dir}/${file}" ] || echo "${line}"
generate <"${include_dir}/${file}" "${include_dir}" \
"${@}" ${extra_vars} || echo "${line}"
;;
*)
echo "${line}"
;;
esac
done | sed_with_vars "${@}"
}
# Validates that the manual page has been properly generated.
#
# In particular, this checks if any directives or common replacement patterns
# have been left in place.
#
# Returns:
# True if the manual page is valid; false otherwise.
validate_file() {
local filename="${1}"
if grep '__[A-Za-z0-9]*__' "${filename}" >/dev/null; then
return 1
else
return 0
fi
}
# Program entry point.
main() {
local vars=
while getopts :v: arg; do
case "${arg}" in
v)
vars="${vars} ${OPTARG}"
;;
\?)
err "Unknown option -${OPTARG}"
;;
esac
done
shift $((${OPTIND} - 1))
[ ${#} -eq 2 ] || err "Must provide input and output names as arguments"
local input="${1}"; shift
local output="${1}"; shift
trap "rm -f '${output}.tmp'" EXIT HUP INT TERM
generate "$(dirname "${input}")" ${vars} \
<"${input}" >"${output}.tmp" \
|| err "Failed to generate ${output}"
if validate_file "${output}.tmp"; then
:
else
err "Failed to generate ${output}; some patterns were left unreplaced"
fi
mv "${output}.tmp" "${output}"
}
main "${@}"

235
doc/manbuild_test.sh Executable file
View File

@ -0,0 +1,235 @@
# Copyright 2014 The Kyua Authors.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# Absolute path to the uninstalled script.
MANBUILD="__MANBUILD__"
atf_test_case empty
empty_body() {
touch input
atf_check "${MANBUILD}" input output
atf_check cat output
}
atf_test_case no_replacements
no_replacements_body() {
cat >input <<EOF
This is a manpage.
With more than one line.
EOF
atf_check "${MANBUILD}" input output
atf_check -o file:input cat output
}
atf_test_case one_replacement
one_replacement_body() {
cat >input <<EOF
This is a manpage.
Where __FOO__ gets replaced.
And nothing more.
EOF
atf_check "${MANBUILD}" -v FOO=this input output
cat >expout <<EOF
This is a manpage.
Where this gets replaced.
And nothing more.
EOF
atf_check -o file:expout cat output
}
atf_test_case some_replacements
some_replacements_body() {
cat >input <<EOF
This is a manpage.
Where __FOO__ gets __BAR__.
And nothing more.
EOF
atf_check "${MANBUILD}" -v FOO=this -v BAR=replaced input output
cat >expout <<EOF
This is a manpage.
Where this gets replaced.
And nothing more.
EOF
atf_check -o file:expout cat output
}
atf_test_case preserve_tricky_lines
preserve_tricky_lines_body() {
cat >input <<EOF
Begin
This line is intended.
This other \\
continues later.
\*(LtAnd this has strange characters\*(Gt
End
EOF
atf_check "${MANBUILD}" input output
cat >expout <<EOF
Begin
This line is intended.
This other \\
continues later.
\*(LtAnd this has strange characters\*(Gt
End
EOF
atf_check -o file:expout cat output
}
atf_test_case includes_ok
includes_ok_body() {
mkdir doc doc/subdir
cat >doc/input <<EOF
This is a manpage.
__include__ subdir/chunk
There is more...
__include__ chunk
And done!
EOF
cat >doc/subdir/chunk <<EOF
This is the first inclusion
and worked __OK__.
EOF
cat >doc/chunk <<EOF
This is the second inclusion.
EOF
atf_check "${MANBUILD}" -v OK=ok doc/input output
cat >expout <<EOF
This is a manpage.
This is the first inclusion
and worked ok.
There is more...
This is the second inclusion.
And done!
EOF
atf_check -o file:expout cat output
}
atf_test_case includes_parameterized
includes_parameterized_body() {
cat >input <<EOF
__include__ chunk value=first
__include__ chunk value=second
EOF
cat >chunk <<EOF
This is a chunk with value: __value__.
EOF
atf_check "${MANBUILD}" input output
cat >expout <<EOF
This is a chunk with value: first.
This is a chunk with value: second.
EOF
atf_check -o file:expout cat output
}
atf_test_case includes_fail
includes_fail_body() {
cat >input <<EOF
This is a manpage.
__include__ missing
EOF
atf_check -s exit:1 -o ignore \
-e match:"manbuild.sh: Failed to generate output.*left unreplaced" \
"${MANBUILD}" input output
[ ! -f output ] || atf_fail "Output file was generated but it should" \
"not have been"
}
atf_test_case generate_fail
generate_fail_body() {
touch input
atf_check -s exit:1 -o ignore \
-e match:"manbuild.sh: Failed to generate output" \
"${MANBUILD}" -v 'malformed&name=value' input output
[ ! -f output ] || atf_fail "Output file was generated but it should" \
"not have been"
}
atf_test_case validate_fail
validate_fail_body() {
cat >input <<EOF
This is a manpage.
Where __FOO__ gets replaced.
But where __BAR__ doesn't.
EOF
atf_check -s exit:1 -o ignore \
-e match:"manbuild.sh: Failed to generate output.*left unreplaced" \
"${MANBUILD}" -v FOO=this input output
[ ! -f output ] || atf_fail "Output file was generated but it should" \
"not have been"
}
atf_test_case bad_args
bad_args_body() {
atf_check -s exit:1 \
-e match:'manbuild.sh: Must provide input and output names' \
"${MANBUILD}"
atf_check -s exit:1 \
-e match:'manbuild.sh: Must provide input and output names' \
"${MANBUILD}" foo
atf_check -s exit:1 \
-e match:'manbuild.sh: Must provide input and output names' \
"${MANBUILD}" foo bar baz
}
atf_test_case bad_option
bad_option_body() {
atf_check -s exit:1 -e match:'manbuild.sh: Unknown option -Z' \
"${MANBUILD}" -Z
}
atf_init_test_cases() {
atf_add_test_case empty
atf_add_test_case no_replacements
atf_add_test_case one_replacement
atf_add_test_case some_replacements
atf_add_test_case preserve_tricky_lines
atf_add_test_case includes_ok
atf_add_test_case includes_parameterized
atf_add_test_case includes_fail
atf_add_test_case generate_fail
atf_add_test_case validate_fail
atf_add_test_case bad_args
atf_add_test_case bad_option
}

53
doc/results-file-flag-read.mdoc Normal file
View File

@ -0,0 +1,53 @@
.\" Copyright 2014 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Specifies the results file to operate on.
Defaults to
.Sq LATEST ,
which causes
.Nm
to automatically load the latest results file from the current test suite.
.Pp
The following values are accepted:
.Bl -tag -width XX
.It Sq LATEST
Requests the load of the latest results file available for the test suite rooted
at the current directory.
.It Directory
Requests the load of the latest results file available for the test suite rooted
at the given directory.
.It Test suite name
Requests the load of the latest results file available for the given test suite.
.It Results identifier
Requests the load of a specific results file.
.It Explicit file name (aka everything else)
Load the specified results file.
.El
.Pp
See
.Sx Results files
for more details.

46
doc/results-file-flag-write.mdoc Normal file
View File

@ -0,0 +1,46 @@
.\" Copyright 2014 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Specifies the results file to create.
Defaults to
.Sq LATEST ,
which causes
.Nm
to automatically generate a new results file for the test run.
.Pp
The following values are accepted:
.Bl -tag -width XX
.It Sq NEW
Requests the automatic generation of a new results filename based on the test
suite being run and the current time.
.It Explicit filename (aka everything else)
Store the results file where indicated.
.El
.Pp
See
.Sx Results files
for more details.

32
doc/results-files-report-example.mdoc Normal file
View File

@ -0,0 +1,32 @@
.Ss Workflow with results files
If one runs the following command twice in a row:
.Bd -literal -offset indent
kyua test -k /usr/tests/Kyuafile
.Ed
.Pp
the two executions will generate two different files with names like:
.Bd -literal -offset indent
~/.kyua/store/results.usr_tests.20140731-150500-196784.db
~/.kyua/store/results.usr_tests.20140731-151730-997451.db
.Ed
.Pp
Taking advantage of the default naming scheme, the following commands would all
generate a report for the results of the
.Em latest
execution of the test suite:
.Bd -literal -offset indent
cd /usr/tests && kyua __REPORT_COMMAND__
cd /usr/tests && kyua __REPORT_COMMAND__ --results-file=LATEST
kyua __REPORT_COMMAND__ --results-file=/usr/tests
kyua __REPORT_COMMAND__ --results-file=usr_tests
kyua __REPORT_COMMAND__ --results-file=usr_tests.20140731-151730-997451
.Ed
.Pp
But it is also possible to explicitly load data for older runs or from
explicitly-named files:
.Bd -literal -offset indent
kyua __REPORT_COMMAND__ \\
--results-file=usr_tests.20140731-150500-196784
kyua __REPORT_COMMAND__ \\
--results-file=~/.kyua/store/results.usr_tests.20140731-150500-196784.db
.Ed

68
doc/results-files.mdoc Normal file
View File

@ -0,0 +1,68 @@
.\" Copyright 2014 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Results files contain, as their name implies, the results of the execution of a
test suite.
Each test suite executed by
.Xr kyua-test 1
generates a new results file, and such results files can be loaded later on by
inspection commands such as
.Xr kyua-report 1
to analyze their contents.
.Pp
Results files support identifier-based lookups and also path name lookups.
The differences between the two are described below.
.Pp
The default naming scheme for the results files provides simple support for
identifier-based lookups and historical recording of test suite runs.
Each results file is given an identifier derived from the test suite that
generated it and the time the test suite was run.
Kyua can later look up results files by these fields.
.Pp
The identifier follows this pattern:
.Bd -literal -offset indent
\*(Lttest_suite\*(Gt.\*(LtYYYYMMDD\*(Gt-\*(LtHHMMSS\*(Gt-\*(Ltuuuuuu\*(Gt
.Ed
.Pp
where
.Sq test_suite
is the path to the root of the test suite that was run with all slashes replaced
by underscores and
.Sq YYYYMMDD-HHMMSS-uuuuuu
is a timestamp with microsecond resolution.
.Pp
When using the default naming scheme, results files are stored in the
.Pa ~/.kyua/store/
subdirectory and each file holds a name of the form:
.Bd -literal -offset indent
~/.kyua/store/results.\*(Ltidentifier\*(Gt.db
.Ed
.Pp
Results files are simple SQLite databases with the schema described in the
.Pa __STOREDIR__/schema_v?.sql
files.
For details on the schema, please refer to the heavily commented SQL file.

40
doc/test-filters.mdoc Normal file
View File

@ -0,0 +1,40 @@
.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
A
.Em test filter
is a string that is used to match test cases or test programs in a test suite.
Filters have the following form:
.Bd -literal -offset indent
test_program_name[:test_case_name]
.Ed
.Pp
Where
.Sq test_program_name
is the name of a test program or a subdirectory in the test suite, and
.Sq test_case_name
is the name of a test case.

112
doc/test-isolation.mdoc Normal file
View File

@ -0,0 +1,112 @@
.\" Copyright 2014 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
The test programs and test cases run by
.Nm
are all executed in a deterministic environment.
This known, clean environment serves to make the test execution as
reproducible as possible and also to prevent clashes between tests that may,
for example, create auxiliary files with overlapping names.
.Pp
For plain test programs and for TAP test programs, the whole test program
is run under a single instance of the environment described in this page.
For ATF test programs (see
.Xr atf 7 ) ,
each individual test case
.Em and
test cleanup routine are executed in separate environments.
.Bl -tag -width XX
.It Process space
Each test is executed in an independent processes.
Corollary: the test can do whatever it wants to the current process (such
as modify global variables) without having to undo such changes.
.It Session and process group
The test is executed in its own session and its own process group.
There is no controlling terminal attached to the session.
.Pp
Should the test spawn any children, the children should maintain the same
session and process group.
Modifying any of these settings prevents
.Nm
from being able to kill any stray subprocess as part of the cleanup phase.
If modifying these settings is necessary, or if any subprocess started by
the test decides to use a different process group or session, it is the
responsibility of the test to ensure those subprocesses are forcibly
terminated during cleanup.
.It Work directory
The test is executed in a temporary directory automatically created by the
runtime engine.
Corollary: the test can write to its current directory
without needing to clean any files and/or directories it creates.
The runtime engine takes care to recursively delete the temporary directories
after the execution of a test case.
Any file systems mounted within the temporary directory are also unmounted.
.It Home directory
The
.Va HOME
environment variable is set to the absolute path of the work directory.
.It Umask
The value of the umask is set to 0022.
.It Environment
The
.Va LANG ,
.Va LC_ALL ,
.Va LC_COLLATE ,
.Va LC_CTYPE ,
.Va LC_MESSAGES ,
.Va LC_MONETARY ,
.Va LC_NUMERIC
and
.Va LC_TIME
variables are unset.
.Pp
The
.Va TZ
variable is set to
.Sq UTC .
.Pp
The
.Va TMPDIR
variable is set to the absolute path of the work directory.
This is to prevent the test from mistakenly using a temporary directory
outside of the automatically-managed work directory, should the test use the
.Xr mktemp 3
familiy of functions.
.It Process limits
The maximum soft core size limit is raised to its corresponding hard limit.
This is a simple, best-effort attempt at allowing tests to dump core for
further diagnostic purposes.
.It Configuration varibles
The test engine may pass run-time configuration variables to the test program
via the environment.
The name of the configuration variable is prefixed with
.Sq TEST_ENV_
so that a configuration variable of the form
.Sq foo=bar
becomes accessible in the environment as
.Sq TEST_ENV_foo=bar .
.El

7
drivers/Kyuafile Normal file
View File

@ -0,0 +1,7 @@
syntax(2)
test_suite("kyua")
atf_test_program{name="list_tests_test"}
atf_test_program{name="report_junit_test"}
atf_test_program{name="scan_results_test"}

Some files were not shown because too many files have changed in this diff Show More