8fee91db34
Experimental version released on February 7th, 2014. This is the last release to bundle the code for the deprecated tools. The next release will drop their code and will stop worrying about backwards compatibility between the ATF libraries and what the old tools may or may not support. If you still require the old tools for some reason, grab a copy of the 'tools' directory now. The code in this directory is standalone and does not depend on any internal details of atf-c++ any longer. * Various fixes and improvements to support running as part of the FreeBSD test suite. * Project hosting moved from Google Code (as a subproject of Kyua) to GitHub (as a first-class project). The main reason for the change is the suppression of binary downloads in Google Code on Jan 15th, 2014. See https://github.com/jmmv/atf/ * Removed builtin help from atf-sh(1) and atf-check(1) for simplicity reasons. In other words, their -h option is gone. * Moved the code of the deprecated tools into a 'tools' directory and completely decoupled their code from the internals of atf-c++. The reason for this is to painlessly allow a third-party to maintain a copy of these tools after we delete them because upcoming changes to atf-c++ would break the stale tools.
203 lines
6.0 KiB
Groff
203 lines
6.0 KiB
Groff
.\"
|
|
.\" Automated Testing Framework (atf)
|
|
.\"
|
|
.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
.\" modification, are permitted provided that the following conditions
|
|
.\" are met:
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
|
|
.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
|
|
.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
|
.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
|
|
.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
|
|
.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
|
|
.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
|
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
|
|
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
|
|
.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
|
|
.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
.\"
|
|
.Dd November 1, 2010
|
|
.Dt ATF-RUN 1
|
|
.Os
|
|
.Sh NAME
|
|
.Nm atf-run
|
|
.Nd executes a collection of test programs
|
|
.Sh SYNOPSIS
|
|
.Nm
|
|
.Op Fl v Ar var1=value1 Op .. Fl v Ar varN=valueN
|
|
.Op Ar test_program1 Op Ar .. test_programN
|
|
.Nm
|
|
.Fl h
|
|
.Sh DESCRIPTION
|
|
.Nm
|
|
executes a collection of test programs or, in other words, a complete
|
|
test suite.
|
|
The results of each test program are collected by the tool, and are then
|
|
multiplexed into a single machine-parseable report; see
|
|
.Xr atf-formats 5
|
|
for more details.
|
|
This report can later be transformed into many different and saner formats
|
|
using the
|
|
.Nm atf-report
|
|
tool.
|
|
.Pp
|
|
The list of test programs to execute is read from an
|
|
.Pa Atffile
|
|
present in the current directory.
|
|
This file describes the test suite stored in the directory it lives in,
|
|
which aside from the list of test programs also includes meta-data and
|
|
configuration variables.
|
|
.Pp
|
|
.Nm
|
|
is also in charge of reading the configuration files that tune the behavior
|
|
of each test program and passing down the necessary variables to them.
|
|
More details on how this is done are given in the
|
|
.Sx Configuration
|
|
section.
|
|
.Pp
|
|
In the first synopsis form,
|
|
.Nm
|
|
parses the
|
|
.Pa Atffile
|
|
in the current directory and runs all the test programs specified in it.
|
|
If any test program names are given as part of the command line, those are
|
|
the ones executed instead of the complete list.
|
|
.Pp
|
|
In the second synopsis form,
|
|
.Nm
|
|
will print information about all supported options and their purpose.
|
|
.Pp
|
|
The following options are available:
|
|
.Bl -tag -width XvXvarXvalueXX
|
|
.It Fl h
|
|
Shows a short summary of all available options and their purpose.
|
|
.It Fl v Ar var=value
|
|
Sets the configuration variable
|
|
.Ar var
|
|
to the given value
|
|
.Ar value .
|
|
.El
|
|
.Ss Configuration
|
|
.Nm
|
|
reads configuration data from multiple places.
|
|
After all of these places have been analyzed, a list of variable-value
|
|
pairs are passed to the test programs to be run.
|
|
.Pp
|
|
The following locations are scanned for configuration data, in order.
|
|
Items down the list override values defined above them:
|
|
.Bl -enum
|
|
.It
|
|
Configuration variables defined in the
|
|
.Pa Atffile .
|
|
.It
|
|
Configuration variables defined in the system-wide configuration file
|
|
shared among all test suites.
|
|
This lives in
|
|
.Pa ${ATF_CONFDIR}/common.conf .
|
|
.It
|
|
Configuration variables defined in the system-wide test-suite-specific
|
|
configuration file.
|
|
This lives in
|
|
.Pa ${ATF_CONFDIR}/<test-suite>.conf .
|
|
.It
|
|
Configuration variables defined in the user-specific configuration file
|
|
shared among all test suites.
|
|
This lives in
|
|
.Pa ${HOME}/.atf/common.conf .
|
|
.It
|
|
Configuration variables defined in the user-specific test-suite-specific
|
|
configuration file.
|
|
This lives in
|
|
.Pa ${HOME}/.atf/<test-suite>.conf .
|
|
.It
|
|
Configuration variables provided as part of the command line through the
|
|
.Fl v
|
|
option.
|
|
.El
|
|
.Pp
|
|
The value of
|
|
.Va ATF_CONFDIR
|
|
in the above list determined as detailed in
|
|
.Xr atf-config 1 .
|
|
.Pp
|
|
The following configuration variables are globally recognized:
|
|
.Bl -tag -width XunprivilegedXuserXX
|
|
.It Va unprivileged-user
|
|
The name of the system user that atf-run will drop root privileges into
|
|
for test cases defining
|
|
.Sq require.user=unprivileged .
|
|
Note that this is
|
|
.Em not provided for security purposes ;
|
|
this feature is only for the convenience of the user.
|
|
.El
|
|
.Ss Hooks
|
|
.Nm Ns 's
|
|
internal behavior can be customized by the system administrator and the
|
|
user by means of hooks.
|
|
These hooks are written in the shell script language for simplicity and
|
|
are stored in the following files, which are read in the order provided
|
|
below:
|
|
.Bl -enum
|
|
.It
|
|
${ATF_CONFDIR}/atf-run.hooks
|
|
.It
|
|
${HOME}/.atf/atf-run.hooks
|
|
.El
|
|
.Pp
|
|
The following hooks are supported:
|
|
.Bl -tag -width infoXstartXhookXX
|
|
.It info_start_hook
|
|
Called before
|
|
.Nm
|
|
executes any test program.
|
|
The purpose of this hook is to write additional
|
|
.Sq info
|
|
stanzas to the top of the output report; these are defined by the
|
|
.Sq application/X-atf-tps format
|
|
described in
|
|
.Xr atf-formats 5 .
|
|
Always use the
|
|
.Sq atf_tps_writer_info
|
|
function to print these.
|
|
.Pp
|
|
This takes no parameters.
|
|
.It info_end_hook
|
|
Similar to
|
|
.Sq info_start_hook
|
|
but executed after all test programs have been run so that additional
|
|
.Sq info
|
|
stanzas can be added to the bottom of the output report.
|
|
.Pp
|
|
This takes no parameters.
|
|
.El
|
|
.Pp
|
|
All hooks are accompanied by a function named
|
|
.Sq default_<hook_name>
|
|
that can be executed by them to invoke the default behavior built into
|
|
.Nm .
|
|
For example, in order to extend the default
|
|
.Sq info_start_hook
|
|
hook, we could write the following function:
|
|
.Bd -literal -offset indent
|
|
info_start_hook()
|
|
{
|
|
default_info_start_hook "${@}"
|
|
|
|
atf_tps_writer_info "uptime" "$(uptime)"
|
|
}
|
|
.Ed
|
|
.Sh SEE ALSO
|
|
.Xr atf-report 1 ,
|
|
.Xr atf-test-program 1 ,
|
|
.Xr atf 7
|