2012-09-04 23:07:32 +00:00
|
|
|
.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
|
|
|
|
|
.\" All rights reserved.
|
|
|
|
|
.\"
|
|
|
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
|
|
|
.\" modification, are permitted provided that the following conditions
|
|
|
|
|
.\" are met:
|
|
|
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
|
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
|
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
|
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
|
|
|
.\"
|
|
|
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
|
|
|
|
|
.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
|
|
|
|
|
.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
|
|
|
|
.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
|
|
|
|
|
.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
|
|
|
|
|
.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
|
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
|
|
|
|
|
.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
|
|
|
|
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
|
|
|
|
|
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
|
|
|
|
|
.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
|
|
|
|
|
.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
Import atf-0.21:
Released on October 23rd, 2014.
* Restored the atf(7) manual page to serve as a reference to all the other
manual pages shipped by ATF.
* Added the -s flag to atf-sh to support specifying the shell interpreter
to be used.
* Removed ATF_WORKDIR. The only remaining consumers have been converted to
use the standard TMPDIR environment variable. As a benefit, and because
Kyua forces the TMPDIR to live within the test case's work directory,
any stale files left behind by ATF will be automatically cleaned up.
* Documented the environment variables recognized by each component in the
relevant manual pages. This information was lost with the atf-config(1)
removal.
* Added a new "require.diskspace" metadata property to test cases so that
they can specify the minimum amount of disk space required for the test
to run.
* Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for
discoverability purposes. Symbolic links are provided for the time
being to still make the old names visible.
* Issue #5: Recommend the (expected, actual) idiom for calls to the test
macros in the manual pages.
* Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This
propagates the crash to the caller, which in turn allows it to obtain
proper debugging information. In particular, Kyua should now be able to
extract a stacktrace pinpointing the problem.
* Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang
that ships with FreeBSD 11.0-CURRENT.
* Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better
explaining how they relate to each other.
* Issue #14: Stopped setting 'set -e' in atf-sh. This setting was
initially added as a way to enable a "strict" mode in the library and to
make test cases fail fast when they run unprotected commands. However,
doing so in the library is surprising as the responsibility of enabling
'set -e' should be on the user's code. Also, 'set -e' introduces
inconsistent behavior on subshells and users do not expect that.
* Issue #15: Fixed atf_utils_{fork,wait} to support nested calls.
* Issue #16: Fixed test failures (by removing a long-standing hack) on
systems that lack \e support in printf(1).
* Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
|
|
|
.Dd March 2, 2014
|
2012-09-04 23:07:32 +00:00
|
|
|
.Dt ATF-TEST-PROGRAM 1
|
|
|
|
|
.Os
|
|
|
|
|
.Sh NAME
|
|
|
|
|
.Nm atf-test-program
|
|
|
|
|
.Nd common interface to ATF test programs
|
|
|
|
|
.Sh SYNOPSIS
|
|
|
|
|
.Nm
|
|
|
|
|
.Op Fl r Ar resfile
|
|
|
|
|
.Op Fl s Ar srcdir
|
|
|
|
|
.Op Fl v Ar var1=value1 Op .. Fl v Ar varN=valueN
|
|
|
|
|
.Ar test_case
|
|
|
|
|
.Nm
|
|
|
|
|
.Fl l
|
|
|
|
|
.Sh DESCRIPTION
|
|
|
|
|
Test programs written using the ATF libraries all share a common user
|
|
|
|
|
interface, which is what this manual page describes.
|
|
|
|
|
.Em NOTE: There is no binary known as
|
|
|
|
|
.Nm ;
|
|
|
|
|
.Em what is described in this manual page is the command-line interface
|
|
|
|
|
.Em exposed by the atf-c, atf-c++ and atf-sh bindings .
|
|
|
|
|
.Pp
|
|
|
|
|
In the first synopsis form, the test program will execute the provided
|
|
|
|
|
test case and print its results to the standard output, unless otherwise
|
|
|
|
|
stated by the
|
|
|
|
|
.Fl r
|
|
|
|
|
flag.
|
|
|
|
|
Optionally, the test case name can be suffixed by
|
|
|
|
|
.Sq :cleanup ,
|
|
|
|
|
in which case the cleanup routine of the test case will be executed
|
|
|
|
|
instead of the test case body; see
|
|
|
|
|
.Xr atf-test-case 4 .
|
|
|
|
|
Note that the test case is
|
|
|
|
|
.Em executed without isolation ,
|
|
|
|
|
so it can and probably will create and modify files in the current directory.
|
Import atf-0.21:
Released on October 23rd, 2014.
* Restored the atf(7) manual page to serve as a reference to all the other
manual pages shipped by ATF.
* Added the -s flag to atf-sh to support specifying the shell interpreter
to be used.
* Removed ATF_WORKDIR. The only remaining consumers have been converted to
use the standard TMPDIR environment variable. As a benefit, and because
Kyua forces the TMPDIR to live within the test case's work directory,
any stale files left behind by ATF will be automatically cleaned up.
* Documented the environment variables recognized by each component in the
relevant manual pages. This information was lost with the atf-config(1)
removal.
* Added a new "require.diskspace" metadata property to test cases so that
they can specify the minimum amount of disk space required for the test
to run.
* Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for
discoverability purposes. Symbolic links are provided for the time
being to still make the old names visible.
* Issue #5: Recommend the (expected, actual) idiom for calls to the test
macros in the manual pages.
* Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This
propagates the crash to the caller, which in turn allows it to obtain
proper debugging information. In particular, Kyua should now be able to
extract a stacktrace pinpointing the problem.
* Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang
that ships with FreeBSD 11.0-CURRENT.
* Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better
explaining how they relate to each other.
* Issue #14: Stopped setting 'set -e' in atf-sh. This setting was
initially added as a way to enable a "strict" mode in the library and to
make test cases fail fast when they run unprotected commands. However,
doing so in the library is surprising as the responsibility of enabling
'set -e' should be on the user's code. Also, 'set -e' introduces
inconsistent behavior on subshells and users do not expect that.
* Issue #15: Fixed atf_utils_{fork,wait} to support nested calls.
* Issue #16: Fixed test failures (by removing a long-standing hack) on
systems that lack \e support in printf(1).
* Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
|
|
|
To execute test cases in a controller manner, you need a runtime engine
|
|
|
|
|
that understands the ATF interface.
|
|
|
|
|
The recommended runtime engine is
|
|
|
|
|
.Xr kyua 1 .
|
2012-09-04 23:07:32 +00:00
|
|
|
You should only execute test cases by hand for debugging purposes.
|
|
|
|
|
.Pp
|
|
|
|
|
In the second synopsis form, the test program will list all available
|
|
|
|
|
test cases alongside their meta-data properties in a format that is
|
|
|
|
|
machine parseable.
|
|
|
|
|
This list is processed by
|
Import atf-0.21:
Released on October 23rd, 2014.
* Restored the atf(7) manual page to serve as a reference to all the other
manual pages shipped by ATF.
* Added the -s flag to atf-sh to support specifying the shell interpreter
to be used.
* Removed ATF_WORKDIR. The only remaining consumers have been converted to
use the standard TMPDIR environment variable. As a benefit, and because
Kyua forces the TMPDIR to live within the test case's work directory,
any stale files left behind by ATF will be automatically cleaned up.
* Documented the environment variables recognized by each component in the
relevant manual pages. This information was lost with the atf-config(1)
removal.
* Added a new "require.diskspace" metadata property to test cases so that
they can specify the minimum amount of disk space required for the test
to run.
* Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for
discoverability purposes. Symbolic links are provided for the time
being to still make the old names visible.
* Issue #5: Recommend the (expected, actual) idiom for calls to the test
macros in the manual pages.
* Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This
propagates the crash to the caller, which in turn allows it to obtain
proper debugging information. In particular, Kyua should now be able to
extract a stacktrace pinpointing the problem.
* Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang
that ships with FreeBSD 11.0-CURRENT.
* Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better
explaining how they relate to each other.
* Issue #14: Stopped setting 'set -e' in atf-sh. This setting was
initially added as a way to enable a "strict" mode in the library and to
make test cases fail fast when they run unprotected commands. However,
doing so in the library is surprising as the responsibility of enabling
'set -e' should be on the user's code. Also, 'set -e' introduces
inconsistent behavior on subshells and users do not expect that.
* Issue #15: Fixed atf_utils_{fork,wait} to support nested calls.
* Issue #16: Fixed test failures (by removing a long-standing hack) on
systems that lack \e support in printf(1).
* Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
|
|
|
.Xr kyua 1
|
2012-09-04 23:07:32 +00:00
|
|
|
to know how to execute the test cases of a given test program.
|
|
|
|
|
.Pp
|
|
|
|
|
The following options are available:
|
|
|
|
|
.Bl -tag -width XvXvarXvalueXX
|
|
|
|
|
.It Fl l
|
|
|
|
|
Lists available test cases alongside a brief description for each of them.
|
|
|
|
|
.It Fl r Ar resfile
|
|
|
|
|
Specifies the file that will receive the test case result.
|
|
|
|
|
If not specified, the test case prints its results to stdout.
|
|
|
|
|
If the result of a test case needs to be parsed by another program, you must
|
|
|
|
|
use this option to redirect the result to a file and then read the resulting
|
|
|
|
|
file from the other program.
|
|
|
|
|
Note:
|
|
|
|
|
.Em do not try to process the stdout of the test case
|
|
|
|
|
because your program may break in the future.
|
|
|
|
|
.It Fl s Ar srcdir
|
|
|
|
|
The path to the directory where the test program is located.
|
|
|
|
|
This is needed in all cases, except when the test program is being executed
|
|
|
|
|
from the current directory.
|
|
|
|
|
The test program will use this path to locate any helper data files or
|
|
|
|
|
utilities.
|
|
|
|
|
.It Fl v Ar var=value
|
|
|
|
|
Sets the configuration variable
|
|
|
|
|
.Ar var
|
|
|
|
|
to the value
|
|
|
|
|
.Ar value .
|
|
|
|
|
.El
|
|
|
|
|
.Sh SEE ALSO
|
Import atf-0.21:
Released on October 23rd, 2014.
* Restored the atf(7) manual page to serve as a reference to all the other
manual pages shipped by ATF.
* Added the -s flag to atf-sh to support specifying the shell interpreter
to be used.
* Removed ATF_WORKDIR. The only remaining consumers have been converted to
use the standard TMPDIR environment variable. As a benefit, and because
Kyua forces the TMPDIR to live within the test case's work directory,
any stale files left behind by ATF will be automatically cleaned up.
* Documented the environment variables recognized by each component in the
relevant manual pages. This information was lost with the atf-config(1)
removal.
* Added a new "require.diskspace" metadata property to test cases so that
they can specify the minimum amount of disk space required for the test
to run.
* Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for
discoverability purposes. Symbolic links are provided for the time
being to still make the old names visible.
* Issue #5: Recommend the (expected, actual) idiom for calls to the test
macros in the manual pages.
* Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This
propagates the crash to the caller, which in turn allows it to obtain
proper debugging information. In particular, Kyua should now be able to
extract a stacktrace pinpointing the problem.
* Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang
that ships with FreeBSD 11.0-CURRENT.
* Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better
explaining how they relate to each other.
* Issue #14: Stopped setting 'set -e' in atf-sh. This setting was
initially added as a way to enable a "strict" mode in the library and to
make test cases fail fast when they run unprotected commands. However,
doing so in the library is surprising as the responsibility of enabling
'set -e' should be on the user's code. Also, 'set -e' introduces
inconsistent behavior on subshells and users do not expect that.
* Issue #15: Fixed atf_utils_{fork,wait} to support nested calls.
* Issue #16: Fixed test failures (by removing a long-standing hack) on
systems that lack \e support in printf(1).
* Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
|
|
|
.Xr kyua 1
|
