a817576754
Experimental version released on February 7th, 2014. This is the last release to bundle the code for the deprecated tools. The next release will drop their code and will stop worrying about backwards compatibility between the ATF libraries and what the old tools may or may not support. If you still require the old tools for some reason, grab a copy of the 'tools' directory now. The code in this directory is standalone and does not depend on any internal details of atf-c++ any longer. * Various fixes and improvements to support running as part of the FreeBSD test suite. * Project hosting moved from Google Code (as a subproject of Kyua) to GitHub (as a first-class project). The main reason for the change is the suppression of binary downloads in Google Code on Jan 15th, 2014. See https://github.com/jmmv/atf/ * Removed builtin help from atf-sh(1) and atf-check(1) for simplicity reasons. In other words, their -h option is gone. * Moved the code of the deprecated tools into a 'tools' directory and completely decoupled their code from the internals of atf-c++. The reason for this is to painlessly allow a third-party to maintain a copy of these tools after we delete them because upcoming changes to atf-c++ would break the stale tools.
193 lines
5.6 KiB
Groff
193 lines
5.6 KiB
Groff
.\"
|
|
.\" Automated Testing Framework (atf)
|
|
.\"
|
|
.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
.\" modification, are permitted provided that the following conditions
|
|
.\" are met:
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
|
|
.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
|
|
.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
|
.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
|
|
.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
|
|
.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
|
|
.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
|
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
|
|
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
|
|
.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
|
|
.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
.\"
|
|
.Dd August 28, 2010
|
|
.Dt ATF 7
|
|
.Os
|
|
.Sh NAME
|
|
.Nm ATF
|
|
.Nd introduction to the Automated Testing Framework
|
|
.Sh DESCRIPTION
|
|
.Em IMPORTANT: If you are here because you want to know how to run the tests in
|
|
.Pa __TESTSDIR__ ,
|
|
.Em you most likely want to read the
|
|
.Xr tests 7
|
|
.Em manual page instead.
|
|
.Pp
|
|
The Automated Testing Framework
|
|
.Pf ( Nm )
|
|
is a collection of libraries and utilities designed to ease unattended
|
|
application testing in the hands of developers and end users of a specific
|
|
piece of software.
|
|
.Pp
|
|
As regards developers,
|
|
.Nm
|
|
provides the necessary means to easily create
|
|
test suites composed of multiple test programs, which in turn are a
|
|
collection of test cases.
|
|
It also attempts to simplify the debugging of problems when these test
|
|
cases detect an error by providing as much information as possible
|
|
about the failure.
|
|
.Pp
|
|
As regards users, it simplifies the process of running the test suites and,
|
|
in special, encourages end users to run them often: they do not need to
|
|
have source trees around nor any other development tools installed to be
|
|
able to certify that a given piece of software works on their machine as
|
|
advertised.
|
|
.Pp
|
|
If your operating systems distributes
|
|
.Nm ,
|
|
it is possible that it provides an introductory
|
|
.Xr tests 7
|
|
manual page.
|
|
You are encouraged to read it now.
|
|
.Ss License
|
|
.Nm
|
|
is distributed under the terms of the TNF License, a 2-clause BSD license.
|
|
For more details please see:
|
|
.Bd -literal -offset indent
|
|
.Pa __DOCDIR__/COPYING
|
|
.Ed
|
|
.Ss Components
|
|
.Nm
|
|
is a highly modular piece of software.
|
|
It provides a couple of libraries to ease the implementation of test
|
|
programs: one for the C and C++ languages and another one for shell
|
|
scripts.
|
|
It also includes multiple small utilities that follow the principle of
|
|
doing a single thing but doing it right.
|
|
This section outlines which these components are.
|
|
.Pp
|
|
Public utilities:
|
|
.Bl -tag -width atfXtestXprogramXXXXX
|
|
.It Xr atf-check 1
|
|
Executes a command and checks that its exit code, its standard output
|
|
and its standard error output match pre-specified expected values.
|
|
.It Xr atf-config 1
|
|
Queries static configuration information.
|
|
.It Xr atf-report 1
|
|
Converts the output of
|
|
.Nm atf-run
|
|
to user-friendly and/or machine-parseable reports.
|
|
.It Xr atf-run 1
|
|
Automates the execution of a series of test programs and collects their
|
|
results in a unified report.
|
|
.It Xr atf-sh 1
|
|
Shell interpreter for shell-based test programs.
|
|
.El
|
|
.Pp
|
|
Programming interfaces:
|
|
.Bl -tag -width atfXtestXprogramXXXXX
|
|
.It Xr atf-c-api 3
|
|
C programming interface for test programs.
|
|
.It Xr atf-c++-api 3
|
|
C++ programming interface for test programs.
|
|
.It Xr atf-sh-api 3
|
|
POSIX shell programming interface for test programs.
|
|
.El
|
|
.Pp
|
|
Other:
|
|
.Bl -tag -width atfXtestXprogramXXXXX
|
|
.It Xr atf-formats 5
|
|
Description of the machine-parseable data formats used by the tools.
|
|
.It Xr atf-test-case 4
|
|
Generic description of test cases, independent of the language they are
|
|
implemented in.
|
|
.It Xr atf-test-program 1
|
|
Common interface provided by the test programs written using the
|
|
.Nm
|
|
libraries.
|
|
.El
|
|
.Ss Recommended reading order
|
|
For end users wishing to run tests:
|
|
.Bl -enum -compact
|
|
.It
|
|
.Xr tests 7
|
|
(only if provided by your operating system).
|
|
.It
|
|
.Xr atf-test-program 1
|
|
.It
|
|
.Xr atf-run 1
|
|
.It
|
|
.Xr atf-report 1
|
|
.It
|
|
.Xr atf-config 1
|
|
.El
|
|
.Pp
|
|
For developers wanting to write their own tests:
|
|
.Bl -enum -compact
|
|
.It
|
|
Everything recommended to users.
|
|
.It
|
|
.Xr atf-test-case 4
|
|
.It
|
|
.Xr atf-c-api 3
|
|
.It
|
|
.Xr atf-c++-api 3
|
|
.It
|
|
.Xr atf-sh-api 3
|
|
.It
|
|
.Xr atf-sh 1
|
|
.It
|
|
.Xr atf-check 1
|
|
.El
|
|
.Pp
|
|
For those interested in
|
|
.Nm
|
|
internals:
|
|
.Bl -enum -compact
|
|
.It
|
|
Everything recommended to users.
|
|
.It
|
|
Everything recommended to developers.
|
|
.It
|
|
.Xr atf-formats 5
|
|
.El
|
|
.Sh SEE ALSO
|
|
.Xr tests 7
|
|
.Sh HISTORY
|
|
.Nm
|
|
started as a Google Summer of Code 2007 project mentored by The NetBSD
|
|
Foundation.
|
|
Its original goal was to provide a testing framework for The NetBSD
|
|
Operating System, but it grew as an independent project because the
|
|
framework itself did not need to be tied to a specific operating system.
|
|
.Pp
|
|
For more details on this subject, please see:
|
|
.Bd -literal -offset indent
|
|
.Pa __DOCDIR__/NEWS
|
|
.Pa __DOCDIR__/ROADMAP
|
|
.Ed
|
|
.Sh AUTHORS
|
|
For more details on the people that made
|
|
.Nm
|
|
possible, please see:
|
|
.Bd -literal -offset indent
|
|
.Pa __DOCDIR__/AUTHORS
|
|
.Ed
|