freebsd-skq/contrib/atf/atf-sh/integration_test.sh

161 lines
4.0 KiB
Bash
Raw Normal View History

# Copyright (c) 2010 The NetBSD Foundation, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
# CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
# INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
# GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
# IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Import atf-0.21: Released on October 23rd, 2014. * Restored the atf(7) manual page to serve as a reference to all the other manual pages shipped by ATF. * Added the -s flag to atf-sh to support specifying the shell interpreter to be used. * Removed ATF_WORKDIR. The only remaining consumers have been converted to use the standard TMPDIR environment variable. As a benefit, and because Kyua forces the TMPDIR to live within the test case's work directory, any stale files left behind by ATF will be automatically cleaned up. * Documented the environment variables recognized by each component in the relevant manual pages. This information was lost with the atf-config(1) removal. * Added a new "require.diskspace" metadata property to test cases so that they can specify the minimum amount of disk space required for the test to run. * Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for discoverability purposes. Symbolic links are provided for the time being to still make the old names visible. * Issue #5: Recommend the (expected, actual) idiom for calls to the test macros in the manual pages. * Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This propagates the crash to the caller, which in turn allows it to obtain proper debugging information. In particular, Kyua should now be able to extract a stacktrace pinpointing the problem. * Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang that ships with FreeBSD 11.0-CURRENT. * Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better explaining how they relate to each other. * Issue #14: Stopped setting 'set -e' in atf-sh. This setting was initially added as a way to enable a "strict" mode in the library and to make test cases fail fast when they run unprotected commands. However, doing so in the library is surprising as the responsibility of enabling 'set -e' should be on the user's code. Also, 'set -e' introduces inconsistent behavior on subshells and users do not expect that. * Issue #15: Fixed atf_utils_{fork,wait} to support nested calls. * Issue #16: Fixed test failures (by removing a long-standing hack) on systems that lack \e support in printf(1). * Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
: ${ATF_SH:="__ATF_SH__"}
create_test_program() {
Import atf-0.21: Released on October 23rd, 2014. * Restored the atf(7) manual page to serve as a reference to all the other manual pages shipped by ATF. * Added the -s flag to atf-sh to support specifying the shell interpreter to be used. * Removed ATF_WORKDIR. The only remaining consumers have been converted to use the standard TMPDIR environment variable. As a benefit, and because Kyua forces the TMPDIR to live within the test case's work directory, any stale files left behind by ATF will be automatically cleaned up. * Documented the environment variables recognized by each component in the relevant manual pages. This information was lost with the atf-config(1) removal. * Added a new "require.diskspace" metadata property to test cases so that they can specify the minimum amount of disk space required for the test to run. * Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for discoverability purposes. Symbolic links are provided for the time being to still make the old names visible. * Issue #5: Recommend the (expected, actual) idiom for calls to the test macros in the manual pages. * Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This propagates the crash to the caller, which in turn allows it to obtain proper debugging information. In particular, Kyua should now be able to extract a stacktrace pinpointing the problem. * Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang that ships with FreeBSD 11.0-CURRENT. * Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better explaining how they relate to each other. * Issue #14: Stopped setting 'set -e' in atf-sh. This setting was initially added as a way to enable a "strict" mode in the library and to make test cases fail fast when they run unprotected commands. However, doing so in the library is surprising as the responsibility of enabling 'set -e' should be on the user's code. Also, 'set -e' introduces inconsistent behavior on subshells and users do not expect that. * Issue #15: Fixed atf_utils_{fork,wait} to support nested calls. * Issue #16: Fixed test failures (by removing a long-standing hack) on systems that lack \e support in printf(1). * Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
local output="${1}"; shift
echo "#! ${ATF_SH} ${*}" >"${output}"
cat >>"${output}"
chmod +x "${output}"
}
atf_test_case no_args
no_args_body()
{
cat >experr <<EOF
atf-sh: ERROR: No test program provided
atf-sh: See atf-sh(1) for usage details.
EOF
Import atf-0.21: Released on October 23rd, 2014. * Restored the atf(7) manual page to serve as a reference to all the other manual pages shipped by ATF. * Added the -s flag to atf-sh to support specifying the shell interpreter to be used. * Removed ATF_WORKDIR. The only remaining consumers have been converted to use the standard TMPDIR environment variable. As a benefit, and because Kyua forces the TMPDIR to live within the test case's work directory, any stale files left behind by ATF will be automatically cleaned up. * Documented the environment variables recognized by each component in the relevant manual pages. This information was lost with the atf-config(1) removal. * Added a new "require.diskspace" metadata property to test cases so that they can specify the minimum amount of disk space required for the test to run. * Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for discoverability purposes. Symbolic links are provided for the time being to still make the old names visible. * Issue #5: Recommend the (expected, actual) idiom for calls to the test macros in the manual pages. * Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This propagates the crash to the caller, which in turn allows it to obtain proper debugging information. In particular, Kyua should now be able to extract a stacktrace pinpointing the problem. * Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang that ships with FreeBSD 11.0-CURRENT. * Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better explaining how they relate to each other. * Issue #14: Stopped setting 'set -e' in atf-sh. This setting was initially added as a way to enable a "strict" mode in the library and to make test cases fail fast when they run unprotected commands. However, doing so in the library is surprising as the responsibility of enabling 'set -e' should be on the user's code. Also, 'set -e' introduces inconsistent behavior on subshells and users do not expect that. * Issue #15: Fixed atf_utils_{fork,wait} to support nested calls. * Issue #16: Fixed test failures (by removing a long-standing hack) on systems that lack \e support in printf(1). * Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
atf_check -s eq:1 -o ignore -e file:experr "${ATF_SH}"
}
atf_test_case missing_script
missing_script_body()
{
cat >experr <<EOF
atf-sh: ERROR: The test program 'non-existent' does not exist
EOF
Import atf-0.21: Released on October 23rd, 2014. * Restored the atf(7) manual page to serve as a reference to all the other manual pages shipped by ATF. * Added the -s flag to atf-sh to support specifying the shell interpreter to be used. * Removed ATF_WORKDIR. The only remaining consumers have been converted to use the standard TMPDIR environment variable. As a benefit, and because Kyua forces the TMPDIR to live within the test case's work directory, any stale files left behind by ATF will be automatically cleaned up. * Documented the environment variables recognized by each component in the relevant manual pages. This information was lost with the atf-config(1) removal. * Added a new "require.diskspace" metadata property to test cases so that they can specify the minimum amount of disk space required for the test to run. * Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for discoverability purposes. Symbolic links are provided for the time being to still make the old names visible. * Issue #5: Recommend the (expected, actual) idiom for calls to the test macros in the manual pages. * Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This propagates the crash to the caller, which in turn allows it to obtain proper debugging information. In particular, Kyua should now be able to extract a stacktrace pinpointing the problem. * Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang that ships with FreeBSD 11.0-CURRENT. * Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better explaining how they relate to each other. * Issue #14: Stopped setting 'set -e' in atf-sh. This setting was initially added as a way to enable a "strict" mode in the library and to make test cases fail fast when they run unprotected commands. However, doing so in the library is surprising as the responsibility of enabling 'set -e' should be on the user's code. Also, 'set -e' introduces inconsistent behavior on subshells and users do not expect that. * Issue #15: Fixed atf_utils_{fork,wait} to support nested calls. * Issue #16: Fixed test failures (by removing a long-standing hack) on systems that lack \e support in printf(1). * Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
atf_check -s eq:1 -o ignore -e file:experr "${ATF_SH}" non-existent
}
atf_test_case arguments
arguments_body()
{
create_test_program tp <<EOF
main() {
echo ">>>\${0}<<<"
while test \${#} -gt 0; do
echo ">>>\${1}<<<"
shift
done
true
}
EOF
cat >expout <<EOF
>>>./tp<<<
>>> a b <<<
>>>foo<<<
EOF
atf_check -s eq:0 -o file:expout -e empty ./tp ' a b ' foo
cat >expout <<EOF
>>>tp<<<
>>> hello bye <<<
>>>foo bar<<<
EOF
Import atf-0.21: Released on October 23rd, 2014. * Restored the atf(7) manual page to serve as a reference to all the other manual pages shipped by ATF. * Added the -s flag to atf-sh to support specifying the shell interpreter to be used. * Removed ATF_WORKDIR. The only remaining consumers have been converted to use the standard TMPDIR environment variable. As a benefit, and because Kyua forces the TMPDIR to live within the test case's work directory, any stale files left behind by ATF will be automatically cleaned up. * Documented the environment variables recognized by each component in the relevant manual pages. This information was lost with the atf-config(1) removal. * Added a new "require.diskspace" metadata property to test cases so that they can specify the minimum amount of disk space required for the test to run. * Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for discoverability purposes. Symbolic links are provided for the time being to still make the old names visible. * Issue #5: Recommend the (expected, actual) idiom for calls to the test macros in the manual pages. * Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This propagates the crash to the caller, which in turn allows it to obtain proper debugging information. In particular, Kyua should now be able to extract a stacktrace pinpointing the problem. * Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang that ships with FreeBSD 11.0-CURRENT. * Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better explaining how they relate to each other. * Issue #14: Stopped setting 'set -e' in atf-sh. This setting was initially added as a way to enable a "strict" mode in the library and to make test cases fail fast when they run unprotected commands. However, doing so in the library is surprising as the responsibility of enabling 'set -e' should be on the user's code. Also, 'set -e' introduces inconsistent behavior on subshells and users do not expect that. * Issue #15: Fixed atf_utils_{fork,wait} to support nested calls. * Issue #16: Fixed test failures (by removing a long-standing hack) on systems that lack \e support in printf(1). * Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
atf_check -s eq:0 -o file:expout -e empty "${ATF_SH}" tp \
' hello bye ' 'foo bar'
}
atf_test_case custom_shell__command_line
custom_shell__command_line_body()
{
cat >expout <<EOF
This is the custom shell
This is the test program
EOF
cat >custom-shell <<EOF
#! /bin/sh
echo "This is the custom shell"
exec /bin/sh "\${@}"
EOF
chmod +x custom-shell
echo 'main() { echo "This is the test program"; }' | create_test_program tp
atf_check -s eq:0 -o file:expout -e empty "${ATF_SH}" -s ./custom-shell tp
}
atf_test_case custom_shell__shebang
custom_shell__shebang_body()
{
cat >expout <<EOF
This is the custom shell
This is the test program
EOF
cat >custom-shell <<EOF
#! /bin/sh
echo "This is the custom shell"
exec /bin/sh "\${@}"
EOF
chmod +x custom-shell
echo 'main() { echo "This is the test program"; }' | create_test_program \
tp "-s$(pwd)/custom-shell"
atf_check -s eq:0 -o file:expout -e empty ./tp
}
atf_test_case set_e
set_e_head()
{
atf_set "descr" "Simple test to validate that atf-sh works even when" \
"set -e is enabled"
}
set_e_body()
{
cat >custom-shell <<EOF
#! /bin/sh
exec /bin/sh -e "\${@}"
EOF
chmod +x custom-shell
cat >tp <<EOF
atf_test_case helper
helper_body() {
atf_skip "reached"
}
atf_init_test_cases() {
atf_add_test_case helper
}
EOF
atf_check -s eq:0 -o match:skipped.*reached \
"${ATF_SH}" -s ./custom-shell tp helper
}
atf_init_test_cases()
{
atf_add_test_case no_args
atf_add_test_case missing_script
atf_add_test_case arguments
Import atf-0.21: Released on October 23rd, 2014. * Restored the atf(7) manual page to serve as a reference to all the other manual pages shipped by ATF. * Added the -s flag to atf-sh to support specifying the shell interpreter to be used. * Removed ATF_WORKDIR. The only remaining consumers have been converted to use the standard TMPDIR environment variable. As a benefit, and because Kyua forces the TMPDIR to live within the test case's work directory, any stale files left behind by ATF will be automatically cleaned up. * Documented the environment variables recognized by each component in the relevant manual pages. This information was lost with the atf-config(1) removal. * Added a new "require.diskspace" metadata property to test cases so that they can specify the minimum amount of disk space required for the test to run. * Renamed the atf-{c,c++,sh}-api(3) manual pages to atf-{c,c++,sh}(3) for discoverability purposes. Symbolic links are provided for the time being to still make the old names visible. * Issue #5: Recommend the (expected, actual) idiom for calls to the test macros in the manual pages. * Issue #7: Stopped catching unhandled exceptions in atf-c++ tests. This propagates the crash to the caller, which in turn allows it to obtain proper debugging information. In particular, Kyua should now be able to extract a stacktrace pinpointing the problem. * Issue #8: Fixed atf-c/macros_test:use test failures spotted by the clang that ships with FreeBSD 11.0-CURRENT. * Issue #12: Improved documentation of atf-sh(3) and atf-check(1) by better explaining how they relate to each other. * Issue #14: Stopped setting 'set -e' in atf-sh. This setting was initially added as a way to enable a "strict" mode in the library and to make test cases fail fast when they run unprotected commands. However, doing so in the library is surprising as the responsibility of enabling 'set -e' should be on the user's code. Also, 'set -e' introduces inconsistent behavior on subshells and users do not expect that. * Issue #15: Fixed atf_utils_{fork,wait} to support nested calls. * Issue #16: Fixed test failures (by removing a long-standing hack) on systems that lack \e support in printf(1). * Issue #19: Removed stale references to atf-config and atf-run.
2014-10-30 17:14:07 +00:00
atf_add_test_case custom_shell__command_line
atf_add_test_case custom_shell__shebang
atf_add_test_case set_e
}
# vim: syntax=sh:expandtab:shiftwidth=4:softtabstop=4