freebsd-dev/crypto/openssh/regress
Ed Maste 2f513db72b Upgrade to OpenSSH 7.9p1.
MFC after:	2 months
Sponsored by:	The FreeBSD Foundation
2020-02-14 19:06:59 +00:00
..
misc Upgrade to OpenSSH 7.9p1. 2020-02-14 19:06:59 +00:00
unittests Upgrade to OpenSSH 7.9p1. 2020-02-14 19:06:59 +00:00
addrmatch.sh
agent-getpeereid.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
agent-pkcs11.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
agent-ptrace.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
agent-timeout.sh
agent.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
allow-deny-users.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
authinfo.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
banner.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
broken-pipe.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
brokenkeys.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
cert-file.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
cert-hostkey.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
cert-userkey.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
cfginclude.sh Upgrade to OpenSSH 7.3p1. 2017-03-02 00:11:32 +00:00
cfgmatch.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
cfgmatchlisten.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
cfgparse.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
check-perm.c Upgrade to OpenSSH 7.2p2. 2016-03-11 00:15:29 +00:00
cipher-speed.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
conch-ciphers.sh
connect-privsep.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
connect-uri.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
connect.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
dhgex.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
dsa_ssh2.prv
dsa_ssh2.pub
dynamic-forward.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
envpass.sh
exit-status.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
forcecommand.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
forward-control.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
forwarding.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
host-expand.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
hostkey-agent.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
hostkey-rotate.sh Upgrade to OpenSSH 7.2p2. 2016-03-11 00:15:29 +00:00
integrity.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
kextype.sh Upgrade to OpenSSH 6.9p1. 2016-01-19 18:55:44 +00:00
key-options.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
keygen-change.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
keygen-convert.sh
keygen-knownhosts.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
keygen-moduli.sh Upgrade to OpenSSH 7.4p1. 2017-03-06 01:37:05 +00:00
keys-command.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
keyscan.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
keytype.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
krl.sh Upgrade to OpenSSH 7.9p1. 2020-02-14 19:06:59 +00:00
limit-keytype.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
localcommand.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
login-timeout.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
Makefile Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
mkdtemp.c Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
modpipe.c Upgrade to OpenSSH 7.3p1. 2017-03-02 00:11:32 +00:00
moduli.in Upgrade to OpenSSH 7.4p1. 2017-03-06 01:37:05 +00:00
multiplex.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
multipubkey.sh
netcat.c Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
portnum.sh
principals-command.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
proto-mismatch.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
proto-version.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
proxy-connect.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
putty-ciphers.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
putty-kex.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
putty-transfer.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
README.regress Upgrade to OpenSSH 7.9p1. 2020-02-14 19:06:59 +00:00
reconfigure.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
reexec.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
rekey.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
rsa_openssh.prv
rsa_openssh.pub
rsa_ssh2.prv
scp-ssh-wrapper.sh
scp-uri.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
scp.sh
setuid-allowed.c Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
sftp-badcmds.sh
sftp-batch.sh
sftp-chroot.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
sftp-cmds.sh
sftp-glob.sh
sftp-perm.sh
sftp-uri.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
sftp.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
ssh2putty.sh Upgrade to OpenSSH 6.9p1. 2016-01-19 18:55:44 +00:00
ssh-com-client.sh
ssh-com-keygen.sh
ssh-com-sftp.sh
ssh-com.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
sshcfgparse.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
sshd-log-wrapper.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
stderr-after-eof.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
stderr-data.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
t4.ok
t5.ok
t11.ok
test-exec.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
transfer.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
try-ciphers.sh Upgrade to OpenSSH 7.6p1. This will be followed shortly by 7.7p1. 2018-05-08 23:13:11 +00:00
valgrind-unit.sh Upgrade to OpenSSH 7.8p1. 2018-09-10 16:20:12 +00:00
yes-head.sh Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00

Overview.

$ ./configure && make tests

You'll see some progress info. A failure will cause either the make to
abort or the driver script to report a "FATAL" failure.

The test consists of 2 parts. The first is the file-based tests which is
driven by the Makefile, and the second is a set of network or proxycommand
based tests, which are driven by a driver script (test-exec.sh) which is
called multiple times by the Makefile.

Failures in the first part will cause the Makefile to return an error.
Failures in the second part will print a "FATAL" message for the failed
test and continue.

OpenBSD has a system-wide regression test suite. OpenSSH Portable's test
suite is based on OpenBSD's with modifications.


Environment variables.

SKIP_UNIT: Skip unit tests.
SUDO: path to sudo/doas command, if desired. Note that some systems
	(notably systems using PAM) require sudo to execute some tests.
LTESTS: Whitespace separated list of tests (filenames without the .sh
	extension) to run.
OBJ: used by test scripts to access build dir.
TEST_SHELL: shell used for running the test scripts.
TEST_SSH_FAIL_FATAL: set to "yes" to make any failure abort the test
	currently in progress.
TEST_SSH_PORT: TCP port to be used for the listening tests.
TEST_SSH_QUIET: set to "yes" to suppress non-fatal output.
TEST_SSH_SSHD_CONFOPTS: Configuration directives to be added to sshd_config
	before running each test.
TEST_SSH_SSH_CONFOPTS: Configuration directives to be added to
	ssh_config before running each test.
TEST_SSH_TRACE: set to "yes" for verbose output from tests 
TEST_SSH_x: path to "ssh" command under test, where x is one of
	SSH, SSHD, SSHAGENT, SSHADD, SSHKEYGEN, SSHKEYSCAN, SFTP or
	SFTPSERVER
USE_VALGRIND: Run the tests under valgrind memory checker.


Individual tests.

You can run an individual test from the top-level Makefile, eg:
$ make tests LTESTS=agent-timeout

If you need to manipulate the environment more you can invoke test-exec.sh
directly if you set up the path to find the binaries under test and the
test scripts themselves, for example:

$ cd regress
$ PATH=`pwd`/..:$PATH:. TEST_SHELL=/bin/sh sh test-exec.sh `pwd` \
    agent-timeout.sh
ok agent timeout test


Files.

test-exec.sh: the main test driver. Sets environment, creates config files
and keys and runs the specified test.

At the time of writing, the individual tests are:
agent-timeout.sh:	agent timeout test
agent.sh:		simple agent test
broken-pipe.sh:		broken pipe test
connect-privsep.sh:	proxy connect with privsep
connect.sh:		simple connect
exit-status.sh:		remote exit status
forwarding.sh:		local and remote forwarding
keygen-change.sh:	change passphrase for key
keyscan.sh:		keyscan
proto-mismatch.sh:	protocol version mismatch
proto-version.sh:	sshd version with different protocol combinations
proxy-connect.sh:	proxy connect
sftp.sh:		basic sftp put/get
ssh-com-client.sh:	connect with ssh.com client
ssh-com-keygen.sh:	ssh.com key import
ssh-com-sftp.sh:	basic sftp put/get with ssh.com server
ssh-com.sh:		connect to ssh.com server
stderr-after-eof.sh:	stderr data after eof
stderr-data.sh:		stderr data transfer
transfer.sh:		transfer data
try-ciphers.sh:		try ciphers
yes-head.sh:		yes pipe head


Problems?

Run the failing test with shell tracing (-x) turned on:
$ PATH=`pwd`/..:$PATH:. sh -x test-exec.sh `pwd` agent-timeout.sh

Failed tests can be difficult to diagnose. Suggestions:
- run the individual test via ./test-exec.sh `pwd` [testname]
- set LogLevel to VERBOSE in test-exec.sh and enable syslogging of
  auth.debug (eg to /var/log/authlog).


Known Issues.

- Similarly, if you do not have "scp" in your system's $PATH then the
  multiplex scp tests will fail (since the system's shell startup scripts
  will determine where the shell started by sshd will look for scp).

- Recent GNU coreutils deprecate "head -[n]": this will cause the yes-head
  test to fail.  The old behaviour can be restored by setting (and
  exporting) _POSIX2_VERSION=199209 before running the tests.