The tests are failing because the return value and output have changed, but
before test code structure adjusted, removing these test cases help people
be able to focus on more important cases.
Discussed with: emaste
MFC with: r348206
Sponsored by: The FreeBSD Foundation
Final cleanup routines shouldn't be called from testcases; it should be called
from the testcase cleanup routine.
Furthermore, `geli_test_cleanup` should take care of cleaning up geli providers
and the memory disks used for the geli providers. `geli_test_cleanup` will always
be executed whereas the equivalent logic in `geli_test_body`, may not have been
executed if the test failed prior to the logic being run.
Prior to this change, the test case was trying to clean up `$md` twice: once in
at the end of the test case body function, and the other in the cleanup function.
The cleanup function logic was failing because there wasn't anything to clean up
in the cleanup function and the errors weren't being ignored.
This fixes FreeBSD test suite runs after r345864.
PR: 237128
Reviewed by: asomers, pjd
Approved by: emaste (mentor)
MFC with: r345864
Differential Revision: https://reviews.freebsd.org/D19854
- init, init -R
- onetime, onetime -R
- 512 and 4k sectors
- encryption only
- encryption and authentication
- configure -r/-R for detached providers
- configure -r/-R for attached providers
- all keys allocated (10, 20 and 30MB provider sizes)
- keys allocated on demand (10, 20 and 30PB provider sizes)
- reading and writing to provider after expansion (10-30MB only)
- checking if metadata in old location is cleared.
Obtained from: Fudo Security
Re-apply r341665 with format strings fixed.
If we happen to taste a stale mirror component first, don't reject valid,
newer components that have differing metadata from the stale component
(during STARTING). Instead, update our view of the most recent metadata as
we taste components.
Like mediasize beforehand, remove some checks from g_mirror_check_metadata
which would evict valid components due to metadata that can change over a
mirror's lifetime. g_mirror_check_metadata is invoked long before we check
genid/syncid and decide which component(s) are newest and whether or not we
have quorum.
Before checking if we can enter RUNNING (i.e., we have quorum) after a NEW
component is added, first remove any known stale or inconsistent disks from
the mirrorset, rather than removing them *after* deciding we have quorum.
Check if we have quorum after removing these components.
Additionally, add a knob, kern.geom.mirror.launch_mirror_before_timeout, to
force gmirrors to wait out the full timeout (kern.geom.mirror.timeout)
before transitioning from STARTING to RUNNING. This is a kludge to help
ensure all eligible, boot-time available mirror components are tasted before
RUNNING a gmirror.
Add a basic test case for STARTING -> RUNNING startup behavior around stale
genids.
PR: 232671, 232835
Submitted by: Cindy Yang <cyang AT isilon.com> (previous version)
Reviewed by: markj (kernel portions)
Discussed with: asomers, Cindy Yang
Tested by: pho
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D18062
r341392 changed common test cleanup routines in a way that allowed them to
be used by TAP tests as well as ATF tests. However, a late change made
during code review resulted in cleanup being broken for ATF tests, which
source geom_subr.sh separately during the body and cleanup phases of the
test. The result was that md(4) devices wouldn't get cleaned up.
MFC after: 2 weeks
X-MFC-With: 341392
If we happen to taste a stale mirror component first, don't reject valid,
newer components that have differing metadata from the stale component
(during STARTING). Instead, update our view of the most recent metadata as
we taste components.
Like mediasize beforehand, remove some checks from g_mirror_check_metadata
which would evict valid components due to metadata that can change over a
mirror's lifetime. g_mirror_check_metadata is invoked long before we check
genid/syncid and decide which component(s) are newest and whether or not we
have quorum.
Before checking if we can enter RUNNING (i.e., we have quorum) after a NEW
component is added, first remove any known stale or inconsistent disks from
the mirrorset, rather than removing them *after* deciding we have quorum.
Check if we have quorum after removing these components.
Additionally, add a knob, kern.geom.mirror.launch_mirror_before_timeout, to
force gmirrors to wait out the full timeout (kern.geom.mirror.timeout)
before transitioning from STARTING to RUNNING. This is a kludge to help
ensure all eligible, boot-time available mirror components are tasted before
RUNNING a gmirror.
When we are instructed to forget mirror components, bump the generation id
to avoid confusion with such stale components later.
Add a basic test case for STARTING -> RUNNING startup behavior around stale
genids.
PR: 232671, 232835
Submitted by: Cindy Yang <cyang AT isilon.com> (previous version)
Reviewed by: markj (kernel portions)
Discussed with: asomers, Cindy Yang
Tested by: pho
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D18062
The problem with the logic prior to this commit was twofold:
1. The wrong set of idioms (TAP-compatible) were being applied to the ATF
testcases when run, resulting in confusing ATF failure results on setup.
2. The cleanup subroutines were broken when the geom classes could not be
loaded as they exited with 0 unexpectedly.
This commit changes the test code to source the class-specific configuration
(conf.sh) once globally, instead of sourcing it per testcase and per cleanup
subroutine, and to call the ATF-specific setup subroutine(s) inline in
the testcases.
The refactoring done is effectively a no-op for the TAP testcases, modulo
any refactoring done to create common code between the ATF and TAP
testcases.
This unbreaks the geli testcases converted to ATF in r327662 and r327683,
and the gmirror testcases added in r327780, respectively, when the geom
class could not be loaded.
tests/sys/geom/class/mirror/...
While here, ignore errors when turning debug failpoint sysctl off, which
could occur if the gmirror class was not loaded.
Submitted by: ngie
MFC after: 2 weeks
Pull Request: https://github.com/freebsd/freebsd/pull/241
If the underlying provider's physical path is null, then the gpart device's
physical path will be, too. Otherwise, it will append the partition name,
such as "/p1" or "/s1/a". This will make gpart work better with zfsd(8).
PR: 224965
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D14010
If the underlying provider's physical path is null, then the geli device's
physical path will be, too. Otherwise, it will append "/eli". This will make
geli work better with zfsd(8).
PR: 224962
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D13979
Reduce the geli tests' runtime by about a third:
* In integrity_test:copy, use a file-backed md(4) device instead of a
malloc'd one. That way we can corrupt the underlying storage without
needing to detach and reattach the geli device.
* In integrity_test:{copy, hmac, data} and onetime_test:{onetime,
onetime_a}, move reads of /dev/random out of the loop.
MFC after: 2 weeks
The trick is not to destroy an md(4) device during a test. That can create
a "double-free" situation, because we also destroy md devices during test
cleanup.
MFC after: 2 weeks
I'm leaving readonly_test and nokey_test alone for now. In a future commit
they should be broken up into several smaller test cases and distributed
between multiple files.
Reviewed by: ngie
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D13717
The resize test used bsdlabel(8), which is not available on all
architectures. Change it to use gpart(8) instead, which should be available
everywhere.
PR: 221763
Reported by: andrew
MFC after: 2 weeks
This change mostly reverts r293436, which introduced the bug due to a belief
that geli(8) would allocate md(4) devices by itself. However, that belief is
incorrect. Instead of using linear probing to find available md(4) numbers,
it's best to use the existing attach_md function.
Reviewed by: ngie
MFC after: 2 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13666
^/head@r323923 changed when MODIFIED is printed at exit. It's better to follow the
documented way of determining whether or not a filesystem is clean per fsck_ffs, i.e.,
ensure that the exit code is either 0 or 7.
The pass/fail determination is brittle prior to this commit, and ^/head@r323923 made
the issue apparent -- thus this needs to be fixed independent of ^/head@r323923.
PR: 222780
MFC after: 1 week
MFC with: r323923
Reported by: Jenkins
Non-tests/... changes:
- Add HAS_TESTS= to Makefiles with libraries and programs to enable iteration
and propagate the appropriate environment down to *.test.mk.
tests/... changes:
- Add appropriate support Makefile.inc's to set HAS_TESTS in a minimal manner,
since tests/... is a special subdirectory tree compared to the others.
MFC after: 2 months
MFC with: r322511
Reviewed by: arch (silence), testing (silence)
Differential Revision: D12014
Executable bits should be set at install time instead of in the repo.
Setting executable bits on files triggers false positives with Phabricator.
MFC after: 2 months
tests/... is a special snowflake directory and using HAS_TESTS would
result in a nasty layering violation between bsd.tests.mk and
bsd.prog.mk.
Add reachover Makefile.inc's which get the default value from
Makefile.inc0 (inspired by gnu/usr.bin/binutils/Makefile.inc0).
The test code prior to r311893 loaded geom_gate at test start if necessary and
skipped the tests if it couldn't be loaded.
The ATF-ifcation of this test done in r311893 unfortunately dropped this
functionality.
This change restores the geom_gate module load and skips the test(s) if unavailable
in an ATF-like way.
MFC after: 1 month
PR: 220164
Reported by: gjb
This change moves the tests added in r313962 to an existing directory
structure used by the geli TAP tests. It also, renames the test from
pbkdf2 to pbkdf2_test .
The changes to ObsoleteFiles.inc are being committed separately as they
aren't needed for the MFC to ^/stable/11, etc, if the MFC for the tests
is done all in one commit.
MFC after: 2 weeks
X-MFC with: r313962, r313972-r313973
Reviewed by: allanjude
Sponsored by: Dell EMC Isilon
Differential Revision: D9985
committed in r313972
The code committed in r313962 implicitly relies on python 2.x to generate
testvect.h . There are a handful of issues with this approach:
- python is not an explicit build dependency for FreeBSD
- python 2.x is deprecated and will be removed sometime in the future
(potentially before 11.x's EOL), and the script does not function with
python 3.5 (it uses deprecated idioms and incompatible function calls).
- python(1) (by default) lives in /usr/local/bin (${LOCALBASE}/bin) and
gentestvect.py is a dependency of testvect.h (prior to r313972) which
means that if the mtime of the generator script was newer than the
mtime of the test vector, it could cause a spurious build failure in
build time or at install time.
A better solution using C/C++ should be devised.
Discussed with: allanjude
MFC after: 2 weeks
X-MFC with: r313962, r313972
Sponsored by: Dell EMC Isilon
Using relative paths imply working directory (in this case .OBJDIR), whereas the
sources live in the .CURDIR-relative path.
MFC after: 2 weeks
X-MFC with: r313962
Pointyhat to: allanjude
Sponsored by: Dell EMC Isilon
The PBKDF2 in sys/geom/eli/pkcs5v2.c is around half the speed it could be
GELI's PBKDF2 uses a simple benchmark to determine a number of iterations
that will takes approximately 2 seconds. The security provided is actually
half what is expected, because an attacker could use the optimized
algorithm to brute force the key in half the expected time.
With this change, all newly generated GELI keys will be approximately 2x
as strong. Previously generated keys will talk half as long to calculate,
resulting in faster mounting of encrypted volumes. Users may choose to
rekey, to generate a new key with the larger default number of iterations
using the geli(8) setkey command.
Security of existing data is not compromised, as ~1 second per brute force
attempt is still a very high threshold.
PR: 202365
Original Research: https://jbp.io/2015/08/11/pbkdf2-performance-matters/
Submitted by: Joe Pixton <jpixton@gmail.com> (Original Version), jmg (Later Version)
Reviewed by: ed, pjd, delphij
Approved by: secteam, pjd (maintainer)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D8236
results of commands
As noted in r313008, the underlying issue was that geom_gate device
creation wasn't created at ggatel command completion, but some short
time after. ggatec(8) employs similar logic when creating geom_gate(4)
devices.
Switch from retry loops (after the ggatec/dd write calls) to
wait_for_ggate_device function calls after calling ggatec(8) instead
to detect the presence of the /dev/ggate* device, as this function is
sufficient for determining whether or not the character device is ready
for testing
While here, use atf_check consistently with all dd calls to ensure that
data output is as expected.
MFC after: 1 week
Reviewed by: asomers
Differential Revision: D9409
Sponsored by: Dell EMC Isilon
The test assumed that `ggatel create` created a device on completion, but that's
incorrect. This squashes the race by waiting for the device to appear, as
`ggatel create` daemonizes before issuing an ioctl to geom_gate(4) if not called
with `-v`.
Discussed with: asomers
MFC after: 1 week
PR: 204616
Sponsored by: Dell EMC Isilon
This ensures their cleanup routines will be run even if they should timeout.
tests/sys/geom/class/gate/ggate_test.sh
tests/sys/geom/class/gate/Makefile
Add an ATF test with three testcases, one for each TAP test. Use
ATF-style cleanup functions, and convert sleeps to polling loops.
ObsoleteFiles.inc
tests/sys/geom/class/gate/conf.sh
tests/sys/geom/class/gate/1_test.sh
tests/sys/geom/class/gate/2_test.sh
tests/sys/geom/class/gate/3_test.sh
Delete TAP test files
Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D8891
Also, add test cases for the -p, -P, and -s options to gnop create
Reviewed by: ngie
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D8892
There is no reduction in test coverage. On my system runtime is reduced
from 38m32s to 6m24s.
tests/sys/geom/class/eli/conf.sh
tests/sys/geom/class/eli/init_a_test.sh
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/integrity_copy_test.sh
tests/sys/geom/class/eli/integrity_data_test.sh
tests/sys/geom/class/eli/integrity_hmac_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
tests/sys/geom/class/eli/onetime_test.sh
Move the looping code into common functions in conf.sh, and remove
alias ciphers from the list.
tests/sys/geom/class/eli/init_a_test.sh
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/integrity_copy_test.sh
tests/sys/geom/class/eli/integrity_data_test.sh
tests/sys/geom/class/eli/integrity_hmac_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
Move a few commands that don't need to be in the inner loop out.
tests/sys/geom/class/eli/init_test.sh
tests/sys/geom/class/eli/onetime_a_test.sh
Reduce the sector count
tests/sys/geom/class/eli/Makefile
tests/sys/geom/class/eli/init_alias_test.sh
Add a test for initializing a GELI device using one of the cipher
aliases, and check that the alias is correctly interpreted.
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D8814
have both big and little-endian images in tree).
Also we don't known the endianness of the platform the
image was generated on.
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
