numam-spdk/test/pmem/test_plan.md

PMEM bdev feature test plan

Objective

The purpose of these tests is to verify possibility of using pmem bdev configuration in SPDK by running functional tests FIO traffic verification tests.

Configuration

Configuration in tests is to be done using example stub application (spdk/example/bdev/io/bdev_io). All possible management is done using RPC calls with the exception of use of split bdevs which have to be configured in .conf file.

Functional tests are executed as scenarios - sets of smaller test steps in which results and return codes of RPC calls are validated. Some configuration calls may also additionally be validated by use of "get" (e.g. get_bdevs) RPC calls, which provide additional information for veryfing results. In some steps additional write/read operations will be performed on PMEM bdevs in order to check IO path correct behavior.

FIO traffic verification tests will serve as integration tests and will be executed to config correct behavior of PMEM bdev when working with vhost, nvmf_tgt and iscsi_tgt applications.

Functional tests

pmem_pool_info

pmem_pool_info_tc1

Negative test for checking pmem pool file. Call with missing path argument. Steps & expected results:

• Call pmem_pool_info with missing path argument
• Check that return code != 0 and error code =

pmem_pool_info_tc2

Negative test for checking pmem pool file. Call with non-existant path argument. Steps & expected results:

• Call pmem_pool_info with path argument that points to not existing file.
• Check that return code != 0 and error code = ENODEV

pmem_pool_info_tc3

Negative test for checking pmem pool file. Call with other type of pmem pool file. Steps & expected results:

• Using pmem utility tools create pool of OBJ type instead of BLK (if needed utility tools are not available - create random file in filesystem)
• Call pmem_pool_info and point to file created in previous step.
• Check that return code != 0 and error code = ENODEV

pmem_pool_info_tc4

Positive test for checking pmem pool file. Call with existing pmem pool file. Steps & expected results:

• Call pmem_pool_info with path argument that points to existing file.
• Check that return code == 0

create_pmem_pool

From libpmemblk documentation:

• PMEM block size has to be bigger than 512 internal blocks; if lower value is used then PMEM library will silently round it up to 512 which is defined in nvml/libpmemblk.h file as PMEMBLK_MIN_BLK.
• Total pool size cannot be less than 16MB which is defined i nvml/libpmemblk.h file as PMEMBLK_MIN_POOL
• Total number of segments in PMEP pool file cannot be less than 256

create_pmem_pool_tc1

Negative test case for creating a new pmem. Call create_pmem_pool with missing arguments. Steps & expected results:

• call create_pmem_pool without path argument
• call return code != 0
• call pmem_pool_info and check that pmem pool file was not created
• call return code != 0
• call create_pmem_pool with path but without size and block size arguments
• call return code != 0
• call pmem_pool_info and check that pmem pool file was not created
• call return code != 0
• call create_pmem_pool with path and size but without block size arguments
• call return code != 0
• call pmem_pool_info and check that pmem pool file was not created
• call return code != 0

create_pmem_pool_tc2

Negative test case for creating a new pmem. Call create_pmem_pool with non existing path argument. Steps & expected results:

• call create_pmem_pool with path that does not exist
• call return code != 0
• call pmem_pool_info and check that pmem pool file was not created
• call return code != 0

create_pmem_pool_tc3

Positive test case for creating a new pmem pool on disk space. Steps & expected results:

• call create_pmem_pool with correct path argument, blocksize=512 and total size=256MB
• call return code = 0
• call pmem_pool_info and check that pmem file was created
• call return code = 0
• call delete_pmem_pool on previously created pmem
• return code = 0 and no error code

create_pmem_pool_tc4

Positive test case for creating a new pmem pool in RAM space.

TODO: Research test steps for creating a pool in RAM!!!

Steps & expected results:

• call create_pmem_pool with correct path argument, blocksize=512 and total size=256MB
• call return code = 0
• call pmem_pool_info and check that pmem file was created
• call return code = 0
• call delete_pmem_pool on previously created pmem
• return code = 0 and no error code

create_pmem_pool_tc5

Negative test case for creating two pmems with same path. Steps & expected results:

• call create_pmem_pool with correct path argument, blocksize=512 and total size=256MB
• call return code = 0
• call pmem_pool_info and check that pmem file was created
• call return code = 0
• call create_pmem_pool with the same path argument as before, blocksize=4096 and total size=512MB
• call return code != 0, error code = EEXIST
• call create_pmem_pool and check that first pmem pool file is still available and not modified (block size and total size stay the same)
• call return code = 0
• call delete_pmem_pool on first created pmem pool
• return code =0 and no error code

create_pmem_pool_tc6

Positive test case for creating pmem pool file with various block sizes. Steps & expected results:

• call create_pmem_pool with correct path argument, total size=256MB with different block size arguments - 1, 511, 512, 513, 1024, 4096, 128k and 256k
• call pmem_pool_info on each of created pmem pool and check if it was created; For pool files created with block size <512 their block size should be rounded up to 512; other pool files should have the same block size as specified in create command
• call return code = 0; block sizes as expected
• call delete_pmem_pool on all created pool files

create_pmem_pool_tc7

Negative test case for creating pmem pool file with total size of less than 16MB. Steps & expected results:

• call create_pmem_pool with correct path argument, block size=512 and total size less than 16MB
• return code !=0 and error code !=0
• call pmem_pool_info to verify pmem pool file was not created
• return code = 0

create_pmem_pool_tc8

Negative test case for creating pmem pool file with less than 256 blocks. Steps & expected results:

• call create_pmem_pool with correct path argument, block size=128k and total size=30MB
• return code !=0 and error code !=0
• call pmem_pool_info to verify pmem pool file was not created
• return code = 0

delete_pmem_pool

delete_pmem_pool_tc1

Negative test case for deleting a pmem. Call delete_pmem_pool on non-exisiting pmem. Steps & expected results:

• call delete_pmem_pool on non-existing pmem.
• return code !=0 and error code = ENOENT

delete_pmem_pool_tc2

Negative test case for deleting a pmem. Call delete_pmem_pool on a file of wrong type Steps & expected results:

• Using pmem utility tools create pool of OBJ type instead of BLK (if needed utility tools are not available - create random file in filesystem)
• Call delete_pmem_pool and point to file created in previous step.
• return code !=0 and error code = ENOTBLK

delete_pmem_pool_tc3

Positive test case for creating and deleting a pemem. Steps & expected results:

• call create_pmem_pool with correct arguments
• return code = 0 and no error code
• using pmem_pool_info check that pmem was created
• return code = 0 and no error code
• call delete_pmem_pool on previously created pmem
• return code = 0 and no error code
• using pmem_pool_info check that pmem no longer exists
• return code !=0 and error code = ENODEV

delete_pmem_pool_tc4

Negative test case for creating and deleting a pemem. Steps & expected results:

• run scenario from test case 3
• call delete_pmem_pool on already deleted pmem pool
• return code !=0 and error code = ENODEV

construct_pmem_bdev

construct_pmem_bdev_tc1

Negative test for constructing new pmem bdev. Call create_pmem_bdev with missing argument. Steps & expected results:

• Call construct_pmem_bdev with missing path argument.
• Check that return code != 0

construct_pmem_bdev_tc2

Negative test for constructing new pmem bdev. Call construct_pmem_bdev with not existing path argument. Steps & expected results:

• call construct_pmem_bdev with incorrect (not existing) path
• call return code != 0 and error code = ENODEV
• using get_bdevs check that no pmem bdev was created

construct_pmem_bdev_tc3

Negative test for constructing pmem bdevs with random file instead of pmemblk pool. Steps & expected results:

• using a system tool (like dd) create a random file
• call construct_pmem_bdev with path pointing to that file
• return code != 0, error code = ENOTBLK

construct_pmem_bdev_tc4

Negative test for constructing pmem bdevs with pmemobj instead of pmemblk pool. Steps & expected results:

• Using pmem utility tools create pool of OBJ type instead of BLK (if needed utility tools are not available - create random file in filesystem)
• call construct_pmem_bdev with path pointing to that pool
• return code != 0, error code = ENOTBLK

construct_pmem_bdev_tc5

Positive test for constructing pmem bdev. Steps & expected results:

• call create_pmem_pool with correct arguments
• return code = 0, no errors
• call pmem_pool_info and check if pmem files exists
• return code = 0, no errors
• call construct_pmem_bdev with with correct arguments to create a pmem bdev
• return code = 0, no errors
• using get_bdevs check that pmem bdev was created
• delete pmem bdev using delete_bdev
• return code = 0, no error code
• delete previously created pmem pool
• return code = 0, no error code

construct_pmem_bdev_tc6

Negative test for constructing pmem bdevs twice on the same pmem. Steps & expected results:

• call create_pmem_pool with correct arguments
• return code = 0, no errors
• call pmem_pool_info and check if pmem files exists
• return code = 0, no errors
• call construct_pmem_bdev with with correct arguments to create a pmem bdev
• return code = 0, no errors
• using get_bdevs check that pmem bdev was created
• call construct_pmem_bdev again on the same pmem file
• return code != 0, error code = EEXIST
• delete pmem bdev using delete_bdev
• return code = 0, no error code
• delete previously created pmem pool
• return code = 0, no error code

delete_bdev

delete_bdev_tc1

Positive test for deleting pmem bdevs using common delete_bdev call. Steps & expected results:

• construct malloc and aio bdevs (also NVMe if possible)
• all calls - return code = 0, no errors; bdevs created
• call create_pmem_pool with correct path argument, block size=512, total size=256M
• return code = 0, no errors
• call pmem_pool_info and check if pmem file exists
• return code = 0, no errors
• call construct_pmem_bdev and create a pmem bdev
• return code = 0, no errors
• using get_bdevs check that pmem bdev was created
• delete pmem bdev using delete_bdev
• return code = 0, no errors
• using get_bdevs confirm that pmem bdev was deleted and other bdevs were unaffected.

delete_bdev_tc2

Negative test for deleting pmem bdev twice. Steps & expected results:

• call create_pmem_pool with correct path argument, block size=512, total size=256M
• return code = 0, no errors
• call pmem_pool_info and check if pmem file exists
• return code = 0, no errors
• call construct_pmem_bdev and create a pmem bdev
• return code = 0, no errors
• using get_bdevs check that pmem bdev was created
• delete pmem bdev using delete_bdev
• return code = 0, no errors
• using get_bdevs confirm that pmem bdev was deleted
• delete pmem bdev using delete_bdev second time
• return code != 0, error code = ENODEV

Integration tests

Description of integration tests which run FIO verification traffic against pmem_bdevs used in vhost, iscsi_tgt and nvmf_tgt applications can be found in test directories for these components:

• spdk/test/vhost
• spdk/test/nvmf
• spdk/test/iscsi_tgt