Add readme and script for fedora 26 vm configuration
This readme and bash script can be used by developers to quickly deploy a new vm in which to pretest their spdk patches. Change-Id: I12f805e925dae9120ea8c451c9c5b07df4ffaf4b Signed-off-by: Seth Howell <seth.howell@intel.com> Reviewed-on: https://review.gerrithub.io/387673 Tested-by: SPDK Automated Test System <sys_sgsw@intel.com> Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com> Reviewed-by: Jim Harris <james.r.harris@intel.com>
This commit is contained in:
Seth Howell
Jim Harris
parent
c427d7bf92
commit
7e4a7b62e1
79
test/config/README.md
Normal file
79
test/config/README.md
Normal file
@ -0,0 +1,79 @@
|
||||
# Virtual Test Configuration
|
||||
|
||||
This readme and the associated bash script, vm_setup.sh, are intended to assist developers in quickly
|
||||
preparing a virtual test environment on which to run the SPDK validation tests rooted at autorun.sh.
|
||||
This file contains basic information about SPDK environment requirements, an introduction to the
|
||||
autorun-spdk.conf files used to moderate which tests are run by autorun.sh, and step-by-step instructions
|
||||
for spinning up a VM capable of running the SPDK test suite.
|
||||
There is no need for external hardware to run these tests. The linux kernel comes with the drivers necessary
|
||||
to emulate an RDMA enabled NIC. NVMe controllers can also be virtualized in emulators such as QEMU.
|
||||
|
||||
|
||||
## VM Envronment Requirements (Host):
|
||||
- 8 GiB of RAM (for DPDK)
|
||||
- Enable intel_kvm on the host machine from the bios.
|
||||
- Enable nesting for VMs in kernel command line (for vhost tests).
|
||||
- In `/etc/default/grub` append the following to the GRUB_CMDLINE_LINUX line: intel_iommu=on kvm-intel.nested=1.
|
||||
|
||||
## VM Specs
|
||||
When creating the user during the fedora installation, it is best to use the name sys_sgsw. Efforts are being made
|
||||
to remove all references to this user, or files specific to this user from the codebase, but there are still some
|
||||
trailing references to it.
|
||||
|
||||
## Autorun-spdk.conf
|
||||
Every machine that runs the autotest scripts should include a file titled autorun-spdk.conf in the home directory
|
||||
of the user that will run them. This file consists of several lines of the form 'variable_name=0/1'. autorun.sh sources
|
||||
this file each time it is run, and determines which tests to attempt based on which variables are defined in the
|
||||
configuration file. For a full list of the variable declarations available for autorun-spdk.conf, please see
|
||||
`scripts/autotest_common.sh` starting at line 13.
|
||||
|
||||
## Steps for Configuring the VM
|
||||
1. Download a fresh Fedora 26 image.
|
||||
2. Perform the installation of Fedora 26 server.
|
||||
3. Create an admin user sys_sgsw (enabling passwordless sudo for this account will make life easier during the tests).
|
||||
4. Run the vm_setup.sh script which will install all proper dependencies.
|
||||
5. Modify the autorun-spdk.conf file in the home directory.
|
||||
6. Reboot the VM.
|
||||
7. Run autorun.sh for SPDK. Any output files will be placed in `~/spdk_repo/output/`.
|
||||
|
||||
## Additional Steps for Preparing the Vhost Tests
|
||||
The Vhost tests also require the creation of a second virtual machine nested inside of the test VM.
|
||||
Please follow the directions below to complete that installation. Note that host refers to the Fedora VM
|
||||
created above and guest or VM refer to the Ubuntu VM created in this section.
|
||||
1. Create an image file for the VM. It does not have to be large, about 3.5G should suffice.
|
||||
2. Create an ssh keypair for host-guest communications (performed on the host):
|
||||
- Generate an ssh keypair with the name spdk_vhost_id_rsa and save it in `/root/.ssh`.
|
||||
- Make sure that only root has read access to the private key.
|
||||
3. Install the OS in the VM image (performed on guest):
|
||||
- Use the latest Ubuntu server (Currently 16.04 LTS).
|
||||
- When partitioning the disk, make one partion that consumes the whole disk mounted at /. Do not encrypt the disk or enable LVM.
|
||||
- Choose the OpenSSH server packages during install.
|
||||
4. Post installation configuration (performed on guest):
|
||||
- Run the following commands to enable all necessary dependencies:
|
||||
~~~{.sh}
|
||||
sudo apt update
|
||||
sudo apt upgrade
|
||||
sudo apt install fio sg3-utils bc
|
||||
~~~
|
||||
- Enable the root user: "sudo passwd root -> root".
|
||||
- Enable root login over ssh: vim `/etc/ssh/sshd_config` -> PermitRootLogin=yes.
|
||||
- Disable DNS for ssh: `/etc/ssh/sshd_config` -> UseDNS=no.
|
||||
- Add the spdk_vhost key to root's known hosts: `/root/.ssh/authorized_keys` -> add spdk_vhost_id_rsa.pub key to authorized keys.
|
||||
Remember to save the private key in `~/.ssh/spdk_vhost_id_rsa` on the host.
|
||||
- Change the grub boot options for the guest as follows:
|
||||
- Add "console=ttyS0 earlyprintk=ttyS0" to the boot options in `/etc/default/grub` (for serial output redirect).
|
||||
- Add "scsi_mod.use_blk_mq=1" to boot options in `/etc/default/grub`.
|
||||
~~~{.sh}
|
||||
sudo update-grub
|
||||
~~~
|
||||
- Reboot the VM.
|
||||
- Remove any unnecessary packages (this is to make booting the VM faster):
|
||||
~~~{.sh}
|
||||
apt purge snapd
|
||||
apt purge Ubuntu-core-launcher
|
||||
apt purge squashfs-tools
|
||||
apt purge unattended-upgrades
|
||||
~~~
|
||||
5. Copy the fio binary from the guest location `/usr/bin/fio` to the host location `/home/sys_sgsw/fio_ubuntu`.
|
||||
6. Place the guest VM in the host at the following location: `/home/sys_sgsw/vhost_vm_image.qcow2`.
|
||||
7. On the host, edit the `~/autorun-spdk.conf` file to include the following line: SPDK_TEST_VHOST=1.
|
||||
168
test/config/vm_setup.sh
Normal file
168
test/config/vm_setup.sh
Normal file
@ -0,0 +1,168 @@
|
||||
#! /usr/bin/env bash
|
||||
|
||||
# Virtual Machine environment requirements:
|
||||
# 8 GiB of RAM (for DPDK)
|
||||
# enable intel_kvm on your host machine
|
||||
|
||||
# The purpose of this script is to provide a simple procedure for spinning up a new
|
||||
# virtual test environment capable of running our whole test suite. This script, when
|
||||
# applied to a fresh install of fedora 26 server will install all of the necessary dependencies
|
||||
# to run almost the complete test suite. The main exception being VHost. Vhost requires the
|
||||
# configuration of a second virtual machine. instructions for how to configure
|
||||
# that vm are included in the file TEST_ENV_SETUP_README inside this repository
|
||||
|
||||
# it is important to enable nesting for vms in kernel command line of your machine for the vhost tests.
|
||||
# in /etc/default/grub
|
||||
# append the following to the GRUB_CMDLINE_LINUX line
|
||||
# intel_iommu=on kvm-intel.nested=1
|
||||
|
||||
# We have made a lot of progress with removing hardcoded paths from the tests,
|
||||
# but it may be easiest if you create your user with the name sys_sgsw for now.
|
||||
|
||||
set -e
|
||||
|
||||
sudo dnf upgrade -y
|
||||
sudo dnf install -y git
|
||||
sudo dnf install -y jq
|
||||
sudo dnf install -y valgrind
|
||||
sudo dnf install -y nvme-cli
|
||||
sudo dnf install -y ceph
|
||||
sudo dnf install -y gdb
|
||||
sudo dnf install -y sg3_utils
|
||||
sudo dnf install -y fio
|
||||
sudo dnf install -y librbd-devel
|
||||
sudo dnf install -y kernel-devel
|
||||
sudo dnf install -y gflags-devel
|
||||
sudo dnf install -y libubsan
|
||||
sudo dnf install -y autoconf
|
||||
sudo dnf install -y automake
|
||||
sudo dnf install -y libtool
|
||||
sudo dnf install -y libmount-devel
|
||||
sudo dnf install -y isns-utils-devel
|
||||
sudo dnf install -y openssl-devel
|
||||
sudo dnf install -y numactl-devel
|
||||
sudo dnf install -y libaio-devel
|
||||
sudo dnf install -y CUnit-devel
|
||||
sudo dnf install -y clang-analyzer
|
||||
sudo dnf install -y libpmemblk-devel
|
||||
sudo dnf install -y libibverbs libibverbs-devel librdmacm librdmacm-devel
|
||||
sudo dnf install -y perl-open
|
||||
sudo dnf install -y glib2-devel
|
||||
sudo dnf install -y pixman-devel
|
||||
sudo dnf install -y libiscsi-devel
|
||||
sudo dnf install -y doxygen
|
||||
|
||||
cd ~
|
||||
|
||||
mkdir spdk_repo
|
||||
|
||||
# the configurations of username and email are needed later for applying patches to iscsiadm.
|
||||
git config --global user.name none
|
||||
git config --global user.email none
|
||||
|
||||
# The librxe-dev repository provides a command line tool called rxe_cfg which makes it
|
||||
# very easy to use Soft-RoCE. The build pool utilizes this command line tool in the absence
|
||||
# of any real RDMA NICs to simulate one for the NVMe-oF tests.
|
||||
git clone https://github.com/SoftRoCE/librxe-dev.git
|
||||
sudo dnf install -y perl-Switch librdmacm-utils libibverbs-utils
|
||||
|
||||
cd librxe-dev
|
||||
./configure --libdir=/usr/lib64/ --prefix=
|
||||
make
|
||||
sudo make install
|
||||
cd ~
|
||||
|
||||
cd spdk_repo
|
||||
mkdir output
|
||||
git clone https://review.gerrithub.io/spdk/spdk
|
||||
cd spdk
|
||||
git submodule update --init --recursive
|
||||
cd ~
|
||||
|
||||
# The version of iscsiadm that ships with fedora 26 was broken as of November 3 2017.
|
||||
# There is already a bug report out about it, and hopefully it is fixed soon, but in the event that
|
||||
# that version is still broken when you do your setup, the below steps will fix the issue.
|
||||
CURRENT_VERSION=$(iscsiadm --version)
|
||||
OPEN_ISCSI_VER='iscsiadm version 6.2.0.874'
|
||||
if [ "$CURRENT_VERSION" == "$OPEN_ISCSI_VER" ]; then
|
||||
mkdir open-iscsi-install
|
||||
cd open-iscsi-install
|
||||
sudo dnf download --source iscsi-initiator-utils
|
||||
rpm2cpio iscsi-initiator-utils-6.2.0.874-3.git86e8892.fc26.src.rpm | cpio -idmv
|
||||
mkdir patches
|
||||
mv 00* patches/
|
||||
git clone https://github.com/open-iscsi/open-iscsi
|
||||
|
||||
cd open-iscsi
|
||||
git checkout 86e8892
|
||||
for patch in `ls ../patches`; do
|
||||
git am ../patches/$patch
|
||||
done
|
||||
sed -i '427s/.*/-1);/' usr/session_info.c
|
||||
make
|
||||
sudo make install
|
||||
cd ~
|
||||
fi
|
||||
|
||||
|
||||
sudo mkdir -p /usr/src
|
||||
|
||||
# Rocksdb is installed for use with the blobfs tests.
|
||||
git clone https://review.gerrithub.io/spdk/rocksdb
|
||||
git -C ./rocksdb checkout spdk-v5.6.1
|
||||
sudo mv rocksdb /usr/src/
|
||||
|
||||
git clone http://git.kernel.dk/fio.git
|
||||
sudo mv fio /usr/src/
|
||||
(
|
||||
cd /usr/src/fio &&
|
||||
make &&
|
||||
sudo make install
|
||||
)
|
||||
cd ~
|
||||
|
||||
git clone https://github.com/brendangregg/FlameGraph.git
|
||||
mkdir -p /usr/local
|
||||
sudo mv FlameGraph /usr/local/FlameGraph
|
||||
|
||||
|
||||
# The SPDK branch of Qemu contains some extra functionality needed by SPDK. However, that branch is not currently
|
||||
# compatible with GCC 7. Some of the SPDK changes are currently out for review on Qemu's main branch. Until those
|
||||
# changes are merged, this specific review provides support for both spdk and GCC 7.
|
||||
mkdir vhost
|
||||
cd vhost
|
||||
git clone https://review.gerrithub.io/spdk/qemu
|
||||
cd qemu
|
||||
./configure --prefix=/home/sys_sgsw/spdk_repo/root --target-list="x86_64-softmmu" --enable-kvm --enable-linux-aio --enable-numa
|
||||
sudo make install
|
||||
cd ~
|
||||
|
||||
# create autorun-spdk.conf in home folder. This is sourced by the autotest_common.sh file.
|
||||
# By setting any one of the values below to 0, you can skip that specific test. If you are
|
||||
# using your autotest platform to do sanity checks before uploading to the build pool, it is
|
||||
# probably best to only run the tests that you believe your changes have modified along with
|
||||
# Scanbuild and check format. This is because running the whole suite of tests in series can
|
||||
# take ~40 minutes to complete.
|
||||
cat > ~/autorun-spdk.conf << EOF
|
||||
# assign a value of 1 to all of the pertinent tests
|
||||
SPDK_BUILD_DOC=1
|
||||
SPDK_BUILD_IOAT_KMOD=1
|
||||
SPDK_RUN_CHECK_FORMAT=1
|
||||
SPDK_RUN_SCANBUILD=1
|
||||
SPDK_RUN_VALGRIND=1
|
||||
SPDK_TEST_UNITTEST=1
|
||||
SPDK_TEST_ISCSI=1
|
||||
SPDK_TEST_NVME=1
|
||||
SPDK_TEST_NVMF=1
|
||||
SPDK_TEST_RBD=1
|
||||
# requires some extra configuration. see TEST_ENV_SETUP_README
|
||||
SPDK_TEST_VHOST=0
|
||||
SPDK_TEST_BLOCKDEV=1
|
||||
# doesn't work on vm
|
||||
SPDK_TEST_IOAT=0
|
||||
SPDK_TEST_EVENT=1
|
||||
SPDK_TEST_BLOBFS=1
|
||||
SPDK_TEST_NVML=1
|
||||
SPDK_RUN_ASAN=1
|
||||
SPDK_RUN_UBSAN=1
|
||||
EOF
|
||||
Loading…
x
Reference in New Issue
Block a user