doc: improve Linux guide layout
Fixed Linux Getting Started Guide rst layout to improve rendering in PDF. Signed-off-by: John McNamara <john.mcnamara@intel.com>
This commit is contained in:
John McNamara
Thomas Monjalon
parent
cacb49c55d
commit
29c673401c
@ -28,14 +28,13 @@
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
.. _linux_gsg_compiling_dpdk:
|
||||
|
||||
Compiling the DPDK Target from Source
|
||||
=====================================
|
||||
|
||||
.. note::
|
||||
|
||||
Parts of this process can also be done using the setup script described in Chapter 6 of this document.
|
||||
Parts of this process can also be done using the setup script described in
|
||||
the :ref:`linux_setup_script` section of this document.
|
||||
|
||||
Install the DPDK and Browse Sources
|
||||
-----------------------------------
|
||||
@ -44,10 +43,12 @@ First, uncompress the archive and move to the uncompressed DPDK source directory
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
user@host:~$ unzip DPDK-<version>.zip
|
||||
user@host:~$ cd DPDK-<version>
|
||||
user@host:~/DPDK-<version>$ ls
|
||||
app/ config/ drivers/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile mk/ scripts/ tools/
|
||||
unzip DPDK-<version>.zip
|
||||
cd DPDK-<version>
|
||||
|
||||
ls
|
||||
app/ config/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile
|
||||
mk/ scripts/ tools/
|
||||
|
||||
The DPDK is composed of several directories:
|
||||
|
||||
@ -64,19 +65,19 @@ The DPDK is composed of several directories:
|
||||
Installation of DPDK Target Environments
|
||||
----------------------------------------
|
||||
|
||||
The format of a DPDK target is:
|
||||
The format of a DPDK target is::
|
||||
|
||||
ARCH-MACHINE-EXECENV-TOOLCHAIN
|
||||
|
||||
where:
|
||||
|
||||
* ARCH can be: i686, x86_64, ppc_64
|
||||
* ``ARCH`` can be: ``i686``, ``x86_64``, ``ppc_64``
|
||||
|
||||
* MACHINE can be: native, ivshmem, power8
|
||||
* ``MACHINE`` can be: ``native``, ``ivshmem``, ``power8``
|
||||
|
||||
* EXECENV can be: linuxapp, bsdapp
|
||||
* ``EXECENV`` can be: ``linuxapp``, ``bsdapp``
|
||||
|
||||
* TOOLCHAIN can be: gcc, icc
|
||||
* ``TOOLCHAIN`` can be: ``gcc``, ``icc``
|
||||
|
||||
The targets to be installed depend on the 32-bit and/or 64-bit packages and compilers installed on the host.
|
||||
Available targets can be found in the DPDK/config directory.
|
||||
@ -84,13 +85,13 @@ The defconfig\_ prefix should not be used.
|
||||
|
||||
.. note::
|
||||
|
||||
Configuration files are provided with the RTE_MACHINE optimization level set.
|
||||
Within the configuration files, the RTE_MACHINE configuration value is set to native,
|
||||
Configuration files are provided with the ``RTE_MACHINE`` optimization level set.
|
||||
Within the configuration files, the ``RTE_MACHINE`` configuration value is set to native,
|
||||
which means that the compiled software is tuned for the platform on which it is built.
|
||||
For more information on this setting, and its possible values, see the *DPDK Programmers Guide*.
|
||||
|
||||
When using the Intel® C++ Compiler (icc), one of the following commands should be invoked for 64-bit or 32-bit use respectively.
|
||||
Notice that the shell scripts update the $PATH variable and therefore should not be performed in the same session.
|
||||
Notice that the shell scripts update the ``$PATH`` variable and therefore should not be performed in the same session.
|
||||
Also, verify the compiler's installation directory since the path may be different:
|
||||
|
||||
.. code-block:: console
|
||||
@ -98,7 +99,7 @@ Also, verify the compiler's installation directory since the path may be differe
|
||||
source /opt/intel/bin/iccvars.sh intel64
|
||||
source /opt/intel/bin/iccvars.sh ia32
|
||||
|
||||
To install and make targets, use the make install T=<target> command in the top-level DPDK directory.
|
||||
To install and make targets, use the ``make install T=<target>`` command in the top-level DPDK directory.
|
||||
|
||||
For example, to compile a 64-bit target using icc, run:
|
||||
|
||||
@ -113,7 +114,7 @@ To compile a 32-bit build using gcc, the make command should be:
|
||||
make install T=i686-native-linuxapp-gcc
|
||||
|
||||
To prepare a target without building it, for example, if the configuration changes need to be made before compilation,
|
||||
use the make config T=<target> command:
|
||||
use the ``make config T=<target>`` command:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -121,10 +122,10 @@ use the make config T=<target> command:
|
||||
|
||||
.. warning::
|
||||
|
||||
Any kernel modules to be used, e.g. igb_uio, kni, must be compiled with the
|
||||
Any kernel modules to be used, e.g. ``igb_uio``, ``kni``, must be compiled with the
|
||||
same kernel as the one running on the target.
|
||||
If the DPDK is not being built on the target machine,
|
||||
the RTE_KERNELDIR environment variable should be used to point the compilation at a copy of the kernel version to be used on the target machine.
|
||||
the ``RTE_KERNELDIR`` environment variable should be used to point the compilation at a copy of the kernel version to be used on the target machine.
|
||||
|
||||
Once the target environment is created, the user may move to the target environment directory and continue to make code changes and re-compile.
|
||||
The user may also make modifications to the compile-time DPDK configuration by editing the .config file in the build directory.
|
||||
@ -147,21 +148,22 @@ A kmod directory is also present that contains kernel modules which may be load
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ ls x86_64-native-linuxapp-gcc
|
||||
ls x86_64-native-linuxapp-gcc
|
||||
|
||||
app build hostapp include kmod lib Makefile
|
||||
|
||||
Loading Modules to Enable Userspace IO for DPDK
|
||||
-----------------------------------------------
|
||||
|
||||
To run any DPDK application, a suitable uio module can be loaded into the running kernel.
|
||||
In many cases, the standard uio_pci_generic module included in the Linux kernel
|
||||
In many cases, the standard ``uio_pci_generic`` module included in the Linux kernel
|
||||
can provide the uio capability. This module can be loaded using the command
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
sudo modprobe uio_pci_generic
|
||||
|
||||
As an alternative to the uio_pci_generic, the DPDK also includes the igb_uio
|
||||
As an alternative to the ``uio_pci_generic``, the DPDK also includes the igb_uio
|
||||
module which can be found in the kmod subdirectory referred to above. It can
|
||||
be loaded as shown below:
|
||||
|
||||
@ -173,7 +175,7 @@ be loaded as shown below:
|
||||
.. note::
|
||||
|
||||
For some devices which lack support for legacy interrupts, e.g. virtual function
|
||||
(VF) devices, the igb_uio module may be needed in place of uio_pci_generic.
|
||||
(VF) devices, the ``igb_uio`` module may be needed in place of ``uio_pci_generic``.
|
||||
|
||||
Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
|
||||
for platforms that support using VFIO.
|
||||
@ -181,7 +183,7 @@ for platforms that support using VFIO.
|
||||
Loading VFIO Module
|
||||
-------------------
|
||||
|
||||
To run an DPDK application and make use of VFIO, the vfio-pci module must be loaded:
|
||||
To run an DPDK application and make use of VFIO, the ``vfio-pci`` module must be loaded:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -196,29 +198,31 @@ Also, to use VFIO, both kernel and BIOS must support and be configured to use IO
|
||||
For proper operation of VFIO when running DPDK applications as a non-privileged user, correct permissions should also be set up.
|
||||
This can be done by using the DPDK setup script (called setup.sh and located in the tools directory).
|
||||
|
||||
.. _linux_gsg_binding_kernel:
|
||||
|
||||
Binding and Unbinding Network Ports to/from the Kernel Modules
|
||||
----------------------------------------------------------------------
|
||||
--------------------------------------------------------------
|
||||
|
||||
As of release 1.4, DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use.
|
||||
Instead, all ports that are to be used by an DPDK application must be bound to the
|
||||
uio_pci_generic, igb_uio or vfio-pci module before the application is run.
|
||||
``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module before the application is run.
|
||||
Any network ports under Linux* control will be ignored by the DPDK poll-mode drivers and cannot be used by the application.
|
||||
|
||||
.. warning::
|
||||
|
||||
The DPDK will, by default, no longer automatically unbind network ports from the kernel driver at startup.
|
||||
Any ports to be used by an DPDK application must be unbound from Linux* control and
|
||||
bound to the uio_pci_generic, igb_uio or vfio-pci module before the application is run.
|
||||
bound to the ``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module before the application is run.
|
||||
|
||||
To bind ports to the uio_pci_generic, igb_uio or vfio-pci module for DPDK use,
|
||||
To bind ports to the ``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module for DPDK use,
|
||||
and then subsequently return ports to Linux* control,
|
||||
a utility script called dpdk_nic _bind.py is provided in the tools subdirectory.
|
||||
This utility can be used to provide a view of the current state of the network ports on the system,
|
||||
and to bind and unbind those ports from the different kernel modules, including the uio and vfio modules.
|
||||
The following are some examples of how the script can be used.
|
||||
A full description of the script and its parameters can be obtained by calling the script with the --help or --usage options.
|
||||
A full description of the script and its parameters can be obtained by calling the script with the ``--help`` or ``--usage`` options.
|
||||
Note that the uio or vfio kernel modules to be used, should be loaded into the kernel before
|
||||
running the dpdk_nic_bind.py script.
|
||||
running the ``dpdk_nic_bind.py`` script.
|
||||
|
||||
.. warning::
|
||||
|
||||
@ -239,38 +243,38 @@ To see the status of all network ports on the system:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
root@host:DPDK# ./tools/dpdk_nic_bind.py --status
|
||||
./tools/dpdk_nic_bind.py --status
|
||||
|
||||
Network devices using DPDK-compatible driver
|
||||
============================================
|
||||
0000:82:00.0 '82599EB 10-Gigabit SFI/SFP+ Network Connection' drv=uio_pci_generic unused=ixgbe
|
||||
0000:82:00.1 '82599EB 10-Gigabit SFI/SFP+ Network Connection' drv=uio_pci_generic unused=ixgbe
|
||||
0000:82:00.0 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe
|
||||
0000:82:00.1 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe
|
||||
|
||||
Network devices using kernel driver
|
||||
===================================
|
||||
0000:04:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=uio_pci_generic *Active*
|
||||
0000:04:00.1 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=uio_pci_generic
|
||||
0000:04:00.2 'I350 Gigabit Network Connection' if=eth2 drv=igb unused=uio_pci_generic
|
||||
0000:04:00.3 'I350 Gigabit Network Connection' if=eth3 drv=igb unused=uio_pci_generic
|
||||
0000:04:00.0 'I350 1-GbE NIC' if=em0 drv=igb unused=uio_pci_generic *Active*
|
||||
0000:04:00.1 'I350 1-GbE NIC' if=eth1 drv=igb unused=uio_pci_generic
|
||||
0000:04:00.2 'I350 1-GbE NIC' if=eth2 drv=igb unused=uio_pci_generic
|
||||
0000:04:00.3 'I350 1-GbE NIC' if=eth3 drv=igb unused=uio_pci_generic
|
||||
|
||||
Other network devices
|
||||
=====================
|
||||
<none>
|
||||
|
||||
To bind device eth1, 04:00.1, to the uio_pci_generic driver:
|
||||
To bind device ``eth1``,``04:00.1``, to the ``uio_pci_generic`` driver:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=uio_pci_generic 04:00.1
|
||||
./tools/dpdk_nic_bind.py --bind=uio_pci_generic 04:00.1
|
||||
|
||||
or, alternatively,
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=uio_pci_generic eth1
|
||||
./tools/dpdk_nic_bind.py --bind=uio_pci_generic eth1
|
||||
|
||||
To restore device 82:00.0 to its original kernel binding:
|
||||
To restore device ``82:00.0`` to its original kernel binding:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=ixgbe 82:00.0
|
||||
./tools/dpdk_nic_bind.py --bind=ixgbe 82:00.0
|
||||
|
||||
@ -36,59 +36,62 @@ It also provides a pointer to where sample applications are stored.
|
||||
|
||||
.. note::
|
||||
|
||||
Parts of this process can also be done using the setup script described in **Chapter 6** of this document.
|
||||
Parts of this process can also be done using the setup script described the
|
||||
:ref:`linux_setup_script` section of this document.
|
||||
|
||||
Compiling a Sample Application
|
||||
------------------------------
|
||||
|
||||
Once an DPDK target environment directory has been created (such as x86_64-native-linuxapp-gcc),
|
||||
Once an DPDK target environment directory has been created (such as ``x86_64-native-linuxapp-gcc``),
|
||||
it contains all libraries and header files required to build an application.
|
||||
|
||||
When compiling an application in the Linux* environment on the DPDK, the following variables must be exported:
|
||||
|
||||
* RTE_SDK - Points to the DPDK installation directory.
|
||||
* ``RTE_SDK`` - Points to the DPDK installation directory.
|
||||
|
||||
* RTE_TARGET - Points to the DPDK target environment directory.
|
||||
* ``RTE_TARGET`` - Points to the DPDK target environment directory.
|
||||
|
||||
The following is an example of creating the helloworld application, which runs in the DPDK Linux environment.
|
||||
This example may be found in the ${RTE_SDK}/examples directory.
|
||||
The following is an example of creating the ``helloworld`` application, which runs in the DPDK Linux environment.
|
||||
This example may be found in the ``${RTE_SDK}/examples`` directory.
|
||||
|
||||
The directory contains the main.c file. This file, when combined with the libraries in the DPDK target environment,
|
||||
The directory contains the ``main.c`` file. This file, when combined with the libraries in the DPDK target environment,
|
||||
calls the various functions to initialize the DPDK environment,
|
||||
then launches an entry point (dispatch application) for each core to be utilized.
|
||||
By default, the binary is generated in the build directory.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
user@host:~/DPDK$ cd examples/helloworld/
|
||||
user@host:~/DPDK/examples/helloworld$ export RTE_SDK=$HOME/DPDK
|
||||
user@host:~/DPDK/examples/helloworld$ export RTE_TARGET=x86_64-native-linuxapp-gcc
|
||||
user@host:~/DPDK/examples/helloworld$ make
|
||||
cd examples/helloworld/
|
||||
export RTE_SDK=$HOME/DPDK
|
||||
export RTE_TARGET=x86_64-native-linuxapp-gcc
|
||||
|
||||
make
|
||||
CC main.o
|
||||
LD helloworld
|
||||
INSTALL-APP helloworld
|
||||
INSTALL-MAP helloworld.map
|
||||
|
||||
user@host:~/DPDK/examples/helloworld$ ls build/app
|
||||
ls build/app
|
||||
helloworld helloworld.map
|
||||
|
||||
.. note::
|
||||
|
||||
In the above example, helloworld was in the directory structure of the DPDK.
|
||||
In the above example, ``helloworld`` was in the directory structure of the DPDK.
|
||||
However, it could have been located outside the directory structure to keep the DPDK structure intact.
|
||||
In the following case, the helloworld application is copied to a new directory as a new starting point.
|
||||
In the following case, the ``helloworld`` application is copied to a new directory as a new starting point.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
user@host:~$ export RTE_SDK=/home/user/DPDK
|
||||
user@host:~$ cp -r $(RTE_SDK)/examples/helloworld my_rte_app
|
||||
user@host:~$ cd my_rte_app/
|
||||
user@host:~$ export RTE_TARGET=x86_64-native-linuxapp-gcc
|
||||
user@host:~/my_rte_app$ make
|
||||
CC main.o
|
||||
LD helloworld
|
||||
INSTALL-APP helloworld
|
||||
INSTALL-MAP helloworld.map
|
||||
export RTE_SDK=/home/user/DPDK
|
||||
cp -r $(RTE_SDK)/examples/helloworld my_rte_app
|
||||
cd my_rte_app/
|
||||
export RTE_TARGET=x86_64-native-linuxapp-gcc
|
||||
|
||||
make
|
||||
CC main.o
|
||||
LD helloworld
|
||||
INSTALL-APP helloworld
|
||||
INSTALL-MAP helloworld.map
|
||||
|
||||
Running a Sample Application
|
||||
----------------------------
|
||||
@ -100,7 +103,7 @@ Running a Sample Application
|
||||
.. warning::
|
||||
|
||||
Any ports to be used by the application must be already bound to an appropriate kernel
|
||||
module, as described in Section 3.5, prior to running the application.
|
||||
module, as described in :ref:`linux_gsg_binding_kernel`, prior to running the application.
|
||||
|
||||
The application is linked with the DPDK target environment's Environmental Abstraction Layer (EAL) library,
|
||||
which provides some options that are generic to every DPDK application.
|
||||
@ -109,55 +112,75 @@ The following is the list of options that can be given to the EAL:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
./rte-app -n NUM [-c COREMASK] [-b <domain:bus:devid.func>] [--socket-mem=MB,...] [-m MB] [-r NUM] [-v] [--file-prefix] [--proc-type <primary|secondary|auto>] [-- xen-dom0]
|
||||
./rte-app -c COREMASK [-n NUM] [-b <domain:bus:devid.func>] \
|
||||
[--socket-mem=MB,...] [-m MB] [-r NUM] [-v] [--file-prefix] \
|
||||
[--proc-type <primary|secondary|auto>] [-- xen-dom0]
|
||||
|
||||
The EAL options are as follows:
|
||||
|
||||
* -c COREMASK: An hexadecimal bit mask of the cores to run on. Note that core numbering can change between platforms and should be determined beforehand.
|
||||
* ``-c COREMASK``:
|
||||
An hexadecimal bit mask of the cores to run on. Note that core numbering can
|
||||
change between platforms and should be determined beforehand.
|
||||
|
||||
* -n NUM: Number of memory channels per processor socket
|
||||
* ``-n NUM``:
|
||||
Number of memory channels per processor socket.
|
||||
|
||||
* -b <domain:bus:devid.func>: blacklisting of ports; prevent EAL from using specified PCI device (multiple -b options are allowed)
|
||||
* ``-b <domain:bus:devid.func>``:
|
||||
Blacklisting of ports; prevent EAL from using specified PCI device
|
||||
(multiple ``-b`` options are allowed).
|
||||
|
||||
* --use-device: use the specified Ethernet device(s) only. Use comma-separate <[domain:]bus:devid.func> values. Cannot be used with -b option
|
||||
* ``--use-device``:
|
||||
use the specified Ethernet device(s) only. Use comma-separate
|
||||
``[domain:]bus:devid.func`` values. Cannot be used with ``-b`` option.
|
||||
|
||||
* --socket-mem: Memory to allocate from hugepages on specific sockets
|
||||
* ``--socket-mem``:
|
||||
Memory to allocate from hugepages on specific sockets.
|
||||
|
||||
* -m MB: Memory to allocate from hugepages, regardless of processor socket. It is recommended that --socket-mem be used instead of this option.
|
||||
* ``-m MB``:
|
||||
Memory to allocate from hugepages, regardless of processor socket. It is
|
||||
recommended that ``--socket-mem`` be used instead of this option.
|
||||
|
||||
* -r NUM: Number of memory ranks
|
||||
* ``-r NUM``:
|
||||
Number of memory ranks.
|
||||
|
||||
* -v: Display version information on startup
|
||||
* ``-v``:
|
||||
Display version information on startup.
|
||||
|
||||
* --huge-dir: The directory where hugetlbfs is mounted
|
||||
* ``--huge-dir``:
|
||||
The directory where hugetlbfs is mounted.
|
||||
|
||||
* --file-prefix: The prefix text used for hugepage filenames
|
||||
* ``--file-prefix``:
|
||||
The prefix text used for hugepage filenames.
|
||||
|
||||
* --proc-type: The type of process instance
|
||||
* ``--proc-type``:
|
||||
The type of process instance.
|
||||
|
||||
* --xen-dom0: Support application running on Xen Domain0 without hugetlbfs
|
||||
* ``--xen-dom0``:
|
||||
Support application running on Xen Domain0 without hugetlbfs.
|
||||
|
||||
* --vmware-tsc-map: use VMware TSC map instead of native RDTSC
|
||||
* ``--vmware-tsc-map``:
|
||||
Use VMware TSC map instead of native RDTSC.
|
||||
|
||||
* --base-virtaddr: specify base virtual address
|
||||
* ``--base-virtaddr``:
|
||||
Specify base virtual address.
|
||||
|
||||
* --vfio-intr: specify interrupt type to be used by VFIO (has no effect if VFIO is not used)
|
||||
* ``--vfio-intr``:
|
||||
Specify interrupt type to be used by VFIO (has no effect if VFIO is not used).
|
||||
|
||||
The -c and the -n options are mandatory; the others are optional.
|
||||
The ``-c`` and option is mandatory; the others are optional.
|
||||
|
||||
Copy the DPDK application binary to your target, then run the application as follows
|
||||
(assuming the platform has four memory channels per processor socket,
|
||||
and that cores 0-3 are present and are to be used for running the application):
|
||||
and that cores 0-3 are present and are to be used for running the application)::
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
user@target:~$ ./helloworld -c f -n 4
|
||||
./helloworld -c f -n 4
|
||||
|
||||
.. note::
|
||||
|
||||
The --proc-type and --file-prefix EAL options are used for running multiple DPDK processes.
|
||||
See the “Multi-process Sample Application” chapter in the *DPDK Sample Applications User Guide* and
|
||||
the *DPDK Programmers Guide* for more details.
|
||||
The ``--proc-type`` and ``--file-prefix`` EAL options are used for running
|
||||
multiple DPDK processes. See the "Multi-process Sample Application"
|
||||
chapter in the *DPDK Sample Applications User Guide* and the *DPDK
|
||||
Programmers Guide* for more details.
|
||||
|
||||
Logical Core Use by Applications
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@ -168,16 +191,14 @@ Since these logical core numbers, and their mapping to specific cores on specifi
|
||||
it is recommended that the core layout for each platform be considered when choosing the coremask to use in each case.
|
||||
|
||||
On initialization of the EAL layer by an DPDK application, the logical cores to be used and their socket location are displayed.
|
||||
This information can also be determined for all cores on the system by examining the /proc/cpuinfo file, for example, by running cat /proc/cpuinfo.
|
||||
This information can also be determined for all cores on the system by examining the ``/proc/cpuinfo`` file, for example, by running cat ``/proc/cpuinfo``.
|
||||
The physical id attribute listed for each processor indicates the CPU socket to which it belongs.
|
||||
This can be useful when using other processors to understand the mapping of the logical cores to the sockets.
|
||||
|
||||
.. note::
|
||||
|
||||
A more graphical view of the logical core layout may be obtained using the lstopo Linux utility.
|
||||
On Fedora* Linux, this may be installed and run using the following command:
|
||||
|
||||
.. code-block:: console
|
||||
A more graphical view of the logical core layout may be obtained using the ``lstopo`` Linux utility.
|
||||
On Fedora Linux, this may be installed and run using the following command::
|
||||
|
||||
sudo yum install hwloc
|
||||
./lstopo
|
||||
@ -191,25 +212,25 @@ Hugepage Memory Use by Applications
|
||||
|
||||
When running an application, it is recommended to use the same amount of memory as that allocated for hugepages.
|
||||
This is done automatically by the DPDK application at startup,
|
||||
if no -m or --socket-mem parameter is passed to it when run.
|
||||
if no ``-m`` or ``--socket-mem`` parameter is passed to it when run.
|
||||
|
||||
If more memory is requested by explicitly passing a -m or --socket-mem value, the application fails.
|
||||
However, the application itself can also fail if the user requests less memory than the reserved amount of hugepage-memory, particularly if using the -m option.
|
||||
If more memory is requested by explicitly passing a ``-m`` or ``--socket-mem`` value, the application fails.
|
||||
However, the application itself can also fail if the user requests less memory than the reserved amount of hugepage-memory, particularly if using the ``-m`` option.
|
||||
The reason is as follows.
|
||||
Suppose the system has 1024 reserved 2 MB pages in socket 0 and 1024 in socket 1.
|
||||
If the user requests 128 MB of memory, the 64 pages may not match the constraints:
|
||||
|
||||
* The hugepage memory by be given to the application by the kernel in socket 1 only.
|
||||
In this case, if the application attempts to create an object, such as a ring or memory pool in socket 0, it fails.
|
||||
To avoid this issue, it is recommended that the -- socket-mem option be used instead of the -m option.
|
||||
To avoid this issue, it is recommended that the ``--socket-mem`` option be used instead of the ``-m`` option.
|
||||
|
||||
* These pages can be located anywhere in physical memory, and, although the DPDK EAL will attempt to allocate memory in contiguous blocks,
|
||||
it is possible that the pages will not be contiguous. In this case, the application is not able to allocate big memory pools.
|
||||
|
||||
The socket-mem option can be used to request specific amounts of memory for specific sockets.
|
||||
This is accomplished by supplying the --socket-mem flag followed by amounts of memory requested on each socket,
|
||||
for example, supply --socket-mem=0,512 to try and reserve 512 MB for socket 1 only.
|
||||
Similarly, on a four socket system, to allocate 1 GB memory on each of sockets 0 and 2 only, the parameter --socket-mem=1024,0,1024 can be used.
|
||||
This is accomplished by supplying the ``--socket-mem`` flag followed by amounts of memory requested on each socket,
|
||||
for example, supply ``--socket-mem=0,512`` to try and reserve 512 MB for socket 1 only.
|
||||
Similarly, on a four socket system, to allocate 1 GB memory on each of sockets 0 and 2 only, the parameter ``--socket-mem=1024,0,1024`` can be used.
|
||||
No memory will be reserved on any CPU socket that is not explicitly referenced, for example, socket 3 in this case.
|
||||
If the DPDK cannot allocate enough memory on each socket, the EAL initialization fails.
|
||||
|
||||
|
||||
@ -47,11 +47,9 @@ The BIOS is typically accessed by pressing F2 while the platform is starting up.
|
||||
The user can then navigate to the HPET option. On the Crystal Forest platform BIOS, the path is:
|
||||
**Advanced -> PCH-IO Configuration -> High Precision Timer ->** (Change from Disabled to Enabled if necessary).
|
||||
|
||||
On a system that has already booted, the following command can be issued to check if HPET is enabled:
|
||||
On a system that has already booted, the following command can be issued to check if HPET is enabled::
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# grep hpet /proc/timer_list
|
||||
grep hpet /proc/timer_list
|
||||
|
||||
If no entries are returned, HPET must be enabled in the BIOS (as per the instructions above) and the system rebooted.
|
||||
|
||||
@ -59,75 +57,84 @@ Linux Kernel Support
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The DPDK makes use of the platform HPET timer by mapping the timer counter into the process address space, and as such,
|
||||
requires that the HPET_MMAP kernel configuration option be enabled.
|
||||
requires that the ``HPET_MMAP`` kernel configuration option be enabled.
|
||||
|
||||
.. warning::
|
||||
|
||||
On Fedora*, and other common distributions such as Ubuntu*, the HPET_MMAP kernel option is not enabled by default.
|
||||
On Fedora, and other common distributions such as Ubuntu, the ``HPET_MMAP`` kernel option is not enabled by default.
|
||||
To recompile the Linux kernel with this option enabled, please consult the distributions documentation for the relevant instructions.
|
||||
|
||||
Enabling HPET in the DPDK
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
By default, HPET support is disabled in the DPDK build configuration files.
|
||||
To use HPET, the CONFIG_RTE_LIBEAL_USE_HPET setting should be changed to “y”, which will enable the HPET settings at compile time.
|
||||
To use HPET, the ``CONFIG_RTE_LIBEAL_USE_HPET`` setting should be changed to ``y``, which will enable the HPET settings at compile time.
|
||||
|
||||
For an application to use the rte_get_hpet_cycles() and rte_get_hpet_hz() API calls,
|
||||
For an application to use the ``rte_get_hpet_cycles()`` and ``rte_get_hpet_hz()`` API calls,
|
||||
and optionally to make the HPET the default time source for the rte_timer library,
|
||||
the new rte_eal_hpet_init() API call should be called at application initialization.
|
||||
the new ``rte_eal_hpet_init()`` API call should be called at application initialization.
|
||||
This API call will ensure that the HPET is accessible, returning an error to the application if it is not,
|
||||
for example, if HPET_MMAP is not enabled in the kernel.
|
||||
for example, if ``HPET_MMAP`` is not enabled in the kernel.
|
||||
The application can then determine what action to take, if any, if the HPET is not available at run-time.
|
||||
|
||||
.. note::
|
||||
|
||||
For applications that require timing APIs, but not the HPET timer specifically,
|
||||
it is recommended that the rte_get_timer_cycles() and rte_get_timer_hz() API calls be used instead of the HPET-specific APIs.
|
||||
These generic APIs can work with either TSC or HPET time sources, depending on what is requested by an application call to rte_eal_hpet_init(),
|
||||
it is recommended that the ``rte_get_timer_cycles()`` and ``rte_get_timer_hz()`` API calls be used instead of the HPET-specific APIs.
|
||||
These generic APIs can work with either TSC or HPET time sources, depending on what is requested by an application call to ``rte_eal_hpet_init()``,
|
||||
if any, and on what is available on the system at runtime.
|
||||
|
||||
Running DPDK Applications Without Root Privileges
|
||||
--------------------------------------------------------
|
||||
|
||||
Although applications using the DPDK use network ports and other hardware resources directly,
|
||||
with a number of small permission adjustments it is possible to run these applications as a user other than “root”.
|
||||
with a number of small permission adjustments it is possible to run these applications as a user other than "root".
|
||||
To do so, the ownership, or permissions, on the following Linux file system objects should be adjusted to ensure that
|
||||
the Linux user account being used to run the DPDK application has access to them:
|
||||
|
||||
* All directories which serve as hugepage mount points, for example, /mnt/huge
|
||||
* All directories which serve as hugepage mount points, for example, ``/mnt/huge``
|
||||
|
||||
* The userspace-io device files in /dev, for example, /dev/uio0, /dev/uio1, and so on
|
||||
* The userspace-io device files in ``/dev``, for example, ``/dev/uio0``, ``/dev/uio1``, and so on
|
||||
|
||||
* The userspace-io sysfs config and resource files, for example for uio0: /sys/class/uio/uio0/device/config /sys/class/uio/uio0/device/resource*
|
||||
* The userspace-io sysfs config and resource files, for example for ``uio0``::
|
||||
|
||||
* If the HPET is to be used, /dev/hpet
|
||||
/sys/class/uio/uio0/device/config
|
||||
/sys/class/uio/uio0/device/resource*
|
||||
|
||||
* If the HPET is to be used, ``/dev/hpet``
|
||||
|
||||
.. note::
|
||||
|
||||
On some Linux installations, /dev/hugepages is also a hugepage mount point created by default.
|
||||
On some Linux installations, ``/dev/hugepages`` is also a hugepage mount point created by default.
|
||||
|
||||
Power Management and Power Saving Functionality
|
||||
-----------------------------------------------
|
||||
|
||||
Enhanced Intel SpeedStep® Technology must be enabled in the platform BIOS if the power management feature of DPDK is to be used.
|
||||
Otherwise, the sys file folder /sys/devices/system/cpu/cpu0/cpufreq will not exist, and the CPU frequency- based power management cannot be used.
|
||||
Otherwise, the sys file folder ``/sys/devices/system/cpu/cpu0/cpufreq`` will not exist, and the CPU frequency- based power management cannot be used.
|
||||
Consult the relevant BIOS documentation to determine how these settings can be accessed.
|
||||
|
||||
For example, on some Intel reference platform BIOS variants, the path to Enhanced Intel SpeedStep® Technology is:
|
||||
For example, on some Intel reference platform BIOS variants, the path to Enhanced Intel SpeedStep® Technology is::
|
||||
|
||||
**Advanced->Processor Configuration->Enhanced Intel SpeedStep® Tech**
|
||||
Advanced
|
||||
-> Processor Configuration
|
||||
-> Enhanced Intel SpeedStep® Tech
|
||||
|
||||
In addition, C3 and C6 should be enabled as well for power management. The path of C3 and C6 on the same platform BIOS is:
|
||||
In addition, C3 and C6 should be enabled as well for power management. The path of C3 and C6 on the same platform BIOS is::
|
||||
|
||||
**Advanced->Processor Configuration->Processor C3 Advanced->Processor Configuration-> Processor C6**
|
||||
Advanced
|
||||
-> Processor Configuration
|
||||
-> Processor C3 Advanced
|
||||
-> Processor Configuration
|
||||
-> Processor C6
|
||||
|
||||
Using Linux* Core Isolation to Reduce Context Switches
|
||||
------------------------------------------------------
|
||||
Using Linux Core Isolation to Reduce Context Switches
|
||||
-----------------------------------------------------
|
||||
|
||||
While the threads used by an DPDK application are pinned to logical cores on the system,
|
||||
it is possible for the Linux scheduler to run other tasks on those cores also.
|
||||
To help prevent additional workloads from running on those cores,
|
||||
it is possible to use the isolcpus Linux* kernel parameter to isolate them from the general Linux scheduler.
|
||||
it is possible to use the ``isolcpus`` Linux kernel parameter to isolate them from the general Linux scheduler.
|
||||
|
||||
For example, if DPDK applications are to run on logical cores 2, 4 and 6,
|
||||
the following should be added to the kernel parameter list:
|
||||
@ -137,38 +144,38 @@ the following should be added to the kernel parameter list:
|
||||
isolcpus=2,4,6
|
||||
|
||||
Loading the DPDK KNI Kernel Module
|
||||
-----------------------------------------
|
||||
----------------------------------
|
||||
|
||||
To run the DPDK Kernel NIC Interface (KNI) sample application, an extra kernel module (the kni module) must be loaded into the running kernel.
|
||||
The module is found in the kmod sub-directory of the DPDK target directory.
|
||||
Similar to the loading of the igb_uio module, this module should be loaded using the insmod command as shown below
|
||||
Similar to the loading of the ``igb_uio`` module, this module should be loaded using the insmod command as shown below
|
||||
(assuming that the current directory is the DPDK target directory):
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
#insmod kmod/rte_kni.ko
|
||||
insmod kmod/rte_kni.ko
|
||||
|
||||
.. note::
|
||||
|
||||
See the “Kernel NIC Interface Sample Application” chapter in the *DPDK Sample Applications User Guide* for more details.
|
||||
See the "Kernel NIC Interface Sample Application" chapter in the *DPDK Sample Applications User Guide* for more details.
|
||||
|
||||
Using Linux IOMMU Pass-Through to Run DPDK with Intel® VT-d
|
||||
-----------------------------------------------------------
|
||||
|
||||
To enable Intel® VT-d in a Linux kernel, a number of kernel configuration options must be set. These include:
|
||||
|
||||
* IOMMU_SUPPORT
|
||||
* ``IOMMU_SUPPORT``
|
||||
|
||||
* IOMMU_API
|
||||
* ``IOMMU_API``
|
||||
|
||||
* INTEL_IOMMU
|
||||
* ``INTEL_IOMMU``
|
||||
|
||||
In addition, to run the DPDK with Intel® VT-d, the iommu=pt kernel parameter must be used when using igb_uio driver.
|
||||
In addition, to run the DPDK with Intel® VT-d, the ``iommu=pt`` kernel parameter must be used when using ``igb_uio`` driver.
|
||||
This results in pass-through of the DMAR (DMA Remapping) lookup in the host.
|
||||
Also, if INTEL_IOMMU_DEFAULT_ON is not set in the kernel, the intel_iommu=on kernel parameter must be used too.
|
||||
Also, if ``INTEL_IOMMU_DEFAULT_ON`` is not set in the kernel, the ``intel_iommu=on`` kernel parameter must be used too.
|
||||
This ensures that the Intel IOMMU is being initialized as expected.
|
||||
|
||||
Please note that while using iommu=pt is compulsory for igb_uio driver, the vfio-pci driver can actually work with both iommu=pt and iommu=on.
|
||||
Please note that while using ``iommu=pt`` is compulsory for ``igb_uio driver``, the ``vfio-pci`` driver can actually work with both ``iommu=pt`` and ``iommu=on``.
|
||||
|
||||
High Performance of Small Packets on 40G NIC
|
||||
--------------------------------------------
|
||||
@ -182,12 +189,12 @@ DPDK release, so currently the validated firmware version is 4.2.6.
|
||||
Enabling Extended Tag and Setting Max Read Request Size
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
PCI configurations of extended_tag and max _read_requ st_size have big impacts on performance of small packets on 40G NIC.
|
||||
Enabling extended_tag and setting max _read_requ st_size to small size such as 128 bytes provide great helps to high performance of small packets.
|
||||
PCI configurations of ``extended_tag`` and max _read_requ st_size have big impacts on performance of small packets on 40G NIC.
|
||||
Enabling extended_tag and setting ``max_read_request_size`` to small size such as 128 bytes provide great helps to high performance of small packets.
|
||||
|
||||
* These can be done in some BIOS implementations.
|
||||
|
||||
* For other BIOS implementations, PCI configurations can be changed by using command of setpci, or special configurations in DPDK config file of common_linux.
|
||||
* For other BIOS implementations, PCI configurations can be changed by using command of ``setpci``, or special configurations in DPDK config file of ``common_linux``.
|
||||
|
||||
* Bits 7:5 at address of 0xA8 of each PCI device is used for setting the max_read_request_size,
|
||||
and bit 8 of 0xA8 of each PCI device is used for enabling/disabling the extended_tag.
|
||||
@ -195,24 +202,24 @@ Enabling extended_tag and setting max _read_requ st_size to small size such as 1
|
||||
|
||||
* In config file of common_linux, below three configurations can be changed for the same purpose.
|
||||
|
||||
CONFIG_RTE_PCI_CONFIG
|
||||
``CONFIG_RTE_PCI_CONFIG``
|
||||
|
||||
CONFIG_RTE_PCI_EXTENDED_TAG
|
||||
``CONFIG_RTE_PCI_EXTENDED_TAG``
|
||||
|
||||
CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE
|
||||
``CONFIG_RTE_PCI_MAX_READ_REQUEST_SIZE``
|
||||
|
||||
Use 16 Bytes RX Descriptor Size
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
As i40e PMD supports both 16 and 32 bytes RX descriptor sizes, and 16 bytes size can provide helps to high performance of small packets.
|
||||
Configuration of CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC in config files can be changed to use 16 bytes size RX descriptors.
|
||||
Configuration of ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC`` in config files can be changed to use 16 bytes size RX descriptors.
|
||||
|
||||
High Performance and per Packet Latency Tradeoff
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Due to the hardware design, the interrupt signal inside NIC is needed for per
|
||||
packet descriptor write-back. The minimum interval of interrupts could be set
|
||||
at compile time by CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL in configuration files.
|
||||
at compile time by ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` in configuration files.
|
||||
Though there is a default configuration, the interval could be tuned by the
|
||||
users with that configuration item depends on what the user cares about more,
|
||||
performance or per packet latency.
|
||||
|
||||
@ -33,7 +33,7 @@ Introduction
|
||||
|
||||
This document contains instructions for installing and configuring the Intel® Data Plane Development Kit (DPDK) software.
|
||||
It is designed to get customers up and running quickly.
|
||||
The document describes how to compile and run a DPDK application in a Linux* application (linuxapp) environment,
|
||||
The document describes how to compile and run a DPDK application in a Linux application (linuxapp) environment,
|
||||
without going deeply into detail.
|
||||
|
||||
Documentation Roadmap
|
||||
@ -48,7 +48,7 @@ The following is a list of DPDK documents in the suggested reading order:
|
||||
|
||||
* Programmer's Guide: Describes:
|
||||
|
||||
* The software architecture and how to use it (through examples), specifically in a Linux* application (linuxapp) environment
|
||||
* The software architecture and how to use it (through examples), specifically in a Linux application (linuxapp) environment
|
||||
|
||||
* The content of the DPDK, the build system (including the commands that can be used in the root DPDK Makefile to build the development kit and
|
||||
an application) and guidelines for porting an application
|
||||
@ -61,7 +61,3 @@ The following is a list of DPDK documents in the suggested reading order:
|
||||
|
||||
* Sample Applications User Guide: Describes a set of sample applications.
|
||||
Each chapter describes a sample application that showcases specific functionality and provides instructions on how to compile, run and use the sample application.
|
||||
|
||||
.. note::
|
||||
|
||||
These documents are available for download as a separate documentation package at the same location as the DPDK code package.
|
||||
|
||||
@ -28,6 +28,8 @@
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
.. _linux_setup_script:
|
||||
|
||||
Quick Start Setup Script
|
||||
========================
|
||||
|
||||
@ -51,7 +53,7 @@ The setup.sh script, found in the tools subdirectory, allows the user to perform
|
||||
|
||||
* Look at hugepages in the meminfo
|
||||
|
||||
* List hugepages in /mnt/huge
|
||||
* List hugepages in ``/mnt/huge``
|
||||
|
||||
* Remove built DPDK libraries
|
||||
|
||||
@ -106,7 +108,7 @@ Some options in the script prompt the user for further data before proceeding.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
user@host:~/rte$ source tools/setup.sh
|
||||
source tools/setup.sh
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
@ -202,7 +204,7 @@ Some options in the script prompt the user for further data before proceeding.
|
||||
|
||||
Option:
|
||||
|
||||
The following selection demonstrates the creation of the x86_64-native-linuxapp-gcc DPDK library.
|
||||
The following selection demonstrates the creation of the ``x86_64-native-linuxapp-gcc`` DPDK library.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -214,7 +216,7 @@ The following selection demonstrates the creation of the x86_64-native-linuxapp-
|
||||
== Build lib
|
||||
...
|
||||
Build complete
|
||||
RTE_TARGET exported as x86_64-native -linuxapp-gcc
|
||||
RTE_TARGET exported as x86_64-native-linuxapp-gcc
|
||||
|
||||
The following selection demonstrates the starting of the DPDK UIO driver.
|
||||
|
||||
@ -277,15 +279,16 @@ the logical core layout of the platform should be determined when selecting a co
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
rte@rte-desktop:~/rte/examples$ cd helloworld/
|
||||
rte@rte-desktop:~/rte/examples/helloworld$ make
|
||||
CC main.o
|
||||
LD helloworld
|
||||
INSTALL-APP helloworld
|
||||
INSTALL-MAP helloworld.map
|
||||
cd helloworld/
|
||||
make
|
||||
CC main.o
|
||||
LD helloworld
|
||||
INSTALL-APP helloworld
|
||||
INSTALL-MAP helloworld.map
|
||||
|
||||
rte@rte-desktop:~/rte/examples/helloworld$ sudo ./build/app/helloworld -c 0xf -n 3
|
||||
sudo ./build/app/helloworld -c 0xf -n 3
|
||||
[sudo] password for rte:
|
||||
|
||||
EAL: coremask set to f
|
||||
EAL: Detected lcore 0 as core 0 on socket 0
|
||||
EAL: Detected lcore 1 as core 0 on socket 1
|
||||
|
||||
@ -28,7 +28,6 @@
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
|
||||
System Requirements
|
||||
===================
|
||||
|
||||
@ -37,7 +36,7 @@ This chapter describes the packages required to compile the DPDK.
|
||||
.. note::
|
||||
|
||||
If the DPDK is being used on an Intel® Communications Chipset 89xx Series platform,
|
||||
please consult the *Intel® Communications Chipset 89xx Series Software for Linux* Getting Started Guide*.
|
||||
please consult the *Intel® Communications Chipset 89xx Series Software for Linux Getting Started Guide*.
|
||||
|
||||
BIOS Setting Prerequisite on x86
|
||||
--------------------------------
|
||||
@ -45,7 +44,7 @@ BIOS Setting Prerequisite on x86
|
||||
For the majority of platforms, no special BIOS settings are needed to use basic DPDK functionality.
|
||||
However, for additional HPET timer and power management functionality,
|
||||
and high performance of small packets on 40G NIC, BIOS setting changes may be needed.
|
||||
Consult :ref:`Chapter 5. Enabling Additional Functionality <Enabling_Additional_Functionality>`
|
||||
Consult the section on :ref:`Enabling Additional Functionality <Enabling_Additional_Functionality>`
|
||||
for more information on the required changes.
|
||||
|
||||
Compilation of the DPDK
|
||||
@ -55,17 +54,17 @@ Compilation of the DPDK
|
||||
|
||||
.. note::
|
||||
|
||||
Testing has been performed using Fedora* 18. The setup commands and installed packages needed on other systems may be different.
|
||||
Testing has been performed using Fedora 18. The setup commands and installed packages needed on other systems may be different.
|
||||
For details on other Linux distributions and the versions tested, please consult the DPDK Release Notes.
|
||||
|
||||
* GNU make
|
||||
* GNU ``make``.
|
||||
|
||||
* coreutils: cmp, sed, grep, arch
|
||||
* coreutils: ``cmp``, ``sed``, ``grep``, ``arch``, etc.
|
||||
|
||||
* gcc: versions 4.5.x or later is recommended for i686/x86_64. versions 4.8.x or later is recommended
|
||||
for ppc_64 and x86_x32 ABI. On some distributions, some specific compiler flags and linker flags are enabled by
|
||||
default and affect performance (- fstack-protector, for example). Please refer to the documentation
|
||||
of your distribution and to gcc -dumpspecs.
|
||||
* gcc: versions 4.5.x or later is recommended for ``i686/x86_64``. Versions 4.8.x or later is recommended
|
||||
for ``ppc_64`` and ``x86_x32`` ABI. On some distributions, some specific compiler flags and linker flags are enabled by
|
||||
default and affect performance (``-fstack-protector``, for example). Please refer to the documentation
|
||||
of your distribution and to ``gcc -dumpspecs``.
|
||||
|
||||
* libc headers (glibc-devel.i686 / libc6-dev-i386; glibc-devel.x86_64 for 64-bit compilation on Intel
|
||||
architecture; glibc-devel.ppc64 for 64 bit IBM Power architecture;)
|
||||
@ -75,9 +74,9 @@ Compilation of the DPDK
|
||||
|
||||
* Additional packages required for 32-bit compilation on 64-bit systems are:
|
||||
|
||||
glibc.i686, libgcc.i686, libstdc++.i686 and glibc-devel.i686 for Intel i686/x86_64;
|
||||
* glibc.i686, libgcc.i686, libstdc++.i686 and glibc-devel.i686 for Intel i686/x86_64;
|
||||
|
||||
glibc.ppc64, libgcc.ppc64, libstdc++.ppc64 and glibc-devel.ppc64 for IBM ppc_64;
|
||||
* glibc.ppc64, libgcc.ppc64, libstdc++.ppc64 and glibc-devel.ppc64 for IBM ppc_64;
|
||||
|
||||
.. note::
|
||||
|
||||
@ -86,21 +85,20 @@ Compilation of the DPDK
|
||||
|
||||
.. note::
|
||||
|
||||
Python, version 2.6 or 2.7, to use various helper scripts included in the DPDK package
|
||||
Python, version 2.6 or 2.7, to use various helper scripts included in the DPDK package.
|
||||
|
||||
|
||||
**Optional Tools:**
|
||||
|
||||
* Intel® C++ Compiler (icc). For installation, additional libraries may be required.
|
||||
* Intel® C++ Compiler (icc). For installation, additional libraries may be required.
|
||||
See the icc Installation Guide found in the Documentation directory under the compiler installation.
|
||||
This release has been tested using version 12.1.
|
||||
|
||||
* IBM® Advance ToolChain for Powerlinux. This is a set of open source development tools and runtime libraries
|
||||
which allows users to take leading edge advantage of IBM's latest POWER hardware features on Linux. To install
|
||||
it, see the IBM official installation document.
|
||||
|
||||
* libpcap headers and libraries (libpcap-devel) to compile and use the libpcap-based poll-mode driver.
|
||||
This driver is disabled by default and can be enabled by setting CONFIG_RTE_LIBRTE_PMD_PCAP=y in the build time config file.
|
||||
This driver is disabled by default and can be enabled by setting ``CONFIG_RTE_LIBRTE_PMD_PCAP=y`` in the build time config file.
|
||||
|
||||
Running DPDK Applications
|
||||
-------------------------
|
||||
@ -114,29 +112,17 @@ System Software
|
||||
|
||||
* Kernel version >= 2.6.34
|
||||
|
||||
The kernel version in use can be checked using the command:
|
||||
|
||||
.. code-block:: console
|
||||
The kernel version in use can be checked using the command::
|
||||
|
||||
uname -r
|
||||
|
||||
* glibc >= 2.7 (for features related to cpuset)
|
||||
|
||||
The version can be checked using the ldd --version command. A sample output is shown below:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# ldd --version
|
||||
|
||||
ldd (GNU libc) 2.14.90
|
||||
Copyright (C) 2011 Free Software Foundation, Inc.
|
||||
This is free software; see the source for copying conditions. There is NO
|
||||
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
Written by Roland McGrath and Ulrich Drepper.
|
||||
The version can be checked using the ``ldd --version`` command.
|
||||
|
||||
* Kernel configuration
|
||||
|
||||
In the Fedora* OS and other common distributions, such as Ubuntu*, or Red Hat Enterprise Linux*,
|
||||
In the Fedora OS and other common distributions, such as Ubuntu, or Red Hat Enterprise Linux,
|
||||
the vendor supplied kernel configurations can be used to run most DPDK applications.
|
||||
|
||||
For other kernel builds, options which should be enabled for DPDK include:
|
||||
@ -148,15 +134,15 @@ System Software
|
||||
* PROC_PAGE_MONITOR support
|
||||
|
||||
* HPET and HPET_MMAP configuration options should also be enabled if HPET support is required.
|
||||
See :ref:`Section 5.1 High Precision Event Timer (HPET) Functionality <High_Precision_Event_Timer>` for more details.
|
||||
See the section on :ref:`High Precision Event Timer (HPET) Functionality <High_Precision_Event_Timer>` for more details.
|
||||
|
||||
.. _linux_gsg_hugepages:
|
||||
|
||||
Use of Hugepages in the Linux* Environment
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Use of Hugepages in the Linux Environment
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Hugepage support is required for the large memory pool allocation used for packet buffers
|
||||
(the HUGETLBFS option must be enabled in the running kernel as indicated in Section 2.3).
|
||||
(the HUGETLBFS option must be enabled in the running kernel as indicated the previous section).
|
||||
By using hugepage allocations, performance is increased since fewer pages are needed,
|
||||
and therefore less Translation Lookaside Buffers (TLBs, high speed translation caches),
|
||||
which reduce the time it takes to translate a virtual page address to a physical page address.
|
||||
@ -167,19 +153,15 @@ Reserving Hugepages for DPDK Use
|
||||
|
||||
The allocation of hugepages should be done at boot time or as soon as possible after system boot
|
||||
to prevent memory from being fragmented in physical memory.
|
||||
To reserve hugepages at boot time, a parameter is passed to the Linux* kernel on the kernel command line.
|
||||
To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.
|
||||
|
||||
For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use:
|
||||
|
||||
.. code-block:: console
|
||||
For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use::
|
||||
|
||||
hugepages=1024
|
||||
|
||||
For other hugepage sizes, for example 1G pages, the size must be specified explicitly and
|
||||
can also be optionally set as the default hugepage size for the system.
|
||||
For example, to reserve 4G of hugepage memory in the form of four 1G pages, the following options should be passed to the kernel:
|
||||
|
||||
.. code-block:: console
|
||||
For example, to reserve 4G of hugepage memory in the form of four 1G pages, the following options should be passed to the kernel::
|
||||
|
||||
default_hugepagesz=1G hugepagesz=1G hugepages=4
|
||||
|
||||
@ -197,21 +179,17 @@ In the case of a dual-socket NUMA system,
|
||||
the number of hugepages reserved at boot time is generally divided equally between the two sockets
|
||||
(on the assumption that sufficient memory is present on both sockets).
|
||||
|
||||
See the Documentation/kernel-parameters.txt file in your Linux* source tree for further details of these and other kernel options.
|
||||
See the Documentation/kernel-parameters.txt file in your Linux source tree for further details of these and other kernel options.
|
||||
|
||||
**Alternative:**
|
||||
|
||||
For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
|
||||
This is done by echoing the number of hugepages required to a nr_hugepages file in the /sys/devices/ directory.
|
||||
For a single-node system, the command to use is as follows (assuming that 1024 pages are required):
|
||||
|
||||
.. code-block:: console
|
||||
This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory.
|
||||
For a single-node system, the command to use is as follows (assuming that 1024 pages are required)::
|
||||
|
||||
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
|
||||
|
||||
On a NUMA machine, pages should be allocated explicitly on separate nodes:
|
||||
|
||||
.. code-block:: console
|
||||
On a NUMA machine, pages should be allocated explicitly on separate nodes::
|
||||
|
||||
echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
|
||||
echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
|
||||
@ -223,29 +201,23 @@ On a NUMA machine, pages should be allocated explicitly on separate nodes:
|
||||
Using Hugepages with the DPDK
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:
|
||||
|
||||
.. code-block:: console
|
||||
Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps::
|
||||
|
||||
mkdir /mnt/huge
|
||||
mount -t hugetlbfs nodev /mnt/huge
|
||||
|
||||
The mount point can be made permanent across reboots, by adding the following line to the /etc/fstab file:
|
||||
|
||||
.. code-block:: console
|
||||
The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file::
|
||||
|
||||
nodev /mnt/huge hugetlbfs defaults 0 0
|
||||
|
||||
For 1GB pages, the page size must be specified as a mount option:
|
||||
|
||||
.. code-block:: console
|
||||
For 1GB pages, the page size must be specified as a mount option::
|
||||
|
||||
nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
|
||||
|
||||
Xen Domain0 Support in the Linux* Environment
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Xen Domain0 Support in the Linux Environment
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The existing memory management implementation is based on the Linux* kernel hugepage mechanism.
|
||||
The existing memory management implementation is based on the Linux kernel hugepage mechanism.
|
||||
On the Xen hypervisor, hugepage support for DomainU (DomU) Guests means that DPDK applications work as normal for guests.
|
||||
|
||||
However, Domain0 (Dom0) does not support hugepages.
|
||||
@ -263,11 +235,9 @@ Furthermore, the CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID setting should also be chang
|
||||
Loading the DPDK rte_dom0_mm Module
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
To run any DPDK application on Xen Dom0, the rte_dom0_mm module must be loaded into the running kernel with rsv_memsize option.
|
||||
To run any DPDK application on Xen Dom0, the ``rte_dom0_mm`` module must be loaded into the running kernel with rsv_memsize option.
|
||||
The module is found in the kmod sub-directory of the DPDK target directory.
|
||||
This module should be loaded using the insmod command as shown below (assuming that the current directory is the DPDK target directory):
|
||||
|
||||
.. code-block:: console
|
||||
This module should be loaded using the insmod command as shown below (assuming that the current directory is the DPDK target directory)::
|
||||
|
||||
sudo insmod kmod/rte_dom0_mm.ko rsv_memsize=X
|
||||
|
||||
@ -278,19 +248,15 @@ Configuring Memory for DPDK Use
|
||||
|
||||
After the rte_dom0_mm.ko kernel module has been loaded, the user must configure the memory size for DPDK usage.
|
||||
This is done by echoing the memory size to a memsize file in the /sys/devices/ directory.
|
||||
Use the following command (assuming that 2048 MB is required):
|
||||
|
||||
.. code-block:: console
|
||||
Use the following command (assuming that 2048 MB is required)::
|
||||
|
||||
echo 2048 > /sys/kernel/mm/dom0-mm/memsize-mB/memsize
|
||||
|
||||
The user can also check how much memory has already been used:
|
||||
|
||||
.. code-block:: console
|
||||
The user can also check how much memory has already been used::
|
||||
|
||||
cat /sys/kernel/mm/dom0-mm/memsize-mB/memsize_rsvd
|
||||
|
||||
Xen Domain0 does not support NUMA configuration, as a result the --socket-mem command line option is invalid for Xen Domain0.
|
||||
Xen Domain0 does not support NUMA configuration, as a result the ``--socket-mem`` command line option is invalid for Xen Domain0.
|
||||
|
||||
.. note::
|
||||
|
||||
@ -299,4 +265,4 @@ Xen Domain0 does not support NUMA configuration, as a result the --socket-mem co
|
||||
Running the DPDK Application on Xen Domain0
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
To run the DPDK application on Xen Domain0, an extra command line option --xen-dom0 is required.
|
||||
To run the DPDK application on Xen Domain0, an extra command line option ``--xen-dom0`` is required.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user