d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

doc: update thunderx driver guide

Browse Source 
Updating the thunderx with loopback port support,
debugging options, new features added and correcting
some formatting issues.

Signed-off-by: Harman Kalra <hkalra@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
This commit is contained in:
Harman Kalra 2020-02-07 15:32:48 +05:30 committed by Ferruh Yigit
parent 21e59432df
commit 5d24d6a550
1 changed files with 87 additions and 64 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

151
doc/guides/nics/thunderx.rst
View File

@ -25,6 +25,7 @@ Features of the ThunderX PMD are:
- Port hardware statistics - Port hardware statistics
- Jumbo frames - Jumbo frames
- Link state information - Link state information
- Setting up link state.
- Scattered and gather for TX and RX - Scattered and gather for TX and RX
- VLAN stripping - VLAN stripping
- SR-IOV VF - SR-IOV VF
@ -242,6 +243,12 @@ driver' list, secondary VFs are on the remaining on the remaining part of the li
Depending on the hardware used, the kernel driver sets a threshold ``vf_id``. VFs that try to attached with an id below or equal to Depending on the hardware used, the kernel driver sets a threshold ``vf_id``. VFs that try to attached with an id below or equal to
this boundary are considered primary VFs. VFs that try to attach with an id above this boundary are considered secondary VFs. this boundary are considered primary VFs. VFs that try to attach with an id above this boundary are considered secondary VFs.
LBK HW Access
~~~~~~~~~~~~~
Loopback HW Unit (LBK) receives packets from NIC-RX and sends packets back to NIC-TX.
The loopback block has N channels and contains data buffering that is shared across
all channels. Four primary VFs are reserved as loopback ports.
Example device binding Example device binding
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
@ -263,36 +270,40 @@ on a non-NUMA machine.
Network devices using kernel driver Network devices using kernel driver
=================================== ===================================
0000:01:10.0 'Device a026' if= drv=thunder-BGX unused=vfio-pci,uio_pci_generic 0000:01:10.0 'THUNDERX BGX (Common Ethernet Interface) a026' if= drv=thunder-BGX unused=vfio-pci
0000:01:10.1 'Device a026' if= drv=thunder-BGX unused=vfio-pci,uio_pci_generic 0000:01:10.1 'THUNDERX BGX (Common Ethernet Interface) a026' if= drv=thunder-BGX unused=vfio-pci
0002:01:00.0 'Device a01e' if= drv=thunder-nic unused=vfio-pci,uio_pci_generic 0001:01:00.0 'THUNDERX Network Interface Controller a01e' if= drv=thunder-nic unused=vfio-pci
0002:01:00.1 'Device 0011' if=eth0 drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:00.1 'Device a034' if=eth0 drv=thunder-nicvf unused=vfio-pci
0002:01:00.2 'Device 0011' if=eth1 drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:00.2 'Device a034' if=eth1 drv=thunder-nicvf unused=vfio-pci
0002:01:00.3 'Device 0011' if=eth2 drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:00.3 'Device a034' if=eth2 drv=thunder-nicvf unused=vfio-pci
0002:01:00.4 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:00.4 'Device a034' if=eth3 drv=thunder-nicvf unused=vfio-pci
0002:01:00.5 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:00.5 'Device a034' if=eth4 drv=thunder-nicvf unused=vfio-pci
0002:01:00.6 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:00.6 'Device a034' if=lbk0 drv=thunder-nicvf unused=vfio-pci
0002:01:00.7 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:00.7 'Device a034' if=lbk1 drv=thunder-nicvf unused=vfio-pci
0002:01:01.0 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.0 'Device a034' if=lbk2 drv=thunder-nicvf unused=vfio-pci
0002:01:01.1 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.1 'Device a034' if=lbk3 drv=thunder-nicvf unused=vfio-pci
0002:01:01.2 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.2 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:01.3 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.3 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:01.4 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.4 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:01.5 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.5 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:01.6 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.6 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:01.7 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:01.7 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:02.0 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:02.0 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:02.1 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:02.1 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
0002:01:02.2 'Device 0011' if= drv=thunder-nicvf unused=vfio-pci,uio_pci_generic 0001:01:02.2 'Device a034' if= drv=thunder-nicvf unused=vfio-pci
Other network devices Other network devices
===================== =====================
0002:00:03.0 'Device a01f' unused=vfio-pci,uio_pci_generic 0002:00:03.0 'Device a01f' unused=vfio-pci,uio_pci_generic
.. note::
Here total no of primary VFs = 5 (variable, depends on no of ethernet ports present) + 4 (fixed, loopback ports).
Ethernet ports are indicated as `if=eth0` while loopback ports as `if=lbk0`.
We want to bind two physical interfaces with 24 queues each device, we attach two primary VFs We want to bind two physical interfaces with 24 queues each device, we attach two primary VFs
and four secondary queues. In our example we choose two 10G interfaces eth1 (0002:01:00.2) and eth2 (0002:01:00.3). and four secondary VFs. In our example we choose two 10G interfaces eth1 (0002:01:00.2) and eth2 (0002:01:00.3).
We will choose four secondary queue sets from the ending of the list (0002:01:01.7-0002:01:02.2). We will choose four secondary queue sets from the ending of the list (0001:01:01.2-0002:01:02.2).
#. Bind two primary VFs to the ``vfio-pci`` driver: #. Bind two primary VFs to the ``vfio-pci`` driver:
@ -313,56 +324,68 @@ We will choose four secondary queue sets from the ending of the list (0002:01:01
The nicvf thunderx driver will make use of attached secondary VFs automatically during the interface configuration stage. The nicvf thunderx driver will make use of attached secondary VFs automatically during the interface configuration stage.
driver thunder-nic VF's Thunder-nic VF's
~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~
Use sysfs to distinguish thunder-nic primary VFs and secondary VFs. Use sysfs to distinguish thunder-nic primary VFs and secondary VFs.
.. code-block:: console .. code-block:: console
ll /sys/bus/pci/drivers/thunder-nic/ ls -l /sys/bus/pci/drivers/thunder-nic/
total 0 total 0
drwxr-xr-x 2 root root 0 Jan 22 11:19 ./ drwxr-xr-x 2 root root 0 Jan 22 11:19 ./
drwxr-xr-x 86 root root 0 Jan 22 11:07 ../ drwxr-xr-x 86 root root 0 Jan 22 11:07 ../
lrwxrwxrwx 1 root root 0 Jan 22 11:19 0001:01:00.0 -> '../../../../devices/platform/soc@0/849000000000.pci/pci0001:00/0001:00:10.0/0001:01:00.0'/ lrwxrwxrwx 1 root root 0 Jan 22 11:19 0001:01:00.0 -> '../../../../devices/platform/soc@0/849000000000.pci/pci0001:00/0001:00:10.0/0001:01:00.0'/
cat /sys/bus/pci/drivers/thunder-nic/0001\:01\:00.0/sriov_sqs_assignment .. code-block:: console
12
0 0001:01:00.1 vfio-pci +: 12 13 cat /sys/bus/pci/drivers/thunder-nic/0001\:01\:00.0/sriov_sqs_assignment
1 0001:01:00.2 thunder-nicvf -: 12
2 0001:01:00.3 thunder-nicvf -: 0 0001:01:00.1 vfio-pci +: 12 13
3 0001:01:00.4 thunder-nicvf -: 1 0001:01:00.2 thunder-nicvf -:
4 0001:01:00.5 thunder-nicvf -: 2 0001:01:00.3 thunder-nicvf -:
5 0001:01:00.6 thunder-nicvf -: 3 0001:01:00.4 thunder-nicvf -:
6 0001:01:00.7 thunder-nicvf -: 4 0001:01:00.5 thunder-nicvf -:
7 0001:01:01.0 thunder-nicvf -: 5 0001:01:00.6 thunder-nicvf -:
8 0001:01:01.1 thunder-nicvf -: 6 0001:01:00.7 thunder-nicvf -:
9 0001:01:01.2 thunder-nicvf -: 7 0001:01:01.0 thunder-nicvf -:
10 0001:01:01.3 thunder-nicvf -: 8 0001:01:01.1 thunder-nicvf -:
11 0001:01:01.4 thunder-nicvf -: 9 0001:01:01.2 thunder-nicvf -:
12 0001:01:01.5 vfio-pci: 0 10 0001:01:01.3 thunder-nicvf -:
13 0001:01:01.6 vfio-pci: 0 11 0001:01:01.4 thunder-nicvf -:
14 0001:01:01.7 thunder-nicvf: 255 12 0001:01:01.5 vfio-pci: 0
15 0001:01:02.0 thunder-nicvf: 255 13 0001:01:01.6 vfio-pci: 0
16 0001:01:02.1 thunder-nicvf: 255 14 0001:01:01.7 thunder-nicvf: 255
17 0001:01:02.2 thunder-nicvf: 255 15 0001:01:02.0 thunder-nicvf: 255
18 0001:01:02.3 thunder-nicvf: 255 16 0001:01:02.1 thunder-nicvf: 255
19 0001:01:02.4 thunder-nicvf: 255 17 0001:01:02.2 thunder-nicvf: 255
20 0001:01:02.5 thunder-nicvf: 255 18 0001:01:02.3 thunder-nicvf: 255
21 0001:01:02.6 thunder-nicvf: 255 19 0001:01:02.4 thunder-nicvf: 255
22 0001:01:02.7 thunder-nicvf: 255 20 0001:01:02.5 thunder-nicvf: 255
23 0001:01:03.0 thunder-nicvf: 255 21 0001:01:02.6 thunder-nicvf: 255
24 0001:01:03.1 thunder-nicvf: 255 22 0001:01:02.7 thunder-nicvf: 255
25 0001:01:03.2 thunder-nicvf: 255 23 0001:01:03.0 thunder-nicvf: 255
26 0001:01:03.3 thunder-nicvf: 255 24 0001:01:03.1 thunder-nicvf: 255
27 0001:01:03.4 thunder-nicvf: 255 25 0001:01:03.2 thunder-nicvf: 255
28 0001:01:03.5 thunder-nicvf: 255 26 0001:01:03.3 thunder-nicvf: 255
29 0001:01:03.6 thunder-nicvf: 255 27 0001:01:03.4 thunder-nicvf: 255
30 0001:01:03.7 thunder-nicvf: 255 28 0001:01:03.5 thunder-nicvf: 255
31 0001:01:04.0 thunder-nicvf: 255 29 0001:01:03.6 thunder-nicvf: 255
30 0001:01:03.7 thunder-nicvf: 255
31 0001:01:04.0 thunder-nicvf: 255
Every column that ends with 'thunder-nicvf: number' can be used as secondary VF. Every column that ends with 'thunder-nicvf: number' can be used as secondary VF.
In printout above all entres after '14 0001:01:01.7 thunder-nicvf: 255' can be used as secondary VF. In printout above all entres after '14 0001:01:01.7 thunder-nicvf: 255' can be used as secondary VF.
Debugging Options
-----------------
EAL command option to change log level
.. code-block:: console
--log-level=pmd.net.thunderx.driver:info
or
--log-level=pmd.net.thunderx.driver,7
Module params Module params
-------------- --------------