d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

doc: add l3fwd-graph application user guide

Browse Source 
Adding the user guide for l3fwd graph application.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
This commit is contained in:
Nithin Dabilpuram 2020-04-11 19:44:28 +05:30 committed by Thomas Monjalon
parent 4dc6d8e63c
commit cafe124259
5 changed files with 348 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
MAINTAINERS
View File

@ -1480,6 +1480,7 @@ F: doc/guides/prog_guide/graph_lib.rst
F: app/test/test_graph*
M: Nithin Dabilpuram <ndabilpuram@marvell.com>
F: examples/l3fwd-graph/
F: doc/guides/sample_app_ug/l3_forward_graph.rst
Nodes - EXPERIMENTAL
M: Nithin Dabilpuram <ndabilpuram@marvell.com>

8
doc/guides/rel_notes/release_20_05.rst
View File

@ -236,6 +236,14 @@ New Features
that can be configured through ``rte_node_ip4_rewrite_add`` function.
* Packet drop node: Frees the packets received to their respective mempool.
* **Added new l3fwd-graph sample application.**
Added an example application ``l3fwd-graph``. It demonstrates the usage of graph
library and node library for packet processing. In addition to the library usage
demonstration, this application can use for performance comparison with existing
``l3fwd`` (The static code without any nodes) with the modular ``l3fwd-graph``
approach.
Removed Items
-------------

1
doc/guides/sample_app_ug/index.rst
View File

@ -29,6 +29,7 @@ Sample Applications User Guides
l2_forward_event
l2_forward_cat
l3_forward
l3_forward_graph
l3_forward_power_man
l3_forward_access_ctrl
link_status_intr

4
doc/guides/sample_app_ug/intro.rst
View File

@ -54,6 +54,10 @@ examples are highlighted below.
forwarding, or ``l3fwd`` application does forwarding based on Internet
Protocol, IPv4 or IPv6 like a simple router.
* :doc:`Network Layer 3 forwarding Graph<l3_forward_graph>`: The Network Layer3
forwarding Graph, or ``l3fwd_graph`` application does forwarding based on IPv4
like a simple router with DPDK Graph framework.
* :doc:`Hardware packet copying<ioat>`: The Hardware packet copying,
or ``ioatfwd`` application demonstrates how to use IOAT rawdev driver for
copying packets between two threads.

334
doc/guides/sample_app_ug/l3_forward_graph.rst Normal file
View File

@ -0,0 +1,334 @@
.. SPDX-License-Identifier: BSD-3-Clause
Copyright(C) 2020 Marvell International Ltd.
L3 Forwarding Graph Sample Application
======================================
The L3 Forwarding Graph application is a simple example of packet processing
using the DPDK Graph framework. The application performs L3 forwarding using
Graph framework and nodes written for graph framework.
Overview
--------
The application demonstrates the use of the graph framework and graph nodes
``ethdev_rx``, ``ip4_lookup``, ``ip4_rewrite``, ``ethdev_tx`` and ``pkt_drop`` in DPDK to
implement packet forwarding.
The initialization is very similar to those of the :doc:`l3_forward`.
There is also additional initialization of graph for graph object creation
and configuration per lcore.
Run-time path is main thing that differs from L3 forwarding sample application.
Difference is that forwarding logic starting from Rx, followed by LPM lookup,
TTL update and finally Tx is implemented inside graph nodes. These nodes are
interconnected in graph framework. Application main loop needs to walk over
graph using ``rte_graph_walk()`` with graph objects created one per slave lcore.
The lookup method is as per implementation of ``ip4_lookup`` graph node.
The ID of the output interface for the input packet is the next hop returned by
the LPM lookup. The set of LPM rules used by the application is statically
configured and provided to ``ip4_lookup`` graph node and ``ip4_rewrite`` graph node
using node control API ``rte_node_ip4_route_add()`` and ``rte_node_ip4_rewrite_add()``.
In the sample application, only IPv4 forwarding is supported as of now.
Compiling the Application
-------------------------
To compile the sample application see :doc:`compiling`.
The application is located in the ``l3fwd-graph`` sub-directory.
Running the Application
-----------------------
The application has a number of command line options similar to l3fwd::
./l3fwd-graph [EAL options] -- -p PORTMASK
[-P]
--config(port,queue,lcore)[,(port,queue,lcore)]
[--eth-dest=X,MM:MM:MM:MM:MM:MM]
[--enable-jumbo [--max-pkt-len PKTLEN]]
[--no-numa]
[--per-port-pool]
Where,
* ``-p PORTMASK:`` Hexadecimal bitmask of ports to configure
* ``-P:`` Optional, sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address.
Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted.
* ``--config (port,queue,lcore)[,(port,queue,lcore)]:`` Determines which queues from which ports are mapped to which cores.
* ``--eth-dest=X,MM:MM:MM:MM:MM:MM:`` Optional, ethernet destination for port X.
* ``--enable-jumbo:`` Optional, enables jumbo frames.
* ``--max-pkt-len:`` Optional, under the premise of enabling jumbo, maximum packet length in decimal (64-9600).
* ``--no-numa:`` Optional, disables numa awareness.
* ``--per-port-pool:`` Optional, set to use independent buffer pools per port. Without this option, single buffer pool is used for all ports.
For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0,
while cores 8-15 and 24-31 appear on socket 1.
To enable L3 forwarding between two ports, assuming that both ports are in the same socket, using two cores, cores 1 and 2,
(which are in the same socket too), use the following command:
.. code-block:: console
./build/l3fwd-graph -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)"
In this command:
* The -l option enables cores 1, 2
* The -p option enables ports 0 and 1
* The --config option enables one queue on each port and maps each (port,queue) pair to a specific core.
The following table shows the mapping in this example:
+----------+-----------+-----------+-------------------------------------+
| **Port** | **Queue** | **lcore** | **Description** |
| | | | |
+----------+-----------+-----------+-------------------------------------+
| 0 | 0 | 1 | Map queue 0 from port 0 to lcore 1. |
| | | | |
+----------+-----------+-----------+-------------------------------------+
| 1 | 0 | 2 | Map queue 0 from port 1 to lcore 2. |
| | | | |
+----------+-----------+-----------+-------------------------------------+
Refer to the *DPDK Getting Started Guide* for general information on running applications and
the Environment Abstraction Layer (EAL) options.
.. _l3_fwd_graph_explanation:
Explanation
-----------
The following sections provide some explanation of the sample application code.
As mentioned in the overview section, the initialization is similar to that of
the :doc:`l3_forward`. Run-time path though similar in functionality to that of
:doc:`l3_forward`, major part of the implementation is in graph nodes via used
via ``librte_node`` library.
The following sections describe aspects that are specific to the L3 Forwarding
Graph sample application.
Graph Node Pre-Init Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
After device configuration and device Rx, Tx queue setup is complete,
a minimal config of port id, num_rx_queues, num_tx_queues, mempools etc will
be passed to *ethdev_** node ctrl API ``rte_node_eth_config()``. This will be
lead to the clone of ``ethdev_rx`` and ``ethdev_tx`` nodes as ``ethdev_rx-X-Y`` and
``ethdev_tx-X`` where X, Y represent port id and queue id associated with them.
In case of ``ethdev_tx-X`` nodes, tx queue id assigned per instance of the node
is same as graph id.
These cloned nodes along with existing static nodes such as ``ip4_lookup`` and
``ip4_rewrite`` will be used in graph creation to associate node's to lcore
specific graph object.
.. code-block:: c
RTE_ETH_FOREACH_DEV(portid)
{
/* ... */
ret = rte_eth_dev_configure(portid, nb_rx_queue,
n_tx_queue, &local_port_conf);
/* ... */
/* Init one TX queue per couple (lcore,port) */
queueid = 0;
for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
/* ... */
ret = rte_eth_tx_queue_setup(portid, queueid, nb_txd,
socketid, txconf);
/* ... */
queueid++;
}
/* Setup ethdev node config */
ethdev_conf[nb_conf].port_id = portid;
ethdev_conf[nb_conf].num_rx_queues = nb_rx_queue;
ethdev_conf[nb_conf].num_tx_queues = n_tx_queue;
if (!per_port_pool)
ethdev_conf[nb_conf].mp = pktmbuf_pool[0];
else
ethdev_conf[nb_conf].mp = pktmbuf_pool[portid];
ethdev_conf[nb_conf].mp_count = NB_SOCKETS;
nb_conf++;
printf("\n");
}
for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
/* Init RX queues */
for (queue = 0; queue < qconf->n_rx_queue; ++queue) {
/* ... */
if (!per_port_pool)
ret = rte_eth_rx_queue_setup(portid, queueid, nb_rxd, socketid,
&rxq_conf, pktmbuf_pool[0][socketid]);
else
ret = rte_eth_rx_queue_setup(portid, queueid, nb_rxd, socketid,
&rxq_conf, pktmbuf_pool[portid][socketid]);
/* ... */
}
}
/* Ethdev node config, skip rx queue mapping */
ret = rte_node_eth_config(ethdev_conf, nb_conf, nb_graphs);
Graph Initialization
~~~~~~~~~~~~~~~~~~~~
Now a graph needs to be created with a specific set of nodes for every lcore.
A graph object returned after graph creation is a per lcore object and
cannot be shared between lcores. Since ``ethdev_tx-X`` node is per port node,
it can be associated with all the graphs created as all the lcores should have
Tx capability for every port. But ``ethdev_rx-X-Y`` node is created per
(port, rx_queue_id), so they should be associated with a graph based on
the application argument ``--config`` specifying rx queue mapping to lcore.
.. note::
The Graph creation will fail if the passed set of shell node pattern's
are not sufficient to meet their inter-dependency or even one node is not
found with a given regex node pattern.
.. code-block:: c
static const char *const default_patterns[] = {
"ip4*",
"ethdev_tx-*",
"pkt_drop",
};
const char **node_patterns;
uint16_t nb_pattern;
/* ... */
/* Create a graph object per lcore with common nodes and
* lcore specific nodes based on application arguments
*/
nb_patterns = RTE_DIM(default_patterns);
node_patterns = malloc((MAX_RX_QUEUE_PER_LCORE + nb_patterns) *
sizeof(*node_patterns));
memcpy(node_patterns, default_patterns,
nb_patterns * sizeof(*node_patterns));
memset(&graph_conf, 0, sizeof(graph_conf));
/* Common set of nodes in every lcore's graph object */
graph_conf.node_patterns = node_patterns;
for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
/* ... */
/* Skip graph creation if no source exists */
if (!qconf->n_rx_queue)
continue;
/* Add rx node patterns of this lcore based on --config */
for (i = 0; i < qconf->n_rx_queue; i++) {
graph_conf.node_patterns[nb_patterns + i] =
qconf->rx_queue_list[i].node_name;
}
graph_conf.nb_node_patterns = nb_patterns + i;
graph_conf.socket_id = rte_lcore_to_socket_id(lcore_id);
snprintf(qconf->name, sizeof(qconf->name), "worker_%u", lcore_id);
graph_id = rte_graph_create(qconf->name, &graph_conf);
/* ... */
qconf->graph = rte_graph_lookup(qconf->name);
/* ... */
}
Forwarding data(Route, Next-Hop) addition
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Once graph objects are created, node specific info like routes and rewrite
headers will be provided run-time using ``rte_node_ip4_route_add()`` and
``rte_node_ip4_rewrite_add()`` API.
.. note::
Since currently ``ip4_lookup`` and ``ip4_rewrite`` nodes don't support
lock-less mechanisms(RCU, etc) to add run-time forwarding data like route and
rewrite data, forwarding data is added before packet processing loop is
launched on slave lcore.
.. code-block:: c
/* Add route to ip4 graph infra */
for (i = 0; i < IPV4_L3FWD_LPM_NUM_ROUTES; i++) {
/* ... */
dst_port = ipv4_l3fwd_lpm_route_array[i].if_out;
next_hop = i;
/* ... */
ret = rte_node_ip4_route_add(ipv4_l3fwd_lpm_route_array[i].ip,
ipv4_l3fwd_lpm_route_array[i].depth, next_hop,
RTE_NODE_IP4_LOOKUP_NEXT_REWRITE);
/* ... */
memcpy(rewrite_data, val_eth + dst_port, rewrite_len);
/* Add next hop for a given destination */
ret = rte_node_ip4_rewrite_add(next_hop, rewrite_data,
rewrite_len, dst_port);
RTE_LOG(INFO, L3FWD_GRAPH, "Added route %s, next_hop %u\n",
route_str, next_hop);
}
Packet Forwarding using Graph Walk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now that all the device configurations are done, graph creations are done and
forwarding data is updated with nodes, slave lcores will be launched with graph
main loop. Graph main loop is very simple in the sense that it needs to
continuously call a non-blocking API ``rte_graph_walk()`` with it's lcore
specific graph object that was already created.
.. note::
rte_graph_walk() will walk over all the sources nodes i.e ``ethdev_rx-X-Y``
associated with a given graph and Rx the available packets and enqueue them
to the following node ``ip4_lookup`` which then will enqueue them to ``ip4_rewrite``
node if LPM lookup succeeds. ``ip4_rewrite`` node then will update Ethernet header
as per next-hop data and transmit the packet via port 'Z' by enqueuing
to ``ethdev_tx-Z`` node instance in its graph object.
.. code-block:: c
/* Main processing loop */
static int
graph_main_loop(void *conf)
{
// ...
lcore_id = rte_lcore_id();
qconf = &lcore_conf[lcore_id];
graph = qconf->graph;
RTE_LOG(INFO, L3FWD_GRAPH,
"Entering main loop on lcore %u, graph %s(%p)\n", lcore_id,
qconf->name, graph);
/* Walk over graph until signal to quit */
while (likely(!force_quit))
rte_graph_walk(graph);
return 0;
}