e2a94f9ad3
While make has been deprecated for DPDK, it's still applicable for some example apps to be built standalone, this patch adjusts the guides to take that into consideration. Signed-off-by: Ciara Power <ciara.power@intel.com> Acked-by: Nicolas Chautru <nicolas.chautru@intel.com> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
335 lines
13 KiB
ReStructuredText
335 lines
13 KiB
ReStructuredText
.. SPDX-License-Identifier: BSD-3-Clause
|
|
Copyright(C) 2020 Marvell International Ltd.
|
|
|
|
L3 Forwarding Graph Sample Application
|
|
======================================
|
|
|
|
The L3 Forwarding Graph application is a simple example of packet processing
|
|
using the DPDK Graph framework. The application performs L3 forwarding using
|
|
Graph framework and nodes written for graph framework.
|
|
|
|
Overview
|
|
--------
|
|
|
|
The application demonstrates the use of the graph framework and graph nodes
|
|
``ethdev_rx``, ``ip4_lookup``, ``ip4_rewrite``, ``ethdev_tx`` and ``pkt_drop`` in DPDK to
|
|
implement packet forwarding.
|
|
|
|
The initialization is very similar to those of the :doc:`l3_forward`.
|
|
There is also additional initialization of graph for graph object creation
|
|
and configuration per lcore.
|
|
Run-time path is main thing that differs from L3 forwarding sample application.
|
|
Difference is that forwarding logic starting from Rx, followed by LPM lookup,
|
|
TTL update and finally Tx is implemented inside graph nodes. These nodes are
|
|
interconnected in graph framework. Application main loop needs to walk over
|
|
graph using ``rte_graph_walk()`` with graph objects created one per worker lcore.
|
|
|
|
The lookup method is as per implementation of ``ip4_lookup`` graph node.
|
|
The ID of the output interface for the input packet is the next hop returned by
|
|
the LPM lookup. The set of LPM rules used by the application is statically
|
|
configured and provided to ``ip4_lookup`` graph node and ``ip4_rewrite`` graph node
|
|
using node control API ``rte_node_ip4_route_add()`` and ``rte_node_ip4_rewrite_add()``.
|
|
|
|
In the sample application, only IPv4 forwarding is supported as of now.
|
|
|
|
Compiling the Application
|
|
-------------------------
|
|
|
|
To compile the sample application see :doc:`compiling`.
|
|
|
|
The application is located in the ``l3fwd-graph`` sub-directory.
|
|
|
|
Running the Application
|
|
-----------------------
|
|
|
|
The application has a number of command line options similar to l3fwd::
|
|
|
|
./dpdk-l3fwd-graph [EAL options] -- -p PORTMASK
|
|
[-P]
|
|
--config(port,queue,lcore)[,(port,queue,lcore)]
|
|
[--eth-dest=X,MM:MM:MM:MM:MM:MM]
|
|
[--enable-jumbo [--max-pkt-len PKTLEN]]
|
|
[--no-numa]
|
|
[--per-port-pool]
|
|
|
|
Where,
|
|
|
|
* ``-p PORTMASK:`` Hexadecimal bitmask of ports to configure
|
|
|
|
* ``-P:`` Optional, sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address.
|
|
Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted.
|
|
|
|
* ``--config (port,queue,lcore)[,(port,queue,lcore)]:`` Determines which queues from which ports are mapped to which cores.
|
|
|
|
* ``--eth-dest=X,MM:MM:MM:MM:MM:MM:`` Optional, ethernet destination for port X.
|
|
|
|
* ``--enable-jumbo:`` Optional, enables jumbo frames.
|
|
|
|
* ``--max-pkt-len:`` Optional, under the premise of enabling jumbo, maximum packet length in decimal (64-9600).
|
|
|
|
* ``--no-numa:`` Optional, disables numa awareness.
|
|
|
|
* ``--per-port-pool:`` Optional, set to use independent buffer pools per port. Without this option, single buffer pool is used for all ports.
|
|
|
|
For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0,
|
|
while cores 8-15 and 24-31 appear on socket 1.
|
|
|
|
To enable L3 forwarding between two ports, assuming that both ports are in the same socket, using two cores, cores 1 and 2,
|
|
(which are in the same socket too), use the following command:
|
|
|
|
.. code-block:: console
|
|
|
|
./<build_dir>/examples/dpdk-l3fwd-graph -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)"
|
|
|
|
In this command:
|
|
|
|
* The -l option enables cores 1, 2
|
|
|
|
* The -p option enables ports 0 and 1
|
|
|
|
* The --config option enables one queue on each port and maps each (port,queue) pair to a specific core.
|
|
The following table shows the mapping in this example:
|
|
|
|
+----------+-----------+-----------+-------------------------------------+
|
|
| **Port** | **Queue** | **lcore** | **Description** |
|
|
| | | | |
|
|
+----------+-----------+-----------+-------------------------------------+
|
|
| 0 | 0 | 1 | Map queue 0 from port 0 to lcore 1. |
|
|
| | | | |
|
|
+----------+-----------+-----------+-------------------------------------+
|
|
| 1 | 0 | 2 | Map queue 0 from port 1 to lcore 2. |
|
|
| | | | |
|
|
+----------+-----------+-----------+-------------------------------------+
|
|
|
|
Refer to the *DPDK Getting Started Guide* for general information on running applications and
|
|
the Environment Abstraction Layer (EAL) options.
|
|
|
|
.. _l3_fwd_graph_explanation:
|
|
|
|
Explanation
|
|
-----------
|
|
|
|
The following sections provide some explanation of the sample application code.
|
|
As mentioned in the overview section, the initialization is similar to that of
|
|
the :doc:`l3_forward`. Run-time path though similar in functionality to that of
|
|
:doc:`l3_forward`, major part of the implementation is in graph nodes via used
|
|
via ``librte_node`` library.
|
|
The following sections describe aspects that are specific to the L3 Forwarding
|
|
Graph sample application.
|
|
|
|
Graph Node Pre-Init Configuration
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
After device configuration and device Rx, Tx queue setup is complete,
|
|
a minimal config of port id, num_rx_queues, num_tx_queues, mempools etc will
|
|
be passed to *ethdev_** node ctrl API ``rte_node_eth_config()``. This will be
|
|
lead to the clone of ``ethdev_rx`` and ``ethdev_tx`` nodes as ``ethdev_rx-X-Y`` and
|
|
``ethdev_tx-X`` where X, Y represent port id and queue id associated with them.
|
|
In case of ``ethdev_tx-X`` nodes, tx queue id assigned per instance of the node
|
|
is same as graph id.
|
|
|
|
These cloned nodes along with existing static nodes such as ``ip4_lookup`` and
|
|
``ip4_rewrite`` will be used in graph creation to associate node's to lcore
|
|
specific graph object.
|
|
|
|
.. code-block:: c
|
|
|
|
RTE_ETH_FOREACH_DEV(portid)
|
|
{
|
|
|
|
/* ... */
|
|
ret = rte_eth_dev_configure(portid, nb_rx_queue,
|
|
n_tx_queue, &local_port_conf);
|
|
/* ... */
|
|
|
|
/* Init one TX queue per couple (lcore,port) */
|
|
queueid = 0;
|
|
for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
|
|
/* ... */
|
|
ret = rte_eth_tx_queue_setup(portid, queueid, nb_txd,
|
|
socketid, txconf);
|
|
/* ... */
|
|
queueid++;
|
|
}
|
|
|
|
/* Setup ethdev node config */
|
|
ethdev_conf[nb_conf].port_id = portid;
|
|
ethdev_conf[nb_conf].num_rx_queues = nb_rx_queue;
|
|
ethdev_conf[nb_conf].num_tx_queues = n_tx_queue;
|
|
if (!per_port_pool)
|
|
ethdev_conf[nb_conf].mp = pktmbuf_pool[0];
|
|
else
|
|
ethdev_conf[nb_conf].mp = pktmbuf_pool[portid];
|
|
ethdev_conf[nb_conf].mp_count = NB_SOCKETS;
|
|
|
|
nb_conf++;
|
|
printf("\n");
|
|
}
|
|
|
|
for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
|
|
/* Init RX queues */
|
|
for (queue = 0; queue < qconf->n_rx_queue; ++queue) {
|
|
/* ... */
|
|
if (!per_port_pool)
|
|
ret = rte_eth_rx_queue_setup(portid, queueid, nb_rxd, socketid,
|
|
&rxq_conf, pktmbuf_pool[0][socketid]);
|
|
else
|
|
ret = rte_eth_rx_queue_setup(portid, queueid, nb_rxd, socketid,
|
|
&rxq_conf, pktmbuf_pool[portid][socketid]);
|
|
/* ... */
|
|
}
|
|
}
|
|
|
|
/* Ethdev node config, skip rx queue mapping */
|
|
ret = rte_node_eth_config(ethdev_conf, nb_conf, nb_graphs);
|
|
|
|
Graph Initialization
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Now a graph needs to be created with a specific set of nodes for every lcore.
|
|
A graph object returned after graph creation is a per lcore object and
|
|
cannot be shared between lcores. Since ``ethdev_tx-X`` node is per port node,
|
|
it can be associated with all the graphs created as all the lcores should have
|
|
Tx capability for every port. But ``ethdev_rx-X-Y`` node is created per
|
|
(port, rx_queue_id), so they should be associated with a graph based on
|
|
the application argument ``--config`` specifying rx queue mapping to lcore.
|
|
|
|
.. note::
|
|
|
|
The Graph creation will fail if the passed set of shell node pattern's
|
|
are not sufficient to meet their inter-dependency or even one node is not
|
|
found with a given regex node pattern.
|
|
|
|
.. code-block:: c
|
|
|
|
static const char *const default_patterns[] = {
|
|
"ip4*",
|
|
"ethdev_tx-*",
|
|
"pkt_drop",
|
|
};
|
|
const char **node_patterns;
|
|
uint16_t nb_pattern;
|
|
|
|
/* ... */
|
|
|
|
/* Create a graph object per lcore with common nodes and
|
|
* lcore specific nodes based on application arguments
|
|
*/
|
|
nb_patterns = RTE_DIM(default_patterns);
|
|
node_patterns = malloc((MAX_RX_QUEUE_PER_LCORE + nb_patterns) *
|
|
sizeof(*node_patterns));
|
|
memcpy(node_patterns, default_patterns,
|
|
nb_patterns * sizeof(*node_patterns));
|
|
|
|
memset(&graph_conf, 0, sizeof(graph_conf));
|
|
|
|
/* Common set of nodes in every lcore's graph object */
|
|
graph_conf.node_patterns = node_patterns;
|
|
|
|
for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
|
|
/* ... */
|
|
|
|
/* Skip graph creation if no source exists */
|
|
if (!qconf->n_rx_queue)
|
|
continue;
|
|
|
|
/* Add rx node patterns of this lcore based on --config */
|
|
for (i = 0; i < qconf->n_rx_queue; i++) {
|
|
graph_conf.node_patterns[nb_patterns + i] =
|
|
qconf->rx_queue_list[i].node_name;
|
|
}
|
|
|
|
graph_conf.nb_node_patterns = nb_patterns + i;
|
|
graph_conf.socket_id = rte_lcore_to_socket_id(lcore_id);
|
|
|
|
snprintf(qconf->name, sizeof(qconf->name), "worker_%u", lcore_id);
|
|
|
|
graph_id = rte_graph_create(qconf->name, &graph_conf);
|
|
|
|
/* ... */
|
|
|
|
qconf->graph = rte_graph_lookup(qconf->name);
|
|
|
|
/* ... */
|
|
}
|
|
|
|
Forwarding data(Route, Next-Hop) addition
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Once graph objects are created, node specific info like routes and rewrite
|
|
headers will be provided run-time using ``rte_node_ip4_route_add()`` and
|
|
``rte_node_ip4_rewrite_add()`` API.
|
|
|
|
.. note::
|
|
|
|
Since currently ``ip4_lookup`` and ``ip4_rewrite`` nodes don't support
|
|
lock-less mechanisms(RCU, etc) to add run-time forwarding data like route and
|
|
rewrite data, forwarding data is added before packet processing loop is
|
|
launched on worker lcore.
|
|
|
|
.. code-block:: c
|
|
|
|
/* Add route to ip4 graph infra */
|
|
for (i = 0; i < IPV4_L3FWD_LPM_NUM_ROUTES; i++) {
|
|
/* ... */
|
|
|
|
dst_port = ipv4_l3fwd_lpm_route_array[i].if_out;
|
|
next_hop = i;
|
|
|
|
/* ... */
|
|
ret = rte_node_ip4_route_add(ipv4_l3fwd_lpm_route_array[i].ip,
|
|
ipv4_l3fwd_lpm_route_array[i].depth, next_hop,
|
|
RTE_NODE_IP4_LOOKUP_NEXT_REWRITE);
|
|
|
|
/* ... */
|
|
|
|
memcpy(rewrite_data, val_eth + dst_port, rewrite_len);
|
|
|
|
/* Add next hop for a given destination */
|
|
ret = rte_node_ip4_rewrite_add(next_hop, rewrite_data,
|
|
rewrite_len, dst_port);
|
|
|
|
RTE_LOG(INFO, L3FWD_GRAPH, "Added route %s, next_hop %u\n",
|
|
route_str, next_hop);
|
|
}
|
|
|
|
Packet Forwarding using Graph Walk
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Now that all the device configurations are done, graph creations are done and
|
|
forwarding data is updated with nodes, worker lcores will be launched with graph
|
|
main loop. Graph main loop is very simple in the sense that it needs to
|
|
continuously call a non-blocking API ``rte_graph_walk()`` with it's lcore
|
|
specific graph object that was already created.
|
|
|
|
.. note::
|
|
|
|
rte_graph_walk() will walk over all the sources nodes i.e ``ethdev_rx-X-Y``
|
|
associated with a given graph and Rx the available packets and enqueue them
|
|
to the following node ``ip4_lookup`` which then will enqueue them to ``ip4_rewrite``
|
|
node if LPM lookup succeeds. ``ip4_rewrite`` node then will update Ethernet header
|
|
as per next-hop data and transmit the packet via port 'Z' by enqueuing
|
|
to ``ethdev_tx-Z`` node instance in its graph object.
|
|
|
|
.. code-block:: c
|
|
|
|
/* Main processing loop */
|
|
static int
|
|
graph_main_loop(void *conf)
|
|
{
|
|
// ...
|
|
|
|
lcore_id = rte_lcore_id();
|
|
qconf = &lcore_conf[lcore_id];
|
|
graph = qconf->graph;
|
|
|
|
RTE_LOG(INFO, L3FWD_GRAPH,
|
|
"Entering main loop on lcore %u, graph %s(%p)\n", lcore_id,
|
|
qconf->name, graph);
|
|
|
|
/* Walk over graph until signal to quit */
|
|
while (likely(!force_quit))
|
|
rte_graph_walk(graph);
|
|
return 0;
|
|
}
|