d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

examples/multi_process: remove l2fwd fork example

Browse Source 
l2fwd_fork relies on a multiprocess model that DPDK does not support
(calling rte_eal_init() before fork()), in particular in light of recent
EAL changes like the multiproess communication channel.

This example can mislead users into thinking this is a supported
multiprocess model; hence, this commit removes this example and the
corresponding user guide documentation as well.

This patch was made following this mailing list discussion:
http://mails.dpdk.org/archives/dev/2018-July/108106.html

Signed-off-by: Gage Eads <gage.eads@intel.com>
This commit is contained in:
Gage Eads 2018-07-30 10:10:00 -05:00 committed by Thomas Monjalon
parent bc57bef7a9
commit 3504db92b5
9 changed files with 0 additions and 2091 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
doc/guides/sample_app_ug/img/master_slave_proc.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 191 KiB

BIN
doc/guides/sample_app_ug/img/slave_proc_recov.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 83 KiB

4
doc/guides/sample_app_ug/index.rst
View File

@ -83,10 +83,6 @@ Sample Applications User Guides
:numref:`figure_client_svr_sym_multi_proc_app` :ref:`figure_client_svr_sym_multi_proc_app`
:numref:`figure_master_slave_proc` :ref:`figure_master_slave_proc`
:numref:`figure_slave_proc_recov` :ref:`figure_slave_proc_recov`
:numref:`figure_qos_sched_app_arch` :ref:`figure_qos_sched_app_arch`
:numref:`figure_pipeline_overview` :ref:`figure_pipeline_overview`

406
doc/guides/sample_app_ug/multi_process.rst
View File

@ -321,409 +321,3 @@ In both the server and the client processes, outgoing packets are buffered befor
so as to allow the sending of multiple packets in a single burst to improve efficiency.
For example, the client process will buffer packets to send,
until either the buffer is full or until we receive no further packets from the server.
Master-slave Multi-process Example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The fourth example of DPDK multi-process support demonstrates a master-slave model that
provide the capability of application recovery if a slave process crashes or meets unexpected conditions.
In addition, it also demonstrates the floating process,
which can run among different cores in contrast to the traditional way of binding a process/thread to a specific CPU core,
using the local cache mechanism of mempool structures.
This application performs the same functionality as the L2 Forwarding sample application,
therefore this chapter does not cover that part but describes functionality that is introduced in this multi-process example only.
Please refer to :doc:`l2_forward_real_virtual` for more information.
Unlike previous examples where all processes are started from the command line with input arguments, in this example,
only one process is spawned from the command line and that process creates other processes.
The following section describes this in more detail.
Master-slave Process Models
^^^^^^^^^^^^^^^^^^^^^^^^^^^
The process spawned from the command line is called the *master process* in this document.
A process created by the master is called a *slave process*.
The application has only one master process, but could have multiple slave processes.
Once the master process begins to run, it tries to initialize all the resources such as
memory, CPU cores, driver, ports, and so on, as the other examples do.
Thereafter, it creates slave processes, as shown in the following figure.
.. _figure_master_slave_proc:
.. figure:: img/master_slave_proc.*
Master-slave Process Workflow
The master process calls the rte_eal_mp_remote_launch() EAL function to launch an application function for each pinned thread through the pipe.
Then, it waits to check if any slave processes have exited.
If so, the process tries to re-initialize the resources that belong to that slave and launch them in the pinned thread entry again.
The following section describes the recovery procedures in more detail.
For each pinned thread in EAL, after reading any data from the pipe, it tries to call the function that the application specified.
In this master specified function, a fork() call creates a slave process that performs the L2 forwarding task.
Then, the function waits until the slave exits, is killed or crashes. Thereafter, it notifies the master of this event and returns.
Finally, the EAL pinned thread waits until the new function is launched.
After discussing the master-slave model, it is necessary to mention another issue, global and static variables.
For multiple-thread cases, all global and static variables have only one copy and they can be accessed by any thread if applicable.
So, they can be used to sync or share data among threads.
In the previous examples, each process has separate global and static variables in memory and are independent of each other.
If it is necessary to share the knowledge, some communication mechanism should be deployed, such as, memzone, ring, shared memory, and so on.
The global or static variables are not a valid approach to share data among processes.
For variables in this example, on the one hand, the slave process inherits all the knowledge of these variables after being created by the master.
On the other hand, other processes cannot know if one or more processes modifies them after slave creation since that
is the nature of a multiple process address space.
But this does not mean that these variables cannot be used to share or sync data; it depends on the use case.
The following are the possible use cases:
#. The master process starts and initializes a variable and it will never be changed after slave processes created. This case is OK.
#. After the slave processes are created, the master or slave cores need to change a variable, but other processes do not need to know the change.
This case is also OK.
#. After the slave processes are created, the master or a slave needs to change a variable.
In the meantime, one or more other process needs to be aware of the change.
In this case, global and static variables cannot be used to share knowledge. Another communication mechanism is needed.
A simple approach without lock protection can be a heap buffer allocated by rte_malloc or mem zone.
Slave Process Recovery Mechanism
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Before talking about the recovery mechanism, it is necessary to know what is needed before a new slave instance can run if a previous one exited.
When a slave process exits, the system returns all the resources allocated for this process automatically.
However, this does not include the resources that were allocated by the DPDK. All the hardware resources are shared among the processes,
which include memzone, mempool, ring, a heap buffer allocated by the rte_malloc library, and so on.
If the new instance runs and the allocated resource is not returned, either resource allocation failed or the hardware resource is lost forever.
When a slave process runs, it may have dependencies on other processes.
They could have execution sequence orders; they could share the ring to communicate; they could share the same port for reception and forwarding;
they could use lock structures to do exclusive access in some critical path.
What happens to the dependent process(es) if the peer leaves?
The consequence are varied since the dependency cases are complex.
It depends on what the processed had shared.
However, it is necessary to notify the peer(s) if one slave exited.
Then, the peer(s) will be aware of that and wait until the new instance begins to run.
Therefore, to provide the capability to resume the new slave instance if the previous one exited, it is necessary to provide several mechanisms:
#. Keep a resource list for each slave process.
Before a slave process run, the master should prepare a resource list.
After it exits, the master could either delete the allocated resources and create new ones,
or re-initialize those for use by the new instance.
#. Set up a notification mechanism for slave process exit cases. After the specific slave leaves,
the master should be notified and then help to create a new instance.
This mechanism is provided in Section `Master-slave Process Models`_.
#. Use a synchronization mechanism among dependent processes.
The master should have the capability to stop or kill slave processes that have a dependency on the one that has exited.
Then, after the new instance of exited slave process begins to run, the dependency ones could resume or run from the start.
The example sends a STOP command to slave processes dependent on the exited one, then they will exit.
Thereafter, the master creates new instances for the exited slave processes.
The following diagram describes slave process recovery.
.. _figure_slave_proc_recov:
.. figure:: img/slave_proc_recov.*
Slave Process Recovery Process Flow
Floating Process Support
^^^^^^^^^^^^^^^^^^^^^^^^
When the DPDK application runs, there is always a -c option passed in to indicate the cores that are enabled.
Then, the DPDK creates a thread for each enabled core.
By doing so, it creates a 1:1 mapping between the enabled core and each thread.
The enabled core always has an ID, therefore, each thread has a unique core ID in the DPDK execution environment.
With the ID, each thread can easily access the structures or resources exclusively belonging to it without using function parameter passing.
It can easily use the rte_lcore_id() function to get the value in every function that is called.
For threads/processes not created in that way, either pinned to a core or not, they will not own a unique ID and the
rte_lcore_id() function will not work in the correct way.
However, sometimes these threads/processes still need the unique ID mechanism to do easy access on structures or resources.
For example, the DPDK mempool library provides a local cache mechanism
(refer to :ref:`mempool_local_cache`)
for fast element allocation and freeing.
If using a non-unique ID or a fake one,
a race condition occurs if two or more threads/ processes with the same core ID try to use the local cache.
Therefore, unused core IDs from the passing of parameters with the -c option are used to organize the core ID allocation array.
Once the floating process is spawned, it tries to allocate a unique core ID from the array and release it on exit.
A natural way to spawn a floating process is to use the fork() function and allocate a unique core ID from the unused core ID array.
However, it is necessary to write new code to provide a notification mechanism for slave exit
and make sure the process recovery mechanism can work with it.
To avoid producing redundant code, the Master-Slave process model is still used to spawn floating processes,
then cancel the affinity to specific cores.
Besides that, clear the core ID assigned to the DPDK spawning a thread that has a 1:1 mapping with the core mask.
Thereafter, get a new core ID from the unused core ID allocation array.
Run the Application
^^^^^^^^^^^^^^^^^^^
This example has a command line similar to the L2 Forwarding sample application with a few differences.
To run the application, start one copy of the l2fwd_fork binary in one terminal.
Unlike the L2 Forwarding example,
this example requires at least three cores since the master process will wait and be accountable for slave process recovery.
The command is as follows:
.. code-block:: console
#./build/l2fwd_fork -l 2-4 -n 4 -- -p 3 -f
This example provides another -f option to specify the use of floating process.
If not specified, the example will use a pinned process to perform the L2 forwarding task.
To verify the recovery mechanism, proceed as follows: First, check the PID of the slave processes:
.. code-block:: console
#ps -fe | grep l2fwd_fork
root 5136 4843 29 11:11 pts/1 00:00:05 ./build/l2fwd_fork
root 5145 5136 98 11:11 pts/1 00:00:11 ./build/l2fwd_fork
root 5146 5136 98 11:11 pts/1 00:00:11 ./build/l2fwd_fork
Then, kill one of the slaves:
.. code-block:: console
#kill -9 5145
After 1 or 2 seconds, check whether the slave has resumed:
.. code-block:: console
#ps -fe | grep l2fwd_fork
root 5136 4843 3 11:11 pts/1 00:00:06 ./build/l2fwd_fork
root 5247 5136 99 11:14 pts/1 00:00:01 ./build/l2fwd_fork
root 5248 5136 99 11:14 pts/1 00:00:01 ./build/l2fwd_fork
It can also monitor the traffic generator statics to see whether slave processes have resumed.
Explanation
^^^^^^^^^^^
As described in previous sections,
not all global and static variables need to change to be accessible in multiple processes;
it depends on how they are used.
In this example,
the statics info on packets dropped/forwarded/received count needs to be updated by the slave process,
and the master needs to see the update and print them out.
So, it needs to allocate a heap buffer using rte_zmalloc.
In addition, if the -f option is specified,
an array is needed to store the allocated core ID for the floating process so that the master can return it
after a slave has exited accidentally.
.. code-block:: c
static int
l2fwd_malloc_shared_struct(void)
{
port_statistics = rte_zmalloc("port_stat", sizeof(struct l2fwd_port_statistics) * RTE_MAX_ETHPORTS, 0);
if (port_statistics == NULL)
return -1;
/* allocate mapping_id array */
if (float_proc) {
int i;
mapping_id = rte_malloc("mapping_id", sizeof(unsigned) * RTE_MAX_LCORE, 0);
if (mapping_id == NULL)
return -1;
for (i = 0 ;i < RTE_MAX_LCORE; i++)
mapping_id[i] = INVALID_MAPPING_ID;
}
return 0;
}
For each slave process, packets are received from one port and forwarded to another port that another slave is operating on.
If the other slave exits accidentally, the port it is operating on may not work normally,
so the first slave cannot forward packets to that port.
There is a dependency on the port in this case. So, the master should recognize the dependency.
The following is the code to detect this dependency:
.. code-block:: c
for (portid = 0; portid < nb_ports; portid++) {
/* skip ports that are not enabled */
if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
continue;
/* Find pair ports' lcores */
find_lcore = find_pair_lcore = 0;
pair_port = l2fwd_dst_ports[portid];
for (i = 0; i < RTE_MAX_LCORE; i++) {
if (!rte_lcore_is_enabled(i))
continue;
for (j = 0; j < lcore_queue_conf[i].n_rx_port;j++) {
if (lcore_queue_conf[i].rx_port_list[j] == portid) {
lcore = i;
find_lcore = 1;
break;
}
if (lcore_queue_conf[i].rx_port_list[j] == pair_port) {
pair_lcore = i;
find_pair_lcore = 1;
break;
}
}
if (find_lcore && find_pair_lcore)
break;
}
if (!find_lcore || !find_pair_lcore)
rte_exit(EXIT_FAILURE, "Not find port=%d pair\\n", portid);
printf("lcore %u and %u paired\\n", lcore, pair_lcore);
lcore_resource[lcore].pair_id = pair_lcore;
lcore_resource[pair_lcore].pair_id = lcore;
}
Before launching the slave process,
it is necessary to set up the communication channel between the master and slave so that
the master can notify the slave if its peer process with the dependency exited.
In addition, the master needs to register a callback function in the case where a specific slave exited.
.. code-block:: c
for (i = 0; i < RTE_MAX_LCORE; i++) {
if (lcore_resource[i].enabled) {
/* Create ring for master and slave communication */
ret = create_ms_ring(i);
if (ret != 0)
rte_exit(EXIT_FAILURE, "Create ring for lcore=%u failed",i);
if (flib_register_slave_exit_notify(i,slave_exit_cb) != 0)
rte_exit(EXIT_FAILURE, "Register master_trace_slave_exit failed");
}
}
After launching the slave process, the master waits and prints out the port statics periodically.
If an event indicating that a slave process exited is detected,
it sends the STOP command to the peer and waits until it has also exited.
Then, it tries to clean up the execution environment and prepare new resources.
Finally, the new slave instance is launched.
.. code-block:: c
while (1) {
sleep(1);
cur_tsc = rte_rdtsc();
diff_tsc = cur_tsc - prev_tsc;
/* if timer is enabled */
if (timer_period > 0) {
/* advance the timer */
timer_tsc += diff_tsc;
/* if timer has reached its timeout */
if (unlikely(timer_tsc >= (uint64_t) timer_period)) {
print_stats();
/* reset the timer */
timer_tsc = 0;
}
}
prev_tsc = cur_tsc;
/* Check any slave need restart or recreate */
rte_spinlock_lock(&res_lock);
for (i = 0; i < RTE_MAX_LCORE; i++) {
struct lcore_resource_struct *res = &lcore_resource[i];
struct lcore_resource_struct *pair = &lcore_resource[res->pair_id];
/* If find slave exited, try to reset pair */
if (res->enabled && res->flags && pair->enabled) {
if (!pair->flags) {
master_sendcmd_with_ack(pair->lcore_id, CMD_STOP);
rte_spinlock_unlock(&res_lock);
sleep(1);
rte_spinlock_lock(&res_lock);
if (pair->flags)
continue;
}
if (reset_pair(res->lcore_id, pair->lcore_id) != 0)
rte_exit(EXIT_FAILURE, "failed to reset slave");
res->flags = 0;
pair->flags = 0;
}
}
rte_spinlock_unlock(&res_lock);
}
When the slave process is spawned and starts to run, it checks whether the floating process option is applied.
If so, it clears the affinity to a specific core and also sets the unique core ID to 0.
Then, it tries to allocate a new core ID.
Since the core ID has changed, the resource allocated by the master cannot work,
so it remaps the resource to the new core ID slot.
.. code-block:: c
static int
l2fwd_launch_one_lcore( attribute ((unused)) void *dummy)
{
unsigned lcore_id = rte_lcore_id();
if (float_proc) {
unsigned flcore_id;
/* Change it to floating process, also change it's lcore_id */
clear_cpu_affinity();
RTE_PER_LCORE(_lcore_id) = 0;
/* Get a lcore_id */
if (flib_assign_lcore_id() < 0 ) {
printf("flib_assign_lcore_id failed\n");
return -1;
}
flcore_id = rte_lcore_id();
/* Set mapping id, so master can return it after slave exited */
mapping_id[lcore_id] = flcore_id;
printf("Org lcore_id = %u, cur lcore_id = %u\n",lcore_id, flcore_id);
remapping_slave_resource(lcore_id, flcore_id);
}
l2fwd_main_loop();
/* return lcore_id before return */
if (float_proc) {
flib_free_lcore_id(rte_lcore_id());
mapping_id[lcore_id] = INVALID_MAPPING_ID;
}
return 0;
}

1
examples/multi_process/Makefile
View File

@ -13,6 +13,5 @@ include $(RTE_SDK)/mk/rte.vars.mk
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += client_server_mp
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += simple_mp
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += symmetric_mp
DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += l2fwd_fork
include $(RTE_SDK)/mk/rte.extsubdir.mk

22
examples/multi_process/l2fwd_fork/Makefile
View File

@ -1,22 +0,0 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2010-2014 Intel Corporation
# binary name
APP = l2fwd-fork
# all source are stored in SRCS-y
SRCS-y := main.c flib.c
ifeq ($(RTE_SDK),)
$(error "Please define RTE_SDK environment variable")
endif
# Default target, can be overridden by command line or environment
RTE_TARGET ?= x86_64-native-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
include $(RTE_SDK)/mk/rte.extapp.mk

280
examples/multi_process/l2fwd_fork/flib.c
View File

@ -1,280 +0,0 @@
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright(c) 2010-2014 Intel Corporation
*/
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <inttypes.h>
#include <sys/types.h>
#include <sys/queue.h>
#include <sys/wait.h>
#include <sys/prctl.h>
#include <netinet/in.h>
#include <setjmp.h>
#include <stdarg.h>
#include <ctype.h>
#include <errno.h>
#include <getopt.h>
#include <dirent.h>
#include <signal.h>
#include <rte_common.h>
#include <rte_log.h>
#include <rte_malloc.h>
#include <rte_memory.h>
#include <rte_memcpy.h>
#include <rte_eal.h>
#include <rte_launch.h>
#include <rte_atomic.h>
#include <rte_cycles.h>
#include <rte_prefetch.h>
#include <rte_lcore.h>
#include <rte_per_lcore.h>
#include <rte_branch_prediction.h>
#include <rte_interrupts.h>
#include <rte_random.h>
#include <rte_debug.h>
#include <rte_ether.h>
#include <rte_ethdev.h>
#include <rte_mempool.h>
#include <rte_mbuf.h>
#include <rte_string_fns.h>
#include "flib.h"
#define SIG_PARENT_EXIT SIGUSR1
struct lcore_stat {
pid_t pid; /**< pthread identifier */
lcore_function_t *f; /**< function to call */
void *arg; /**< argument of function */
slave_exit_notify *cb_fn;
} __rte_cache_aligned;
static struct lcore_stat *core_cfg;
static uint16_t *lcore_cfg = NULL;
/* signal handler to be notified after parent leaves */
static void
sighand_parent_exit(int sig)
{
printf("lcore = %u : Find parent leaves, sig=%d\n", rte_lcore_id(),
sig);
printf("Child leaving\n");
exit(0);
return;
}
/**
* Real function entrance ran in slave process
**/
static int
slave_proc_func(void)
{
struct rte_config *config;
unsigned slave_id = rte_lcore_id();
struct lcore_stat *cfg = &core_cfg[slave_id];
if (prctl(PR_SET_PDEATHSIG, SIG_PARENT_EXIT, 0, 0, 0, 0) != 0)
printf("Warning: Slave can't register for being notified in"
"case master process exited\n");
else {
struct sigaction act;
memset(&act, 0 , sizeof(act));
act.sa_handler = sighand_parent_exit;
if (sigaction(SIG_PARENT_EXIT, &act, NULL) != 0)
printf("Fail to register signal handler:%d\n", SIG_PARENT_EXIT);
}
/* Set slave process to SECONDARY to avoid operation like dev_start/stop etc */
config = rte_eal_get_configuration();
if (NULL == config)
printf("Warning:Can't get rte_config\n");
else
config->process_type = RTE_PROC_SECONDARY;
printf("Core %u is ready (pid=%d)\n", slave_id, (int)cfg->pid);
exit(cfg->f(cfg->arg));
}
/**
* function entrance ran in master thread, which will spawn slave process and wait until
* specific slave exited.
**/
static int
lcore_func(void *arg __attribute__((unused)))
{
unsigned slave_id = rte_lcore_id();
struct lcore_stat *cfg = &core_cfg[slave_id];
int pid, stat;
if (rte_get_master_lcore() == slave_id)
return cfg->f(cfg->arg);
/* fork a slave process */
pid = fork();
if (pid == -1) {
printf("Failed to fork\n");
return -1;
} else if (pid == 0) /* child */
return slave_proc_func();
else { /* parent */
cfg->pid = pid;
waitpid(pid, &stat, 0);
cfg->pid = 0;
cfg->f = NULL;
cfg->arg = NULL;
/* Notify slave's exit if applicable */
if(cfg->cb_fn)
cfg->cb_fn(slave_id, stat);
return stat;
}
}
static int
lcore_id_init(void)
{
int i;
/* Setup lcore ID allocation map */
lcore_cfg = rte_zmalloc("LCORE_ID_MAP",
sizeof(uint16_t) * RTE_MAX_LCORE,
RTE_CACHE_LINE_SIZE);
if(lcore_cfg == NULL)
rte_panic("Failed to malloc\n");
for (i = 0; i < RTE_MAX_LCORE; i++) {
if (rte_lcore_is_enabled(i))
lcore_cfg[i] = 1;
}
return 0;
}
int
flib_assign_lcore_id(void)
{
unsigned i;
int ret;
/**
* thread assigned a lcore id previously, or a slave thread. But still have
* a bug here: If the core mask includes core 0, and that core call this
* function, it still can get a new lcore id.
**/
if (rte_lcore_id() != 0)
return -1;
do {
/* Find a lcore id not used yet, avoid to use lcore ID 0 */
for (i = 1; i < RTE_MAX_LCORE; i++) {
if (lcore_cfg[i] == 0)
break;
}
if (i == RTE_MAX_LCORE)
return -1;
/* Assign new lcore id to this thread */
ret = rte_atomic16_cmpset(&lcore_cfg[i], 0, 1);
} while (unlikely(ret == 0));
RTE_PER_LCORE(_lcore_id) = i;
return i;
}
void
flib_free_lcore_id(unsigned lcore_id)
{
/* id is not valid or belongs to pinned core id */
if (lcore_id >= RTE_MAX_LCORE || lcore_id == 0 ||
rte_lcore_is_enabled(lcore_id))
return;
lcore_cfg[lcore_id] = 0;
}
int
flib_register_slave_exit_notify(unsigned slave_id,
slave_exit_notify *cb)
{
if (cb == NULL)
return -EFAULT;
if (!rte_lcore_is_enabled(slave_id))
return -ENOENT;
core_cfg[slave_id].cb_fn = cb;
return 0;
}
enum slave_stat
flib_query_slave_status(unsigned slave_id)
{
if (!rte_lcore_is_enabled(slave_id))
return ST_FREEZE;
/* pid only be set when slave process spawned */
if (core_cfg[slave_id].pid != 0)
return ST_RUN;
else
return ST_IDLE;
}
int
flib_remote_launch(lcore_function_t *f,
void *arg, unsigned slave_id)
{
if (f == NULL)
return -1;
if (!rte_lcore_is_enabled(slave_id))
return -1;
/* Wait until specific lcore state change to WAIT */
rte_eal_wait_lcore(slave_id);
core_cfg[slave_id].f = f;
core_cfg[slave_id].arg = arg;
return rte_eal_remote_launch(lcore_func, NULL, slave_id);
}
int
flib_mp_remote_launch(lcore_function_t *f, void *arg,
enum rte_rmt_call_master_t call_master)
{
int i;
RTE_LCORE_FOREACH_SLAVE(i) {
core_cfg[i].arg = arg;
core_cfg[i].f = f;
}
return rte_eal_mp_remote_launch(lcore_func, NULL, call_master);
}
int
flib_init(void)
{
if ((core_cfg = rte_zmalloc("core_cfg",
sizeof(struct lcore_stat) * RTE_MAX_LCORE,
RTE_CACHE_LINE_SIZE)) == NULL ) {
printf("rte_zmalloc failed\n");
return -1;
}
if (lcore_id_init() != 0) {
printf("lcore_id_init failed\n");
return -1;
}
return 0;
}

120
examples/multi_process/l2fwd_fork/flib.h
View File

@ -1,120 +0,0 @@
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright(c) 2010-2014 Intel Corporation
*/
#ifndef __FLIB_H
#define __FLIB_H
/* callback function pointer when specific slave leaves */
typedef void (slave_exit_notify)(unsigned slaveid, int stat);
enum slave_stat{
ST_FREEZE = 1,
ST_IDLE,
ST_RUN,
ST_ZOMBIE, /* Not implemented yet */
};
/**
* Initialize the fork lib.
*
* @return
* - 0 : fork lib initialized successfully
* - -1 : fork lib initialized failed
*/
int flib_init(void);
/**
* Check that every SLAVE lcores are in WAIT state, then call
* flib_remote_launch() for all of them. If call_master is true
* (set to CALL_MASTER), also call the function on the master lcore.
*
* @param f:
* function pointer need to run
* @param arg:
* argument for f to carry
* @param call_master
* - SKIP_MASTER : only launch function on slave lcores
* - CALL_MASTER : launch function on master and slave lcores
* @return
* - 0 : function execute successfully
* - -1 : function execute failed
*/
int flib_mp_remote_launch(lcore_function_t *f,
void *arg, enum rte_rmt_call_master_t call_master);
/**
* Send a message to a slave lcore identified by slave_id to call a
* function f with argument arg.
*
* @param f:
* function pointer need to run
* @param arg:
* argument for f to carry
* @param slave_id
* slave lcore id to run on
* @return
* - 0 : function execute successfully
* - -1 : function execute failed
*/
int flib_remote_launch(lcore_function_t *f,
void *arg, unsigned slave_id);
/**
* Query the running stat for specific slave, wont' work in with master id
*
* @param slave_id:
* lcore id which should not be master id
* @return
* - ST_FREEZE : lcore is not in enabled core mask
* - ST_IDLE : lcore is idle
* - ST_RUN : lcore is running something
*/
enum slave_stat
flib_query_slave_status(unsigned slave_id);
/**
* Register a callback function to be notified in case specific slave exit.
*
* @param slave_id:
* lcore id which should not be master id
* @param cb:
* callback pointer to register
* @return
* - 0 : function execute successfully
* - -EFAULT : argument error
* - -ENOENT : slave_id not correct
*/
int flib_register_slave_exit_notify(unsigned slave_id,
slave_exit_notify *cb);
/**
* Assign a lcore ID to non-slave thread. Non-slave thread refers to thread that
* not created by function rte_eal_remote_launch or rte_eal_mp_remote_launch.
* These threads can either bind lcore or float among different lcores.
* This lcore ID will be unique in multi-thread or multi-process DPDK running
* environment, then it can benefit from using the cache mechanism provided in
* mempool library.
* After calling successfully, use rte_lcore_id() to get the assigned lcore ID, but
* other lcore funtions can't guarantee to work correctly.
*
* @return
* - -1 : can't assign a lcore id with 3 possibilities.
* - it's not non-slave thread.
* - it had assign a lcore id previously
* - the lcore id is running out.
* - > 0 : the assigned lcore id.
*/
int flib_assign_lcore_id(void);
/**
* Free the lcore_id that assigned in flib_assign_lcore_id().
* call it in case non-slave thread is leaving or left.
*
* @param lcore_id
* The identifier of the lcore, which MUST be between 1 and
* RTE_MAX_LCORE-1.
*/
void flib_free_lcore_id(unsigned lcore_id);
#endif /* __FLIB_H */

1258
examples/multi_process/l2fwd_fork/main.c
View File

File diff suppressed because it is too large Load Diff