d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

examples/service_cores: add new sample application

Browse Source 
This commit adds a new sample app, which showcases the value
of running services. In particular it allows the application
to dynamically schedule services to service-cores.

The sample app itself registers a number of dummy services,
and applies different profiles to them at runtime. Note that
this sample application does not forward any traffic - it
demonstrates advanced usage of the service cores API.

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Gage Eads <gage.eads@intel.com>
This commit is contained in:
Harry van Haaren 2017-10-19 11:31:11 +01:00 committed by Thomas Monjalon
parent 9c9befea4f
commit cbdd342060
6 changed files with 479 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
MAINTAINERS
View File

@ -977,6 +977,10 @@ M: John McNamara <john.mcnamara@intel.com>
F: examples/rxtx_callbacks/
F: doc/guides/sample_app_ug/rxtx_callbacks.rst
M: Harry van Haaren <harry.van.haaren@intel.com>
F: examples/service_cores/
F: doc/guides/sample_app_ug/service_cores.rst
M: Bruce Richardson <bruce.richardson@intel.com>
M: John McNamara <john.mcnamara@intel.com>
F: examples/skeleton/

1
doc/guides/sample_app_ug/index.rst
View File

@ -58,6 +58,7 @@ Sample Applications User Guides
link_status_intr
load_balancer
server_node_efd
service_cores
multi_process
qos_metering
qos_scheduler

172
doc/guides/sample_app_ug/service_cores.rst Normal file
View File

@ -0,0 +1,172 @@
.. BSD LICENSE
Copyright(c) 2017 Intel Corporation. All rights reserved.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
* Neither the name of Intel Corporation nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Service Cores Sample Application
================================
The service cores sample application demonstrates the service cores capabilities
of DPDK. The service cores infrastructure is part of the DPDK EAL, and allows
any DPDK component to register a service. A service is a work item or task, that
requires CPU time to perform its duty.
This sample application registers 5 dummy services. These 5 services are used
to show how the service_cores API can be used to orchestrate these services to
run on different service lcores. This orchestration is done by calling the
service cores APIs, however the sample application introduces a "profile"
concept to contain the service mapping details. Note that the profile concept
is application specific, and not a part of the service cores API.
Compiling the Application
-------------------------
#. Go to the example directory:
.. code-block:: console
export RTE_SDK=/path/to/rte_sdk
cd ${RTE_SDK}/examples/service_cores
#. Set the target (a default target is used if not specified). For example:
.. code-block:: console
export RTE_TARGET=x86_64-native-linuxapp-gcc
See the *DPDK Getting Started* Guide for possible RTE_TARGET values.
#. Build the application:
.. code-block:: console
make
Running the Application
-----------------------
To run the example, just execute the binary. Since the application dynamically
adds service cores in the application code itself, there is no requirement to
pass a service core-mask as an EAL argument at startup time.
.. code-block:: console
$ ./build/service_cores
Explanation
-----------
The following sections provide some explanation of code focusing on
registering applications from an applications point of view, and modifying the
service core counts and mappings at runtime.
Registering a Service
~~~~~~~~~~~~~~~~~~~~~
The following code section shows how to register a service as an application.
Note that the service component header must be included by the application in
order to register services: ``rte_service_component.h``, in addition
to the ordinary service cores header ``rte_service.h`` which provides
the runtime functions to add, remove and remap service cores.
.. code-block:: c
struct rte_service_spec service = {
.name = "service_name",
};
int ret = rte_service_component_register(services, &id);
if (ret)
return -1;
/* set the service itself to be ready to run. In the case of
* ethdev, eventdev etc PMDs, this will be set when the
* appropriate configure or setup function is called.
*/
rte_service_component_runstate_set(id, 1);
/* Collect statistics for the service */
rte_service_set_stats_enable(id, 1);
/* The application sets the service to running state. Note that this
* function enables the service to run - while the 'component' version
* of this function (as above) marks the service itself as ready */
ret = rte_service_runstate_set(id, 1);
Controlling A Service Core
~~~~~~~~~~~~~~~~~~~~~~~~~~
This section demonstrates how to add a service core. The ``rte_service.h``
header file provides the functions for dynamically adding and removing cores.
The APIs to add and remove cores use lcore IDs similar to existing DPDK
functions.
These are the functions to start a service core, and have it run a service:
.. code-block:: c
/* the lcore ID to use as a service core */
uint32_t service_core_id = 7;
ret = rte_service_lcore_add(service_core_id);
if(ret)
return -1;
/* service cores are in "stopped" state when added, so start it */
ret = rte_service_lcore_start(service_core_id);
if(ret)
return -1;
/* map a service to the service core, causing it to run the service */
uint32_t service_id; /* ID of a registered service */
uint32_t enable = 1; /* 1 maps the service, 0 unmaps */
ret = rte_service_map_lcore_set(service_id, service_core_id, enable);
if(ret)
return -1;
Removing A Service Core
~~~~~~~~~~~~~~~~~~~~~~~
To remove a service core, the steps are similar to adding but in reverse order.
Note that it is not allowed to remove a service core if the service is running,
and the service-core is the only core running that service (see documentation
for ``rte_service_lcore_stop`` function for details).
Conclusion
~~~~~~~~~~
The service cores infrastructure provides DPDK with two main features. The first
is to abstract away hardware differences: the service core can CPU cycles to
a software fallback implementation, allowing the application to be abstracted
from the difference in HW / SW availability. The second feature is a flexible
method of registering functions to be run, allowing the running of the
functions to be scaled across multiple CPUs.

1
examples/Makefile
View File

@ -84,6 +84,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_METER) += qos_meter
DIRS-$(CONFIG_RTE_LIBRTE_SCHED) += qos_sched
DIRS-y += quota_watermark
DIRS-$(CONFIG_RTE_ETHDEV_RXTX_CALLBACKS) += rxtx_callbacks
DIRS-y += service_cores
DIRS-y += skeleton
ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination

54
examples/service_cores/Makefile Normal file
View File

@ -0,0 +1,54 @@
# BSD LICENSE
#
# Copyright(c) 2017 Intel Corporation. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# * Neither the name of Intel Corporation nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
ifeq ($(RTE_SDK),)
$(error "Please define RTE_SDK environment variable")
endif
# Default target, can be overridden by command line or environment
RTE_TARGET ?= x86_64-native-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
# binary name
APP = service_cores
# all source are stored in SRCS-y
SRCS-y := main.c
CFLAGS += $(WERROR_FLAGS)
# workaround for a gcc bug with noreturn attribute
# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
CFLAGS_main.o += -Wno-return-type
endif
include $(RTE_SDK)/mk/rte.extapp.mk

247
examples/service_cores/main.c Normal file
View File

@ -0,0 +1,247 @@
/*
* BSD LICENSE
*
* Copyright(c) 2017 Intel Corporation. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in
* the documentation and/or other materials provided with the
* distribution.
* * Neither the name of Intel Corporation nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <errno.h>
#include <sys/queue.h>
#include <rte_memory.h>
#include <rte_memzone.h>
#include <rte_launch.h>
#include <rte_eal.h>
#include <rte_debug.h>
#include <rte_cycles.h>
/* allow application scheduling of the services */
#include <rte_service.h>
/* Allow application registration of its own services. An application does not
* have to register services, but it can be useful if it wishes to run a
* function on a core that is otherwise in use as a service core. In this
* example, all services are dummy services registered by the sample app itself.
*/
#include <rte_service_component.h>
#define PROFILE_CORES_MAX 8
#define PROFILE_SERVICE_PER_CORE 5
/* dummy function to do "work" */
static int32_t service_func(void *args)
{
RTE_SET_USED(args);
rte_delay_us(2000);
return 0;
}
static struct rte_service_spec services[] = {
{"service_1", service_func, NULL, 0, 0},
{"service_2", service_func, NULL, 0, 0},
{"service_3", service_func, NULL, 0, 0},
{"service_4", service_func, NULL, 0, 0},
{"service_5", service_func, NULL, 0, 0},
};
#define NUM_SERVICES RTE_DIM(services)
/* this struct holds the mapping of a particular core to all services */
struct profile_for_core {
uint32_t mapped_services[PROFILE_SERVICE_PER_CORE];
};
/* struct that can be applied as the service core mapping. Items in this
* struct will be passed to the ordinary rte_service_* APIs to configure the
* service cores at runtime, based on the requirements.
*
* These profiles can be considered a "configuration" for the service cores,
* where switching profile just changes the number of cores and the mappings
* for each of them. As a result, the core requirements and performance of the
* application scales.
*/
struct profile {
char name[64];
uint32_t num_cores;
struct profile_for_core cores[PROFILE_CORES_MAX];
};
static struct profile profiles[] = {
/* profile 0: high performance */
{
.name = "High Performance",
.num_cores = 5,
.cores[0] = {.mapped_services = {1, 0, 0, 0, 0} },
.cores[1] = {.mapped_services = {0, 1, 0, 0, 0} },
.cores[2] = {.mapped_services = {0, 0, 1, 0, 0} },
.cores[3] = {.mapped_services = {0, 0, 0, 1, 0} },
.cores[4] = {.mapped_services = {0, 0, 0, 0, 1} },
},
/* profile 1: mid performance with single service priority */
{
.name = "Mid-High Performance",
.num_cores = 3,
.cores[0] = {.mapped_services = {1, 1, 0, 0, 0} },
.cores[1] = {.mapped_services = {0, 0, 1, 1, 0} },
.cores[2] = {.mapped_services = {0, 0, 0, 0, 1} },
.cores[3] = {.mapped_services = {0, 0, 0, 0, 0} },
.cores[4] = {.mapped_services = {0, 0, 0, 0, 0} },
},
/* profile 2: mid performance with single service priority */
{
.name = "Mid-Low Performance",
.num_cores = 2,
.cores[0] = {.mapped_services = {1, 1, 1, 0, 0} },
.cores[1] = {.mapped_services = {1, 1, 0, 1, 1} },
.cores[2] = {.mapped_services = {0, 0, 0, 0, 0} },
.cores[3] = {.mapped_services = {0, 0, 0, 0, 0} },
.cores[4] = {.mapped_services = {0, 0, 0, 0, 0} },
},
/* profile 3: scale down performance on single core */
{
.name = "Scale down performance",
.num_cores = 1,
.cores[0] = {.mapped_services = {1, 1, 1, 1, 1} },
.cores[1] = {.mapped_services = {0, 0, 0, 0, 0} },
.cores[2] = {.mapped_services = {0, 0, 0, 0, 0} },
.cores[3] = {.mapped_services = {0, 0, 0, 0, 0} },
.cores[4] = {.mapped_services = {0, 0, 0, 0, 0} },
},
};
#define NUM_PROFILES RTE_DIM(profiles)
static void
apply_profile(int profile_id)
{
uint32_t i;
uint32_t s;
int ret;
struct profile *p = &profiles[profile_id];
const uint8_t core_off = 1;
for (i = 0; i < p->num_cores; i++) {
uint32_t core = i + core_off;
ret = rte_service_lcore_add(core);
if (ret && ret != -EALREADY)
printf("core %d added ret %d\n", core, ret);
ret = rte_service_lcore_start(core);
if (ret && ret != -EALREADY)
printf("core %d start ret %d\n", core, ret);
for (s = 0; s < NUM_SERVICES; s++) {
if (rte_service_map_lcore_set(s, core,
p->cores[i].mapped_services[s]))
printf("failed to map lcore %d\n", core);
}
}
for ( ; i < PROFILE_CORES_MAX; i++) {
uint32_t core = i + core_off;
for (s = 0; s < NUM_SERVICES; s++) {
ret = rte_service_map_lcore_set(s, core, 0);
if (ret && ret != -EINVAL) {
printf("%s %d: map lcore set = %d\n", __func__,
__LINE__, ret);
}
}
ret = rte_service_lcore_stop(core);
if (ret && ret != -EALREADY) {
printf("%s %d: lcore stop = %d\n", __func__,
__LINE__, ret);
}
ret = rte_service_lcore_del(core);
if (ret && ret != -EINVAL) {
printf("%s %d: lcore del = %d\n", __func__,
__LINE__, ret);
}
}
}
int
main(int argc, char **argv)
{
int ret;
ret = rte_eal_init(argc, argv);
if (ret < 0)
rte_panic("Cannot init EAL\n");
uint32_t i;
for (i = 0; i < NUM_SERVICES; i++) {
services[i].callback_userdata = 0;
uint32_t id;
ret = rte_service_component_register(&services[i], &id);
if (ret)
rte_exit(-1, "service register() failed");
/* set the service itself to be ready to run. In the case of
* ethdev, eventdev etc PMDs, this will be set when the
* appropriate configure or setup function is called.
*/
rte_service_component_runstate_set(id, 1);
/* Collect statistics for the service */
rte_service_set_stats_enable(id, 1);
/* the application sets the service to be active. Note that the
* previous component_runstate_set() is the PMD indicating
* ready, while this function is the application setting the
* service to run. Applications can choose to not run a service
* by setting runstate to 0 at any time.
*/
ret = rte_service_runstate_set(id, 1);
if (ret)
return -ENOEXEC;
}
i = 0;
while (1) {
const char clr[] = { 27, '[', '2', 'J', '\0' };
const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
printf("%s%s", clr, topLeft);
apply_profile(i);
printf("\n==> Profile: %s\n\n", profiles[i].name);
sleep(1);
rte_service_dump(stdout, UINT32_MAX);
sleep(5);
rte_service_dump(stdout, UINT32_MAX);
i++;
if (i >= NUM_PROFILES)
i = 0;
}
return 0;
}