Arrays of type uint64_t/int/string can now be included within an array
or dict. One level of embedded containers is supported. This is
necessary to allow for instances such as the ethdev queue stats to be
reported as a list of uint64_t values, rather than having multiple dict
entries with one uint64_t value for each queue stat.
The memory management APIs provided by telemetry simplify the memory
allocation/free aspect of the embedded container. The rte_tel_data_alloc
function is called in the library/app callback to return a pointer to a
container that has been allocated memory. When adding this container
to an array/dict, a parameter is passed to indicate if the memory
should be freed by telemetry after use. This will allow reuse of the
allocated memory if the library/app wishes to do so.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Telemetry only passed the first param to the command handler if multiple
were entered by the user, separated by commas. Telemetry is required to
pass the full params string to the command, by splitting by a comma
delimiter only once to remove the command part of the string. This will
enable future commands to take multiple param values.
Fixes: b1ad0e1245 ("rawdev: add telemetry callbacks")
Fixes: c190daedb9 ("ethdev: add telemetry callbacks")
Fixes: 6dd571fd07 ("telemetry: introduce new functionality")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
A decision was made [1] to no longer support Make in DPDK, this patch
removes all Makefiles that do not make use of pkg-config, along with
the mk directory previously used by make.
[1] https://mails.dpdk.org/archives/dev/2020-April/162839.html
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
This patch limits the number of client connections to the new telemetry
socket. The limit is set to 10.
Signed-off-by: Ciara Power <ciara.power@intel.com>
The following libraries are experimental, all of their functions can
be changed or removed:
- librte_bbdev
- librte_bpf
- librte_compressdev
- librte_fib
- librte_flow_classify
- librte_graph
- librte_ipsec
- librte_node
- librte_rcu
- librte_rib
- librte_stack
- librte_telemetry
Their status is properly announced in MAINTAINERS.
Remind this status in their headers in a common fashion (aligned to ABI
docs).
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Initially, printf was used to indicate and error/warning resulting from
telemetry initialisation. This is now fixed to use EAL logs for
notices, and the unnecessary printf for an error is removed.
Fixes: eeb486f3ba ("eal: add telemetry as dependency")
Fixes: dd6275a424 ("telemetry: fix error log output")
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
If 1024 bytes were received over the socket, this caused
buffer_recvf[bytes] to overrun the array. The size of the buffer - 1 is
now passed to the read function.
Coverity issue: 358442
Fixes: b80fe1805e ("telemetry: introduce backward compatibility")
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
The return value from the socket function is now checked, as it can
return a negative value on error.
Coverity issue: 358443
Fixes: b80fe1805e ("telemetry: introduce backward compatibility")
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
The socket fd is now being closed when the connection fails.
Coverity issue: 358444
Fixes: b80fe1805e ("telemetry: introduce backward compatibility")
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
The strchr function return was not being checked which could lead to
NULL deferencing later in the function.
Coverity issue: 358438, 358445
Fixes: b80fe1805e ("telemetry: introduce backward compatibility")
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
The threads for listening on the telemetry sockets are control threads
and should be separated from those on the data plane. Since telemetry
cannot use the rte_ctrl_thread_create() API, as it does not depend on
EAL, we pass the ctrl thread cpu_set to telemetry init and use it
directly to ensure that telemetry cannot interfere with the data plane
threads.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Caught while running testpmd:
No telemetry legacy support- No legacy callbacks, legacy socket not createdInteractive-mode selected
Add missing \n.
Fixes: 6dd571fd07 ("telemetry: introduce new functionality")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
telemetry can not depend on EAL anymore but it still wants to get arch
headers.
We directly point at the right source directories by using the same logic
than EAL. However the special case of armv7 has been missed.
Fix this by defaulting ARCH_DIR to RTE_ARCH.
Caught on OBS:
[ 162s] SYMLINK-FILE include/rte_telemetry.h
[ 162s] CC telemetry.o
[ 162s] CC telemetry_data.o
[ 162s] CC telemetry_legacy.o
[ 162s] .../lib/librte_telemetry/telemetry.c:15:10: fatal error:
rte_spinlock.h: No such file or directory
[ 162s] #include <rte_spinlock.h>
[ 162s] ^~~~~~~~~~~~~~~~
[ 162s] compilation terminated.
Fixes: 6dd571fd07 ("telemetry: introduce new functionality")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
This patch moves telemetry further down the build, and adds it as a
dependency for EAL. Telemetry V2 is now configured to build by default,
and the legacy support is built when the telemetry config flag is set.
Telemetry now has EAL flags, shown below:
"--telemetry" = Enables telemetry (this is default if no flags given)
"--no-telemetry" = Disables telemetry
When telemetry is enabled, it will attempt to open the new socket
version, and also the legacy support socket (this will depend on Jansson
external dependency and telemetry config flag, as before).
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
This patch removes the existing telemetry files, which are now redundant
as the new version of telemetry has backward compatibility for their
functionality.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
The new telemetry will now open a socket using the old telemetry path,
to ensure backward compatibility. This is not yet initialised, as it
would clash with the existing telemetry, to be removed in a later patch.
This means that both old and new telemetry socket interfaces are
handled in a common way.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
The default commands are now added to provide the list of commands
available, help text for a specified command, and also information
about DPDK and telemetry.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
The functions added in this patch will help applications build
up data in reply to a telemetry request.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
This patch introduces a new telemetry connection socket and handling
functionality. Like the existing telemetry implementation (which is
unaffected by this change) it uses a unix socket, but unlike the
existing one it does not have a fixed list of commands - instead
libraries or applications can register telemetry commands and callbacks
to provide a full-extensible solution for all kinds of telemetry across
DPDK.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
The functions added in this patch will make it easier for telemetry
to convert data to correct JSON responses to telemetry requests.
Tests are also added for these json utility functions.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
Rather than having the telemetry library depend on the metrics
lib we invert the dependency so that metrics instead depends
on telemetry lib, and registers the needed functions with it
at init time. This prepares the way for a cleaner telemetry
architecture to be applied in later patches.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
The telemetry code that was moved into the metrics library can be
shortened, while still maintaining the same functionality.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
This commit moves some of the telemetry library code to a new file in
the metrics library. No modifications are made to the moved code,
except what is needed to allow it to compile and run. The additional
code in metrics is built only when the Jansson library is present.
Telemetry functions as normal, using the functions from the
metrics_telemetry file. This move will enable code be reused by the new
version of telemetry in a later commit, to support backward
compatibility with the existing telemetry usage.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
Remove setting ALLOW_EXPERIMENTAL_API individually for each Makefile and
meson.build. Instead, enable ALLOW_EXPERIMENTAL_API flag across app, lib
and drivers.
This changes reduces the clutter across the project while still
maintaining the functionality of ALLOW_EXPERIMENTAL_API i.e. warning
external applications about experimental API usage.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
The ep struct is used to track what type of stats are required by the
client. For PORT_STATS type, it contains the lists of port and metric
ids to query, and the number of ids in each list.
The ep struct has values set (num of port and metric ids) when a request
for port stats values by name is received. However, after this value
assignment, the struct is reset to all 0 values, meaning the number of
port and metric ids required now both show as 0, and the client will not
receive the requested data in response. To fix this issue, the memset
call is now moved above the ep struct value assignment.
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Since the library versioning for both stable and experimental ABI's is
now managed globally, the LIBABIVER and version variables no longer
serve any useful purpose, and can be removed.
The replacement in Makefiles was done using the following regex:
^(#.*\n)?LIBABIVER\s*:=\s*\d+\n(\s*\n)?
(LIBABIVER := numbers, optionally preceded by a comment and optionally
succeeded by an empty line)
The replacement for meson files was done using the following regex:
^(#.*\n)?version\s*=\s*\d+\n(\s*\n)?
(version = numbers, optionally preceded by a comment and optionally
succeeded by an empty line)
[David]: those variables are manually removed for the files:
- drivers/common/qat/Makefile
- lib/librte_eal/meson.build
[David]: the LIBABIVER is restored for the external ethtool example
library.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Suppress the unaligned packed member address warnings by extending
the telemetry library build flags with -Wno-address-of-packed-member
option, through the WERROR_FLAGS makefile variable.
With this change additional warnings are turned on to be treated as errors,
which causes the following build issues to be seen:
- no previous prototype [-Werror=missing-prototypes]
- initialization discards ‘const’ qualifier from pointer target type
[-Werror=discarded-qualifiers]
- old-style function definition [-Werror=old-style-definition]
- variable may be used before its value is set (when using icc compiler).
Fixes: 0fe3a37924 ("telemetry: format json response when sending stats")
Fixes: ee5ff0d329 ("telemetry: add client feature and sockets")
Fixes: 8877ac688b ("telemetry: introduce infrastructure")
Fixes: 1b756087db ("telemetry: add parser for client socket messages")
Fixes: fff6df7bf5 ("telemetry: fix using ports of different types")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Signed-off-by: Flavia Musatescu <flavia.musatescu@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
For each library where we optionally disable it, add in the reason why it's
being disabled, so the user knows how to fix it.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Putting a '__attribute__((deprecated))' in the middle of a function
prototype does not result in the expected result with gcc (while clang
is fine with this syntax).
$ cat deprecated.c
void * __attribute__((deprecated)) incorrect() { return 0; }
__attribute__((deprecated)) void *correct(void) { return 0; }
int main(int argc, char *argv[]) { incorrect(); correct(); return 0; }
$ gcc -o deprecated.o -c deprecated.c
deprecated.c: In function ‘main’:
deprecated.c:3:1: warning: ‘correct’ is deprecated (declared at
deprecated.c:2) [-Wdeprecated-declarations]
int main(int argc, char *argv[]) { incorrect(); correct(); return 0; }
^
Move the tag on a separate line and make it the first thing of function
prototypes.
This is not perfect but we will trust reviewers to catch the other not
so easy to detect patterns.
sed -i \
-e '/^\([^#].*\)\?__rte_experimental */{' \
-e 's//\1/; s/ *$//; i\' \
-e __rte_experimental \
-e '/^$/d}' \
$(git grep -l __rte_experimental -- '*.h')
Special mention for rte_mbuf_data_addr_default():
There is either a bug or a (not yet understood) issue with gcc.
gcc won't drop this inline when unused and rte_mbuf_data_addr_default()
calls rte_mbuf_buf_addr() which itself is experimental.
This results in a build warning when not accepting experimental apis
from sources just including rte_mbuf.h.
For this specific case, we hide the call to rte_mbuf_buf_addr() under
the ALLOW_EXPERIMENTAL_API flag.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
We had some inconsistencies between functions prototypes and actual
definitions.
Let's avoid this by only adding the experimental tag to the prototypes.
Tests with gcc and clang show it is enough.
git grep -l __rte_experimental |grep \.c$ |while read file; do
sed -i -e '/^__rte_experimental$/d' $file;
sed -i -e 's/ *__rte_experimental//' $file;
sed -i -e 's/__rte_experimental *//' $file;
done
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Build error:
../lib/librte_telemetry/rte_telemetry.c:558:28:
error: comparison of unsigned expression < 0 is always false
[-Werror,-Wtautological-compare]
Build error not observed in default make build because telemetry library
disabled by default but easier to reproduce via meson.
Fixing by converting unsigned variables to signed.
Fixes: 0fe3a37924 ("telemetry: format json response when sending stats")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
telemetry has support for fetching port based stats
from metrics library.
Metrics library also has global stats which are
not fetched by telemetry, so extend telemetry to
fetch the global metrics.
Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Free the `values` pointer before returning
from rte_telemetry_command_ports_all_stat_values()
to avoid memory leak.
Fixes: c12cefa379 ("telemetry: fix mapping of statistics")
Cc: stable@dpdk.org
Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
If we have two NIC ports which have a different set of NIC stats we can
end up having two different stats registered with xstats with the same
name. [Since the stats are updated in bulk as a contiguous set, the
second driver re-using the registration of the first is not possible.]
This causes issues with the invalid stat for one driver being found due to
a lookup by name which is unnecessary. Instead of getting stat names
involved do the lookup by ID instead.
Fixes: 1b756087db ("telemetry: add parser for client socket messages")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Whenever possible (if the library ships a pkg-config file) use meson's
dependency() function to look for it, as it will automatically add it
to the Requires.private list if needed, to allow for static builds to
succeed for reverse dependencies of DPDK. Otherwise the recursive
dependencies are not parsed, and users doing static builds have to
resolve them manually by themselves.
When using this API avoid additional checks that are superfluous and
take extra time, and avoid adding the linker flag manually which causes
it to be duplicated.
Signed-off-by: Luca Boccassi <bluca@debian.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Add a usage string field in rte_option, allowing to display
help to the user and describe which options are currently available.
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Current options name can be passed with arbitrary format.
Force the use of "--" prefix and thus POSIX long options format.
This restricts the ability to introduce surprising options and will help
future additional checks.
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Different NIC ports can have different numbers of xstats on them, which
means that we can't just use the xstats list from the first port registered
in the telemetry library. Instead, we need to check the type of each port -
by checking its ops structure pointer - and register each port type once
with the metrics lib.
Fixes: fdbdb3f9ce ("telemetry: add initial connection socket")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
This patch adds telemetry as a dependecy to all applications. Without these
changes, the --telemetry flag will not be recognised and applications will
fail to run if they want to enable telemetry.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch adds functionality to enable/disable the selftest.
This functionality will be extended in future to make the
enabling/disabling more dynamic and remove this 'hardcoded' approach. We
are temporarily using this approach due to the design changes (vdev vs eal)
made to the library.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch adds functionality to create a JSON message in
order to send it to a client socket.
When stats are requested by a client, they are retrieved from
the metrics library and encoded in JSON format.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch adds functionality to update the statistics in
the metrics library with values from the ethdev stats.
Values need to be updated before they are encoded into a JSON
message and sent to the client that requested them. The JSON encoding
will be added in a subsequent patch.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch adds the parser file. This is used to parse any
messages that are received on any of the client sockets.
Currently, the unregister functionality works using the parser.
Functionality relating to getting statistic values for certain ports
will be added in a subsequent patch, however the parsing involved
for that command is added in this patch.
Some of the parser code included is in preparation for future
functionality, that is not implemented yet in this patchset.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch introduces clients to the telemetry API.
When a client makes a connection through the initial telemetry
socket, they can send a message through the socket to be
parsed. Register messages are expected through this socket, to
enable clients to register and have a client socket setup for
future communications.
A TAILQ is used to store all clients information. Using this, the
client sockets are polled for messages, which will later be parsed
and dealt with accordingly.
Functionality that make use of the client sockets were introduced
in this patch also, such as writing to client sockets, and sending
error responses.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch adds the telemetry UNIX socket. It is used to
allow connections from external clients.
On the initial connection from a client, ethdev stats are
registered in the metrics library, to allow for their retrieval
at a later stage.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch adds the infrastructure and initial code for the telemetry
library.
The telemetry init is registered with eal_init(). We can then check to see
if --telemetry was passed as an eal option. If --telemetry was parsed, then
we call telemetry init at the end of eal init.
Control threads are used to get CPU cycles for telemetry, which are
configured in this patch also.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
