The default page size can be None, and the page size from user request
can be 0 kB if lower than 1024. In these cases, a division will fail.
In order to avoid a Python exception, the page size is checked
and an error message "Invalid page size" is printed.
A similar error message is printed in set_hugepages()
if the size is not supported, except at this stage the message can be
completed with "Valid page sizes".
Unfortunately the first check is too early to print such information.
A third error message can be printed in a different place (get_memsize)
in case of a format issue, e.g. a negative size.
The function get_memsize() is also used for total requested size,
so the error message "not a valid page size" was potentially wrong.
This message is replaced with the more general "is not a valid size".
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
When the dpdk-telemetry client connects to a DPDK instance, we can use the
PID provided in the initial connection message to query from /proc the name
of the process we are connected to, and display that to the user. We use
the "cmdline" procfs entry for the query since that is available on both
Linux and FreeBSD (assuming procfs is mounted on the BSD instance).
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Currently the dpdk-telemetry.py script connects to all running DPDK apps
consecutively. With the addition of this file-prefix argument, we can limit
the amount of information returned providing improved consumability and
precision to the user.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
A driver can be loaded as a dynamic module or a built-in module.
In commit 681a672886 ("usertools: check if module is loaded
before binding"), the script only checks modules in /sys/module/.
However, for built-in kernel driver, it only shows up in /sys/module/,
if it has a version or at least one parameter. So add check for
modules in /lib/modules/$(uname -r)/modules.builtin.
Fixes: 681a672886 ("usertools: check if module is loaded before binding")
Cc: stable@dpdk.org
Signed-off-by: Yongxin Liu <yongxin.liu@windriver.com>
Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
If user requests a hugepage size which is not supported by the system,
currently user gets an error message saying that the requested size
is not a valid system huge page size. In addition to this if we display
the valid hugepage sizes it will be convenient for the user to request
the right size next time.
Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Sometimes the system is unable to reserve the requested hugepages because
enough space is not available in the RAM. In that case, currently the
script displays no error message hence the user can be under the delusion
that the hugepages requested are all successfully reserved. This patch
displays an error message if the pages reserved are different from the
requested pages.
Signed-off-by: Sarosh Arif <sarosh.arif@emumba.com>
The IBM PowerNV systems include NUMA nodes that don't have associated
CPUs or hugepage memory. Here is an example on an IBM AC922 system:
$ lscpu
...
NUMA node0 CPU(s): 0-63
NUMA node8 CPU(s): 64-127
NUMA node252 CPU(s):
...
$ numastat -m
...
Node 0 Node 8 Node 252
--------------- --------------- ---------------
MemTotal 126763.19 130785.06 0.00
MemFree 119513.38 125294.44 0.00
MemUsed 7249.81 5490.62 0.00
...
HugePages_Total 4.00 1734.00 0.00
HugePages_Free 0.00 4.00 0.00
HugePages_Surp 4.00 1730.00 0.00
...
Modify dpdk-hugepages.py to test for the ../hugepages directory before
attempting to parse the hugepage entries.
Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
This old script relied on deprecated stuff, and especially make.
It also applied some scary 666 permissions on files under /dev/vfio.
Its deprecation had been notified in a previous release, remove it.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The "misc" and "regex" device classes were missing from the list used to
check arguments, preventing them from being used with "--status-dev"
flag to list only devices of those types.
When adding them to the list, the list is also sorted alphabetically for
consistency.
Bugzilla ID: 582
Fixes: 81255f27c6 ("usertools: replace optparse with argparse")
Reported-by: Wei Ling <weix.ling@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Tested-by: Yu Jiang <yux.jiang@intel.com>
This is an improved version of the setup of huge pages
bases on earlier DPDK setup.
Differences are:
* autodetects NUMA vs non NUMA
* allows setting different page sizes
recent kernels support multiple sizes.
* accepts a parameter in bytes (not pages).
* can display current hugepage settings.
Most users will just use --setup argument but if necessary
the steps of clearing old settings and mounting/umounting
can be done individually.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
In the display_pmd_info_strings function, the script parses the section
until to find a byte between 32 and 127, and get all data
until a byte equals to 0.
After, it searches "PMD_INFO_STRING" in the data and passes the whole
string in the parse_pmd_info_string function, which split the string
with "=" and convert it in python dict with json.loads().
But the string may contain a "=" before "PMD_INFO_STRING",
so it is not correctly split and will lead to an error
(json.decoder.JSONDecodeError).
Example of a string encountered that leads to an error:
"Ag%=C£°ÐÊ+Ë®{0´wË-£0òjB·;¾¬úPMD_INFO_STRING= {"name" :
"net_octeontx", "params" : "nr_port=<int> ", "pci_ids" : []}"
Fixes: c67c9a5c64 ("tools: query binaries for HW and other support information")
Cc: stable@dpdk.org
Signed-off-by: Julien Massonneau <julien.massonneau@6wind.com>
Python lint suggests using in instead of multiple comparisons.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Address python lint complaints about unused imports.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Python lint warns about using len(SEQUENCE) to determine if sequence is empty.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
The optparse module is deprecated and replaced with new argparse.
Using the python standard argument parser instead of C library
style getopt gives a number of advantages such as checking
for conflicting arguments, restricting choices, and automatically
generating help messages.
Some of the help messages are now less wordy.
The code now enforces the rule that only one of the pmdinfo formats
can be specified: raw or json.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Python lint complains about indentation and missing spaces around commas.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Python lint complains:
Unnecessary parens after 'if' keyword
Unnecessary parens after 'not' keyword
Unnecessary "else" after "return"
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
The builtin open() is the recommended approach in python3.
io.open was for compatibility with older versions.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
In python3 the standard way to split strings is to use the
split() on the string object itself. The old way is broken
and would cause a traceback.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
The pycodestyle tool flagged the following issues, which are now fixed.
$ pycodestyle cpu_layout.py
cpu_layout.py:18:5: E722 do not use bare 'except'
cpu_layout.py:62:14: E231 missing whitespace after ','
Fixes: deb87e6777 ("usertools: use sysfs for CPU layout")
Fixes: c9208f1dc9 ("usertools: fix CPU layout with python 3")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
This script inspects an ELF file (binary or shared library) and its
linked dependencies by following DT_NEEDED tags.
So far a simple librte_pmd prefix was used as a filter to only parse
DPDK drivers dependencies.
While the reason is not clear from the commitlog of the patch that
introduced this filter, it was probably added for performance reasons,
since going through all dependencies can be quite long.
Testing with a DPDK built before the driver name changes:
- running the script takes ~0.3s with the filter,
- running the script takes ~9s without the filter,
Now that we changed the driver library names, it becomes more difficult
to identify only DPDK drivers, but we can just filter on the librte_
prefix to identify DPDK libraries: the script later checks for the
PMD_INFO_STRING string in .rodata and it is enough to differentiate the
DPDK drivers from the other DPDK libraries.
Running the script with this patch takes ~0.5s.
A debug message was logged for each inspected file, it gives no useful
information and is removed.
Fixes: a20b2c01a7 ("build: standardize component names and defines")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Update the devbind script with new section of regex devices, also
added OCTEONTX2 REE device ID to regex device list
Signed-off-by: Guy Kaneti <guyk@marvell.com>
Intel Data Streaming Accelerator (Intel DSA) is a high-performance data
copy and transformation accelerator which will be integrated in future
Intel processors [1].
Add DSA device support to dpdk-devbind.py script.
[1] https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add NTB device support (4th generation) for Intel Ice Lake platform.
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
Changed scripts to explicitly use Python 3 only, to avoid
maintaining Python 2.
Removed deprecation notices.
Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
When binding or unbinding a range of devices, it can be useful to use
wildcards to specify the devices rather than repeating the same prefix
multiple times. We can use the python "glob" module to give us this
functionality - at least for PCI devices - by checking /sys for matching
files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.*
./dpdk-devbind.py -u 80:04.[2-7]
The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The
second then unbinds six of those devices, 80:04.2..80:04.7, from any
driver.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
The "if", or interface, field in the status display of dpdk-devbind is only
relevant for network interfaces, so don't display it for other device
types.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
As agreed in the deprecation notice, the "generic" script
to setup the Linux environment will be removed in DPDK 20.11.
Some specific parts may be converted into dedicated scripts.
In the meantime, a warning will be printed to users of this script.
Suggested-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Prepare for python2 removal in 20.11.
Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>
The path to the socket when running the script as a regular user needed
to be updated to match the logic in EAL.
Fixes: 6a2967c112 ("usertools: add new telemetry script")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
Fixes the case where a PCI device string identifier
contains non-ASCII UTF-8
A particular example is Mellanox Connext-X 5 EN MT27800:
28:00.0 Ethernet controller: Mellanox Technologies
MT27800 Family [ConnectX-5]
Subsystem: Mellanox Technologies ConnectX®-5 EN network
interface card, 100GbE single-port QSFP28, PCIe3.0 x16,
tall bracket; MCX515A-CCAT
Signed-off-by: Christos Ricudis <ricudis@niometrics.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Add support for Ice Lake IOAT DMA engine PCI Device ID.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
This patch adds a python script that can be used with the new telemetry
socket. It connects as a client to the socket, and allows the user send
a command and see the JSON response.
The example usage below shows the script connecting to the new telemetry
socket, and sending three default telemetry commands entered by the user.
The response for each command is shown below the user input.
Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
{"version": "DPDK 20.05.0-rc0", "pid": 32794, "max_output_len": 16384}
--> /
{"/": ["/", "/help", "/info"]}
--> /info
{"/info": {"version": "DPDK 20.05.0-rc0", "pid": 32794, \
"max_output_len": 16384}}
--> /help,/info
{"/help": {"/info": "Returns DPDK Telemetry information. \
Takes no parameters"}}
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
Debian and Ubuntu switched years ago from /usr/share/hwdata to
/usr/share/misc, and the former is just a compat symlink now.
We are starting to get bug reports to nudge us into changing.
So check the new path first, and the old one as a fallback.
Cc: stable@dpdk.org
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
Acked-by: David Marchand <david.marchand@redhat.com>
The client script for use with the telemetry library did not support
Python3, as the data being sent over the socket was in string format.
Python3 requires the data be explicitly converted to bytes before being
sent. Similarly, the received bytes need to be decoded into string
format.
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Fixes: fe35622659 ("usertools: fix telemetry client with python 3")
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Silent the following warning when running script with python 3.8:
> /usr/bin/dpdk-pmdinfo:542: SyntaxWarning: "is" with a literal.
> Did you mean "=="?
> if (autoload_path is None or autoload_path is ""):
As autoload_path can only be None or a string, directly check its bool
value.
Fixes: c67c9a5c64 ("tools: query binaries for HW and other support information")
Cc: stable@dpdk.org
Signed-off-by: Thomas Faivre <thomas.faivre@6wind.com>
There is a typo in the SPDX tag, which is down as an "SPDK" tag.
One-character change should be all that is needed.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Cc: stable@dpdk.org
Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Some kernel modules use '-' in their name when registering through
`pci_register_driver` and the same name is populated in
'/sys/bus/pci/drivers/'.
But the kernel always populates modules names replacing '-' with '_'
in '/sys/module/'.
Example:
# ./usertools/dpdk-devbind.py -b octeontx2-nicpf 0002:03:00.0
Error: Driver 'octeontx2-nicpf' is not loaded.
# ls /sys/bus/pci/drivers/octeontx2-nicpf
bind module new_id remove_id uevent unbind
# ls /sys/module/octeontx2_nicpf/
drivers uevent version
The patch addresses it by always replacing '-' with '_' when looking in
'/sys/module/'
Signed-off-by: Phanendra Vukkisala <pvukkisala@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
When running the dpdk-telemetry-client.py with python 3, we get the
following syntax errors:
File "usertools/dpdk-telemetry-client.py", line 70
print "\nResponse: \n", str(data)
^
SyntaxError: invalid syntax
File "usertools/dpdk-telemetry-client.py", line 93
print "\nResponse: \n", str(data)
^
SyntaxError: invalid syntax
File "usertools/dpdk-telemetry-client.py", line 111
file_path = sys.argv[1]
^
TabError: inconsistent use of tabs and spaces in indentation
Import print_function from __future__ and add parentheses where missing.
Also, use spaces for indentation everywhere.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Running dpdk-pmdinfo.py on Ubuntu 18.04 (bionic) with python 3 and
pyelftools installed produces no output but no error is reported
neither:
~$ python3 usertools/dpdk-pmdinfo.py -r build/app/testpmd
~$ echo $?
0
While with python 2, it works:
~# python2 usertools/dpdk-pmdinfo.py -r build/app/testpmd
{"pci_ids": [], "name": "dpio"}
{"pci_ids": [], "name": "dpbp"}
{"pci_ids": [], "name": "dpaa2_qdma"}
.....
On Ubuntu 18.04, pyelftools is version 0.24. The change log of
pyelftools v0.24 says:
- Symbol/section names are strings internally now, not bytestrings
(this may affect API usage in Python 3) (#76).
We cannot guess which version of pyelftools is actually being used. The
elftools.__version__ symbol is not consistent with each distro's package
version. For example, on Ubuntu 16.04 (xenial), the .deb package version
is '0.23-2' but elftools.__version__ contains '0.25'. This is certainly
due to partial backports.
To have a more consistent behaviour of this script across all versions
of python, add the unicode_literals future import so that literal
strings are now always "unicode".
Add 2 utility functions to force a string into bytes or bytes into an
unicode string.
Force pyelftools return values to unicode strings (will do nothing with
recent version of pyelftools).
If elffile.get_section_by_name returns None with a unicode section name,
try with the same one encoded as bytes.
Also, replace all open() calls by io.open() which behaves like the
builtin open in python 3. The only non-binary opened file is
/usr/share/hwdata/pci.ids which is UTF-8 encoded text. Explicitly
specify that encoding.
Link: https://github.com/eliben/pyelftools/blob/v0.24/CHANGES#L7
Link: https://github.com/eliben/pyelftools/commit/108eaea9e75a8b5a
Fixes: 54ca545dce ("make python scripts python2/3 compliant")
Cc: stable@dpdk.org
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
This commit removes the unnecesarry ast.literal_eval() function call
from the input handling, which now relies just on raw_input() to get
its input.
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Cc: stable@dpdk.org
Signed-off-by: Andrius Sirvys <andrius.sirvys@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
LGTM static code analysis tool reports that the function 'input' is
unsafe. Changed to use raw_input which then converts it using
ast.literal_eval() which is safe.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Cc: stable@dpdk.org
Signed-off-by: Andrius Sirvys <andrius.sirvys@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Bring consistency to error messages and output them to stderr.
Also, whenever the script tells the user to "check usage", don't
tell the user to do it and just display usage instead.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Currently, if an attempt is made to bind a device to a driver that
is not loaded, a confusing and misleading error message appears.
Fix it so that, before binding to the driver, we actually check if
it is loaded in the kernel first.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
A common user error is to forget driver to which the PCI devices should
be bound to. Currently, the error message in this case looks unhelpful
misleading and indecipherable to anyone but people who know how devbind
works.
Fix this by checking if the driver string is actually a valid device
string. If it is, we assume that the user has just forgot to specify the
driver, and display appropriate error. We also assume that no one will
name their driver in a format that looks like a PCI address, but that
seems like a reasonable assumption to make.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Add in the list of registers for the device.
And enable NTB device ops for Intel Skylake platform.
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
Update the devbind script with new section of DMA devices, also
added OCTEONTX2 DMA device ID to DMA device list
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Satha Rao <skoteshwar@marvell.com>
Currently clear_data (dpdk-devbind.py) doesn't work as expected
since "global devices" is missing and so "devices" is considered
a local variable.
This commit changes "clear_data" function in order to really clear
devices by adding "global devices".
Fixes: ea9f00f728 ("usertools: refactor NIC and crypto binding details")
Cc: stable@dpdk.org
Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Reviewed-by: David Marchand <david.marchand@redhat.com>
