The pycodestyle tool flagged the following issues, which are now fixed.
$ pycodestyle cpu_layout.py
cpu_layout.py:18:5: E722 do not use bare 'except'
cpu_layout.py:62:14: E231 missing whitespace after ','
Fixes: deb87e6777 ("usertools: use sysfs for CPU layout")
Fixes: c9208f1dc9 ("usertools: fix CPU layout with python 3")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
This script inspects an ELF file (binary or shared library) and its
linked dependencies by following DT_NEEDED tags.
So far a simple librte_pmd prefix was used as a filter to only parse
DPDK drivers dependencies.
While the reason is not clear from the commitlog of the patch that
introduced this filter, it was probably added for performance reasons,
since going through all dependencies can be quite long.
Testing with a DPDK built before the driver name changes:
- running the script takes ~0.3s with the filter,
- running the script takes ~9s without the filter,
Now that we changed the driver library names, it becomes more difficult
to identify only DPDK drivers, but we can just filter on the librte_
prefix to identify DPDK libraries: the script later checks for the
PMD_INFO_STRING string in .rodata and it is enough to differentiate the
DPDK drivers from the other DPDK libraries.
Running the script with this patch takes ~0.5s.
A debug message was logged for each inspected file, it gives no useful
information and is removed.
Fixes: a20b2c01a7 ("build: standardize component names and defines")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Update the devbind script with new section of regex devices, also
added OCTEONTX2 REE device ID to regex device list
Signed-off-by: Guy Kaneti <guyk@marvell.com>
Intel Data Streaming Accelerator (Intel DSA) is a high-performance data
copy and transformation accelerator which will be integrated in future
Intel processors [1].
Add DSA device support to dpdk-devbind.py script.
[1] https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add NTB device support (4th generation) for Intel Ice Lake platform.
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
Changed scripts to explicitly use Python 3 only, to avoid
maintaining Python 2.
Removed deprecation notices.
Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
When binding or unbinding a range of devices, it can be useful to use
wildcards to specify the devices rather than repeating the same prefix
multiple times. We can use the python "glob" module to give us this
functionality - at least for PCI devices - by checking /sys for matching
files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.*
./dpdk-devbind.py -u 80:04.[2-7]
The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The
second then unbinds six of those devices, 80:04.2..80:04.7, from any
driver.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
The "if", or interface, field in the status display of dpdk-devbind is only
relevant for network interfaces, so don't display it for other device
types.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
As agreed in the deprecation notice, the "generic" script
to setup the Linux environment will be removed in DPDK 20.11.
Some specific parts may be converted into dedicated scripts.
In the meantime, a warning will be printed to users of this script.
Suggested-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Prepare for python2 removal in 20.11.
Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>
The path to the socket when running the script as a regular user needed
to be updated to match the logic in EAL.
Fixes: 6a2967c112 ("usertools: add new telemetry script")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
Fixes the case where a PCI device string identifier
contains non-ASCII UTF-8
A particular example is Mellanox Connext-X 5 EN MT27800:
28:00.0 Ethernet controller: Mellanox Technologies
MT27800 Family [ConnectX-5]
Subsystem: Mellanox Technologies ConnectX®-5 EN network
interface card, 100GbE single-port QSFP28, PCIe3.0 x16,
tall bracket; MCX515A-CCAT
Signed-off-by: Christos Ricudis <ricudis@niometrics.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Add support for Ice Lake IOAT DMA engine PCI Device ID.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
This patch adds a python script that can be used with the new telemetry
socket. It connects as a client to the socket, and allows the user send
a command and see the JSON response.
The example usage below shows the script connecting to the new telemetry
socket, and sending three default telemetry commands entered by the user.
The response for each command is shown below the user input.
Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
{"version": "DPDK 20.05.0-rc0", "pid": 32794, "max_output_len": 16384}
--> /
{"/": ["/", "/help", "/info"]}
--> /info
{"/info": {"version": "DPDK 20.05.0-rc0", "pid": 32794, \
"max_output_len": 16384}}
--> /help,/info
{"/help": {"/info": "Returns DPDK Telemetry information. \
Takes no parameters"}}
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
Debian and Ubuntu switched years ago from /usr/share/hwdata to
/usr/share/misc, and the former is just a compat symlink now.
We are starting to get bug reports to nudge us into changing.
So check the new path first, and the old one as a fallback.
Cc: stable@dpdk.org
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
Acked-by: David Marchand <david.marchand@redhat.com>
The client script for use with the telemetry library did not support
Python3, as the data being sent over the socket was in string format.
Python3 requires the data be explicitly converted to bytes before being
sent. Similarly, the received bytes need to be decoded into string
format.
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Fixes: fe35622659 ("usertools: fix telemetry client with python 3")
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Silent the following warning when running script with python 3.8:
> /usr/bin/dpdk-pmdinfo:542: SyntaxWarning: "is" with a literal.
> Did you mean "=="?
> if (autoload_path is None or autoload_path is ""):
As autoload_path can only be None or a string, directly check its bool
value.
Fixes: c67c9a5c64 ("tools: query binaries for HW and other support information")
Cc: stable@dpdk.org
Signed-off-by: Thomas Faivre <thomas.faivre@6wind.com>
There is a typo in the SPDX tag, which is down as an "SPDK" tag.
One-character change should be all that is needed.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Cc: stable@dpdk.org
Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Some kernel modules use '-' in their name when registering through
`pci_register_driver` and the same name is populated in
'/sys/bus/pci/drivers/'.
But the kernel always populates modules names replacing '-' with '_'
in '/sys/module/'.
Example:
# ./usertools/dpdk-devbind.py -b octeontx2-nicpf 0002:03:00.0
Error: Driver 'octeontx2-nicpf' is not loaded.
# ls /sys/bus/pci/drivers/octeontx2-nicpf
bind module new_id remove_id uevent unbind
# ls /sys/module/octeontx2_nicpf/
drivers uevent version
The patch addresses it by always replacing '-' with '_' when looking in
'/sys/module/'
Signed-off-by: Phanendra Vukkisala <pvukkisala@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
When running the dpdk-telemetry-client.py with python 3, we get the
following syntax errors:
File "usertools/dpdk-telemetry-client.py", line 70
print "\nResponse: \n", str(data)
^
SyntaxError: invalid syntax
File "usertools/dpdk-telemetry-client.py", line 93
print "\nResponse: \n", str(data)
^
SyntaxError: invalid syntax
File "usertools/dpdk-telemetry-client.py", line 111
file_path = sys.argv[1]
^
TabError: inconsistent use of tabs and spaces in indentation
Import print_function from __future__ and add parentheses where missing.
Also, use spaces for indentation everywhere.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Running dpdk-pmdinfo.py on Ubuntu 18.04 (bionic) with python 3 and
pyelftools installed produces no output but no error is reported
neither:
~$ python3 usertools/dpdk-pmdinfo.py -r build/app/testpmd
~$ echo $?
0
While with python 2, it works:
~# python2 usertools/dpdk-pmdinfo.py -r build/app/testpmd
{"pci_ids": [], "name": "dpio"}
{"pci_ids": [], "name": "dpbp"}
{"pci_ids": [], "name": "dpaa2_qdma"}
.....
On Ubuntu 18.04, pyelftools is version 0.24. The change log of
pyelftools v0.24 says:
- Symbol/section names are strings internally now, not bytestrings
(this may affect API usage in Python 3) (#76).
We cannot guess which version of pyelftools is actually being used. The
elftools.__version__ symbol is not consistent with each distro's package
version. For example, on Ubuntu 16.04 (xenial), the .deb package version
is '0.23-2' but elftools.__version__ contains '0.25'. This is certainly
due to partial backports.
To have a more consistent behaviour of this script across all versions
of python, add the unicode_literals future import so that literal
strings are now always "unicode".
Add 2 utility functions to force a string into bytes or bytes into an
unicode string.
Force pyelftools return values to unicode strings (will do nothing with
recent version of pyelftools).
If elffile.get_section_by_name returns None with a unicode section name,
try with the same one encoded as bytes.
Also, replace all open() calls by io.open() which behaves like the
builtin open in python 3. The only non-binary opened file is
/usr/share/hwdata/pci.ids which is UTF-8 encoded text. Explicitly
specify that encoding.
Link: https://github.com/eliben/pyelftools/blob/v0.24/CHANGES#L7
Link: https://github.com/eliben/pyelftools/commit/108eaea9e75a8b5a
Fixes: 54ca545dce ("make python scripts python2/3 compliant")
Cc: stable@dpdk.org
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
This commit removes the unnecesarry ast.literal_eval() function call
from the input handling, which now relies just on raw_input() to get
its input.
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Cc: stable@dpdk.org
Signed-off-by: Andrius Sirvys <andrius.sirvys@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
LGTM static code analysis tool reports that the function 'input' is
unsafe. Changed to use raw_input which then converts it using
ast.literal_eval() which is safe.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Cc: stable@dpdk.org
Signed-off-by: Andrius Sirvys <andrius.sirvys@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Bring consistency to error messages and output them to stderr.
Also, whenever the script tells the user to "check usage", don't
tell the user to do it and just display usage instead.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Currently, if an attempt is made to bind a device to a driver that
is not loaded, a confusing and misleading error message appears.
Fix it so that, before binding to the driver, we actually check if
it is loaded in the kernel first.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
A common user error is to forget driver to which the PCI devices should
be bound to. Currently, the error message in this case looks unhelpful
misleading and indecipherable to anyone but people who know how devbind
works.
Fix this by checking if the driver string is actually a valid device
string. If it is, we assume that the user has just forgot to specify the
driver, and display appropriate error. We also assume that no one will
name their driver in a format that looks like a PCI address, but that
seems like a reasonable assumption to make.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Add in the list of registers for the device.
And enable NTB device ops for Intel Skylake platform.
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
Update the devbind script with new section of DMA devices, also
added OCTEONTX2 DMA device ID to DMA device list
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Satha Rao <skoteshwar@marvell.com>
Currently clear_data (dpdk-devbind.py) doesn't work as expected
since "global devices" is missing and so "devices" is considered
a local variable.
This commit changes "clear_data" function in order to really clear
devices by adding "global devices".
Fixes: ea9f00f728 ("usertools: refactor NIC and crypto binding details")
Cc: stable@dpdk.org
Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Reviewed-by: David Marchand <david.marchand@redhat.com>
In order to allow binding/unbinding of devices for use by the
ioat_rawdev, we need to update the devbind script to add a new class
of device, and add device ids for the specific HW instances.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Jiayu Hu <jiayu.hu@intel.com>
Tested-by: Harry van Haaren <harry.van.haaren@intel.com>
telemetry has support for fetching port based stats
from metrics library.
Metrics library also has global stats which are
not fetched by telemetry, so extend telemetry to
fetch the global metrics.
Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Rather than using linuxapp and bsdapp everywhere, we can change things to
use the, more readable, terms "linux" and "freebsd" in our build configs.
Rather than renaming the configs we can just duplicate the existing ones
with the new names using symlinks, and use the new names exclusively
internally. ["make showconfigs" also only shows the new names to keep the
list short] The result is that backward compatibility is kept fully but any
new builds or development can be done using the newer names, i.e. both
"make config T=x86_64-native-linuxapp-gcc" and "T=x86_64-native-linux-gcc"
work.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Add #! line to the top of the script and mark it as executable so it can
be run directly rather than having to pass it to python interpreter.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
If there aren't any devices of a particular category on user's
system, we still display them, which is bad for usability. Fix
devbind to not print out a category unless there are devices in
it.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: David Hunt <david.hunt@intel.com>
On some distributions (such as CentOS 7) lspci may not be installed
by default, causing exceptions which are difficult to interpret.
Fix devbind script to check if lspci is installed at script startup.
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: Rami Rosen <roszenrami@gmail.com>
This patch adds a python script which can be used as a demo
client. The script is interactive and will allow the user to
register, request statistics, and unregister.
To run the script, an argument for the client file path must
be passed in: "python telemetry_client.py <file_path>".
This script is useful to see how the Telemetry API for DPDK
is used, and how to make the initial connection.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
lspci reports kernel modules in "Module" string, but devbind
expects it to be "Module_str". Fix it up similar to how we fix
up "Driver" to be "Driver_str".
Fixes: c3ce205d57 ("usertools: optimize lspci invocation")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Have the "usertools" scripts installed when doing a DPDK install using
ninja. They will be copies to $prefix/bin, generally /usr/local/bin,
alongside testpmd.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Keith Wiles <keith.wiles@intel.com>
Acked-by: Luca Boccassi <luca.boccassi@gmail.com>
Replace the BSD license header with the SPDX tag for
scripting files with only an Intel copyright on them.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
