The client script for use with the telemetry library did not support
Python3, as the data being sent over the socket was in string format.
Python3 requires the data be explicitly converted to bytes before being
sent. Similarly, the received bytes need to be decoded into string
format.
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Fixes: fe35622659 ("usertools: fix telemetry client with python 3")
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Silent the following warning when running script with python 3.8:
> /usr/bin/dpdk-pmdinfo:542: SyntaxWarning: "is" with a literal.
> Did you mean "=="?
> if (autoload_path is None or autoload_path is ""):
As autoload_path can only be None or a string, directly check its bool
value.
Fixes: c67c9a5c64 ("tools: query binaries for HW and other support information")
Cc: stable@dpdk.org
Signed-off-by: Thomas Faivre <thomas.faivre@6wind.com>
There is a typo in the SPDX tag, which is down as an "SPDK" tag.
One-character change should be all that is needed.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Cc: stable@dpdk.org
Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Some kernel modules use '-' in their name when registering through
`pci_register_driver` and the same name is populated in
'/sys/bus/pci/drivers/'.
But the kernel always populates modules names replacing '-' with '_'
in '/sys/module/'.
Example:
# ./usertools/dpdk-devbind.py -b octeontx2-nicpf 0002:03:00.0
Error: Driver 'octeontx2-nicpf' is not loaded.
# ls /sys/bus/pci/drivers/octeontx2-nicpf
bind module new_id remove_id uevent unbind
# ls /sys/module/octeontx2_nicpf/
drivers uevent version
The patch addresses it by always replacing '-' with '_' when looking in
'/sys/module/'
Signed-off-by: Phanendra Vukkisala <pvukkisala@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
When running the dpdk-telemetry-client.py with python 3, we get the
following syntax errors:
File "usertools/dpdk-telemetry-client.py", line 70
print "\nResponse: \n", str(data)
^
SyntaxError: invalid syntax
File "usertools/dpdk-telemetry-client.py", line 93
print "\nResponse: \n", str(data)
^
SyntaxError: invalid syntax
File "usertools/dpdk-telemetry-client.py", line 111
file_path = sys.argv[1]
^
TabError: inconsistent use of tabs and spaces in indentation
Import print_function from __future__ and add parentheses where missing.
Also, use spaces for indentation everywhere.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Fixes: 4080e46c80 ("telemetry: support global metrics")
Cc: stable@dpdk.org
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Running dpdk-pmdinfo.py on Ubuntu 18.04 (bionic) with python 3 and
pyelftools installed produces no output but no error is reported
neither:
~$ python3 usertools/dpdk-pmdinfo.py -r build/app/testpmd
~$ echo $?
0
While with python 2, it works:
~# python2 usertools/dpdk-pmdinfo.py -r build/app/testpmd
{"pci_ids": [], "name": "dpio"}
{"pci_ids": [], "name": "dpbp"}
{"pci_ids": [], "name": "dpaa2_qdma"}
.....
On Ubuntu 18.04, pyelftools is version 0.24. The change log of
pyelftools v0.24 says:
- Symbol/section names are strings internally now, not bytestrings
(this may affect API usage in Python 3) (#76).
We cannot guess which version of pyelftools is actually being used. The
elftools.__version__ symbol is not consistent with each distro's package
version. For example, on Ubuntu 16.04 (xenial), the .deb package version
is '0.23-2' but elftools.__version__ contains '0.25'. This is certainly
due to partial backports.
To have a more consistent behaviour of this script across all versions
of python, add the unicode_literals future import so that literal
strings are now always "unicode".
Add 2 utility functions to force a string into bytes or bytes into an
unicode string.
Force pyelftools return values to unicode strings (will do nothing with
recent version of pyelftools).
If elffile.get_section_by_name returns None with a unicode section name,
try with the same one encoded as bytes.
Also, replace all open() calls by io.open() which behaves like the
builtin open in python 3. The only non-binary opened file is
/usr/share/hwdata/pci.ids which is UTF-8 encoded text. Explicitly
specify that encoding.
Link: https://github.com/eliben/pyelftools/blob/v0.24/CHANGES#L7
Link: https://github.com/eliben/pyelftools/commit/108eaea9e75a8b5a
Fixes: 54ca545dce ("make python scripts python2/3 compliant")
Cc: stable@dpdk.org
Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
This commit removes the unnecesarry ast.literal_eval() function call
from the input handling, which now relies just on raw_input() to get
its input.
Fixes: 53f293c9a7 ("usertools: replace unsafe input function")
Cc: stable@dpdk.org
Signed-off-by: Andrius Sirvys <andrius.sirvys@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
LGTM static code analysis tool reports that the function 'input' is
unsafe. Changed to use raw_input which then converts it using
ast.literal_eval() which is safe.
Fixes: d1b94da4a4 ("usertools: add client script for telemetry")
Cc: stable@dpdk.org
Signed-off-by: Andrius Sirvys <andrius.sirvys@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Bring consistency to error messages and output them to stderr.
Also, whenever the script tells the user to "check usage", don't
tell the user to do it and just display usage instead.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Currently, if an attempt is made to bind a device to a driver that
is not loaded, a confusing and misleading error message appears.
Fix it so that, before binding to the driver, we actually check if
it is loaded in the kernel first.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
A common user error is to forget driver to which the PCI devices should
be bound to. Currently, the error message in this case looks unhelpful
misleading and indecipherable to anyone but people who know how devbind
works.
Fix this by checking if the driver string is actually a valid device
string. If it is, we assume that the user has just forgot to specify the
driver, and display appropriate error. We also assume that no one will
name their driver in a format that looks like a PCI address, but that
seems like a reasonable assumption to make.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Add in the list of registers for the device.
And enable NTB device ops for Intel Skylake platform.
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
Update the devbind script with new section of DMA devices, also
added OCTEONTX2 DMA device ID to DMA device list
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Satha Rao <skoteshwar@marvell.com>
Currently clear_data (dpdk-devbind.py) doesn't work as expected
since "global devices" is missing and so "devices" is considered
a local variable.
This commit changes "clear_data" function in order to really clear
devices by adding "global devices".
Fixes: ea9f00f728 ("usertools: refactor NIC and crypto binding details")
Cc: stable@dpdk.org
Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Reviewed-by: David Marchand <david.marchand@redhat.com>
In order to allow binding/unbinding of devices for use by the
ioat_rawdev, we need to update the devbind script to add a new class
of device, and add device ids for the specific HW instances.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Jiayu Hu <jiayu.hu@intel.com>
Tested-by: Harry van Haaren <harry.van.haaren@intel.com>
telemetry has support for fetching port based stats
from metrics library.
Metrics library also has global stats which are
not fetched by telemetry, so extend telemetry to
fetch the global metrics.
Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
Rather than using linuxapp and bsdapp everywhere, we can change things to
use the, more readable, terms "linux" and "freebsd" in our build configs.
Rather than renaming the configs we can just duplicate the existing ones
with the new names using symlinks, and use the new names exclusively
internally. ["make showconfigs" also only shows the new names to keep the
list short] The result is that backward compatibility is kept fully but any
new builds or development can be done using the newer names, i.e. both
"make config T=x86_64-native-linuxapp-gcc" and "T=x86_64-native-linux-gcc"
work.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Add #! line to the top of the script and mark it as executable so it can
be run directly rather than having to pass it to python interpreter.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
If there aren't any devices of a particular category on user's
system, we still display them, which is bad for usability. Fix
devbind to not print out a category unless there are devices in
it.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: David Hunt <david.hunt@intel.com>
On some distributions (such as CentOS 7) lspci may not be installed
by default, causing exceptions which are difficult to interpret.
Fix devbind script to check if lspci is installed at script startup.
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: Rami Rosen <roszenrami@gmail.com>
This patch adds a python script which can be used as a demo
client. The script is interactive and will allow the user to
register, request statistics, and unregister.
To run the script, an argument for the client file path must
be passed in: "python telemetry_client.py <file_path>".
This script is useful to see how the Telemetry API for DPDK
is used, and how to make the initial connection.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Brian Archbold <brian.archbold@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
lspci reports kernel modules in "Module" string, but devbind
expects it to be "Module_str". Fix it up similar to how we fix
up "Driver" to be "Driver_str".
Fixes: c3ce205d57 ("usertools: optimize lspci invocation")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Have the "usertools" scripts installed when doing a DPDK install using
ninja. They will be copies to $prefix/bin, generally /usr/local/bin,
alongside testpmd.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Keith Wiles <keith.wiles@intel.com>
Acked-by: Luca Boccassi <luca.boccassi@gmail.com>
Replace the BSD license header with the SPDX tag for
scripting files with only an Intel copyright on them.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
When using Python 3, dpdk-devbind.py fails to detect modules other than
igb_uio.
Fixes: bb9f408550 ("tools: support binding to built-in kernel modules")
Signed-off-by: Omri Mor <omrimor2@illinois.edu>
Replace the incorrect reference to "Cavium Networks", "Cavium Ltd"
company name with correct the "Cavium, Inc" company name in
copyright headers.
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
If multiple devices of same class are added to a device type,
only devices that match first device listed in device type list are
processed.
Fixing it in device_type_match() by returning false after iterating
through all the devices listed in a device type list.
Fixes: 8ad08a2879 ("usertools: define DPDK PCI functional device")
Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
These differences in Python 3 were causing errors:
- xrange is replaced by range
- dict values are a view (instead of list)
- has_key is removed
Fixes: deb87e6777 ("usertools: use sysfs for CPU layout")
Fixes: 63985c5f10 ("usertools: fix CPU layout for more than 2 threads")
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Gowrishankar Muthukrishnan <gowrishankar.m@linux.vnet.ibm.com>
The existing code used to search for module files via modinfo has
several corner cases which can result in it failing where it should be
successful.
The call to lower() would cause results returned by 'modinfo' to be
forced to lowercase, results which were subsequently passed to
exists() which is case sensitive. This was most likely done to capture
all variants of failure strings modinfo might return
(ie. ERROR/Error/error/...) without thought negative effect to the
later call to exists(). For many this is a nonissue but if the module
path included non-lowercase alpha characters, something which is
easily possible with a non-lowercase kernel-extraversion string, this
would cause an issue.
We could move the call to lower() to the check for "error" but this
still leaves possible corner cases, for modules or module paths with
'error' in them.
Instead we will prevent modinfo's stderr from being used as a "good
value" for path, meaning we either get a valid path from modinfo, or
nothing at all. This removes all corner cases.
Ultimately these preliminary checks are unnecessary as exists() will
only return True if it is passed a valid path, passing it modinfo's
stderr would fail. In keeping with the original code, however, we do
some preliminary checks, but we are now free of corner cases.
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Current usertools/cpu_layout.py is broken to handle multithreads
of count more than 2 as in IBM powerpc P8 servers.
Below patch addressed this issue.
Also, added minor exception catch on failing to open unavailable
sys file in case of multithread=off configuration in server.
Patch has been verified not to break existing topology configurations
and also not changing anything in current output.
Signed-off-by: Gowrishankar Muthukrishnan <gowrishankar.m@linux.vnet.ibm.com>
Reviewed-by: Andriy Berestovskyy <andriy.berestovskyy@caviumnetworks.com>
Script displays status for all device types and output is much
longer than it used to be. This makes harder to read script output.
This patch adds new --status-dev argument to the script to select
a device group to display status.
Supported device groups:
net
crypto
event
mempool
Sample usage:
./usertools/dpdk-devbind.py --status-dev mempool
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
The following sequence of operation gives error in binding devices
1) Bind a device using dpdk-devbind.py
2) Unbind the device using kernel tools(/sys/bus/pci/device/driver/unbind)
3) Bind the device using kernel tools(/sys/bus/pci/driver/new_id and
/sys/bus/pci/driver/bind)
The bind failure was due to cached driver name in 'driver_override'.
Fix it by writing 'null' to driver_override just after binding a
device so that any method of binding/unbinding can be used.
Fixes: 2fc3502935 ("usertools: use optimized driver override scheme to bind")
Reported-by: Lijuan A Tu <lijuanx.a.tu@intel.com>
Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
This patch creates the framework to define the DPDK PCI functional
device by specifying the pci attributes like Vendor ID, Device ID,
Sub Vendor ID, Sub Device ID and Class.This enables a flexible way to
add DPDK function devices based on PCI attributes.
Crypto devices can belong to Encryption class(0x10) or Processor
class(0x0b) based on the vendor preference.
Using this framework, The above disparity can be encoded in the following
format
encryption_class = [{'Class': '10', 'Vendor': None,
'Device': None, 'SVendor': None, 'SDevice': None}]
intel_processor_class = [{'Class': '0b', 'Vendor': '8086', 'Device': None,
'SVendor': None, 'SDevice': None}]
crypto_devices = [encryption_class, intel_processor_class]
Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
The current device bind model uses /sys/bus/pci/driver/new_id scheme to
bind devices to the driver. This scheme has following operations to bind
a device to the driver.
1) Write device ID and vendor ID to /sys/bus/pci/driver/new_id
2) Write PCI BDF number to /sys/bus/pci/driver/bind
3) On step (1), _All_ the devices that match the device ID and vendor ID
get bound to the driver
4) Except for requested devices, Unbind the remaining devices
In kernels >= 3.15, An alternative scheme driver_override can be used to
bind a device to driver.This scheme has following operations to bind a
device to driver.
1) Write driver to /sys/bus/pci/device/driver_override
2) Write PCI BDF number to /sys/bus/pci/driver/bind
This script detects the presence of /sys/bus/pci/device/driver_override,
if available use optimized bind scheme to bind it
Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
lspci invoked twice over all the pci devices in the system.
The first pass is to extract Numeric IDs and second pass to get extended
device details.
As an optimization, Used lspci with -nn option in get_device_details()
to obtain Numeric ID and extended device details in one shot.
In addition to this, After binding the PCI device, lspci needs to be
invoked again to confirm the proper bind operation. Used a boolean
argument to express this case in get_pci_device_details()
Signed-off-by: Guduri Prathyusha <gprathyusha@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
