Originally, the ena_com memzone counter was shared by ports, which
caused the memzones to be harder to identify and could potentially
lead to race and because of that the counter had to be atomic.
This atomic counter was global variable and it couldn't work in the
multiprocess implementation.
The memzone is now being identified by the local to port memzone counter
and the port ID - both of those information can be found in the shared
data, so it can be probed easily.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Dawid Gorecki <dgr@semihalf.com>
Reviewed-by: Shai Brandes <shaibran@amazon.com>
As the default behavior for arm64 is to alias rte_memcpy as memcpy, ENA
cannot redefine memcpy as rte_memcpy as it would cause nested
declaration.
To make it possible to use optimized memcpy in the ena_com layer on Arm,
the driver now redefines memcpy when it is beneficial:
* For arm64 only when the flag RTE_ARCH_ARM64_MEMCPY was defined
* For arm only when the flag RTE_ARCH_ARM_NEON_MEMCPY was defined
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Dawid Gorecki <dgr@semihalf.com>
Reviewed-by: Shai Brandes <shaibran@amazon.com>
ENA defined its own logger flags for Tx and Rx, but they weren't
technically used anywhere. Those data path loggers weren't used anywhere
after the definition.
This commit uses the generic RTE_ETHDEV_DEBUG_RX and RTE_ETHDEV_DEBUG_TX
flags to define PMD_TX_LOG and PMD_RX_LOG which are now being used on
the data path. The PMD_TX_FREE_LOG was removed, as it has no usage in
the current version of the driver.
RTE_ETH_DEBUG_[TR]X now wraps extra checks for the driver state in the
IO path - this saves extra conditionals on the hot path.
ena_com logger is no longer optional (previously it had to be explicitly
enabled by defining this flag: RTE_LIBRTE_ENA_COM_DEBUG). Having this
logger optional makes tracing of ena_com errors much harder.
Due to ena_com design, it's impossible to separate IO path logs
from the management path logs, so for now they will be always enabled.
Default levels for the affected loggers were modified. Hot path loggers
are initialized with the default level of DEBUG instead of NOTICE, as
they have to be explicitly enabled. ena_com logging level was reduced
from NOTICE to WARNING - as it's no longer optional, the driver should
report just a warnings in the ena_com layer.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shai Brandes <shaibran@amazon.com>
This version update contains:
* memcpy mapping to the dpdk-optimized version.
* ena_com (HAL) update to the latest version (from 18.09.2020).
* Bug fixes for the large LLQ headers and devargs parsing.
* Bug fix for the default ring size.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
pthread_cond_timedwait() may spuriously wakeup according to POSIX.
Therefore it is required to check whether predicate is actually true
before finishing the waiting loop.
Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>
1. As memzone uses unique names, changed alloc coherent macro to use
64 bit size atomic variable to increase the memzone name space
2. "handle" param name change to be consistent with other macros
3. Variable definition displacement
4. Backslash alignment to column 80
Signed-off-by: Amit Bernstein <amitbern@amazon.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>
As in the v20.11 rte_prefetch0_write API was added, it should be used
in the platform file for the definition of the macro prefetchw, instead
of using simply prefetch0.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Amit Bernstein <amitbern@amazon.com>
This feature wasn't used by any of the drivers. Because of that, it was
removed.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>
The RSS indirection table shouldn't be included as a part of the
device features context.
If the driver has to acquire it, it can be done using the separate API
whenever the RSS configuration is needed.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Amit Bernstein <amitbern@amazon.com>
Although the ENA DPDK PMD doesn't have to perform any actions for
destroying the wait event, some other platforms have to.
The macro "ENA_WAIT_EVENT_DESTROY" was renamed to
"ENA_WAIT_EVENTS_DESTROY" and also whole implementation responsible for
that was moved to a separate function for better readability.
Fixes: 3adcba9a8987 ("net/ena: update HAL to the newer version")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
To silence error messages from the static code analysis, make the type
conversions explicit where they're intended.
Also fix the type for the DMA width value.
Fixes: 99ecfbf845b3 ("ena: import communication layer")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
List of changes:
* Comment style was adjusted for the functions
* The keys_num at "struct ena_admin_feature_rss_flow_hash_control" was
renamed to the key_parts to better describe it's meaning
* The RSS indirection table was called "REDIRECTION" -> changed to
INDIRECTION
* Change AENQ field "syndrom" -> "syndrome"
* Calculate number of the RSS key parts or whole key by using the
common way: sizeof of the first element of the RSS key
* Add description of the "enum ena_admin_aq_feature_id"
* Rename "map_rx_buf_bidirectional" field as "rx_buf_mirroring"
* Other minor style fixes (remove extra spaces, add missing line break,
improve indentation)
* Remove unused macros ENA_ADMIN_EXTRA_PROPERTIES_*
* Restructure the "if {} else if {} else" conditional statement for
setting up the meta descriptor
Fixes: 99ecfbf845b3 ("ena: import communication layer")
Fixes: b68309be44c0 ("net/ena/base: update communication layer for the ENAv2")
Fixes: b2b02edeb0d6 ("net/ena/base: upgrade HAL for new HW features")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Some platforms may make use of the device argument to make the logs more
verbose and specific for the appropriate device.
As it's not used by the ENA DPDK PMD for the logging, the type is just
defined, but never used.
It may be reconsidered to change this in the future by adding port ID
to the message logs, but as for now the logging behavior won't change.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Instead of using 'queue' for struct ena_com_admin_queue and 'dev' for
struct ena_com_dev variables, use more descriptive 'admin_queue' and
'ena_dev'.
This also unifies the names of variables of the type struct ena_com_dev
in the driver.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
memcpy is now mapped to rte_memcpy macro on x86 architectures.
Fixes: 9ba7981ec992 ("ena: add communication layer for DPDK")
Cc: stable@dpdk.org
Signed-off-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Artur Rojek <ar@semihalf.com>
Instead of verifying the Rx descriptor each time it's being used in the
driver code, now the verification happens on the HAL side.
This simplifies code a lot as instead of doing 2 validations, only
single one is needed. The driver have to check the rc value returned
by the ena_com upon reading the Rx descriptor and trigger the reset
if needed. It was previously the responsibility of the
validate_rx_req_id() function.
As part of the change, the version of the driver was bumped to v2.2.1.
Fixes: 2061fe41f212 ("net/ena: linearize Tx mbuf")
Cc: stable@dpdk.org
Signed-off-by: Ido Segev <idose@amazon.com>
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Latest generation HW requires IO completion queue descriptors to be
aligned to a 4K in order to achieve the best performance.
Because of that, the new allocation macros were added, which allows
driver to allocate the memory with specified alignment.
The previous allocation macros are now wrappers around the macros
doing the alignment, with the alignment value equal to cacheline size.
Fixes: b68309be44c0 ("net/ena/base: update communication layer for the ENAv2")
Cc: stable@dpdk.org
Signed-off-by: Ido Segev <idose@amazon.com>
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Amit Bernstein <amitbern@amazon.com>
This assert macro is not called anymore.
This also fixes an invalid reference to RTE_LOGTYPE_ERR that does not
exist.
Fixes: 3adcba9a8987 ("net/ena: update HAL to the newer version")
Fixes: 6f1c9df9e9cc ("net/ena: use dynamic log type for debug logging")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>
The current ena_com version was generated on 26.04.2020.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Checking for the cdesc not being NULL doesn't have any sense if the idx
argument is not 0, so it can be skipped, as the error won't be detected
anyway.
To simplify that, only the 'i' value is being verified and the code is
breaking from the infinite loop in case when all descriptors were copied
into the buffer.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
When filling out meta descriptor, all values should be converted to the
desired type (u32 in case of the meta descriptor) to prevent losing the
data.
For example, io_sq->phase is of type u8. If
ENA_ETH_IO_TX_META_DESC_PHASE_SHIFT would be greater then 8, all data
would be lost.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Static code analysis showed up, that it's possible for meta_desc being
NULL. To avoid dereference of the NULL pointer, extra check was added if
the pointer is in fact valid.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
To minimize chance of integer overflow, the type of admin statistics was
changed from u32 to u64.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
To align the error checking code with other parts of the ena_com,
the conditional check is being tested for the error was wrapped inside
unlikely().
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
* Function argument style improvement (space after *)
* Align indentation of the define
* Typo fix in the documentation
* Remove extra empty line after license (aligned with other files)
* Extra alignment of one line was fixed
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Setting RSS hash function could not be supported by the device. In that
situation there is no need to fill in default hash key or even allocate
hash key.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
It's well defined how the RSS key buffer looks from the device
perspective, so the constant value should be used instead of magic
number. Also it doesn't has to be calculated dynamically.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
There is no need to keep single function for both hash function and
the key. If the caller want's to get only single value, then it had to
pass NULL as one of the values, making the API harder to use.
Except reading functions from the device, one can also use function
ena_com_get_current_hash_function() to get the integer value, which
is representing current hash function stored in the ena_com layer.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
The Elastic Netfwork Interface (ENI) stats can be acquired from the HW.
They can provide advanced values which can be further used by the
application for better flow management.
It isn't available to the DPDK application, yet. The PMD must expose
them directly.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
The purpose of this change is general code simplification and
type safety improvement for the logical values.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
As there is no replacement for mmiowb() and there is no need to use both
versions in the DPDK, this ifdef was simply removed.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
The wait event is being accessed without making sure it the completion
context exists. The check for that is just below, so it could be used
for releasing wait even safely.
Fixes: 3adcba9a8987 ("net/ena: update HAL to the newer version")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Instead of the fixes, 5 ms delay in the polling functions, use
values into given range (by default from 100 us 5000 us) and increase
them exponentially each time, the operation isn't finished.
This change can improve responsiveness of the driver for the fast
operations.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
The admin command could return ENA_ADMIN_RESOURCE_BUSY status, which
is meaning that currently the given resource cannot be used.
However, the request can be repeated, so it's being converted to the
ENA_COM_TRY_AGAIN error code.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
ENA_MSLEEP() and ENA_UDELAY() were expecting different behavior - the
first one is expecting driver to sleep, while the other, to busy wait.
For both cases, the rte_delay_(u|m)s() function was used, which could
be either sleep or block, depending on the configuration.
To make the macros valid, the operations should be specified directly.
Because of that, the rte_delay_us_sleep() and rte_delay_us_block() are
now being used.
Fixes: 9ba7981ec992 ("ena: add communication layer for DPDK")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Usage of RTE_MIN(MAX) in ENA_MIN32, ENA_MIN16, ENA_MIN8 (and same for
the MAX), was not enough, as the HAL code is assuming that those macros
will convert both arguments to the specified uintX_t type.
As RTE_MIN(MAX) is using 'typeof' operator, the behavior won't be the
same, especially if arguments has different types (and it could cause
compilation warnings).
To satisfy that, the ENA_MIN_T and ENA_MAX_T macros were added, which
are converting both arguments to the type which is being passed as an
argument.
Fixes: 9ba7981ec992 ("ena: add communication layer for DPDK")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
This patch adds a missing LLQ-related check in the
ena_com_is_doorbell_needed() routine, which is relevant for the feature
supported by the next generation HW of the ENA.
Fixes: b2b02edeb0d6 ("net/ena/base: upgrade HAL for new HW features")
CC: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Signed-off-by: Artur Rojek <ar@semihalf.com>
The current ena_com version was generated on 25.09.2019.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
As the alignment of the defines wasn't valid, it was removed at all, so
instead of using multiple spaces or tabs, the single space after define
name is being used.
Fixes: 99ecfbf845b3 ("ena: import communication layer")
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
Because ena_com is being used by multiple platforms which are using
different C versions, PRIu64 cannot be used directly and must be defined
in the platform file.
Fixes: b2b02edeb0d6 ("net/ena/base: upgrade HAL for new HW features")
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
ENA device is using 48-bit memory for IO. Because of that, the upper
limit had to be updated.
From the driver perspective, it's just a cosmetic change to make
definition of the structure 'ena_common_mem_addr' more descriptive and
the address value was verified anyway for the valid range in the
function 'ena_com_mem_addr_set()'.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
To make the debugging easier, the error logs were added in the Tx path.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
The spaces instead of tabs were used for the indent.
Fixes: 3adcba9a8987 ("net/ena: update HAL to the newer version")
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
The documentation format was aligned and few typos were fixed.
Fixes: 99ecfbf845b3 ("ena: import communication layer")
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
In order to use the accelerated LLQ (Low-lateny queue) mode, the driver
must limit the Tx burst and be aware that the device has the meta
caching disabled. In that situation, the meta descriptor must be valid
on each Tx packet.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
This buffer was never used by the ENA PMD. It could be used for
debugging, but it's presence is redundant now.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
This feature allows for adaptive interrupt moderation. It's not used by
the DPDK PMD, but is a part of the newest HAL version.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
After the indirection table is being saved in the device, there is no
need to convert it back, as it's already saved in host_rss_ind_tbl
array.
As a result, the call to the ena_com_ind_tbl_convert_from_device() is
not needed.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
There was a bug in ena_com_fill_hash_function(), which was causing bit to
be shifted left one bit too much.
To fix that, the ENA_FFS macro is being used (returning the location of
the first bit set), hash_function value is being subtracted by 1 if any
hash function is supported by the device and BIT macro is used for
shifting for better verbosity.
Fixes: 99ecfbf845b3 ("ena: import communication layer")
Cc: stable@dpdk.org
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Although the RSS key still cannot be set, it is now being generated
every time the driver is being initialized.
Multiple devices can still have the same key if they're used by the same
driver.
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
