variables 'out_pkts_total' and 'out_pkts_total' will be in registers.
Hence shifting the code after the loop, helps the update from registers.
Signed-off-by: Vipin Varghese <vipin.varghese@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
For most run cases 'sw->started' holds true. Adding a branch prediction
suggestion to compiler helps as this is first conditional check just
after entering the function.
Signed-off-by: Vipin Varghese <vipin.varghese@intel.com>
Acked-by: Kevin Laatz <kevin.laatz@intel.com>
This commit enables dynamic logging with the SW pmd.
The string "pmd.event.sw" is used to change the verbosity
of the logging output, as per the newly defined log naming.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Add checks during build to ensure that all symbols in the EXPERIMENTAL
version map section have __experimental tags on their definitions, and
enable the warnings needed to announce their use. Also add an
ALLOW_EXPERIMENTAL_APIS define to allow individual libraries and files
to declare the acceptability of experimental api usage
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
align the config option name with config/common_base
Fixes: aaa4a221da26 ("event/sw: add new software-only eventdev driver")
Cc: stable@dpdk.org
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Update software eventdev ops to invoke selftest when application invokes
`rte_event_dev_selftest`.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Modify test_eventdev_sw to be standalone selftest independent of test
framework.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Move software eventdev specific test (test_eventdev_sw) to
driver/event/sw/.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit fixes a bug in which, when the sw PMD is reconfigured, it would
leave stale IQ chunk pointers in each queue's IQ structure. Now, the PMD
initializes all IQs at eventdev start time and releases all IQ chunk
pointers at eventdev stop time (which has the consequence that any events
in a queue when the eventdev is stopped will be lost). This approach should
be resilient to any reconfiguration done between the stop and start, such
as adding or removing queues.
This commit also fixes two potential issues in iq_chunk.h. iq_init()
now initializes the IQ's count field to 0, and iq_dequeue_burst() sets
iq->head to the appropriate next pointer.
Fixes: dca926ca9faa ("event/sw: use dynamically-sized IQs")
Reported-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
This commit modifies the sw PMD credit scheme such that credits are
consumed when enqueueing a NEW event and released when an event is
released -- typically, the beginning and end of a pipeline. Workers that
simply forward events do not interact with the credit pool.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit introduces a capability for disabling the "implicit" release
functionality for a port, which prevents the eventdev PMD from issuing
outstanding releases for previously dequeued events when dequeuing a new
batch of events.
If a PMD does not support this capability, the application will receive an
error if it attempts to setup a port with implicit releases disabled.
Otherwise, if the port is configured with implicit releases disabled, the
application must release each dequeued event by invoking
rte_event_enqueue_burst() with RTE_EVENT_OP_RELEASE or
RTE_EVENT_OP_FORWARD.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit introduces dynamically-sized IQs, by switching the underlying
data structure from a fixed-size ring to a linked list of queue 'chunks.'
This has a number of benefits:
- Certain corner cases were observed in which all of a pipeline's flows
could be pinned to one port for extended periods, effectively turning a
multi-core pipeline into single-core one. This was caused by an event
producer having a larger new_event_threshold than the IQ depth, and
injecting large numbers of packets that are ultimately backpressured in a
worker's rx_ring, causing those packets' flows to be scheduled to that
port.
The dynamically sized IQ does not have this problem because each IQ can
grow large enough to store all the system's events, such that
backpressure will not reach the worker_ring.
- Slight performance improvement (~1-2%) in high throughput scenarios,
tested with eventdev_pipeline_sw_pmd.
This implementation has a small increase in the queue storage memory
footprint (~70KB). This commit also removes the iq_size xstat, which no
longer applies to this implementation.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit reinitializes a queue before it is reconfigured, such that
reorder buffer memory is not leaked.
This bug masked a few other problems, which this commit corrects as well:
- sw_port_link() allowed a port to link to a queue twice, such that the
port could then successfully unlink the queue twice. Now the link
function checks whether a port is already linked to the queue, and if so
returns success but doesn't assign the a port a second slot in the
queue's cq map.
- test_eventdev.c's test_eventdev_unlink() was unlinking a queue twice
from the same port, and expecting the second unlink to succeed. Now the
test unlinks, links, then unlinks again.
- test_eventdev.c's test_eventdev_link_get() was linking a single queue but
expecting the unlink function to return nb_queues (where nb_queues > 1).
The test now checks for a return value of 1.
Fixes: 5ffb2f142d95 ("event/sw: support event queues")
Fixes: 371a688fc159 ("event/sw: support linking queues to ports")
Fixes: f8f9d233ea0e ("test/eventdev: add unit tests")
Cc: stable@dpdk.org
Signed-off-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Replace the BSD license header with the SPDX tag for files
with only an Intel copyright on them.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Move the vdev bus from lib/librte_eal to drivers/bus.
As the crypto vdev helper function refers to data structure
in rte_vdev.h, so we move those helper function into drivers/bus
too.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
The memzone header is often included without good reason.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
remove eventdev schedule api and enforce sw driver to use service core
feature for event scheduling.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Extend the service capability of the sw event device by exposing service id
to the application.
The application can use service id to configure service cores to run event
scheduling.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
With the current scheme of event queue configuration the cfg schedule
type macros (RTE_EVENT_QUEUE_CFG_*_ONLY) are inconsistent with the
event schedule type (RTE_SCHED_TYPE_*) this requires unnecessary
conversion between the fastpath and slowpath API's while scheduling
events or configuring event queues.
This patch aims to fix such inconsistency by using event schedule
types (RTE_SCHED_TYPE_*) for event queue configuration.
This patch also fixes example/eventdev_pipeline_sw_pmd as it doesn't
convert RTE_EVENT_QUEUE_CFG_*_ONLY to RTE_SCHED_TYPE_* which leads to
improper events being enqueued to the eventdev.
Fixes: adb5d5486c39 ("examples/eventdev_pipeline_sw_pmd: add sample app")
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit allows the xstats_get() API to return just a single
value based on its ID. Previously, the "ret_n_lt_stats" value
set to 1 ensured a check to take place that the array was larger
than the available xstats.
The xstats_get() API allows retriving of individual stats - hence
this check should be removed - by setting "ret_n_lt_stats" to zero.
Fixes: c1ad03df7ad5 ("event/sw: support xstats")
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
When forwarding or releasing events, the operation would fail if the port
has 0 inflight credits and cannot acquire more, or the inflight count
exceeds the port's new event threshold.
This patch fixes that by counting the number of new events in the burst,
and applying the credit and new event threshold checks accordingly.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
The list of libraries in LDLIBS was generated from the DEPDIRS-xyz
variable. This is valid when the subdirectory name match the library
name, but it's not always the case, especially for PMDs.
The patches removes this feature and explicitly adds the proper
libraries in LDLIBS.
Some DEPDIRS-xyz variables become useless, remove them.
Reported-by: Gage Eads <gage.eads@intel.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
Naming convention for event drivers is "rte_pmd_<name>_event_version.map"
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit reworks the API to move from two separate start
and stop functions, to a "runstate" API which allows setting
the runstate. The is_running API is replaced with an function
to query the runstate. The runstate functions take a id value
for service. Unit tests and the eventdev sw pmd are updated.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
This commit reworks the service register function to accept
an extra parameter. The parameter is a uint32_t *, which when
provided will be set to the integer service_id that the newly
registered service is represented by.
This is useful for services that wish to validate settings at
a later point in time - they need to know their own service id.
This commit updates the eventdev sw pmd, as well as unit tests
to use the new register API.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
This commit shows how easy it is to enable a specific
DPDK component with a service callback, in order to get
CPU cycles for it.
The beauty of this method is that the service is unaware
of how much CPU time it is getting - the application can
decide how to split and slice cores and map them to the
registered services.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Now that we have a standard event ring implementation for passing events
core-to-core, use that in place of the custom event rings in the software
eventdev.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Introducing the rte_event_enqueue_new_burst() for enabling the
PMD, an optimization opportunity to optimize if all the events in
the enqueue burst has the op type of RTE_EVENT_OP_FORWARD.
If a PMD does not have any optimization opportunity
for this operation then the PMD can choose the generic enqueue
burst PMD callback as the fallback.
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Introducing the rte_event_enqueue_new_burst() for enabling the
PMD, an optimization opportunity to optimize if all the events in
the enqueue burst has the op type of RTE_EVENT_OP_NEW.
If a PMD does not have any optimization opportunity
for this operation then the PMD can choose the generic enqueue
burst PMD callback as the fallback.
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Gage Eads <gage.eads@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Made libeventdev library independent of VDEV bus by moving vdev pmd
specific function to rte_eventdev_pmd_vdev.h header file. Eventdev VDEV
PMD can include that for generic eventdev VDEV init and uninit function
enablement.
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
This commit fixes the counting of mapped queues to a port,
when the type of queue type is PARALLEL. Not incrementing
the count here could lead to an underflow of the count when
unlinking at a later date.
Fixes: 371a688fc159 ("event/sw: support linking queues to ports")
Reported-by: Jesse Bruni <jesse.bruni@intel.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Single-link optimized ports previously did not correctly track
credits when dequeued, and re-enqueued as a FORWARD type. This
could "inflate" the number of credits in the system.
A unit test is added to reproduce and verify the issue, and the
fixed implementation counts FORWARD packets, and reduces the
number of credits the port has if it is of single-link type.
Fixes: 656af9180014 ("event/sw: add worker core functions")
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Gage Eads <gage.eads@intel.com>
This commit adds a new statistic to the SW eventdev PMD.
The statistic shows how many packets were sent from a
queue to a port. This provides information on how traffic
from a specific queue is being load-balanced to worker cores.
Note that these numbers should be compared across all queue
stages - the load-balancing does not try to perfectly share
each queue's traffic, rather it balances the overall traffic
from all queues to the ports.
The statistic is printed from the rte_eventdev_dump() function,
as well as being made available via the xstats API.
Unit tests have been updated to expect more per-queue statistics,
and the correctness of counts and counts after reset is verified.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Different drivers use internal macros like force_inline for compiler
always inline feature.
Standardizing it through __rte_always_inline macro.
Verified the change by comparing the output binary file.
No difference found in the output binary file with this change.
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
When taking events from a port, we checked the history list to check if the
event needed to be put back in order i.e. originally came from a reordered
queue type. The check for reordering involved checking if the reorder
buffer entry pointer was null. However, after that pointer was used it was
never cleared to null again.
This caused problems when we had mixed reordered and atomic or parallel
events, as the events from the latter two queue types were misidentified as
needing reordering. This let in some cases to crashes, but mostly led to
dropping events, and then application lock-up.
Fixes: 617995dfc5b2 ("event/sw: add scheduling logic")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
This patch returns a credit when an rte_event is
enqueued with an invalid queue_id. Previously a
credit was leaked from the system.
Note that the eventdev instance does not attempt
to free any resources that the rte_event owns. As
a result, resources owned by the rte_event are leaked.
Eg. if the rte_event represents an rte_mbuf, the mbuf
will not be freed, and causes a leak from the mempool.
Fixes: 656af9180014 ("event/sw: add worker core functions")
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>
The flow id of packets was not being hashed on ingress
on an ordered queue. Fix by applying same hashing as is
applied in the atomic queue case. The hashing itself is
broken out into a macro to avoid duplication of code.
Fixes: 617995dfc5b2 ("event/sw: add scheduling logic")
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
This is a preparation to embed the generic rte_device into the rte_eth_dev
also for virtual devices.
Signed-off-by: Jan Blunck <jblunck@infradead.org>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
build error:
.../event/sw/sw_evdev_worker.c: In function ‘sw_event_release’:
.../event/sw/sw_evdev_worker.c:52:3: error: unknown field ‘op’ specified
in initializer
Fixed by updating struct initialization.
Fixes: 656af9180014 ("event/sw: add worker core functions")
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Add support for xstats to report out on the state of the eventdev.
Useful for debugging and for unit tests, as well as observability
at runtime and performance tuning of apps to work well with the
scheduler.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>
Segfault issue resolved when only partially configured and
rte_event_dev_dump() is called before start(),
Reported-by: Vipin Varghese <vipin.varghese@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Add in the scheduling function which takes the events from the
producer queues and buffers them before scheduling them to consumer
queues. The scheduling logic includes support for atomic, reordered,
and parallel scheduling of flows.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Gage Eads <gage.eads@intel.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>
