This patch fixes the situation where data path does not work properly
when vhost reconnects to virtio in server mode with packed ring.
Currently, virtio and vhost share memory of vring. For split ring, vhost
can read the status of descriptors directly from the available ring and
the used ring during reconnection. Therefore, the data path can
continue.
But for packed ring, when reconnecting to virtio, vhost cannot get the
status of descriptors via the descriptor ring. By resetting Tx
and Rx queues, the data path can restart from the beginning.
Fixes: 4c3f5822eb ("net/virtio: add packed virtqueue defines")
Cc: stable@dpdk.org
Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The function vhost_kernel_tap_set_offload() could return errors,
the return value need to be checked. And there is no need to fail
when error is -ENOTSUP.
Fixes: 1db4d2330b ("net/virtio-user: check negotiated features before set")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
This patch supports the feature that the driver passes extra data
(besides identifying the virtqueue) in its device notifications,
expanding the notifications to include the avail index and avail
wrap counter.
Signed-off-by: Cheng Jiang <cheng1.jiang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Do not close the tap fds when disabling queue pairs, instead,
we just need to unbind the backend. Otherwise, tap port can be
destroyed unexpectedly.
Fixes: e3b434818b ("net/virtio-user: support kernel vhost")
Cc: stable@dpdk.org
Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Valgrind complains that virtio_user is calling close(-1).
Fix this by adding check in virtio that is similar to existing code.
Fixes: 37a7eb2ae8 ("net/virtio-user: add device emulation layer")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
The "addr" param has been used since the event callbacks are enabled
for external memory. So the "__rte_unused" should be dropped.
Besides, slightly refine the coding style by consistently assuming
tabs are 8 characters.
Fixes: f32c7c9de9 ("malloc: enable event callbacks for external memory")
Cc: stable@dpdk.org
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Merge all versions in linker version script files to DPDK_20.0.
This commit was generated by running the following command:
:~/DPDK$ buildtools/update-abi.sh 20.0
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Since the library versioning for both stable and experimental ABI's is
now managed globally, the LIBABIVER and version variables no longer
serve any useful purpose, and can be removed.
The replacement in Makefiles was done using the following regex:
^(#.*\n)?LIBABIVER\s*:=\s*\d+\n(\s*\n)?
(LIBABIVER := numbers, optionally preceded by a comment and optionally
succeeded by an empty line)
The replacement for meson files was done using the following regex:
^(#.*\n)?version\s*=\s*\d+\n(\s*\n)?
(version = numbers, optionally preceded by a comment and optionally
succeeded by an empty line)
[David]: those variables are manually removed for the files:
- drivers/common/qat/Makefile
- lib/librte_eal/meson.build
[David]: the LIBABIVER is restored for the external ethtool example
library.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
As doc mentioned, Rx/Mac/vlan filters are all supported by best effort.
These control commands should return success.
Fixes: f9b9d1a557 ("net/virtio-user: add multiple queues in device emulation")
Cc: stable@dpdk.org
Signed-off-by: Marvin Liu <yong.liu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Virtio spec only set rule that packed ring maximum size is up to 2^15
entries. Should not limit packed ring size to power of two.
Fixes: aea29aa5d3 ("net/virtio: enable packed virtqueues by default")
Cc: stable@dpdk.org
Signed-off-by: Marvin Liu <yong.liu@intel.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Missing parenthesis around expression before type cast to struct
virtio_net_hdr pointer makes the arithmetic to be in
sizeof(struct virtio_net_hdr) units.
Use rte_pktmbuf_mtod_offset() to fix the problem.
Type of head_size is changed to signed since some compilers bark
on unary minus applied to unsigned.
Fixes: 1ae55ad38e ("net/virtio: fix mbuf data and packet length mismatch")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This driver supports none of DCB or VMDQ modes, therefore must
check and return error if configured incorrectly.
Fixes: c1f86306a0 ("virtio: add new driver")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This driver supports none of DCB, RSS or VMDQ modes, therefore must
check and return error if configured incorrectly.
Virtio can distribute Rx packets across multi-queue, but there is
no controls (algorithm, redirection table, hash function) except
number of Rx queues and ETH_MQ_RX_NONE is the best fit meaning
no method is enforced on how to route packets to MQs.
Fixes: c1f86306a0 ("virtio: add new driver")
Cc: stable@dpdk.org
Signed-off-by: Dilshod Urazov <dilshod.urazov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Deferred start Tx queue is not supported by the driver.
Fixes: 0748be2cf9 ("ethdev: queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Dilshod Urazov <dilshod.urazov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Deferred start Rx queue is not supported by the driver.
Fixes: 0748be2cf9 ("ethdev: queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Dilshod Urazov <dilshod.urazov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Previous fix removes usage of rte_pktmbuf_prepend() to get pointer
to virtio net header which changes mbuf data_off and data_len.
Size of virtio net header is added to segment length when Tx descriptor
is composed, but segment address (calculated using data_off) is not
adjusted to take size of virtio net header into account.
Fixes: 1ae55ad38e ("net/virtio: fix mbuf data and packet length mismatch")
Cc: stable@dpdk.org
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
The loop to read packets does not take all packets as the number of
available packets (nb_used) is decremented in the loop.
Fixes: 52b5a707e6 ("net/virtio: add Altivec Rx")
Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
With vectorized functions, only the rx stats for number of packets is
incremented.
Update also the other statistics.
Fixes: 52b5a707e6 ("net/virtio: add Altivec Rx")
Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
In case VIRTIO_F_ORDER_PLATFORM(36) is not negotiated, then the frontend
and backend are assumed to be implemented in software, that is they can
run on identical CPUs in an SMP configuration.
Thus a weak form of memory barriers like rte_smp_r/wmb, other than
rte_cio_r/wmb, is sufficient for this case(vq->hw->weak_barriers == 1)
and yields better performance.
For the above case, this patch helps yielding even better performance
by replacing the two-way barriers with C11 one-way barriers for used
flags in packed ring.
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
In case VIRTIO_F_ORDER_PLATFORM(36) is not negotiated, then the frontend
and backend are assumed to be implemented in software, that is they can
run on identical CPUs in an SMP configuration.
Thus a weak form of memory barriers like rte_smp_r/wmb, other than
rte_cio_r/wmb, is sufficient for this case(vq->hw->weak_barriers == 1)
and yields better performance.
For the above case, this patch helps yielding even better performance
by replacing the two-way barriers with C11 one-way barriers for avail
flags in packed ring.
Meanwhile, a read barrier is required to ensure ordering between
descriptor's flags and content reads [1]. With C11, load-acquire can
enforce the ordering instead of rmb barrier.
[1] https://patchwork.dpdk.org/patch/49109/
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
On x86, iopl permissions are only available to root user (or users that
have the CAP_SYS_RAWIO capability).
But those permissions are only needed when the virtio devices accesses
are done with inb/outb instructions, which is when the device is bound
to a UIO kernel module.
So far, the virtio driver was refusing to register based on the check
on IO permissions.
This check does not make sense when binding the device to vfio.
Now that the check on IO permissions has been abstracted in the ioport
API, we can remove it on virtio side.
We still need to call rte_eal_iopl_init() in the virtio constructor so
that the interrupt thread inherits this permission in the case it could
be used with UIO later.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Tiwei Bie <tiwei.bie@intel.com>
If reserve virtio header room by function rte_pktmbuf_prepend, both
segment data length and packet length of mbuf will be increased.
Data length will be equal to descriptor length, while packet length
should be decreased as virtio-net header won't be taken into packet.
Thus will cause mismatch in mbuf structure. Fix this issue by access
mbuf data directly and increase descriptor length if it is needed.
Fixes: 58169a9c81 ("net/virtio: support Tx checksum offload")
Fixes: 892dc798fa ("net/virtio: implement Tx path for packed queues")
Fixes: 4905ed3a52 ("net/virtio: optimize Tx enqueue for packed ring")
Fixes: e5f456a98d ("net/virtio: support in-order Rx and Tx")
Cc: stable@dpdk.org
Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Marvin Liu <yong.liu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The loop to read packets does not take all packets as the number of
available packets (nb_used) is decremented in the loop.
Take all available packets provides a performance improvement of 3%.
Fixes: fc3d66212f ("virtio: add vector Rx")
Cc: stable@dpdk.org
Signed-off-by: Thibaut Collet <thibaut.collet@6wind.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
With vectorized functions, only the rx stats for number of packets is
incremented.
Update also the other statistics.
Performance impact is about 2%
Fixes: fc3d66212f ("virtio: add vector Rx")
Cc: stable@dpdk.org
Signed-off-by: Thibaut Collet <thibaut.collet@6wind.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Check whether space are enough before burst enqueue operation. If more
space is needed, will try to clean up used descriptors for space on
demand. It can give more chances to free used descriptors, thus will
help RFC2544 performance. Also deduct failed xmit packets from total
xmit number.
Fixes: e5f456a98d ("net/virtio: support in-order Rx and Tx")
Cc: stable@dpdk.org
Signed-off-by: Marvin Liu <yong.liu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
When doing xmit in-order enqueue, packets are buffered and then flushed
into avail ring. Buffered packets can be dropped due to insufficient
space. Moving stats update action just after successful avail ring
updates can guarantee correctness.
Fixes: e5f456a98d ("net/virtio: support in-order Rx and Tx")
Cc: stable@dpdk.org
Signed-off-by: Marvin Liu <yong.liu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Enabling/disabling of allmulticast mode is not always successful and
it should be taken into account to be able to handle it properly.
When correct return status is unclear from driver code, -EAGAIN is used.
Signed-off-by: Ivan Ilchenko <ivan.ilchenko@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Hyong Youb Kim <hyonkim@cisco.com>
Change return value of the callbacks from void to int. Make
implementations across all drivers return negative errno
values in case of error conditions.
Both callbacks are updated together because a large number of
drivers assign the same function to both callbacks.
Signed-off-by: Igor Romanov <igor.romanov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Enabling/disabling of promiscuous mode is not always successful and
it should be taken into account to be able to handle it properly.
When correct return status is unclear from driver code, -EAGAIN is used.
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Acked-by: Hyong Youb Kim <hyonkim@cisco.com>
Added the file virtio_rxtx_simple_altivec.c which implements Altivec
code for the virtio vectorized RX functions. Updated the various build
files.
Signed-off-by: David Christensen <drc@linux.vnet.ibm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Change eth_dev_infos_get_t return value from void to int.
Make eth_dev_infos_get_t implementations across all drivers to return
negative errno values if case of error conditions.
Signed-off-by: Ivan Ilchenko <ivan.ilchenko@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Restrict this header inclusion to its real users.
Fixes: 028669bc9f ("eal: hide shared memory config")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
The simple Tx path in virtio has been removed in below commit.
This patch removes an undefined function declaration of simple
Tx and all related descriptions in the doc.
Fixes: 57f818963d ("net/virtio: remove simple Tx path")
Cc: stable@dpdk.org
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Add extern to variable declaration to avoid some compiler treating it
as variable definition.
build error log:
lib/librte_pmd_virtio.a(vhost_kernel.o):(.rodata+0x110):
multiple definition of `vhost_msg_strings'
lib/librte_pmd_virtio.a(vhost_user.o):(.data.rel.ro.local+0x0):
first defined here
lib/librte_pmd_virtio.a(virtio_user_dev.o):(.rodata+0xe8):
multiple definition of `vhost_msg_strings'
lib/librte_pmd_virtio.a(vhost_user.o):(.data.rel.ro.local+0x0):
first defined here
Fixes: 33d24d65fe ("net/virtio-user: abstract backend operations")
Cc: stable@dpdk.org
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This Tx counter has never been used.
Fixes: 9658d17da2 ("virtio: maintain stats per queue")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
The rte_vdev_driver is declared twice.
The first one is not necessary.
Fixes: 050fe6e9ff ("drivers/net: use ethdev allocation helper for vdev")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Replace rte_intr_enable() with rte_intr_ack() API
for acking an interrupt in interrupt handlers and
rx_queue_intr_enable() callbacks of PMD's.
This is inline with original intent of this change in PMDs
to ack interrupts after handling is completed if
device is backed by UIO, IGB_UIO or VFIO(with INTx).
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Shahed Shaikh <shshaikh@marvell.com>
Tested-by: Shahed Shaikh <shshaikh@marvell.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Use rte_ether_unformat_addr rather than sscanf.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Currently, the memory hotplug is locked automatically by all
memory-related _walk() functions, but sometimes locking the
memory subsystem outside of them is needed. There is no
public API to do that, so it creates a dependency on shared
memory config to be public. Fix this by introducing a new
API to lock/unlock the memory hotplug subsystem.
Create a new common file for all things mem config, and a
new API namespace rte_mcfg_*, and search-and-replace all
usages of the locks with the new API.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: David Marchand <david.marchand@redhat.com>
We should check the descriptor state instead of vq's internal
free count (i.e. the number of descriptors that we haven't made
available) for the remaining mergeable packets.
Fixes: a76290c8f1 ("net/virtio: implement Rx path for packed queues")
Cc: stable@dpdk.org
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
When there is no enough segments for a packet in mergeable
packed Rx path, we should free the whole mbuf chain instead
of just recycling the last segment.
Fixes: a76290c8f1 ("net/virtio: implement Rx path for packed queues")
Cc: stable@dpdk.org
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
When there is no enough segments for a packet in mergeable
Rx path, we should free the whole mbuf chain instead of just
recycling the last segment.
Fixes: bcac5aa207 ("net/virtio: improve batching in mergeable path")
Cc: stable@dpdk.org
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
When there is no enough segments for a packet in in-order
mergeable Rx path, we should free the whole mbuf chain instead
of just recycling the last segment.
Fixes: e5f456a98d ("net/virtio: support in-order Rx and Tx")
Cc: stable@dpdk.org
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
VLAN tag insertion should be in Tx prepare, not in Tx burst functions.
One of Tx prepare goals is to be able to do preparations in advance
(possibly on different CPU core) and then transmit it fast.
Also Tx prepare can report that a packet does not pass Tx offloads
check. E.g. has no enough headroom to insert VLAN header.
Fixes: 4fb7e803eb ("ethdev: add Tx preparation")
Cc: stable@dpdk.org
Signed-off-by: Dilshod Urazov <dilshod.urazov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Virtio requires pseudo-header checksum in TCP/UDP checksum to do
offload, but it was lost when Tx prepare is introduced. Also
rte_validate_tx_offload() should be used to validate Tx offloads.
Also it is incorrect to do virtio_tso_fix_cksum() after prepend
to mbuf without taking prepended size into account, since layer 2/3/4
lengths provide incorrect offsets after prepend.
Fixes: 4fb7e803eb ("ethdev: add Tx preparation")
Cc: stable@dpdk.org
Signed-off-by: Dilshod Urazov <dilshod.urazov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Tiwei Bie <tiwei.bie@intel.com>
Set RTE_ETH_DEV_CLOSE_REMOVE upon probe so all the private
resources for the port can be freed by rte_eth_dev_close().
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
We should unmap the device when we failed to initialize the device.
Fixes: 6ba1f63b5a ("virtio: support specification 1.0")
Cc: stable@dpdk.org
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
For legacy devices, we should also unmap the port IO
resource on device removal.
Fixes: b8f04520ad ("virtio: use PCI ioport API")
Cc: stable@dpdk.org
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
