- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.
Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.
To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.
For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.
- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.
This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).
In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.
- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.
To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.
- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.
- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.
Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117
This was enumerated with exhaustive search for sys/eventhandler.h includes,
cross-referenced against EVENTHANDLER_* usage with the comm(1) utility. Manual
checking was performed to avoid redundant includes in some drivers where a
common os_bsd.h (for example) included sys/eventhandler.h indirectly, but it is
possible some of these are redundant with driver-specific headers in ways I
didn't notice.
(These CUs did not show up as missing eventhandler.h in tinderbox.)
X-MFC-With: r347984
There is no reason to re-create the command workqueue during healthcare.
This also fixes an issue where a previous work struct may refer to a
destroyed workqueue.
MFC after: 3 days
Sponsored by: Mellanox Technologies
Present code uses lock-less accesses to the dump data to prevent top
level ioctls from blocking bottom-level call to dump. Unfortunately, this
depends on the type stability of the dump data structure, which makes it
non-functional during driver teardown.
Switch to the mutex locking scheme where top levels use the mutex in the
bound regions, while copyouts and drain for completion utilize condvars.
The mutex lifetime is guaranteed to be strictly larger than the time
interval where driver can initiate dump, and most of the control fields
of the old struct mlx5_dump_data are directly embedded into struct
mlx5_core_dev.
Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies
Avoid race for command completion when triggering a command completions event.
Serialize operation by queueing all commands on the same work queue.
This can happen when healthcare triggers.
MFC after: 3 days
Sponsored by: Mellanox Technologies
The command timeout is terribly long, whole two hours. Make it 60s so if
things do go wrong, the user gets feedback in relatively short time, so
they can take corrective actions and/or investigate using tools and such.
Linux commit:
6b6c07bdcdc97ccac2596063bfc32a5faddfe884
MFC after: 3 days
Sponsored by: Mellanox Technologies
Function 'mlx5e_alloc_rx_wqe' can never be inlined because it uses alloca
(override using the always_inline attribute)
MFC after: 3 days
Sponsored by: Mellanox Technologies
Allow more macro arguments and split the variable type and name into
separate arguments. This allows simple and powerful copy and extraction
of values from IFC based structures into SYSCTLs with the use of a single
macro.
MFC after: 3 days
Sponsored by: Mellanox Technologies
Implement a watchdog as part of the healtcare subsystem which
reads the PCI power status during startup and upon the PCI
power status change event and store it into the core device
structure. This value is then exported to user-space via a
read-only SYSCTL. A dmesg print has been added to inform
the admin about the PCI power status.
MFC after: 3 days
Sponsored by: Mellanox Technologies
CM layer calls ib_modify_port() regardless of the link layer.
For the Ethernet ports, qkey violation and Port capabilities
are meaningless. Therefore, always return success for ib_modify_port
calls on the Ethernet ports.
Linux Commit:
ec2558796d25e6024071b6bcb8e11392538d57bf
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
mlx5en(4).
IFM_10G_LR and IFM_40G_ER4 media should be added only if the device
has the needed capability bit set for it.
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
If generation is cleared due to hardware clock failure, check for it before
the divisor is used. Actually clear generation when failure occurs.
While there, stop doing the calculations inside the generation loop. Since
all members of mlx5e_clbr_point are used for calculations, get the
local copy of the structure and use it after generation stabilized.
Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies
This fixes counting issues when the firmware resets the counter during
allocation of the counter set where the counter belongs.
MFC after: 3 days
Sponsored by: Mellanox Technologies
The physical- and virtual- port counters might not reflect the amount
of data received after address filtering. Use the software counters
instead for rx_packets and rx_bytes to know exactly how much data
was received.
MFC after: 3 days
Sponsored by: Mellanox Technologies
mlx5_pci_disable_device is calling pci_disable_device which disables
bus master. No need to explicitly call pci_clear_master.
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
Add mlx5 implementation for the ones defined by the mlxfw
shared module to be used while flashing the device firmware.
The callbacks do their job through the MCQI, MCC and MCDA registers.
Linux commit:
62bd22cf326dc4ac5be673c11cef4602dc1f5e47
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
The current mapping of driver counters to netstat counters is wrong.
For example, a single jabber packet, will cause the Ierrs counter to
count three times.
The work for mapping the hardware and software counters to their right
place in netstat counters were already done in Linux, take that as is
to the FreeBSD driver.
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
To be used by the mlx5 callbacks exposed to the mlxfw module.
Linux commit:
d2ad488b0073bd1a2c3f5d2ea50a7eb632103e5d
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
Enhance MCAM to allow the driver to query which access regs are
supported. For now, expose the regs needed for FW flashing.
Linux commit:
0ab87743cc8c5bcd482daf71961ed5fc45349e01
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
MCC (Management Component Control) allows to control a firmware
component update.
MCDA (Management Component Data Access) allows to read and write
a firmware component.
MCQI (Management Component Query Information) allows to query
information about firmware components.
Linux commit:
4717628938423fcba0aa8fa889e9fed4eb6a655f
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
On load_one, we now cache our capabilities registers internally, similar
to QUERY_HCA_CAP. Capabilities can later be queried using macros
introduced in this patch.
Linux commit:
71862561f3a62015a11de16d1c306481e8415c08
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
Introduced registers will expose capabilities of new registers and
features related to port/management.
Driver will query MCAM and PCAM in order to avoid failing on old
firmwares with lack of support.
Linux commit:
c835ad64683bd3e2d1b31ed2cb1ff4366932edb1
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
PCAM: Ports capabilities mask register.
MCAM: Management capabilities mask register.
PCAM and MCAM registers will provide information regarding firmware
support for different features, in order to avoid cases where new driver
combined with old firmware results in syndromes (for ex. PCIe counters
before this patchset).
Linux commit:
cfdcbceaeffc669b70d904d80a2df9c86c232566
Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies
