Convert PCIe hot plug support over to asking the firmware, if any, for
permission to use the HotPlug hardware. Implement pci_request_feature
for ACPI. All other host pci connections to allowing all valid feature
requests.
Sponsored by: Netflix
pcib_request_feature allows drivers to request the firmware (ACPI)
release certain features it may be using. ACPI normally manages things
like hot plug, advanced error reporting and other features until the
OS requests ACPI to relenquish control since it is taking over.
Sponsored by: Netflix
As of r313097, the HotPlug code requires the link to support
reporting of the data-link status. Remove tests for this capability
from code that can now assume its presence.
Suggested by: jhb
Reviewed by: jhb
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D9431
Some PCI-e bridges report that they support HotPlug in the slot
capabilities but do not report support for Data Layer Active events
in the link capabilities register. These bridges do not work correctly
when HotPlug is used. Further, while the description of HotPlug in
the spec does not mention that DL active events are required, the
description of the link capabilities register says that DL active is
required for HotPlug. Thanks to Dave Baukus for finding that language
in the spec.
PR: 211699
Submitted by: Dave Baukus <daveb@spectralogic.com>
Reviewed by: vangyzen
MFC after: 3 days
Replace archaic "busses" with modern form "buses."
Intentionally excluded:
* Old/random drivers I didn't recognize
* Old hardware in general
* Use of "busses" in code as identifiers
No functional change.
http://grammarist.com/spelling/buses-busses/
PR: 216099
Reported by: bltsrc at mail.ru
Sponsored by: Dell EMC Isilon
This patch solves IRQ generation problems using the mlx5en(4) driver
with xenserver v6.5.0 in SRIOV and PCI-passthrough modes.
Until further the hw.pci.msix_rewrite_table quirk must be set manually
in /boot/loader.conf .
Reviewed by: jhb @
Sponsored by: Mellanox Technologies
MFC after: 2 weeks
ARM GIC specification in device trees use 3 cells, so the current
limit of 2 causes the last cell to be dropped. This in turn can
cause the interrupt polarity and trigger settings to be incorrect.
Increase the limit to 4 which should handle all reasonable cases.
This fixes issues seen in QEMU when registering PCI interrupts.
FDT attachment to a new file. A separate ACPI attachment will then be added
to allow arm64 servers with ACPI to use it over FDT.
This should also help with merging this with the ofwpci driver, with
further work needed to remove restrictions this driver places on resource
allocation.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D7319
It's unsafe to update the BAR when the related EN bit is set.
Submitted by: Dexuan Cui <decui microsoft com>
Reviewed by: jhb
MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7914
During a bus rescan the check for an invalid vendor ID of a subfunction
used the wrong constant.
Submitted by: Dexuan Cui <decui@microsoft.com>
MFC after: 3 days
Add routines to trigger a function level reset (FLR) of a PCI-express
device via the PCI-express device control register. This also includes
support routines to wait for pending transactions to complete as well
as calculating the maximum completion timeout permitted by a device.
Change the ppt(4) driver to reset pass through devices before attaching
to a VM during startup and before detaching from a VM during shutdown.
Reviewed by: imp, wblock (earlier version)
MFC after: 1 month
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7751
When the I/O MMU is active in bhyve, all PCI devices need valid entries
in the DMAR context tables. The I/O MMU code does a single enumeration
of the available PCI devices during initialization to add all existing
devices to a domain representing the host. The ppt(4) driver then moves
pass through devices in and out of domains for virtual machines as needed.
However, when new PCI devices were added at runtime either via SR-IOV or
HotPlug, the I/O MMU tables were not updated.
This change adds a new set of EVENTHANDLERS that are invoked when PCI
devices are added and deleted. The I/O MMU driver in bhyve installs
handlers for these events which it uses to add and remove devices to
the "host" domain.
Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7667
Other files including pci_host_generic.h failed to compile
due to missing declaration of enum pci_id_type.
Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Reviewed by: wma
Differential Revision: https://reviews.freebsd.org/D7561
It seems Killer E2200/E2400 has a BIOS misconfiguration or silicon
bug which triggers DMA write errors when driver uses advertised
maximum payload size. Force the maximum payload size to 128 bytes
in DMA configuration.
This change should fix occasional DMA write errors reported on
Killer E2200.
Tested by: <psy0nic@sys-tek.org>
- Read interrupt properties at bus enumeration time and store
it into global mapping table.
- At bus_activate_resource() time, given mapping entry is resolved and
connected to real interrupt source. A copy of mapping entry is attached
to given resource.
- At bus_setup_intr() time, mapping entry stored in resource is used
for delivery of requested interrupt configuration.
- For MSI/MSIX interrupts, mapping entry is created within
pci_alloc_msi()/pci_alloc_msix() call.
- For legacy PCI interrupts, mapping entry must be created within
pcib_route_interrupt() by pcib driver itself.
Reviewed by: nwhitehorn, andrew
Differential Revision: https://reviews.freebsd.org/D7493
Some devices report that they have an MRL when they actually
do not. Since they always report that the MRL is open, child
devices would be ignored. Try to detect these devices and
ignore their claim of HotPlug support. Specifically,
if there is an open MRL but the Data Link Layer is active,
the MRL is not real.
Revert r303645 to re-enable HotPlug support for slots with
power controllers, since it works correctly in my testing.
Start the DLL state-change timer if Presence /or/ MRL state changes,
along with other conditions. Previously, we started the timer iff
Presence changed. If there is an MRL, it must be closed for power
to be turned on, so Presence is unlikely to change on an MRL-close event.
Add a printf() of interesting registers on HotPlug interrupts and
commands (one from erj@). These were very useful for debugging.
Guard them with bootverbose, since they're spam in normal operation.
In collaboration with: jhb
Reviewed by: jhb
MFC after: 1 day
Relnotes: yes (re-enable HotPlug support for slots with power controllers)
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D7509
Several files use the internal name of `struct device` instead of
`device_t` which is part of the public API. This patch changes all
`struct device *` to `device_t`.
The remaining occurrences of `struct device` are those referring to the
Linux or OpenBSD version of the structure, or the code is not built on
FreeBSD and it's unclear what to do.
Submitted by: Matthew Macy <mmacy@nextbsd.org> (previous version)
Approved by: emaste, jhibbits, sbruno
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D7447
Previously the loop in PCIIOCGETCONF would terminate as soon as it
found enough matches. Now it will continue iterating through the
PCI device list and only terminate if it finds another matching device
for which it has no room to store a conf structure. This means that
PCI_GETCONF_LAST_DEVICE is reliably returned when the number of
matching devices is equal to the number of slots in the matches
buffer. For example, if a program requests the conf structure for a
single PCI function with a specified domain/bus/slot/function it will
now get PCI_GETCONF_LAST_DEVICE instead of PCI_GETCONF_MORE_DEVS.
While here, simplify the loop conditional a bit more by explicitly
breaking out of the loop if copyout() fails and removing a redundant
i < pci_numdevs check.
Reviewed by: vangyzen, imp
MFC after: 1 month
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7445
The interpretation of the Electromechanical Interlock Status was
inverted, so we disengaged the EI if a card was inserted.
Fix it to engage the EI if a card is inserted.
When displaying the slot capabilites/status with pciconf:
- We inverted the sense of the Power Controller Control bit,
saying the power was off when it was really on (according to
this bit). Fix that.
- Display the status of the Electromechanical Interlock:
EI(engaged)
EI(disengaged)
Reviewed by: jhb
MFC after: 3 days
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D7426
The PCI_IOV option creates character devices in /dev/iov for each PF
device driver that registers support for creating VFs. By default the
character device is named after the PF device (e.g. /dev/iov/foo0).
This change adds a variant of pci_iov_attach() called pci_iov_attach_name()
that allows the name of the /dev/iov entry to be specified by the
driver.
Reviewed by: rstone
MFC after: 1 month
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7400
After further review of the spec, I do not think the current HotPlug
code handles slots with power controllers correctly. In particular,
the power state of the slot is to be inferred from other events, not
from examining the state of the power control bit in SLOT_CTL. For now,
disable PCI hotplug support on such slots.
PR: 211081
Tested by: Jeffrey E Pieper <jeffrey.e.pieper@intel.com>
MFC after: 3 days
Some systems and/or devices (such as riser cards) do not include a
non-compliant implementation of PCI-e HotPlug that can result in devices
not being attached (e.g. the HotPlug code might assume that a card is
being unplugged and will power the slot off and detach it). This
tunable can be set to 0 to disable support for PCI-e HotPlug ignoring
the incorrect HotPlug state on these slots.
PR: 211081
Reported by: Sergey Renkas <serg_ic@mail.ru> (SuperMicro X7 riser card)
Reported by: Jeffrey E Pieper <jeffrey.e.pieper@intel.com>
(Intel X520 adapter)
MFC after: 1 week
Relnotes: yes
this to be the case. This will mean we don't try and handle the cache in
bus_dmamap_sync when it is not needed.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6605
tested on the Pass 1.1 and 2.0 ThunderX machines in the Netperf cluster.
Reviewed by: jhb
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6453
- Add a pcib_detach() function for the PCI-PCI bridge driver. It
tears down the NEW_PCIB and hotplug state including destroying
resource managers, deleting child devices, and disabling hotplug
events.
- Add a detach method to the ACPI PCI-PCI bridge driver which calls
pcib_detach() and then frees the copy of the _PRT interrupt routing
table.
- Add a detach method to the PCI-Cardbus bridge driver which frees
the PCI bus resources in addition to calling cbb_detach().
- Explicitly clear any pending hotplug events during attach to ensure
future events will generate an interrupt.
- If a the Command Completed bit is set in the slot status register
when the command completion timeout fires, treat it as if the
command completed and the completion interrupt was just lost rather
than forcing a detach.
- Don't wait for a Command Completed notification if Command Completion
interrupts are disabled. The spec explicitly says no interrupt is
enabled when clearing CCIE, and on my T400 no interrupt is generated
when CCIE is changed from cleared to set, either. In addition, the
T400 doesn't appear to set the Command Completed bit in the cases
where it doesn't generate an interrupt, so don't schedule the timer
either. (If the CC bit were always set, one could always set the timer
and rely on the logic of treating CC set as a missed interrupt.)
Reviewed by: imp (older version)
Differential Revision: https://reviews.freebsd.org/D6424
Previously the command completion interrupt would post any pending
command immediately before pcib_pcie_hotplug_update() had been
run to inspect the current status. Now, the command completion
interrupt merely clears the flag and stops the timer assuming that
the caller is always going to call pcib_pcie_hotplug_update() to
generate the next hotplug command if one is needed.
While here, fix a bug for systems with command completion where the
old (existing) value was written to the slot control register instead
of the new value. This fixes the complaint about a missing hotplug
interrupt on my T400.
Differential Revision: https://reviews.freebsd.org/D6363
translate the pci rid to a controller ID. The translation could be based
on the 'msi-map' OFW property, a similar ACPI option, or hard-coded for
hardware lacking the above options.
Reviewed by: wma
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Add a new get_id interface to pci and pcib. This will allow us to both
detect failures, and get different PCI IDs.
For the former the interface returns an int to signal an error. The ID is
returned at a uintptr_t * argument.
For the latter there is a type argument that allows selecting the ID type.
This only specifies a single type, however a MSI type will be added
to handle the need to find the ID the hardware passes to the ARM GICv3
interrupt controller.
A follow up commit will be made to remove pci_get_rid.
Reviewed by: jhb, rstone (previous version)
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6239
interface with 5 methods to mirror the 5 MSI/MSI-X methods in the pcib
interface. The pcib driver will need to perform a device specific lookup
to find the MSI controller and pass this to intrng as the xref. Intrng
will finally find the controller and have it handle the requested operation.
Obtained from: ABT Systems Ltd
MFH: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5985
which leads to end being before start and thus a signed extended very large
number of size later on, which kva_alloc() will fail upon and we will panic.
Add the missing call.
Debugged with: andrew
Reviewed by: br, andrew
Sponsored by: DARPA/AFRL
Found: while using virtio with gem5
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D6337
detect failures, and get different PCI IDs.
For the former the interface returns an int to signal an error. The ID is
returned at a uintptr_t * argument.
For the latter there is a type argument that allows selecting the ID type.
This only specifies a single type, however a MSI type will be added
to handle the need to find the ID the hardware passes to the ARM GICv3
interrupt controller.
A follow up commit will be made to remove pci_get_rid.
Reviewed by: jhb, rstone
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6239
When devctl was added, the location string for PCI devices was changed to
use the PCI "selector" that pciconf and devctl accept. However, devd
assumes that location strings are formatted as a list of name=value pairs.
As a result, devd is no longer parsing any of the values out of PCI
device events. Restore the previous format of the PCI location strings
to restore the location and slot keywords in case any devd scripts are
using this. Add the "selector" as a new 'dbsf' location variable.
Reviewed by: imp
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D6253
PCI-express HotPlug support is implemented via bits in the slot
registers of the PCI-express capability of the downstream port along
with an interrupt that triggers when bits in the slot status register
change.
This is implemented for FreeBSD by adding HotPlug support to the
PCI-PCI bridge driver which attaches to the virtual PCI-PCI bridges
representing downstream ports on HotPlug slots. The PCI-PCI bridge
driver registers an interrupt handler to receive HotPlug events. It
also uses the slot registers to determine the current HotPlug state
and drive an internal HotPlug state machine. For simplicty of
implementation, the PCI-PCI bridge device detaches and deletes the
child PCI device when a card is removed from a slot and creates and
attaches a PCI child device when a card is inserted into the slot.
The PCI-PCI bridge driver provides a bus_child_present which claims
that child devices are present on HotPlug-capable slots only when a
card is inserted. Rather than requiring a timeout in the RC for
config accesses to not-present children, the pcib_read/write_config
methods fail all requests when a card is not present (or not yet
ready).
These changes include support for various optional HotPlug
capabilities such as a power controller, mechanical latch,
electro-mechanical interlock, indicators, and an attention button.
It also includes support for devices which require waiting for
command completion events before initiating a subsequent HotPlug
command. However, it has only been tested on ExpressCard systems
which support surprise removal and have none of these optional
capabilities.
PCI-express HotPlug support is conditional on the PCI_HP option
which is enabled by default on arm64, x86, and powerpc.
Reviewed by: adrian, imp, vangyzen (older versions)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D6136
