- First off, device drivers really do need to know if they are allocating
MSI or MSI-X messages. MSI requires allocating powerof2() messages for
example where MSI-X does not. To address this, split out the MSI-X
support from pci_msi_count() and pci_alloc_msi() into new driver-visible
functions pci_msix_count() and pci_alloc_msix(). As a result,
pci_msi_count() now just returns a count of the max supported MSI
messages for the device, and pci_alloc_msi() only tries to allocate MSI
messages. To get a count of the max supported MSI-X messages, use
pci_msix_count(). To allocate MSI-X messages, use pci_alloc_msix().
pci_release_msi() still handles both MSI and MSI-X messages, however.
As a result of this change, drivers using the existing API will only
use MSI messages and will no longer try to use MSI-X messages.
- Because MSI-X allows for each message to have its own data and address
values (and thus does not require all of the messages to have their
MD vectors allocated as a group), some devices allow for "sparse" use
of MSI-X message slots. For example, if a device supports 8 messages
but the OS is only able to allocate 2 messages, the device may make the
best use of 2 IRQs if it enables the messages at slots 1 and 4 rather
than default of using the first N slots (or indicies) at 1 and 2. To
support this, add a new pci_remap_msix() function that a driver may call
after a successful pci_alloc_msix() (but before allocating any of the
SYS_RES_IRQ resources) to allow the allocated IRQ resources to be
assigned to different message indices. For example, from the earlier
example, after pci_alloc_msix() returned a value of 2, the driver would
call pci_remap_msix() passing in array of integers { 1, 4 } as the
new message indices to use. The rid's for the SYS_RES_IRQ resources
will always match the message indices. Thus, after the call to
pci_remap_msix() the driver would be able to access the first message
in slot 1 at SYS_RES_IRQ rid 1, and the second message at slot 4 at
SYS_RES_IRQ rid 4. Note that the message slots/indices are 1-based
rather than 0-based so that they will always correspond to the rid
values (SYS_RES_IRQ rid 0 is reserved for the legacy INTx interrupt).
To support this API, a new PCIB_REMAP_MSIX() method was added to the
pcib interface to change the message index for a single IRQ.
Tested by: scottl
- Add 3 new functions to the pci_if interface along with suitable wrappers
to provide the device driver visible API:
- pci_alloc_msi(dev, int *count) backed by PCI_ALLOC_MSI(). '*count'
here is an in and out parameter. The driver stores the desired number
of messages in '*count' before calling the function. On success,
'*count' holds the number of messages allocated to the device. Also on
success, the driver can access the messages as SYS_RES_IRQ resources
starting at rid 1. Note that the legacy INTx interrupt resource will
not be available when using MSI. Note that this function will allocate
either MSI or MSI-X messages depending on the devices capabilities and
the 'hw.pci.enable_msix' and 'hw.pci.enable_msi' tunables. Also note
that the driver should activate the memory resource that holds the
MSI-X table and pending bit array (PBA) before calling this function
if the device supports MSI-X.
- pci_release_msi(dev) backed by PCI_RELEASE_MSI(). This function
releases the messages allocated for this device. All of the
SYS_RES_IRQ resources need to be released for this function to succeed.
- pci_msi_count(dev) backed by PCI_MSI_COUNT(). This function returns
the maximum number of MSI or MSI-X messages supported by this device.
MSI-X is preferred if present, but this function will honor the
'hw.pci.enable_msix' and 'hw.pci.enable_msi' tunables. This function
should return the largest value that pci_alloc_msi() can return
(assuming the MD code is able to allocate sufficient backing resources
for all of the messages).
- Add default implementations for these 3 methods to the pci_driver generic
PCI bus driver. (The various other PCI bus drivers such as for ACPI and
OFW will inherit these default implementations.) This default
implementation depends on 4 new pcib_if methods that bubble up through
the PCI bridges to the MD code to allocate IRQ values and perform any
needed MD setup code needed:
- PCIB_ALLOC_MSI() attempts to allocate a group of MSI messages.
- PCIB_RELEASE_MSI() releases a group of MSI messages.
- PCIB_ALLOC_MSIX() attempts to allocate a single MSI-X message.
- PCIB_RELEASE_MSIX() releases a single MSI-X message.
- Add default implementations for these 4 methods that just pass the
request up to the parent bus's parent bridge driver and use the
default implementation in the various MI PCI bridge drivers.
- Add MI functions for use by MD code when managing MSI and MSI-X
interrupts:
- pci_enable_msi(dev, address, data) programs the MSI capability address
and data registers for a group of MSI messages
- pci_enable_msix(dev, index, address, data) initializes a single MSI-X
message in the MSI-X table
- pci_mask_msix(dev, index) masks a single MSI-X message
- pci_unmask_msix(dev, index) unmasks a single MSI-X message
- pci_pending_msix(dev, index) returns true if the specified MSI-X
message is currently pending
- Save the MSI capability address and data registers in the pci_cfgreg
block in a PCI devices ivars and restore the values when a device is
resumed. Note that the MSI-X table is not currently restored during
resume.
- Add constants for MSI-X register offsets and fields.
- Record interesting data about any MSI-X capability blocks we come
across in the pci_cfgreg block in the ivars for PCI devices.
Tested on: em (i386, MSI), bce (amd64/i386, MSI), mpt (amd64, MSI-X)
Reviewed by: scottl, grehan, jfv
MFC after: 2 months
share devclass pointers, a mistake I've encouraged in the past) and
move the declaration of the pci_driver kobj class from cardbus.c to
pci_private.h so that other drivers can inherit from pci_driver.
force allocation of unallocated BARs (cardbus uses this to preallocate
everything). Add a prefetchmask to allow for busses that get prefetch
hints to set them. Addjust pci_add_map and pci_ata_maps to take a new
force flag which pci_add_resources will pass in. Implement 'force' in
pci_add_map. Write new value of allocated resource into the bar, if
the allocation succeeded (we should have done this before, but with
the new force the bug was very obvious).
to search for a specific extended capability. If the specified capability
is found for the given device, then the function returns success and
optionally returns the offset of that capability. If the capability is
not found, the function returns an error.
o Save and restore bars for suspend/resume as well as for D3->D0
transitions.
o preallocate resources that the PCI devices use to avoid resource
conflicts
o lazy allocation of resources not allocated by the BIOS.
o set unattached drivers to state D3. Set power state to D0
before probe/attach. Right now there's two special cases
for this (display and memory devices) that need work in other
areas of the tree.
Please report any bugs to me.
interrupt to be used for a device. This is intended solely for internal
use of PCI bus implementations, and exists so that PCI bus drivers
implementing special interrupt assignment methods which require
additional work at the bus level to work right can be easily derived
from the generic driver (or any other one) without resorting to hacks.
It will be used in the sparc64 ofw_pcibus driver, which will be
committed shortly.
Make use of this method in the generic implementation, and add it to
the method table of bus drivers derived from the PCI one.
Reviewed by: imp, -hackers
pci busses implement this.
Also minor comment smithing in cardbus. Fix copyright to this year
with my name on it since I've been doing a lot to this file.
Reviewed by: jhb
when the first PCI bus attaches.
- Create /dev/pci during MOD_LOAD as well.
- Destroy /dev/pci during MOD_UNLOAD (not that you can kldunload pci, but
might as well get the code right)
- Make the pci devclass a global variable.
- Add child devices in pci_attach() instead of pci_probe(). Change
pci_probe() to just check for a valid bus number from the associated
bridge and return -1000 if successful. This allows subclasses of the
PCI bus driver to override the generic driver.
- Move the code to load the vendor data into its own public function.
Really though, doing this at attach is just plain wrong. This should
really be done in the module load routine instead. As a side effect,
the 'busno' variable in pci_attach() is now no longer static (minor
bug that was harmless so far.)
- Change pci_add_children() to take an extra argument that is the size of
the device info structure passed to pci_read_device() and make it public
so subclasses of the PCI bus can call it in their attach routines.
- Move the bits to attach a probed PCI child to a PCI bus into a global
pci_add_child() function. This will allow subclasses that can detect
a PCI device not found in the normal PCI probe to add those devices in
their own attach routine. (I have seen this in the ACPI tree on my
laptop for example.) As a side effect, change the static function
pci_add_resources() to get the busno, slot, and func from the passed
in dinfo structure instead of requiring them as function arguments.
Tested on: i386, alpha, ia64, sparc64
code into cardbus and s/pci/cardbus. This exposes a few pci_*
functions that are now static.
This work is similar to work Justin posted to the mobile list about a
year or two ago, which I have neglected since then.
This is a subset of his current work with the multiple inheritance
newbus architecutre. When completed, that will eliminate the need for
pci/pci_private.h.
Similar work is needed for the cardbus_cis and pccard_cis code as well.
