device's, not the bridge's, softc to be used to check the
PCIB_DISABLE_MSI flag. This resulted in randomly allowing
or denying MSI interrupts based on whatever value the driver
happened to store at sizeof(device_t) bytes into its softc.
I noticed this when I stopped getting MSI interrupts
after slighly re-arranging mxge's softc yesterday.
- Simplify the amount of work that has be done for each architecture by
pushing more of the truly MI code down into the PCI bus driver.
- Don't bind MSI-X indicies to IRQs so that we can allow a driver to map
multiple MSI-X messages into a single IRQ when handling a message
shortage.
The changes include:
- Add a new pcib_if method: PCIB_MAP_MSI() which is called by the PCI bus
to calculate the address and data values for a given MSI/MSI-X IRQ.
The x86 nexus drivers map this into a call to a new 'msi_map()' function
in msi.c that does the mapping.
- Retire the pcib_if method PCIB_REMAP_MSIX() and remove the 'index'
parameter from PCIB_ALLOC_MSIX(). MD code no longer has any knowledge
of the MSI-X index for a given MSI-X IRQ.
- The PCI bus driver now stores more MSI-X state in a child's ivars.
Specifically, it now stores an array of IRQs (called "message vectors" in
the code) that have associated address and data values, and a small
virtual version of the MSI-X table that specifies the message vector
that a given MSI-X table entry uses. Sparse mappings are permitted in
the virtual table.
- The PCI bus driver now configures the MSI and MSI-X address/data
registers directly via custom bus_setup_intr() and bus_teardown_intr()
methods. pci_setup_intr() invokes PCIB_MAP_MSI() to determine the
address and data values for a given message as needed. The MD code
no longer has to call back down into the PCI bus code to set these
values from the nexus' bus_setup_intr() handler.
- The PCI bus code provides a callout (pci_remap_msi_irq()) that the MD
code can call to force the PCI bus to re-invoke PCIB_MAP_MSI() to get
new values of the address and data fields for a given IRQ. The x86
MSI code uses this when an MSI IRQ is moved to a different CPU, requiring
a new value of the 'address' field.
- The x86 MSI psuedo-driver loses a lot of code, and in fact the separate
MSI/MSI-X pseudo-PICs are collapsed down into a single MSI PIC driver
since the only remaining diff between the two is a substring in a
bootverbose printf.
- The PCI bus driver will now restore MSI-X state (including programming
entries in the MSI-X table) on device resume.
- The interface for pci_remap_msix() has changed. Instead of accepting
indices for the allocated vectors, it accepts a mini-virtual table
(with a new length parameter). This table is an array of u_ints, where
each value specifies which allocated message vector to use for the
corresponding MSI-X message. A vector of 0 forces a message to not
have an associated IRQ. The device may choose to only use some of the
IRQs assigned, in which case the unused IRQs must be at the "end" and
will be released back to the system. This allows a driver to use the
same remap table for different shortage values. For example, if a driver
wants 4 messages, it can use the same remap table (which only uses the
first two messages) for the cases when it only gets 2 or 3 messages and
in the latter case the PCI bus will release the 3rd IRQ back to the
system.
MFC after: 1 month
- First off, device drivers really do need to know if they are allocating
MSI or MSI-X messages. MSI requires allocating powerof2() messages for
example where MSI-X does not. To address this, split out the MSI-X
support from pci_msi_count() and pci_alloc_msi() into new driver-visible
functions pci_msix_count() and pci_alloc_msix(). As a result,
pci_msi_count() now just returns a count of the max supported MSI
messages for the device, and pci_alloc_msi() only tries to allocate MSI
messages. To get a count of the max supported MSI-X messages, use
pci_msix_count(). To allocate MSI-X messages, use pci_alloc_msix().
pci_release_msi() still handles both MSI and MSI-X messages, however.
As a result of this change, drivers using the existing API will only
use MSI messages and will no longer try to use MSI-X messages.
- Because MSI-X allows for each message to have its own data and address
values (and thus does not require all of the messages to have their
MD vectors allocated as a group), some devices allow for "sparse" use
of MSI-X message slots. For example, if a device supports 8 messages
but the OS is only able to allocate 2 messages, the device may make the
best use of 2 IRQs if it enables the messages at slots 1 and 4 rather
than default of using the first N slots (or indicies) at 1 and 2. To
support this, add a new pci_remap_msix() function that a driver may call
after a successful pci_alloc_msix() (but before allocating any of the
SYS_RES_IRQ resources) to allow the allocated IRQ resources to be
assigned to different message indices. For example, from the earlier
example, after pci_alloc_msix() returned a value of 2, the driver would
call pci_remap_msix() passing in array of integers { 1, 4 } as the
new message indices to use. The rid's for the SYS_RES_IRQ resources
will always match the message indices. Thus, after the call to
pci_remap_msix() the driver would be able to access the first message
in slot 1 at SYS_RES_IRQ rid 1, and the second message at slot 4 at
SYS_RES_IRQ rid 4. Note that the message slots/indices are 1-based
rather than 0-based so that they will always correspond to the rid
values (SYS_RES_IRQ rid 0 is reserved for the legacy INTx interrupt).
To support this API, a new PCIB_REMAP_MSIX() method was added to the
pcib interface to change the message index for a single IRQ.
Tested by: scottl
bridge if it doesn't pass MSI messages up correctly. We set the flag
in pcib_attach() if the device ID is disabled via a PCI quirk.
- Disable MSI for devices behind the AMD 8131 HT-PCIX bridge. Linux has
the same quirk.
Tested by: no one despite repeated calls for testers
- Add 3 new functions to the pci_if interface along with suitable wrappers
to provide the device driver visible API:
- pci_alloc_msi(dev, int *count) backed by PCI_ALLOC_MSI(). '*count'
here is an in and out parameter. The driver stores the desired number
of messages in '*count' before calling the function. On success,
'*count' holds the number of messages allocated to the device. Also on
success, the driver can access the messages as SYS_RES_IRQ resources
starting at rid 1. Note that the legacy INTx interrupt resource will
not be available when using MSI. Note that this function will allocate
either MSI or MSI-X messages depending on the devices capabilities and
the 'hw.pci.enable_msix' and 'hw.pci.enable_msi' tunables. Also note
that the driver should activate the memory resource that holds the
MSI-X table and pending bit array (PBA) before calling this function
if the device supports MSI-X.
- pci_release_msi(dev) backed by PCI_RELEASE_MSI(). This function
releases the messages allocated for this device. All of the
SYS_RES_IRQ resources need to be released for this function to succeed.
- pci_msi_count(dev) backed by PCI_MSI_COUNT(). This function returns
the maximum number of MSI or MSI-X messages supported by this device.
MSI-X is preferred if present, but this function will honor the
'hw.pci.enable_msix' and 'hw.pci.enable_msi' tunables. This function
should return the largest value that pci_alloc_msi() can return
(assuming the MD code is able to allocate sufficient backing resources
for all of the messages).
- Add default implementations for these 3 methods to the pci_driver generic
PCI bus driver. (The various other PCI bus drivers such as for ACPI and
OFW will inherit these default implementations.) This default
implementation depends on 4 new pcib_if methods that bubble up through
the PCI bridges to the MD code to allocate IRQ values and perform any
needed MD setup code needed:
- PCIB_ALLOC_MSI() attempts to allocate a group of MSI messages.
- PCIB_RELEASE_MSI() releases a group of MSI messages.
- PCIB_ALLOC_MSIX() attempts to allocate a single MSI-X message.
- PCIB_RELEASE_MSIX() releases a single MSI-X message.
- Add default implementations for these 4 methods that just pass the
request up to the parent bus's parent bridge driver and use the
default implementation in the various MI PCI bridge drivers.
- Add MI functions for use by MD code when managing MSI and MSI-X
interrupts:
- pci_enable_msi(dev, address, data) programs the MSI capability address
and data registers for a group of MSI messages
- pci_enable_msix(dev, index, address, data) initializes a single MSI-X
message in the MSI-X table
- pci_mask_msix(dev, index) masks a single MSI-X message
- pci_unmask_msix(dev, index) unmasks a single MSI-X message
- pci_pending_msix(dev, index) returns true if the specified MSI-X
message is currently pending
- Save the MSI capability address and data registers in the pci_cfgreg
block in a PCI devices ivars and restore the values when a device is
resumed. Note that the MSI-X table is not currently restored during
resume.
- Add constants for MSI-X register offsets and fields.
- Record interesting data about any MSI-X capability blocks we come
across in the pci_cfgreg block in the ivars for PCI devices.
Tested on: em (i386, MSI), bce (amd64/i386, MSI), mpt (amd64, MSI-X)
Reviewed by: scottl, grehan, jfv
MFC after: 2 months
only those bars that had addresses assigned by the BIOS and where the
bridges were properly programmed. Now even unprogrammed ones work.
This was needed for sun4v. We still only implement up to 2GB memory
ranges, even for 64-bit bars. PCI standards at least through 2.2 say
that this is the max (or 1GB is, I only know it is < 32bits).
o Always define pci_addr_t as uint64_t. A pci address is always 64-bits,
but some hosts can't address all of them.
o Preserve the upper half of the 64-bit word during resource probing.
o Test to make sure that 64-bit values can fit in a u_long (true on some
platforms, but not others). Don't use those that can't.
o minor pedantry about data sizes.
o Better bridge resource reporting in bootverbose case.
o Minor formatting changes to cope with different data types on different
platforms.
Submitted by: jmg, with many changes by me to fully support 64-bit
addresses.
various pcib drivers to use their own private devclass_t variables for
their modules.
- Use the DEFINE_CLASS_0() macro to declare drivers for the various pcib
drivers while I'm here.
theoretically unload pci bridges or pci drivers. It will also allow
detach to work if one needed to detach a subtree.
This is inspired by looking at the p4 commits from bms to his 5.4
tree, but I didn't look at the final results.
for the VGA I/O or memory ranges, when it's not within the default
ranges decoded by the bridge. When allocation for VGA addresses is
attempted, check that the bridge has the VGA Enable bit set before
allowing it.
As such, newbusified VGA drivers can allocate their resources when
the VGA adapter is behind a PCI-to-PCI bridge.
Reviewed by: imp@, jhb@
like a valid range. We already do this in the memory case (although
the code there is somewhat different than the I/o case because we have
to deal with different kinds of memory). Since most laptops don't
have non-subtractive bridges, this wasn't seen in practice.
Evidentally the Compaq R3000 hits this problem with PC Cards.
Some minor style fixes while I'm here.
Submitted by: Jung-uk Kim
o Save and restore bars for suspend/resume as well as for D3->D0
transitions.
o preallocate resources that the PCI devices use to avoid resource
conflicts
o lazy allocation of resources not allocated by the BIOS.
o set unattached drivers to state D3. Set power state to D0
before probe/attach. Right now there's two special cases
for this (display and memory devices) that need work in other
areas of the tree.
Please report any bugs to me.
pain and suffering. Attempt to back it out by removing the 'if the
requested range is larger than the window, clip to the window' code.
This is a band-aide until the issues are better understood and the
issues with the lazy allocation patches are resolved.
signals to addresses to the child busses. Typically, ProgIf of 1
means a subtractive bridge. However, Intel has a whole lot of ones
with a ProgIf of 80 that are also subtractive. We cope with these
bridges too. This eliminates hw.pci.allow_unsupported_io_range
because that had almost the same effect as these patches (almost means
'buggy'). Remove the bogus checks for ISA bus locations: these cycles
aren't special and are only passed by transparent bridges.
We allow any range to succeed. If the range is a superset of the
range that's decoded, trim the resource to that range. Otherwise,
pass the range unchanged. This will change the location that PC Card
and CardBus cards are attached. This might bogusly cause some
overlapping allocation that wasn't present before, but the overlapping
fixes need to be in the pci level.
There's also a few formatting changes here.
We allow the request to go through if it matches either a prefetchable
or a non-prefetchable part of the bridge. We do not check to make
sure it is the right kind of memory because most drivers to not yet
properly set RF_PREFETCHABLE (only cardbus seems to do so, and I'm not
entirely sure it does it right). RF_PREFETCHABLE was invented for
cardbus, so hasn't been properly documented yet.
This is still overridable by hw.pci.allow_unsupported_io_ranges, but
the need for that is greatly reduced, especially for the nvida driver.
Approved by: re
Reviewed by: jhb and many testers
Submitted by: Matt Emmerton (although this has been reworked somewhat)
- If a PCI device is not present, then a 32-bit read_config() is going to
return 0xffffffff not 0xffff.
- For the 82454NX chipset, the MIOC that we read the bus numbers of the
various host-PCI bridges from is at function (slot) 0x10 not 0x0.
Approved by: re (rwatson)
OOP speak, you would mark these as 'protected' members. Specifically:
- Make the pcib_softc struct public so it can be used by subclasses.
- Make pcib_{read,write}_ivar(), pcib_alloc_resource(), pcib_maxslots(),
and pcib_{read,write}_config() globals that can be used by subclasses.
- Make the pcib devclass a global variable.
- Move most of the pcib_attach() function into a global
pcib_attach_common() function that can be called by the attach routines
of subclasses.
Tested on: i386, alpha, sparc64, ia64
of I/O in 1.5. It looks like I got it right only for some of the
cases. Instead, allow ISA addresses as a special case. Most PCI
bridges decode this range. I need to investigate PCI bridges better
to know if this is always true or not, but for now assume that it is
since that seems to be the most common case.
# We need to allocate addresses better for the pccard stuff...
Submitted by: phk, mitsunaga-san
request to one that's supported by the bridge. I'm not 100% sure this
is correct, but it makes it easier for the cardbus bridge to allocate
its memory.
Similar code is needed for the I/O range. Also, I'm not sure if I
should be doing this based on memory or pmemory (but likely should do
it based on some flag that tells us to prefetch or not).
Talked about a long time ago with: msmith
with weird PCI-PCI bridge configurations to work. Defining
PCI_ALLOW_UNSUPPORTED_IO_RANGE causes the sanity checks to pass even
with out of range values.
Reviewed by: msmith
- Break out the /dev/pci driver into a separate file.
- Kill the COMPAT_OLDPCI support.
- Make the EISA bridge attach a bit more like the old code; explicitly
check for the existence of eisa0/isa0 and only attach if they don't
already exist. Only make one bus_generic_attach() pass over the
bridge, once both busses are attached. Note that the stupid Intel
bridge's class is entirely unpredictable.
- Add prototypes and re-layout the core PCI modules in line with
current coding standards (not a major whitespace change, just moving
the module data to the top of the file).
- Remove redundant type-2 bridge support from the core PCI code; the
PCI-CardBus code does this itself internally. Remove the now
entirely redundant header-class-specific support, as well as the
secondary and subordinate bus number fields. These are bridge
attributes now.
- Add support for PCI Extended Capabilities.
- Add support for PCI Power Management. The interface currently
allows a driver to query and set the power state of a device.
- Add helper functions to allow drivers to enable/disable busmastering
and the decoding of I/O and memory ranges.
- Use PCI_SLOTMAX and PCI_FUNCMAX rather than magic numbers in some
places.
- Make the PCI-PCI bridge code a little more paranoid about valid
I/O and memory decodes.
- Add some more PCI register definitions for the command and status
registers. Correct another bogus definition for type-1 bridges.
rather than finding our parent pcib and using its PCI_READ_CONFIG
method.
- Fix the defines for the 32-bit I/O decode registers, and properly
process the 16-bit versions. Now we will correctly check that I/O
resources behind the bridge are going to be decoded.
- Bring the quirk for the Orion PCI:PCI bridge in here (since it
seems to want to set the secondary/supplementary bus numbers).
- Use PCI_SLOTMAX rather than a magic number.
- Move PCI core code to dev/pci.
- Split bridge code out into separate modules.
- Remove the descriptive strings from the bridge drivers. If you
want to know what a device is, use pciconf. Add support for
broadly identifying devices based on class/subclass, and for
parsing a preloaded device identification database so that if
you want to waste the memory, you can identify *anything* we know
about.
- Remove machine-dependant code from the core PCI code. APIC interrupt
mapping is performed by shadowing the intline register in machine-
dependant code.
- Bring interrupt routing support to the Alpha
(although many platforms don't yet support routing or mapping
interrupts entirely correctly). This resulted in spamming
<sys/bus.h> into more places than it really should have gone.
- Put sys/dev on the kernel/modules include path. This avoids
having to change *all* the pci*.h includes.
