d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
025730aad6
freebsd-skq/sys/dev/cxgbe/t4_iov.c

373 lines
9.3 KiB
C
Raw Normal View History

Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
/*-
* Copyright (c) 2015-2016 Chelsio Communications, Inc.
* All rights reserved.
* Written by: John Baldwin <jhb@FreeBSD.org>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h>
#include <sys/bus.h>
#include <sys/kernel.h>
#include <sys/module.h>
#include <sys/systm.h>
#include <dev/pci/pcivar.h>
#ifdef PCI_IOV
#include <sys/nv.h>
#include <sys/iov_schema.h>
#include <dev/pci/pci_iov.h>
#endif
Fix a couple of issues with t4iov probe and attach. - Check for Chelsio vendor ID in probe routines. - Fail attach instead of faulting if pci_find_dbsf() doesn't find a device. PR: 216539 Reported by: asomers Tested by: Dave Baukus <daveb@spectralogic.com> MFC after: 3 days Sponsored by: Chelsio Communications
2017-01-31 18:54:13 +00:00
#include "common/common.h"
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
#include "t4_if.h"
struct t4iov_softc {
device_t sc_dev;
device_t sc_main;
bool sc_attached;
};
struct {
uint16_t device;
char *desc;
} t4iov_pciids[] = {
{0x4000, "Chelsio T440-dbg"},
{0x4001, "Chelsio T420-CR"},
{0x4002, "Chelsio T422-CR"},
{0x4003, "Chelsio T440-CR"},
{0x4004, "Chelsio T420-BCH"},
{0x4005, "Chelsio T440-BCH"},
{0x4006, "Chelsio T440-CH"},
{0x4007, "Chelsio T420-SO"},
{0x4008, "Chelsio T420-CX"},
{0x4009, "Chelsio T420-BT"},
{0x400a, "Chelsio T404-BT"},
{0x400e, "Chelsio T440-LP-CR"},
}, t5iov_pciids[] = {
{0x5000, "Chelsio T580-dbg"},
{0x5001, "Chelsio T520-CR"}, /* 2 x 10G */
{0x5002, "Chelsio T522-CR"}, /* 2 x 10G, 2 X 1G */
{0x5003, "Chelsio T540-CR"}, /* 4 x 10G */
{0x5007, "Chelsio T520-SO"}, /* 2 x 10G, nomem */
{0x5009, "Chelsio T520-BT"}, /* 2 x 10GBaseT */
{0x500a, "Chelsio T504-BT"}, /* 4 x 1G */
{0x500d, "Chelsio T580-CR"}, /* 2 x 40G */
{0x500e, "Chelsio T540-LP-CR"}, /* 4 x 10G */
{0x5010, "Chelsio T580-LP-CR"}, /* 2 x 40G */
{0x5011, "Chelsio T520-LL-CR"}, /* 2 x 10G */
{0x5012, "Chelsio T560-CR"}, /* 1 x 40G, 2 x 10G */
{0x5014, "Chelsio T580-LP-SO-CR"}, /* 2 x 40G, nomem */
{0x5015, "Chelsio T502-BT"}, /* 2 x 1G */
cxgbev(4): Catch up with the pciids in the PF driver. MFC after: 3 days Sponsored by: Chelsio Communications
2019-11-15 18:48:14 +00:00
{0x5018, "Chelsio T540-BT"}, /* 4 x 10GBaseT */
{0x5019, "Chelsio T540-LP-BT"}, /* 4 x 10GBaseT */
{0x501a, "Chelsio T540-SO-BT"}, /* 4 x 10GBaseT, nomem */
{0x501b, "Chelsio T540-SO-CR"}, /* 4 x 10G, nomem */
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
}, t6iov_pciids[] = {
cxgbe(4): Update the list of PCIe devices claimed by the driver. At this point any board with a T6 should just work. Obtained from: Chelsio Communications MFC after: 1 week Sponsored by: Chelsio Communications
2017-05-05 00:54:23 +00:00
{0x6000, "Chelsio T6-DBG-25"}, /* 2 x 10/25G, debug */
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
{0x6001, "Chelsio T6225-CR"}, /* 2 x 10/25G */
{0x6002, "Chelsio T6225-SO-CR"}, /* 2 x 10/25G, nomem */
cxgbe(4): Update the list of PCIe devices claimed by the driver. At this point any board with a T6 should just work. Obtained from: Chelsio Communications MFC after: 1 week Sponsored by: Chelsio Communications
2017-05-05 00:54:23 +00:00
{0x6003, "Chelsio T6425-CR"}, /* 4 x 10/25G */
cxgbe(4): Update the VF device ids too. This should have been part of r317820. Reported by: jhb@ MFC after: 1 week Sponsored by: Chelsio Communications
2017-05-05 16:52:25 +00:00
{0x6004, "Chelsio T6425-SO-CR"}, /* 4 x 10/25G, nomem */
cxgbe(4): Update the list of PCIe devices claimed by the driver. At this point any board with a T6 should just work. Obtained from: Chelsio Communications MFC after: 1 week Sponsored by: Chelsio Communications
2017-05-05 00:54:23 +00:00
{0x6005, "Chelsio T6225-OCP-SO"}, /* 2 x 10/25G, nomem */
{0x6006, "Chelsio T62100-OCP-SO"}, /* 2 x 40/50/100G, nomem */
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
{0x6007, "Chelsio T62100-LP-CR"}, /* 2 x 40/50/100G */
{0x6008, "Chelsio T62100-SO-CR"}, /* 2 x 40/50/100G, nomem */
cxgbe(4): Update the list of PCIe devices claimed by the driver. At this point any board with a T6 should just work. Obtained from: Chelsio Communications MFC after: 1 week Sponsored by: Chelsio Communications
2017-05-05 00:54:23 +00:00
{0x6009, "Chelsio T6210-BT"}, /* 2 x 10GBASE-T */
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
{0x600d, "Chelsio T62100-CR"}, /* 2 x 40/50/100G */
cxgbe(4): Update the list of PCIe devices claimed by the driver. At this point any board with a T6 should just work. Obtained from: Chelsio Communications MFC after: 1 week Sponsored by: Chelsio Communications
2017-05-05 00:54:23 +00:00
{0x6010, "Chelsio T6-DBG-100"}, /* 2 x 40/50/100G, debug */
{0x6011, "Chelsio T6225-LL-CR"}, /* 2 x 10/25G */
{0x6014, "Chelsio T61100-OCP-SO"}, /* 1 x 40/50/100G, nomem */
{0x6015, "Chelsio T6201-BT"}, /* 2 x 1000BASE-T */
/* Custom */
{0x6080, "Chelsio T6225 80"},
{0x6081, "Chelsio T62100 81"},
cxgbev(4): Catch up with the pciids in the PF driver. MFC after: 3 days Sponsored by: Chelsio Communications
2019-11-15 18:48:14 +00:00
{0x6082, "Chelsio T6225-CR 82"},
{0x6083, "Chelsio T62100-CR 83"},
{0x6084, "Chelsio T64100-CR 84"},
{0x6085, "Chelsio T6240-SO 85"},
{0x6086, "Chelsio T6225-SO-CR 86"},
{0x6087, "Chelsio T6225-CR 87"},
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
};
static int t4iov_attach_child(device_t dev);
static int
t4iov_probe(device_t dev)
{
Whitespace nits.
2016-09-15 22:31:49 +00:00
uint16_t d;
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
size_t i;
Fix a couple of issues with t4iov probe and attach. - Check for Chelsio vendor ID in probe routines. - Fail attach instead of faulting if pci_find_dbsf() doesn't find a device. PR: 216539 Reported by: asomers Tested by: Dave Baukus <daveb@spectralogic.com> MFC after: 3 days Sponsored by: Chelsio Communications
2017-01-31 18:54:13 +00:00
if (pci_get_vendor(dev) != PCI_VENDOR_ID_CHELSIO)
return (ENXIO);
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
d = pci_get_device(dev);
for (i = 0; i < nitems(t4iov_pciids); i++) {
if (d == t4iov_pciids[i].device) {
device_set_desc(dev, t4iov_pciids[i].desc);
Use the port device name for the iov device for Chelsio T4/T5 cards. Chelsio T4/T5 adapters are multifunction cards. The main driver uses physical function 4 (PF4). However, VF devices for SR-IOV are only supported on physical functions 0 through 3, where PF0 creates VFs tied to port 0, etc. The t4iov/t5iov driver was previously added to create VF devices for ports that are present on each adapter. This change uses the recently added pci_iov_attach_name() function to name the character device in /dev/iov after the associated port on the card (e.g. /dev/iov/cxl0 is used to create VFs that share the cxl0 port). With this in place, mark the t4iov/t5iov devices quiet to prevent them from cluttering dmesg. Reviewed by: rstone Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D7402
2016-08-03 17:11:08 +00:00
device_quiet(dev);
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
return (BUS_PROBE_DEFAULT);
}
}
return (ENXIO);
}
static int
t5iov_probe(device_t dev)
{
Whitespace nits.
2016-09-15 22:31:49 +00:00
uint16_t d;
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
size_t i;
Fix a couple of issues with t4iov probe and attach. - Check for Chelsio vendor ID in probe routines. - Fail attach instead of faulting if pci_find_dbsf() doesn't find a device. PR: 216539 Reported by: asomers Tested by: Dave Baukus <daveb@spectralogic.com> MFC after: 3 days Sponsored by: Chelsio Communications
2017-01-31 18:54:13 +00:00
if (pci_get_vendor(dev) != PCI_VENDOR_ID_CHELSIO)
return (ENXIO);
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
d = pci_get_device(dev);
for (i = 0; i < nitems(t5iov_pciids); i++) {
if (d == t5iov_pciids[i].device) {
device_set_desc(dev, t5iov_pciids[i].desc);
Use the port device name for the iov device for Chelsio T4/T5 cards. Chelsio T4/T5 adapters are multifunction cards. The main driver uses physical function 4 (PF4). However, VF devices for SR-IOV are only supported on physical functions 0 through 3, where PF0 creates VFs tied to port 0, etc. The t4iov/t5iov driver was previously added to create VF devices for ports that are present on each adapter. This change uses the recently added pci_iov_attach_name() function to name the character device in /dev/iov after the associated port on the card (e.g. /dev/iov/cxl0 is used to create VFs that share the cxl0 port). With this in place, mark the t4iov/t5iov devices quiet to prevent them from cluttering dmesg. Reviewed by: rstone Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D7402
2016-08-03 17:11:08 +00:00
device_quiet(dev);
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
return (BUS_PROBE_DEFAULT);
}
}
return (ENXIO);
}
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
static int
t6iov_probe(device_t dev)
{
uint16_t d;
size_t i;
Fix a couple of issues with t4iov probe and attach. - Check for Chelsio vendor ID in probe routines. - Fail attach instead of faulting if pci_find_dbsf() doesn't find a device. PR: 216539 Reported by: asomers Tested by: Dave Baukus <daveb@spectralogic.com> MFC after: 3 days Sponsored by: Chelsio Communications
2017-01-31 18:54:13 +00:00
if (pci_get_vendor(dev) != PCI_VENDOR_ID_CHELSIO)
return (ENXIO);
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
d = pci_get_device(dev);
for (i = 0; i < nitems(t6iov_pciids); i++) {
if (d == t6iov_pciids[i].device) {
device_set_desc(dev, t6iov_pciids[i].desc);
device_quiet(dev);
return (BUS_PROBE_DEFAULT);
}
}
return (ENXIO);
}
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
static int
t4iov_attach(device_t dev)
{
struct t4iov_softc *sc;
sc = device_get_softc(dev);
sc->sc_dev = dev;
sc->sc_main = pci_find_dbsf(pci_get_domain(dev), pci_get_bus(dev),
pci_get_slot(dev), 4);
Fix a couple of issues with t4iov probe and attach. - Check for Chelsio vendor ID in probe routines. - Fail attach instead of faulting if pci_find_dbsf() doesn't find a device. PR: 216539 Reported by: asomers Tested by: Dave Baukus <daveb@spectralogic.com> MFC after: 3 days Sponsored by: Chelsio Communications
2017-01-31 18:54:13 +00:00
if (sc->sc_main == NULL)
return (ENXIO);
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
if (T4_IS_MAIN_READY(sc->sc_main) == 0)
return (t4iov_attach_child(dev));
return (0);
}
static int
t4iov_attach_child(device_t dev)
{
struct t4iov_softc *sc;
#ifdef PCI_IOV
nvlist_t *pf_schema, *vf_schema;
#endif
Use the port device name for the iov device for Chelsio T4/T5 cards. Chelsio T4/T5 adapters are multifunction cards. The main driver uses physical function 4 (PF4). However, VF devices for SR-IOV are only supported on physical functions 0 through 3, where PF0 creates VFs tied to port 0, etc. The t4iov/t5iov driver was previously added to create VF devices for ports that are present on each adapter. This change uses the recently added pci_iov_attach_name() function to name the character device in /dev/iov after the associated port on the card (e.g. /dev/iov/cxl0 is used to create VFs that share the cxl0 port). With this in place, mark the t4iov/t5iov devices quiet to prevent them from cluttering dmesg. Reviewed by: rstone Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D7402
2016-08-03 17:11:08 +00:00
device_t pdev;
int error;
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
sc = device_get_softc(dev);
MPASS(!sc->sc_attached);
/*
* PF0-3 are associated with a specific port on the NIC (PF0
Use the port device name for the iov device for Chelsio T4/T5 cards. Chelsio T4/T5 adapters are multifunction cards. The main driver uses physical function 4 (PF4). However, VF devices for SR-IOV are only supported on physical functions 0 through 3, where PF0 creates VFs tied to port 0, etc. The t4iov/t5iov driver was previously added to create VF devices for ports that are present on each adapter. This change uses the recently added pci_iov_attach_name() function to name the character device in /dev/iov after the associated port on the card (e.g. /dev/iov/cxl0 is used to create VFs that share the cxl0 port). With this in place, mark the t4iov/t5iov devices quiet to prevent them from cluttering dmesg. Reviewed by: rstone Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D7402
2016-08-03 17:11:08 +00:00
* with port 0, etc.). Ask the PF4 driver for the device for
* this function's associated port to determine if the port is
* present.
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
*/
Use the port device name for the iov device for Chelsio T4/T5 cards. Chelsio T4/T5 adapters are multifunction cards. The main driver uses physical function 4 (PF4). However, VF devices for SR-IOV are only supported on physical functions 0 through 3, where PF0 creates VFs tied to port 0, etc. The t4iov/t5iov driver was previously added to create VF devices for ports that are present on each adapter. This change uses the recently added pci_iov_attach_name() function to name the character device in /dev/iov after the associated port on the card (e.g. /dev/iov/cxl0 is used to create VFs that share the cxl0 port). With this in place, mark the t4iov/t5iov devices quiet to prevent them from cluttering dmesg. Reviewed by: rstone Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D7402
2016-08-03 17:11:08 +00:00
error = T4_READ_PORT_DEVICE(sc->sc_main, pci_get_function(dev), &pdev);
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
if (error)
return (0);
#ifdef PCI_IOV
pf_schema = pci_iov_schema_alloc_node();
vf_schema = pci_iov_schema_alloc_node();
Use the port device name for the iov device for Chelsio T4/T5 cards. Chelsio T4/T5 adapters are multifunction cards. The main driver uses physical function 4 (PF4). However, VF devices for SR-IOV are only supported on physical functions 0 through 3, where PF0 creates VFs tied to port 0, etc. The t4iov/t5iov driver was previously added to create VF devices for ports that are present on each adapter. This change uses the recently added pci_iov_attach_name() function to name the character device in /dev/iov after the associated port on the card (e.g. /dev/iov/cxl0 is used to create VFs that share the cxl0 port). With this in place, mark the t4iov/t5iov devices quiet to prevent them from cluttering dmesg. Reviewed by: rstone Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D7402
2016-08-03 17:11:08 +00:00
error = pci_iov_attach_name(dev, pf_schema, vf_schema, "%s",
device_get_nameunit(pdev));
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
if (error) {
device_printf(dev, "Failed to initialize SR-IOV: %d\n", error);
return (0);
}
#endif
sc->sc_attached = true;
return (0);
}
static int
t4iov_detach_child(device_t dev)
{
struct t4iov_softc *sc;
#ifdef PCI_IOV
int error;
#endif
sc = device_get_softc(dev);
if (!sc->sc_attached)
return (0);
#ifdef PCI_IOV
error = pci_iov_detach(dev);
if (error != 0) {
device_printf(dev, "Failed to disable SR-IOV\n");
return (error);
}
#endif
sc->sc_attached = false;
return (0);
}
static int
t4iov_detach(device_t dev)
{
struct t4iov_softc *sc;
Use device_verbose() to undo device_quiet() when detaching from t[45]iovX. The device quiet flag is not automatically reset on detach, so it is inherited by other device drivers (e.g. when switching a device driver over to ppt for PCI pass through). Cope with this behavior by explicitly marking the device verbose during detach so that the next driver can make its own decision. Sponsored by: Chelsio Communications
2016-08-29 22:47:14 +00:00
int error;
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
sc = device_get_softc(dev);
Use device_verbose() to undo device_quiet() when detaching from t[45]iovX. The device quiet flag is not automatically reset on detach, so it is inherited by other device drivers (e.g. when switching a device driver over to ppt for PCI pass through). Cope with this behavior by explicitly marking the device verbose during detach so that the next driver can make its own decision. Sponsored by: Chelsio Communications
2016-08-29 22:47:14 +00:00
if (sc->sc_attached) {
error = t4iov_detach_child(dev);
if (error)
return (error);
}
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
return (0);
}
#ifdef PCI_IOV
static int
t4iov_iov_init(device_t dev, uint16_t num_vfs, const struct nvlist *config)
{
/* XXX: The Linux driver sets up a vf_monitor task on T4 adapters. */
return (0);
}
static void
t4iov_iov_uninit(device_t dev)
{
}
static int
t4iov_add_vf(device_t dev, uint16_t vfnum, const struct nvlist *config)
{
return (0);
}
#endif
static device_method_t t4iov_methods[] = {
DEVMETHOD(device_probe, t4iov_probe),
DEVMETHOD(device_attach, t4iov_attach),
DEVMETHOD(device_detach, t4iov_detach),
#ifdef PCI_IOV
DEVMETHOD(pci_iov_init, t4iov_iov_init),
DEVMETHOD(pci_iov_uninit, t4iov_iov_uninit),
DEVMETHOD(pci_iov_add_vf, t4iov_add_vf),
#endif
DEVMETHOD(t4_attach_child, t4iov_attach_child),
DEVMETHOD(t4_detach_child, t4iov_detach_child),
DEVMETHOD_END
};
static driver_t t4iov_driver = {
"t4iov",
t4iov_methods,
sizeof(struct t4iov_softc)
};
static device_method_t t5iov_methods[] = {
DEVMETHOD(device_probe, t5iov_probe),
DEVMETHOD(device_attach, t4iov_attach),
DEVMETHOD(device_detach, t4iov_detach),
#ifdef PCI_IOV
DEVMETHOD(pci_iov_init, t4iov_iov_init),
DEVMETHOD(pci_iov_uninit, t4iov_iov_uninit),
DEVMETHOD(pci_iov_add_vf, t4iov_add_vf),
#endif
DEVMETHOD(t4_attach_child, t4iov_attach_child),
DEVMETHOD(t4_detach_child, t4iov_detach_child),
DEVMETHOD_END
};
static driver_t t5iov_driver = {
"t5iov",
t5iov_methods,
sizeof(struct t4iov_softc)
};
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
static device_method_t t6iov_methods[] = {
DEVMETHOD(device_probe, t6iov_probe),
DEVMETHOD(device_attach, t4iov_attach),
DEVMETHOD(device_detach, t4iov_detach),
#ifdef PCI_IOV
DEVMETHOD(pci_iov_init, t4iov_iov_init),
DEVMETHOD(pci_iov_uninit, t4iov_iov_uninit),
DEVMETHOD(pci_iov_add_vf, t4iov_add_vf),
#endif
DEVMETHOD(t4_attach_child, t4iov_attach_child),
DEVMETHOD(t4_detach_child, t4iov_detach_child),
DEVMETHOD_END
};
static driver_t t6iov_driver = {
"t6iov",
t6iov_methods,
sizeof(struct t4iov_softc)
};
static devclass_t t4iov_devclass, t5iov_devclass, t6iov_devclass;
Add a driver to create VF devices on Chelsio T4/T5 NICs. Chelsio NICs are a bit unique compared to some other NICs in that they expose different functionality on different physical functions. In particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex'). However, PF4 is not able to create VF devices. Instead, VFs are only supported by physical functions 0 through 3. This commit adds 't4iov' and 't5iov' drivers that attach to PF0-3. One extra wrinkle is that the iov devices cannot enable SR-IOV until the firwmare has been initialized by the main PF4 driver. To handle this case, a new t4_if kobj interface has been added to permit cross-calls between the PF drivers. The PF4 driver notifies sibling drivers when it is fully attached. It also requests sibling drivers to detach before it detaches. Sibling drivers query the PF4 driver during their attach routine to see if it is attached. If not, the sibling drivers defer their attach actions until the PF4 driver informs them it is attached. VF devices are associated with a single port on the NIC. VF devices created from PF0 are associated with the first port on the NIC, VFs from PF1 are associated with the second port, etc. VF devices can only be created from a PF device that has an associated port. Thus, on a 2-port card, VFs are only supported on PF0 and PF1. Reviewed by: np (earlier versions) MFC after: 1 month Sponsored by: Chelsio Communications
2016-07-22 22:46:41 +00:00
DRIVER_MODULE(t4iov, pci, t4iov_driver, t4iov_devclass, 0, 0);
MODULE_VERSION(t4iov, 1);
DRIVER_MODULE(t5iov, pci, t5iov_driver, t5iov_devclass, 0, 0);
MODULE_VERSION(t5iov, 1);
cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will come up as 't6nex' nexus devices with 'cc' ports hanging off them. The T6 firmware and configuration files will be added as soon as they are released. For now the driver will try to work with whatever firmware and configuration is on the card's flash. Sponsored by: Chelsio Communications
2016-09-16 00:08:37 +00:00
DRIVER_MODULE(t6iov, pci, t6iov_driver, t6iov_devclass, 0, 0);
MODULE_VERSION(t6iov, 1);
Reference in New Issue Copy Permalink