d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

bhyve: passthru: enable BARs before possibly mmap(2)ing them

Browse Source 
The first time we start bhyve with a passthru device everything is fine
as on boot we do enable BARs.  If a driver (unload) inside bhyve disables
the BAR(s) as some Linux drivers do, we need to make sure we re-enable
them on next bhyve start.

If we are trying to mmap a disabled BAR for MSI-X (PCIOCBARMMAP)
the kernel will give us an EBUSY.
While we were re-enabling the BAR(s) in the current code loop
cfginit() was writing the changes out too late to the real hardware.

Move the call to init_msix_table() after the register on the real
hardware was updated.  That way the kernel will be happy and the
mmap will succeed and bhyve will start.
Also simplify the code given the last argument to init_msix_table()
is unused we do not need to do checks for each bar. [1]

MFC after:	3 days
PR:		260148
Pointed out by:	markj [1]
Sponsored by:	The FreeBSD Foundation
Reviewed by:	markj
Differential Revision: https://reviews.freebsd.org/D33628
This commit is contained in:
Bjoern A. Zeeb 2021-12-23 14:59:49 +00:00
parent b4a58b3d58
commit f1442847c9
1 changed files with 12 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
usr.sbin/bhyve/pci_passthru.c
View File

@ -420,7 +420,7 @@ msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_softc *sc,
}
static int
init_msix_table(struct vmctx *ctx, struct passthru_softc *sc, uint64_t base)
init_msix_table(struct vmctx *ctx, struct passthru_softc *sc)
{
struct pci_devinst *pi = sc->psc_pi;
struct pci_bar_mmap pbm;
@ -548,13 +548,6 @@ cfginitbar(struct vmctx *ctx, struct passthru_softc *sc)
sc->psc_bar[i].lobits = lobits;
pi->pi_bar[i].lobits = lobits;
/* The MSI-X table needs special handling */
if (i == pci_msix_table_bar(pi)) {
error = init_msix_table(ctx, sc, base);
if (error)
return (-1);
}
/*
* 64-bit BAR takes up two slots so skip the next one.
*/
@ -596,7 +589,17 @@ cfginit(struct vmctx *ctx, struct pci_devinst *pi, int bus, int slot, int func)
write_config(&sc->psc_sel, PCIR_COMMAND, 2,
pci_get_cfgdata16(pi, PCIR_COMMAND));
error = 0; /* success */
/*
* We need to do this after PCIR_COMMAND got possibly updated, e.g.,
* a BAR was enabled, as otherwise the PCIOCBARMMAP might fail on us.
*/
error = init_msix_table(ctx, sc);
if (error != 0) {
warnx("failed to initialize MSI-X table for PCI %d/%d/%d: %d",
bus, slot, func, error);
goto done;
}
done:
return (error);
}