If an emulation uses those functions instead of set_config_value_node
or set_config_value, it allows the config values to get
overwritten. Introducing new functions is much more readable than
if else statements in the emulation code.
Reviewed by: khng
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33770
The starting address passed to mprotect was wrong, so in the case where
the last page containing the table is not the last page of the BAR, the
wrong region would be unmapped.
Reported by: Andy Fiddaman <andy@omniosce.org>
Reviewed by: jhb
Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for passthrough")
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33739
The PBA and MSI-X table can reside in different BARs.
Reported by: Andy Fiddaman <andy@omniosce.org>
Reviewed by: jhb
Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for passthrough")
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33739
Bhyve allocates small 64 bit BARs below 4 GB and generates ACPI tables
based on this allocation. If the guest decides to relocate those BARs
above 4 GB, it could lead to mismatching ACPI tables. Especially
when using OVMF with enabled bus enumeration it could cause
issues. OVMF relocates all 64 bit BARs above 4 GB. The guest OS
may be unable to recover from this situation and disables some PCI
devices because their BARs are located outside of the MMIO space
reported by ACPI. Avoid this situation by giving the guest more
space for relocating BARs.
Let's be paranoid. The available space for BARs below 4 GB is 512 MB
large. Use a slop of 512 MB. It'll allow the guest to relocate all
BARs below 4 GB to an address above 4 GB. We could run into issues
when we exceeding the memlimit above 4 GB. However, this space has
a size of 32 GB. Even when using many PCI device with large BARs
like framebuffer or when using multiple PCI busses, it's very
unlikely that we run out of space due to the large slop.
Additionally, this situation will occur on startup and not at runtime
which is much better.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33118
At the moment, you only have one single chance to read the fwctl
signature. At boot bhyve is in the state IDENT_WAIT. It's then
possible to switch to IDENT_SEND. After bhyve sends the signature,
it switches to REQ. From now on it's impossible to switch back to
IDENT_SEND to read the signature. For that reason, only a single
driver can read the signature. A guest can't use two drivers to
identify that fwctl is present. It gets even worse when using
OVMF. OVMF uses a library to access fwctl. Therefore, every single
OVMF driver would try to read the signature. Currently, only a
single OVMF driver accesses the fwctl. So, there's no issue with
it yet. However, no OS driver would have a chance to detect fwctl when
using OVMF because it's signature was already consumed by OVMF.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D31981
E.g. Framebuffers can require large space and BARs need to be aligned
by their size. If BARs aren't allocated by size, it'll cause much
fragmentation of the MMIO space. Reduce fragmentation by ordering
the BAR allocation on their size to reduce the risk of
OUT_OF_MMIO_SPACE issues.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D28278
Some passthru devices only support MSI instead of MSI-X. For those
devices the initialization of MSI-X table will fail. Re-add the
check erroneously removed in f1442847c9404d4bc5f5524a0c3362dd39cb14f9.
MFC after: 3 days
X-MFC with: f1442847c9404d4bc5f5524a0c3362dd39cb14f9
PR: 260148
Reviewed by: manu, bz
Differential Revision: https://reviews.freebsd.org/D33728
If you look at the SMBIOS specification, we'll find something is
missing. In particular at offset 0Dh is supposed to be the OEM-defined
field. This should go between security and height. It is not legal to
actually skip this and will lead to other folks not properly
interpreting later parts of the table.
https://www.illumos.org/issues/14312
Reviewed by: jhb
Submitted by: Robert Mustacchi <rm@fingolfin.org>
Obtained from: ilumos
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33682
The first time we start bhyve with a passthru device everything is fine
as on boot we do enable BARs. If a driver (unload) inside bhyve disables
the BAR(s) as some Linux drivers do, we need to make sure we re-enable
them on next bhyve start.
If we are trying to mmap a disabled BAR for MSI-X (PCIOCBARMMAP)
the kernel will give us an EBUSY.
While we were re-enabling the BAR(s) in the current code loop
cfginit() was writing the changes out too late to the real hardware.
Move the call to init_msix_table() after the register on the real
hardware was updated. That way the kernel will be happy and the
mmap will succeed and bhyve will start.
Also simplify the code given the last argument to init_msix_table()
is unused we do not need to do checks for each bar. [1]
MFC after: 3 days
PR: 260148
Pointed out by: markj [1]
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D33628
Register a "block resize" callback to be notified of changes to the
backing storage for the Namespace. Use this to generate an Asynchronous
Event Notification, Namespace Attributes Changed when the guest OS
provides an Asynchronous Event Request.
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D32953
When an EVFILT_VNODE filter event is triggered, reset it.
This fixes the issue where a virtio-blk resize event would cause the
mevent thread to consume 100% of the cpu.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D33326
OVMF creates two separate .fd files, a _CODE.fd file containing
the UEFI code, and a _VARS.fd file containing a template of an
empty UEFI variable store.
OVMF decides to write variables to the memory range just below the
boot rom code if it detects a CFI flash device. So here we add
just the barest facsimile of CFI command handling to bootrom.c
that is needed to placate OVMF.
Submitted by: D Scott Phillips <d.scott.phillips@intel.com>
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D19976
MFC After: 1 week
NVMe conformance tests for the Format command failed if the
backing-storage for the bhyve device was a file instead of a Zvol. The
tests (and the specification) expect a Format to destroy all previously
written data. The bhyve NVMe emulation implements this by trimming /
deallocating all data from the backing-storage.
The blockif_candelete() function indicated the file did not support
deallocation (i.e. fpathconf(..., _PC_DEALLOC_PRESENT) returned FALSE)
even though the kernel supported file hole punching. This occurs on
builds with Capsicum enabled because blockif did not allow the
fpathconf(2) right.
Fix is to add CAP_FPATHCONF to the cap_rights_init(3) call.
PR: 260081
Reviewed by: allanjude, markj, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33203
Reads of the MSI-X capabilites aren't emulated by passthru devices
yet. The guest will read the host MSI-X capabilites which could
cause issues.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D32686
Sponsored by: Beckhoff Automation GmbH & Co. KG
On startup all virtual BARs are registered.
Additionally, the encoding bit in the virtual cmd register is set.
After that, the passthru emulation overwrites the virtual cmd register with
the physical one.
This could lead to a mismatch between registered BARs and the encoding
bits in the cmd register.
Instead of writing the physical to the virtual cmd register,
write the virtual to the physical cmd register to solve this issue.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D32687
Sponsored by: Beckhoff Automation GmbH & Co. KG
OVMF always uses 0xC0000000 as base address for 32 bit PCI MMIO space.
For that reason, we should use that address too.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D31051
Sponsored by: Beckhoff Automation GmbH & Co. KG
OVMF will fail, if large 64 bit BARs are used. GCD-Map doesn't cover
64 bit addresses of BARs.
OVMF assumes that 64 bit addresses of BARS are located on next 32 GB
boundary behind Top of High RAM.
This patch moves 64 bit BARs on next 32 GB boundary behind Top of High
RAM to match OVMF assumptions.
Differential Revision: https://reviews.freebsd.org/D27970
Sponsored by: Beckhoff Automation GmbH & Co. KG
Allocating a BAR will call baraddr which maps the framebuffer. No need
to allocate it explicitly on init.
Reviewed by: grehan
Sponsored by: Beckhoff Autmation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D32596
Tell the guest whether a BAR uses prefetched memory or not for
passthru devices by using the same lobits as the physical device.
Reviewed by: grehan
Sponsored by: Beckhoff Autmation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D32685
Bump the firmware version to 14.0 and set the firmware release date
to today.
Reviewed by: jhb, bz, imp
Differential Revision: https://reviews.freebsd.org/D32534
Note, this breaks compatibility with snapshots generated by older builds
of bhyve(8).
Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for passthrough")
Reported by: Greg V <greg@unrelenting.technology>
Reviewed by: grehan, bz
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32523
Bhyve could emulate wrong PCI registers.
In the best case, the guest reads wrong registers and the device driver would
report some errors.
In the worst case, the guest writes to wrong PCI registers and could brick
hardware when using PCI passthrough.
According to Intels specification, low bits of CFGADR should be
ignored. Some OS like linux may rely on it. Otherwise, bhyve could
emulate a wrong PCI register.
E.g.
If linux would like to read 2 bytes from offset 0x02, following would
happen.
linux:
outl 0x80000002 at CFGADR
inw at CFGDAT + 2
bhyve:
cfgoff = 0x80000002 & 0xFF = 0x02
coff = cfgoff + (port - CFGDAT) = 0x02 + 0x02 = 0x04
Bhyve would emulate the register at offset 0x04 not 0x02.
Reviewed By: #bhyve, grehan
Differential Revision: https://reviews.freebsd.org/D31819
Sponsored by: Beckhoff Automation GmbH & Co. KG
Let's make it clear to users that -k is for configuration files.
Also, point to bhyve_config(5) in the paragraph describing the flag.
Reviewed by: jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32467
It is possible for the PBA to reside in the same page as the MSI-X
table. And, while devices are not supposed to do this, at least some
Intel wifi devices place registers in a page shared with the MSI-X
table. To handle the first case we currently map the PBA page using
/dev/mem, and the second case is not handled.
Kill two birds with one stone: map the MSI-X table BAR using the
PCIOCBARMMAP ioctl instead of /dev/mem, and map the entire table so that
accesses beyond the bounds of the table can be emulated. Regions of the
BAR not containing the table are left unmapped.
Reviewed by: bz, grehan, jhb
MFC after: 3 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32359
Allow user to specify the IP address available for gdb debugger.
Reviewed by: jhb, grehan, rgrimes, bcr (man pages)
Differential Revision: https://reviews.freebsd.org/D29607
This removes the dependency on /dev/io.
PR: 251046
Reviewed by: jhb
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31308
bhyve can also make use of fspacectl(2) to implement BOP_DELETE with
hole-punching. Since it is not desirable to do zero-filling for large
DEALLOCATE/UNMAP range, candelete is not set if pathconf(2) indicates
that the underlying file system does not support native
VOP_DEALLOCATE(9).
Sponsored by: The FreeBSD Foundation
Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D28880
If you want communications with the outside world and tell bhyve to
create an interfaces then it should be usable as well.
Rather than relying on the sysctl net.link.tap.up_on_open automatically
try to IFF_UP the opened tap device.
MFC after: 10 days
Reviewed by: markj, grehan
Differential Revision: https://reviews.freebsd.org/D31342
The UEFI driver included with Rocky Linux 8.4 uncovered an existing bug
in the NVMe emulation's construction of iovec's.
By default, NVMe data transfer operations use a scatter-gather list in
which all entries point to a fixed size memory region. For example, if
the Memory Page Size is 4KiB, a 2MiB IO requires 512 entries. Lists
themselves are also fixed size (default is 512 entries).
Because the list size is fixed, the last entry is special. If the IO
requires more than 512 entries, the last entry in the list contains the
address of the next list of entries. But if the IO requires exactly 512
entries, the last entry points to data.
The NVMe emulation missed this logic and unconditionally treated the
last entry as a pointer to the next list. Fix is to check if the
remaining data is greater than the page size before using the last entry
as a pointer to the next list.
PR: 256422
Reported by: dave@syix.com
Tested by: jason@tubnor.net
MFC after: 5 days
Relnotes: yes
Reviewed by: imp, grehan
Differential Revision: https://reviews.freebsd.org/D30897
Register a resize callback with the blockif interface. When the
callback fires, update the size of the disk and notify the guest via a
configuration change interrupt.
Reviewed by: grehan, markj
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D30506
This allows device models to assert VirtIO interrupts for reasons
other than publishing changes to a VirtIO ring such as configuration
changes.
Reviewed by: grehan, markj
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D30505
Allow clients of blockif to register a resize callback handler. When
a callback is registered, register an EVFILT_VNODE kevent watching the
backing store for a change in the file's attributes. If the size has
changed when the kevent fires, invoke the clients' callback.
Currently resize detection is limited to backing stores that support
EVFILT_VNODE kevents such as regular files.
Reviewed by: grehan, markj
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D30504
