Advertise Namespace Attribute Notices events in the Optional
Asynchronous Events Supported (OAES) field of the Identify Controller
data structure. Additionally, rename the enums and macros to clarify
these are AEN's related to Notices and not generic information.
Reported by: andy@omniosce.org
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D34331
The nvlist for a checkpoint request will now look like:
{ cmd="checkpoint", suspend="true/false", filename="afilename" }
Reviewed by: jhb
Suggested by: jhb
Differential Revision: https://reviews.freebsd.org/D34237
Switch to using an nvlist with nvlist_send()/nvlist_recv() to
communicate from bhyvectl(8) to bhyve(8).
The idea is that a bhyve process receives a command with with a set of
arguments. The nvlist here is structured to reflect that premise.
For example, to snapshot the vm, the expected nvlist looks like:
{ cmd=START_CHECKPOINT, filename="filename" }
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D33977
Section 4.1.2.1 of the virtio spec states that the transitional PCI
device id for a scsi device is 0x1004.
Fix suggested by reporter.
PR: 259961
Reported by: me@nanaya.pro
Reviewed by: imp, jhb
Fixes: f9c005a17f4e ("Add bhyve virtio-scsi storage backend support.")
Differential Revision: https://reviews.freebsd.org/D34103
If the NVMe Controller doesn't support Namespace Management, it should
return "Invalid Namespace or Format" when the Host request Identify
Namespace with the global NSID value.
Fixes UNH IOL 16.0 Test 9.1, Case 6
Reviewed by: imp, allanjude
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33578
NVMe Controllers which do not support Endurance Groups must return an
error when the Endurance Group Event Aggregate Log Change Notices bit is
set in Set Features, Asynchronous Event Configuration.
Fixes UNH IOL Test 3.12, Case 8
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33577
The function which checks for a valid LBA range mistakenly named an
input value as NLB ("Number of Logical Blocks") instead of "number of
blocks". The NVMe specification defines NLB as a zero-based value (i.e.
NLB=0x0 represents 1 block, 0x1 is 2 blocks, etc.), but the passed
parameter is a 1's-based value.
Fix is to rename the variable to avoid future confusion.
While in the neighborhood, also check that the starting LBA is less than
the size of the backing storage to avoid an integer overflow.
Reviewed by: imp, allanjude, jhb
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33575
Implement basic support for the SEL field of Get Features. This returns
information about Namespace Specific features.
Fixes UNH ILO 16.0 Test 1.2, Case 13
Reviewed by: imp, allanjude
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33574
Compliant v1.4 Controllers must report a Controller Type (CNTRLTYPE).
Also, do not advertise secure erase functionality in the Format NVM
Attributes field of the Identify Controller data structure as the
Controller does not implement secure erase.
Fixes UNH ILO Test 1.1, Case 2
Reviewed by: imp, allanjude
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33573
This adds the ability for a guest OS to send Set / Get Feature,
Temperature Threshold commands. The implementation assumes a constant
temperature and will generate an Asynchronous Event Notification if the
specified threshold is above/below this value. Although the
specification allows 9 temperature values, this implementation only
implements the Composite Temperature.
While in the neighborhood, move the clear of the CSTS register in the
reset function after all other cleanup. This avoids a race with the
guest thinking the reset is complete (i.e. CSTS.RDY = 0) before the NVMe
emulation is actually complete with the reset.
Fixes UNH IOL 16.0 Test 1.7, cases 1, 2, and 4.
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33572
Be more conservative and only support the Features mandatory for an I/O
Controller.
Avoids a "hang" in UNH test 1.2.10 associated with Predictable Latency
Mode Configuration and Host Behavior Support features.
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33571
The NVMe emulation checked if the Asynchronous Event Request Limit
(a.k.a AERL) would be exceeded in pci_nvme_aer_add(), but this function
is only called from nvme_opc_async_event_req() which also checks for
exceeding the AERL.
Reviewed by: imp, allanjude
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33570
Modify the Get Log Page command to parse the Log Page Offset fields to
support more recent versions of the NVMe specification.
Fixes various tests for UNH Test 1.3.*
Reviewed by: imp, allanjude
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33568
Return an error if the feature specified in Set Features is Namespace
specific but the Namespace ID uses the Global Namespace tag.
Fixes UNH Test 1.2.7
Reviewed by: imp, allanjude
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33566
The NVM Format command is unique among the Admin commands in that it
needs to finish asynchronously. For this reason, the emulation code
invented a synthetic completion status (NVME_NO_STATUS) to indicate that
the command was still in progress and the command processing loop should
not generate a completion message. The implementation used the value
0xffff for the synthetic value as this set both the Status Code and
Status Code Type fields to reserved values.
Format initialized the completion status to this value and expected
error cases to override it with a status code/type appropriate to the
situation. The macros used to set the NVMe status are careful not to
modify bit 0 (i.e. the phase bit), which with the synthetic completion
status, causes the phase bit to get out of sync. When running tests in a
guest with illegal NVM Format commands, Admin commands would eventually
hang because it appeared there were no completions due to the incorrect
phase bit value.
Fix is to only set NVME_NO_STATUS if the blockif delete command
succeeds. While in the neighborhood, add a missing break statement when
NVM Format is not supported.
Reviewed by: imp, allanjude
Tested by: jason@tubnor.net
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33565
This is needed to get mediasize of the device after a resize event.
I missed this earlier as I was building WITH_BHYVE_SNAPSHOT, which
disables capsicum.
Reviewed by: khng, markj
Fixes: ae9ea22e14bf ("bhyve: get mediasize for character devices when ...")
Differential Revision: https://reviews.freebsd.org/D34013
An AHCI controller may be specified with no connected ports. Avoid
dumping core in this case for compatibility with existing VM configs.
Reviewed by: khng, jhb
Fixes: 621b5090487de Refactor configuration management in bhyve.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D33969
Part two: Append bhyve -K option for specified keyboard layout
with layout setting files every languages.
Since the cmd option '-k' was used in the meantime
it was changed to '-K'
PR: 246121
Submitted by: koinec@yahoo.co.jp
Reviewed by: grehan@
Differential Revision: https://reviews.freebsd.org/D29473
MFC after: 4 weeks
pthread_create() returns 0 on success or an error number on failure.
Reviewed by: khng, markj
Differential Revision: https://reviews.freebsd.org/D33930
If an emulation uses those functions instead of set_config_value_node
or set_config_value, it allows the config values to get
overwritten. Introducing new functions is much more readable than
if else statements in the emulation code.
Reviewed by: khng
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33770
The starting address passed to mprotect was wrong, so in the case where
the last page containing the table is not the last page of the BAR, the
wrong region would be unmapped.
Reported by: Andy Fiddaman <andy@omniosce.org>
Reviewed by: jhb
Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for passthrough")
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33739
The PBA and MSI-X table can reside in different BARs.
Reported by: Andy Fiddaman <andy@omniosce.org>
Reviewed by: jhb
Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for passthrough")
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33739
Bhyve allocates small 64 bit BARs below 4 GB and generates ACPI tables
based on this allocation. If the guest decides to relocate those BARs
above 4 GB, it could lead to mismatching ACPI tables. Especially
when using OVMF with enabled bus enumeration it could cause
issues. OVMF relocates all 64 bit BARs above 4 GB. The guest OS
may be unable to recover from this situation and disables some PCI
devices because their BARs are located outside of the MMIO space
reported by ACPI. Avoid this situation by giving the guest more
space for relocating BARs.
Let's be paranoid. The available space for BARs below 4 GB is 512 MB
large. Use a slop of 512 MB. It'll allow the guest to relocate all
BARs below 4 GB to an address above 4 GB. We could run into issues
when we exceeding the memlimit above 4 GB. However, this space has
a size of 32 GB. Even when using many PCI device with large BARs
like framebuffer or when using multiple PCI busses, it's very
unlikely that we run out of space due to the large slop.
Additionally, this situation will occur on startup and not at runtime
which is much better.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33118
At the moment, you only have one single chance to read the fwctl
signature. At boot bhyve is in the state IDENT_WAIT. It's then
possible to switch to IDENT_SEND. After bhyve sends the signature,
it switches to REQ. From now on it's impossible to switch back to
IDENT_SEND to read the signature. For that reason, only a single
driver can read the signature. A guest can't use two drivers to
identify that fwctl is present. It gets even worse when using
OVMF. OVMF uses a library to access fwctl. Therefore, every single
OVMF driver would try to read the signature. Currently, only a
single OVMF driver accesses the fwctl. So, there's no issue with
it yet. However, no OS driver would have a chance to detect fwctl when
using OVMF because it's signature was already consumed by OVMF.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D31981
E.g. Framebuffers can require large space and BARs need to be aligned
by their size. If BARs aren't allocated by size, it'll cause much
fragmentation of the MMIO space. Reduce fragmentation by ordering
the BAR allocation on their size to reduce the risk of
OUT_OF_MMIO_SPACE issues.
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D28278
Some passthru devices only support MSI instead of MSI-X. For those
devices the initialization of MSI-X table will fail. Re-add the
check erroneously removed in f1442847c9404d4bc5f5524a0c3362dd39cb14f9.
MFC after: 3 days
X-MFC with: f1442847c9404d4bc5f5524a0c3362dd39cb14f9
PR: 260148
Reviewed by: manu, bz
Differential Revision: https://reviews.freebsd.org/D33728
If you look at the SMBIOS specification, we'll find something is
missing. In particular at offset 0Dh is supposed to be the OEM-defined
field. This should go between security and height. It is not legal to
actually skip this and will lead to other folks not properly
interpreting later parts of the table.
https://www.illumos.org/issues/14312
Reviewed by: jhb
Submitted by: Robert Mustacchi <rm@fingolfin.org>
Obtained from: ilumos
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33682
The first time we start bhyve with a passthru device everything is fine
as on boot we do enable BARs. If a driver (unload) inside bhyve disables
the BAR(s) as some Linux drivers do, we need to make sure we re-enable
them on next bhyve start.
If we are trying to mmap a disabled BAR for MSI-X (PCIOCBARMMAP)
the kernel will give us an EBUSY.
While we were re-enabling the BAR(s) in the current code loop
cfginit() was writing the changes out too late to the real hardware.
Move the call to init_msix_table() after the register on the real
hardware was updated. That way the kernel will be happy and the
mmap will succeed and bhyve will start.
Also simplify the code given the last argument to init_msix_table()
is unused we do not need to do checks for each bar. [1]
MFC after: 3 days
PR: 260148
Pointed out by: markj [1]
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D33628
Register a "block resize" callback to be notified of changes to the
backing storage for the Namespace. Use this to generate an Asynchronous
Event Notification, Namespace Attributes Changed when the guest OS
provides an Asynchronous Event Request.
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D32953
When an EVFILT_VNODE filter event is triggered, reset it.
This fixes the issue where a virtio-blk resize event would cause the
mevent thread to consume 100% of the cpu.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D33326
OVMF creates two separate .fd files, a _CODE.fd file containing
the UEFI code, and a _VARS.fd file containing a template of an
empty UEFI variable store.
OVMF decides to write variables to the memory range just below the
boot rom code if it detects a CFI flash device. So here we add
just the barest facsimile of CFI command handling to bootrom.c
that is needed to placate OVMF.
Submitted by: D Scott Phillips <d.scott.phillips@intel.com>
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D19976
MFC After: 1 week
NVMe conformance tests for the Format command failed if the
backing-storage for the bhyve device was a file instead of a Zvol. The
tests (and the specification) expect a Format to destroy all previously
written data. The bhyve NVMe emulation implements this by trimming /
deallocating all data from the backing-storage.
The blockif_candelete() function indicated the file did not support
deallocation (i.e. fpathconf(..., _PC_DEALLOC_PRESENT) returned FALSE)
even though the kernel supported file hole punching. This occurs on
builds with Capsicum enabled because blockif did not allow the
fpathconf(2) right.
Fix is to add CAP_FPATHCONF to the cap_rights_init(3) call.
PR: 260081
Reviewed by: allanjude, markj, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33203
Reads of the MSI-X capabilites aren't emulated by passthru devices
yet. The guest will read the host MSI-X capabilites which could
cause issues.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D32686
Sponsored by: Beckhoff Automation GmbH & Co. KG
On startup all virtual BARs are registered.
Additionally, the encoding bit in the virtual cmd register is set.
After that, the passthru emulation overwrites the virtual cmd register with
the physical one.
This could lead to a mismatch between registered BARs and the encoding
bits in the cmd register.
Instead of writing the physical to the virtual cmd register,
write the virtual to the physical cmd register to solve this issue.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D32687
Sponsored by: Beckhoff Automation GmbH & Co. KG
OVMF always uses 0xC0000000 as base address for 32 bit PCI MMIO space.
For that reason, we should use that address too.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D31051
Sponsored by: Beckhoff Automation GmbH & Co. KG
OVMF will fail, if large 64 bit BARs are used. GCD-Map doesn't cover
64 bit addresses of BARs.
OVMF assumes that 64 bit addresses of BARS are located on next 32 GB
boundary behind Top of High RAM.
This patch moves 64 bit BARs on next 32 GB boundary behind Top of High
RAM to match OVMF assumptions.
Differential Revision: https://reviews.freebsd.org/D27970
Sponsored by: Beckhoff Automation GmbH & Co. KG
