vmm and libvmmapi already have handlers for that. When adding debug
cpus, they were only used for the debug stub. Over time, they were
reused by other parts like snapshots or idle APs.
Reviewed by: corvink, jhb
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D40804
Unfortunately, this feature didn't receive much feedback in the past.
However, after committing this, some people came up and complain that
this feature requires some more discussion before upstreaming it.
Additionally, it wasn't a good idea to start this new feature by adding
a new command line parameter as it fixes the user interface.
This reverts commit c9fdd4f3cc.
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
This is required to enable capsicum for the snapshot code.
Reviewed by: corvink
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38858
This replaces the 'struct vm, int vcpuid' tuple passed to most API
calls and is similar to the changes recently made in vmm(4) in the
kernel.
struct vcpu is an opaque type managed by libvmmapi. For now it stores
a pointer to the VM context and an integer id.
As an immediate effect this removes the divergence between the kernel
and userland for the instruction emulation code introduced by the
recent vmm(4) changes.
Since this is a major change to the vmmapi API, bump VMMAPI_VERSION to
0x200 (2.0) and the shared library major version.
While here (and since the major version is bumped), remove unused
vcpu argument from vm_setup_pptdev_msi*().
Add new functions vm_suspend_all_cpus() and vm_resume_all_cpus() for
use by the debug server. The underyling ioctl (which uses a vcpuid of
-1) remains unchanged, but the userlevel API now uses separate
functions for global CPU suspend/resume.
Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38124
When using the --suspend and --checkpoint parameter, bhyvectl will
produce two checkpoint images and the exits. This is slightly ambiguous.
So, permit only one of theses parameters at the same time.
Reviewed by: corvink, markj
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38887
Avoid unaligned accesses in cpu_vendor_intel() and address a few other
nits. No functional change intended.
Reviewed by: corvink, rew, jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D38839
Now that nvlist_send()/nvlist_recv() are being used, ditch the datagram
socket.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D34863
The nvlist for a checkpoint request will now look like:
{ cmd="checkpoint", suspend="true/false", filename="afilename" }
Reviewed by: jhb
Suggested by: jhb
Differential Revision: https://reviews.freebsd.org/D34237
Switch to using an nvlist with nvlist_send()/nvlist_recv() to
communicate from bhyvectl(8) to bhyve(8).
The idea is that a bhyve process receives a command with with a set of
arguments. The nvlist here is structured to reflect that premise.
For example, to snapshot the vm, the expected nvlist looks like:
{ cmd=START_CHECKPOINT, filename="filename" }
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D33977
This is a step towards sending messages (other than suspend/checkpoint)
from bhyvectl to bhyve.
Introduce a new struct, ipc_message - this struct stores the type of
message and a union containing message specific structures for the type
of message being sent.
Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D30221
The save/restore feature uses a unix domain socket to send messages
from bhyvectl(8) to a bhyve(8) process. A datagram socket will suffice
for this.
An added benefit of using a datagram socket is simplified code. For
bhyve, the listen/accept calls are dropped; and for bhyvectl, the
connect() call is dropped.
EPRINTLN handles raw mode for bhyve(8), use it to print error messages.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D28983
Use errno to print a more descriptive error message when vm_open() fails
libvmm: preserve errno when vm_device_open() fails
vm_destroy() squashes errno by making a dive into sysctlbyname() - we
can safely skip vm_destroy() here since it's not doing any critical
clean up at this point. Replace vm_destroy() with a free() call.
PR: 250671
MFC after: 3 days
Submitted by: marko@apache.org
Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D29109
MAX_SNAPSHOT_VMNAME is a macro used to set the size of a character
buffer that stores a filename or the path to a file - this file is used
by the save/restore feature.
Since the file doesn't have anything to do with a vm name, rename
MAX_SNAPSHOT_VMNAME to MAX_SNAPSHOT_FILENAME. Bump the size to PATH_MAX
while here.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D28879
Combine send_start_checkpoint() and send_start_suspend() into a
single function named snapshot_request().
snapshot_request() is equivalent to send_start_checkpoint() and
send_start_suspend() except that it takes an additional argument. The
additional argument, enum ipc_opcode, is used to determine the type of
snapshot request being performed. Also, switch to using strlcpy instead
of strncpy.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D28878
Add /var/run/bhyve/ to BSD.var.dist so we don't have to call mkdir when
creating the unix domain socket for a given bhyve vm.
The path to the unix domain socket for a bhyve vm will now be
/var/run/bhyve/vmname instead of /var/run/bhyve/checkpoint/vmname
Move BHYVE_RUN_DIR from snapshot.c to snapshot.h so it can be shared
to bhyvectl(8).
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D28783
struct checkpoint_op, enum checkpoint_opcodes, and
MAX_SNAPSHOT_VMNAME are not vmm specific, move them out of the vmmapi
header.
They are used for the save/restore functionality that bhyve(8)
provides and are better suited in usr.sbin/bhyve/snapshot.h
Since bhyvectl(8) requires these, the Makefile for bhyvectl has been
modified to include usr.sbin/bhyve/snapshot.h
Reviewed by: kevans, grehan
Differential Revision: https://reviews.freebsd.org/D28410
Don't permit setting the exception bitmap or VMCS entry interrupt
information. These are not generally useful to set. If it is needed
in the future, dedicated pseudo registers can be added for these that
would be used with vm_set_register().
Discussed with: grehan
MFC after: 1 week
Save and restore (also known as suspend and resume) permits a snapshot
to be taken of a guest's state that can later be resumed. In the
current implementation, bhyve(8) creates a UNIX domain socket that is
used by bhyvectl(8) to send a request to save a snapshot (and
optionally exit after the snapshot has been taken). A snapshot
currently consists of two files: the first holds a copy of guest RAM,
and the second file holds other guest state such as vCPU register
values and device model state.
To resume a guest, bhyve(8) must be started with a matching pair of
command line arguments to instantiate the same set of device models as
well as a pointer to the saved snapshot.
While the current implementation is useful for several uses cases, it
has a few limitations. The file format for saving the guest state is
tied to the ABI of internal bhyve structures and is not
self-describing (in that it does not communicate the set of device
models present in the system). In addition, the state saved for some
device models closely matches the internal data structures which might
prove a challenge for compatibility of snapshot files across a range
of bhyve versions. The file format also does not currently support
versioning of individual chunks of state. As a result, the current
file format is not a fixed binary format and future revisions to save
and restore will break binary compatiblity of snapshot files. The
goal is to move to a more flexible format that adds versioning,
etc. and at that point to commit to providing a reasonable level of
compatibility. As a result, the current implementation is not enabled
by default. It can be enabled via the WITH_BHYVE_SNAPSHOT=yes option
for userland builds, and the kernel option BHYVE_SHAPSHOT.
Submitted by: Mihai Tiganus, Flavius Anton, Darius Mihai
Submitted by: Elena Mihailescu, Mihai Carabas, Sergiu Weisz
Relnotes: yes
Sponsored by: University Politehnica of Bucharest
Sponsored by: Matthew Grooms (student scholarships)
Sponsored by: iXsystems
Differential Revision: https://reviews.freebsd.org/D19495
from userland without the need to use sysctls, it allows the old
sysctls to continue to function, but deprecates them at
FreeBSD_version 1200060 (Relnotes for deprecate).
The command line of bhyve is maintained in a backwards compatible way.
The API of libvmmapi is maintained in a backwards compatible way.
The sysctl's are maintained in a backwards compatible way.
Added command option looks like:
bhyve -c [[cpus=]n][,sockets=n][,cores=n][,threads=n][,maxcpus=n]
The optional parts can be specified in any order, but only a single
integer invokes the backwards compatible parse. [,maxcpus=n] is
hidden by #ifdef until kernel support is added, though the api
is put in place.
bhyvectl --get-cpu-topology option added.
Reviewed by: grehan (maintainer, earlier version),
Reviewed by: bcr (manpages)
Approved by: bde (mentor), phk (mentor)
Tested by: Oleg Ginzburg <olevole@olevole.ru> (cbsd)
MFC after: 1 week
Relnotes: Y
Differential Revision: https://reviews.freebsd.org/D9930
Currently most of the debug registers are not saved and restored
during VM transitions allowing guest and host debug register values to
leak into the opposite context. One result is that hardware
watchpoints do not work reliably within a guest under VT-x.
Due to differences in SVM and VT-x, slightly different approaches are
used.
For VT-x:
- Enable debug register save/restore for VM entry/exit in the VMCS for
DR7 and MSR_DEBUGCTL.
- Explicitly save DR0-3,6 of the guest.
- Explicitly save DR0-3,6-7, MSR_DEBUGCTL, and the trap flag from
%rflags for the host. Note that because DR6 is "software" managed
and not stored in the VMCS a kernel debugger which single steps
through VM entry could corrupt the guest DR6 (since a single step
trap taken after loading the guest DR6 could alter the DR6
register). To avoid this, explicitly disable single-stepping via
the trace flag before loading the guest DR6. A determined debugger
could still defeat this by setting a breakpoint after the guest DR6
was loaded and then single-stepping.
For SVM:
- Enable debug register caching in the VMCB for DR6/DR7.
- Explicitly save DR0-3 of the guest.
- Explicitly save DR0-3,6-7, and MSR_DEBUGCTL for the host. Since SVM
saves the guest DR6 in the VMCB, the race with single-stepping
described for VT-x does not exist.
For both platforms, expose all of the guest DRx values via --get-drX
and --set-drX flags to bhyvectl.
Discussed with: avg, grehan
Tested by: avg (SVM), myself (VT-x)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D13229
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
No functional change intended.
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.
Currently rescue/rescue and sys/modules are still not connected.
Sponsored by: EMC / Isilon Storage Division
devmem is used to represent MMIO devices like the boot ROM or a VESA framebuffer
where doing a trap-and-emulate for every access is impractical. devmem is a
hybrid of system memory (sysmem) and emulated device models.
devmem is mapped in the guest address space via nested page tables similar
to sysmem. However the address range where devmem is mapped may be changed
by the guest at runtime (e.g. by reprogramming a PCI BAR). Also devmem is
usually mapped RO or RW as compared to RWX mappings for sysmem.
Each devmem segment is named (e.g. "bootrom") and this name is used to
create a device node for the devmem segment (e.g. /dev/vmm/testvm.bootrom).
The device node supports mmap(2) and this decouples the host mapping of
devmem from its mapping in the guest address space (which can change).
Reviewed by: tychon
Discussed with: grehan
Differential Revision: https://reviews.freebsd.org/D2762
MFC after: 4 weeks
