Commit Graph

19 Commits

Author SHA1 Message Date
Neel Natu
9b1aa8d622 Restructure memory allocation in bhyve to support "devmem".
devmem is used to represent MMIO devices like the boot ROM or a VESA framebuffer
where doing a trap-and-emulate for every access is impractical. devmem is a
hybrid of system memory (sysmem) and emulated device models.

devmem is mapped in the guest address space via nested page tables similar
to sysmem. However the address range where devmem is mapped may be changed
by the guest at runtime (e.g. by reprogramming a PCI BAR). Also devmem is
usually mapped RO or RW as compared to RWX mappings for sysmem.

Each devmem segment is named (e.g. "bootrom") and this name is used to
create a device node for the devmem segment (e.g. /dev/vmm/testvm.bootrom).
The device node supports mmap(2) and this decouples the host mapping of
devmem from its mapping in the guest address space (which can change).

Reviewed by:	tychon
Discussed with:	grehan
Differential Revision:	https://reviews.freebsd.org/D2762
MFC after:	4 weeks
2015-06-18 06:00:17 +00:00
Allan Jude
0db293c171 Fix off-by-one in array index bounds check
bhyveload would allow you to create 33 entries on an array that only has 32 slots

Differential Revision:	https://reviews.freebsd.org/D2569
Reviewed by:	araujo
Approved by:	neel
MFC after:	1 week
Sponsored by:	ScaleEngine Inc.
2015-05-18 19:45:46 +00:00
John Baldwin
cde1f5b8a0 Sort command flags in usage output and the manpages. 2014-06-27 15:20:34 +00:00
Neel Natu
be679db4cd Provide APIs to directly get 'lowmem' and 'highmem' size directly.
Previously the sizes were inferred indirectly based on the size of the mappings
at 0 and 4GB respectively. This works fine as long as size of the allocation is
identical to the size of the mapping in the guest's address space. However, if
the mapping is disjoint then this assumption falls apart (e.g., due to the
legacy BIOS hole between 640KB and 1MB).
2014-06-24 02:02:51 +00:00
Neel Natu
5fcf252f41 Add ioctl(VM_REINIT) to reinitialize the virtual machine state maintained
by vmm.ko. This allows the virtual machine to be restarted without having
to destroy it first.

Reviewed by:	grehan
2014-06-07 21:36:52 +00:00
Peter Grehan
cf087c12c2 ZFS boot support for bhyveload.
Modelled after the i386 zfsloader. However, with no
2nd stage zfsboot to search for a bootable dataset,
attempt a ZFS boot if there is more than one ZFS
dataset found during the disk probe.

sys/boot/userboot/zfs
 - build the ZFS boot library

sys/boot/userboot/userboot/
 conf.c
  - Add the ZFS pool and filesystem tables
 devicename.c
  - correctly format ZFS devices
 main.c
  - increase the size of the libstand malloc pool
  to account for the increased usage from ZFS buffers
  - probe for a ZFS dataset, and if one is
  found, attempt to boot from it.

usr.sbin/bhyveload/bhyveload.c
 - allow multiple invocations of the '-d' option
 to specify multiple disks e.g. a raidz set.
 Up to 32 disks are supported.

Tested with various combinations of GPT, MBR, single
and multiple disks, RAID-Z, mirrors.

Reviewed by:	neel
Discussed with:	avg
Tested by:	Michael Dexter and others
MFC after:	3 weeks
2014-02-22 07:18:06 +00:00
John Baldwin
00f3efe1bd Add support for FreeBSD/i386 guests under bhyve.
- Similar to the hack for bootinfo32.c in userboot, define
  _MACHINE_ELF_WANT_32BIT in the load_elf32 file handlers in userboot.
  This allows userboot to load 32-bit kernels and modules.
- Copy the SMAP generation code out of bootinfo64.c and into its own
  file so it can be shared with bootinfo32.c to pass an SMAP to the i386
  kernel.
- Use uint32_t instead of u_long when aligning module metadata in
  bootinfo32.c in userboot, as otherwise the metadata used 64-bit
  alignment which corrupted the layout.
- Populate the basemem and extmem members of the bootinfo struct passed
  to 32-bit kernels.
- Fix the 32-bit stack in userboot to start at the top of the stack
  instead of the bottom so that there is room to grow before the
  kernel switches to its own stack.
- Push a fake return address onto the 32-bit stack in addition to the
  arguments normally passed to exec() in the loader.  This return
  address is needed to convince recover_bootinfo() in the 32-bit
  locore code that it is being invoked from a "new" boot block.
- Add a routine to libvmmapi to setup a 32-bit flat mode register state
  including a GDT and TSS that is able to start the i386 kernel and
  update bhyveload to use it when booting an i386 kernel.
- Use the guest register state to determine the CPU's current instruction
  mode (32-bit vs 64-bit) and paging mode (flat, 32-bit, PAE, or long
  mode) in the instruction emulation code.  Update the gla2gpa() routine
  used when fetching instructions to handle flat mode, 32-bit paging, and
  PAE paging in addition to long mode paging.  Don't look for a REX
  prefix when the CPU is in 32-bit mode, and use the detected mode to
  enable the existing 32-bit mode code when decoding the mod r/m byte.

Reviewed by:	grehan, neel
MFC after:	1 month
2014-02-05 04:39:03 +00:00
Peter Grehan
384305ff48 Don't create an initial value for the host filesystem of "/".
This has the unintended effect of booting the host kernel if
a disk image open fails.

Discussed with:	neel
2013-11-27 06:07:03 +00:00
Peter Grehan
6380102c7f Allow bhyve and bhyveload to attach to tty devices.
bhyveload: introduce the -c <device> parameter
 to select a tty for output (or "stdio")

bhyve: allow the puc and lpc-com backends to
 accept a tty in addition to "stdio"

When used in conjunction with the null-modem device,
nmdm(4), this allows attach/detach to the guest console
and multiple concurrent serial ports. kgdb on a serial
port is now functional.

Reviewed by:	neel
Requested by:	Almost everyone that has used bhyve
MFC after:	10.0
2013-11-27 00:21:37 +00:00
Neel Natu
b5331f4d88 Tidy usage messages for bhyve and bhyveload.
Submitted by:	jhb
2013-10-23 21:42:53 +00:00
Neel Natu
b6afa84b8c Add an option to bhyveload(8) that allows setting a loader environment variable
from the command line.

The option syntax is "-e <name=value>". It may be used multiple times to set
multiple environment variables.

Reviewed by:	grehan
Requested by:	alfred
2013-10-17 00:28:35 +00:00
Neel Natu
200758f114 Parse the memory size parameter using expand_number() to allow specifying
the memory size more intuitively (e.g. 512M, 4G etc).

Submitted by:	rodrigc
Reviewed by:	grehan
Approved by:	re (blanket)
2013-10-09 03:56:07 +00:00
Neel Natu
318224bbe6 Merge projects/bhyve_npt_pmap into head.
Make the amd64/pmap code aware of nested page table mappings used by bhyve
guests. This allows bhyve to associate each guest with its own vmspace and
deal with nested page faults in the context of that vmspace. This also
enables features like accessed/dirty bit tracking, swapping to disk and
transparent superpage promotions of guest memory.

Guest vmspace:
Each bhyve guest has a unique vmspace to represent the physical memory
allocated to the guest. Each memory segment allocated by the guest is
mapped into the guest's address space via the 'vmspace->vm_map' and is
backed by an object of type OBJT_DEFAULT.

pmap types:
The amd64/pmap now understands two types of pmaps: PT_X86 and PT_EPT.

The PT_X86 pmap type is used by the vmspace associated with the host kernel
as well as user processes executing on the host. The PT_EPT pmap is used by
the vmspace associated with a bhyve guest.

Page Table Entries:
The EPT page table entries as mostly similar in functionality to regular
page table entries although there are some differences in terms of what
bits are used to express that functionality. For e.g. the dirty bit is
represented by bit 9 in the nested PTE as opposed to bit 6 in the regular
x86 PTE. Therefore the bitmask representing the dirty bit is now computed
at runtime based on the type of the pmap. Thus PG_M that was previously a
macro now becomes a local variable that is initialized at runtime using
'pmap_modified_bit(pmap)'.

An additional wrinkle associated with EPT mappings is that older Intel
processors don't have hardware support for tracking accessed/dirty bits in
the PTE. This means that the amd64/pmap code needs to emulate these bits to
provide proper accounting to the VM subsystem. This is achieved by using
the following mapping for EPT entries that need emulation of A/D bits:
               Bit Position           Interpreted By
PG_V               52                 software (accessed bit emulation handler)
PG_RW              53                 software (dirty bit emulation handler)
PG_A               0                  hardware (aka EPT_PG_RD)
PG_M               1                  hardware (aka EPT_PG_WR)

The idea to use the mapping listed above for A/D bit emulation came from
Alan Cox (alc@).

The final difference with respect to x86 PTEs is that some EPT implementations
do not support superpage mappings. This is recorded in the 'pm_flags' field
of the pmap.

TLB invalidation:
The amd64/pmap code has a number of ways to do invalidation of mappings
that may be cached in the TLB: single page, multiple pages in a range or the
entire TLB. All of these funnel into a single EPT invalidation routine called
'pmap_invalidate_ept()'. This routine bumps up the EPT generation number and
sends an IPI to the host cpus that are executing the guest's vcpus. On a
subsequent entry into the guest it will detect that the EPT has changed and
invalidate the mappings from the TLB.

Guest memory access:
Since the guest memory is no longer wired we need to hold the host physical
page that backs the guest physical page before we can access it. The helper
functions 'vm_gpa_hold()/vm_gpa_release()' are available for this purpose.

PCI passthru:
Guest's with PCI passthru devices will wire the entire guest physical address
space. The MMIO BAR associated with the passthru device is backed by a
vm_object of type OBJT_SG. An IOMMU domain is created only for guest's that
have one or more PCI passthru devices attached to them.

Limitations:
There isn't a way to map a guest physical page without execute permissions.
This is because the amd64/pmap code interprets the guest physical mappings as
user mappings since they are numerically below VM_MAXUSER_ADDRESS. Since PG_U
shares the same bit position as EPT_PG_EXECUTE all guest mappings become
automatically executable.

Thanks to Alan Cox and Konstantin Belousov for their rigorous code reviews
as well as their support and encouragement.

Thanks for John Baldwin for reviewing the use of OBJT_SG as the backing
object for pci passthru mmio regions.

Special thanks to Peter Holm for testing the patch on short notice.

Approved by:	re
Discussed with:	grehan
Reviewed by:	alc, kib
Tested by:	pho
2013-10-05 21:22:35 +00:00
Neel Natu
b060ba5024 Simplify the assignment of memory to virtual machines by requiring a single
command line option "-m <memsize in MB>" to specify the memory size.

Prior to this change the user needed to explicitly specify the amount of
memory allocated below 4G (-m <lowmem>) and the amount above 4G (-M <highmem>).

The "-M" option is no longer supported by 'bhyveload' and 'bhyve'.

The start of the PCI hole is fixed at 3GB and cannot be directly changed
using command line options. However it is still possible to change this in
special circumstances via the 'vm_set_lowmem_limit()' API provided by
libvmmapi.

Submitted by:	Dinakar Medavaram (initial version)
Reviewed by:	grehan
Obtained from:	NetApp
2013-03-18 22:38:30 +00:00
Neel Natu
e37bc32183 Reduce the default memory allocation for a VM from 768MB to 128MB.
Obtained from:	NetApp
2013-01-08 01:56:05 +00:00
Neel Natu
a10c6f5544 IFC @ r242684 2012-11-11 03:26:14 +00:00
Neel Natu
c3e9ce3312 Use the new userboot 'getenv' callback to set a couple of environment variables
in the guest.

The variables are: smbios.bios.vendor=BHYVE and boot_serial=1

The FreeBSD guest uses the "smbios.bios.vendor" environment variable to
detect whether or not it is running as a guest inside a hypervisor.

The "boot_serial=1" is temporary and will be dropped when bhyve can do VGA
emulation.

Obtained from:	NetApp
2012-11-06 21:48:45 +00:00
Peter Grehan
38f1b189cd IFC @ r234692
sys/amd64/include/cpufunc.h
sys/amd64/include/fpu.h
sys/amd64/amd64/fpu.c
sys/amd64/vmm/vmm.c

 - Add API to allow vmm FPU state init/save/restore.

FP stuff discussed with: kib
2012-04-26 07:52:28 +00:00
Neel Natu
c487da1e19 'bhyveload' is a userspace FreeBSD loader that can load the kernel + metadata
inside a BHyVe-based virtual machine.

It is a thin wrapper on top of userboot.so which is a variant of the FreeBSD
loader packaged as a shared library. 'bhyveload' provides callbacks that are
utilized by userboot.so to do things like console i/o, disk i/o,
set virtual machine registers etc.

Thanks for Doug Rabson (dfr@) for making this happen.
2011-07-06 22:38:09 +00:00