* nsegs must be at most BLKIF_MAX_SEGMENTS_PER_REQUEST (since we specify
that limit to bus_dma_tag_create), so KASSERT that rather than silently
adjusting the request.
* block_segs is now a synonym for nsegs, so garbage collect that variable.
* nsegs is never read during or after the while loop, so remove the dead
decrement from the loop.
These were all left behind from the pre-r284296 support for a "segment
block" extension.
The FreeBSD extension adds a new request type, called blkif_segment_block
which has a size of 112bytes for both i386 and amd64. This is fine on
amd64, since requests have a size of 112B there also. But this is not true
for i386, where requests have a size of 108B. So on i386 we basically
overrun the ring slot when queuing a request of type blkif_segment_block_t,
which is very bad.
Remove this extension (including a cleanup of the public blkif.h header
file) from blkfront and blkback.
Sponsored by: Citrix Systems R&D
Tested-by: cperciva
years for head. However, it is continuously misused as the mpsafe argument
for callout_init(9). Deprecate the flag and clean up callout_init() calls
to make them more consistent.
Differential Revision: https://reviews.freebsd.org/D2613
Reviewed by: jhb
MFC after: 2 weeks
Netfront has to wait for the backend to switch to state XenbusStateConnected
before sending the ARP request, or else the backend might not be connected
and thus the packet will be lost.
Sponsored by: Citrix Systems R&D
MFC after: 1 week
In order to map memory from other domains when running on Xen FreeBSD uses
unused physical memory regions. Until now this memory has been allocated
using bus_alloc_resource, but this is not completely safe as we can end up
using unreclaimed MMIO or ACPI regions.
Fix this by introducing a new newbus method that can be used by Xen drivers
to request for unused memory regions. On amd64 we make sure this memory
comes from regions above 4GB in order to prevent clashes with MMIO/ACPI
regions. On i386 there's nothing we can do, so just fall back to the
previous mechanism.
Sponsored by: Citrix Systems R&D
Tested by: Gustau Pérez <gperez@entel.upc.edu>
remains. Xen is planning to phase out support for PV upstream since it
is harder to maintain and has more overhead. Modern x86 CPUs include
virtualization extensions that support HVM guests instead of PV guests.
In addition, the PV code was i386 only and not as well maintained recently
as the HVM code.
- Remove the i386-only NATIVE option that was used to disable certain
components for PV kernels. These components are now standard as they
are on amd64.
- Remove !XENHVM bits from PV drivers.
- Remove various shims required for XEN (e.g. PT_UPDATES_FLUSH, LOAD_CR3,
etc.)
- Remove duplicate copy of <xen/features.h>.
- Remove unused, i386-only xenstored.h.
Differential Revision: https://reviews.freebsd.org/D2362
Reviewed by: royger
Tested by: royger (i386/amd64 HVM domU and amd64 PVH dom0)
Relnotes: yes
Handle the VIRQ_DEBUG signal and print a stack trace of each vCPU on the Xen
console. This is only used for debug purposes and is triggered by the
administrator of the Xen host.
Sponsored by: Citrix Systems R&D
MFC after: 1 week
The current GSO implementation in netback is broken and causes errors on the
guest tx path. While this is fixed disable GSO in order to have a working
netback.
Sponsored by: Citrix Systems R&D
Discussed with: gibbs
Xenstore user-space device has two problems currently:
- It does not correctly handle concurrent clients, because it's storing
each client data in dev->si_drv1.
- It does not correctly free this data when the client closes the device.
In order to solve both of this issues store the per-client data using
cdevpriv, which also comes with a hook in order to perform the necessary
cleanup on device close.
While there also make the device eternal.
Sponsored by: Citrix Systems R&D
Reported and Tested by: thompsa
MFC after: 2 weeks
KVM clock shares the same data structures between the guest and the host
as Xen so it makes sense to just have a single copy of this code.
Differential Revision: https://reviews.freebsd.org/D1429
Reviewed by: royger (eariler version)
MFC after: 1 month
Before this change, the current code handles SIOCGIFADDR the same
way with SIOCSIFADDR, which involves full arp_ifinit, et al. They
should be unnecessary for SIOCGIFADDR case.
Differential Revision: https://reviews.freebsd.org/D1508
Reviewed by: glebius
MFC after: 2 weeks
socket-buffer implementations, introduce a return value for MCLGET()
(and m_cljget() that underlies it) to allow the caller to avoid testing
M_EXT itself. Update all callers to use the return value.
With this change, very few network device drivers remain aware of
M_EXT; the primary exceptions lie in mbuf-chain pretty printers for
debugging, and in a few cases, custom mbuf and cluster allocation
implementations.
NB: This is a difficult-to-test change as it touches many drivers for
which I don't have physical devices. Instead we've gone for intensive
review, but further post-commit review would definitely be appreciated
to spot errors where changes could not easily be made mechanically,
but were largely mechanical in nature.
Differential Revision: https://reviews.freebsd.org/D1440
Reviewed by: adrian, bz, gnn
Sponsored by: EMC / Isilon Storage Division
This allows the Grant-table code to attach directly to the xenpv bus,
allowing us to remove the grant-table initialization done in xenpv.
Sponsored by: Citrix Systems R&D
Mave the grant table code into the dev/xen folder in preparation for turning
it into a device using the newbus interface. This is just code motion, no
functional changes.
Sponsored by: Citrix Systems R&D
When running as a Xen PVH Dom0 we need to add custom buses that override
some of the functionality present in the ACPI PCI Bus and the PCI Bus. We
currently override the ACPI PCI Bus, but not the PCI Bus, so add a new
override for the PCI Bus and share the generic functions between them.
Reported by: David P. Discher <dpd@dpdtech.com>
Sponsored by: Citrix Systems R&D
conf/files.amd64:
- Add the new files.
x86/xen/xen_pci_bus.c:
- Generic file that contains the PCI overrides so they can be used by the
several PCI specific buses.
xen/xen_pci.h:
- Prototypes for the generic overried functions.
dev/xen/pci/xen_pci.c:
- Xen specific override for the PCI bus.
dev/xen/pci/xen_acpi_pci.c:
- Xen specific override for the ACPI PCI bus.
Current FreeBSD netback names the interface with xnb<device unit>, but
this is not suitable for usage with the Xen toolstack, which expects
something similar to <prefix><domid><handle>. In order to solve this,
change the netback naming convention to use xnb<domid>.<handle>.
Sponsored by: Citrix Systems R&D
dev/xen/netback/netback.c:
- Change netback to use the nomenclature stated above.
This device is only attached to priviledged domains, and allows the
toolstack to interact with Xen. The two functions of the privcmd
interface is to allow the execution of hypercalls from user-space, and
the mapping of foreign domain memory.
Sponsored by: Citrix Systems R&D
i386/include/xen/hypercall.h:
amd64/include/xen/hypercall.h:
- Introduce a function to make generic hypercalls into Xen.
xen/interface/xen.h:
xen/interface/memory.h:
- Import the new hypercall XENMEM_add_to_physmap_range used by
auto-translated guests to map memory from foreign domains.
dev/xen/privcmd/privcmd.c:
- This device has the following functions:
- Allow user-space applications to make hypercalls into Xen.
- Allow user-space applications to map memory from foreign domains,
this is accomplished using the newly introduced hypercall
(XENMEM_add_to_physmap_range).
xen/privcmd.h:
- Public ioctl interface for the privcmd device.
x86/xen/hvm.c:
- Remove declaration of hypercall_page, now it's declared in
hypercall.h.
conf/files:
- Add the privcmd device to the build process.
The user-space event channel device is used by applications to receive
and send event channel interrupts. This device is based on the Linux
evtchn device.
Sponsored by: Citrix Systems R&D
xen/evtchn/evtchn_dev.c:
- Remove the old event channel device, which was already disabled in
the build system.
dev/xen/evtchn/evtchn_dev.c:
- Import a new event channel device based on the one present in
Linux.
- This device allows the following operations:
- Bind VIRQ event channels (ioctl).
- Bind regular event channels (ioctl).
- Create and bind new event channels (ioctl).
- Unbind event channels (ioctl).
- Send notifications to event channels (ioctl).
- Reset the device shared memory ring (ioctl).
- Unmask event channels (write).
- Receive event channel upcalls (read).
- The new code is MP safe, and can be used concurrently.
conf/files:
- Add the new device to the build system.
- Wrong integer type was specified.
- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.
- Logical OR where binary OR was expected.
- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.
- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.
- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.
- Updated "EXAMPLES" section in SYSCTL manual page.
MFC after: 3 days
Sponsored by: Mellanox Technologies
Fix a problem where the blockback driver could run out of requests,
despite the fact that we allocate enough request and reqlist
structures to satisfy the maximum possible number of requests.
The problem was that we were sending responses back to the other
end (blockfront) before freeing resources. The Citrix Windows
driver is pretty agressive about queueing, and would queue more I/O
to us immediately after we sent responses to it. We would run into
a resource shortage and stall out I/O until we freed resources.
It isn't clear whether the request shortage condition was an
indirect cause of the I/O hangs we've been seeing between Windows
with the Citrix PV drivers and FreeBSD's blockback, but the above
problem is certainly a bug.
Sponsored by: Spectra Logic
Submitted by: ken
Reviewed by: royger
dev/xen/blkback/blkback.c:
- Break xbb_send_response() into two sub-functions,
xbb_queue_response() and xbb_push_responses().
Remove xbb_send_response(), because it is no longer
used.
- Adjust xbb_complete_reqlist() so that it calls the
two new functions, and holds the mutex around both
calls. The mutex insures that another context
can't come along and push responses before we've
freed our resources.
- Change xbb_release_reqlist() so that it requires
the mutex to be held instead of acquiring the mutex
itself. Both callers could easily hold the mutex
while calling it, and one really needs to hold the
mutex during the call.
- Add two new counters, accessible via sysctl
variables. The first one counts the number of
I/Os that are queued and waiting to be pushed
(reqs_queued_for_completion). The second one
(reqs_completed_with_error) counts the number of
requests we've completed with an error status.
Using realmem on PVH is not realiable, since in this case the realmem value
is computed from Maxmem, which contains the higher memory address found. Use
HYPERVISOR_start_info->nr_pages instead, which is set by the hypervisor and
contains the exact number of memory pages assigned to the domain.
Sponsored by: Citrix Systems R&D
This device is used by the user-space daemon that runs xenstore
(xenstored). It allows xenstored to map the xenstore memory page, and
reports the event channel xenstore is using.
Sponsored by: Citrix Systems R&D
dev/xen/xenstore/xenstored_dev.c:
- Add the xenstored character device that's used to map the xenstore
memory into user-space, and to report the event channel used by
xenstore.
conf/files:
- Add the device to the build process.
Convert the xenstore user-space device (/dev/xen/xenstore) to a device
using the newbus interface. This allows us to make the device
initialization dependant on the initialization of xenstore itself in
the kernel.
Sponsored by: Citrix Systems R&D
dev/xen/xenstore/xenstore.c:
- Convert to a newbus device, this removes the xs_dev_init function.
xen/xenstore/xenstore_internal.h:
- Remove xs_dev_init prototype.
dev/xen/xenstore/xenstore.c:
- Don't call xs_dev_init anymore, the device will attach itself when
xenstore is started.
The xenstore related devices in the kernel cannot be started until
xenstored is running, which will happen later in the Dom0 case. If
start_info_t doesn't contain a valid xenstore event channel, defer all
xenstore related devices attachment to later.
Sponsored by: Citrix Systems R&D
dev/xen/xenstore/xenstore.c:
- Prevent xenstore from trying to attach it's descendant devices if
xenstore is not initialized.
- Add a callback in the xenstore interrupt filter that will trigger
the plug of xenstore descendant devices on the first received
interrupt. This interrupt is generated when xenstored attaches to
the event channel, and serves as a notification that xenstored is
running.
Move xenstore related devices (xenstore.c and xenstore_dev.c) from
xen/xenstore to dev/xen/xenstore. This is just code motion, no
functional changes.
Sponsored by: Citrix Systems R&D
This is done so we can prevent the Xen Balloon driver from attaching
before xenstore is setup.
Sponsored by: Citrix Systems R&D
dev/xen/balloon/balloon.c:
- Make xen balloon a driver that depends on xenstore.
The current TSO limitation feature only takes the total number of
bytes in an mbuf chain into account and does not limit by the number
of mbufs in a chain. Some kinds of hardware is limited by two
factors. One is the fragment length and the second is the fragment
count. Both of these limits need to be taken into account when doing
TSO. Else some kinds of hardware might have to drop completely valid
mbuf chains because they cannot loaded into the given hardware's DMA
engine. The new way of doing TSO limitation has been made backwards
compatible as input from other FreeBSD developers and will use
defaults for values not set.
Reviewed by: adrian, rmacklem
Sponsored by: Mellanox Technologies
MFC after: 1 week
The current TSO limitation feature only takes the total number of
bytes in an mbuf chain into account and does not limit by the number
of mbufs in a chain. Some kinds of hardware is limited by two
factors. One is the fragment length and the second is the fragment
count. Both of these limits need to be taken into account when doing
TSO. Else some kinds of hardware might have to drop completely valid
mbuf chains because they cannot loaded into the given hardware's DMA
engine. The new way of doing TSO limitation has been made backwards
compatible as input from other FreeBSD developers and will use
defaults for values not set.
MFC after: 1 week
Sponsored by: Mellanox Technologies
Current busdma code for unmapped bios will not properly align the segment
size, causing corruption on blkfront devices. Revert the commit until
busdma code is fixed.
Reported by: mav
MFC after: 1 day
Fix some frees incorrectly assigned to M_XENBUS when the memory is
allocated with M_XENSTORE.
Sponsored by: Citrix Systems R&D
MFC after: 1 week
dev/xen/blkback/blkback.c:
- Fix incorrect frees.
This patch contains the following fixes for netback:
- Only unbind the evtchn if it has been bound.
- Set xnb->bridge to NULL after free to prevent double-freeing it.
- Set the MAC address for the host-facing interface to a dummy value.
Sponsored by: Citrix Systems R&D
dev/xen/netback/netback.c:
- Prevent trying to unbind if the evtchn has not been bounded.
- Prevent double-freeing xnb->bridge.
- Set the MAC address of the host-facing interface to a dummy value,
so it can work when the interface is added to a bridge.
Using unmapped IO is really beneficial when running inside of a VM,
since it avoids IPIs to other vCPUs in order to invalidate the
mappings.
This patch adds unmapped IO support to blkfront. The following tests
results have been obtained when running on a Xen host without HAP:
PVHVM
3165.84 real 6354.17 user 4483.32 sys
PVHVM with unmapped IO
2099.46 real 4624.52 user 2967.38 sys
This is because when running using shadow page tables TLB flushes and
range invalidations are much more expensive, so using unmapped IO
provides a very important performance boost.
Sponsored by: Citrix Systems R&D
Tested by: robak
MFC after: 1 week
PR: 191173
dev/xen/blkfront/blkfront.c:
- Add and announce support for unmapped IO.
Minor fixes to make the Xen Dom0 console work. This includes always
returning there's pending input in xencons_has_input, because on Dom0
there's no shared ring and we cannot test the indexes. The second
fix is to use the CONSOLEIO_read hypercall in order to read input
data from the Xen console.
Sponsored by: Citrix Systems R&D
dev/xen/console/xencons_ring.c:
- Always return true in xencons_has_input for Dom0.
- Implement Dom0 console support for xencons_handle_input.
Prevent the Xen and VirtIO balloon drivers from marking pages as
wired. This prevents them from increasing the system wired page count,
which can lead to mlock failing because of hitting the limit in
vm.max_wired.
In the Xen case make sure pages are zeroed before giving them back to
the hypervisor, or else we might be leaking data. Also remove the
balloon_{append/retrieve} and link pages directly into the
ballooned_pages queue using the plinks.q field in the page struct.
Sponsored by: Citrix Systems R&D
Reviewed by: kib, bryanv
Approved by: gibbs
dev/virtio/balloon/virtio_balloon.c:
- Don't allocate pages with VM_ALLOC_WIRED.
dev/xen/balloon/balloon.c:
- Don't allocate pages with VM_ALLOC_WIRED.
- Make sure pages are zeroed before giving them back to the
hypervisor.
- Remove the balloon_entry struct and the balloon_{append/retrieve}
functions and use the page plinks.q entry to link the pages
directly into the ballooned_pages queue.
the queue where to enqueue pages that are going to be unwired.
- Add stronger checks to the enqueue/dequeue for the pagequeues when
adding and removing pages to them.
Of course, for unmanaged pages the queue parameter of vm_page_unwire() will
be ignored, just as the active parameter today.
This makes adding new pagequeues quicker.
This change effectively modifies the KPI. __FreeBSD_version will be,
however, bumped just when the full cache of free pages will be
evicted.
Sponsored by: EMC / Isilon storage division
Reviewed by: alc
Tested by: pho
Switch the initialization of gnttab to use an unused physical memory
range for both PVHVM and PVH.
In the past PVHVM was using the xenpci BAR, but there's no reason to
do that, and in fact FreeBSD was probably doing it because it was the
way it was done in Windows, were drivers cannot probably request for
unused physical memory ranges, but it was never enforced in the
hypervisor.
Sponsored by: Citrix Systems R&D
Approved by: gibbs
xen/gnttab.c:
- Allocate contiguous physical memory for grant table frames for both
PVHVM and PVH.
- Since gnttab is not a device, use the xenpv device in order to
request for this allocation.
dev/xen/xenpci/xenpcivar.h:
dev/xen/xenpci/xenpci.c:
- Remove the now unused xenpci_alloc_space and xenpci_alloc_space_int
functions.
xen/gnttab.h:
- Change the prototype of gnttab_init and gnttab_resume, that now
takes a device_t parameter.
dev/xen/control/control.c:
x86/xen/xenpv.c:
- Changes to accomodate the new prototype of gnttab_init and
gnttab_resume.
Add the PV shutdown hook to PVH.
Sponsored by: Citrix Systems R&D
Approved by: gibbs
dev/xen/control/control.c:
- Make xen_pv_shutdown_final available on XENHVM builds.
- Register the Xen PV shutdown hook for PVH guests.
Create a dummy bus so top level Xen devices can attach to it (instead
of attaching directly to the nexus). This allows to have all the Xen
related devices grouped under a single bus.
Sponsored by: Citrix Systems R&D
Approved by: gibbs
x86/xen/xenpv.c:
- Attach the xenpv bus when running as a Xen guest.
- Attach the ISA bus if needed, in order to attach syscons.
conf/files.amd6:
conf/files.i386:
- Include the xenpv.c file in the build of i386/amd64 kernels using
XENHVM.
dev/xen/console/console.c:
dev/xen/timer/timer.c:
xen/xenstore/xenstore.c:
- Attach to the xenpv bus instead of the Nexus.
dev/xen/xenpci/xenpci.c:
- Xen specific devices on PVHVM guests are no longer attached to the
xenpci device, they are instead attached to the xenpv bus, remove
the now unused methods.
Replace usage of db_active in Xen console with kdb_active.
Reported by: Andrzej Tobola <ato@iem.pw.edu.pl>
Approved by: gibbs
Sponsored by: Citrix Systems R&D
When running as a PVH guest, there's no emulated i8254, so we need to
use the Xen PV timer as the early source for DELAY. This change allows
for different implementations of the early DELAY function and
implements a Xen variant for it.
Approved by: gibbs
Sponsored by: Citrix Systems R&D
dev/xen/timer/timer.c:
dev/xen/timer/timer.h:
- Implement Xen early delay functions using the PV timer and declare
them.
x86/include/init.h:
- Add hooks for early clock source initialization and early delay
functions.
i386/i386/machdep.c:
pc98/pc98/machdep.c:
amd64/amd64/machdep.c:
- Set early delay hooks to use the i8254 on bare metal.
- Use clock_init (that will in turn make use of init_ops) to
initialize the early clock source.
amd64/include/clock.h:
i386/include/clock.h:
- Declare i8254_delay and clock_init.
i386/xen/clock.c:
- Rename DELAY to i8254_delay.
x86/isa/clock.c:
- Introduce clock_init that will take care of initializing the early
clock by making use of the init_ops hooks.
- Move non ISA related delay functions to the newly introduced delay
file.
x86/x86/delay.c:
- Add moved delay related functions.
- Implement generic DELAY function that will use the init_ops hooks.
x86/xen/pv.c:
- Set PVH hooks for the early delay related functions in init_ops.
conf/files.amd64:
conf/files.i386:
conf/files.pc98:
- Add delay.c to the kernel build.
This should not introduce any functional change, and makes the
functions suitable to be called before we have actually mapped the
vcpu_info struct on a per-cpu basis.
Approved by: gibbs
Sponsored by: Citrix Systems R&D
dev/xen/timer/timer.c:
- Remove citrical_{enter/exit}, the clock code will already be called
with preemption disabled when needed. Add a comment to that regard
in xentimer_get_timecount.
- Allow xen_fetch_vcpu_time to be called with a specifc vcpu_info
that will be used to fetch current time.
- Assert that xentimer_et_start will always be called with preemption
disabled.
This adds and enables the PV console used on XEN kernels to
GENERIC/XENHVM kernels in order for it to be used on PVH.
Approved by: gibbs
Sponsored by: Citrix Systems R&D
dev/xen/console/console.c:
- Define console_page.
- Move xc_printf debug function from i386 XEN code to generic console
code.
- Rework xc_printf.
- Use xen_initial_domain instead of open-coded checks for Dom0.
- Gate the attach of the PV console to PV(H) guests.
dev/xen/console/xencons_ring.c:
- Allow the PV Xen console to output earlier by directly signaling
the event channel in start_info if the event channel is not yet
initialized.
- Use HYPERVISOR_start_info instead of xen_start_info.
i386/include/xen/xen-os.h:
- Remove prototype for xc_printf since it's now declared in global
xen-os.h
i386/xen/xen_machdep.c:
- Remove previous version of xc_printf.
- Remove definition of console_page (now it's defined in the console
itself).
- Fix some printf formatting errors.
x86/xen/pv.c:
- Add some early boot debug messages using xc_printf.
- Set console_page based on the value passed in start_info.
xen/xen-os.h:
- Declare console_page and add prototype for xc_printf.
In its stead use the Solaris / illumos approach of emulating '-' (dash)
in probe names with '__' (two consecutive underscores).
Reviewed by: markj
MFC after: 3 weeks
option, unbreak the lock tracing release semantic by embedding
calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined
version of the releasing functions for mutex, rwlock and sxlock.
Failing to do so skips the lockstat_probe_func invokation for
unlocking.
- As part of the LOCKSTAT support is inlined in mutex operation, for
kernel compiled without lock debugging options, potentially every
consumer must be compiled including opt_kdtrace.h.
Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the
dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES
is linked there and it is only used as a compile-time stub [0].
[0] immediately shows some new bug as DTRACE-derived support for debug
in sfxge is broken and it was never really tested. As it was not
including correctly opt_kdtrace.h before it was never enabled so it
was kept broken for a while. Fix this by using a protection stub,
leaving sfxge driver authors the responsibility for fixing it
appropriately [1].
Sponsored by: EMC / Isilon storage division
Discussed with: rstone
[0] Reported by: rstone
[1] Discussed with: philip
sys/dev/xen/balloon/balloon.c:
Remove unused and commented out code.
Fix deadlock caused by performing a sleepable malloc
while holding the balloon mutex.
Perform proper accounting of the memory used by the domain.
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
MFC after: 2 days
makes FreeBSD halt but not poweroff (as expected when issuing a
shutdown from the VM manager). Fix this by using the same handler
for both "halt" and "poweroff".
NB: The "halt" signal seems to be used on XenServer only. The OSS
Xen toolstack (xl) uses "poweroff" instead.
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
MFC after: 2 days
resist easy conversion since they implement a great deal of their attach
logic inside probe(). Some of this could be fixed by moving it to attach(),
but some requires something more subtle than BUS_PROBE_NOWILDCARD.
This shuld have been a problem since r230587. Not exactly sure why it
was not detected the last weeks with the tinderbox. I would assume
r255744 is what started to cause it.
MFC after: 1 week
prior releases.
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Approved by: re (gjb)
sys/dev/xen/blkfront/blkfront.c:
On XenServer versions up to an including 6.2, paravirtualized
CDROM support is broken. When running in an HVM domain,
ignore paravirtualized instances of CDROM media, and instead
rely on native drivers attaching to emulated hardware. This
functions correctly on all currently known Xen based
platforms.
field. Perform vcpu enumeration for Xen PV and HVM environments
and convert all Xen drivers to use vcpu_id instead of a hard coded
assumption of the mapping algorithm (acpi or apic ID) in use.
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Approved by: re (blanket Xen)
amd64/include/pcpu.h:
i386/include/pcpu.h:
Add vcpu_id to the amd64 and i386 pcpu structures.
dev/xen/timer/timer.c
x86/xen/xen_intr.c
Use new vcpu_id instead of assuming acpi_id == vcpu_id.
i386/xen/mp_machdep.c:
i386/xen/mptable.c
x86/xen/hvm.c:
Perform Xen HVM and Xen full PV vcpu_id mapping.
x86/xen/hvm.c:
x86/acpica/madt.c
Change SYSINIT ordering of acpi CPU enumeration so that it
is guaranteed to be available at the time of Xen HVM vcpu
id mapping.
Xen PVHVM guest.
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Approved by: re (blanket Xen)
MFC after: 2 weeks
sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
- Make sure that are no MMU related IPIs pending on migration.
- Reset pending IPI_BITMAP on resume.
- Init vcpu_info on resume.
sys/amd64/include/intr_machdep.h:
sys/i386/include/intr_machdep.h:
sys/x86/acpica/acpi_wakeup.c:
sys/x86/x86/intr_machdep.c:
sys/x86/isa/atpic.c:
sys/x86/x86/io_apic.c:
sys/x86/x86/local_apic.c:
- Add a "suspend_cancelled" parameter to pic_resume(). For the
Xen PIC, restoration of interrupt services differs between
the aborted suspend and normal resume cases, so we must provide
this information.
sys/dev/acpica/acpi_timer.c:
sys/dev/xen/timer/timer.c:
sys/timetc.h:
- Don't swap out "suspend safe" timers across a suspend/resume
cycle. This includes the Xen PV and ACPI timers.
sys/dev/xen/control/control.c:
- Perform proper suspend/resume process for PVHVM:
- Suspend all APs before going into suspension, this allows us
to reset the vcpu_info on resume for each AP.
- Reset shared info page and callback on resume.
sys/dev/xen/timer/timer.c:
- Implement suspend/resume support for the PV timer. Since FreeBSD
doesn't perform a per-cpu resume of the timer, we need to call
smp_rendezvous in order to correctly resume the timer on each CPU.
sys/dev/xen/xenpci/xenpci.c:
- Don't reset the PCI interrupt on each suspend/resume.
sys/kern/subr_smp.c:
- When suspending a PVHVM domain make sure there are no MMU IPIs
in-flight, or we will get a lockup on resume due to the fact that
pending event channels are not carried over on migration.
- Implement a generic version of restart_cpus that can be used by
suspended and stopped cpus.
sys/x86/xen/hvm.c:
- Implement resume support for the hypercall page and shared info.
- Clear vcpu_info so it can be reset by APs when resuming from
suspension.
sys/dev/xen/xenpci/xenpci.c:
sys/x86/xen/hvm.c:
sys/x86/xen/xen_intr.c:
- Support UP kernel configurations.
sys/x86/xen/xen_intr.c:
- Properly rebind per-cpus VIRQs and IPIs on resume.
Properly round-trip the "operation code" for client requests.
sys/dev/xen/blkback/blkback.c:
In xbb_dispatch_dev() when processing a flush request,
correctly set bio->bio_caller1 to the request list (not
bare request) for the operation, as is expected by the
completion handler xbb_bio_done().
In xbb_get_resources(), initialize "operation" in the
driver's internal request object from the client's "ring
request", so it is correct when used to populate the reply
when this operation completes.
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Initialize the request id for requests in xbb_get_resources()
instead of its previous location in xbb_dispatch_io(). This
guarantees that all request types (e.g. BLKIF_OP_FLUSH_DISKCACHE)
have the front-end specified id recorded.
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
value. Some hosts do not publish "extended" disk IDs via virtual-device in
an attempt to avoid confusing old blkfront drivers, and without this change
we failed to attach such disks.
In particular, this commit allows all 24 ephemeral disks on EC2 hs1.8xlarge
instances to be used, instead of only the first 15.
MFC after: 3 days
Use this new driver for both PV and HVM instances.
This driver requires a Xen hypervisor that supports vector callbacks,
VCPUOP hypercalls, and reports that it has a "safe PV clock".
New timer driver:
Submitted by: will
Sponsored by: Spectra Logic Corporation
PV port to new driver, and bug fixes:
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
sys/dev/xen/timer/timer.c:
- Register a PV timer device driver which (currently)
implements device_{identify,probe,attach} and stubs
device_detach. The detach routine requires functionality
not provided by timecounters(4). The suspend and resume
routines need additional work (due to Xen requiring that
the hypercalls be executed on the target VCPU), and aren't
needed for our purposes.
- Make sure there can only be one device instance of this
driver, and that it only registers one eventtimers(4) and
one timecounters(4) device interface. Make both interfaces
use PCPU data as needed.
- Match, with a few style cleanups & API differences, the
Xen versions of the "fetch time" functions.
- Document the magic scale_delta() better for the i386 version.
- When registering the event timer, bind a separate event
channel for the timer VIRQ to the device's event timer
interrupt handler for each active VCPU. Describe each
interrupt as "xen_et:c%d", so they can be identified per
CPU in "vmstat -i" or "show intrcnt" in KDB.
- When scheduling a timer into the hypervisor, try up to
60 times if the hypervisor rejects the time as being in
the past. In the common case, this retry shouldn't happen,
and if it does, it should only happen once. This is
because the event timer advertises a minimum period of
100usec, which is only less than the usual hypercall round
trip time about 1 out of every 100 tries. (Unlike other
similar drivers, this one actually checks whether the
hypervisor accepted the singleshot timer set hypercall.)
- Implement a RTC PV clock based on the hypervisor wallclock.
sys/conf/files:
- Add dev/xen/timer/timer.c if the kernel configuration
includes either the XEN or XENHVM options.
sys/conf/files.i386:
sys/i386/include/xen/xen_clock_util.h:
sys/i386/xen/clock.c:
sys/i386/xen/xen_clock_util.c:
sys/i386/xen/mp_machdep.c:
sys/i386/xen/xen_rtc.c:
- Remove previous PV timer used in i386 XEN PV kernels, the
new timer introduced in this change is used instead (so
we share the same code between PVHVM and PV).
MFC after: 2 weeks
Re-structure Xen HVM support so that:
- Xen is detected and hypercalls can be performed very
early in system startup.
- Xen interrupt services are implemented using FreeBSD's native
interrupt delivery infrastructure.
- the Xen interrupt service implementation is shared between PV
and HVM guests.
- Xen interrupt handlers can optionally use a filter handler
in order to avoid the overhead of dispatch to an interrupt
thread.
- interrupt load can be distributed among all available CPUs.
- the overhead of accessing the emulated local and I/O apics
on HVM is removed for event channel port events.
- a similar optimization can eventually, and fairly easily,
be used to optimize MSI.
Early Xen detection, HVM refactoring, PVHVM interrupt infrastructure,
and misc Xen cleanups:
Sponsored by: Spectra Logic Corporation
Unification of PV & HVM interrupt infrastructure, bug fixes,
and misc Xen cleanups:
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
sys/x86/x86/local_apic.c:
sys/amd64/include/apicvar.h:
sys/i386/include/apicvar.h:
sys/amd64/amd64/apic_vector.S:
sys/i386/i386/apic_vector.s:
sys/amd64/amd64/machdep.c:
sys/i386/i386/machdep.c:
sys/i386/xen/exception.s:
sys/x86/include/segments.h:
Reserve IDT vector 0x93 for the Xen event channel upcall
interrupt handler. On Hypervisors that support the direct
vector callback feature, we can request that this vector be
called directly by an injected HVM interrupt event, instead
of a simulated PCI interrupt on the Xen platform PCI device.
This avoids all of the overhead of dealing with the emulated
I/O APIC and local APIC. It also means that the Hypervisor
can inject these events on any CPU, allowing upcalls for
different ports to be handled in parallel.
sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
Map Xen per-vcpu area during AP startup.
sys/amd64/include/intr_machdep.h:
sys/i386/include/intr_machdep.h:
Increase the FreeBSD IRQ vector table to include space
for event channel interrupt sources.
sys/amd64/include/pcpu.h:
sys/i386/include/pcpu.h:
Remove Xen HVM per-cpu variable data. These fields are now
allocated via the dynamic per-cpu scheme. See xen_intr.c
for details.
sys/amd64/include/xen/hypercall.h:
sys/dev/xen/blkback/blkback.c:
sys/i386/include/xen/xenvar.h:
sys/i386/xen/clock.c:
sys/i386/xen/xen_machdep.c:
sys/xen/gnttab.c:
Prefer FreeBSD primatives to Linux ones in Xen support code.
sys/amd64/include/xen/xen-os.h:
sys/i386/include/xen/xen-os.h:
sys/xen/xen-os.h:
sys/dev/xen/balloon/balloon.c:
sys/dev/xen/blkback/blkback.c:
sys/dev/xen/blkfront/blkfront.c:
sys/dev/xen/console/xencons_ring.c:
sys/dev/xen/control/control.c:
sys/dev/xen/netback/netback.c:
sys/dev/xen/netfront/netfront.c:
sys/dev/xen/xenpci/xenpci.c:
sys/i386/i386/machdep.c:
sys/i386/include/pmap.h:
sys/i386/include/xen/xenfunc.h:
sys/i386/isa/npx.c:
sys/i386/xen/clock.c:
sys/i386/xen/mp_machdep.c:
sys/i386/xen/mptable.c:
sys/i386/xen/xen_clock_util.c:
sys/i386/xen/xen_machdep.c:
sys/i386/xen/xen_rtc.c:
sys/xen/evtchn/evtchn_dev.c:
sys/xen/features.c:
sys/xen/gnttab.c:
sys/xen/gnttab.h:
sys/xen/hvm.h:
sys/xen/xenbus/xenbus.c:
sys/xen/xenbus/xenbus_if.m:
sys/xen/xenbus/xenbusb_front.c:
sys/xen/xenbus/xenbusvar.h:
sys/xen/xenstore/xenstore.c:
sys/xen/xenstore/xenstore_dev.c:
sys/xen/xenstore/xenstorevar.h:
Pull common Xen OS support functions/settings into xen/xen-os.h.
sys/amd64/include/xen/xen-os.h:
sys/i386/include/xen/xen-os.h:
sys/xen/xen-os.h:
Remove constants, macros, and functions unused in FreeBSD's Xen
support.
sys/xen/xen-os.h:
sys/i386/xen/xen_machdep.c:
sys/x86/xen/hvm.c:
Introduce new functions xen_domain(), xen_pv_domain(), and
xen_hvm_domain(). These are used in favor of #ifdefs so that
FreeBSD can dynamically detect and adapt to the presence of
a hypervisor. The goal is to have an HVM optimized GENERIC,
but more is necessary before this is possible.
sys/amd64/amd64/machdep.c:
sys/dev/xen/xenpci/xenpcivar.h:
sys/dev/xen/xenpci/xenpci.c:
sys/x86/xen/hvm.c:
sys/sys/kernel.h:
Refactor magic ioport, Hypercall table and Hypervisor shared
information page setup, and move it to a dedicated HVM support
module.
HVM mode initialization is now triggered during the
SI_SUB_HYPERVISOR phase of system startup. This currently
occurs just after the kernel VM is fully setup which is
just enough infrastructure to allow the hypercall table
and shared info page to be properly mapped.
sys/xen/hvm.h:
sys/x86/xen/hvm.c:
Add definitions and a method for configuring Hypervisor event
delievery via a direct vector callback.
sys/amd64/include/xen/xen-os.h:
sys/x86/xen/hvm.c:
sys/conf/files:
sys/conf/files.amd64:
sys/conf/files.i386:
Adjust kernel build to reflect the refactoring of early
Xen startup code and Xen interrupt services.
sys/dev/xen/blkback/blkback.c:
sys/dev/xen/blkfront/blkfront.c:
sys/dev/xen/blkfront/block.h:
sys/dev/xen/control/control.c:
sys/dev/xen/evtchn/evtchn_dev.c:
sys/dev/xen/netback/netback.c:
sys/dev/xen/netfront/netfront.c:
sys/xen/xenstore/xenstore.c:
sys/xen/evtchn/evtchn_dev.c:
sys/dev/xen/console/console.c:
sys/dev/xen/console/xencons_ring.c
Adjust drivers to use new xen_intr_*() API.
sys/dev/xen/blkback/blkback.c:
Since blkback defers all event handling to a taskqueue,
convert this task queue to a "fast" taskqueue, and schedule
it via an interrupt filter. This avoids an unnecessary
ithread context switch.
sys/xen/xenstore/xenstore.c:
The xenstore driver is MPSAFE. Indicate as much when
registering its interrupt handler.
sys/xen/xenbus/xenbus.c:
sys/xen/xenbus/xenbusvar.h:
Remove unused event channel APIs.
sys/xen/evtchn.h:
Remove all kernel Xen interrupt service API definitions
from this file. It is now only used for structure and
ioctl definitions related to the event channel userland
device driver.
Update the definitions in this file to match those from
NetBSD. Implementing this interface will be necessary for
Dom0 support.
sys/xen/evtchn/evtchnvar.h:
Add a header file for implemenation internal APIs related
to managing event channels event delivery. This is used
to allow, for example, the event channel userland device
driver to access low-level routines that typical kernel
consumers of event channel services should never access.
sys/xen/interface/event_channel.h:
sys/xen/xen_intr.h:
Standardize on the evtchn_port_t type for referring to
an event channel port id. In order to prevent low-level
event channel APIs from leaking to kernel consumers who
should not have access to this data, the type is defined
twice: Once in the Xen provided event_channel.h, and again
in xen/xen_intr.h. The double declaration is protected by
__XEN_EVTCHN_PORT_DEFINED__ to ensure it is never declared
twice within a given compilation unit.
sys/xen/xen_intr.h:
sys/xen/evtchn/evtchn.c:
sys/x86/xen/xen_intr.c:
sys/dev/xen/xenpci/evtchn.c:
sys/dev/xen/xenpci/xenpcivar.h:
New implementation of Xen interrupt services. This is
similar in many respects to the i386 PV implementation with
the exception that events for bound to event channel ports
(i.e. not IPI, virtual IRQ, or physical IRQ) are further
optimized to avoid mask/unmask operations that aren't
necessary for these edge triggered events.
Stubs exist for supporting physical IRQ binding, but will
need additional work before this implementation can be
fully shared between PV and HVM.
sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
sys/i386/xen/mp_machdep.c
sys/x86/xen/hvm.c:
Add support for placing vcpu_info into an arbritary memory
page instead of using HYPERVISOR_shared_info->vcpu_info.
This allows the creation of domains with more than 32 vcpus.
sys/i386/i386/machdep.c:
sys/i386/xen/clock.c:
sys/i386/xen/xen_machdep.c:
sys/i386/xen/exception.s:
Add support for new event channle implementation.
transparent layering and better fragmentation.
- Normalize functions that allocate memory to use kmem_*
- Those that allocate address space are named kva_*
- Those that operate on maps are named kmap_*
- Implement recursive allocation handling for kmem_arena in vmem.
Reviewed by: alc
Tested by: pho
Sponsored by: EMC / Isilon Storage Division
support cache flush and write barrier commands.
sys/dev/xen/blkfront/block.h:
Add per-command flag that specifies that the I/O queue must
be frozen after this command is dispatched. This is used
to implement "single-stepping".
Remove the unused per-command flag that indicates a polled
command.
Add block device instance flags to record backend features.
Add a block device instance flag to indicate the I/O queue
is frozen until all outstanding I/O completes.
Enhance the queue API to allow the number of elements in a
queue to be interrogated.
Prefer "inline" to "__inline".
sys/dev/xen/blkfront/blkfront.c:
Formalize queue freeze semantics by adding methods for both
global and command-associated queue freezing.
Provide mechanism to freeze the I/O queue until all outstanding
I/O completes. Use this to implement barrier semantics
(BIO_ORDERED) when the backend does not support
BLKIF_OP_WRITE_BARRIER commands.
Implement BIO_FLUSH as either a BLKIF_OP_FLUSH_DISKCACHE
command or a 0 byte write barrier. Currently, all publicly
available backends perform a diskcache flush when processing
barrier commands, and this frontend behavior matches what
is done in Linux.
Simplify code by using new queue length API.
Report backend features during device attach and via sysctl.
Submitted by: Roger Pau Monné
Submitted by: gibbs (Merge with new driver queue API, sysctl support)
only re-enable I/O when all reasons have cleared.
sys/dev/xen/blkfront/block.h:
In the block front driver softc, replace the boolean
XBDF_FROZEN flag with a count of commands and driver global
issues that freeze the I/O queue. So long xbd_qfrozen_cnt
is non-zero, I/O is halted.
Add flags to xbd_flags for tracking grant table entry and
free command resource shortages. Each of these classes can
increment xbd_qfrozen_cnt at most once.
Add a command flag (XBDCF_ASYNC_MAPPING) that is set whenever
the initial mapping attempt of a command fails with EINPROGRESS.
sys/dev/xen/blkfront/blkfront.c:
In xbd_queue_cb(), use new XBDCF_ASYNC_MAPPING flag to definitively
know if an async bus dmamap load has occurred.
Add xbd_freeze() and xbd_thaw() helper methods for managing
xbd_qfrozen_cnt and use them to implement all queue freezing logic.
Add missing "thaw" to restart I/O processing once grant references
become available.
Sponsored by: Spectra Logic Corporation
scheme for defining inline command queuing functions.
Prefer enums to #defines.
sys/dev/xen/blkfront/block.h
Replace inline function generation performed by the
XBDQ_COMMAND_QUEUE() macro with single instances of each
inline function (init, enqueue, dequeue, remove). This was
made possible by using queue indexes instead of bit flags
in the command structure, and passing the index enum as
an argument to the functions.
Improve panic/assert messages in the queue functions.
Combine queue data and stats into a single data structure
and declare an array of them instead of each queue individually.
Convert command flags, softc state, and softc flags to enums.
sys/dev/xen/blkfront/blkfront.c
Mechanical adjustments for new queue api.
Sponsored by: Spectra Logic Corporation
MFC after: 1 week
In netif_free(), call ifmedia_removeall() after ether_ifdetach()
so that bpf listeners are detached, any link state processing
is completed, and there is no chance for external reference to media
information.
Suggested by: yongari
MFC after: 1 week
Xen host side can handle after defragmentation.
This prevents the driver from throwing away too long TSO chains and
improves the performance on Amazon AWS instances with 10GigE virtual
interfaces to the normally expected throughput.
Submitted by: cperciva (earlier version)
Reviewed by: cperciva
Tested by: cperciva
MFC after: 1 week
Remove local, and incorrect, definition for the value of an invalid
grant reference.
Extract ring cleanup code into xbd_free_ring() function for
symetry with xbd_alloc_ring(). This process also eliminated
an initialized but unused variable.
Sponsored by: Spectra Logic Corporation
MFC after: 1 week
o Group functions by by their functionality.
o Remove superfluous declarations.
o Remove more unused (#ifdef'd out) code.
Sponsored by: Spectra Logic Corporation
o This driver is the "xbd" driver, not the "blkfront", "blkif", "xbf", or
"xb" driver. Use the "xbd_" naming conventions for all functions,
structures, and constants.
o The prevailing convention for structure fields in this driver is to
prefix them with an abreviation of the structure type. Update
"recently added" fields to match this style.
o Remove unused data structures.
o Remove superfluous casts.
o Make a pass over the whole driver and bring it closer to
style(9) conformance.
Sponsored by: Spectra Logic Corporation
MFC after: 1 week
dev/xen/netfront:
In netif_free(), properly stop the interface and drain any pending
timers prior to disconnecting from the backend device.
Remove all media and detach our interface object from the system
prior to deleting it.
PR: kern/176471
Submitted by: Roger Pau Monne <roger.pau@citrix.com>
Reviewed by: gibbs
MFC after: 1 week
__func__ and add some missing ones.
- Remove a stale comment.
- Remove unused NUM_ELEMENTS macro.
- Remove extra empty lines.
- Use DEVMETHOD_END.
- Use NULL rather than 0 for pointers.
MFC after: 3 days
- Replace incorrect function names in printf(9) strings with __func__.
- Make xctrl_shutdown_reasons table const.
- Use nitems() rather than rolling an own version.
- Use DEVMETHOD_END.
- Use NULL rather than 0 for pointers.
MFC after: 3 days
In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.
The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.
Conducted and reviewed by: attilio
Tested by: pho
