ff662b5c98
o Add support for backend devices (e.g. blkback) o Implement extensions to the Xen para-virtualized block API to allow for larger and more outstanding I/Os. o Import a completely rewritten block back driver with support for fronting I/O to both raw devices and files. o General cleanup and documentation of the XenBus and XenStore support code. o Robustness and performance updates for the block front driver. o Fixes to the netfront driver. Sponsored by: Spectra Logic Corporation sys/xen/xenbus/init.txt: Deleted: This file explains the Linux method for XenBus device enumeration and thus does not apply to FreeBSD's NewBus approach. sys/xen/xenbus/xenbus_probe_backend.c: Deleted: Linux version of backend XenBus service routines. It was never ported to FreeBSD. See xenbusb.c, xenbusb_if.m, xenbusb_front.c xenbusb_back.c for details of FreeBSD's XenBus support. sys/xen/xenbus/xenbusvar.h: sys/xen/xenbus/xenbus_xs.c: sys/xen/xenbus/xenbus_comms.c: sys/xen/xenbus/xenbus_comms.h: sys/xen/xenstore/xenstorevar.h: sys/xen/xenstore/xenstore.c: Split XenStore into its own tree. XenBus is a software layer built on top of XenStore. The old arrangement and the naming of some structures and functions blurred these lines making it difficult to discern what services are provided by which layer and at what times these services are available (e.g. during system startup and shutdown). sys/xen/xenbus/xenbus_client.c: sys/xen/xenbus/xenbus.c: sys/xen/xenbus/xenbus_probe.c: sys/xen/xenbus/xenbusb.c: sys/xen/xenbus/xenbusb.h: Split up XenBus code into methods available for use by client drivers (xenbus.c) and code used by the XenBus "bus code" to enumerate, attach, detach, and service bus drivers. sys/xen/reboot.c: sys/dev/xen/control/control.c: Add a XenBus front driver for handling shutdown, reboot, suspend, and resume events published in the XenStore. Move all PV suspend/reboot support from reboot.c into this driver. sys/xen/blkif.h: New file from Xen vendor with macros and structures used by a block back driver to service requests from a VM running a different ABI (e.g. amd64 back with i386 front). sys/conf/files: Adjust kernel build spec for new XenBus/XenStore layout and added Xen functionality. sys/dev/xen/balloon/balloon.c: sys/dev/xen/netfront/netfront.c: sys/dev/xen/blkfront/blkfront.c: sys/xen/xenbus/... sys/xen/xenstore/... o Rename XenStore APIs and structures from xenbus_* to xs_*. o Adjust to use of M_XENBUS and M_XENSTORE malloc types for allocation of objects returned by these APIs. o Adjust for changes in the bus interface for Xen drivers. sys/xen/xenbus/... sys/xen/xenstore/... Add Doxygen comments for these interfaces and the code that implements them. sys/dev/xen/blkback/blkback.c: o Rewrite the Block Back driver to attach properly via newbus, operate correctly in both PV and HVM mode regardless of domain (e.g. can be in a DOM other than 0), and to deal with the latest metadata available in XenStore for block devices. o Allow users to specify a file as a backend to blkback, in addition to character devices. Use the namei lookup of the backend path to automatically configure, based on file type, the appropriate backend method. The current implementation is limited to a single outstanding I/O at a time to file backed storage. sys/dev/xen/blkback/blkback.c: sys/xen/interface/io/blkif.h: sys/xen/blkif.h: sys/dev/xen/blkfront/blkfront.c: sys/dev/xen/blkfront/block.h: Extend the Xen blkif API: Negotiable request size and number of requests. This change extends the information recorded in the XenStore allowing block front/back devices to negotiate for optimal I/O parameters. This has been achieved without sacrificing backward compatibility with drivers that are unaware of these protocol enhancements. The extensions center around the connection protocol which now includes these additions: o The back-end device publishes its maximum supported values for, request I/O size, the number of page segments that can be associated with a request, the maximum number of requests that can be concurrently active, and the maximum number of pages that can be in the shared request ring. These values are published before the back-end enters the XenbusStateInitWait state. o The front-end waits for the back-end to enter either the InitWait or Initialize state. At this point, the front end limits it's own capabilities to the lesser of the values it finds published by the backend, it's own maximums, or, should any back-end data be missing in the store, the values supported by the original protocol. It then initializes it's internal data structures including allocation of the shared ring, publishes its maximum capabilities to the XenStore and transitions to the Initialized state. o The back-end waits for the front-end to enter the Initalized state. At this point, the back end limits it's own capabilities to the lesser of the values it finds published by the frontend, it's own maximums, or, should any front-end data be missing in the store, the values supported by the original protocol. It then initializes it's internal data structures, attaches to the shared ring and transitions to the Connected state. o The front-end waits for the back-end to enter the Connnected state, transitions itself to the connected state, and can commence I/O. Although an updated front-end driver must be aware of the back-end's InitWait state, the back-end has been coded such that it can tolerate a front-end that skips this step and transitions directly to the Initialized state without waiting for the back-end. sys/xen/interface/io/blkif.h: o Increase BLKIF_MAX_SEGMENTS_PER_REQUEST to 255. This is the maximum number possible without changing the blkif request header structure (nr_segs is a uint8_t). o Add two new constants: BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK, and BLKIF_MAX_SEGMENTS_PER_SEGMENT_BLOCK. These respectively indicate the number of segments that can fit in the first ring-buffer entry of a request, and for each subsequent (sg element only) ring-buffer entry associated with the "header" ring-buffer entry of the request. o Add the blkif_request_segment_t typedef for segment elements. o Add the BLKRING_GET_SG_REQUEST() macro which wraps the RING_GET_REQUEST() macro and returns a properly cast pointer to an array of blkif_request_segment_ts. o Add the BLKIF_SEGS_TO_BLOCKS() macro which calculates the number of ring entries that will be consumed by a blkif request with the given number of segments. sys/xen/blkif.h: o Update for changes in interface/io/blkif.h macros. o Update the BLKIF_MAX_RING_REQUESTS() macro to take the ring size as an argument to allow this calculation on multi-page rings. o Add a companion macro to BLKIF_MAX_RING_REQUESTS(), BLKIF_RING_PAGES(). This macro determines the number of ring pages required in order to support a ring with the supplied number of request blocks. sys/dev/xen/blkback/blkback.c: sys/dev/xen/blkfront/blkfront.c: sys/dev/xen/blkfront/block.h: o Negotiate with the other-end with the following limits: Reqeust Size: MAXPHYS Max Segments: (MAXPHYS/PAGE_SIZE) + 1 Max Requests: 256 Max Ring Pages: Sufficient to support Max Requests with Max Segments. o Dynamically allocate request pools and segemnts-per-request. o Update ring allocation/attachment code to support a multi-page shared ring. o Update routines that access the shared ring to handle multi-block requests. sys/dev/xen/blkfront/blkfront.c: o Track blkfront allocations in a blkfront driver specific malloc pool. o Strip out XenStore transaction retry logic in the connection code. Transactions only need to be used when the update to multiple XenStore nodes must be atomic. That is not the case here. o Fully disable blkif_resume() until it can be fixed properly (it didn't work before this change). o Destroy bus-dma objects during device instance tear-down. o Properly handle backend devices with powef-of-2 sector sizes larger than 512b. sys/dev/xen/blkback/blkback.c: Advertise support for and implement the BLKIF_OP_WRITE_BARRIER and BLKIF_OP_FLUSH_DISKCACHE blkif opcodes using BIO_FLUSH and the BIO_ORDERED attribute of bios. sys/dev/xen/blkfront/blkfront.c: sys/dev/xen/blkfront/block.h: Fix various bugs in blkfront. o gnttab_alloc_grant_references() returns 0 for success and non-zero for failure. The check for < 0 is a leftover Linuxism. o When we negotiate with blkback and have to reduce some of our capabilities, print out the original and reduced capability before changing the local capability. So the user now gets the correct information. o Fix blkif_restart_queue_callback() formatting. Make sure we hold the mutex in that function before calling xb_startio(). o Fix a couple of KASSERT()s. o Fix a check in the xb_remove_* macro to be a little more specific. sys/xen/gnttab.h: sys/xen/gnttab.c: Define GNTTAB_LIST_END publicly as GRANT_REF_INVALID. sys/dev/xen/netfront/netfront.c: Use GRANT_REF_INVALID instead of driver private definitions of the same constant. sys/xen/gnttab.h: sys/xen/gnttab.c: Add the gnttab_end_foreign_access_references() API. This API allows a client to batch the release of an array of grant references, instead of coding a private for loop. The implementation takes advantage of this batching to reduce lock overhead to one acquisition and release per-batch instead of per-freed grant reference. While here, reduce the duration the gnttab_list_lock is held during gnttab_free_grant_references() operations. The search to find the tail of the incoming free list does not rely on global state and so can be performed without holding the lock. sys/dev/xen/xenpci/evtchn.c: sys/dev/xen/evtchn/evtchn.c: sys/xen/xen_intr.h: o Implement the bind_interdomain_evtchn_to_irqhandler API for HVM mode. This allows an HVM domain to serve back end devices to other domains. This API is already implemented for PV mode. o Synchronize the API between HVM and PV. sys/dev/xen/xenpci/xenpci.c: o Scan the full region of CPUID space in which the Xen VMM interface may be implemented. On systems using SuSE as a Dom0 where the Viridian API is also exported, the VMM interface is above the region we used to search. o Pass through bus_alloc_resource() calls so that XenBus drivers attaching on an HVM system can allocate unused physical address space from the nexus. The block back driver makes use of this facility. sys/i386/xen/xen_machdep.c: Use the correct type for accessing the statically mapped xenstore metadata. sys/xen/interface/hvm/params.h: sys/xen/xenstore/xenstore.c: Move hvm_get_parameter() to the correct global header file instead of as a private method to the XenStore. sys/xen/interface/io/protocols.h: Sync with vendor. sys/xeninterface/io/ring.h: Add macro for calculating the number of ring pages needed for an N deep ring. To avoid duplication within the macros, create and use the new __RING_HEADER_SIZE() macro. This macro calculates the size of the ring book keeping struct (producer/consumer indexes, etc.) that resides at the head of the ring. Add the __RING_PAGES() macro which calculates the number of shared ring pages required to support a ring with the given number of requests. These APIs are used to support the multi-page ring version of the Xen block API. sys/xeninterface/io/xenbus.h: Add Comments. sys/xen/xenbus/... o Refactor the FreeBSD XenBus support code to allow for both front and backend device attachments. o Make use of new config_intr_hook capabilities to allow front and back devices to be probed/attached in parallel. o Fix bugs in probe/attach state machine that could cause the system to hang when confronted with a failure either in the local domain or in a remote domain to which one of our driver instances is attaching. o Publish all required state to the XenStore on device detach and failure. The majority of the missing functionality was for serving as a back end since the typical "hot-plug" scripts in Dom0 don't handle the case of cleaning up for a "service domain" that is not itself. o Add dynamic sysctl nodes exposing the generic ivars of XenBus devices. o Add doxygen style comments to the majority of the code. o Cleanup types, formatting, etc. sys/xen/xenbus/xenbusb.c: Common code used by both front and back XenBus busses. sys/xen/xenbus/xenbusb_if.m: Method definitions for a XenBus bus. sys/xen/xenbus/xenbusb_front.c: sys/xen/xenbus/xenbusb_back.c: XenBus bus specialization for front and back devices. MFC after: 1 month
879 lines
22 KiB
C
879 lines
22 KiB
C
/******************************************************************************
|
|
* Copyright (C) 2010 Spectra Logic Corporation
|
|
* Copyright (C) 2008 Doug Rabson
|
|
* Copyright (C) 2005 Rusty Russell, IBM Corporation
|
|
* Copyright (C) 2005 Mike Wray, Hewlett-Packard
|
|
* Copyright (C) 2005 XenSource Ltd
|
|
*
|
|
* This file may be distributed separately from the Linux kernel, or
|
|
* incorporated into other software packages, subject to the following license:
|
|
*
|
|
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
* of this source file (the "Software"), to deal in the Software without
|
|
* restriction, including without limitation the rights to use, copy, modify,
|
|
* merge, publish, distribute, sublicense, and/or sell copies of the Software,
|
|
* and to permit persons to whom the Software is furnished to do so, subject to
|
|
* the following conditions:
|
|
*
|
|
* The above copyright notice and this permission notice shall be included in
|
|
* all copies or substantial portions of the Software.
|
|
*
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
|
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
|
|
* IN THE SOFTWARE.
|
|
*/
|
|
|
|
/**
|
|
* \file xenbusb.c
|
|
*
|
|
* \brief Shared support functions for managing the NewBus busses that contain
|
|
* Xen front and back end device instances.
|
|
*
|
|
* The NewBus implementation of XenBus attaches a xenbusb_front and xenbusb_back
|
|
* child bus to the xenstore device. This strategy allows the small differences
|
|
* in the handling of XenBus operations for front and back devices to be handled
|
|
* as overrides in xenbusb_front/back.c. Front and back specific device
|
|
* classes are also provided so device drivers can register for the devices they
|
|
* can handle without the need to filter within their probe routines. The
|
|
* net result is a device hierarchy that might look like this:
|
|
*
|
|
* xenstore0/
|
|
* xenbusb_front0/
|
|
* xn0
|
|
* xbd0
|
|
* xbd1
|
|
* xenbusb_back0/
|
|
* xbbd0
|
|
* xnb0
|
|
* xnb1
|
|
*/
|
|
#include <sys/cdefs.h>
|
|
__FBSDID("$FreeBSD$");
|
|
|
|
#include <sys/param.h>
|
|
#include <sys/bus.h>
|
|
#include <sys/kernel.h>
|
|
#include <sys/lock.h>
|
|
#include <sys/malloc.h>
|
|
#include <sys/module.h>
|
|
#include <sys/sbuf.h>
|
|
#include <sys/sysctl.h>
|
|
#include <sys/syslog.h>
|
|
#include <sys/systm.h>
|
|
#include <sys/sx.h>
|
|
#include <sys/taskqueue.h>
|
|
|
|
#include <machine/xen/xen-os.h>
|
|
#include <machine/stdarg.h>
|
|
|
|
#include <xen/gnttab.h>
|
|
#include <xen/xenstore/xenstorevar.h>
|
|
#include <xen/xenbus/xenbusb.h>
|
|
#include <xen/xenbus/xenbusvar.h>
|
|
|
|
/*------------------------- Private Functions --------------------------------*/
|
|
/**
|
|
* \brief Deallocate XenBus device instance variables.
|
|
*
|
|
* \param ivars The instance variable block to free.
|
|
*/
|
|
static void
|
|
xenbusb_free_child_ivars(struct xenbus_device_ivars *ivars)
|
|
{
|
|
if (ivars->xd_otherend_watch.node != NULL) {
|
|
xs_unregister_watch(&ivars->xd_otherend_watch);
|
|
free(ivars->xd_otherend_watch.node, M_XENBUS);
|
|
ivars->xd_otherend_watch.node = NULL;
|
|
}
|
|
|
|
if (ivars->xd_node != NULL) {
|
|
free(ivars->xd_node, M_XENBUS);
|
|
ivars->xd_node = NULL;
|
|
}
|
|
|
|
if (ivars->xd_type != NULL) {
|
|
free(ivars->xd_type, M_XENBUS);
|
|
ivars->xd_type = NULL;
|
|
}
|
|
|
|
if (ivars->xd_otherend_path != NULL) {
|
|
free(ivars->xd_otherend_path, M_XENBUS);
|
|
ivars->xd_otherend_path = NULL;
|
|
}
|
|
|
|
free(ivars, M_XENBUS);
|
|
}
|
|
|
|
/**
|
|
* XenBus watch callback registered against the "state" XenStore
|
|
* node of the other-end of a split device connection.
|
|
*
|
|
* This callback is invoked whenever the state of a device instance's
|
|
* peer changes.
|
|
*
|
|
* \param watch The xs_watch object used to register this callback
|
|
* function.
|
|
* \param vec An array of pointers to NUL terminated strings containing
|
|
* watch event data. The vector should be indexed via the
|
|
* xs_watch_type enum in xs_wire.h.
|
|
* \param vec_size The number of elements in vec.
|
|
*
|
|
* \return The device_t of the found device if any, or NULL.
|
|
*
|
|
* \note device_t is a pointer type, so it can be compared against
|
|
* NULL for validity.
|
|
*/
|
|
static void
|
|
xenbusb_otherend_changed(struct xs_watch *watch, const char **vec,
|
|
unsigned int vec_size __unused)
|
|
{
|
|
struct xenbus_device_ivars *ivars;
|
|
device_t dev;
|
|
enum xenbus_state newstate;
|
|
|
|
ivars = (struct xenbus_device_ivars *) watch;
|
|
dev = ivars->xd_dev;
|
|
|
|
if (!ivars->xd_otherend_path
|
|
|| strncmp(ivars->xd_otherend_path, vec[XS_WATCH_PATH],
|
|
strlen(ivars->xd_otherend_path)))
|
|
return;
|
|
|
|
newstate = xenbus_read_driver_state(ivars->xd_otherend_path);
|
|
XENBUS_OTHEREND_CHANGED(dev, newstate);
|
|
}
|
|
|
|
/**
|
|
* Search our internal record of configured devices (not the XenStore)
|
|
* to determine if the XenBus device indicated by \a node is known to
|
|
* the system.
|
|
*
|
|
* \param dev The XenBus bus instance to search for device children.
|
|
* \param node The XenStore node path for the device to find.
|
|
*
|
|
* \return The device_t of the found device if any, or NULL.
|
|
*
|
|
* \note device_t is a pointer type, so it can be compared against
|
|
* NULL for validity.
|
|
*/
|
|
static device_t
|
|
xenbusb_device_exists(device_t dev, const char *node)
|
|
{
|
|
device_t *kids;
|
|
device_t result;
|
|
struct xenbus_device_ivars *ivars;
|
|
int i, count;
|
|
|
|
if (device_get_children(dev, &kids, &count))
|
|
return (FALSE);
|
|
|
|
result = NULL;
|
|
for (i = 0; i < count; i++) {
|
|
ivars = device_get_ivars(kids[i]);
|
|
if (!strcmp(ivars->xd_node, node)) {
|
|
result = kids[i];
|
|
break;
|
|
}
|
|
}
|
|
free(kids, M_TEMP);
|
|
|
|
return (result);
|
|
}
|
|
|
|
static void
|
|
xenbusb_delete_child(device_t dev, device_t child)
|
|
{
|
|
struct xenbus_device_ivars *ivars;
|
|
|
|
ivars = device_get_ivars(child);
|
|
|
|
/*
|
|
* We no longer care about the otherend of the
|
|
* connection. Cancel the watch now so that we
|
|
* don't try to handle an event for a partially
|
|
* detached child.
|
|
*/
|
|
if (ivars->xd_otherend_watch.node != NULL)
|
|
xs_unregister_watch(&ivars->xd_otherend_watch);
|
|
|
|
device_delete_child(dev, child);
|
|
xenbusb_free_child_ivars(ivars);
|
|
}
|
|
|
|
/**
|
|
* \param dev The NewBus device representing this XenBus bus.
|
|
* \param child The NewBus device representing a child of dev%'s XenBus bus.
|
|
*/
|
|
static void
|
|
xenbusb_verify_device(device_t dev, device_t child)
|
|
{
|
|
if (xs_exists(XST_NIL, xenbus_get_node(child), "") == 0) {
|
|
|
|
/*
|
|
* Device tree has been removed from Xenbus.
|
|
* Tear down the device.
|
|
*/
|
|
xenbusb_delete_child(dev, child);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* \brief Enumerate the devices on a XenBus bus and register them with
|
|
* the NewBus device tree.
|
|
*
|
|
* xenbusb_enumerate_bus() will create entries (in state DS_NOTPRESENT)
|
|
* for nodes that appear in the XenStore, but will not invoke probe/attach
|
|
* operations on drivers. Probe/Attach processing must be separately
|
|
* performed via an invocation of xenbusb_probe_children(). This is usually
|
|
* done via the xbs_probe_children task.
|
|
*
|
|
* \param xbs XenBus Bus device softc of the owner of the bus to enumerate.
|
|
*
|
|
* \return On success, 0. Otherwise an errno value indicating the
|
|
* type of failure.
|
|
*/
|
|
static int
|
|
xenbusb_enumerate_bus(struct xenbusb_softc *xbs)
|
|
{
|
|
const char **types;
|
|
u_int type_idx;
|
|
u_int type_count;
|
|
int error;
|
|
|
|
error = xs_directory(XST_NIL, xbs->xbs_node, "", &type_count, &types);
|
|
if (error)
|
|
return (error);
|
|
|
|
for (type_idx = 0; type_idx < type_count; type_idx++)
|
|
XENBUSB_ENUMERATE_TYPE(xbs->xbs_dev, types[type_idx]);
|
|
|
|
free(types, M_XENSTORE);
|
|
|
|
return (0);
|
|
}
|
|
|
|
/**
|
|
* Handler for all generic XenBus device systcl nodes.
|
|
*/
|
|
static int
|
|
xenbusb_device_sysctl_handler(SYSCTL_HANDLER_ARGS)
|
|
{
|
|
device_t dev;
|
|
const char *value;
|
|
|
|
dev = (device_t)arg1;
|
|
switch (arg2) {
|
|
case XENBUS_IVAR_NODE:
|
|
value = xenbus_get_node(dev);
|
|
break;
|
|
case XENBUS_IVAR_TYPE:
|
|
value = xenbus_get_type(dev);
|
|
break;
|
|
case XENBUS_IVAR_STATE:
|
|
value = xenbus_strstate(xenbus_get_state(dev));
|
|
break;
|
|
case XENBUS_IVAR_OTHEREND_ID:
|
|
return (sysctl_handle_int(oidp, NULL,
|
|
xenbus_get_otherend_id(dev),
|
|
req));
|
|
/* NOTREACHED */
|
|
case XENBUS_IVAR_OTHEREND_PATH:
|
|
value = xenbus_get_otherend_path(dev);
|
|
break;
|
|
default:
|
|
return (EINVAL);
|
|
}
|
|
return (SYSCTL_OUT(req, value, strlen(value)));
|
|
}
|
|
|
|
/**
|
|
* Create read-only systcl nodes for xenbusb device ivar data.
|
|
*
|
|
* \param dev The XenBus device instance to register with sysctl.
|
|
*/
|
|
static void
|
|
xenbusb_device_sysctl_init(device_t dev)
|
|
{
|
|
struct sysctl_ctx_list *ctx;
|
|
struct sysctl_oid *tree;
|
|
|
|
ctx = device_get_sysctl_ctx(dev);
|
|
tree = device_get_sysctl_tree(dev);
|
|
|
|
SYSCTL_ADD_PROC(ctx,
|
|
SYSCTL_CHILDREN(tree),
|
|
OID_AUTO,
|
|
"xenstore_path",
|
|
CTLFLAG_RD,
|
|
dev,
|
|
XENBUS_IVAR_NODE,
|
|
xenbusb_device_sysctl_handler,
|
|
"A",
|
|
"XenStore path to device");
|
|
|
|
SYSCTL_ADD_PROC(ctx,
|
|
SYSCTL_CHILDREN(tree),
|
|
OID_AUTO,
|
|
"xenbus_dev_type",
|
|
CTLFLAG_RD,
|
|
dev,
|
|
XENBUS_IVAR_TYPE,
|
|
xenbusb_device_sysctl_handler,
|
|
"A",
|
|
"XenBus device type");
|
|
|
|
SYSCTL_ADD_PROC(ctx,
|
|
SYSCTL_CHILDREN(tree),
|
|
OID_AUTO,
|
|
"xenbus_connection_state",
|
|
CTLFLAG_RD,
|
|
dev,
|
|
XENBUS_IVAR_STATE,
|
|
xenbusb_device_sysctl_handler,
|
|
"A",
|
|
"XenBus state of peer connection");
|
|
|
|
SYSCTL_ADD_PROC(ctx,
|
|
SYSCTL_CHILDREN(tree),
|
|
OID_AUTO,
|
|
"xenbus_peer_domid",
|
|
CTLFLAG_RD,
|
|
dev,
|
|
XENBUS_IVAR_OTHEREND_ID,
|
|
xenbusb_device_sysctl_handler,
|
|
"I",
|
|
"Xen domain ID of peer");
|
|
|
|
SYSCTL_ADD_PROC(ctx,
|
|
SYSCTL_CHILDREN(tree),
|
|
OID_AUTO,
|
|
"xenstore_peer_path",
|
|
CTLFLAG_RD,
|
|
dev,
|
|
XENBUS_IVAR_OTHEREND_PATH,
|
|
xenbusb_device_sysctl_handler,
|
|
"A",
|
|
"XenStore path to peer device");
|
|
}
|
|
|
|
/**
|
|
* \brief Verify the existance of attached device instances and perform
|
|
* probe/attach processing for newly arrived devices.
|
|
*
|
|
* \param dev The NewBus device representing this XenBus bus.
|
|
*
|
|
* \return On success, 0. Otherwise an errno value indicating the
|
|
* type of failure.
|
|
*/
|
|
static int
|
|
xenbusb_probe_children(device_t dev)
|
|
{
|
|
device_t *kids;
|
|
struct xenbus_device_ivars *ivars;
|
|
int i, count;
|
|
|
|
if (device_get_children(dev, &kids, &count) == 0) {
|
|
for (i = 0; i < count; i++) {
|
|
if (device_get_state(kids[i]) != DS_NOTPRESENT) {
|
|
/*
|
|
* We already know about this one.
|
|
* Make sure it's still here.
|
|
*/
|
|
xenbusb_verify_device(dev, kids[i]);
|
|
continue;
|
|
}
|
|
|
|
if (device_probe_and_attach(kids[i])) {
|
|
/*
|
|
* Transition device to the closed state
|
|
* so the world knows that attachment will
|
|
* not occur.
|
|
*/
|
|
xenbus_set_state(kids[i], XenbusStateClosed);
|
|
|
|
/*
|
|
* Remove our record of this device.
|
|
* So long as it remains in the closed
|
|
* state in the XenStore, we will not find
|
|
* it again. The state will only change
|
|
* if the control domain actively reconfigures
|
|
* this device.
|
|
*/
|
|
xenbusb_delete_child(dev, kids[i]);
|
|
|
|
continue;
|
|
}
|
|
/*
|
|
* Augment default newbus provided dynamic sysctl
|
|
* variables with the standard ivar contents of
|
|
* XenBus devices.
|
|
*/
|
|
xenbusb_device_sysctl_init(kids[i]);
|
|
|
|
/*
|
|
* Now that we have a driver managing this device
|
|
* that can receive otherend state change events,
|
|
* hook up a watch for them.
|
|
*/
|
|
ivars = device_get_ivars(kids[i]);
|
|
xs_register_watch(&ivars->xd_otherend_watch);
|
|
}
|
|
free(kids, M_TEMP);
|
|
}
|
|
|
|
return (0);
|
|
}
|
|
|
|
/**
|
|
* \brief Task callback function to perform XenBus probe operations
|
|
* from a known safe context.
|
|
*
|
|
* \param arg The NewBus device_t representing the bus instance to
|
|
* on which to perform probe processing.
|
|
* \param pending The number of times this task was queued before it could
|
|
* be run.
|
|
*/
|
|
static void
|
|
xenbusb_probe_children_cb(void *arg, int pending __unused)
|
|
{
|
|
device_t dev = (device_t)arg;
|
|
|
|
/*
|
|
* Hold Giant until the Giant free newbus changes are committed.
|
|
*/
|
|
mtx_lock(&Giant);
|
|
xenbusb_probe_children(dev);
|
|
mtx_unlock(&Giant);
|
|
}
|
|
|
|
/**
|
|
* \brief XenStore watch callback for the root node of the XenStore
|
|
* subtree representing a XenBus.
|
|
*
|
|
* This callback performs, or delegates to the xbs_probe_children task,
|
|
* all processing necessary to handle dynmaic device arrival and departure
|
|
* events from a XenBus.
|
|
*
|
|
* \param watch The XenStore watch object associated with this callback.
|
|
* \param vec The XenStore watch event data.
|
|
* \param len The number of fields in the event data stream.
|
|
*/
|
|
static void
|
|
xenbusb_devices_changed(struct xs_watch *watch, const char **vec,
|
|
unsigned int len)
|
|
{
|
|
struct xenbusb_softc *xbs;
|
|
device_t dev;
|
|
char *node;
|
|
char *bus;
|
|
char *type;
|
|
char *id;
|
|
char *p;
|
|
u_int component;
|
|
|
|
xbs = (struct xenbusb_softc *)watch;
|
|
dev = xbs->xbs_dev;
|
|
|
|
if (len <= XS_WATCH_PATH) {
|
|
device_printf(dev, "xenbusb_devices_changed: "
|
|
"Short Event Data.\n");
|
|
return;
|
|
}
|
|
|
|
node = strdup(vec[XS_WATCH_PATH], M_XENBUS);
|
|
p = strchr(node, '/');
|
|
if (p == NULL)
|
|
goto out;
|
|
bus = node;
|
|
*p = 0;
|
|
type = p + 1;
|
|
|
|
p = strchr(type, '/');
|
|
if (p == NULL)
|
|
goto out;
|
|
*p++ = 0;
|
|
|
|
/*
|
|
* Extract the device ID. A device ID has one or more path
|
|
* components separated by the '/' character.
|
|
*
|
|
* e.g. "<frontend vm id>/<frontend dev id>" for backend devices.
|
|
*/
|
|
id = p;
|
|
for (component = 0; component < xbs->xbs_id_components; component++) {
|
|
p = strchr(p, '/');
|
|
if (p == NULL)
|
|
break;
|
|
p++;
|
|
}
|
|
if (p != NULL)
|
|
*p = 0;
|
|
|
|
if (*id != 0 && component >= xbs->xbs_id_components - 1) {
|
|
xenbusb_add_device(xbs->xbs_dev, type, id);
|
|
taskqueue_enqueue(taskqueue_thread, &xbs->xbs_probe_children);
|
|
}
|
|
out:
|
|
free(node, M_XENBUS);
|
|
}
|
|
|
|
/**
|
|
* \brief Interrupt configuration hook callback associated with xbs_attch_ch.
|
|
*
|
|
* Since interrupts are always functional at the time of XenBus configuration,
|
|
* there is nothing to be done when the callback occurs. This hook is only
|
|
* registered to hold up boot processing while XenBus devices come online.
|
|
*
|
|
* \param arg Unused configuration hook callback argument.
|
|
*/
|
|
static void
|
|
xenbusb_nop_confighook_cb(void *arg __unused)
|
|
{
|
|
}
|
|
|
|
/**
|
|
* \brief Decrement the number of XenBus child devices in the
|
|
* connecting state by one and release the xbs_attch_ch
|
|
* interrupt configuration hook if the connecting count
|
|
* drops to zero.
|
|
*
|
|
* \param xbs XenBus Bus device softc of the owner of the bus to enumerate.
|
|
*/
|
|
static void
|
|
xenbusb_release_confighook(struct xenbusb_softc *xbs)
|
|
{
|
|
mtx_lock(&xbs->xbs_lock);
|
|
KASSERT(xbs->xbs_connecting_children > 0,
|
|
("Connecting device count error\n"));
|
|
xbs->xbs_connecting_children--;
|
|
if (xbs->xbs_connecting_children == 0
|
|
&& (xbs->xbs_flags & XBS_ATTACH_CH_ACTIVE) != 0) {
|
|
xbs->xbs_flags &= ~XBS_ATTACH_CH_ACTIVE;
|
|
mtx_unlock(&xbs->xbs_lock);
|
|
config_intrhook_disestablish(&xbs->xbs_attach_ch);
|
|
} else {
|
|
mtx_unlock(&xbs->xbs_lock);
|
|
}
|
|
}
|
|
|
|
/*--------------------------- Public Functions -------------------------------*/
|
|
/*--------- API comments for these methods can be found in xenbusb.h ---------*/
|
|
void
|
|
xenbusb_identify(driver_t *driver __unused, device_t parent)
|
|
{
|
|
/*
|
|
* A single instance of each bus type for which we have a driver
|
|
* is always present in a system operating under Xen.
|
|
*/
|
|
BUS_ADD_CHILD(parent, 0, driver->name, 0);
|
|
}
|
|
|
|
int
|
|
xenbusb_add_device(device_t dev, const char *type, const char *id)
|
|
{
|
|
struct xenbusb_softc *xbs;
|
|
struct sbuf *devpath_sbuf;
|
|
char *devpath;
|
|
struct xenbus_device_ivars *ivars;
|
|
int error;
|
|
|
|
xbs = device_get_softc(dev);
|
|
devpath_sbuf = sbuf_new_auto();
|
|
sbuf_printf(devpath_sbuf, "%s/%s/%s", xbs->xbs_node, type, id);
|
|
sbuf_finish(devpath_sbuf);
|
|
devpath = sbuf_data(devpath_sbuf);
|
|
|
|
ivars = malloc(sizeof(*ivars), M_XENBUS, M_ZERO|M_WAITOK);
|
|
error = ENXIO;
|
|
|
|
if (xs_exists(XST_NIL, devpath, "") != 0) {
|
|
device_t child;
|
|
enum xenbus_state state;
|
|
char *statepath;
|
|
|
|
child = xenbusb_device_exists(dev, devpath);
|
|
if (child != NULL) {
|
|
/*
|
|
* We are already tracking this node
|
|
*/
|
|
error = 0;
|
|
goto out;
|
|
}
|
|
|
|
state = xenbus_read_driver_state(devpath);
|
|
if (state != XenbusStateInitialising) {
|
|
/*
|
|
* Device is not new, so ignore it. This can
|
|
* happen if a device is going away after
|
|
* switching to Closed.
|
|
*/
|
|
printf("xenbusb_add_device: Device %s ignored. "
|
|
"State %d\n", devpath, state);
|
|
error = 0;
|
|
goto out;
|
|
}
|
|
|
|
sx_init(&ivars->xd_lock, "xdlock");
|
|
ivars->xd_flags = XDF_CONNECTING;
|
|
ivars->xd_node = strdup(devpath, M_XENBUS);
|
|
ivars->xd_type = strdup(type, M_XENBUS);
|
|
ivars->xd_state = XenbusStateInitialising;
|
|
|
|
error = XENBUSB_GET_OTHEREND_NODE(dev, ivars);
|
|
if (error) {
|
|
printf("xenbus_update_device: %s no otherend id\n",
|
|
devpath);
|
|
goto out;
|
|
}
|
|
|
|
statepath = malloc(strlen(ivars->xd_otherend_path)
|
|
+ strlen("/state") + 1, M_XENBUS, M_WAITOK);
|
|
sprintf(statepath, "%s/state", ivars->xd_otherend_path);
|
|
|
|
ivars->xd_otherend_watch.node = statepath;
|
|
ivars->xd_otherend_watch.callback = xenbusb_otherend_changed;
|
|
|
|
mtx_lock(&xbs->xbs_lock);
|
|
xbs->xbs_connecting_children++;
|
|
mtx_unlock(&xbs->xbs_lock);
|
|
|
|
child = device_add_child(dev, NULL, -1);
|
|
ivars->xd_dev = child;
|
|
device_set_ivars(child, ivars);
|
|
}
|
|
|
|
out:
|
|
sbuf_delete(devpath_sbuf);
|
|
if (error != 0)
|
|
xenbusb_free_child_ivars(ivars);
|
|
|
|
return (error);
|
|
}
|
|
|
|
int
|
|
xenbusb_attach(device_t dev, char *bus_node, u_int id_components)
|
|
{
|
|
struct xenbusb_softc *xbs;
|
|
|
|
xbs = device_get_softc(dev);
|
|
mtx_init(&xbs->xbs_lock, "xenbusb softc lock", NULL, MTX_DEF);
|
|
xbs->xbs_node = bus_node;
|
|
xbs->xbs_id_components = id_components;
|
|
xbs->xbs_dev = dev;
|
|
|
|
/*
|
|
* Since XenBus busses are attached to the XenStore, and
|
|
* the XenStore does not probe children until after interrupt
|
|
* services are available, this config hook is used solely
|
|
* to ensure that the remainder of the boot process (e.g.
|
|
* mount root) is deferred until child devices are adequately
|
|
* probed. We unblock the boot process as soon as the
|
|
* connecting child count in our softc goes to 0.
|
|
*/
|
|
xbs->xbs_attach_ch.ich_func = xenbusb_nop_confighook_cb;
|
|
xbs->xbs_attach_ch.ich_arg = dev;
|
|
config_intrhook_establish(&xbs->xbs_attach_ch);
|
|
xbs->xbs_flags |= XBS_ATTACH_CH_ACTIVE;
|
|
xbs->xbs_connecting_children = 1;
|
|
|
|
/*
|
|
* The subtree for this bus type may not yet exist
|
|
* causing initial enumeration to fail. We still
|
|
* want to return success from our attach though
|
|
* so that we are ready to handle devices for this
|
|
* bus when they are dynamically attached to us
|
|
* by a Xen management action.
|
|
*/
|
|
(void)xenbusb_enumerate_bus(xbs);
|
|
xenbusb_probe_children(dev);
|
|
|
|
xbs->xbs_device_watch.node = bus_node;
|
|
xbs->xbs_device_watch.callback = xenbusb_devices_changed;
|
|
|
|
TASK_INIT(&xbs->xbs_probe_children, 0, xenbusb_probe_children_cb, dev);
|
|
|
|
xs_register_watch(&xbs->xbs_device_watch);
|
|
|
|
xenbusb_release_confighook(xbs);
|
|
|
|
return (0);
|
|
}
|
|
|
|
int
|
|
xenbusb_resume(device_t dev)
|
|
{
|
|
device_t *kids;
|
|
struct xenbus_device_ivars *ivars;
|
|
int i, count, error;
|
|
char *statepath;
|
|
|
|
/*
|
|
* We must re-examine each device and find the new path for
|
|
* its backend.
|
|
*/
|
|
if (device_get_children(dev, &kids, &count) == 0) {
|
|
for (i = 0; i < count; i++) {
|
|
if (device_get_state(kids[i]) == DS_NOTPRESENT)
|
|
continue;
|
|
|
|
ivars = device_get_ivars(kids[i]);
|
|
|
|
xs_unregister_watch(&ivars->xd_otherend_watch);
|
|
ivars->xd_state = XenbusStateInitialising;
|
|
|
|
/*
|
|
* Find the new backend details and
|
|
* re-register our watch.
|
|
*/
|
|
error = XENBUSB_GET_OTHEREND_NODE(dev, ivars);
|
|
if (error)
|
|
return (error);
|
|
|
|
DEVICE_RESUME(kids[i]);
|
|
|
|
statepath = malloc(strlen(ivars->xd_otherend_path)
|
|
+ strlen("/state") + 1, M_XENBUS, M_WAITOK);
|
|
sprintf(statepath, "%s/state", ivars->xd_otherend_path);
|
|
|
|
free(ivars->xd_otherend_watch.node, M_XENBUS);
|
|
ivars->xd_otherend_watch.node = statepath;
|
|
xs_register_watch(&ivars->xd_otherend_watch);
|
|
|
|
#if 0
|
|
/*
|
|
* Can't do this yet since we are running in
|
|
* the xenwatch thread and if we sleep here,
|
|
* we will stop delivering watch notifications
|
|
* and the device will never come back online.
|
|
*/
|
|
sx_xlock(&ivars->xd_lock);
|
|
while (ivars->xd_state != XenbusStateClosed
|
|
&& ivars->xd_state != XenbusStateConnected)
|
|
sx_sleep(&ivars->xd_state, &ivars->xd_lock,
|
|
0, "xdresume", 0);
|
|
sx_xunlock(&ivars->xd_lock);
|
|
#endif
|
|
}
|
|
free(kids, M_TEMP);
|
|
}
|
|
|
|
return (0);
|
|
}
|
|
|
|
int
|
|
xenbusb_print_child(device_t dev, device_t child)
|
|
{
|
|
struct xenbus_device_ivars *ivars = device_get_ivars(child);
|
|
int retval = 0;
|
|
|
|
retval += bus_print_child_header(dev, child);
|
|
retval += printf(" at %s", ivars->xd_node);
|
|
retval += bus_print_child_footer(dev, child);
|
|
|
|
return (retval);
|
|
}
|
|
|
|
int
|
|
xenbusb_read_ivar(device_t dev, device_t child, int index, uintptr_t *result)
|
|
{
|
|
struct xenbus_device_ivars *ivars = device_get_ivars(child);
|
|
|
|
switch (index) {
|
|
case XENBUS_IVAR_NODE:
|
|
*result = (uintptr_t) ivars->xd_node;
|
|
return (0);
|
|
|
|
case XENBUS_IVAR_TYPE:
|
|
*result = (uintptr_t) ivars->xd_type;
|
|
return (0);
|
|
|
|
case XENBUS_IVAR_STATE:
|
|
*result = (uintptr_t) ivars->xd_state;
|
|
return (0);
|
|
|
|
case XENBUS_IVAR_OTHEREND_ID:
|
|
*result = (uintptr_t) ivars->xd_otherend_id;
|
|
return (0);
|
|
|
|
case XENBUS_IVAR_OTHEREND_PATH:
|
|
*result = (uintptr_t) ivars->xd_otherend_path;
|
|
return (0);
|
|
}
|
|
|
|
return (ENOENT);
|
|
}
|
|
|
|
int
|
|
xenbusb_write_ivar(device_t dev, device_t child, int index, uintptr_t value)
|
|
{
|
|
struct xenbus_device_ivars *ivars = device_get_ivars(child);
|
|
enum xenbus_state newstate;
|
|
int currstate;
|
|
|
|
switch (index) {
|
|
case XENBUS_IVAR_STATE:
|
|
{
|
|
int error;
|
|
|
|
newstate = (enum xenbus_state) value;
|
|
sx_xlock(&ivars->xd_lock);
|
|
if (ivars->xd_state == newstate) {
|
|
error = 0;
|
|
goto out;
|
|
}
|
|
|
|
error = xs_scanf(XST_NIL, ivars->xd_node, "state",
|
|
NULL, "%d", &currstate);
|
|
if (error)
|
|
goto out;
|
|
|
|
do {
|
|
error = xs_printf(XST_NIL, ivars->xd_node, "state",
|
|
"%d", newstate);
|
|
} while (error == EAGAIN);
|
|
if (error) {
|
|
/*
|
|
* Avoid looping through xenbus_dev_fatal()
|
|
* which calls xenbus_write_ivar to set the
|
|
* state to closing.
|
|
*/
|
|
if (newstate != XenbusStateClosing)
|
|
xenbus_dev_fatal(dev, error,
|
|
"writing new state");
|
|
goto out;
|
|
}
|
|
ivars->xd_state = newstate;
|
|
|
|
if ((ivars->xd_flags & XDF_CONNECTING) != 0
|
|
&& (newstate == XenbusStateClosed
|
|
|| newstate == XenbusStateConnected)) {
|
|
struct xenbusb_softc *xbs;
|
|
|
|
ivars->xd_flags &= ~XDF_CONNECTING;
|
|
xbs = device_get_softc(dev);
|
|
xenbusb_release_confighook(xbs);
|
|
}
|
|
|
|
wakeup(&ivars->xd_state);
|
|
out:
|
|
sx_xunlock(&ivars->xd_lock);
|
|
return (error);
|
|
}
|
|
|
|
case XENBUS_IVAR_NODE:
|
|
case XENBUS_IVAR_TYPE:
|
|
case XENBUS_IVAR_OTHEREND_ID:
|
|
case XENBUS_IVAR_OTHEREND_PATH:
|
|
/*
|
|
* These variables are read-only.
|
|
*/
|
|
return (EINVAL);
|
|
}
|
|
|
|
return (ENOENT);
|
|
}
|