d/numam-spdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

virtio: don't reinitialize entire virtqueues on mem hotplug

Browse Source 
One of the messages we send on memory hotplug event is
SET_VRING_BASE, which tells vhost e.g. the position in
a vring it should start processing requests from. Sending
this message with any outstanding I/O could cause that
I/O to be never processed as it could be at a vring
position that won't be practically polled.

To fix the above, we don't send SET_VRING_BASE message
on memory hotplug event anymore since it's completely
unnecessary. It was sent together with a couple other
messages that would reinitialize the vring, but we know
vrings occupy a memory buffer that won't be hotremoved
during vring lifetime. We also know that vring GPAs will
never change. Hence we can initialize the vrings just
once on device start now.

We still need to send SET_VRING_ADDR after updating the
memory table, as rte_vhost depends on it to apply that
new memory table. Luckily, this single message doesn't
cause us any trouble.

Change-Id: I2125099f1cf3f8c76e8160ec819bd1a9a3e7823c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439436
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This commit is contained in:
Darek Stojaczyk 2019-01-07 22:26:14 +01:00 committed by Jim Harris
parent 4d74fad8b5
commit 148eda65c1
1 changed files with 15 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
lib/virtio/virtio_user.c
View File

@ -67,11 +67,9 @@ virtio_user_create_queue(struct virtio_dev *vdev, uint32_t queue_sel)
}
static int
virtio_user_kick_queue(struct virtio_dev *vdev, uint32_t queue_sel)
virtio_user_set_vring_addr(struct virtio_dev *vdev, uint32_t queue_sel)
{
struct virtio_user_dev *dev = vdev->ctx;
struct vhost_vring_file file;
struct vhost_vring_state state;
struct vring *vring = &dev->vrings[queue_sel];
struct vhost_vring_addr addr = {
.index = queue_sel,
@ -81,6 +79,17 @@ virtio_user_kick_queue(struct virtio_dev *vdev, uint32_t queue_sel)
.log_guest_addr = 0,
.flags = 0, /* disable log */
};
return dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
}
static int
virtio_user_kick_queue(struct virtio_dev *vdev, uint32_t queue_sel)
{
struct virtio_user_dev *dev = vdev->ctx;
struct vhost_vring_file file;
struct vhost_vring_state state;
struct vring *vring = &dev->vrings[queue_sel];
int rc;
state.index = queue_sel;
@ -97,10 +106,7 @@ virtio_user_kick_queue(struct virtio_dev *vdev, uint32_t queue_sel)
return rc;
}
rc = dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
if (rc < 0) {
return rc;
}
virtio_user_set_vring_addr(vdev, queue_sel);
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
@ -162,7 +168,7 @@ virtio_user_map_notify(void *cb_ctx, struct spdk_mem_map *map,
/* We have to send SET_VRING_ADDR to make rte_vhost flush a pending
* SET_MEM_TABLE...
*/
ret = virtio_user_queue_setup(vdev, virtio_user_kick_queue);
ret = virtio_user_queue_setup(vdev, virtio_user_set_vring_addr);
if (ret < 0) {
return ret;
}
@ -243,7 +249,7 @@ virtio_user_start_device(struct virtio_dev *vdev)
return ret;
}
return 0;
return virtio_user_queue_setup(vdev, virtio_user_kick_queue);
}
static int