Avoid grabbing locks when grabbing the vt(4) console for DDB.

Trying to grab locks during cngrab() when entering the debugger is
deadlock prone as all other CPUs are already halted (and thus unable
to release locks) when cngrab() is invoked.  One could instead use
try-locks.  However, the case that the try-lock fails still has to
be handled.  In addition, if the try-lock works it doesn't provide
any greater ordering guarantees than is already provided by entering
and exiting DDB.  It is simpler to define a simpler path for the
case that the try-lock would fail and always use that when entering
DDB.  Messing with timers, etc. when entering DDB is dubious even if
the try-lock succeeds.

This patch attempts to use the smallest possible set of operations to
grab the vt(4) console when entering DDB without using any locks.

Reviewed by:	emaste
Tested by:	Matthew Macy
MFC after:	1 week
This commit is contained in:
John Baldwin 2018-02-22 02:26:29 +00:00
parent eae594f7d5
commit 642ffab5fc

View File

@ -520,6 +520,27 @@ vt_window_switch(struct vt_window *vw)
struct vt_window *curvw = vd->vd_curwindow;
keyboard_t *kbd;
if (kdb_active) {
/*
* When grabbing the console for the debugger, avoid
* locks as that can result in deadlock. While this
* could use try locks, that wouldn't really make a
* difference as there are sufficient barriers in
* debugger entry/exit to be equivalent to
* successfully try-locking here.
*/
if (curvw == vw)
return (0);
if (!(vw->vw_flags & (VWF_OPENED|VWF_CONSOLE)))
return (EINVAL);
vd->vd_curwindow = vw;
vd->vd_flags |= VDF_INVALID;
if (vd->vd_driver->vd_postswitch)
vd->vd_driver->vd_postswitch(vd);
return (0);
}
VT_LOCK(vd);
if (curvw == vw) {
/* Nothing to do. */