m_reclaim() and re-acquire it when m_reclaim() returns. This means that
we now call the drain routines without holding the mutex lock and
recursing into it. This was done for mainly two reasons:
(i) Avoid the long recursion; long recursions are typically bad and this
is the case here because we block all other code from freeing mbufs
if they need to. Doing that is kind of counter-productive, since we're
really hoping that someone will free.
(ii) More importantly, avoid a potential lock order reversal. Right now,
not all the locks have been added to our networking code; but
without this change, we're introducing the possibility for deadlock.
Consider for example ip_drain(). We will likely eventually introduce
a lock for ipq there, and so ip_freef() will be called with ipq lock
held. But, ip_freef() calls m_freem() which in turn acquires the
mmbfree lock. Since we were previously calling ip_drain() with mmbfree
held, our lock order would be: mmbfree->ipq->mmbfree. Some other code
may very well lock ipq first and then call ip_freef(). This would
result in the regular lock order, ipq->mmbfree. Clearly, we have
deadlock if one thread acquires the ipq lock and sits waiting for
mmbfree while another thread calling m_reclaim() acquires mmbfree
and sits waiting for the ipq lock.
Also, make sure to add a comment above m_reclaim()'s definition briefly
explaining this. Also document this above the call to m_reclaim() in
m_mballoc_wait().
Suggested and reviewed by: alfred
d113d3857e