Update the vm_page.h block comment to reflect recent changes.

Explain the new locking rules for per-page queue state updates.

Reviewed by:	jeff, kib
Sponsored by:	Netflix, Intel
Differential Revision:	https://reviews.freebsd.org/D22884

sys/vm/vm_page.h

@@ -171,34 +171,32 @@
  *	The page daemon must therefore handle the possibility of a concurrent
  *	free of the page.
  *
- *	The queue field is the index of the page queue containing the page,
- *	or PQ_NONE if the page is not enqueued.  The queue lock of a page is
- *	the page queue lock corresponding to the page queue index, or the
- *	page lock (P) for the page if it is not enqueued.  To modify the
- *	queue field, the queue lock for the old value of the field must be
- *	held.  There is one exception to this rule: the page daemon may
- *	transition the queue field from PQ_INACTIVE to PQ_NONE immediately
- *	prior to freeing a page during an inactive queue scan.  At that
- *	point the page has already been physically dequeued and no other
- *	references to that vm_page structure exist.
+ *	The queue state of a page consists of the queue and act_count fields of
+ *	its atomically updated state, and the subset of atomic flags specified
+ *	by PGA_QUEUE_STATE_MASK.  The queue field contains the page's page queue
+ *	index, or PQ_NONE if it does not belong to a page queue.  To modify the
+ *	queue field, the page queue lock corresponding to the old value must be
+ *	held, unless that value is PQ_NONE, in which case the queue index must
+ *	be updated using an atomic RMW operation.  There is one exception to
+ *	this rule: the page daemon may transition the queue field from
+ *	PQ_INACTIVE to PQ_NONE immediately prior to freeing the page during an
+ *	inactive queue scan.  At that point the page is already dequeued and no
+ *	other references to that vm_page structure can exist.  The PGA_ENQUEUED
+ *	flag, when set, indicates that the page structure is physically inserted
+ *	into the queue corresponding to the page's queue index, and may only be
+ *	set or cleared with the corresponding page queue lock held.
  *
- *	To avoid contention on page queue locks, page queue operations
- *	(enqueue, dequeue, requeue) are batched using per-CPU queues.  A
- *	deferred operation is requested by inserting an entry into a batch
- *	queue; the entry is simply a pointer to the page, and the request
- *	type is encoded in the page's aflags field using the values in
- *	PGA_QUEUE_STATE_MASK.  The type-stability of struct vm_pages is
- *	crucial to this scheme since the processing of entries in a given
- *	batch queue may be deferred indefinitely.  In particular, a page may
- *	be freed before its pending batch queue entries have been processed.
- *	The page lock (P) must be held to schedule a batched queue
- *	operation, and the page queue lock must be held in order to process
- *	batch queue entries for the page queue.  There is one exception to
- *	this rule: the thread freeing a page may schedule a dequeue without
- *	holding the page lock.  In this scenario the only other thread which
- *	may hold a reference to the page is the page daemon, which is
- *	careful to avoid modifying the page's queue state once the dequeue
- *	has been requested by setting PGA_DEQUEUE.
+ *	To avoid contention on page queue locks, page queue operations (enqueue,
+ *	dequeue, requeue) are batched using fixed-size per-CPU queues.  A
+ *	deferred operation is requested by setting one of the flags in
+ *	PGA_QUEUE_OP_MASK and inserting an entry into a batch queue.  When a
+ *	queue is full, an attempt to insert a new entry will lock the page
+ *	queues and trigger processing of the pending entries.  The
+ *	type-stability of vm_page structures is crucial to this scheme since the
+ *	processing of entries in a given batch queue may be deferred
+ *	indefinitely.  In particular, a page may be freed with pending batch
+ *	queue entries.  The page queue operation flags must be set using atomic
+ *	RWM operations.
  */
 
 #if PAGE_SIZE == 4096