Document all the sysctl values for the nda devices. Include some minimal

documentation on namespace support for nda devices. Fix a few typos
and formatting nits to apease igor.

share/man/man4/nda.4

@@ -25,7 +25,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd December 20, 2017
+.Dd June 6, 2020
 .Dt NDA 4
 .Os
 .Sh NAME
@@ -48,37 +48,156 @@ variables and
 .Xr loader 8
 tunables:
 .Bl -tag -width 12
+.It Va hw.nvme.use_nvd
+The
+.Xr nvme 4
+driver will create
+.Nm
+device nodes for block storage when set to 0.
+Create
+.Xr nvd 4
+device nodes for block storage when set to 1.
+See
+.Xr nvd 4
+when set to 1.
+.It Va kern.cam.nda.nvd_compat
+When set to 1,
+.Xr nvd 4
+aliases will be created for all
+.Nm
+devices, including partitions and other
+.Xr goem 4
+providers that take their names from the disk's name.
+.Xr nvd
+devices will not, however, be reported in the
+.Va kern.disks
+.Xr sysctl 8 .
 .It Va kern.cam.nda.sort_io_queue
-.Pp
 This variable determines whether the software queued entries are
 sorted in LBA order or not.
 Sorting is almost always a waste of time.
 The default is to not sort.
+.It Va kern.cam.nda.enable_biospeedup
+This variable determines if the
+.Nm
+devices participate in the speedup protocol.
+When the device participates in the speedup, then when the upper layers
+send a
+.Va BIO_SPEEDUP ,
+all current
+.Va BIO_DELETE
+requests not yet sent to the hardware are completed successfully immediate
+without sending them to the hardware.
+Used in low disk space scenarios when the filesystem encounters
+a critical shortage and needs blocks immediately.
+Since trims have maximum benefit when the LBA is unused for a long time,
+skipping the trim when space is needed for immediate writes results in little to
+no excess wear.
+When participation is disabled,
+.Va BIO_SPEEDUP
+requests are ignored.
+.It Va kern.cam.nda.max_trim
+The maximum number of LBA ranges to be collected together for each DSM trims
+send to the hardware.
+Defaults to 256, which is the maximum number of ranges the protocol supports.
+Sometimes poor trim performance can be mitigated by limiting the number of
+ranges sent to the device.
+This value must be between 1 and 256 inclusive.
 .El
 .Pp
 The following report per-device settings, and are read-only unless
-otherwise indicated. Replace
+otherwise indicated.
+Replace
 .Va N
 with the device unit number.
 .Bl -tag -width 12
 .It Va kern.cam.nda.N.rotating
-.Pp
 This variable reports whether the storage volume is spinning or
 flash.
-It's value is hard coded to 0 indicating flash.
+Its value is hard coded to 0 indicating flash.
 .It Va kern.cam.nda.N.unmapped_io
 This variable reports whether the
 .Nm
 driver accepts unmapped I/O for this unit.
+.It Va kern.cam.nda.N.flags
+This variable reports the current flags.
+.Bl -tag -width 12
+.It Va OPEN
+The device is open.
+.It Va DIRTY
+Set when a write to the drive is scheduled.
+Cleared after flush commands.
+.It Va SCTX_INIT
+Internal flag set after
+.Xr sysctl 8
+nodes have been created.
 .El
+.It Va kern.cam.nda.N.sort_io_queue
+Same as the
+.Va kern.cam.nda.sort_io_queue
+tunable.
+.It Va kern.cam.nda.N.trim_ticks
+Writable.
+When greater than zero, hold trims for up to this many ticks before sending
+to the drive.
+Sometimes waiting a little bit to collect more trims to send at one time
+improves trim performance.
+When 0, no delaying of trims are done.
+.It Va kern.cam.nda.N.trim_goal
+Writable.
+When delaying a bit to collect multiple trims, send the accumulated DSM TRIM to
+the drive.
+.It Va kern.cam.nda.N.trim_lbas
+Total number of LBAs that have been trimmed.
+.It Va kern.cam.nda.N.trim_ranges
+Total number of LBA ranges that have been trimmed.
+.It Va kern.cam.nda.N.trim_count
+Total number of trims sent to the hardware.
+.It Va kern.cam.nda.N.deletes
+Total number of
+.Va BIO_DELETE
+requests queued to the device.
+.El
+.Sh NAMESPACE MAPPING
+Each
+.Xr nvme 4
+drive has one or more namespaces associated with it.
+One instance of the
+.Nm
+driver will be created for each of the namespaces on
+the drive.
+All the
+.Nm
+nodes for a
+.Xr nvme 4
+device are at target 0.
+However, the namespace ID maps to the CAM lun, as reported
+in kernel messages and in the
+.Va devlist
+sub command of
+.Xr camcontrol 8 .
+.Pp
+Namespaces are managed with the
+.Va ns
+sub command of
+.Xr nvmecontrol 8 .
+Not all drives support namespace management,
+but all drives support at least one namespace.
+Device nodes for
+.Nm
+will be created and destroyed dynamically as
+namespaces are activated or detached.
 .Sh FILES
 .Bl -tag -width ".Pa /dev/nda*" -compact
 .It Pa /dev/nda*
 NVMe storage device nodes
 .El
 .Sh SEE ALSO
+.Xr cam 4 ,
+.Xr geom 4 ,
 .Xr nvd 4 ,
-.Xr nvme 4
+.Xr nvme 4 ,
+.Xr gpart 8
 .Sh HISTORY
 The
 .Nm