Commit Graph

315 Commits

Author SHA1 Message Date
Kenneth D. Merry
43518607b2 Significant upgrades to sa(4) and mt(1).
The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

 o sa(4) driver status and parameter information is now exported via an
   XML structure.  This will allow for changes and improvements later
   on that will not break userland applications.  The old MTIOCGET
   status ioctl remains, so applications using the existing interface
   will not break.

 o 'mt status' now reports drive-reported tape position information
   as well as the previously available calculated tape position
   information.  These numbers will be different at times, because
   the drive-reported block numbers are relative to BOP (Beginning
   of Partition), but the block numbers calculated previously via
   sa(4) (and still provided) are relative to the last filemark.
   Both numbers are now provided.  'mt status' now also shows the
   drive INQUIRY information, serial number and any position flags
   (BOP, EOT, etc.) provided with the tape position information.
   'mt status -v' adds information on the maximum possible I/O size,
   and the underlying values used to calculate it.

 o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

   The extra devices were originally added as place holders for
   density-specific device nodes.  Some OSes (NetBSD, NetApp's OnTap
   and Solaris) have had device nodes that, when you write to them,
   will automatically select a given density for particular tape drives.

   This is a convenient way of switching densities, but it was never
   implemented in FreeBSD.  Only the device nodes were there, and that
   sometimes confused users.

   For modern tape devices, the density is generally not selectable
   (e.g. with LTO) or defaults to the highest availble density when
   the tape is rewritten from BOT (e.g. TS11X0).  So, for most users,
   density selection won't be necessary.  If they do need to select
   the density, it is easy enough to use 'mt density' to change it.

 o Protection information is now supported.  This is either a
   Reed-Solomon CRC or CRC32 that is included at the end of each block
   read and written.  On write, the tape drive verifies the CRC, and
   on read, the tape drive provides a CRC for the userland application
   to verify.

 o New, extensible tape driver parameter get/set interface.

 o Density reporting information.  For drives that support it,
   'mt getdensity' will show detailed information on what formats the
   tape drive supports, and what formats the tape drive supports.

 o Some mt(1) functionality moved into a new mt(3) library so that
   external applications can reuse the code.

 o The new mt(3) library includes helper routines to aid in parsing
   the XML output of the sa(4) driver, and build a tree of driver
   metadata.

 o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
   (write filemark immediate) ioctls needed by IBM's LTFS
   implementation.

 o Improve device departure behavior for the sa(4) driver.  The previous
   implementation led to hangs when the device was open.

 o This has been tested on the following types of drives:
	IBM TS1150
	IBM TS1140
	IBM LTO-6
	IBM LTO-5
	HP LTO-2
	Seagate DDS-4
	Quantum DLT-4000
	Exabyte 8505
	Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
	Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
	New mt(3) library that contains functions moved from mt(1) and
	new functions needed to interact with the updated sa(4) driver.

	This includes XML parser helper functions that application writers
	can use when writing code to query tape parameters.

rescue/rescue/Makefile:
	Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
	Clarify this man page a bit, and since it contains what is
	essentially the mtio.h header file, add new ioctls and structure
	definitions from mtio.h.

src/share/man/man4/sa.4
	Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
	Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
	functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
	Many tape driver changes, largely outlined above.

	Increase the sa(4) driver read/write timeout from 4 to 32
	minutes.  This is based on the recommended values for IBM LTO
	5/6 drives.  This may also avoid timeouts for other tape
	hardware that can take a long time to do retries and error
	recovery.  Longer term, a better way to handle this is to ask
	the drive for recommended timeout values using the REPORT
	SUPPORTED OPCODES command.  Modern IBM and Oracle tape drives
	at least support that command, and it would allow for more
	accurate timeout values.

	Add XML status generation.  This is done with a series of
	macros to eliminate as much duplicate code as possible.  The
	new XML-based status values are reported through the new
	MTIOCEXTGET ioctl.

	Add XML driver parameter reporting, using the new MTIOCPARAMGET
	ioctl.

	Add a new driver parameter setting interface, using the new
	MTIOCPARAMSET and MTIOCSETLIST ioctls.

	Add a new MTIOCRBLIM ioctl to get block limits information.

	Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
	and scsi_read_position_10().

	scsi_locate_10 implements the LOCATE command, as does the
	existing scsi_set_position() command.  It just supports
	additional arguments and features.  If/when we figure out a
	good way to provide backward compatibility for older
	applications using the old function API, we can just revamp
	scsi_set_position().  The same goes for
	scsi_read_position_10() and the existing scsi_read_position()
	function.

	Revamp sasetpos() to take the new mtlocate structure as an
	argument.  It now will use either scsi_locate_10() or
	scsi_locate_16(), depending upon the arguments the user
	supplies.  As before, once we change position we don't have a
	clear idea of what the current logical position of the tape
	drive is.

	For tape drives that support long form position data, we
	read the current position and store that for later reporting
	after changing the position.  This should help applications
	like Bacula speed tape access under FreeBSD once they are
	modified to support the new ioctls.

	Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
	drives that report SCSI-2 or older, as well as drives that
	report an Illegal Request type error for READ POSITION with
	the long format.  So we should automatically detect drives
	that don't support the long form and stop asking for it after
	an initial try.

	Add a partition number to the sa(4) softc.

	Improve device departure handling. The previous implementation
	led to hangs when the device was open.

	If an application had the sa(4) driver open, and attempted to
	close it after it went away, the cam_periph_release() call in
	saclose() would cause the periph to get destroyed because that
	was the last reference to it.  Because destroy_dev() was
	called from the sa(4) driver's cleanup routine (sacleanup()),
	and would block waiting for the close to happen, a deadlock
	would result.

	So instead of calling destroy_dev() from the cleanup routine,
	call destroy_dev_sched_cb() from saoninvalidate() and wait for
	the callback.

	Acquire a reference for devfs in saregister(), and release it
	in the new sadevgonecb() routine when all devfs devices for
	the particular sa(4) driver instance are gone.

	Add a new function, sasetupdev(), to centralize setting
	per-instance devfs device parameters instead of repeating the
	code in saregister().

	Add an open count to the softc, so we know how many
	peripheral driver references are a result of open
       	sessions.

	Add the D_TRACKCLOSE flag to the cdevsw flags so
	that we get a 1:1 mapping of open to close calls
	instead of a N:1 mapping.

	This should be a no-op for everything except the
	control device, since we don't allow more than one
	open on non-control devices.

	However, since we do allow multiple opens on the
	control device, the combination of the open count
	and the D_TRACKCLOSE flag should result in an
	accurate peripheral driver reference count, and an
	accurate open count.

	The accurate open count allows us to release all
	peripheral driver references that are the result
	of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
	Add a number of new mt(4) ioctls and the requisite data
	structures.  None of the existing interfaces been removed
	or changed.

	This includes definitions for the following new ioctls:

	MTIOCRBLIM      /* get block limits */
	MTIOCEXTLOCATE	/* seek to position */
	MTIOCEXTGET     /* get tape status */
	MTIOCPARAMGET	/* get tape params */
	MTIOCPARAMSET	/* set tape params */
	MTIOCSETLIST	/* set N params */

usr.bin/mt/Makefile:
	mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
	Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
	Implement support for mt(1) subcommands that need to
	use getopt(3) for their arguments.

	Implement a new 'mt status' command to replace the old
	'mt status' command.  The old status command has been
	renamed 'ostatus'.

	The new status function uses the MTIOCEXTGET ioctl, and
	therefore parses the XML data to determine drive status.
	The -x argument to 'mt status' allows the user to dump out
	the raw XML reported by the kernel.

	The new status display is mostly the same as the old status
	display, except that it doesn't print the redundant density
	mode information, and it does print the current partition
	number and position flags.

	Add a new command, 'mt locate', that will supersede the
	old 'mt setspos' and 'mt sethpos' commands.  'mt locate'
	implements all of the functionality of the MTIOCEXTLOCATE
	ioctl, and allows the user to change the logical position
	of the tape drive in a number of ways.  (Partition,
	block number, file number, set mark number, end of data.)
	The immediate bit and the explicit address bits are
	implemented, but not documented in the man page.

	Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
	This allows the user to ask the drive to write a filemark
	without waiting around for the operation to complete.

	Add a new 'mt getdensity' command that gets the XML-based
	tape drive density report from the sa(4) driver and displays
	it.  This uses the SCSI REPORT DENSITY SUPPORT command
	to get comprehensive information from the tape drive about
	what formats it is able to read and write.

	Add a new 'mt protect' command that allows getting and setting
	tape drive protection information.  The protection information
	is a CRC tacked on to the end of every read/write from and to
	the tape drive.

Sponsored by:	Spectra Logic
MFC after:	1 month
2015-02-23 21:59:30 +00:00
Ulrich Spörlein
9e90c87023 Pull up vendor changes up to 2014-01-29
- move local overrides into mdoc.local
- syncs us with git commit 819839b66c80e8dabe6cb24ea6319c26c9a2be14

Discussed with: ru
MFC after: 2 weeks
2014-01-31 12:26:30 +00:00
Sean Bruno
b51b786d73 Quiesce warnings -Wdangling-else
Thanks to jilles for some reviews and tips.
2013-10-28 18:10:35 +00:00
Pawel Jakub Dawidek
8e70aa6c47 Add libsbuf. 2012-09-22 17:47:37 +00:00
Ruslan Ermilov
ca0a2184d6 Pull up vendor changes to mdoc(7). 2012-07-26 11:12:38 +00:00
Ruslan Ermilov
a16720da65 Backed out r228904, and added libstdthreads support to mdoc(7) to where
it belongs.
2012-07-26 11:10:25 +00:00
Ruslan Ermilov
770b69c0b7 Backed out r236255, and added FreeBSD 9.1 support to mdoc(7) to where
it belongs.
2012-07-26 10:58:30 +00:00
David E. O'Brien
585ed8cb79 Add the latest FreeBSD versions. 2012-05-29 19:49:52 +00:00
Ulrich Spörlein
cabdddae57 Pull up vendor changes to mdoc(7)
This switches us to using -isoC-2011 as the symbol name which is used by
groff and mdocml. It follows the change to 4 digit years as done with
IEEE Std 1003 post-1999.

MFC after:	2 weeks (groff changes only)
2012-01-05 21:36:12 +00:00
Ed Schouten
fc6f0665ba Add libstdthreads.
This library implements the C11 threads interface on top of the pthreads
library.  As discussed on the lists, the preferred way to implement
this, is as a separate library.

It is unlikely that these functions will be used a lot in the future. It
would have been easier if the C11 working group standardized (a subset
of) pthreads and clock_nanosleep(). Having it as a separate library
allows the embedded people to omit it from their system.

Discussed on:	arch@, threads@
2011-12-26 21:51:53 +00:00
Ed Schouten
5b7b8966c1 Add the C11 standard to groff. This allows us to refer to C11 as -isoC-11.
MFC after:	2 months
2011-12-25 22:05:33 +00:00
Ruslan Ermilov
dd5e6f8a24 Pull up vendor changes to mdoc(7). 2011-12-08 13:54:06 +00:00
Ruslan Ermilov
c8c215e682 Pull up all vendor changes to mdoc(7).
This also replaces the local fix in r219209 that made .Ac emit
ASCII angle quotes with an official fix.  In the official fix,
ASCII quotes are output when using the .Aq, .Ao and .Ac calls,
but only when nested into the .An macro.

PR:		gnu/154822
2011-06-08 08:08:42 +00:00
Ruslan Ermilov
0829f5978b Re-enable SGR support (ANSI color escapes) in grotty(1) by default.
Our man(1) and bsd.doc.mk still disable it for POLA reasons via the
-c option to grotty(1).

PR:		gnu/82353
2011-06-03 12:02:53 +00:00
Ben Laurie
d74336e632 Fix clang warnings.
Note: possible minor security issues fixed (untrusted string used as
printf format string).

Approved by:	philip (mentor)
2011-05-18 21:04:29 +00:00
Ulrich Spörlein
49cd9b4388 Unbreak .Aq for non-ASCII output like -Tps or -Tutf8.
groff will try to produce fancy angle brackets like
	Foo ⟨foo@FreeBSD.org⟩
This is nice and well, but no email client will understand them. For
ease of copy&paste keep the one-true pair of brackets 0x3c/0x3e.

See:		RFC 822, RFC 2822
PR:		gnu/154822
Submitted by:	Dominic Fandrey <kamikaze@bsdforen.de>
MFC after:	2 weeks
2011-03-02 21:59:42 +00:00
Ulrich Spörlein
7ea7457aac Update groff manpage and symbols with what has been submitted upstream.
Also remove local overrides that are now in the contrib tree.

This is a direct commit to contrib/ as we will no longer import any
newer groff snapshots, due to licensing issues.

MFC after:	3 weeks
2011-01-19 20:16:38 +00:00
Ruslan Ermilov
18adde964d Pull up vendor changes. 2010-01-15 15:10:29 +00:00
Ruslan Ermilov
1b2ef7353d Moved the doc-str-Lb-libulog string definition to where it belongs. 2010-01-15 14:05:06 +00:00
Ruslan Ermilov
4d433b8ccd Pull up vendor changes. The following local changes made obsolete:
- Addition of several FreeBSD versions.
- r192561 that attempted to fix UTF-8 issues.
2010-01-15 13:59:50 +00:00
Ruslan Ermilov
0e96a56a89 Flatten out vendor tree. 2010-01-15 12:01:25 +00:00
Ed Schouten
2208eadf43 Add a new library: libulog.
One of the things I really want to do, is to get rid of the limitations
of our current utmp(5) mechanism:

- It only allows 8 byte TTY device names.
- The hostname only allows 16 bytes of storage.

I'm not a big fan of <utmpx.h>, but I think we should at least try to
add parts of it. Unfortunately we cannot implement <utmpx.h>, because we
miss various fields, such as ut_id, ut_pid, etc. The API provided by
libulog shares some similarities with <utmpx.h>, so it shouldn't be too
hard to port these applications eventually. In most simple cases, it
should just be a matter of removing the ulog_ prefix everywhere.

As a bonus, it also implements a function called ulog_login_pseudo(),
which allows unprivileged applications to write log entries, provided
they have a valid file descriptor to a pseudo-terminal master device.

libulog will allow a smoother transition to a new file format by adding
a library interface to deal with utmp/wtmp/lastlog files. I initially
thought about adding the functionality to libutil, but because I'm not
planning on keeping this library around forever, we'd better keep it
separated.

Next items on the todo list:

1. Port applications in the base system (and ports) to libulog, instead
   of letting them use <utmp.h>.
2. Remove <utmp.h>, implement <utmpx.h> and reimplement this library on
   top.
3. Port as many applications as possible back to <utmpx.h>.
2009-12-03 15:48:24 +00:00
Dag-Erling Smørgrav
62aa21ae86 When man pages are formatted in UTF-8, .Fl is encoded as U+2212 "MINUS
SIGN" instead of U+002D "HYPHEN-MINUS".  This is unfortunate for two
reasons: 1) this is not the character which is actually used on the
command line, and 2) it makes it impossible to search a man page for a
specific command-line option.

This patch fixes this, but there are other unresolved issues, such as
confusion between -, \- and hy: while the latter is always (and only)
used for hyphenation, both - and \- are used for negation and
subtraction, and \- is used for command-line options and sometimes
also for parenthesis.  IMHO, the correct Unicode characters are:

 - hyphenation: either U+2010 or U+00AD, most likely the former (the
   latter is the so-called soft hyphen, used to indicate a point at
   which a text processor is allowed to hyphenate a word)

 - negation and subtraction: U+2212

 - parenthesis: in English, U+2214, with spaces suppressed before and
   after; in some others (such as Norwegian), U+2213 with spaces
   retained.

 - command-line options: U+002D, because that is what is actually used
   on the command line.

However, fixing this would require extensive modifications to (at least)
the doc and man macro sets...

MFC after:	1 week
2009-05-21 17:56:00 +00:00
Ruslan Ermilov
80cc270287 MFV: recent mdoc(7) changes.
Approved by:	re (kensmith)
2007-10-04 04:38:23 +00:00
Ruslan Ermilov
95261a8d0f Pull up current -mdoc version from vendor. This includes the
following changes:

: 2007-10-02  Ruslan Ermilov  <ru@FreeBSD.org>
:
:         * tmac/doc-common, tmac/groff_doc.man: Add FreeBSD 6.2, document
:         FreeBSD 5.5 and 7.0.
:
:         * tmac/doc-syms: Give better names for System V releases.

: 2007-05-30  Werner LEMBERG <wl@gnu.org>
:
:         * tmac/groff_mdoc.man: Convert `-' to `\-' where appropriate.
:         Recommend `tbl' instead of `-column' lists for more complicated
:         cases.
2007-10-03 06:04:40 +00:00
Ruslan Ermilov
2cd96fdf2c This commit was generated by cvs2svn to compensate for changes in r172423,
which included commits to RCS files with non-trunk default branches.
2007-10-03 06:04:40 +00:00
Ruslan Ermilov
3d4c5c90cb MFV: Sync with vendor branch. 2006-12-25 09:03:10 +00:00
Ruslan Ermilov
5bba0737f1 Pull up current -mdoc version from vendor.
- New Darwin, FreeBSD, and NetBSD versions.
- DragonFly support including the new .Dx macro.
- New .St strings: -isoC-amd1, -isoC-tcor1, -isoC-tcor2, and -ieee1275-94.
2006-12-25 08:59:09 +00:00
Ruslan Ermilov
16fa9cc022 This commit was generated by cvs2svn to compensate for changes in r165538,
which included commits to RCS files with non-trunk default branches.
2006-12-25 08:59:09 +00:00
Ruslan Ermilov
b281aca4ad Pull up from the FSF branch. 2006-01-12 15:31:41 +00:00
Ruslan Ermilov
b99439c39a Merge support for new BSD releases from upstream:
- Darwin 8.[0-3].0
- FreeBSD 4.11, 5.4, 6.0, 6.1
- NetBSD 3.0
2006-01-12 15:26:11 +00:00
Ruslan Ermilov
401070f083 This commit was generated by cvs2svn to compensate for changes in r154258,
which included commits to RCS files with non-trunk default branches.
2006-01-12 15:26:11 +00:00
Ruslan Ermilov
5ca88fcbc5 Update upgrade instructions. 2005-10-20 11:18:24 +00:00
Ruslan Ermilov
ed6ac7236a Merge Groff 1.19.2 changes. 2005-10-20 10:53:15 +00:00
Ruslan Ermilov
982fd8a3e4 Use stock (FSF) version of this file. 2005-10-20 10:52:34 +00:00
Ruslan Ermilov
22481ce32f Removed files not present in v1.19.2 import. 2005-10-20 10:47:26 +00:00
Ruslan Ermilov
d870518220 This commit was generated by cvs2svn to compensate for changes in r151500,
which included commits to RCS files with non-trunk default branches.
2005-10-20 10:47:26 +00:00
Ruslan Ermilov
22e7cbb874 Virgin import of FSF groff v1.19.2 2005-10-20 10:45:19 +00:00
Ruslan Ermilov
1c7b34d8d7 This commit was generated by cvs2svn to compensate for changes in r151497,
which included commits to RCS files with non-trunk default branches.
2005-10-20 10:45:19 +00:00
Ruslan Ermilov
2ed83d261b MFV: Latest mdoc(7) fixes. 2005-01-25 09:32:56 +00:00
Ruslan Ermilov
bc33253bf8 Bring in latest mdoc(7) fixes over here. 2005-01-25 09:29:48 +00:00
Ruslan Ermilov
3899af05ce This commit was generated by cvs2svn to compensate for changes in r140801,
which included commits to RCS files with non-trunk default branches.
2005-01-25 09:29:48 +00:00
Ruslan Ermilov
32fe84f7e8 Pull up a vendor fix for a problem exposed by tr/tr.1,v 1.30. 2005-01-14 09:50:45 +00:00
Ruslan Ermilov
7643035a1c This commit was generated by cvs2svn to compensate for changes in r140229,
which included commits to RCS files with non-trunk default branches.
2005-01-14 09:50:45 +00:00
Joseph Koshy
548c9b4376 Revert rev 1.66. This file needs to track its upstream source.
Requested by:	ru
2004-12-17 16:29:54 +00:00
Joseph Koshy
e791235b73 Support for ".Fx 6.0".
Reviewed by:	ru
2004-12-17 14:10:44 +00:00
Ruslan Ermilov
6f9868d148 Pull up the latest minor fixes. 2004-10-05 06:14:29 +00:00
Ruslan Ermilov
04e67b43ff This commit was generated by cvs2svn to compensate for changes in r136136,
which included commits to RCS files with non-trunk default branches.
2004-10-05 06:14:29 +00:00
Ruslan Ermilov
abcbe24601 Sync with FSF. 2004-07-30 08:14:00 +00:00
Ruslan Ermilov
e4bda9ead6 Pull up latest mdoc(7) changes:
: 2004-05-10  Werner LEMBERG  <wl@gnu.org>
:
: 	* tmac/doc-syms (doc-str-Lb-libposix, doc-str-Lb-libpthread,
: 	doc-str-Lb-librt): Reset font.
: 	(Lb): Rename `doc-str-Lb' to `doc-str-Lb1'.
: 	Provide `doc-str-Lb' to reset font.
: 	* tmac/groff_mdoc.man: Updated.

: 2004-07-08  Thomas Klausner  <wiz@netbsd.org>
:
: 	* tmac/doc-syms (doc-str-St--p1003.1-2004): New string.
: 	* tmac/groff_mdoc.man: Updated.
2004-07-30 08:12:09 +00:00