This is based on a change from OpenBSD:
"Fix restore so that it can actually restore files larger than 4GB by
changing the type of "size" to off_t in getfiles() plus little dependent
type cleanup, from Daniel Lucq."
It is an important for machines with 32 bit longs.
While here unsign the flags, also from OpenBSD.
Obtained from: OpenBSD (through bitrig, I hate CVS)
MFC after: 2 weeks
a seek for every block... For large (Exabyte sized files) this would
issue lots of unneeded seeks, so combine them...
Thanks for the work Jan, sorry took so long to commit... And an item
completed off the IdeasPage!
Submitted by: Jan Sucan
The restore utility already knows the full-resolution timestamps, so the
only thing to do is to stop converting the timespecs to timevals and use
futimens() and utimensat().
Differential Revision: https://reviews.freebsd.org/D2020
Reviewed by: jhb
restore was failing because ZFS was reporting a blocksize that was
not a multiple of 1024. Replace restore's failed assertion with
code that writes restored files in a blocksize that works for
restore (a multiple of 1024) despite being non-optimal for ZFS.
Submitted by: Dmitry Morozovsky
Tested by: Dmitry Morozovsky
MFC after: 1 week
current version of FreeBSD, this isn't guarenteed by the API.
Custom security modules, or future implementations of the setuid and
setgid may fail.
Submitted by: Erik Cederstrand
Approved by: cperciva
MFC after: 3 days
handled when reading from pipes.
- Remove dead code related to the -P option from getvol(). pipein and
pipecmdin are never set at the same time.
PR: bin/121502
Approved by: trasz (mentor)
MFC after: 2 weeks
in format strings.
- Use (void) instead of (void *) when discarding strcat(3) return value.
- Format string fixes to match variable types.
- Change canon() len parameter and getcmd() size parameter type from
int to size_t.
- Style Makefile and increase WARNS to 2.
PR: bin/140061
Submitted by: uqs
Approved by: trasz (mentor)
on a 7.0 or later system that were created on a pre-5.0 system.
We must ensure that restore zeros out the previously undefined
birthtime and external attribute size fields when reading dump
tapes made by the UFS1 dump program.
The problem is that UFS2 dump carefully zeros out the unused
birthtime and external attribute size fields in the dump header
when dumping UFS1 filesystems, but the UFS1 dump didn't know about
those fields (they were spares) so just left whatever random junk
was in them. So, when restoring one of these pre-UFS2 dumps,
the new restore would eventually trip across a header that had
a non-zero external attribute size and try to extract it. That
consumed several tape blocks which left it totally out of sync
and very unhappy (i.e., the panic). The fix is in the gethead()
function which modernizes old headers by copying old fields to
their new location (and with this fix) zeroing out previously
undefined fields.
PR: bin/120881
Review by: David Malone & Scott Lambert
MFC after: 1 week
Must ensure that dump tapes from UFS1 filesystems properly copy
old fields of dump headers to new locations. Move check of dumpdate
to follow the code which ensures that the appropriate fields have
been copied.
PR: bin/118087
Help from: David Malone, Scott Lambert, Javier Martín Rueda
MFC after: 2 weeks
<sys/extattr.h> to <ufs/ufs/extattr.h>. Move description
of extended attributes in UFS from man9/extattr.9 to
man5/fs.5.
Note that restore will not compile until <sys/extattr.h>
and <ufs/ufs/extattr.h> have been updated.
Suggested by: Robert Watson
the restore program to restore all dumped extended attributes.
If the restore is running as root, it will always be able
to restore all extended attributes. If it is not running
as root, it makes a best effort to set them. Using the -v
command line flag or the `verbose' command in interactive
mode will display all the extended attributes being set on
files (and at the end on directories) that are being restored.
It will note any extended attributes that could not be set.
The extended attributes are placed on the dump image immediately
following each file's data. Older versions of restore can work
with the newer dump images. Old versions of restore will
correctly restore the file data and then (silently) skip
over the extended attribute data and proceed to the next file.
This resolves PR 93085 which will be closed once the code
has been MFC'ed.
Note that this code will not compile until these header
files have been updated: <protocols/dumprestore.h> and
<sys/extattr.h>.
PR: bin/93085
Comments from: Poul-Henning Kamp and Robert Watson
MFC after: 3 weeks
makes restore less efficient, but it makes a bigger effore to read
corrupted dumps. Specifiacally, when in degreded mode:
1) Restore shifts the input by 1 byte if it sees a problem,
rather than one tape block.
2) It doesn't assume the inodes are stored in ascending order.
3) It turns some panics into warning printfs.
We also verify some fields more carefully than before.
There's probably more a degreded mode could do, but this seems to
help a lot.
Approved by: imp, iedowse, mckusick
MFC after: 3 weeks
o Restore owner/group/mode/atime/mtime of symbolic links, rev. 1.30.
o Extract file flags of symbolic link, rev. 1.42.
o Call getfile() before altering file attributes.
Open file with mode 0600 instead of 0666 so that file won't remain
group or world readable/writable even if getfile() terminated.
Move skipfile() before altering file attributes in IF{CHR,BLK} and
IFIFO case for symmetry, rev. 1.32.
o Use file mode 0600 when creating special file or fifo, revs. 1.33, 1.34.
o Remove redundant -N check.
PR: bin/101660
Submitted by: Andrey V. Elsukov
Obtained from: NetBSD, enami@netbsd
MFC after: 6 weeks
based tapes, but I'm not sure where NFS_MAGIC was introduced after
4.3). When support for the pre-4.4 format was removed (the ability to
read 4.2 and 4.3 BSD tapes), the old format inode conversion was
junked as well. However, FreeBSD 1 dump tapes use the NFS_MAGIC
format, but have this inode format. Before, restore would fail
complaining that '.' wasn't found and the root directory wasn't on
this tape. Since the conversion from the not so old format is
relatively trivial, restore the code to make that conversion.
FreeBSD 1 dumps are once again readable.
MFC After: a few days
for the old (4.4-lite through FreeBSD 4.x and *BSD) format. It looks
like they aren't used for TS_INODE, but conversion costs so little
there that I've not removed them there (in case my grep was wrong).
This makes at least some of the tapes work for me again. Now, to
regresion test all my dusty tapes...
use backup methods other than files and tapes. The -P argument is
a normal sh(1) pipeline with either $DUMP_VOLUME or $RESTORE_VOLUME
defined in the environment, respectively.
For example, I can back up my home to three DVD+R[W]s as so:
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/ad0s2e 40028550 10093140 26733126 27% /home
green# dump -0 -L -C16 -B4589840 -P 'growisofs -Z /dev/cd0=/dev/fd/0' /home
support creation times such as UFS2) to the value of the
modification time if the value of the modification time is older
than the current creation time. See utimes(2) for further details.
Sponsored by: DARPA & NAI Labs.
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.
Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.
Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).
Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
It does not help modern compilers, and some may take some hit from it.
(I also found several functions that listed *every* of its 10 local vars with
"register" -- just how many free registers do people think machines have?)
after an EOT-terminated volume. We keep track of the current record
number, and synchronise it with the c_tapea field each time we read
a header. Avoid the use of c_firstrec because some bugs in dump can
cause it to be set incorrectly.
Move the initialisation of some variables to avoid compiler warnings.
volume if we missed some earlier tapes (the user can still enter
'none' later if the tapes are unavailable). Previously with 'x'
restores, we might not ask for all tapes if the tapes are supplied
in reverse order.
Clarify the message that describes what volume should be mounted
first; reverse order is only efficient when extracting a few files.
to multi-volume restores:
- In findinode(), keep a copy of header->c_type so that we don't
exit the do-while loop until we have processed the current header.
Exiting too early leaves curfile.ino set to 0, which confuses
the logic in createfiles(), so multi-volume restores with the
'x' command don't work if you follow the instructions and supply
the tapes in reverse order. This appears to have been broken
by CSRG revision 5.33 tape.c (Oct 1992).
- The logic in getvol() for deciding how many records to skip after
the volume header was confused; sometimes it would skip too few
records and sometimes too many, leading to "resync restore"
warnings and missing files. Skip to the next header only when
the current action is not `USING'. Work around a dump bug that
sets c_count incorrectly in the volume header of the first tape.
Some of the problems here date back to at least 1991.
- Back out revision 1.23. This appeared to avoid warnings about
missing files in the 'rN' verification case, but it made the
problems with the 'x' command worse by stopping getvol() from
even attempting to find the first inode number on the newly
inserted tape. The bug it addressed is fixed by correcting the
skipping logic as described above.
- Save the value of `tpblksread' in case the wrong volume is
supplied, because it is incremented each time we read a volume
header. We already saved `blksread' for the same reson.
time_to_xxx() and xxx_to_time() functions. e.g. _time_to_xxx()
instead of time_to_xxx(), to make it more obvious that these are
stopgap functions & placemarkers and not meant to create a defacto
standard. They will eventually be replaced when a real standard
comes out of committee.
header for the case where sizeof(time_t) != sizeof(int). dumprestore.h
was embedding time_t when it should have been embedding int32_t.
Use time_to_time32() and time32_to_time() to convert between the
protocoll/file-format time and time_t.
corrects cases where restore would spew an infinite stream of
"Changing volumes on pipe input?" messages, or would loop waiting
for a response to the "set owner/mode for '.'" question.
PR: bin/14250
Reviewed by: dwmalone
