Commit Graph

77 Commits

Author SHA1 Message Date
John Baldwin
58862c0bea fstyp: Remove __packed from struct exfat_de_label.
This fixes a -Waddress-of-packed-member warning about a possibly
unaligned pointer from GCC 9 when calling convert_label().

__packed has to be removed from struct exfat_dirent as well to fix an
alignment warning when casting from a struct exfat_dirent pointer to a
struct exfat_de_label pointer.

Reviewed by:	cem
Differential Revision:	https://reviews.freebsd.org/D32144
2022-01-31 17:33:31 -08:00
Martin Matuska
e92ffd9b62 zfs: merge openzfs/zfs@17b2ae0b2 (master) into main
Notable upstream pull request merges:
  #12766 Fix error propagation from lzc_send_redacted
  #12805 Updated the lz4 decompressor
  #12851 FreeBSD: Provide correct file generation number
  #12857 Verify dRAID empty sectors
  #12874 FreeBSD: Update argument types for VOP_READDIR
  #12896 Reduce number of arc_prune threads
  #12934 FreeBSD: Fix zvol_*_open() locking
  #12947 lz4: Cherrypick fix for CVE-2021-3520
  #12961 FreeBSD: Fix leaked strings in libspl mnttab
  #12964 Fix handling of errors from dmu_write_uio_dbuf() on FreeBSD
  #12981 Introduce a flag to skip comparing the local mac when raw sending
  #12985 Avoid memory allocations in the ARC eviction thread

Obtained from:	OpenZFS
OpenZFS commit:	17b2ae0b24
2022-01-22 23:05:15 +01:00
Ed Maste
3513df4bcc fstyp: fix build WITHOUT_ICONV
Reported by:	Michael Dexter, Build Option Survey
Sponsored by:	The FreeBSD Foundation
2021-11-02 14:54:45 -04:00
Jessica Clarke
63d24336fd Fix off-by-one error in msdosfs FAT32 volume label copying
I dropped the + 1 from the other two instances in each file but failed
to do so for this one, resulting in a more egregious buffer overread
than the one I was fixing (since the read character ended up in the
output if there was space).

Reported by:	Jenkins
Fixes:	34fb1c133c ("Fix intra-object buffer overread for labeled msdosfs volumes")
2021-10-28 01:01:00 +01:00
Jessica Clarke
34fb1c133c Fix intra-object buffer overread for labeled msdosfs volumes
Volume labels, like directory entries, are padded with spaces and so
have no NUL terminator. Whilst the MIN for the dsize argument to strlcpy
ensures that the copy does not overflow the destination, strlcpy is
defined to return the number of characters in the source string,
regardless of the provided dsize, and so keeps reading until it finds a
NUL, which likely exists somewhere within the following fields, but On
CHERI with the subobject bounds enabled in the compiler this buffer
overread will be detected and trap with a bounds violation.

Found by:	CHERI
Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D32579
2021-10-27 18:38:37 +01:00
Piotr Pawel Stefaniak
da0c0e0121 fstyp: bump WARNS to default and work around warnings
Differential Revision:	https://reviews.freebsd.org/D31588
2021-09-19 13:52:12 +02:00
Piotr Pawel Stefaniak
0e92585cde fstyp: add BeFS support
A simple support for detecting BeFS (BeOS) filesystem

Submitted by:	Miguel Gocobachi
Differential Revision:	https://reviews.freebsd.org/D29917
2021-08-17 18:16:07 +02:00
Robert Wing
91f251b2ab fstyp(8): define HAVE_ZFS macro when built with zfs
The HAVE_ZFS macro was dropped from the Makefile during the OpenZFS
merge, bring it back.

While here, drop unused WARNS setting.

PR:             255616
Reported by:    Michael Büker <freebsd@michael-bueker.de>
Submitted by:   Michael Büker <freebsd@michael-bueker.de>
Fixes:          9e5787d228
Reviewed by:    markj
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D30221
2021-05-14 13:00:24 -08:00
Conrad Meyer
ddf6115613 fstyp(8): fix exfat detection
In the presence of high-level errors (spec violations, bad boot blocks
checksum), report non-detection instead of detection.

PR:	252787 (related, but does not fully address)
2021-01-17 11:55:06 -08:00
Edward Tomasz Napierala
bce7ee9d41 Drop "All rights reserved" from all my stuff. This includes
Foundation copyrights, approved by emaste@.  It does not include
files which carry other people's copyrights; if you're one
of those people, feel free to make similar change.

Reviewed by:	emaste, imp, gbe (manpages)
Differential Revision:	https://reviews.freebsd.org/D26980
2020-10-28 13:46:11 +00:00
Matt Macy
9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Pedro F. Giffuni
0d2913adc3 usr.sbin/fstyp: Fix incorrect pfs_type test in ondisk inode
"ipdata.meta.pfs_type & HAMMER2_PFSTYPE_SUPROOT" happened to have
the same result (except HAMMER2_PFSTYPE_DUMMY could also match).

Obtained from: Dragonfly (git 29e6489bbd4f8e237c9c17b300ac8b711f36770)
2020-08-20 05:18:08 +00:00
Li-Wen Hsu
bfd735d96e Revert r362390, those tests are fixed by r362418
PR:		247425
Sponsored by:	The FreeBSD Foundation
2020-06-23 19:14:38 +00:00
Li-Wen Hsu
865c7b713c Skip ufs related tests in fstyp(8) and makefs(8) temporarily
They are failing after r362358 and r362359.

PR:		247425
Sponsored by:	The FreeBSD Foundation
2020-06-19 17:32:30 +00:00
Kirk McKusick
34816cb9ae Move the pointers stored in the superblock into a separate
fs_summary_info structure. This change was originally done
by the CheriBSD project as they need larger pointers that
do not fit in the existing superblock.

This cleanup of the superblock eases the task of the commit
that immediately follows this one.

Suggested by: brooks
Reviewed by:  kib
PR:           246983
Sponsored by: Netflix
2020-06-19 01:02:53 +00:00
Pedro F. Giffuni
9d529ed898 fstyp: sync HAMMER1/2 detection support with DragonFly BSD.
Submitted by:	Tomohiro Kusumi
PR:		243929
2020-02-06 18:04:15 +00:00
Eric van Gyzen
5a12815327 fstyp hammer2: remove dead code
best_i will always be >= 0, so remove code to test otherwise.

Reported by:	Coverity
CID:		1412244
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D23159
2020-01-13 22:36:29 +00:00
Eric van Gyzen
0ab52bd3eb fstyp hammer: use strlcpy
Use strlcpy to guarantee NUL termination.  Due to this, there is
no need for strncmp; simply use strcmp.

Reported by:	Coverity
CID:		1412242
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D23159
2020-01-13 22:33:48 +00:00
Adrian Chadd
350bfebba9 Make this compile under external gcc toolchain if WITH_ICONV isn't defined.
This quietens a bunch of unused variable warnings that are treated as errors.
2019-12-29 06:59:09 +00:00
Pedro F. Giffuni
509798ea65 sbin/fstyp: recgonize Dragonfly's hammer and hammer2.
This is based on DragonFly's implementation from about 2019-09-13. It
only contains the basic code and header information to identify the
disks.

Relnotes:		yes
Differential Revision:	https://reviews.freebsd.org/D13369
2019-12-24 19:00:20 +00:00
Conrad Meyer
5ab1cb52b2 fstyp(8): Fix WITHOUT_ICONV build
Reported by:	olivier
2019-12-23 20:23:02 +00:00
Conrad Meyer
e41d62768d fstyp(8): Detect APFS containers
APFS NXSBs are more like slices (or zvols?) than individual filesystem,
but go ahead and detect them nicely as well.
2019-12-23 05:43:18 +00:00
Conrad Meyer
50c59bbb53 fstyp(8): Detect HFS+ / HFSX volumes 2019-12-23 05:43:01 +00:00
Conrad Meyer
ec80d2eedd fstyp(8): Use iconv(3) to convert NTFS vol labels correctly
Rather than hackily extracting only the ASCII subset of UTF-16LE, go ahead
and convert the label to the user's locale correctly.
2019-12-23 02:41:13 +00:00
Conrad Meyer
85b4c344c8 fstyp(8): Show exFAT volume labels with -l flag
exfat is fundamentally the same design as fat32.  The superblock differs
marginally, and there are some additional optional features irrelevant to
fstype(8); the structure of dirents has changed slightly to enable, among
other things, larger files; the directory entries are no longer DOS 8.3
ASCII or local 8-bit encoding, but instead explicitly UCS-2-LE.

(As a result, this change uses iconv to convert a found exfat volume label
to the user's locale.)

Locating the volume label is identical to FAT32: locate the root directory
and walk through dirents until you find a volume label.  Like FAT32, follow
the FAT chain between root directory clusters as necessary.

PR:		242225
Reported by:	Victor Sudakov <vas AT sibptus.ru>
2019-12-22 03:19:17 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Simon J. Gerraty
5ab1c5846f Add Makefile.depend.options
Leaf directories that have dependencies impacted
by options need a Makefile.depend.options file
to avoid churn in Makefile.depend

DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc
can be set in local.dirdeps-options.mk
which can add to those set in Makefile.depend.options

See share/mk/dirdeps-options.mk

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22469
2019-12-11 17:37:37 +00:00
Alan Somers
809a8352dd Drop "All rights reserved" from the files I own
Also, add SPDX tags where needed.

MFC after:	2 weeks
2019-03-11 22:23:56 +00:00
Kirk McKusick
fb14e73cb4 Normally when an attempt is made to mount a UFS/FFS filesystem whose
superblock has a check-hash error, an error message noting the
superblock check-hash failure is printed and the mount fails. The
administrator then runs fsck to repair the filesystem and when
successful, the filesystem can once again be mounted.

This approach fails if the filesystem in question is a root filesystem
from which you are trying to boot. Here, the loader fails when trying
to access the filesystem to get the kernel to boot. So it is necessary
to allow the loader to ignore the superblock check-hash error and make
a best effort to read the kernel. The filesystem may be suffiently
corrupted that the read attempt fails, but there is no harm in trying
since the loader makes no attempt to write to the filesystem.

Once the kernel is loaded and starts to run, it attempts to mount its
root filesystem. Once again, failure means that it breaks to its prompt
to ask where to get its root filesystem. Unless you have an alternate
root filesystem, you are stuck.

Since the root filesystem is initially mounted read-only, it is
safe to make an attempt to mount the root filesystem with the failed
superblock check-hash. Thus, when asked to mount a root filesystem
with a failed superblock check-hash, the kernel prints a warning
message that the root filesystem superblock check-hash needs repair,
but notes that it is ignoring the error and proceeding. It does
mark the filesystem as needing an fsck which prevents it from being
enabled for writing until fsck has been run on it. The net effect
is that the reboot fails to single user, but at least at that point
the administrator has the tools at hand to fix the problem.

Reported by:    Rick Macklem (rmacklem@)
Discussed with: Warner Losh (imp@)
Sponsored by:   Netflix
2018-12-06 00:09:39 +00:00
Alexander Motin
7fca1b93c4 Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.
2018-08-02 18:55:55 +00:00
Mariusz Zaborski
1afab1fe21 Use capsicum helpers in fstype and ctld.
Reviewed by:	trasz
2018-07-15 17:21:19 +00:00
Xin LI
b6f7731dba Remove "All rights reserved" from my files.
See r333391 for the rationale.

MFC after:	1 week
2018-05-10 06:41:08 +00:00
Kirk McKusick
dffce2150e Refactoring of reading and writing of the UFS/FFS superblock.
Specifically reading is done if ffs_sbget() and writing is done
in ffs_sbput(). These functions are exported to libufs via the
sbget() and sbput() functions which then used in the various
filesystem utilities. This work is in preparation for adding
subperblock check hashes.

No functional change intended.

Reviewed by: kib
2018-01-26 00:58:32 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Enji Cooper
d511b20a69 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Andriy Gapon
d6609e6a12 fstyp: move sys/ include path after zfs include paths
The reason is that FreeBSD refcount.h shadows ZFS refcount.h and that
will lead to a build error after a planned import of the ARC buf data
scatter-ization.
It's possible that some day we will have an opposite problem where
a ZFS header would shadow an essential FreeBSD header.
So, we need to think about a better long term solution.

Discussed with:	allanjude
MFC after:	17 days
2017-06-20 16:45:48 +00:00
Edward Tomasz Napierala
56226f9c30 Make fstyp(8) recognize exFAT even without the -u option.
While it's not directly mountable with mount(8), it's something that's
mountable - differently from GELI or zpools.

MFC after:	2 weeks
2017-04-26 19:34:41 +00:00
Edward Tomasz Napierala
4bcdaf7651 Improve formatting.
MFC after:	2 weeks
2017-03-19 16:17:42 +00:00
Alan Somers
f5c4d58f4a Fix harmless mismerge from r286964
MFC after:	3 days
Sponsored by:	Spectra Logic Corp
2017-03-09 16:22:21 +00:00
Enji Cooper
64a0982bee usr.sbin: normalize paths using SRCTOP-relative paths or :H when possible
This simplifies make logic/output

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 11:38:03 +00:00
Enji Cooper
0372faed69 Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-01 04:00:17 +00:00
Conrad Meyer
4a6ace7328 Fix broken fstyp exfat testcase
Introduced in r312010.

It helps to read the documentation before trying to test something.
2017-01-14 01:08:04 +00:00
Conrad Meyer
b632b00087 fstyp.8: Move initial exFAT blurb to the -u section
Didn't notice the second list in r312003.

Reported by:	trasz@
2017-01-13 16:46:01 +00:00
Enji Cooper
ef7ff3e308 Use dfr-01-xfat.img.bz2, not dfr-01-xfat.dd.bz2 (the latter
case was the full disk image from the website, which was never
checked in to svn)

Regardless, the testcase still fails

PR:		214908
Sponsored by:	Dell EMC Isilon
2017-01-13 04:04:48 +00:00
Enji Cooper
6f81c4d9bb Add testcase for exFAT that currently fails
Disk image obtained from:
http://www.cfreds.nist.gov/dfr-images/dfr-01-xfat.dd.bz2 -- was ripped off the
first GPT partition and verified to be a FAT-like partition with file(1)/hexdump.

This testcase currently fails

PR:		214908
Sponsored by:	Dell EMC Isilon
2017-01-13 04:02:09 +00:00
Enji Cooper
cebf8a2ca7 Add license preamble for r286964; credit to asomers
While here, clean up trailing whitespace

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-01-13 03:42:51 +00:00
Conrad Meyer
e266075649 fstyp(8): Detect exFAT filesystems
Simply detect the exFAT filesystem name in the Volume Boot Record
(superblock).

PR:		214908
Reported by:	<vermaden at interia.pl>
2017-01-13 02:12:58 +00:00
Ruslan Bukin
f2567787aa fstyp tests comes with pre-created EL filesystems,
but fstyp cannot detect EL filesystem on EB machine,
so exclude test files from distribution and skip the
test.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-10-18 12:58:17 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00