Commit Graph

161 Commits

Author SHA1 Message Date
bde
de7f82e077 Use VOP_ADVISE() with POSIX_FADV_DONTNEED instead of IO_DIRECT to
implement not double-caching for reads from vnode-backed md devices.
Use VOP_ADVISE() similarly instead of !IO_DIRECT unsimilarly for writes.
Add a "cache" option to mdconfig to allow changing the default of not
caching.

This depends on a recent commit to fix VOP_ADVISE().  A previous version
had optimizations for sequential i/o's (merge the i/o's and only uncache
for discontiguous i/o's and for full blocks), but optimizations and
knowledge of block boundaries belong in VOP_ADVISE().  Read-ahead should
also be handled better, by supporting it in md and discarding it in
VOP_ADVISE().

POSIX_FADV_DONTNEED is ignored by zfs, but so is IO_DIRECT.

POSIX_FADV_DONTNEED works better than IO_DIRECT if it is not ignored,
since it only discards from the buffer cache immediately, while
IO_DIRECT also discards from the page cache immediately.

IO_DIRECT was not used for writes since it was claimed to be too slow,
but most of the slowness for writes is from doing them synchronously by
default.  Non-synchronous writes still deadlock in many cases.

IO_DIRECT only has a special implementation for ffs reads with DIRECTIO
configured.  Otherwise, if it is not ignored than it uses the buffer and
page caches normally except for discarding everything after each i/o,
and then it has much the same overheads as POSIX_FADV_DONTNEED.  The
overheads for reading with ffs and DIRECTIO were similar in tests of md.

Reviewed by:	kib
2018-12-21 08:15:31 +00:00
bde
f15290be5b Fix missing (sub)options in usage message to prepare for adding a new one.
Reviewed by:	kib
2018-12-21 06:38:13 +00:00
phk
ed1e2ce23a Make (no)ro an alias for (no)readonly 2018-12-16 18:10:55 +00:00
pfg
7551d83c35 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
bdrewery
a598c4b809 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
sobomax
950c3938b7 Add ability to label md(4) devices.
This feature comes from the fact that we rely memory-backed md(4)
in our build process heavily. However, if the build goes haywire
the allocated resources (i.e. swap and memory-backed md(4)'s) need
to be purged. It is extremely useful to have ability to attach
arbitrary labels to each of the virtual disks so that they can
be identified and GC'ed if neecessary.

MFC after:	4 weeks
Differential Revision:	https://reviews.freebsd.org/D10457
2017-08-28 15:54:07 +00:00
ngie
d26727d972 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
ngie
734d081ed1 MFhead@r321912 2017-08-02 08:38:36 +00:00
stevek
aa552394db Add MD_VERIFY option to enable O_VERIFY in open for vnode type.
Add -o [no]verify option to mdconfig (and document in man page.)
Implement GEOM attribute MNT::verified to ask md if the backing vnode is
  verified.
Check for MNT::verified in cd9660 mount to flag the mount as MNT_VERIFIED if
  the underlying device has been verified.

Reviewed by:	rwatson
Approved by:	sjg (mentor)
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D2902
2017-05-31 21:18:11 +00:00
trasz
881423faa5 Make mdconfig(8) not segfault when the kernel returns garbage data.
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2017-02-19 16:08:58 +00:00
stevek
20012536fd For MD_PRELOAD type md(4) devices, if there is a file name in the preloaded
meta-data, copy it into the softc structure.

When returning md(4) device details to the caller, include the file name in
any MD_PRELOAD type devices if it is set (first character is not NUL.)

In mdconfig, for "preload" type md(4) devices, if there is file config
available, print it in the file column of the output.

Reviewed by:	brooks
Approved by:	sjg (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D9529
2017-02-13 17:44:07 +00:00
ngie
92100036c8 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
gjb
1c7e318a9a MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
bdrewery
aab40fdc3d DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
gjb
4e38aaedb8 Explicitly add more files to the 'runtime' package.
Sponsored by:	The FreeBSD Foundation
2016-02-09 20:19:31 +00:00
gjb
a6998ad84f First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
ngie
dd1f618367 Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
trasz
8eb11bc852 Add 'p' postfix to mdconfig(8).
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2015-10-10 09:37:31 +00:00
trasz
c05bbe4d23 It's 2015, and some people are still trying to use fdisk and then
go asking what debug flags to set for GEOM to make it work.  Advice
them to use gpart(8) instead.

Something similar should probably done with disklabel,
but I need to rewrite the disklabel examples first.

Reviewed by:	wblock@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3315
2015-09-02 14:08:43 +00:00
trasz
06d7ad38a7 Whoops, wrong flag.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2015-08-06 07:49:34 +00:00
trasz
041feea99f Tweak mdconfig(8) manual page, in particular revise the EXAMPLES
section.  This removes stuff that doesn't really belong there,
and simplifies examples for the basic operations.

Reviewed by:	wblock@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3198
2015-08-06 07:47:13 +00:00
sjg
008d7c831f Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
sjg
75a137820d dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
sjg
65145fa4c8 Merge sync of head 2015-05-27 01:19:58 +00:00
ngie
244d05e746 Convert sbin/mdconfig/tests from prove format tests to ATF format tests
As a side effect...
1. The tests now checks for the root user before continuing with kyua, which is
   more visible than the test being skipped with the TAP protocol
2. The tests work with devices that aren't /dev/md0 by caching the device
   attached during the test to a file, and later use the cached information to
   detach the device in the cleanup routine
3. The tests no longer require perl to run

MFC after: 1 week
PR: 191191
Sponsored by: EMC / Isilon Storage Division
2014-11-27 06:04:00 +00:00
bapt
6fdc031751 Convert sbin/ to LIBADD
Reduce overlinking
2014-11-25 11:23:12 +00:00
sjg
d7cd1d425c Merge head from 7/28 2014-08-19 06:50:54 +00:00
bapt
42044eb342 use .Mt to mark up email addresses consistently (part1)
PR:		191174
Submitted by:	Franco Fichtner  <franco@lastsummer.de>
2014-06-20 09:40:43 +00:00
sjg
5860f0d106 Updated dependencies 2014-05-16 14:09:51 +00:00
sjg
1a7e48acf1 Updated dependencies 2014-05-10 05:16:28 +00:00
sjg
ed3fc70bf5 Merge from head 2014-05-08 23:54:15 +00:00
imp
2118f42afd Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.
2014-05-06 04:22:01 +00:00
sjg
5e568154a0 Merge head 2014-04-28 07:50:45 +00:00
jmmv
4dc52363bb Fix path to the run.pl script to let these tests run.
Submitted by:	Peel, Casey
Obtained from:	freebsd-testing
2014-03-27 13:15:22 +00:00
jmmv
0ee5980c22 Add Makefile missed in r263220. 2014-03-16 02:32:44 +00:00
jmmv
f965a606e8 Migrate tools/regression/sbin/ to the new tests layout.
Pretty much all that this change does is shuffles the code around and hooks
it into the regular build.  The code of the old tests has not changed.
2014-03-16 02:07:08 +00:00
eadler
118094e60b multiple: Remove 3rd clause from BSD license where approved by the
regents and renumber.

This patch skips files in contrib/ and crypto/

Acked by:	imp
Discussed with:	emaste
2014-03-14 03:07:51 +00:00
trasz
de454232c8 Add "null" backend to mdconfig(8). This does exactly what the name
suggests, and is somewhat useful for benchmarking.

MFC after:	1 month
No objections from:	kib
Sponsored by:	The FreeBSD Foundation
2013-12-04 07:38:23 +00:00
jmg
a9c5382253 bump date forgotten in r257165 2013-10-26 18:23:43 +00:00
jmg
2a361a8088 Document that -a will output the device name when -u is not specified..
when -u is specified it is not...

update the docs to say that you can use full device names w/ -u, and
update the examples...

Submitted by:	#vbsdcon
MFC after:	3 days
2013-10-26 15:05:27 +00:00
hrs
3beb1e4f49 Return 0 if:
1. "-u N" specified, no -f, and mdN found,
  2. no -u, "-f /pathname" specified, and mdN associated with /pathname found,
  3. "-u N" specified, "-f /pathname" specified, and both of them found,
  4. "-l" specified and no -f,
  5. "-l" specified, "-f /pathname" specified, and /pathname found.

otherwise return -1.

Spotted by:	Julian H. Stacey
2013-10-24 01:06:44 +00:00
sjg
62bb106222 Merge from head 2013-09-05 20:18:59 +00:00
delphij
e07cc71980 Resolve fflag with realpath().
MFC after:	2 weeks
2013-07-31 07:06:58 +00:00
delphij
1559c2306b When listing with -f, skip all memory disks that are not vnode-backed.
Noticed by:	kevlo
MFC after:	3 days
2013-07-02 19:42:47 +00:00
joel
fb34da0b96 mdoc: remove EOL whitespace. 2013-06-29 16:05:44 +00:00
hrs
0743eae837 - Use length of _PATH_DEV instead of a hardcoded value[*].
- Use MD_NAME for "md".

Pointed out by:	trasz [*]
MFC after:	1 week
2013-06-21 01:47:22 +00:00
hrs
53795e1374 eturn -1 when the specified backing store file is not found in the md
device list.

MFC after:	1 week
2013-06-20 17:01:02 +00:00
hrs
7d32f7dfe8 - Add "-f file" support to listing mode (-l). When a -f option is
specified, only md(4) devices which have the specified file as backing
  store are displayed.
- Use MD_NAME instead of "md".
- Use _PATH_DEV instead of "/dev/".

MFC after:	1 week
2013-06-20 14:30:16 +00:00
sjg
6d37b86f2b Updated dependencies 2013-03-11 17:21:52 +00:00
sjg
0ee5295509 Updated dependencies 2013-02-16 01:23:54 +00:00