Commit Graph

161 Commits

Author SHA1 Message Date
Bruce Evans
9e5ed8593f Use VOP_ADVISE() with POSIX_FADV_DONTNEED instead of IO_DIRECT to
implement not double-caching for reads from vnode-backed md devices.
Use VOP_ADVISE() similarly instead of !IO_DIRECT unsimilarly for writes.
Add a "cache" option to mdconfig to allow changing the default of not
caching.

This depends on a recent commit to fix VOP_ADVISE().  A previous version
had optimizations for sequential i/o's (merge the i/o's and only uncache
for discontiguous i/o's and for full blocks), but optimizations and
knowledge of block boundaries belong in VOP_ADVISE().  Read-ahead should
also be handled better, by supporting it in md and discarding it in
VOP_ADVISE().

POSIX_FADV_DONTNEED is ignored by zfs, but so is IO_DIRECT.

POSIX_FADV_DONTNEED works better than IO_DIRECT if it is not ignored,
since it only discards from the buffer cache immediately, while
IO_DIRECT also discards from the page cache immediately.

IO_DIRECT was not used for writes since it was claimed to be too slow,
but most of the slowness for writes is from doing them synchronously by
default.  Non-synchronous writes still deadlock in many cases.

IO_DIRECT only has a special implementation for ffs reads with DIRECTIO
configured.  Otherwise, if it is not ignored than it uses the buffer and
page caches normally except for discarding everything after each i/o,
and then it has much the same overheads as POSIX_FADV_DONTNEED.  The
overheads for reading with ffs and DIRECTIO were similar in tests of md.

Reviewed by:	kib
2018-12-21 08:15:31 +00:00
Bruce Evans
e6f6d8853c Fix missing (sub)options in usage message to prepare for adding a new one.
Reviewed by:	kib
2018-12-21 06:38:13 +00:00
Poul-Henning Kamp
96a3750174 Make (no)ro an alias for (no)readonly 2018-12-16 18:10:55 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Maxim Sobolev
f7ca2bbe44 Add ability to label md(4) devices.
This feature comes from the fact that we rely memory-backed md(4)
in our build process heavily. However, if the build goes haywire
the allocated resources (i.e. swap and memory-backed md(4)'s) need
to be purged. It is extremely useful to have ability to attach
arbitrary labels to each of the virtual disks so that they can
be identified and GC'ed if neecessary.

MFC after:	4 weeks
Differential Revision:	https://reviews.freebsd.org/D10457
2017-08-28 15:54:07 +00:00
Enji Cooper
d511b20a69 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Stephen J. Kiernan
9a81ba0f24 Add MD_VERIFY option to enable O_VERIFY in open for vnode type.
Add -o [no]verify option to mdconfig (and document in man page.)
Implement GEOM attribute MNT::verified to ask md if the backing vnode is
  verified.
Check for MNT::verified in cd9660 mount to flag the mount as MNT_VERIFIED if
  the underlying device has been verified.

Reviewed by:	rwatson
Approved by:	sjg (mentor)
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D2902
2017-05-31 21:18:11 +00:00
Edward Tomasz Napierala
30545786bb Make mdconfig(8) not segfault when the kernel returns garbage data.
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2017-02-19 16:08:58 +00:00
Stephen J. Kiernan
d2e6391342 For MD_PRELOAD type md(4) devices, if there is a file name in the preloaded
meta-data, copy it into the softc structure.

When returning md(4) device details to the caller, include the file name in
any MD_PRELOAD type devices if it is set (first character is not NUL.)

In mdconfig, for "preload" type md(4) devices, if there is file config
available, print it in the file column of the output.

Reviewed by:	brooks
Approved by:	sjg (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D9529
2017-02-13 17:44:07 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Glen Barber
406d87b1c3 Explicitly add more files to the 'runtime' package.
Sponsored by:	The FreeBSD Foundation
2016-02-09 20:19:31 +00:00
Glen Barber
43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Enji Cooper
b2d48be1bc Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
Edward Tomasz Napierala
d5b0fb47eb Add 'p' postfix to mdconfig(8).
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2015-10-10 09:37:31 +00:00
Edward Tomasz Napierala
b8c19fd719 It's 2015, and some people are still trying to use fdisk and then
go asking what debug flags to set for GEOM to make it work.  Advice
them to use gpart(8) instead.

Something similar should probably done with disklabel,
but I need to rewrite the disklabel examples first.

Reviewed by:	wblock@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3315
2015-09-02 14:08:43 +00:00
Edward Tomasz Napierala
3bb6078985 Whoops, wrong flag.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2015-08-06 07:49:34 +00:00
Edward Tomasz Napierala
6be30a3001 Tweak mdconfig(8) manual page, in particular revise the EXAMPLES
section.  This removes stuff that doesn't really belong there,
and simplifies examples for the basic operations.

Reviewed by:	wblock@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3198
2015-08-06 07:47:13 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Enji Cooper
66ca0717df Convert sbin/mdconfig/tests from prove format tests to ATF format tests
As a side effect...
1. The tests now checks for the root user before continuing with kyua, which is
   more visible than the test being skipped with the TAP protocol
2. The tests work with devices that aren't /dev/md0 by caching the device
   attached during the test to a file, and later use the cached information to
   detach the device in the cleanup routine
3. The tests no longer require perl to run

MFC after: 1 week
PR: 191191
Sponsored by: EMC / Isilon Storage Division
2014-11-27 06:04:00 +00:00
Baptiste Daroussin
13eb765f2d Convert sbin/ to LIBADD
Reduce overlinking
2014-11-25 11:23:12 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Baptiste Daroussin
bd0891ceb3 use .Mt to mark up email addresses consistently (part1)
PR:		191174
Submitted by:	Franco Fichtner  <franco@lastsummer.de>
2014-06-20 09:40:43 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Simon J. Gerraty
76b28ad6ab Updated dependencies 2014-05-10 05:16:28 +00:00
Simon J. Gerraty
cc3f4b9965 Merge from head 2014-05-08 23:54:15 +00:00
Warner Losh
c6063d0da8 Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.
2014-05-06 04:22:01 +00:00
Simon J. Gerraty
3b8f084595 Merge head 2014-04-28 07:50:45 +00:00
Julio Merino
42cd4d514d Fix path to the run.pl script to let these tests run.
Submitted by:	Peel, Casey
Obtained from:	freebsd-testing
2014-03-27 13:15:22 +00:00
Julio Merino
1110970b4c Add Makefile missed in r263220. 2014-03-16 02:32:44 +00:00
Julio Merino
c68de7484a Migrate tools/regression/sbin/ to the new tests layout.
Pretty much all that this change does is shuffles the code around and hooks
it into the regular build.  The code of the old tests has not changed.
2014-03-16 02:07:08 +00:00
Eitan Adler
dda5b39711 multiple: Remove 3rd clause from BSD license where approved by the
regents and renumber.

This patch skips files in contrib/ and crypto/

Acked by:	imp
Discussed with:	emaste
2014-03-14 03:07:51 +00:00
Edward Tomasz Napierala
0efd9bfd47 Add "null" backend to mdconfig(8). This does exactly what the name
suggests, and is somewhat useful for benchmarking.

MFC after:	1 month
No objections from:	kib
Sponsored by:	The FreeBSD Foundation
2013-12-04 07:38:23 +00:00
John-Mark Gurney
52a522a283 bump date forgotten in r257165 2013-10-26 18:23:43 +00:00
John-Mark Gurney
8cde1e8a69 Document that -a will output the device name when -u is not specified..
when -u is specified it is not...

update the docs to say that you can use full device names w/ -u, and
update the examples...

Submitted by:	#vbsdcon
MFC after:	3 days
2013-10-26 15:05:27 +00:00
Hiroki Sato
4103b0b025 Return 0 if:
1. "-u N" specified, no -f, and mdN found,
  2. no -u, "-f /pathname" specified, and mdN associated with /pathname found,
  3. "-u N" specified, "-f /pathname" specified, and both of them found,
  4. "-l" specified and no -f,
  5. "-l" specified, "-f /pathname" specified, and /pathname found.

otherwise return -1.

Spotted by:	Julian H. Stacey
2013-10-24 01:06:44 +00:00
Simon J. Gerraty
d1d0158641 Merge from head 2013-09-05 20:18:59 +00:00
Xin LI
cab9f71ffe Resolve fflag with realpath().
MFC after:	2 weeks
2013-07-31 07:06:58 +00:00
Xin LI
fa3483e67c When listing with -f, skip all memory disks that are not vnode-backed.
Noticed by:	kevlo
MFC after:	3 days
2013-07-02 19:42:47 +00:00
Joel Dahl
a9ca77e055 mdoc: remove EOL whitespace. 2013-06-29 16:05:44 +00:00
Hiroki Sato
db3fdd7f63 - Use length of _PATH_DEV instead of a hardcoded value[*].
- Use MD_NAME for "md".

Pointed out by:	trasz [*]
MFC after:	1 week
2013-06-21 01:47:22 +00:00
Hiroki Sato
dc017e3401 eturn -1 when the specified backing store file is not found in the md
device list.

MFC after:	1 week
2013-06-20 17:01:02 +00:00
Hiroki Sato
b78ae23fdd - Add "-f file" support to listing mode (-l). When a -f option is
specified, only md(4) devices which have the specified file as backing
  store are displayed.
- Use MD_NAME instead of "md".
- Use _PATH_DEV instead of "/dev/".

MFC after:	1 week
2013-06-20 14:30:16 +00:00
Simon J. Gerraty
7cf3a1c6b2 Updated dependencies 2013-03-11 17:21:52 +00:00
Simon J. Gerraty
f5f7c05209 Updated dependencies 2013-02-16 01:23:54 +00:00