Commit Graph

3893 Commits

Author SHA1 Message Date
Brooks Davis
6469bdcdb6 Move most of the contents of opt_compat.h to opt_global.h.
opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c.  A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by:	kib, cem, jhb, jtl
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14941
2018-04-06 17:35:35 +00:00
Benno Rice
7acb51f681 Add isoboot(8) for booting BIOS systems from HDDs containing ISO images.
This is part of a project for adding the ability to create hybrid CD/USB boot
images. In the BIOS case when booting from something that isn't a CD we need
some extra boot code to actually find our next stage (loader) within an
ISO9660 filesystem. This code will reside in a GPT partition (similar to
gptboot(8) from which it is derived) and looks for /boot/loader in an
ISO9660 filesystem on the image.

Reviewed by:	imp
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14914
2018-04-05 19:40:46 +00:00
Ed Maste
d8ba45e213 Revert r313780 (UFS_ prefix) 2018-03-17 12:59:55 +00:00
Ed Maste
1e2b9afca9 Prefix UFS symbols with UFS_ to reduce namespace pollution
Followup to r313780.  Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by:	kib
Reviewed by:	kib, mckusick
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9623
2018-03-17 01:48:27 +00:00
Hajimu UMEMOTO
9f5fab694c Fix Bad file descriptor error.
MFC after:	1 week
2018-03-09 04:45:24 +00:00
Eitan Adler
40301da899 sys/fuse: fix off by one error
Reported by:	Ilja Van Sprundel <ivansprundel@ioactive.com>
Reported by:	Domagoj Stolfa <domagoj.stolfa@gmail.com>
2018-03-03 20:42:39 +00:00
Pedro F. Giffuni
7cbd6d338e {ext2|ufs}_readdir: Avoid setting negative ncookies.
ncookies cannot be negative or the allocator will fail. This should only
happen if a caller is very broken but we can still try to survive the
event.

We should probably also verify for uio_resid > MAXPHYS but in that case
it is not clear that just clipping the ncookies value is an adequate
response.

MFC after:	2 weeks
2018-02-06 22:38:19 +00:00
Jeff Roberson
e2068d0bcd Use per-domain locks for vm page queue free. Move paging control from
global to per-domain state.  Protect reservations with the free lock
from the domain that they belong to.  Refactor to make vm domains more
of a first class object.

Reviewed by:    markj, kib, gallatin
Tested by:      pho
Sponsored by:   Netflix, Dell/EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D14000
2018-02-06 22:10:07 +00:00
Pedro F. Giffuni
fdc154e44a ext2fs: remove EXT4F_RO_INCOMPAT_SUPP
This was a hack to be able to mount ext4 filesystems read-only while not
supporting all the features. We now support all those features so it
doesn't make sense to keep the undocumented hack.

Discussed with:	fsu
2018-02-05 15:14:01 +00:00
Pedro F. Giffuni
f86f5cd406 ext2fs: Cleanup variable assignments for extents.
Delay the initialization of variables until the are needed.

In the case of ext4_ext_rm_leaf(), make sure 'error' value is not
undefined.

Reported by:		Clang's static analyzer
Differential Revision:	https://reviews.freebsd.org/D14193
2018-02-05 14:30:27 +00:00
Fedor Uporov
7c4fa61e6f Fix mistake in case of zeroed inode check.
Reported by:	pho
MFC after:	6 months
2018-01-29 22:15:46 +00:00
Fedor Uporov
c0f16c65cd Add flex_bg/meta_bg features RW support.
Reviewed by:    pfg
MFC after:      6 months

Differential Revision:    https://reviews.freebsd.org/D13964
2018-01-29 21:54:13 +00:00
Pedro F. Giffuni
040fb18b60 Revert r328479:
{ext2|ufs}_readdir: Set limit on valid ncookies values.

We aren't allowed to set resid like this.

Pointed out by:	kib, imp
2018-01-27 16:34:00 +00:00
Pedro F. Giffuni
ee233ab975 {ext2|ufs}_readdir: Set limit on valid ncookies values.
Sanitize the values that will be assigned to ncookies so that we ensure
they are sane and we can handle them.

Let ncookies signed as it was before r328346. The valid range is such
that unsigned values are not required and we are not able to avoid at
least one cast anyways.

Hinted by:	bde
2018-01-27 15:33:52 +00:00
Conrad Meyer
b97b91b547 nfs: Remove NFSSOCKADDRALLOC, NFSSOCKADDRFREE macros
They were just thin wrappers over malloc(9) w/ M_ZERO and free(9).

Discussed with:	rmacklem, markj
Sponsored by:	Dell EMC Isilon
2018-01-25 22:38:39 +00:00
Conrad Meyer
222daa421f style: Remove remaining deprecated MALLOC/FREE macros
Mechanically replace uses of MALLOC/FREE with appropriate invocations of
malloc(9) / free(9) (a series of sed expressions).  Something like:

* MALLOC(a, b, ... -> a = malloc(...
* FREE( -> free(
* free((caddr_t) -> free(

No functional change.

For now, punt on modifying contrib ipfilter code, leaving a definition of
the macro in its KMALLOC().

Reported by:	jhb
Reviewed by:	cy, imp, markj, rmacklem
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D14035
2018-01-25 22:25:13 +00:00
Pedro F. Giffuni
4ee8f6457c Minor style issue introduced in r328346.
Pointed by:	bde
2018-01-25 18:01:46 +00:00
Pedro F. Giffuni
a94a2945be ext2fs|ufs:Unsign some values related to allocation.
When allocating memory through malloc(9), we always expect the amount of
memory requested to be unsigned as a negative value would either stand for
an error or an overflow.
Unsign some values, found when considering the use of mallocarray(9), to
avoid unnecessary casting. Also consider that indexes should be of
at least the same size/type as the upper limit they pretend to index.

MFC after:	2 weeks
2018-01-24 17:58:48 +00:00
Pedro F. Giffuni
f9834d101a Revert r327781, r328093, r328056:
ufs|ext2fs: Revert uses of mallocarray(9).

These aren't really useful: drop them.
Variable unsigning will be brought again later.
2018-01-24 16:44:57 +00:00
Edward Tomasz Napierala
18dffafade Add SPDX tags to autofs(5).
MFC after:	2 weeks
2018-01-24 16:40:26 +00:00
Pedro F. Giffuni
7d84ca677f extfs: Remove unused variables.
Found by:	scan-build
Reviewed by:	fsu
Differential Revision:	https://reviews.freebsd.org/D14017
2018-01-23 14:17:04 +00:00
Pedro F. Giffuni
d821d36419 Unsign some values related to allocation.
When allocating memory through malloc(9), we always expect the amount of
memory requested to be unsigned as a negative value would either stand for
an error or an overflow.
Unsign some values, found when considering the use of mallocarray(9), to
avoid unnecessary casting. Also consider that indexes should be of
at least the same size/type as the upper limit they pretend to index.

MFC after:	3 weeks
2018-01-22 02:08:10 +00:00
Pedro F. Giffuni
ac2fffa4b7 Revert r327828, r327949, r327953, r328016-r328026, r328041:
Uses of mallocarray(9).

The use of mallocarray(9) has rocketed the required swap to build FreeBSD.
This is likely caused by the allocation size attributes which put extra pressure
on the compiler.

Given that most of these checks are superfluous we have to choose better
where to use mallocarray(9). We still have more uses of mallocarray(9) but
hopefully this is enough to bring swap usage to a reasonable level.

Reported by:	wosch
PR:		225197
2018-01-21 15:42:36 +00:00
Emmanuel Vadot
becffb36cc nfs: Do not printf each time a lock structure is freed during module unload
There can be a lot of those structures and printing a line each time we free
one on module unload.

MFC after:	3 days
2018-01-18 15:28:49 +00:00
John Baldwin
b1288166e0 Use long for the last argument to VOP_PATHCONF rather than a register_t.
pathconf(2) and fpathconf(2) both return a long.  The kern_[f]pathconf()
functions now accept a pointer to a long value rather than modifying
td_retval directly.  Instead, the system calls explicitly store the
returned long value in td_retval[0].

Requested by:	bde
Reviewed by:	kib
Sponsored by:	Chelsio Communications
2018-01-17 22:36:58 +00:00
Pedro F. Giffuni
9703326e8f ext2fs: use mallocarray(9).
Focus on code where we are doing multiplications within malloc(9). These
are not likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.
2018-01-16 19:29:32 +00:00
Pedro F. Giffuni
d48d1a6464 nfsclient: make some use of mallocarray(9).
Focus on code where we are doing multiplications within malloc(9). None of
these ire likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.

This initial sweep only covers malloc(9) calls with M_NOWAIT. No good
reason but I started doing the changes before r327796 and at that time it
was convenient to make sure the sorrounding code could handle NULL values.

X-Differential revision: https://reviews.freebsd.org/D13837
2018-01-15 21:14:56 +00:00
Fedor Uporov
512f29d141 Add metadata_csum feature support.
Reviewed by:   pfg (mentor)
Approved by:   pfg (mentor)
MFC after:     6 months

Differential Revision:    https://reviews.freebsd.org/D13810
2018-01-14 20:46:39 +00:00
Edward Tomasz Napierala
ed5cdcb6c3 Make nullfs properly report MNT_AUTOMOUNTED set on the nullfs mount itself,
instead of copying from the underlying filesystem.

PR:		224851
Reported by:	Jamie Landeg-Jones <jamie at dyslexicfish.net>
Tested by:	Jamie Landeg-Jones <jamie at dyslexicfish.net>
MFC after:	2 weeks
2018-01-10 17:51:02 +00:00
John Baldwin
9ae9a3d462 Correct comment. procfs_doprocfile implements 'file', not 'self'. 2018-01-05 18:32:46 +00:00
Fedor Uporov
3acd9182a1 Add 64bit feature support.
Reviewed by:    kevlo, pfg (mentor)
Approved by:    pfg (mentor)
MFC after:      6 months

Differential Revision:    https://reviews.freebsd.org/D11530
2018-01-05 10:04:01 +00:00
Konstantin Belousov
cfb03f67d9 Reuse kern_proc_vmmap_resident() for procfs_map resident count.
The existing algorithm in procfs_map() to calculate count of resident
pages in an entry is too primitive, resulting in too long run time for
large sparse mapping entries.  Re-use the kern_proc_vmmap_resident()
from kern_proc.c which only looks at the existing pages in the
iterations.

Also, this makes procfs to honor kern.proc_vmmap_skip_resident_count,
if user does not need this information.

Reported by:	Glenn Weinberg <glenn.weinberg@intel.com>
PR:	224532
No objections from:	des (procfs maintainer)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D13595
2017-12-28 13:23:13 +00:00
Eitan Adler
caa7e52f3f kernel: Fix several typos and minor errors
- duplicate words
- typos
- references to old versions of FreeBSD

Reviewed by:	imp, benno
2017-12-27 03:23:21 +00:00
Alexander Kabaev
151ba7933a Do pass removing some write-only variables from the kernel.
This reduces noise when kernel is compiled by newer GCC versions,
such as one used by external toolchain ports.

Reviewed by: kib, andrew(sys/arm and sys/arm64), emaste(partial), erj(partial)
Reviewed by: jhb (sys/dev/pci/* sys/kern/vfs_aio.c and sys/kern/kern_synch.c)
Differential Revision: https://reviews.freebsd.org/D10385
2017-12-25 04:48:39 +00:00
Konstantin Belousov
95c8838c2a Fix build for LP64 arches with gcc.
gcc complaints that the comparision is always false due to the value
range, and the cast does not prevent the analysis.  Split the LP64
vs. ILP32 clamping as a workaround.

Sponsored by:	The FreeBSD Foundation
2017-12-21 23:08:10 +00:00
John Baldwin
5538424353 Replace one more LINK_MAX with NFS_LINK_MAX missed in r326991.
Sponsored by:	Chelsio Communications
2017-12-19 22:43:39 +00:00
John Baldwin
f83f3d7986 Update link count handling in fuse for post-ino64.
Set FUSE_LINK_MAX to UINT32_MAX instead of LINK_MAX to match the maximum
link count possible in the 'nlink' field of 'struct fuse_attr'.

Sponsored by:	Chelsio Communications
2017-12-19 22:40:54 +00:00
John Baldwin
b501cc5da6 Rework pathconf handling for FIFOs.
On the one hand, FIFOs should respect other variables not supported by
the fifofs vnode operation (such as _PC_NAME_MAX, _PC_LINK_MAX, etc.).
These values are fs-specific and must come from a fs-specific method.
On the other hand, filesystems that support FIFOs are required to
support _PC_PIPE_BUF on directory vnodes that can contain FIFOs.
Given this latter requirement, once the fs-specific VOP_PATHCONF
method supports _PC_PIPE_BUF for directories, it is also suitable for
FIFOs permitting a single VOP_PATHCONF method to be used for both
FIFOs and non-FIFOs.

To that end, retire all of the FIFO-specific pathconf methods from
filesystems and change FIFO-specific vnode operation switches to use
the existing fs-specific VOP_PATHCONF method.  For fifofs, set it's
VOP_PATHCONF to VOP_PANIC since it should no longer be used.

While here, move _PC_PIPE_BUF handling out of vop_stdpathconf() so that
only filesystems supporting FIFOs will report a value.  In addition,
only report a valid _PC_PIPE_BUF for directories and FIFOs.

Discussed with:	bde
Reviewed by:	kib (part of a larger patch)
MFC after:	1 month
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D12572
2017-12-19 22:39:05 +00:00
John Baldwin
35b1a3abd3 Update tmpfs link count handling for ino64.
Add a new TMPFS_LINK_MAX to use in place of LINK_MAX for link overflow
checks and pathconf() reporting.  Rather than storing a full 64-bit
link count, just use a plain int and use INT_MAX as TMPFS_LINK_MAX.

Discussed with:	bde
Reviewed by:	kib (part of a larger patch)
Sponsored by:	Chelsio Communications
2017-12-19 20:19:07 +00:00
John Baldwin
c24008cc39 Honor NANDFS_LINK_MAX for post-ino64.
This uses NANDFS_LINK_MAX instead of LINK_MAX for link overflow checks
and the value reported by pathconf() / fpathconf().

Sponsored by:	Chelsio Communications
2017-12-19 20:17:07 +00:00
John Baldwin
f6e25ec77c Report INT_MAX for LINK_MAX for devfs' VOP_PATHCONF().
devfs uses int's for link counts internally and already reports the the
full link count via stat() post ino64.

Sponsored by:	Chelsio Communications
2017-12-19 20:07:57 +00:00
John Baldwin
a74da9fb83 Use FUSE_LINK_MAX for LINK_MAX in fuse' VOP_PATHCONF().
Should have included this in r326993.

MFC after:	1 month
Sponsored by:	Chelsio Communications
2017-12-19 19:57:55 +00:00
John Baldwin
418e621276 Handle _PC_FILESIZEBITS and _PC_SYMLINK_MAX for devfs' VOP_PATHCONF().
MFC after:	1 month
Sponsored by:	Chelsio Communications
2017-12-19 19:53:34 +00:00
John Baldwin
599afe53a8 Move NAME_MAX, LINK_MAX, and CHOWN_RESTRICTED out of vop_stdpathconf().
Having all filesystems fall through to default values isn't always correct
and these values can vary for different filesystem implementations.  Most
of these changes just use the existing default values with a few exceptions:
- Don't report CHOWN_RESTRICTED for ZFS since it doesn't do the exact
  permissions check this claims for chown().
- Use NANDFS_NAME_LEN for NAME_MAX for nandfs.
- Don't report a LINK_MAX of 0 on smbfs.  Now fail with EINVAL to
  indicate hard links aren't supported.

Requested by:	bde (though perhaps not this exact implementation)
Reviewed by:	kib (earlier version)
MFC after:	1 month
Sponsored by:	Chelsio Communications
2017-12-19 19:51:36 +00:00
John Baldwin
a0a073b16d Update NFS to handle larger link counts post ino64.
- Define a NFS_LINK_MAX as UINT32_MAX to match the wire protocol.
- Use NFS_LINK_MAX instead of LINK_MAX as the fallback value reported
  for a PATHCONF RPC by the NFS server.
- Use NFS_LINK_MAX instead of LINK_MAX as the default value reported
  by the NFS client pathconf() if not overridden by the NFS server.
- When reading the link count out of an RPC reply, read the full 32
  bits instead of the lower 16 bits.

Reviewed by:	rmacklem (earlier version)
Sponsored by:	Chelsio Communications
2017-12-19 19:18:48 +00:00
John Baldwin
4a62795260 Handle _PC_FILESIZEBITS and _PC_NO_TRUNC for smbfs' VOP_PATHCONF().
MFC after:	1 month
Sponsored by:	Chelsio Communications
2017-12-19 19:14:01 +00:00
John Baldwin
853b3a8ae8 Support _PC_FILESIZEBITS in msdosfs' VOP_PATHCONF().
MFC after:	1 month
Sponsored by:	Chelsio Communications
2017-12-19 19:10:00 +00:00
John Baldwin
746c92e04e Add a custom VOP_PATHCONF method for fuse.
This method handles _PC_FILESIZEBITS, _PC_SYMLINK_MAX, and _PC_NO_TRUNC.
For other values it defers to vop_stdpathconf().

MFC after:	1 month
Sponsored by:	Chelsio Communications
2017-12-19 19:09:06 +00:00
John Baldwin
dd688800e1 Add a custom VOP_PATHCONF method for fdescfs.
The method handles NAME_MAX and LINK_MAX explicitly.  For all other
pathconf variables, the method passes the request down to the underlying
file descriptor.  This requires splitting a kern_fpathconf() syscallsubr
routine out of sys_fpathconf().  Also, to avoid lock order reversals with
vnode locks, the fdescfs vnode is unlocked around the call to
kern_fpathconf(), but with the usecount of the vnode bumped.

MFC after:	1 month
Sponsored by:	Chelsio Communications
2017-12-19 18:20:38 +00:00
Konstantin Belousov
833429229a In devfs_lookupx() dotdot lookup case, avoid dereferencing
dvp->v_mount after dvp is unlocked.

The vnode might be reclaimed after unlock, so v_mount becomes NULL.
Cache the struct mount pointer before the unlock, the struct is
type-stable.

Note that devfs_allocv() reads mp->mnt_data but does not operate on it
further when dirent is doomed.  The unmount cannot proceed until all
dirents are reclaimed.

Reported and tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-12-14 13:41:11 +00:00
Pedro F. Giffuni
62cf53fdac SPDX: some uses of the RSA-MD license. 2017-12-13 16:30:39 +00:00
Fedor Uporov
1806c9ab85 Fix extattr getters in case of neither uio nor buffer was not passed to VOP_*.
Approved by:    pfg (mentor)
MFC after:      2 weeks

Differential Revision:    https://reviews.freebsd.org/D13359
2017-12-12 20:02:48 +00:00
Rick Macklem
219afc4fe2 Define macros used by the pNFS server code.
This commit defines some macros used by the pNFS server code.
They will not be used until the main pNFS server code merge occurs,
which will probably be in April 2018.
2017-12-09 21:04:56 +00:00
Gleb Smirnoff
db357f584a Fix file missed in r326607. 2017-12-06 00:44:49 +00:00
Gleb Smirnoff
135beaf60e Reduce pollution via tmpfs.h. 2017-12-06 00:42:08 +00:00
Rick Macklem
b07bc39a23 Avoid the overhead of acquiring a lock in nfsrv_checkgetattr() when
there are no write delegations issued.

manu@ reported on the freebsd-current@ mailing list that there was
a significant performance hit in nfsrv_checkgetattr() caused by
the acquisition/release of a state lock, even when there were no
write delegations issued.
This patch add a count of outstanding issued write delegations to the
NFSv4 server. This count allows nfsrv_checkgetattr() to return without
acquiring any lock when the count is 0, avoiding the performance hit
for the case where no write delegations are issued.

Reported by:	manu
Reviewed by:	kib
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D13327
2017-12-04 21:50:27 +00:00
Pedro F. Giffuni
7abc09cddb SPDX: Complete license IDs for ext2fs. 2017-12-02 17:22:55 +00:00
Emmanuel Vadot
38c9c65ee4 r326394 is calling malloc with M_WAITOK under a lock, revert for now
Reported by:	andrew
2017-11-30 14:06:54 +00:00
Emmanuel Vadot
31e8b07376 devfs: Avoid a malloc/free if we just need to increment the refcount
MFC after:	1 week
Sponsored by:	Gandi.net
2017-11-30 12:38:42 +00:00
Pedro F. Giffuni
d63027b668 sys/fs: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-27 15:15:37 +00:00
Conrad Meyer
60965b7606 msdosfs(5): Reflect READONLY attribute in file mode
Msdosfs allows setting READONLY by clearing the owner write bit of the file
mode.  (While here, correct the misspelling of S_IWUSR as VWRITE.  No
functional change.)

In msdosfs_getattr, intuitively reflect that READONLY attribute to userspace
in the file mode.

Reported by:	Karl Denninger <karl AT denninger.net>
Sponsored by:	Dell EMC Isilon
2017-11-20 21:38:24 +00:00
Pedro F. Giffuni
51369649b0 sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:43:44 +00:00
Xin LI
11d38f2ca0 Remove unused header. 2017-11-19 03:52:03 +00:00
Xin LI
eafe4941b3 Remove unused header. 2017-11-19 03:51:47 +00:00
Pedro F. Giffuni
df57947f08 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
Jeff Roberson
8d6fbbb867 Replace manyinstances of VM_WAIT with blocking page allocation flags
similar to the kernel memory allocator.

This simplifies NUMA allocation because the domain will be known at wait
time and races between failure and sleeping are eliminated.  This also
reduces boilerplate code and simplifies callers.

A wait primitive is supplied for uma zones for similar reasons.  This
eliminates some non-specific VM_WAIT calls in favor of more explicit
sleeps that may be satisfied without new pages.

Reviewed by:	alc, kib, markj
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
2017-11-08 02:39:37 +00:00
Hans Petter Selasky
56c49c63e7 Allow CUSE(3) to free all memory mapped memory by using regular SWAP objects
instead of malloc(). The SWAP objects are automagically freed when there are no
more consumers. This greatly simplifies the mmap logic inside CUSE(3) in the
kernel. This change fixes an issue where mmapped memory can accumulate and never
get freed, if many different mmap sizes are needed over time. Further this
change fixes memory leaks when the CUSE(3) kernel module is unloaded.

While at it make sure the CUSE_ALLOC_PAGES_MAX limit is treated as an exclusive
limit. CUSE(3) memory maps must be less than CUSE_ALLOC_PAGES_MAX number of pages.

Reviewed by:		kib @
Differential Revision:	https://reviews.freebsd.org/D11392
Sponsored by:		Mellanox Technologies
MFC after:		1 week
2017-11-03 14:10:57 +00:00
Fedor Uporov
ffbde5ea28 Fix physical block number overflow in different places.
Approved by:    pfg (mentor)
MFC after:      6 months
2017-10-24 20:10:08 +00:00
Fedor Uporov
dc262e5bc2 Set doreallocblks sysctl value to zero by default because of
possibility of filesystem corruption.

Approved by:    pfg (mentor)
MFC after:      2 weeks
2017-10-24 19:16:25 +00:00
Fedor Uporov
f0a993bbdb Do not free bufs in case of extents metadata blocks + remove unneeded asserts.
Approved by:    pfg (mentor)
MFC after:      6 months
2017-10-24 19:14:33 +00:00
Matt Joras
ba19246e07 Move clear_unrhdr to tmpfs_free_tmp.
Clearing the unr in tmpfs_unmount is not correct. In the case of
multiple references to the tmpfs mount (e.g. when there are lookup
threads using it) it will not be the one to finish tmpfs_free_tmp. In
those cases tmpfs_free_node_locked will be the final one to execute
tmpfs_free_tmp, and until then the unr must be valid.

Reported by:	pho
Approved/reviewed by:	rstone (mentor)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12749
2017-10-23 15:43:38 +00:00
Mark Johnston
cc1307b67f Delete declarations of struct pfs_bitmap, removed in r143841.
MFC after:	1 week
2017-10-22 20:22:11 +00:00
Fedor Uporov
3fcbb8c0a9 Fix unused variable + style(9) fixes inside the ext4_ext_find_extent()
Approved by:    pfg (mentor)
Reported by:    Coverity
CID:            1381754
MFC after:      6 months
2017-10-19 16:42:03 +00:00
Ed Maste
027bebe8e1 msdosfs: fix build with MSDOSFS_DEBUG
Inspired by a patch submission by longwitz@incore.de with many changes
for ino64 in HEAD.

PR:		199152
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2017-10-19 12:55:11 +00:00
Ryan Libby
ccf2e8123b ext2: delete redundant decl of ext2_fserr
Fix gcc build after r324706.

Reviewed by:	pfg
Differential Revision:	https://reviews.freebsd.org/D12709
2017-10-18 00:41:23 +00:00
Fedor Uporov
b394cd1e28 Add inital extents read-write support.
Approved by:    pfg (mentor)
MFC after:      6 months
RelNotes:       Yes

Differential Revision:    https://reviews.freebsd.org/D12087
2017-10-17 20:45:44 +00:00
Rick Macklem
f49c813c1d Use taskqueue(9) to do writes/commits to mirrored DSs concurrently.
When the NFSv4.1 pNFS client is using a Flexible File Layout specifying
mirrored Data Servers, it must do the writes and commits to all mirrors.
This patch modifies the client to use a taskqueue to perform these writes
and commits concurrently.
The number of threads can't be changed for taskqueue(9), so it is set
to 4 * mp_ncpus by default, but this can be overridden by setting the
sysctl vfs.nfs.pnfsiothreads.

Differential Revision:	https://reviews.freebsd.org/D12632
2017-10-16 23:28:12 +00:00
Rick Macklem
57ef3db3a9 Fix the client IP address reported by nfsdumpstate for 64bit arch and NFSv4.1.
The client IP address was not being reported for some NFSv4 mounts by
nfsdumpstate. Upon investigation, two problems were found for mounts
using IPv4. One was that the code (originally written and tested on i386)
assumed that a "u_long" was a "uint32_t" and would exactly store an
IPv4 host address. Not correct for 64bit arches.
Also, for NFSv4.1 mounts, the field was not being filled in. This was
basically correct, because NFSv4.1 does not use a callback address.
However, it meant that nfsdumpstate could not report the client IP addr.
This patch should fix both of these issues.
For IPv6, the address will still not be reported. The original NFSv4 RFC
only specified IPv4 callback addresses. I think this has changed and, if so,
a future commit to fix reporting of IPv6 addresses will be needed.

Reported by:	manu
PR:		223036
MFC after:	2 weeks
2017-10-15 22:22:27 +00:00
Fedor Uporov
04660064b8 Add extended attributes support to fuse kernel module.
Author:         kem
Reviewed by:    cem, pfg (mentor)
Approved by:    pfg (mentor)
MFC after:      2 weeks

Differential Revision: https://reviews.freebsd.org/D12485
2017-10-14 19:02:52 +00:00
Matt Joras
9aaf913e13 When unmounting a tmpfs, do not call free_unr.
tmpfs uses unr(9) to allocate inodes. Previously when unmounting it
would individually free the units when it freed each vnode. This is
unnecessary as we can use the newly-added unrhdr_clear function to clear
out the unr in onde go. This measurably reduces the time to unmount a
tmpfs with many files.

Reviewed by:	cem, lidl
Approved by:	rstone (mentor)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12591
2017-10-11 21:53:53 +00:00
Rick Macklem
63918d3848 Fix forced dismount when a pNFS mount is hung on a DS.
When a "pnfs" NFSv4.1 mount is hung because of an unresponsive DS,
a forced dismount wouldn't work, because the RPC socket for the DS
was not being closed. This patch fixes this.
This will only affect "pnfs" mounts where the pNFS server's DS
is unresponsive (crashed or network partitioned or...).
Found during testing of the pNFS server.

MFC after:	2 weeks
2017-10-10 21:05:40 +00:00
Rick Macklem
b949cc41d1 Add Flex File Layout support to the NFSv4.1 pNFS client.
This patch adds support for the Flexible File Layout to the pNFS client.
Although the patch is rather large, it should only affect NFS mounts
using the "pnfs" option against pNFS servers that do not support File
Layout.
There are still a couple of things missing from the Flexible File Layout
client implementation:
- The code does not yet do a LayoutReturn with I/O error stats when
  I/O error(s) occur when attempting to do I/O on a DS.
  This will be fixed in a future commit, since it is important for the
  MDS to know that I/O on a DS is failing.
- The current code does writes and commits to mirror DSs serially.
  Making them happen concurrently will be done in a future commit,
  after discussion on freebsd-current@ on the best way to do this.
- The code does not handle NFSv4.0 DSs. Since there is no extant pNFS
  server that implements NFSv4.0 DSs and NFSv4.1 DSs makes more sense
  now, I don't intend to implement this until there is a need for it.
  There is support for NFSv4.1 and NFSv3 DSs.
2017-10-05 20:10:40 +00:00
Hans Petter Selasky
9f16d9c95b Add support for new cuse(3) error code, CUSE_ERR_NO_DEVICE.
This error code is useful when emulating Linux input event
devices from userspace.

PR:			218626
Submitted by:		jan.kokemueller@gmail.com
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2017-10-05 16:42:02 +00:00
Rick Macklem
64059dce60 Add a few definitions for the Flex File Layout.
This patch adds a few definitions for the Flex File Layout.
Until a future commit adds Flex File layout support, these new fields
are not used.
This patch should not affect the "pnfs" option for File Layout.
2017-10-04 22:55:30 +00:00
John Baldwin
61d53d8fce Flesh out pathconf() on UDF.
- Return 64 bits for _PC_FILESIZEBITS.
- Handle _PC_SYMLINK_MAX.
- Defer _PC_PATH_MAX to vop_stdpathconf().

Sponsored by:	Chelsio Communications
2017-10-02 23:31:11 +00:00
John Baldwin
9d1d1d1900 Return 64 for pathconf(_PC_FILESIZEBITS) on tmpfs.
Sponsored by:	Chelsio Communications
2017-10-02 23:23:12 +00:00
John Baldwin
eb3d4ccc48 Handle _PC_FILESIZEBITS and _PC_SYMLINK_MAX pathconf() requests in cd9660.
cd9660 only supports symlinks with Rock Ridge extensions, so
_PC_SYMLINK_MAX is conditional on Rock Ridge.

Sponsored by:	Chelsio Communications
2017-10-02 23:12:02 +00:00
Mateusz Guzik
e3e10c39f1 tmpfs: skip zero-sized page count updates
Such updates consisted of vast majority of modificiations, especially
in tmpfs_reg_resize.

For the case where page count did no change and the size grew we only
need to update tn_size. Use this fact to avoid vm object lock/relock.

MFC after:	1 week
2017-09-30 18:23:45 +00:00
Rick Macklem
efea6b20b9 Add support for Flex File Layout to the pNFS client structures.
This patch modifies the pNFS client layout and deviceinfo structures
to add fields and unions for the Flex File Layout. Until a future
commit adds Flex File layout support, these new fields are not used.
This patch should not affect the "pnfs" option for File Layout.
2017-09-29 23:13:01 +00:00
Rick Macklem
8dd62f2eaa Add the NFS client state flag that enables Flexible File Layout.
This patch adds a NFSSTA_FLEXFILE flag that will be used to enable
Flexible File Layout for the NFSv4.1 pNFS client. It is not yet
used, but will be after a future commit adds Flex File Layout support.
2017-09-28 23:05:08 +00:00
Rick Macklem
be3d32ad6e Change nfsv4_getipaddr() and nfsrpc_fillsa() to not use sockaddr_storage.
This patch changes nfsv4_getipaddr() and nfsrpc_fillsa() to use
a sockaddr_in * and sockaddr_in6 * instead of sockaddr_storage, to
avoid allocating the latter on the stack. It also moves the nfsrpc_fillsa()
call to after the completion of parsing of the DeviceInfo reply from
the server. This patch is in preparation for addition of Flex File
Layout support in a future commit.
It only affects the "pnfs" NFSv4.1 client mount option and should not
have changed its semantics.
2017-09-28 22:33:01 +00:00
Rick Macklem
bd290946e9 Fix a memory leak that occurred in the pNFS client.
When a "pnfs" NFSv4.1 mount was unmounted, it didn't free up the layouts
and deviceinfo structures. This leak only affects "pnfs" mounts and only
when the mount is umounted.
Found while testing the pNFS Flexible File layout client code.

MFC after:	2 weeks
2017-09-27 23:23:41 +00:00
Mark Johnston
47f11baaca Use C99 initializers for DTrace provider methods.
This makes the definitions easier to read and more cscope-friendly.

MFC after:	1 week
2017-09-27 17:46:38 +00:00
Fedor Uporov
72530f91cf Add check to avoid raw inode iblocks fields overflow in case of huge_file feature.
Use the Linux logic for now.

Reviewed by:    pfg (mentor)
Approved by:    pfg (mentor)
MFC after:      2 weeks
Differential Revision: https://reviews.freebsd.org/D12131
2017-09-27 16:12:13 +00:00
Rick Macklem
a8462c582c Add major and minor version arguments to nfscl_reqstart().
This patch adds "vers" and "minorvers" arguments to nfscl_reqstart().
The patch always passes them in as "0" and that implies no change
in semantics. These arguments will be used by a future commit that
adds support for the Flexible File Layout.
2017-09-26 23:42:44 +00:00
John Baldwin
5b01ccb01e Use tmpfs_print for tmpfs FIFOs.
Reviewed by:	kib (part of a larger patch)
2017-09-25 20:26:16 +00:00
Rick Macklem
ce8d06fe87 Change a panic to an error return.
There was a panic() in the NFS server's write operation that didn't
need to be a panic() and could just be an error return.
This patch makes that change.
Found by code inspection during development of the pNFS service.

MFC after:	2 weeks
2017-09-24 20:05:48 +00:00
Rick Macklem
c36e087097 Remove 0 filling from nfsm_uiombuflist().
nfsm_uiombuflist() zero filled the mbuf list to a multiple of 4bytes
as required for XDR. Unfortunately that modified an mbuf list after
it was m_copym()'d and was broken. This patch removes the zero filling code.
Since nfsm_uiombuflist() is not yet used in head/current, this has no
effect on users.
The function will be used by a future commit of code that adds Flex
File Layout support.
2017-09-24 19:43:31 +00:00
John Baldwin
e1d15b892a Only handle _PC_MAX_CANON, _PC_MAX_INPUT, and _PC_VDISABLE for TTY devices.
Move handling of these three pathconf() variables out of vop_stdpathconf()
and into devfs_pathconf() as TTY devices can only be devfs files.  In
addition, only return settings for these three variables for devfs devices
whose device switch has the D_TTY flag set.

Discussed with:	bde, kib
Sponsored by:	Chelsio Communications
2017-09-21 23:05:32 +00:00
Rick Macklem
6b43e06029 Add a few definitions for Flex File Layout for pNFS.
These definitions will be used by a future commit.
2017-09-21 00:41:12 +00:00
Rick Macklem
0f29b8292d Make the nfsrpc_layoutget() function a static.
Make the NFSv4 pNFS client function nfsrpc_layoutget() a static, since it
is only used in sys/fs/nfsclient/nfs_clrpcops.c.
This prepares the code for future patches that add Flex File layout
support.
2017-09-19 23:28:22 +00:00
Rick Macklem
2742a21091 Add a new function called nfsm_uiombuflist(), similar to nfsm_uiombuf().
This patch adds a new function called nfsm_uiombuflist(), which is
similar to nfsm_uiombuf(), but doesn't not use the fields in
struct nfsrv_descript. This new function will be used by the pNFS client
for writing to mirrors using Flex Files layout.
The function is not yet called anywhere.
Also, get rid of #ifndef APPLE, which is ancient cruft left over from
the Mac OSX port of the NFSv4 client.
2017-09-19 21:31:36 +00:00
Rick Macklem
b0932afacc Simplify nfsrpc_layoutreturn() args.
Simplify nfsrpc_layoutreturn() args. in preparation for the addition
of Flex File layout support, since File layout uses a 0 length field.
Flex Files does use a longer field, but that will be added in a
subsequent commit.
2017-09-19 20:45:25 +00:00
Rick Macklem
ab118d04be Simplify nfsrpc_layoutcommit() args.
Simplify nfsrpc_layoutcommit() args. in preparation for the addition
of Flex File layout support, since it also uses a 0 length field.
2017-09-19 20:18:41 +00:00
Rick Macklem
ccf038250a Fix bogus FREAD with NFSV4OPEN_ACCESSREAD. No functional change.
The code in nfscl_doflayoutio() bogusly used FREAD instead of
NFSV4OPEN_ACCESSREAD. Since both happen to be defined as "1", this
worked and the patch doesn't result in a functional change.
Found by inspection during development of Flex File Layout support.

MFC after:	2 weeks
2017-09-17 22:18:01 +00:00
Konstantin Belousov
4eeec01fee Style.
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2017-08-28 21:04:56 +00:00
Konstantin Belousov
fbcbbe78dc Verify that the BPB media descriptor and FAT ID match.
FAT specification requires that for valid FAT, FAT cluster 0 has a
specific value derived from the BPB media descriptor.  The lowest
(little-endian) byte must be equal to bpb.bpbMedia, other bits in the
cluster number must be all 1's.  Implement the check to reduce the
chance of the randomly corrupted FAT to pass the mount attempt.

Submitted by:	Siva Mahadevan <smahadevan@freebsdfoundation.org>
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D12124
2017-08-28 20:52:32 +00:00
Konstantin Belousov
e5cffdd34b Do not drop NFS vnode lock when performing consistency checks.
Currently several paths in the NFS client upgrade the shared vnode
lock to exclusive, which might cause temporal dropping of the lock.
This action appears to be fatal for nullfs mounts over NFS. If the
operation is performed over nullfs vnode, then bypassed down to NFS
VOP, and the lock is dropped, other thread might reclaim the upper
nullfs vnode.  Since on reclaim the nullfs vnode lock and NFS vnode
lock are split, the original lock state of the nullfs vnode is not
restored.  As result, VFS operations receive not locked vnode after a
VOP call.

Stop upgrading the vnode lock when we check the consistency or flush
buffers as result of detected inconsistency.  Instead, allocate a new
lockmgr lock for each NFS node, which is locked exclusive instead of
the vnode lock upgrade.  In other words, the other parallel
modification of the vnode are excluded by either vnode lock conflict
or exclusivity of the new lock when the vnode lock is shared.

Also revert r316529 because now the vnode cannot be reclaimed during
ncl_vinvalbuf().

In collaboration with:	pho
Reviewed by:	rmacklem
Reported and tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D12083
2017-08-20 10:08:45 +00:00
Mark Johnston
a95435cfed Bump the maximum file name length in pseudofs filesystems to 48.
The previous limit of 24 was somewhat restrictive, and with this change
ceil(log2(sizeof(struct pfs_node))) is the same as before in both the ILP32
and LP64 models, so the malloc zone used for allocations of struct pfs_node
is the same as before.

Approved by:	des
2017-08-03 21:35:53 +00:00
Dmitry Chagin
77d3337c9f Implement proper Linux /dev/fd and /proc/self/fd behavior by adding
Linux specific things to the native fdescfs file system.

Unlike FreeBSD, the Linux fdescfs is a directory containing a symbolic
links to the actual files, which the process has open.
A readlink(2) call on this file returns a full path in case of regular file
or a string in a special format (type:[inode], anon_inode:<file-type>, etc..).
As well as in a FreeBSD, opening the file in the Linux fdescfs directory is
equivalent to duplicating the corresponding file descriptor.

Here we have mutually exclusive requirements:
- in case of readlink(2) call fdescfs lookup() method should return VLNK
vnode otherwise our kern_readlink() fail with EINVAL error;
- in the other calls fdescfs lookup() method should return non VLNK vnode.

For what new vnode v_flag VV_READLINK was added, which is set if fdescfs has beed
mounted with linrdlnk option an modified kern_readlinkat() to properly handle it.

For now For Linux ABI compatibility mount fdescfs volume with linrdlnk option:

    mount -t fdescfs -o linrdlnk null /compat/linux/dev/fd

Reviewed by:	kib@
MFC after:	1 week
Relnotes:	yes
2017-08-01 03:40:19 +00:00
Alexander Motin
e9c9826673 Improve FHA locality control for NFS read/write requests.
This change adds two new tunables, allowing to control serialization for
read and write NFS requests separately.  It does not change the default
behavior since there are too many factors to consider, but gives additional
space for further experiments and tuning.

The main motivation for this change is very low write speed in case of ZFS
with sync=always or when NFS clients requests sychronous operation, when
every separate request has to be written/flushed to ZIL, and requests are
processed one at a time.  Setting vfs.nfsd.fha.write=0 in that case allows
to increase ZIL throughput by several times by coalescing writes and cache
flushes.  There is a worry that doing it may increase data fragmentation
on disks, but I suppose it should not happen for pool with SLOG.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2017-07-31 15:23:19 +00:00
Rick Macklem
47cbff34fa Add kernel support for the NFS client forced dismount "umount -N" option.
When an NFS mount is hung against an unresponsive NFS server, the "umount -f"
option can be used to dismount the mount. Unfortunately, "umount -f" gets
hung as well if a "umount" without "-f" has already been done. Usually,
this is because of a vnode lock being held by the "umount" for the mounted-on
vnode.
This patch adds kernel code so that a new "-N" option can be added to "umount",
allowing it to avoid getting hung for this case.
It adds two flags. One indicates that a forced dismount is about to happen
and the other is used, along with setting mnt_data == NULL, to handshake
with the nfs_unmount() VFS call.
It includes a slight change to the interface used between the client and
common NFS modules, so I bumped __FreeBSD_version to ensure both modules are
rebuilt.

Tested by:	pho
Reviewed by:	kib
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D11735
2017-07-29 19:52:47 +00:00
Rick Macklem
23e148e9a4 Fix possible crash for the NFSv4.1 pNFS client.
If the nfsrpc_createlayoutrpc() call in nfsrpc_getcreatelayout() fails,
the code used nfhpp when it might be set NULL. This patch checks for
the error cases (laystat != 0) and avoids using nfhpp for the failure case.
This would only affect NFSv4.1 mounts with the "pnfs" option.
Found while testing the "umount -N" patch not yet in head.

MFC after:	2 weeks
2017-07-29 02:25:49 +00:00
Rick Macklem
16f300fa4a Replace the checks for MNTK_UNMOUNTF with a macro that does the same thing.
This patch defines a macro that checks for MNTK_UNMOUNTF and replaces
explicit checks with this macro. It has no effect on semantics, but
prepares the code for a future patch where there will also be a
NFS specific flag for "forced dismount about to occur".

Suggested by:	kib
MFC after:	2 weeks
2017-07-27 20:55:31 +00:00
Konstantin Belousov
555b7bb4c8 Mark pages after EOF as clean after pageout.
Suppose that a file on NFS has partially filled last page, and this
page is dirty.  NFS VOP_PAGEOUT() method only marks the the page clean
up to the block of the last written byte, leaving other blocks dirty.
Also any page which erronously exists in the vnode vm_object past EOF
is also left marked as dirty.

With the introduction of the buf-cache coherent pager, each pass of
syncer over the object with such page results in creation of B_DELWRI
buffer due to VOP_WRITE() call.  This buffer is noted on next syncer
pass, which results e.g. a visible manifestation of shutdown never
finishing vnode sync.  Note that before buf-cache coherency commit, a
dirty page might left never synced to server if a partial writes
occur.

Fix this by clearing dirty bits after EOF.  Only blocks of the partial
page which are completely after EOF are marked clean, to avoid
possible user data loss.

Reported by:	mav
Reviewed by:	alc, markj
Tested by:	mav, pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D11697
2017-07-26 20:07:05 +00:00
Konstantin Belousov
cc2c26223b Move rtvals initialization out of the region protected by NFS node
lock.

Noted by:	alc
Reviewed by:	alc, markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
X-Differential revision:	https://reviews.freebsd.org/D11697
2017-07-26 20:01:31 +00:00
Dmitry Chagin
f80dc46c3d Replace unnecessary _KERNEL by double-include protection.
MFC after:	2 week
2017-07-25 06:59:35 +00:00
Rick Macklem
f8181b5e0e r320062 introduced a bug when doing NFSv4.1 mounts against some non-FreeBSD servers.
r320062 used nm_rsize, nm_wsize to set the maximum request/response sizes for
the NFSv4.1 session. If rsize,wsize are not specified as options, the
value of nm_rsize, nm_wsize is 0 at session creation, resulting in
values for request/response that are too small.
This patch fixes the problem. A workaround is to specify rsize=N,wsize=N
mount options explicitly, so they are set before session creation.
This bug only affects NFSv4.1 mounts against some non-FreeBSD servers.

MFC after:	1 week
2017-07-21 00:14:43 +00:00
Rick Macklem
06ea10c60b Revert r321308. I'll commit a better fix soon. 2017-07-20 23:59:47 +00:00
Rick Macklem
a9d104fd89 r320062 introduced a bug when doing NFSv4.1 mounts against some non-FreeBSD servers.
r320062 used nm_rsize, nm_wsize to set the maximum request/response sizes for
the NFSv4.1 session. If rsize,wsize are not specified as options, the
value of nm_rsize, nm_wsize is 0 at session creation, resulting in
values for request/response that are too small.
This patch fixes the problem. A workaround is to specify rsize=N,wsize=N
mount options explicitly, so they are set before session creation.
This bug only affects NFSv4.1 mounts against some non-FreeBSD servers.

MFC after:	1 week
2017-07-20 23:15:50 +00:00
Edward Tomasz Napierala
1d2fef9b9a Rename vfs.nfsd.enable_uidtostring to vfs.nfs.enable_uidtostring.
It applies to both NFS client and NFS server, and is useful for both.
This is different from vfs.nfsd.enable_stringtouid, which is specific
to server side.

Reviewed by:	rmacklem@
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2017-07-19 09:59:32 +00:00
John Baldwin
15a88f8158 Consistently use vop_stdpathconf() for default pathconf values.
Update filesystems not currently using vop_stdpathconf() in pathconf
VOPs to use vop_stdpathconf() for any configuration variables that do
not have filesystem-specific values.  vop_stdpathconf() is used for
variables that have system-wide settings as well as providing default
values for some values based on system limits.  Filesystems can still
explicitly override individual settings.

PR:		219851
Reported by:	cem
Reviewed by:	cem, kib, ngie
MFC after:	1 month
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D11541
2017-07-11 21:55:20 +00:00
Pedro F. Giffuni
12b4678b2a Remove stale comments.
There's no real advantage in using memcpy here.

Dicussed with:	bde (long ago)
2017-07-09 15:19:28 +00:00
Dmitry Chagin
d499e3f58e Style(9). Whitespace.
MFC after:	3 weeks
2017-07-09 14:18:22 +00:00
Dmitry Chagin
16e3859b47 Eliminate the bogus casts.
MFC after:	3 weeks
2017-07-09 14:15:51 +00:00
Dmitry Chagin
46d186a9b4 Don't initialize error in declaration.
MFC after:	3 weeks
2017-07-08 21:15:46 +00:00
Dmitry Chagin
18a9ea872a Eliminate the bogus cast.
MFC after:	3 weeks
2017-07-08 21:13:25 +00:00
Dmitry Chagin
11fc6c6dac Eliminate the bogus cast.
MFC after:	3 weeks
2017-07-08 21:12:00 +00:00
Dmitry Chagin
b9d3485fb4 Don't take a lock around atomic operation.
MFC after:	3 weeks
2017-07-08 21:08:22 +00:00
Dmitry Chagin
073b14b469 Remove init from declaration, collapse two int vars declarations into single.
MFC after:	3 weeks
2017-07-08 21:05:28 +00:00
Dmitry Chagin
1901d0d8d3 Remove init from declaration.
MFC after:	3 weeks
2017-07-08 21:04:09 +00:00
Dmitry Chagin
a15cf51f0a Style(9). Add blank line aftr {.
MFC after:	3 weeks
2017-07-08 21:02:40 +00:00
Rick Macklem
25d694a6fa Add support for AF_LOCAL socket upcalls to the nfsuserd daemon.
This patch adds support for AF_LOCAL socket upcalls to an nfsuserd daemon
that supports them. A future patch to the nfsuserd daemon will use AF_LOCAL
sockets to avoid a problem when using upcalls to 127.0.0.1 if jails are
in use.

Suggested by:	dfr
PR:		205193
2017-07-06 00:53:12 +00:00
Pedro F. Giffuni
3d851dbe07 ext2fs: be more verbose about unsupported ext2fs features.
It is useful to know exactly what features may be lacking when trying to
mount ext4 filesystems.

Submitted by:	Fedor Uporov
Differential Revision:	https://reviews.freebsd.org/D11208
2017-07-02 20:47:25 +00:00
Rick Macklem
ad6eb97601 Fix an NFSv3 client case that probably never happens.
If an NFSv3 server were to reply with weak cache consistency attributes,
but not post operation attributes, the client would use garbage attributes
from memory. This was spotted during work on the code for the NFSv4.1 client.
I have never seen evidence that this happens and it wouldn't make sense
for an NFSv3 server to do this, so this patch is basically "theoretical",
but does fix the problem if a server were to do the above.

PR:		219552
MFC after:	2 weeks
2017-06-28 21:37:08 +00:00
Conrad Meyer
bb751fbbc7 Complete support for IO_APPEND flag in fuse
This finishes what r245164 started and makes open(..., O_APPEND) work again
after r299753.

- Pass ioflags, incl. IO_APPEND, down to the direct write backend (r245164
  added it to only the bio backend).
- (r299753 changed the WRONLY backend from bio to direct.)

PR:		220185
Reported by:	Ben RUBSON <ben.rubson at gmail.com>
Reviewed by:	bapt@, rmacklem@
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11348
2017-06-28 13:56:15 +00:00
Enji Cooper
6bfe453238 Fix LINT, broken by a -Wformat warning in r320329 with PFS_DELEN being
changed from %d to a long-width type.

Use uintmax_t casting and %ju to futureproof the format string against
potential changes with either the #define or the implementation-specific
definition for offsetof(..).
2017-06-27 17:01:46 +00:00
Edward Tomasz Napierala
3c264086aa Revert part of r320359, as suggested by rmacklem@. That case is only used
for nfsuserd -manage-gids and shouldn't depend on sysctl.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2017-06-27 15:14:06 +00:00
Pedro F. Giffuni
26f36b55b6 ext2fs: Support e2di_uid_high and e2di_gid_high.
The fields exist on all versions of the filesystem and using them is a mount
option on linux. For FreeBSD, the corresponding i_uid and i_gid are always
long enough so use them by default.

Reviewed by:	Fedor Uporov
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D11354
2017-06-27 15:07:19 +00:00
Edward Tomasz Napierala
6a3450e178 Add vfs.nfsd.nfsd_enable_uidtostring, which works just like
vfs.nfsd.nfsd_enable_stringtouid, but in reverse - when set to 1,
it forces the NFSv4 server to return numeric UIDs and GIDs instead
of "user@domain" strings. This helps with clients that can't
translate returned identifiers, eg when rerooting.

The same can be achieved by just never running nfsuserd(8),
but the sysctl is useful to toggle the behaviour back and forth
without rebooting.

Reviewed by:	rmacklem (earlier version)
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D11326
2017-06-26 13:11:21 +00:00
Rick Macklem
81b07aac10 Add support to the NFSv4.1/pNFS client for commits through the DS.
A NFSv4.1/pNFS server using File Layout can specify that Commit operations
are to be done against the DS instead of MDS. Since no extant pNFS
server did this, the code was untested and "#ifdef notyet".
The FreeBSD pNFS server I am developing does specify that Commits be done
through the DS, so the code has been enabled/tested.
This patch should only affect the case of a pNFS server that specfies
Commits through the DS.

PR:		219551
MFC after:	2 weeks
2017-06-26 00:43:04 +00:00
Dmitry Chagin
0a7c8d302a PFS_DELEN is the sum of the permanent part of the struct dirent and
fixed size for the name buffer PFS_NAMELEN.
As r318736 was commited (ino64 project) the size of the permanent part
of the struct dirent was changed, so calulate PFS_DELEN properly.
2017-06-25 15:21:51 +00:00
Rick Macklem
a351e99ce6 Add two new compound RPCs to the NFSv4.1/pNFS client.
When the NFSv4.1 client is doing pNFS, it needs to get an Open and
a Layout for every file it will be doing I/O on. The current code
does two separate RPCs to get these. This patch adds two new compounds
that do the both the Open and LayoutGet in the same RPC, reducing the
RPC count.
It also factors out the code that sets up and parses the LayoutGet operation
into separate functions, so that the code doesn't get duplicated for
these new RPCs.
This patch is fairly large, but should only affect the NFSv4.1 client
when the "pnfs" option is specified.

PR:		219550
MFC after:	2 weeks
2017-06-24 20:01:21 +00:00
Pedro F. Giffuni
a821bdcfd9 ext2fs: add dir_nlink feature support.
ext4 on linux has always supported more than 32000 directories through
the dir_nlink feature, but FreeBSD was unable to catch up on this feature.
As part of the 64 bit inode changes nlink_t has been extended and this
feature is now possible.

Submitted by:	Fedor Uporov
Differential Revision:	https://reviews.freebsd.org/D11210
2017-06-22 02:43:32 +00:00
Ed Maste
1f7d7cd76a msdosfs: reformat a comment to reduce NetBSD diffs 2017-06-22 01:11:20 +00:00
Rick Macklem
6d7963ecd4 Ensure that the credentials field of the NFSv4 client open structure is
initialized.

bdrewery@ has reported panics "newnfs_copycred: negative nfsc_ngroups".
The only way I can see that this occurs is that the credentials field of
the open structure gets used before being filled in.
I am not sure quite how this happens, but for the file create case, the
code is serialized via the vnode lock on the directory. If, somehow, a
link to the same file gets created just after file creation, this might
occur.

This patch ensures that the credentials field is initialized to a reasonable
set of credentials before the structure is linked into any list, so I
this should ensure it is initialized before use.
I am committing the patch now, since bdrewery@ notes that the panics
are intermittent and it may be months before he knows if the patch fixes
his problem.

Reported by:	bdrewery
MFC after:	2 weeks
2017-06-22 00:17:15 +00:00
Pedro F. Giffuni
aee33af1f1 Attempt to treat "metadata" as a collectively singular noun.
Or at least more consistent.

Input from:	matteo, ian
2017-06-20 20:22:34 +00:00
Pedro F. Giffuni
af2ae31d93 Improve grammar concerning "metadata".
Remove unnecessary space while here.
2017-06-20 14:35:19 +00:00
Pedro F. Giffuni
d23db91ef4 ext2fs: Add uninit_bg feature support.
From the linux tune2fs(8) manpage:
"Allow the kernel to initialize bitmaps and inode tables and keep a high
watermark for the unused inodes in a filesystem, to reduce e2fsck(8) time.
This first e2fsck run after enabling this feature will take the full time,
but subsequent e2fsck runs will take only a fraction of the original time,
depending on how full the file system is."

Submitted by:	Fedor Uporov
Differential Revision:	https://reviews.freebsd.org/D11211
2017-06-20 14:28:51 +00:00
Rick Macklem
ee791357a2 Add the definition of maxbcachebuf to sys/buf.h.
r320070 removed the definition of maxbcachebuf from sys/param.h to
fix the build for arm.
This patch adds the definition of maxbcachebuf to sys/buf.h, which
should be ok, since sys/buf.h is not being included in arm/arm/elf_note.S.

Suggested by:	kib
MFC after:	2 weeks
2017-06-19 22:07:53 +00:00
Rick Macklem
95ac7f1a74 Fix the NFS client/server so that it actually uses the 64bit ino_t filenos.
The code still doesn't use d_off. That will come in a future commit.
The code also removes the checks for servers returning a fileno that
doesn't fit in 32bits, since that should work ok now.
Bump __FreeBSD_version since this patch changes the interface between
the NFS kernel modules.

Reviewed by:	kib
2017-06-18 21:48:31 +00:00
Pedro F. Giffuni
5342d6e0d2 ext2fs: Enable RO huge_file feature support.
We can have support for reading ext4 "huge" files but we can't write
(anything) on ext4. and some filesystem. Formally enable the feature so
that we can mount such filesystems.

Submitted by:	Fedor Uponov
Differential Revision:	https://reviews.freebsd.org/D11209
2017-06-18 20:55:46 +00:00
Rick Macklem
1d9f01b18e Take "extern int maxbcachebuf" out of sys/param.h, since it breaks the
arm build.

In the arm build, elf_note.S includes sys/param.h and then does an
elf macro called ELFNOTE(). Although the compile error doesn't make
sense to me, I believe it just means that an "extern ..." can't exist
in param.h for this inclusion case.
I suspect adding #if !defined(LOCORE) might fix the build, but this
commit just takes the definition out.
I will ask freebsd-current@ what is the best was to deal with this
and do a subsequent commit after that.

Reported by:	melounmichal@gmail.com
2017-06-18 12:28:43 +00:00
Rick Macklem
d1c5e240a8 Make MAXBCACHEBUF a tunable called vfs.maxbcachebuf.
By making MAXBCACHEBUF a tunable, it can be increased to allow for
larger read/write data sizes for the NFS client.
The tunable is limited to MAXPHYS, which is currently 128K.
Making MAXPHYS a tunable or increasing its value is being discussed,
since it would be nice to support a read/write data size of 1Mbyte
for the NFS client when mounting the AmazonEFS file service.

Reviewed by:	kib
MFC after:	2 weeks
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D10991
2017-06-17 22:24:19 +00:00
Rick Macklem
6149361b1e Define NFS_MAXXDR as the upper bound on XDR overhead in an NFS RPC.
This definition is a part of the maxiotune2 patch that will be
committed soon. It is being committed separately to ease merging
with the pNFS projects subversion trees.

MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D10991
2017-06-12 23:41:20 +00:00
Ed Maste
db20c27d28 msdosfs: adjust #ifdefs to be similar to NetBSD
- Add header guards where missing
- Make parts available for use in makefs

Sponsored by:	The FreeBSD Foundation
2017-06-12 20:42:37 +00:00
Pedro F. Giffuni
4d7faf1b26 Remove unnecessary, and mismatched, comment.
Submitted by:	Fedor Uporov
2017-06-11 19:09:10 +00:00
Pedro F. Giffuni
b5f0918be5 extfs: fix the build with no UFS_ACL.
Some people may want to drop UFS-style ACLs for slimmer kernels.
Let's just not assume everyone needs ACLs.

Reported by:	bde
Submitted by:	Fedor Uporov
Differential Revision:	https://reviews.freebsd.org/D11145
2017-06-11 19:05:45 +00:00
Konstantin Belousov
40373cf5b8 Remove msdosfs -o large support.
Its purpose was to translate the values for msdosfs inode numbers,
which is calculated from the msdosfs structures describing the file,
into the range representable by 32bit ino_t.  The translation acted
for filesystems larger than 128Gb, it reserved the range 0xf0000000
(FILENO_FIRST_DYN) to UINT32_MAX and remembered some arbitrary
translation of ino >= FILENO_FIRST_DYN into this range.  It consumed
memory that could be only freed by unmount, and the translation was
not stable across remounts.

With ino_t type extended to 64 bit, there is no such issue and values
can be returned without compaction to 32bit.  That is, for the native
environments, the translation layer is not necessary and adds
significant undeserved code complexity.  For compat ABIs which use
32bit ino_t, the vfs.ino64_trunc_error sysctl provides some measures
to soften the failure mode when inode numbers truncation is not safe.

Discussed with:	bde
Sponsored by:	The FreeBSD Foundation
2017-06-09 12:06:22 +00:00
Conrad Meyer
109f344bb6 ext2fs(4): Fix a null dererence and clean an unclear switch
Coverity warned that the switch statement fell through.  While this was
intentional, the pattern wasn't especially clear.  I just changed it to a
conventional if pattern.

Reported by:	Coverity
CIDs:		1375851 (false positive), 1375853
Sponsored by:	Dell EMC Isilon
2017-06-03 22:39:50 +00:00
Conrad Meyer
a8b9a63980 ext2fs(4): Fix a number of broken flag checks
Introduced in r319071.

Reported by:	Coverity
CIDs:		1375847, 1375848, 1375849
Sponsored by:	Dell EMC Isilon
2017-06-03 22:30:30 +00:00
Ed Maste
6a1c2e1fce msdosfs: use mem{cpy,move,set} instead of bcopy,bzero
This somewhat simplifies use of msdosfs code in userland (for makefs),
reduces diffs with NetBSD and is standard C as of C89.

Reviewed by:	imp
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D11014
2017-06-02 18:39:53 +00:00
Stephen J. Kiernan
9a81ba0f24 Add MD_VERIFY option to enable O_VERIFY in open for vnode type.
Add -o [no]verify option to mdconfig (and document in man page.)
Implement GEOM attribute MNT::verified to ask md if the backing vnode is
  verified.
Check for MNT::verified in cd9660 mount to flag the mount as MNT_VERIFIED if
  the underlying device has been verified.

Reviewed by:	rwatson
Approved by:	sjg (mentor)
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D2902
2017-05-31 21:18:11 +00:00
Hans Petter Selasky
9a83b0971e Minor code optimisation.
Avoid locking the global CUSE lock when the polling flags are zero.

MFC after:	1 week
2017-05-31 21:05:24 +00:00
Konstantin Belousov
a8e7f543af Fix bug in r318997: remove the line which overrides vn_fsid()
calculation.

Noted by:	jhb
Reviewed by:	rmacklem
Sponsored by:	The FreeBSD Foundation
2017-05-30 21:20:54 +00:00
Pedro F. Giffuni
0322275751 Fix potential memory leak.
Moving the allocation forward, just before it's actually needed, seems
sensible.
Add newline character at the last line while here.

Reported by:	pluknet
Differential Revision:	https://reviews.freebsd.org/D10974
2017-05-28 17:48:54 +00:00
Pedro F. Giffuni
39999a6998 Support for linux ext2fs posix-draft ACLs.
This is closely tied to the Extended Attribute implementation.

Submitted by:	Fedor Uporov
Reviewed by:	kevlo, pfg

Differential Revision:	https://reviews.freebsd.org/D10807
2017-05-28 15:39:11 +00:00
Konstantin Belousov
03311f117b Use whole mnt_stat.f_fsid bits for st_dev.
Since ino64 expanded dev_t to 64bit, make VOP_GETATTR(9) provide all
bits of mnt_stat.f_fsid as va_fsid for vnodes on filesystems which use
f_fsid.  In particular, NFSv3 and sometimes NFSv4, and ZFS use this
method or reporting st_dev by stat(2).

Provide a new helper vn_fsid() to avoid duplicating code to copy
f_fsid to va_fsid.

Note that the change is mostly cosmetic.  Its motivation is to avoid
sign-extension of f_fsid[0] into 64bit dev_t value which happens after
dev_t becomes 64bit..

Reviewed by:	avg(zfs), rmacklem (nfs) (both for previous version)
Sponsored by:	The FreeBSD Foundation
2017-05-27 17:00:30 +00:00
Konstantin Belousov
6992112349 Commit the 64-bit inode project.
Extend the ino_t, dev_t, nlink_t types to 64-bit ints.  Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment.  Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.

ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks.  Unfortunately, not everything can be
fixed, especially outside the base system.  For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.

Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.

Struct xvnode changed layout, no compat shims are provided.

For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.

Update note: strictly follow the instructions in UPDATING.  Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.

Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb).  Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver.  Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem).  Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).

Sponsored by:	The FreeBSD Foundation (emaste, kib)
Differential revision:	https://reviews.freebsd.org/D10439
2017-05-23 09:29:05 +00:00
Ed Maste
6403a57bcc msdosfs: constify
Obtained from:	NetBSD
Sponsored by:	The FreeBSD Foundation
2017-05-21 20:00:49 +00:00
Ed Maste
c8b163b617 msdosfs: correct constant and typo in comment
Revisions:
fat.h	1.5 1.6

Obtained from:	NetBSD
Sponsored by:	The FreeBSD Foundation
2017-05-21 19:48:16 +00:00
Ed Maste
9287dbaae9 msdosfs: capitalize FAT appropriately
Diff reduction with NetBSD, including some nearby minor whitespace or
style fixes.

Obtained from:	NetBSD
Sponsored by:	The FreeBSD Foundation
2017-05-21 19:29:28 +00:00
Ed Maste
23c5331208 msdosfs: use C99 types
General cleanup, for diff reduction with NetBSD and future use by FAT
support in makefs.

Submitted by:	Siva Mahadevan <smahadevan@freebsdfoundation.org>
Obtained from:	NetBSD
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10821
2017-05-19 18:13:41 +00:00
Ed Maste
3e85b721d6 Remove register keyword from sys/ and ANSIfy prototypes
A long long time ago the register keyword told the compiler to store
the corresponding variable in a CPU register, but it is not relevant
for any compiler used in the FreeBSD world today.

ANSIfy related prototypes while here.

Reviewed by:	cem, jhb
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10193
2017-05-17 00:34:34 +00:00
Rick Macklem
46adb5dcf8 Make nfscl_mtofh() return ENXIO when *nfhpp == NULL.
r317272 introduced a case where nfscl_mtofh() could return 0 when
*nfhpp is NULL. This patch makes it return ENXIO for this case.

MFC after:	1 week
2017-05-15 13:14:13 +00:00
Pedro F. Giffuni
34f4388801 ext2fs: add read-write support for Extended Attributes.
Extended attributes and their particular implementation in linux are
different from FreeBSD so in this case we have started diverging from
the  UFS EA implementation, which would be the natural reference.

Depending on future progress implementing ACLs this approach may change
but for now bring to the tree an implementation that is consistent and
can be tested.

Submitted by:	Fedor Uporov

Differential Revision:	https://reviews.freebsd.org/D10460
2017-05-03 22:35:41 +00:00
Rick Macklem
845eb84c56 Modify the NFSv4.1/pNFS client to ask for a maximum length of layout.
The code specified the length of a layout as INT64_MAX instead of
UINT64_MAX. This could result in getting a layout for less than the
full file for extremely large files. Although having little practical
effect, this patch corrects this in the code.
Detected during recent testing of the pNFS server.

MFC after:	2 weeks
2017-04-29 00:34:53 +00:00
Rick Macklem
ad81354ceb Fix handling of a NFSv4.1 callback reply from the session cache.
The nfsv4_seqsession() call returns NFSERR_REPLYFROMCACHE when it has a
reply in the session, due to a requestor retry. The code erroneously
assumed a return of 0 for this case. This patch fixes this and adds
a KASSERT(). This would be an extremely rare occurrence. It was found
during code inspection during the pNFS server development.

MFC after:	2 weeks
2017-04-26 21:54:53 +00:00
Rick Macklem
858f6fe327 Allow use of a write open stateid for reading in the NFSv4 server.
The NFSv4 RFCs give a server the option of allowing the use of an open
stateid for write access to be used for a Read operation.
This patch enables this by default and adds a sysctl to disable it,
for anyone who does not want this capability.
Allowing this is particularily useful for a pNFS Data Server (DS), since
they are not permitted to allow the use of special stateids.
Discovered during recent testing of the pNFS server under development.

MFC after:	2 weeks
2017-04-24 20:46:19 +00:00
Rick Macklem
6406db24cb Make the NFSv4 client to use a write open for reading if allowed by the server.
An NFSv4 server has the option of allowing a Read to be done using a Write
Open. If this is not allowed, the server will return NFSERR_OPENMODE.
This patch attempts the read with a write open and then disables this
if the server replies NFSERR_OPENMODE.
This change will avoid some uses of the special stateids. This will be
useful for pNFS/DS Reads, since they cannot use special stateids.
It will also be useful for any NFSv4 server that does not support reading
via the special stateids. It has been tested against both types of NFSv4 server.

MFC after:	2 weeks
2017-04-23 21:51:28 +00:00
Rick Macklem
b845c29a03 Don't set the connection-back-channel flag for DS sessions.
The NFSv4.1/pNFS client does not use/need a backchannel for the Data Server (DS)
sessions, so the flag should only be set for MetaData Server (MDS) sessions.
This patch should have been a part of r317275.

MFC after:	2 weeks
2017-04-23 21:36:32 +00:00
Rick Macklem
4e47dd1885 Fix the NFSv4.1/pNFS client return layout on close.
The "return layout on close" case in the pNFS client was badly broken.
Fortunately, extant pNFS servers that I have tested against do not
do this. This patch fixes it. It also changes the way the layout stateid.seqid
is set for LayoutReturn. I think this change is correct w.r.t. the RFC,
but I am not 100% sure.
This was found during recent testing of the pNFS server under development.

MFC after:	2 weeks
2017-04-22 22:37:44 +00:00
Rick Macklem
c20a721023 Fix some krpc leaks for the NFSv4.1/pNFS client.
The NFSv4.1/pNFS client wasn't doing a newnfs_disconnect() call for the
connection to the Data Server (DS) under some circumstances. The main
effect of this was a leak of malloc'd structures in the krpc. This patch
adds the newnfs_disconnect() calls to fix this.
Detected during recent testing against the pNFS server under development.

MFC after:	2 weeks
2017-04-22 20:55:39 +00:00
Rick Macklem
0596f343f8 Don't set ND_NOMOREDATA for a failed Setattr operation (NFSv4).
The NFSv4 Setattr operation always has reply data even when it fails,
so don't set the ND_NOMOREDATA for it. This would only affect unusual
cases where Setattr fails and the RPC code wants to parse the rest of
the compound. Detected during recent development related to the pNFS server.

MFC after:	2 weeks
2017-04-21 23:01:32 +00:00
Rick Macklem
40f8ff4800 Don't create a backchannel for a DS connection.
An NFSv4.1 client connection to a Data Server (DS) should not have a
backchannel. This patch fixes the NFSv4.1/pNFS client to not do a backchannel
for this case.
Found during recent testing with the pNFS server under development.

MFC after:	2 weeks
2017-04-21 22:38:26 +00:00
Conrad Meyer
ef3c43b4e3 fuse: Implement FOPEN_KEEP_CACHE flag
Implement FUSE open flag FOPEN_KEEP_CACHE.  Without this flag, cached file
contents should be invalidated on open.  Apparently, fusefs-encfs relies
upon this behavior.

PR:		218636
Submitted by:	Ben RUBSON <ben.rubson at gmail.com>
2017-04-21 22:00:22 +00:00
Rick Macklem
e96af29419 Add checks for failed operations to the NFSv4 client function nfscl_mtofh().
The nfscl_mtofh() function didn't check for failed operations and, as such,
would have returned EBADRPC for these cases, due to parsing failure.
This patch adds checks, so that it returns with ND_NOMOREDATA set.
This is needed for future use in the pNFS server and acts as a safety
belt in the meantime.

MFC after:	2 weeks
2017-04-21 21:43:00 +00:00
Rick Macklem
8c1d0d9ce5 Set default uid/gid to nobody/nogroup for NFSv4 mapping.
The default uid/gid for NFSv4 are set by the nfsuserd(8) daemon.
However, they were 0 until the nfsuserd(8) was run. Since it is
possible to use NFSv4 without running the nfsuserd(8) daemon, set them
to nobody/nogroup initially.
Without this patch, the values would be set by the nfsuserd(8) daemon
and left changed even if the nfsuserd(8) daemon was killed. The default
values of 0 meant that setting a group to "wheel" would fail even when
done by root.
It also adds a definition of GID_NOGROUP to sys/conf.h.

Discussed on:	freebsd-current@
MFC after:	2 weeks
2017-04-21 20:08:10 +00:00
Rick Macklem
b843ada7aa Revert r317240. I didn't realize there were defined constants for
uid/gid values in sys/conf.h. I will do another commit using those.
2017-04-21 11:48:12 +00:00
Rick Macklem
1350db1780 Set default uid/gid to nobody/nogroup for NFSv4 mapping.
The default uid/gid for NFSv4 are set by the nfsuserd(8) daemon.
However, they were 0 until the nfsuserd(8) was run. Since it is
possible to use NFSv4 without running the nfsuserd(8) daemon, set them
to nobody/nogroup initially.
Without this patch, the values would be set by the nfsuserd(8) daemon
and left changed even if the nfsuserd(8) daemon was killed. Also, the default
values of 0 meant that setting a group to "wheel" would fail even when
done by root and this patch fixes this issue.

MFC after:	2 weeks
2017-04-21 01:50:41 +00:00
Rick Macklem
dedec68c32 Fix the setting of atime for Linux client NFSv4 mounts.
The FreeBSD NFSv4 server did not set the attribute bit for TimeAccess in
the reply to an Open with exclusive_create, as required by the RFCs.
(This is required since the FreeBSD NFS server stores the create_verifier
 in the va_atime attribute.)
As such, the Linux NFSv4 client did not set the TimeAccess (atime) in
the Setattr done in an RPC after the one with the Open/exclusive_create.
This patch fixes the server to set the TimeAccess bit in the reply.

I believe that storing the create_verifier in an extended attribute for
file systems that support extended attributes might be a good idea,
but I will wait for a discussion of this on the freebsd-fs@ email list
before considering committing a patch to do this.

Reported by:	jim@ks.uiuc.edu
Suggested by:	dfr
MFC after:	2 weeks
2017-04-21 00:17:47 +00:00
Gleb Smirnoff
83c9dea1ba - Remove 'struct vmmeter' from 'struct pcpu', leaving only global vmmeter
in place.  To do per-cpu stats, convert all fields that previously were
  maintained in the vmmeters that sit in pcpus to counter(9).
- Since some vmmeter stats may be touched at very early stages of boot,
  before we have set up UMA and we can do counter_u64_alloc(), provide an
  early counter mechanism:
  o Leave one spare uint64_t in struct pcpu, named pc_early_dummy_counter.
  o Point counter(9) fields of vmmeter to pcpu[0].pc_early_dummy_counter,
    so that at early stages of boot, before counters are allocated we already
    point to a counter that can be safely written to.
  o For sparc64 that required a whole dummy pcpu[MAXCPU] array.

Further related changes:
- Don't include vmmeter.h into pcpu.h.
- vm.stats.vm.v_swappgsout and vm.stats.vm.v_swappgsin changed to 64-bit,
  to match kernel representation.
- struct vmmeter hidden under _KERNEL, and only vmstat(1) is an exclusion.

This is based on benno@'s 4-year old patch:
https://lists.freebsd.org/pipermail/freebsd-arch/2013-July/014471.html

Reviewed by:	kib, gallatin, marius, lidl
Differential Revision:	https://reviews.freebsd.org/D10156
2017-04-17 17:34:47 +00:00
Gleb Smirnoff
ca148cda3b Two more files missed in r317055: these files need sys/vmmeter.h, but now
they got it implicitly included via sys/pcpu.h.
2017-04-17 17:20:48 +00:00
Gleb Smirnoff
9ed01c32e0 All these files need sys/vmmeter.h, but now they got it implicitly
included via sys/pcpu.h.
2017-04-17 17:07:00 +00:00
Rick Macklem
6d4377c1ae Remove unused "cred" argument to ncl_flush().
The "cred" argument of ncl_flush() is unused and it was confusing to have
the code passing in NULL for this argument in some cases. This patch deletes
this argument.
There is no semantic change because of this patch.

MFC after:	2 weeks
2017-04-14 13:25:45 +00:00
Rick Macklem
037a2012e9 Add an NFSv4.1 mount option for "use one openowner".
Some NFSv4.1 servers such as AmazonEFS can only support a small fixed number
of open_owner4s. This patch adds a mount option called "oneopenown" that
can be used for NFSv4.1 mounts to make the client do all Opens with the
same open_owner4 string. This option can only be used with NFSv4.1 and
may not work correctly when Delegations are is use.

Reported by:	cperciva
Tested by:	cperciva
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D8988
2017-04-13 21:54:19 +00:00
Rick Macklem
60287c0c03 Add call to svcpool_close() for the NFSv4 callback pool (svcpool_nfscbd).
A function called svcpool_close() was added to the server side krpc by
r313735, so that a pool could be closed without destroying the data structures.
This little patch adds a call to it for the callback pool (svcpool_nfscbd),
so that the nfscbd daemon can be killed/restarted and continue to work
correctly.

MFC after:	2 weeks
2017-04-13 20:16:29 +00:00
Rick Macklem
0649fcaea5 Fix the NFS client for "text file modified, process killed" mmap'd case.
When an mmap'd text file is written and then executed immediately
afterwards, it was possible that the modify time would change after the
text file was executing, resulting in the process executing the file
being killed. This was usually only observed when the file system's
times were set to higher resolution, but could have occurred for any
time resolution.
This was reported on a recent email list discussion.
This patch adds a VOP_SET_TEXT() to the NFS client which flushed all
dirty pages to the NFS server and then makes sure that n_mtime is up
to date to avoid this from occurring.
Thanks go to kib@ and pho@ for their help with developing this patch.

Tested by:	pho
Reviewed by:	kib
MFC after:	2 weeks
2017-04-12 21:37:12 +00:00
Rick Macklem
7d9b62e1a1 Don't throw away Open state when a NFSv4.1 client recovery fails.
If the ExchangeID/CreateSession operations done by an NFSv4.1 client
after the server crashes/reboots fails, it is possible that some process/thread
is waiting for an open_owner lock. If the client state is free'd, this
can cause a crash.
This would not normally happen, but has been observed on a mount of the
AmazonEFS service.

Reported by:	cperciva
Tested by:	cperciva
PR:		216086
MFC after:	2 weeks
2017-04-11 22:47:02 +00:00
Rick Macklem
9e507a6a00 During a server crash recovery, fix the NFSv4.1 client for a NFSERR_BADSESSION
during recovery.

If the NFSv4.1 client gets a NFSv4.1 NFSERR_BADSESSION reply to an Open/Lock
operation while recovering from the server crash/reboot, allow the opens
to be retained for a subsequent recovery attempt. Since NFSv4.1 servers
should only reply NFSERR_BADSESSION after a crash/reboot that has lost
state, this case should almost never happen.
However, for the AmazonEFS file service, this has been observed when
the client does a fresh TCP connection for RPCs.

Reported by:	cperciva
Tested by:	cperciva
PR:		216088
MFC after:	2 weeks
2017-04-11 20:28:15 +00:00