Commit Graph

3394 Commits

Author SHA1 Message Date
Hans Petter Selasky
9570004318 Don't use POLLNVAL as a return value from the client side poll
function. Many existing clients don't understand POLLNVAL and instead
relies on an error code from the read(), write() or ioctl() system
call. Also make sure we wakeup any client pollers before the cuse
server is closing, so they don't wait forever for an event.
2015-01-13 13:32:18 +00:00
Ed Maste
4882501b5c ANSIfy msdosfs
Add a few cases and style(9) fixes missed in r276887

Sponsored by:	The FreeBSD Foundation
2015-01-12 21:55:48 +00:00
Ed Maste
10c9700f3e ANSIfy sys/fs/msdosfs
There are a number of msdosfs improvements in NetBSD that may be worth
bringing over, and this reduces noise in the comparison.

Differential Revision:	https://reviews.freebsd.org/D1466
Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
2015-01-09 14:50:08 +00:00
Robert Watson
eae6da3db4 Use M_SIZE() instead of hand-crafted (and mostly correct) NFSMSIZ() macro
in the NFS server; garbage collect now-unused NFSMSIZ() and M_HASCL()
macros.  Also garbage collect now-unused versions in headers for the
removed previous NFS client and server.

Reviewed by:	rmacklem
Sponsored by:	EMC / Isilon Storage Division
2015-01-07 17:22:56 +00:00
Mateusz Guzik
cd29b292b8 Convert nullfs hash lock from a mutex to an rwlock. 2014-12-30 21:41:35 +00:00
Rick Macklem
07d491dede r245508 modified the NFS client's Setattr RPC to
use VA_UTIMES_NULL to indicate whether it should
set the time to the current tod on the server.
This had the side effect of making the NFS client
use the client's timestamp for exclusive create,
starting with FreeBSD9.2.
Unfortunately a bug in some Solaris NFS servers
causes these servers to return NFS_OK to the
Setattr RPC done during exclusive create, but not
actually set the file's mode, leaving the file's
mode == 0.
This patch restores the NFS client's behaviour to
use the server's tod for the exclusive open's
Setattr RPC, to avoid the Solaris server bug and
to restore the pre-FreeBSD9.2 NFS behaviour.

Discussed on:	freebsd-fs
PR:	186293
MFC after:	3 months
2014-12-28 21:13:52 +00:00
Rick Macklem
2f88b3d20a Delete some duplicate code that was harmless because
exactly the same code is at the end of the nfscl_checksattr()
function that is called just before it. As such, this code
had already been executed and didn't do anything.

MFC after:	1 week
2014-12-25 22:29:37 +00:00
Rick Macklem
52f1bb38c2 A deadlock in the NFSv4 server with vfs.nfsd.enable_locallocks=1
was reported via email. This was caused by a LOR between the
sleep lock used to serialize the local locking (nfsrv_locklf())
and locking the vnode. I believe this patch fixes the problem
by delaying relocking of the vnode until the sleep lock is
unlocked (nfsrv_unlocklf()). To avoid nfsvno_advlock() having the side
effect of unlocking the vnode, unlocking the vnode was moved to before
the functions that call nfsvno_advlock().
It shouldn't affect the execution of the default case where
vfs.nfsd.enable_locallocks=0.

Reported by:	loic.blot@unix-experience.fr
Discussed with:	kib
MFC after:	1 week
2014-12-25 01:55:17 +00:00
Rick Macklem
62c23db947 Fix kernel builds with "options NFS_DEBUG" that
were broken by r276096. Also delete the two
kernel options NFS_GATHERDELAY, NFS_WDELAYHASHSIZ
which are no longer used.

Reported by:	bz
2014-12-23 14:24:36 +00:00
Rick Macklem
c15882f091 Remove the old NFS client and server from head,
which means that the NFSCLIENT and NFSSERVER
kernel options will no longer work. This commit
only removes the kernel components. Removal of
unused code in the user utilities will be done
later. This commit does not include an addition
to UPDATING, but that will be committed in a
few minutes.

Discussed on: freebsd-fs
2014-12-23 00:47:46 +00:00
Konstantin Belousov
789bdfdbc6 Handle MAKEENTRY cnp flag in the VOP_CREATE(). Curiously, some
fs, e.g. smbfs, already did it.

Tested by:	pho (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-12-21 13:29:33 +00:00
Benno Rice
6d659a5d9b Adjust the test of a KASSERT to better match the intent.
This assertion was added in r246213 as a guard against corrupted mbufs
arriving from drivers, the key distinguishing factor of said mbufs being
that they had a negative length. Given we're in a while loop specifically
designed to skip over zero-length mbufs, panicking on a zero-length mbuf
seems incorrect.

No objection from:	kib
2014-12-19 19:09:22 +00:00
Konstantin Belousov
6c21f6edb8 The VOP_LOOKUP() implementations for CREATE op do not put the name
into namecache, to avoid cache trashing when doing large operations.
E.g., tar archive extraction is not usually followed by access to many
of the files created.

Right now, each VOP_LOOKUP() implementation explicitely knowns about
this quirk and tests for both MAKEENTRY flag presence and op != CREATE
to make the call to cache_enter().  Centralize the handling of the
quirk into VFS, by deciding to cache only by MAKEENTRY flag in VOP.
VFS now sets NOCACHE flag for CREATE namei() calls.

Note that the change in semantic is backward-compatible and could be
merged to the stable branch, and is compatible with non-changed
third-party filesystems which correctly handle MAKEENTRY.

Suggested by:	Chris Torek <torek@pi-coral.com>
Reviewed by:	mckusick
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-12-18 10:01:12 +00:00
Gleb Kurtsou
dde58752db Adjust printf format specifiers for dev_t and ino_t in kernel.
ino_t and dev_t are about to become uint64_t.

Reviewed by:	kib, mckusick
2014-12-17 07:27:19 +00:00
Pedro F. Giffuni
8f87059b41 ext2fs: Fix old out-of-bounds access.
Overrunning buffer pointed to by (caddr_t)&oip->i_db[0] of 48 bytes by
passing it to a function which accesses it at byte offset 59 using
argument 60UL.

The issue was inherited from an older FFS implementation and
fixed there with by merging UFS2 in r98542. We follow the
FFS fix.

Discussed with:	bde
CID:		1007665
MFC after:	3 days
2014-12-09 14:56:00 +00:00
Konstantin Belousov
b2344ab5ff Do not call VFS_SYNC() before VFS_UNMOUNT() for forced unmount.
Since VFS does not/cannot stop writes, sync might run indefinitely, or
be a wrong thing to do at all.  E. g. NFS ignores VFS_SYNC() for
forced unmounts, since non-responding server does not allow sync to
finish.  On the other hand, filesystems can and do stop writes using
fs-specific facilities, and should already fully flush caches in
VFS_UNMOUNT() due to the race.

Adjust msdosfs tp sync in unmount for forced call, to accomodate the
new behaviour.  Note that it is still racy, since writes are not
stopped.

Discussed with:	avg, bjk, mckusick
Reported and tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	3 weeks
2014-12-09 10:00:47 +00:00
Konstantin Belousov
5c7bebf961 The process spin lock currently has the following distinct uses:
- Threads lifetime cycle, in particular, counting of the threads in
  the process, and interlocking with process mutex and thread lock.
  The main reason of this is that turnstile locks are after thread
  locks, so you e.g. cannot unlock blockable mutex (think process
  mutex) while owning thread lock.

- Virtual and profiling itimers, since the timers activation is done
  from the clock interrupt context.  Replace the p_slock by p_itimmtx
  and PROC_ITIMLOCK().

- Profiling code (profil(2)), for similar reason.  Replace the p_slock
  by p_profmtx and PROC_PROFLOCK().

- Resource usage accounting.  Need for the spinlock there is subtle,
  my understanding is that spinlock blocks context switching for the
  current thread, which prevents td_runtime and similar fields from
  changing (updates are done at the mi_switch()).  Replace the p_slock
  by p_statmtx and PROC_STATLOCK().

The split is done mostly for code clarity, and should not affect
scalability.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-11-26 14:10:00 +00:00
Edward Tomasz Napierala
e3d5f1fe3b Implement "automount -c".
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-11-22 16:48:29 +00:00
Edward Tomasz Napierala
836856e3e6 Fix smbfs to not zero out statfs f_flags field. Previously, this
made getmntinfo() return empty flags for smbfs filesystems when
called with MNT_WAIT. It's not visible with mount(8), since it uses
MNT_NOWAIT, but broke autounmount(8) operation.

PR:		195161
Differential Revision:	https://reviews.freebsd.org/D1194
Reviewed by:	kib@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-11-21 06:21:39 +00:00
Pedro F. Giffuni
33587684a6 ifdef ext2_print_inode which is not really used.
ext2_print_inode is not really used but it was nice to
have for initial development work. #ifdef it under a
new EXT2FS_DEBUG knob so that we don't spend time
compiling it.

MFC after:	3 days
2014-11-12 16:23:56 +00:00
Mateusz Guzik
4dba07b216 Fix up some session-related races in devfs.
One was introduced with r272596, the rest was there to begin with.

Noted by: jhb
2014-11-03 03:12:15 +00:00
Edward Tomasz Napierala
2fbe0cff73 Fix handling of "conn" mount_nfs(8) option.
Reviewed by:	rmacklem@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-10-30 09:25:03 +00:00
Edward Tomasz Napierala
5a06ac3540 Add support for "timeo", "actimeo", "noac", and "proto" options
to mount_nfs(8).  They are implemented on Linux, OS X, and Solaris,
and thus can be expected to appear in automounter maps.

Reviewed by:	rmacklem@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-10-30 08:50:01 +00:00
Konstantin Belousov
42ecb595f2 Allow the vfs.nfsd knobs to be set from loader.conf (or using
kenv(8)).  This is useful when nfsd is loaded as module.

Reviewed by:	rmacklem
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-10-27 07:47:13 +00:00
Rick Macklem
6a30c96cdc Clip the settings for the NFS rsize, wsize mount options
to a power of 2. For non-power of 2 settings, intermittent
page faults have been reported. Although the bug that causes
these page faults/crashes has not been identified, it does
not appear to occur when rsize, wsize is a power of 2.

Reported by:	tcberner@gmail.com
MFC after:	2 weeks
2014-10-22 22:27:51 +00:00
Rick Macklem
fcf121d481 Revert r273481 so it can be recoded using fls(), which
some feel will make it more readable.
2014-10-22 21:57:35 +00:00
Rick Macklem
88cc4e92da Clip the settings for the NFS rsize, wsize mount options
to a power of 2. For non-power of 2 settings, intermittent
page faults have been reported. Although the bug that causes
these page faults/crashes has not been identified, it does
not appear to occur when rsize, wsize is a power of 2.

Reported by:	tcberner@gmail.com
MFC after:	2 weeks
2014-10-22 20:47:11 +00:00
Mateusz Guzik
12e2a30ef9 tmpfs: allow shared file lookups
Tested by: pho
2014-10-21 21:27:13 +00:00
Hans Petter Selasky
f0188618f2 Fix multiple incorrect SYSCTL arguments in the kernel:
- Wrong integer type was specified.

- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.

- Logical OR where binary OR was expected.

- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.

- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.

- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.

- Updated "EXAMPLES" section in SYSCTL manual page.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2014-10-21 07:31:21 +00:00
Mateusz Guzik
4fce16e4c9 Provide vfs suspension support only for filesystems which need it, take
two.

nullfs and unionfs need to request suspension if underlying filesystem(s)
use it. Utilize mnt_kern_flag for this purpose.

This is a fixup for 273271.

No strong objections from: kib
Pointy hat to: mjg
MFC after:	2 weeks
2014-10-20 18:00:50 +00:00
Mateusz Guzik
a8a07fd613 unionfs: hold mount interlock while manipulating mnt_flag
This is for consistency with other filesystems.
2014-10-20 17:53:49 +00:00
Mateusz Guzik
020b8f17a0 Provide vfs suspension support only for filesystems which need it.
Need is expressed by providing vfs_susp_clean function in vfsops.

Differential Revision:	D952
Reviewed by:	kib (previous version)
MFC after:	2 weeks
2014-10-19 06:59:33 +00:00
Edward Tomasz Napierala
5742494d29 Remove useless debug.
Sponsored by:	The FreeBSD Foundation
2014-10-17 12:06:48 +00:00
Marcelo Araujo
f9246664f5 Make the sysctl(8) for checkutf8 positively defined and improve
the description of it.

Submitted by:	Ronald Klop <ronald-lists@klop.ws>
Reviewed by:	rmacklem
Approved by:	rmacklem
Sponsored by:	QNAP Systems Inc.
2014-10-17 02:11:09 +00:00
Davide Italiano
2be111bf7d Follow up to r225617. In order to maximize the re-usability of kernel code
in userland rename in-kernel getenv()/setenv() to kern_setenv()/kern_getenv().
This fixes a namespace collision with libc symbols.

Submitted by:   kmacy
Tested by:      make universe
2014-10-16 18:04:43 +00:00
Marcelo Araujo
3dd6b7ff3d Add two sysctl(8) to enable/disable NFSv4 server to check when setting
user nobody and/or setting group nogroup as owner of a file or directory.
Usually at the client side, if there is an username that is not in the
client's passwd database, some clients will send 'nobody@<your.dns.domain>'
in the wire and the NFSv4 server will treat it as an ERROR.
However, if you have a valid user nobody in your passwd database,
the NFSv4 server will treat it as a NFSERR_BADOWNER as its believes the
client doesn't has the username mapped.

Submitted by:	Loic Blot <loic.blot@unix-experience.fr>
Reviewed by:	rmacklem
Approved by:	rmacklem
MFC after:	2 weeks
2014-10-16 02:24:19 +00:00
Konstantin Belousov
e511f63a04 Style changes for deadfs:
- ANSIfy VOPs.
- Remove trivial comments.
- Remove ARGSUSED.
- Remove copies of the vop_XXX_args structure definitions in comments.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-10-15 13:22:33 +00:00
Konstantin Belousov
f12aa60c62 When vnode bypass cannot be performed on the cdev file descriptor for
read/write/poll/ioctl, call standard vnode filedescriptor fop.  This
restores the special handling for terminals by calling the deadfs VOP,
instead of always returning ENXIO for destroyed devices or revoked
terminals.

Since destroyed (and not revoked) device would use devfs_specops VOP
vector, make dead_read/write/poll non-static and fill VOP table with
pointers to the functions, to instead of VOP_PANIC.

Noted and reviewed by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-10-15 13:16:51 +00:00
Konstantin Belousov
dd0218f2bd Change the deadfs poll VOP to return POLLIN|POLLRDNORM if the caller
is interested in i/o state.  Return POLLNVAL for invalid bits, similar
to poll_no_poll().  Note that POLLOUT must not be returned, since
POLLHUP is set.

Noted and reviewed by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-10-15 13:08:53 +00:00
Edward Tomasz Napierala
4cdc52bdef Make automountd(8) inform autofs(4) whether directory being handled can
have wildcards.  This makes it possible for autofs(4) to avoid requesting
automountd(8) action on access to nonexistent nodes - unless wildcards
are actually used.

Note that this change breaks ABI for automountd(8).

Tested by:	dhw@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-10-15 09:28:45 +00:00
Konstantin Belousov
df5c9c0411 Do not set IN_ACCESS flag for read-only mounts. The IN_ACCESS
survives remount in rw, also it is set for vnodes on rootfs before
noatime can be set or clock is adjusted.  All conditions result in
wrong atime for accessed vnodes.

Submitted by:	bde
MFC after:	1 week
2014-10-11 19:09:56 +00:00
Edward Tomasz Napierala
3e4adf76c4 Add assertion to catch duplicated notes.
Sponsored by:	The FreeBSD Foundation
2014-10-11 05:11:23 +00:00
Edward Tomasz Napierala
1609230854 Remove remnants of some cleanup; no functional changes.
Sponsored by:	The FreeBSD Foundation
2014-10-09 18:49:58 +00:00
Edward Tomasz Napierala
5d28b9ed32 Simplify; no functional changes.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-10-08 09:44:02 +00:00
Mateusz Guzik
3a222fe000 devfs: tidy up after 272596
This moves a var to an if statement, no functional changes.

MFC after:	1 week
2014-10-06 07:22:48 +00:00
Mateusz Guzik
1d1b55fbba devfs: don't take proctree_lock unconditionally in devfs_close
MFC after:	1 week
2014-10-06 06:20:35 +00:00
Edward Tomasz Napierala
d19c297e5f Make autofs use shared vnode locks.
Reviewed by:	kib
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-10-04 09:37:40 +00:00
Edward Tomasz Napierala
baf9297e87 Fix autofs debug macros.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-10-03 10:18:22 +00:00
Edward Tomasz Napierala
2eaebf35d6 Make autofs(4) use shared lock for lookups, instead of exclusive one.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2014-10-03 09:58:05 +00:00
Marcelo Araujo
d8a5961f88 Fix failures and warnings reported by newpynfs20090424 test tool.
This fix addresses only issues with the pynfs reports, none of these
issues are know to create problems for extant real clients.

Submitted by:	Bart Hsiao <bart.hsiao@gmail.com>
Reworked by:	myself
Reviewed by:	rmacklem
Approved by:	rmacklem
Sponsored by:	QNAP Systems Inc.
2014-10-03 02:24:41 +00:00
Edward Tomasz Napierala
28a23d3d82 Call uma_zfree() outside of lock, and improve comment.
Sponsored by:	The FreeBSD Foundation
2014-10-02 10:37:56 +00:00
Edward Tomasz Napierala
759489f9e2 Make autofs timeout handling use timeout task instead of callout;
that's because the handler can sleep on sx lock.

Reviewed by:	kib
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2014-10-02 10:31:32 +00:00
Edward Tomasz Napierala
5a66f6b3c4 Fix thinko that, with two map entries like shown below, in that order,
made autofs mix them up: the second one wasn't visible in ls(1) output,
and trying to access it would trigger mount for the first one.

foobar		host:/foobar
foo		host:/foo

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2014-09-23 11:27:43 +00:00
John Baldwin
9696feebe2 Add a new fo_fill_kinfo fileops method to add type-specific information to
struct kinfo_file.
- Move the various fill_*_info() methods out of kern_descrip.c and into the
  various file type implementations.
- Rework the support for kinfo_ofile to generate a suitable kinfo_file object
  for each file and then convert that to a kinfo_ofile structure rather than
  keeping a second, different set of code that directly manipulates
  type-specific file information.
- Remove the shm_path() and ksem_info() layering violations.

Differential Revision:	https://reviews.freebsd.org/D775
Reviewed by:	kib, glebius (earlier version)
2014-09-22 16:20:47 +00:00
Edward Tomasz Napierala
31a4b1aa03 Turns out -1 is a perfectly valid error number, ERESTART. Remove useless
code written under assumption that it wasn't.

Sponsored by:	The FreeBSD Foundation
2014-09-21 10:34:15 +00:00
Edward Tomasz Napierala
787b524980 Fix typos.
Sponsored by:	The FreeBSD Foundation
2014-09-18 10:33:23 +00:00
Konstantin Belousov
d15b55c554 Provide the unique implementation for the VOP_GETPAGES() method used
by ffs and ext2fs.  Remove duplicated call to vm_page_zero_invalid(),
done by VOP and by vm_pager_getpages().  Use vm_pager_free_nonreq().

Reviewed by:	alc (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	6 weeks (after r271596)
2014-09-15 12:28:29 +00:00
Alan Cox
396b3e34b4 Avoid an exclusive acquisition of the object lock on the expected execution
path through the NFS clients' getpages functions.

Introduce vm_pager_free_nonreq().  This function can be used to eliminate
code that is duplicated in many getpages functions.  Also, in contrast to
the code that currently appears in those getpages functions,
vm_pager_free_nonreq() avoids acquiring an exclusive object lock in one
case.

Reviewed by:	kib
MFC after:	6 weeks
Sponsored by:	EMC / Isilon Storage Division
2014-09-14 18:07:55 +00:00
Alan Cox
3e5c84e292 We don't need an exclusive object lock on the expected execution path
through {ext2,ffs}_getpages().

Reviewed by:	kib, pfg
MFC after:	6 weeks
Sponsored by:	EMC / Isilon Storage Division
2014-09-13 18:26:13 +00:00
Pedro F. Giffuni
6e582ca2a5 Extra space from r271467.
MFC after:	2 months
2014-09-12 15:54:18 +00:00
Pedro F. Giffuni
1d0fce9bfe ext2fs: add ext2_getpages().
Literally copy/pasted from ffs_getpages().

Tested with:	fsx
MFC after:	2 months
2014-09-12 15:49:21 +00:00
Gleb Smirnoff
27ad26d8c7 Remove unused arguments for VOP_GETPAGES(), VOP_PUTPAGES(). 2014-09-10 12:36:41 +00:00
Robert Watson
70ac4fa640 Garbage collect NFSMINOFF() from the NFS stack; this unused macro replicates
mbuf-initialisation logic that is best left to centralised mbuf utility
code rather than scattered around the kernel.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2014-09-05 17:05:51 +00:00
Edward Tomasz Napierala
7cb570d0ae Fix bug that, assuming a/ is a root of NFS filesystem mounted on autofs,
prevented "mv a/from a/to" from working, while "cd a && mv from to" was ok.

PR:		192948
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2014-08-24 17:03:52 +00:00
Edward Tomasz Napierala
f81018caf2 Autofs softc needs to be global anyway, so don't pass it as a local
variable, and don't store in autofs_mount.  Also rename it from 'sc'
to 'autofs_softc', since it's global and extern.

MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2014-08-23 11:45:14 +00:00
Edward Tomasz Napierala
a32ba4e63e Add comment explaining one of the quirks in autofs.
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2014-08-23 11:38:31 +00:00
Edward Tomasz Napierala
f5440d1a9d Fix includes.
Suggested by:	pluknet@
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2014-08-21 15:59:25 +00:00
Edward Tomasz Napierala
925fd94584 Use __FBSDID() properly.
Suggested by:	pluknet@
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2014-08-21 15:07:25 +00:00
Edward Tomasz Napierala
2f438a99cd Rework ".." lookup; previous one failed to properly busy the mountpoint.
Reviewed by:	kib@
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2014-08-20 13:46:51 +00:00
Edward Tomasz Napierala
3914ddf8a7 Bring in the new automounter, similar to what's provided in most other
UNIX systems, eg. MacOS X and Solaris.  It uses Sun-compatible map format,
has proper kernel support, and LDAP integration.

There are still a few outstanding problems; they will be fixed shortly.

Reviewed by:	allanjude@, emaste@, kib@, wblock@ (earlier versions)
Phabric:	D523
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2014-08-17 09:44:42 +00:00
Rick Macklem
03738f6076 Change the NFS server's printf related to hitting
the DRC cache's flood level so that it suggests
increasing vfs.nfsd.tcphighwater.

Suggested by:	h.schmalzbauer@omnilan.de
2014-08-10 01:13:32 +00:00
Konstantin Belousov
effc6a3593 VOP_LOOKUP() may relock the directory vnode for some reasons. Since
nullfs vnode shares vnode lock with lower vnode, this allows the
reclamation of nullfs directory vnode in null_lookup().  In this
situation, VOP must return ENOENT.

More, since after the reclamation, the locks of nullfs directory vnode
and lower vnode are no longer shared, the relock of the ldvp does not
restore the correct locking state of dvp, and leaks ldvp lock.
Correct this by unlocking ldvp and locking dvp.

Use cached value of dvp->v_mount.

Reported by:	bdrewery
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-08-08 11:39:05 +00:00
Pedro F. Giffuni
7c1be3f5c7 Revert r269523:
Providing a higher EXT2_LINK_MAX limit is a bad idea for ext2/3.

Discussed with:	bde
2014-08-05 01:25:14 +00:00
Pedro F. Giffuni
55806cca37 set EXT2_LINK_MAX to LINK_MAX
In linux EXT4_LINK_MAX is now 64000.  We can't really do that
since i_nlink and va_nlink are signed so setting higher values
is likely to cause trouble.

This is a system limitation so set the EXT_LINK_MAX to
what the system can handle.

MFC after:	3 days
2014-08-04 16:41:06 +00:00
Warner Losh
2296211265 Set the erase block size properly in the case the underlying media
doesn't advertise an erase block size.

Submitted by: bjg@
Pointy hat to: imp@
2014-08-02 05:05:16 +00:00
Warner Losh
d48bcddafe Follow the ufs practice for disallowing permission changes as well as
writes to files for read-only file systems. Since there are already
checks in nandfs_setattr that return an error, this moves detection of
the error earlier.
2014-08-02 05:05:10 +00:00
Warner Losh
b65c3b8fab Fix a minor style(9) issue. 2014-08-02 05:05:05 +00:00
Konstantin Belousov
e7375b6fa5 Do not generate 1000 unique lock names for nfsrc hash chain locks.
It overflows witness.

Shorten the names of some nfs mutexes.

Reported and tested by:	pho
No objections from:	rmacklem, mav
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-07-31 19:24:44 +00:00
Konstantin Belousov
0ebe0000b6 Assert that nullfs vnode has VV_ROOT set whenever lower vnode has.
Assert that dotdot lookup on the root vnode is not performed.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-07-28 14:20:31 +00:00
Konstantin Belousov
289dd6dd7c Fix typo.
MFC after:	3 days
2014-07-24 23:14:03 +00:00
Warner Losh
1efe717e13 Fix typo in comment: noone -> no one.
Fix minor style(9) nits.
2014-07-23 16:18:51 +00:00
Konstantin Belousov
22bdc15a57 Do not ignore error from tmpfs_alloc_vp(). It results in access to
the random memory.

Reported and tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-16 14:08:01 +00:00
Konstantin Belousov
de75292a5b Remove unused header.
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-16 14:06:16 +00:00
Konstantin Belousov
65589a29f4 Check for the cross-device cross-link attempt in the VFS, instead of
forcing filesystem VOP_LINK() methods to repeat the code.  In
tmpfs_link(), remove redundand check for the type of the source,
already done by VFS.

Note that NFS server already performs this check before calling
VOP_LINK().

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-16 14:04:46 +00:00
Konstantin Belousov
4cda7f7ece Rework the tmpfs unmount.
- Suspend filesystem for unmount.  This prevents new tmpfs nodes from
  instantiating, and also ensures that only unmount thread can destroy
  nodes.

- Do not start tmpfs node deletion until all vnodes are reclaimed,
  which guarantees that no thread can access tmpfs data.  For this,
  call vflush() in the loop, until the mnt_nvnodelistsize is non-zero.
  Note that after mnt_nvnodelistsize becomes 0, insmntque() blocks
  insertion of a vnode germ into the mount list of vnodes.

- Fail node allocation when the filesystem is being unmounted.  This
  is race-free due to the vflush() call in loop.  This is mostly
  cosmetic, avoiding some more work which might be done until
  suspension in unmount is started.

Note that there is currently no way to prevent new vnode instantiation
from readers during the unmount.  Due to this, forced unmount might
live-lock if vflush() loop cannot get to the zero vnode count due to
races with readers.  The unmount would proceed after the load is
lifted.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 09:52:33 +00:00
Konstantin Belousov
b5b3326191 Change forgotten in r268615. Set the OBJ_TMPFS_NODE flag for
vm_object of VREG tmpfs node.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 09:35:14 +00:00
Konstantin Belousov
eb2c06b63a Use tmpfs_vn_get_ino_gen() to handle the races with reclaim in tmpfs
dotdot lookup.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 09:16:55 +00:00
Konstantin Belousov
fd63693dcf Style. Add comment about lock mode.
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 09:13:56 +00:00
Konstantin Belousov
7a41bc2f41 In tmpfs_alloc_file(), code after the 'out' label does only 'return
error;'.  Replace goto's with the return.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 09:02:40 +00:00
Konstantin Belousov
d2ca06cdd2 Add convenience macro to assert tmpfs node lock.
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 08:59:25 +00:00
Konstantin Belousov
55781cb922 Add some assertions for the code handling vm_object for tmpfs vnode.
In particular, vnode must be exclusively locked when the tmpfs vnode
and object are divorced.  When the vnode is opened, the object must be
still alive, since only live vnode can be opened, and the tmpfs node
owns a reference on the object.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 08:55:02 +00:00
Konstantin Belousov
706f80801d The tmpfs_link() must not dereference the filesystem-specific data for
a vnode until it is verified that the vnode indeed belongs to tmpfs
mount.  Otherwise, it might access random memory, at least in the
debug kernel.

Reported and tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 08:45:29 +00:00
Konstantin Belousov
a69452162a Generalize vn_get_ino() to allow filesystems to use custom vnode
producer, instead of hard-coding VFS_VGET().  New function, which
takes callback, is called vn_get_ino_gen(), standard callback for
vn_get_ino() is provided.

Convert inline copies of vn_get_ino() in msdosfs and cd9660 into the
uses of vn_get_ino_gen().

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 08:34:54 +00:00
Konstantin Belousov
fca015d301 Remove code separator lines which do not conform to style(9).
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-07-14 08:17:11 +00:00
Warner Losh
cfe87f0076 Naughty NANDFS was using hidden unused flag, hiding the fact that the
flag was used and wasn't really available. Change the name without
fixing any laying issues that might be present in NANDFS' use of this
flag.
2014-07-07 23:21:07 +00:00
Rick Macklem
6c7d2293d3 The new NFSv3 server did not generate directory postop attributes for
the reply to ReaddirPlus when the server failed within the loop
that calls VFS_VGET(). This failure is most likely an error
return from VFS_VGET() caused by a bogus d_fileno that was
truncated to 32bits.
This patch fixes the server so that it will return directory postop
attributes for the failure. It does not fix the underlying issue caused
by d_fileno being uint32_t when a file system like ZFS generates
a fileno that is greater than 32bits.

Reported by:	jpaetzel
Reviewed by:	jpaetzel
MFC after:	1 month
2014-07-04 22:47:07 +00:00
Rick Macklem
c59e4cc34d Merge the NFSv4.1 server code in projects/nfsv4.1-server over
into head. The code is not believed to have any effect
on the semantics of non-NFSv4.1 server behaviour.
It is a rather large merge, but I am hoping that there will
not be any regressions for the NFS server.

MFC after:	1 month
2014-07-01 20:47:16 +00:00
Bryan Drewery
4fc0f18c20 Change NFS readdir() to only ignore cookies preceding the given offset for
UFS rather than for all but ZFS.  This code was assuming that offsets were
monotonically increasing for all file systems except ZFS and that the
cookies from a previous call may have been rewound to a block boundary.
According to mckusick@ only UFS is known to do this, so only requests against
UFS file systems should remove cookies smaller than the given offset.  This
fixes serving TMPFS over NFS as it too does not have monotonically increasing
offsets.  The comment around the code also indicated it was specific to UFS.

Some of the code using 'not_zfs' is specific to ZFS snapshot handling, so
add a 'is_zfs' variable for those cases.

It's possible that 'is_zfs' check for VFS_VGET() support may not be
specific to ZFS.  This needs more research and testing.

After this fix TMPFS and other file systems can be served over NFS.

To test I compared the results of syncing a /usr/src tree into a tmpfs and
serving that over NFS.  Before the fix 3589 files were missing on the remote
view.  After the fix all files were successfully found.

Reviewed by:	rmacklem
Discussed with:	mckusick, rmacklem via fs@
Discussed at:	http://lists.freebsd.org/pipermail/freebsd-fs/2014-April/019264.html
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2014-07-01 20:00:35 +00:00
Rick Macklem
2d5f835917 There might be a potential race condition for the NFSv4 client
when a newly created file has another open done on it that
update the open mode. This patch moves the code that updates
the open mode up into the block where the mutex is held to
ensure this cannot happen. No bug caused by this potential
race has been observed, but this fix is a safety belt to ensure
it cannot happen.

MFC after:	2 weeks
2014-06-28 21:47:15 +00:00
Hans Petter Selasky
5928e3b9f4 Use existing PHOLD() and PRELE() macros.
Submitted by:	kib @
2014-06-24 18:25:43 +00:00
Konstantin Belousov
7b81a399a4 In msdosfs_setattr(), add a check for result of the utimes(2)
permissions test, forgotten in r164033.

Refactor the permission checks for utimes(2) into vnode helper
function vn_utimes_perm(9), and simplify its code comparing with the
UFS origin, by writing the call to VOP_ACCESSX only once.  Use the
helper for UFS(5), tmpfs(5), devfs(5) and msdosfs(5).

Reported by:	bde
Reviewed by:	bde, trasz
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-06-17 07:11:00 +00:00
Rick Macklem
ca4defd583 The new NFS server would not allow a hard link to be
created to a symlink. This restriction (which was
inherited from OpenBSD) is not required by the NFS RFCs.
Since this is allowed by the old NFS server, it is a
POLA violation to not allow it. This patch modifies the
new NFS server to allow this.

Reported by:	jhb
Reviewed by:	jhb
MFC after:	3 days
2014-06-06 21:38:49 +00:00
Konstantin Belousov
60c5c866aa Allow shared locking for the tmpfs vnodes.
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2014-06-04 15:30:49 +00:00
Hans Petter Selasky
fa0f6e62c6 Initial import of character device in userspace support for FreeBSD.
The CUSE library is a wrapper for the devfs kernel functionality which
is exposed through /dev/cuse . In order to function the CUSE kernel
code must either be enabled in the kernel configuration file or loaded
separately as a module. Currently none of the committed items are
connected to the default builds, except for installing the needed
header files. The CUSE code will be connected to the default world and
kernel builds in a follow-up commit.

The CUSE module was written by Hans Petter Selasky, somewhat inspired
by similar functionality found in FUSE. The CUSE library can be used
for many purposes. Currently CUSE is used when running Linux kernel
drivers in user-space, which need to create a character device node to
communicate with its applications. CUSE has full support for almost
all devfs functionality found in the kernel:
 - kevents
 - read
 - write
 - ioctl
 - poll
 - open
 - close
 - mmap
 - private per file handle data

Requested by several people. Also see "multimedia/cuse4bsd-kmod" in
ports.
2014-05-23 08:46:28 +00:00
Konstantin Belousov
08cf5ceb8e After r254627, the deupdate() started writing the directory entries to
disk.  That has a side effect of corrupting the "." entries names on
rename, since the call to createde() in the msdosfs_rename() sets the
de_Name to the target name.  If any change to the directory attributes
is performed, the wrong name is written back to the on-disk direntry
on update.

Overwrite the de_Name for the directories on rename to correct the dot
name.

Submitted by:	bde
MFC after:	1 week
2014-05-03 16:11:55 +00:00
Rick Macklem
ca20bd924f The new draft specification for NFSv4.0 specifies that a server
should either accept owner and owner_group strings that are just
the digits of the uid/gid or return NFS4ERR_BADOWNER.
This patch adds a sysctl vfs.nfsd.enable_stringtouid, which can
be set to enable the server w.r.t. accepting numeric string. It
also ensures that NFS4ERR_BADOWNER is returned if numeric uid/gid
strings are not enabled. This fixes the server for recent Linux
nfs4 clients that use numeric uid/gid strings by default.

Reported and tested by:	craigyk@gmail.com
MFC after:	2 weeks
2014-05-03 00:13:45 +00:00
Mateusz Guzik
183870cf75 Ignore the error from pipespace_new when creating a pipe.
It can fail if pipe map is exhausted (as a result of too many pipes created),
but it is not fatal and could be provoked by unprivileged users. The only
consequence is worse performance with given pipe.

Reported by:	ivoras
Suggested by:	kib
MFC after:	1 week
2014-05-02 00:52:13 +00:00
Rick Macklem
3c53f923dc The PR reported that the old NFS server did not set uio_td == NULL
for the VOP_READ() call. This patch fixes both the old and new
server for this case.

PR:		185232
Submitted by:	PR had patch for old server
Reviewed by:	kib
MFC after:	2 weeks
2014-04-24 20:47:58 +00:00
Rick Macklem
ab7f24103e Remove an unnecessary level of indirection for an argument.
This simplifies the code and should avoid the clang sparc
port from generating an abort() call.

Requested by:	rdivacky
Submitted by:	jhb
MFC after:	2 weeks
2014-04-23 23:13:46 +00:00
Rick Macklem
c0990edac6 Modify the NFSv4 client's Pathconf RPC (actually a Getattr Op.)
so that it only does the RPC for names that are answered by the RPC.
Doing the RPC for other names is harmless, but unnecessary.

MFC after:	2 weeks
2014-04-23 22:13:10 +00:00
Rick Macklem
9eeef7464b Fixes mkdir for the NFSv2 client that was broken by r264705.
Reported by:	bdrewery
MFC after:	2 weeks
2014-04-22 04:42:46 +00:00
Rick Macklem
c7b560b9b4 For an NFSv4 mount with the "nocto" option, don't get the
up to date file attributes upon close. This reduces the
Getattr RPC count by about 65% for software builds.

MFC after:	2 weeks
2014-04-21 19:10:23 +00:00
Rick Macklem
c3e4a7261c Modify the NFSv4 client create/mkdir RPC so that it acquires
post-create/mkdir directory attributes. This allows the RPC to
name cache the newly created directory and reduces the lookup RPC
count for applications creating a lot of directories.

MFC after:	2 weeks
2014-04-20 22:19:00 +00:00
Rick Macklem
de1a42bd0c Modify the NFSv4 client open/create RPC so that it acquires
post-open/create directory attributes. This allows the RPC to
name cache the newly created file and reduces the lookup RPC
count by about 10% for software builds.

MFC after:	2 weeks
2014-04-19 19:40:20 +00:00
Rick Macklem
a6f8e64e74 Modify the Lookup RPC for NFSv4 so that it acquires directory
attributes. This allows the client to cache directory names
when they are looked up, reducing the Lookup RPC count by
about 40% for software builds.

MFC after:	2 weeks
2014-04-18 22:05:34 +00:00
Warner Losh
1bbf66051b Take out the hack to write -1's to non-NAND. Always do a BIO_DELETE on
the ranges we want to erase. This is nicer to SSDs that want TRIMs
anyway.
2014-04-18 17:03:43 +00:00
Warner Losh
875ac64f3e More properly account for free/reserved segments to avoid deadlock or
worse when filling up a device and then trying to erase files to make
space. Without enough space, you can't do that. Also, ensure that the
metadata writes don't generate ENOSPC. They will be retried later
since the buffers are still dirty...

Submitted by: mjg@
2014-04-18 17:03:35 +00:00
Andrey V. Elsukov
14b2dc3952 Use SMB_QUERY_FS_SIZE_INFO request to populate statfs structure.
When server doesn't support this request, try to use SMB_INFO_ALLOCATION.
And use SMB_COM_QUERY_INFORMATION_DISK request as fallback.

MFC after:	2 weeks
2014-04-15 09:10:01 +00:00
Xin LI
25bfde79d6 Fix NFS deadlock vulnerability. [SA-14:05]
Fix "Heartbleed" vulnerability and ECDSA Cache Side-channel
Attack in OpenSSL. [SA-14:06]
2014-04-08 18:27:32 +00:00
Bryan Drewery
44f1c91610 Rename global cnt to vm_cnt to avoid shadowing.
To reduce the diff struct pcu.cnt field was not renamed, so
PCPU_OP(cnt.field) is still used. pc_cnt and pcpu are also used in
kvm(3) and vmstat(8). The goal was to not affect externally used KPI.

Bump __FreeBSD_version_ in case some out-of-tree module/code relies on the
the global cnt variable.

Exp-run revealed no ports using it directly.

No objection from:	arch@
Sponsored by:	EMC / Isilon Storage Division
2014-03-22 10:26:09 +00:00
Pedro F. Giffuni
ca73017a2d Revert r263449;
ext2fs: minor update to the dirpref policy.

The change in UFS r254996, reverted the change as the
older code seems to work better. This was not visible
in local testing but we can trust UFS is vastly more
exercised in diferent environments.
2014-03-21 04:33:38 +00:00
Pedro F. Giffuni
e23c349230 ext2fs: minor update to the dirpref policy.
Bring in a minor change to the dirpref policy based on r248623.

This is pretty minimal change to keep the implementation in
sync with UFS but other parts from the original change are not
directly applicable so don't expect improvements in fsck times.

MFC after:	2 weeks
2014-03-20 21:19:13 +00:00
Pedro F. Giffuni
ef78ad0290 msdosfs: minor format fix - spaces vs tab
MFC after:	3 days
2014-03-20 20:14:04 +00:00
Robert Watson
4a14441044 Update kernel inclusions of capability.h to use capsicum.h instead; some
further refinement is required as some device drivers intended to be
portable over FreeBSD versions rely on __FreeBSD_version to decide whether
to include capability.h.

MFC after:	3 weeks
2014-03-16 10:55:57 +00:00
Bryan Drewery
504bde017a Add missing FALLTHROUGH comment in tmpfs_dir_getdents for looking up '.' and
'..'.

Reviewed by:	Russell Cattelan
Sponsored by:	EMC / Isilon Storage Division
MFC after:	2 weeks
2014-03-14 13:58:02 +00:00
Bryan Drewery
ac09d109ca Rename cnt to maxcookies and change its use as the condition for when to
lookup cookies to be less obscure.

No functional change.

Since r245115, cnt has not really been needed in tmpfs_dir_getdents().  Keep
it for the MPASS() for now though.

Sponsored by:	EMC / Isilon Storage Division
MFC after:	2 weeks
2014-03-14 13:55:48 +00:00
Bryan Drewery
62dca316da Cleanup redundant logic and add some comments to help explain how
it works in lieu of potentially less clear code.

Sponsored by:	EMC / Isilon Storage Division
Discussed with:	Russell Cattelan
2014-03-14 02:10:30 +00:00
Bryan Drewery
0742ebc98f Fix -o size less than PAGE_SIZE resulting in SIZE_MAX being used.
Discussed with:	kib
MFC after:	2 weeks
2014-03-14 01:43:55 +00:00
Pedro F. Giffuni
157b40af8e ext2fs: Fix a bug when sorting htree entries.
This a typo introduced when bringing the original code from NetBSD.

Reported by:	Mike Ma
MFC after:	3 days
2014-03-06 21:02:16 +00:00
Pedro F. Giffuni
c3b76e1345 ext2fs: small formatting fixes.
Remove some redundant spaces.
No functional change.

MFC after:	3 days
2014-03-01 21:22:20 +00:00
Pedro F. Giffuni
3a54024da4 ext2fs: use of tab vs spaces.
Consistently use a single tab after a #define as mentioned in style(9).
Use tabs instead of space for indenting.
Fix a typo: "hash_vesion".

No functional change.

MFC after:	3 days
2014-02-28 21:25:32 +00:00
Pedro F. Giffuni
67da48a15b ext2fs: fully enable ext4 read-only support.
The ext4 developers tend to tag Ext4-specific flags as
"incompatible" even when such features are not relevant for
read-only support.  This is a consequence of the process
though which this filesystem is implemented without design
and the fact that some new features are not extensible to
ext2/3.

Organize the features according to what we support and sort
them so that we can now read-only mount filesystems with
some features that may be found in newly formatted ext4 fs.

Submitted by:	Zheng Liu
Reviewed by:	pfg
MFC after:	5 days
2014-02-22 22:07:16 +00:00
Dimitry Andric
2de7ba0758 In sys/fs/nandfs/nandfs_vfsops.c, #if 0 an unused static function.
MFC after:	3 days
2014-02-15 11:42:56 +00:00
Pedro F. Giffuni
ad3d96a730 ext2fs: Use i_flag instead of i_flags for Ext4 inode flags.
The ext4 inode flags do not have equivalents for chflags (1)
and hold information that is private to the implementation.
The i_flag field in the inode is a better place to hold the Ext4
inode flags as it saves us from masking flags while setting or
getting attributes.  It should also make things cleaner if we
implement write support for Ext4.

Suggested by:	bde
Tested by:	Mike Ma
MFC after:	3 days
2014-01-28 14:39:05 +00:00
Pedro F. Giffuni
99984d229c ext2fs: Re-enable reallocblk.
The major corruption issues affecting this code have been fixed
a while ago.

MFC after:	1 week
2014-01-24 20:26:00 +00:00
Pedro F. Giffuni
1093104cf7 ext2fs: fix a bug in dirindex and re-enable.
The IN_* flags should be set in i_flag instead of corrupting
i_flags [1].

Re-enable HTree dirindex as the last series of bug fixes
seems to have fixed the issues.

Reported by:	bde [1]
Tested by:	kevlo
MFC after:	1 week
2014-01-24 13:51:38 +00:00
Pedro F. Giffuni
b7bbf8b9f3 ext2fs: fix logic error in the previous change.
Use the bitwise negation instead of bogus boolean negation and move
the flag manipulation with the assignment.
Fix some grammatical errors introduced in the same change.

Reported by:	bde
MFC after:	3 days
2014-01-22 19:09:41 +00:00
Pedro F. Giffuni
a7710d51c4 ext2fs: Translate the EXT4_EXTENTS and EXT4_INDEX to the inode flags.
r260545 cleared the inode flags to fix corruption problems but
we still need to pass some EXT4 flags for the ext4 read-only
mode.  None of these attributes has an equivalent in FreeBSD and
are uninteresting for the system utilities so they should be
innaccessible in ext2_getattrib().

Note: we also use EXT4_HUGE_FILE but we use it directly from the
dinode structure so it is not necessary to translate it,

Suggested by:	bde
MFC after:	3 days
2014-01-21 19:06:29 +00:00
Alexander Motin
6103bae6ae Fix lock leak in purely hypothetical case of TCP connection without SVC_ACK
method.  This change should be NOP now, but it is better to be future safe.

Reported by:	rmacklem
2014-01-14 20:18:38 +00:00
Pedro F. Giffuni
c2e2b77b19 ext2fs: fix inode flag conversion.
After r252890 we are naively attempting to pass through the
inode flags.  This is technically incorrect as the ext2
inode flags don't match the UFS/system values used in
FreeBSD and a clean conversion is needed.

Some filtering was left in place so the change didn't cause
significant changes in FreeBSD but some of the garbage passed
is likely to be the cause for warning messages in linux.

Fix the issue by resetting the flags before conversion as was
done previously. This also means we will not pass the EXT4_*
inode flags into FreeBSD's inode.

PR:		kern/185448
MFC after:	3 days
2014-01-11 15:19:04 +00:00
Alexander Motin
45e18ea7ea Fix off-by-one error in r260229.
Coverity CID:	1148955
2014-01-07 11:43:51 +00:00
Alexander Motin
d473bac729 Rework NFS Duplicate Request Cache cleanup logic.
- Introduce additional hash to group requests by hash of sockref.  This
allows to process TCP acknowledgements without looping though all the cache,
and as result allows to do it every time.
 - Indroduce additional callbacks to notify application layer about sockets
disconnection.  Without this last few requests processed just before socket
disconnection never processed their ACKs and stuck in cache for many hours.
 - Implement transport-specific method for tracking reply acknowledgements.
New implementation does not cross multiple stack layers to get the data and
does not have race conditions that previously made some requests stuck
in cache.  This could be done more efficiently at sockbuf layer, but that
would broke some KBIs, while I don't know other consumers for it aside NFS.
 - Instead of traversing all DRC twice per request, run cleaning only once
per request, and except in some conditions traverse only single hash slot
at a time.

Together this limits NFS DRC growth only to situations of real connectivity
problems.  If network is working well, and so all replies are acknowledged,
cache remains almost empty even after hours of heavy load.  Without this
change on the same test cache was growing to many thousand requests even
with perfectly working local network.

As another result this reduces CPU time spent on the DRC handling during
SPEC NFS benchmark from about 10% to 0.5%.

Sponsored by:	iXsystems, Inc.
2014-01-03 15:09:59 +00:00
Alexander Motin
1555cf04fc Slightly simplify expiration logic introduced in r254337.
- Do not update the histogram for items we are any way deleting from cache.
 - Do not update the histogram if nfsrc_tcphighwater is not set.
 - Remove some extra math operations.
2013-12-25 16:58:42 +00:00
Rick Macklem
43a213bb92 The NFSv4 server would call VOP_SETATTR() with a shared locked vnode
when a Getattr for a file is done by a client other than the one that
holds the file's delegation. This would only happen when delegations
are enabled and the problem is fixed by this patch.

MFC after:	1 week
2013-12-25 01:03:14 +00:00
Rick Macklem
0c695afb96 An intermittent problem with NFSv4 exporting of ZFS snapshots was
reported to the freebsd-fs mailing list. I believe the problem was
caused by the Readdir operation using VFS_VGET() for a snapshot file entry
instead of VOP_LOOKUP(). This would not occur for NFSv3, since it
will do a VFS_VGET() of "." which fails with ENOTSUPP at the beginning
of the directory, whereas NFSv4 does not check "." or "..". This
patch adds a call to VFS_VGET() for the directory being read to check
for ENOTSUPP.
I also observed that the mount_on_fileid and fsid attributes were
not correct at the snapshot's auto mountpoints when looking at packet
traces for the Readdir. This patch fixes the attributes by doing a check
for different v_mount structure, even if the vnode v_mountedhere is not
set.

Reported by:	jas@cse.yorku.ca
Tested by:	jas@cse.yorku.ca
Reviewed by:	asomers
MFC after:	1 week
2013-12-24 22:24:17 +00:00
Rick Macklem
b921158ae0 The NFSv4 client was passing both the p and cred arguments to
nfsv4_fillattr() as NULLs for the Getattr callback. This caused
nfsv4_fillattr() to not fill in the Change attribute for the reply.
I believe this was a violation of the RFC, but had little effect on
server behaviour. This patch passes a non-NULL p argument to fix this.

MFC after:	1 week
2013-12-24 00:48:39 +00:00
Pedro F. Giffuni
b41f53c43b ext2fs: make the hashing algorithm match the linux code.
There appears to be a hash function compatibility issue.
The code is currently disabled but fix it nevertheless.

PR:		kern/183230
MFC after:	3 days
2013-12-23 19:47:34 +00:00
Rick Macklem
6b8fe5d59d The NFSv4.1 client didn't return NFSv4.1 specific error codes
for the Getattr and Recall callbacks. This patch fixes it.
Since the NFSv4.1 specific error codes would only happen for
abnormal circumstances, this patch has little effect, in practice.

MFC after:	1 week
2013-12-23 15:16:53 +00:00
Alexander Motin
10f8f58d4a Fix RPC server threads file handle affinity to work better with ZFS.
Instead of taking 8 specific bytes of file handle to identify file during
RPC thread affitinity handling, use trivial hash of the full file handle.
ZFS's struct zfid_short does not have padding field after the length field,
as result, originally picked 8 bytes are loosing lower 16 bits of object ID,
causing many false matches and unneeded requests affinity to same thread.
  This fix substantially improves NFS server latency and scalability in SPEC
NFS benchmark by more flexible use of multiple NFS threads.

Sponsored by:	iXsystems, Inc.
2013-12-23 08:43:16 +00:00
Konstantin Belousov
f26ca5ecde Do not allow O_EXEC opens for fifo, return EINVAL.
Besides not making sense, open(O_EXEC) for fifo creates fifoinfo with
zero readers and writers counts, which causes premature free of pipes.

Reported and tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2013-12-17 17:28:02 +00:00
Alexander Motin
ca187878c0 Fix long known bug with handling device aliases residing not in devfs root.
Historically creation of device aliases created symbolic links using only
name of target device as a link target, not considering current directory.
Fix that by adding number of "../" chunks to the terget device name,
required to get out of the current directory to devfs root first.

MFC after:	1 month
2013-12-12 11:05:48 +00:00
Rick Macklem
cf766161ff For software builds, the NFS client does many small
synchronous (with FILE_SYNC) writes because non-contiguous
byte ranges in the same buffer cache block are being
written. This patch adds a new mount option "noncontigwr"
which allows the non-contiguous byte ranges to be combined,
with the dirty byte range becoming the superset of the bytes
that are dirty, if the file has not been file locked.
This reduces the number of writes significantly for software
builds. The only case where this change might break existing
applications is where an application is writing
non-overlapping byte ranges within the same buffer cache block
of a file from multiple clients concurrently.
Since such an application would normally do file locking on
the file, avoiding the byte range merge for files that have
been file locked should be sufficient for most (maybe all?) cases.

Submitted by:	jhb (earlier version)
Reviewed by:	kib
MFC after:	3 weeks
2013-12-07 23:05:59 +00:00
Pedro F. Giffuni
244f00cc0d ext2fs: add two new reserved inodes.
According to online documentation [1], Ext4 has two new "special"
inodes so add the new exclude and replica inodes.

Reference:
[1] https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout

Reported by:	Mike Ma
MFC after:	3 weeks
2013-12-04 02:27:52 +00:00
Sergey Kandaurov
0d8dc7cc39 - Nuke a second copy of nfscl_attrcache extern declarations from under
ifdef KDTRACE_HOOKS.  This fixes kernel build with options KDTRACE_HOOKS.
- Fix style inconsistencies.
2013-11-26 22:41:40 +00:00
Gleb Smirnoff
285e7a2d97 Fix build, attempt two. 2013-11-26 20:27:57 +00:00
Gleb Smirnoff
6882b8ea66 Fix build. 2013-11-26 10:34:34 +00:00
Attilio Rao
54366c0bd7 - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging
option, unbreak the lock tracing release semantic by embedding
  calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined
  version of the releasing functions for mutex, rwlock and sxlock.
  Failing to do so skips the lockstat_probe_func invokation for
  unlocking.
- As part of the LOCKSTAT support is inlined in mutex operation, for
  kernel compiled without lock debugging options, potentially every
  consumer must be compiled including opt_kdtrace.h.
  Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the
  dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES
  is linked there and it is only used as a compile-time stub [0].

[0] immediately shows some new bug as DTRACE-derived support for debug
in sfxge is broken and it was never really tested.  As it was not
including correctly opt_kdtrace.h before it was never enabled so it
was kept broken for a while.  Fix this by using a protection stub,
leaving sfxge driver authors the responsibility for fixing it
appropriately [1].

Sponsored by:	EMC / Isilon storage division
Discussed with:	rstone
[0] Reported by:	rstone
[1] Discussed with:	philip
2013-11-25 07:38:45 +00:00
Konstantin Belousov
587430f254 Redo r258088 to avoid relying on signed arithmetic overflow, since
compiler interprets this as an undefined behaviour.  Instead, ensure
that the sum of uio_offset and uio_resid is below OFF_MAX using the
operation which cannot overflow.

Reported and tested by:	pho
Discussed with:	bde
Approved by:	des (pseudofs maintainer)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2013-11-20 19:41:00 +00:00
Konstantin Belousov
5ba4de79a7 Remove useless comparisions of assigned offset and resid with the
sources from uio.  Both uio_offset and offset, and uio_resid and resid
have the same types for some time.

Add check for buflen overflow by comparing the buflen with both offset
and resid (vs. comparing with offset only, as it is currently done).

Reported and tested by:	pho
Approved by:	des (pseudofs maintainer)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2013-11-13 08:55:09 +00:00
Rick Macklem
42b6336a98 Fix an NFSv4.1 client specific case where a forced dismount would hang.
The hang occurred in nfsv4_setsequence() when it couldn't find an
available session slot and is fixed by checking for a forced dismount
in progress and just returning for this case.

MFC after:	1 month
2013-11-09 21:24:56 +00:00
Rick Macklem
cc085ba84d During code inspection, I spotted that there was a code path where
CLNT_CONTROL() would be called on "client" after it was
released via CLNT_RELEASE(). It was unlikely that this
code path gets executed and I have not heard of any problem
report caused by this bug. This patch fixes the code so that
this cannot happen.

MFC after:	2 months
2013-11-03 23:17:30 +00:00
Gleb Smirnoff
76039bc84f The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2013-10-26 17:58:36 +00:00
Pedro F. Giffuni
4b367145f7 UFS2: make di_extsize unsigned.
di_extsize is the EA size and as such it should be unsigned.
Adjust related types for consistency.

Reviewed by:	mckusick (previous version)
MFC after:	3 weeks
2013-10-24 00:33:29 +00:00
Konstantin Belousov
bf3e483b44 Similar to debug.iosize_max_clamp sysctl, introduce
devfs_iosize_max_clamp sysctl, which allows/disables SSIZE_MAX-sized
i/o requests on the devfs files.

Sponsored by:	The FreeBSD Foundation
Reminded by:	Dmitry Sivachenko <trtrmitya@gmail.com>
MFC after:	1 week
2013-10-15 06:33:10 +00:00
Konstantin Belousov
64548150b6 Remove two instances of ARGSUSED comment, and wrap lines nearby the
code that is to be changed.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2013-10-15 06:28:11 +00:00
John-Mark Gurney
c9b24e38e8 NULL stale pointers (should be a no-op as they should no longer be
used)...

Reviewed by:	dteske
Approved by:	re (kib)
Sponsored by:	Vicor
MFC after:	3 days
2013-09-25 02:49:18 +00:00
John-Mark Gurney
fb180e2186 fix a bug where we access a bread buffer after we have brelse'd it...
The kernel normally didn't unmap/context switch away before we accessed
the buffer most of the time, but under heavy I/O pressure and lots of
mount/unmounting this would cause a fault on nofault panic...

Reviewed by:	dteske
Approved by:	re (kib)
Sponsored by:	Vicor
MFC after:	3 days
2013-09-25 02:48:12 +00:00
Dag-Erling Smørgrav
1a05c762b9 Fix the length calculation for the final block of a sendfile(2)
transmission which could be tricked into rounding up to the nearest
page size, leaking up to a page of kernel memory.  [13:11]

In IPv6 and NetATM, stop SIOCSIFADDR, SIOCSIFBRDADDR, SIOCSIFDSTADDR
and SIOCSIFNETMASK at the socket layer rather than pass them on to the
link layer without validation or credential checks.  [SA-13:12]

Prevent cross-mount hardlinks between different nullfs mounts of the
same underlying filesystem.  [SA-13:13]

Security:	CVE-2013-5666
Security:	FreeBSD-SA-13:11.sendfile
Security:	CVE-2013-5691
Security:	FreeBSD-SA-13:12.ifioctl
Security:	CVE-2013-5710
Security:	FreeBSD-SA-13:13.nullfs
Approved by:	re
2013-09-10 10:05:59 +00:00
Pedro F. Giffuni
1f7c9f2bc8 ext2fs: temporarily disable htree directory index.
Our code does not consider yet the case of hash collisions. This
is a rather annoying situation where two or more files that
happen to have the same hash value will not appear accessible.

The situation is not difficult to work-around but given that things
will just work without enabling htree we will save possible
embarrassments for the next release.

Reported by:	Kevin Lo
2013-09-07 02:45:51 +00:00
Pawel Jakub Dawidek
ab568de789 Handle cases where capability rights are not provided.
Reported by:	kib
2013-09-05 11:58:12 +00:00
Pawel Jakub Dawidek
7008be5bd7 Change the cap_rights_t type from uint64_t to a structure that we can extend
in the future in a backward compatible (API and ABI) way.

The cap_rights_t represents capability rights. We used to use one bit to
represent one right, but we are running out of spare bits. Currently the new
structure provides place for 114 rights (so 50 more than the previous
cap_rights_t), but it is possible to grow the structure to hold at least 285
rights, although we can make it even larger if 285 rights won't be enough.

The structure definition looks like this:

	struct cap_rights {
		uint64_t	cr_rights[CAP_RIGHTS_VERSION + 2];
	};

The initial CAP_RIGHTS_VERSION is 0.

The top two bits in the first element of the cr_rights[] array contain total
number of elements in the array - 2. This means if those two bits are equal to
0, we have 2 array elements.

The top two bits in all remaining array elements should be 0.
The next five bits in all array elements contain array index. Only one bit is
used and bit position in this five-bits range defines array index. This means
there can be at most five array elements in the future.

To define new right the CAPRIGHT() macro must be used. The macro takes two
arguments - an array index and a bit to set, eg.

	#define	CAP_PDKILL	CAPRIGHT(1, 0x0000000000000800ULL)

We still support aliases that combine few rights, but the rights have to belong
to the same array element, eg:

	#define	CAP_LOOKUP	CAPRIGHT(0, 0x0000000000000400ULL)
	#define	CAP_FCHMOD	CAPRIGHT(0, 0x0000000000002000ULL)

	#define	CAP_FCHMODAT	(CAP_FCHMOD | CAP_LOOKUP)

There is new API to manage the new cap_rights_t structure:

	cap_rights_t *cap_rights_init(cap_rights_t *rights, ...);
	void cap_rights_set(cap_rights_t *rights, ...);
	void cap_rights_clear(cap_rights_t *rights, ...);
	bool cap_rights_is_set(const cap_rights_t *rights, ...);

	bool cap_rights_is_valid(const cap_rights_t *rights);
	void cap_rights_merge(cap_rights_t *dst, const cap_rights_t *src);
	void cap_rights_remove(cap_rights_t *dst, const cap_rights_t *src);
	bool cap_rights_contains(const cap_rights_t *big, const cap_rights_t *little);

Capability rights to the cap_rights_init(), cap_rights_set(),
cap_rights_clear() and cap_rights_is_set() functions are provided by
separating them with commas, eg:

	cap_rights_t rights;

	cap_rights_init(&rights, CAP_READ, CAP_WRITE, CAP_FSTAT);

There is no need to terminate the list of rights, as those functions are
actually macros that take care of the termination, eg:

	#define	cap_rights_set(rights, ...)				\
		__cap_rights_set((rights), __VA_ARGS__, 0ULL)
	void __cap_rights_set(cap_rights_t *rights, ...);

Thanks to using one bit as an array index we can assert in those functions that
there are no two rights belonging to different array elements provided
together. For example this is illegal and will be detected, because CAP_LOOKUP
belongs to element 0 and CAP_PDKILL to element 1:

	cap_rights_init(&rights, CAP_LOOKUP | CAP_PDKILL);

Providing several rights that belongs to the same array's element this way is
correct, but is not advised. It should only be used for aliases definition.

This commit also breaks compatibility with some existing Capsicum system calls,
but I see no other way to do that. This should be fine as Capsicum is still
experimental and this change is not going to 9.x.

Sponsored by:	The FreeBSD Foundation
2013-09-05 00:09:56 +00:00
Rick Macklem
f7d8291af0 Crashes have been observed for NFSv4.1 mounts when the system
is being shut down which were caused by the nfscbd_pool being
destroyed before the backchannel is disabled. This patch is
believed to fix the problem, by simply avoiding ever destroying
the nfscbd_pool. Since the NFS client module cannot be unloaded,
this should not cause a memory leak.

MFC after:	2 weeks
2013-09-04 22:47:56 +00:00
Rick Macklem
8fe6bddff7 Forced dismounts of NFS mounts can fail when thread(s) are stuck
waiting for an RPC reply from the server while holding the mount
point busy (mnt_lockref incremented). This happens because dounmount()
msleep()s waiting for mnt_lockref to become 0, before calling
VFS_UNMOUNT(). This patch adds a new VFS operation called VFS_PURGE(),
which the NFS client implements as purging RPCs in progress. Making
this call before checking mnt_lockref fixes the problem, by ensuring
that the VOP_xxx() calls will fail and unbusy the mount point.

Reported by:	sbruno
Reviewed by:	kib
MFC after:	2 weeks
2013-09-01 23:02:59 +00:00
Kenneth D. Merry
3b5f179d2a Support storing 7 additional file flags in tmpfs:
UF_SYSTEM, UF_SPARSE, UF_OFFLINE, UF_REPARSE, UF_ARCHIVE, UF_READONLY,
and UF_HIDDEN.

Sort the file flags tmpfs supports alphabetically.  tmpfs now
supports the same flags as UFS, with the exception of SF_SNAPSHOT.

Reported by:	bdrewery, antoine
Sponsored by:	Spectra Logic
2013-08-28 22:12:56 +00:00
John Baldwin
fd77bbb967 Remove most of the remaining sysctl name list macros. They were only
ever intended for use in sysctl(8) and it has not used them for many
years.

Reviewed by:	bde
Tested by:	exp-run by bdrewery
2013-08-26 18:16:05 +00:00
Xin LI
2454886e05 Allow tmpfs be mounted inside jail. 2013-08-23 22:52:20 +00:00
Kenneth D. Merry
7da1a731c6 Expand the use of stat(2) flags to allow storing some Windows/DOS
and CIFS file attributes as BSD stat(2) flags.

This work is intended to be compatible with ZFS, the Solaris CIFS
server's interaction with ZFS, somewhat compatible with MacOS X,
and of course compatible with Windows.

The Windows attributes that are implemented were chosen based on
the attributes that ZFS already supports.

The summary of the flags is as follows:

UF_SYSTEM:	Command line name: "system" or "usystem"
		ZFS name: XAT_SYSTEM, ZFS_SYSTEM
		Windows: FILE_ATTRIBUTE_SYSTEM

		This flag means that the file is used by the
		operating system.  FreeBSD does not enforce any
		special handling when this flag is set.

UF_SPARSE:	Command line name: "sparse" or "usparse"
		ZFS name: XAT_SPARSE, ZFS_SPARSE
		Windows: FILE_ATTRIBUTE_SPARSE_FILE

		This flag means that the file is sparse.  Although
		ZFS may modify this in some situations, there is
		not generally any special handling for this flag.

UF_OFFLINE:	Command line name: "offline" or "uoffline"
		ZFS name: XAT_OFFLINE, ZFS_OFFLINE
		Windows: FILE_ATTRIBUTE_OFFLINE

		This flag means that the file has been moved to
		offline storage.  FreeBSD does not have any special
		handling for this flag.

UF_REPARSE:	Command line name: "reparse" or "ureparse"
		ZFS name: XAT_REPARSE, ZFS_REPARSE
		Windows: FILE_ATTRIBUTE_REPARSE_POINT

		This flag means that the file is a Windows reparse
		point.  ZFS has special handling code for reparse
		points, but we don't currently have the other
		supporting infrastructure for them.

UF_HIDDEN:	Command line name: "hidden" or "uhidden"
		ZFS name: XAT_HIDDEN, ZFS_HIDDEN
		Windows: FILE_ATTRIBUTE_HIDDEN

		This flag means that the file may be excluded from
		a directory listing if the application honors it.
		FreeBSD has no special handling for this flag.

		The name and bit definition for UF_HIDDEN are
		identical to the definition in MacOS X.

UF_READONLY:	Command line name: "urdonly", "rdonly", "readonly"
		ZFS name: XAT_READONLY, ZFS_READONLY
		Windows: FILE_ATTRIBUTE_READONLY

		This flag means that the file may not written or
		appended, but its attributes may be changed.

		ZFS currently enforces this flag, but Illumos
		developers have discussed disabling enforcement.

		The behavior of this flag is different than MacOS X.
		MacOS X uses UF_IMMUTABLE to represent the DOS
		readonly permission, but that flag has a stronger
		meaning than the semantics of DOS readonly permissions.

UF_ARCHIVE:	Command line name: "uarch", "uarchive"
		ZFS_NAME: XAT_ARCHIVE, ZFS_ARCHIVE
		Windows name: FILE_ATTRIBUTE_ARCHIVE

		The UF_ARCHIVED flag means that the file has changed and
		needs to be archived.  The meaning is same as
		the Windows FILE_ATTRIBUTE_ARCHIVE attribute, and
		the ZFS XAT_ARCHIVE and ZFS_ARCHIVE attribute.

		msdosfs and ZFS have special handling for this flag.
		i.e. they will set it when the file changes.

sys/param.h:		Bump __FreeBSD_version to 1000047 for the
			addition of new stat(2) flags.

chflags.1:		Document the new command line flag names
			(e.g. "system", "hidden") available to the
			user.

ls.1:			Reference chflags(1) for a list of file flags
			and their meanings.

strtofflags.c:		Implement the mapping between the new
			command line flag names and new stat(2)
			flags.

chflags.2:		Document all of the new stat(2) flags, and
			explain the intended behavior in a little
			more detail.  Explain how they map to
			Windows file attributes.

			Different filesystems behave differently
			with respect to flags, so warn the
			application developer to take care when
			using them.

zfs_vnops.c:		Add support for getting and setting the
			UF_ARCHIVE, UF_READONLY, UF_SYSTEM, UF_HIDDEN,
			UF_REPARSE, UF_OFFLINE, and UF_SPARSE flags.

			All of these flags are implemented using
			attributes that ZFS already supports, so
			the on-disk format has not changed.

			ZFS currently doesn't allow setting the
			UF_REPARSE flag, and we don't really have
			the other infrastructure to support reparse
			points.

msdosfs_denode.c,
msdosfs_vnops.c:	Add support for getting and setting
			UF_HIDDEN, UF_SYSTEM and UF_READONLY
			in MSDOSFS.

			It supported SF_ARCHIVED, but this has been
			changed to be UF_ARCHIVE, which has the same
			semantics as the DOS archive attribute instead
			of inverse semantics like SF_ARCHIVED.

			After discussion with Bruce Evans, change
			several things in the msdosfs behavior:

			Use UF_READONLY to indicate whether a file
			is writeable instead of file permissions, but
			don't actually enforce it.

			Refuse to change attributes on the root
			directory, because it is special in FAT
			filesystems, but allow most other attribute
			changes on directories.

			Don't set the archive attribute on a directory
			when its modification time is updated.
			Windows and DOS don't set the archive attribute
			in that scenario, so we are now bug-for-bug
			compatible.

smbfs_node.c,
smbfs_vnops.c:		Add support for UF_HIDDEN, UF_SYSTEM,
			UF_READONLY and UF_ARCHIVE in SMBFS.

			This is similar to changes that Apple has
			made in their version of SMBFS (as of
			smb-583.8, posted on opensource.apple.com),
			but not quite the same.

			We map SMB_FA_READONLY to UF_READONLY,
			because UF_READONLY is intended to match
			the semantics of the DOS readonly flag.
			The MacOS X code maps both UF_IMMUTABLE
			and SF_IMMUTABLE to SMB_FA_READONLY, but
			the immutable flags have stronger meaning
			than the DOS readonly bit.

stat.h:			Add definitions for UF_SYSTEM, UF_SPARSE,
			UF_OFFLINE, UF_REPARSE, UF_ARCHIVE, UF_READONLY
			and UF_HIDDEN.

			The definition of UF_HIDDEN is the same as
			the MacOS X definition.

			Add commented-out definitions of
			UF_COMPRESSED and UF_TRACKED.  They are
			defined in MacOS X (as of 10.8.2), but we
			do not implement them (yet).

ufs_vnops.c:		Add support for getting and setting
			UF_ARCHIVE, UF_HIDDEN, UF_OFFLINE, UF_READONLY,
			UF_REPARSE, UF_SPARSE, and UF_SYSTEM in UFS.
			Alphabetize the flags that are supported.

			These new flags are only stored, UFS does
			not take any action if the flag is set.

Sponsored by:	Spectra Logic
Reviewed by:	bde (earlier version)
2013-08-21 23:04:48 +00:00
Konstantin Belousov
c0a46535c4 Make the seek a method of the struct fileops.
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2013-08-21 17:36:01 +00:00
Konstantin Belousov
41cf41fdfd Extract the general-purpose code from tmpfs to perform uiomove from
the page queue of some vm object.

Discussed with:	alc
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2013-08-21 17:23:24 +00:00
Konstantin Belousov
b1dd38f408 Restore the previous sendfile(2) behaviour on the block devices.
Provide valid .fo_sendfile method for several missed struct fileops.

Reviewed by:	glebius
Sponsored by:	The FreeBSD Foundation
2013-08-16 14:22:20 +00:00
Rick Macklem
93c5875b24 Fix several performance related issues in the new NFS server's
DRC for NFS over TCP.
- Increase the size of the hash tables.
- Create a separate mutex for each hash list of the TCP hash table.
- Single thread the code that deletes stale cache entries.
- Add a tunable called vfs.nfsd.tcphighwater, which can be increased
  to allow the cache to grow larger, avoiding the overhead of frequent
  scans to delete stale cache entries.
  (The default value will result in frequent scans to delete stale cache
   entries, analagous to what the pre-patched code does.)
- Add a tunable called vfs.nfsd.cachetcp that can be used to disable
  DRC caching for NFS over TCP, since the old NFS server didn't DRC cache TCP.
It also adjusts the size of nfsrc_floodlevel dynamically, so that it is
always greater than vfs.nfsd.tcphighwater.

For UDP the algorithm remains the same as the pre-patched code, but the
tunable vfs.nfsd.udphighwater can be used to allow the cache to grow
larger and reduce the overhead caused by frequent scans for stale entries.
UDP also uses a larger hash table size than the pre-patched code.

Reported by:	wollman
Tested by:	wollman (earlier version of patch)
Submitted by:	ivoras (earlier patch)
Reviewed by:	jhb (earlier version of patch)
MFC after:	1 month
2013-08-14 21:11:26 +00:00
Pedro F. Giffuni
4a62545173 ext2fs: update format specifiers for ext4 type.
Previous bandaid was not appropriate and didn't really work for
all platforms. While here, cleanup the surrounding code to match
ffs_checkoverlap()

Reported by:	dim, jmallet and bde
MFC after:	3 weeks
2013-08-14 14:22:46 +00:00
Pedro F. Giffuni
88ae190ea0 ext2fs: update format specifiers for ext4 type.
Reported by:	Sam Fourman Jr.
MFC after:	3 weeks
2013-08-13 18:39:36 +00:00
Pedro F. Giffuni
70097aac13 Define ext2fs local types and use them.
Add definitions for e2fs_daddr_t, e4fs_daddr_t in addition
to the already existing e2fs_lbn_t and adjust them for ext4.
Other than making the code more readable these changes should
fix problems related to big filesystems.

Setting the proper types can be tricky so the process was
helped by looking at UFS. In our implementation, logical block
numbers can be negative and the code depends on it. In ext2,
block numbers are unsigned so it is convenient to keep
e2fs_daddr_t unsigned and use the complete 32 bits. In the
case of e4fs_daddr_t, while the value should be unsigned, for
ext4 we only need to support 48 bits so preserving an extra
bit from the sign is not an issue.

While here also drop the ext2_setblock() prototype that was
never used.

Discussed with:	mckusick, bde
MFC after:	3 weeks
2013-08-13 15:40:43 +00:00
Pedro F. Giffuni
d7511a40a7 Add read-only support for extents in ext2fs.
Basic support for extents was implemented by Zheng Liu as part
of his Google Summer of Code in 2010. This support is read-only
at this time.

In addition to extents we also support the huge_file extension
for read-only purposes. This works nicely with the additional
support for birthtime/nanosec timestamps and dir_index that
have been added lately.

The implementation may not work for all ext4 filesystems as
it doesn't support some features that are being enabled by
default on recent linux like flex_bg. Nevertheless, the feature
should be very useful for migration or simple access in
filesystems that have been converted from ext2/3 or don't use
incompatible features.

Special thanks to Zheng Liu for his dedication and continued
work to support ext2 in FreeBSD.

Submitted by:	Zheng Liu (lz@)
Reviewed by:	Mike Ma, Christoph Mallon (previous version)
Sponsored by:	Google Inc.
MFC after:	3 weeks
2013-08-12 21:34:48 +00:00
Attilio Rao
c7aebda8a1 The soft and hard busy mechanism rely on the vm object lock to work.
Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.

Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
  and vm_page_grab are being executed.  This will be very helpful
  once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag

The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.

Sponsored by:	EMC / Isilon storage division
Discussed with:	alc
Reviewed by:	jeff, kib
Tested by:	gavin, bapt (older version)
Tested by:	pho, scottl
2013-08-09 11:11:11 +00:00
Pedro F. Giffuni
95f1f8d262 Small typo.
MFC after:	3 days
2013-08-08 22:07:59 +00:00
Konstantin Belousov
8239a7a878 The tmpfs_alloc_vp() is used to instantiate vnode for the tmpfs node,
in particular, from the tmpfs_lookup VOP method.  If LK_NOWAIT is not
specified in the lkflags, the lookup is supposed to return an alive
vnode whenever the underlying node is valid.

Currently, the tmpfs_alloc_vp() returns ENOENT if the vnode attached
to node exists and is being reclaimed.  This causes spurious ENOENT
errors from lookup on tmpfs and corresponding random 'No such file'
failures from syscalls working with tmpfs files.

Fix this by waiting for the doomed vnode to be detached from the tmpfs
node if sleepable allocation is requested.

Note that filesystems which use vfs_hash.c, correctly handle the case
due to vfs_hash_get() looping when vget() returns ENOENT for sleepable
requests.

Reported and tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2013-08-05 18:53:59 +00:00
Attilio Rao
be99683637 Revert r253939:
We cannot busy a page before doing pagefaults.
Infact, it can deadlock against vnode lock, as it tries to vget().
Other functions, right now, have an opposite lock ordering, like
vm_object_sync(), which acquires the vnode lock first and then
sleeps on the busy mechanism.

Before this patch is reinserted we need to break this ordering.

Sponsored by:	EMC / Isilon storage division
Reported by:	kib
2013-08-05 08:55:35 +00:00
Attilio Rao
3b6714cacb The page hold mechanism is fast but it has couple of fallouts:
- It does not let pages respect the LRU policy
- It bloats the active/inactive queues of few pages

Try to avoid it as much as possible with the long-term target to
completely remove it.
Use the soft-busy mechanism to protect page content accesses during
short-term operations (like uiomove_fromphys()).

After this change only vm_fault_quick_hold_pages() is still using the
hold mechanism for page content access.
There is an additional complexity there as the quick path cannot
immediately access the page object to busy the page and the slow path
cannot however busy more than one page a time (to avoid deadlocks).

Fixing such primitive can bring to complete removal of the page hold
mechanism.

Sponsored by:	EMC / Isilon storage division
Discussed with:	alc
Reviewed by:	jeff
Tested by:	pho
2013-08-04 21:07:24 +00:00
Attilio Rao
878a788734 Remove unnecessary soft busy of the page before to do vn_rdwr() in
kern_sendfile() which is unnecessary.
The page is already wired so it will not be subjected to pagefault.
The content cannot be effectively protected as it is full of races
already.
Multiple accesses to the same indexes are serialized through vn_rdwr().

Sponsored by:	EMC / Isilon storage division
Reviewed by:	alc, jeff
Tested by:	pho
2013-08-04 15:56:19 +00:00
Pedro F. Giffuni
d192e40f77 Add license for the half MD4 algorithm used in ext2_half_md4().
The htree implementation uses code derived from the
RSA Data Security, Inc. MD4 Message-Digest Algorithm.

Add a proper licensing statement for the code and clarify
the corresponding comments.

Approved by:	core (hrs)
2013-08-01 16:04:48 +00:00
Marius Strobl
cd67748bde - Add const-qualifiers to the arguments of isonum_*().
- According to ISO 9660 7.1.2, isonum_712() should return a signed value.
- Try to get isonum_*() closer to style(9).
2013-07-28 12:29:10 +00:00
Andriy Gapon
8e94193e58 make path matching in devfs rules consistent and sane (and safer)
Before this change path matching had the following features:
- for device nodes the patterns were matched against full path
- in the above case '/' in a path could be matched by a wildcard
- for directories and links only the last component was matched

So, for example, a pattern like 're*' could match the following entries:
- re0 device
- responder/u0 device
- zvol/recpool directory

Although it was possible to work around this behavior (once it was spotted
and understood), it was very confusing and contrary to documentation.

Now we always match a full path for all types of devfs entries (devices,
directories, links) and a '/' has to be matched explicitly.
This behavior follows the shell globbing rules.

This change is originally developed by Jaakko Heinonen.
Many thanks!

PR:		kern/122838
Submitted by:	jh
MFC after:	4 weeks
2013-07-26 14:25:58 +00:00
Pedro F. Giffuni
9670f48107 ext2fs: Return EINVAL for negative uio_offset as in UFS.
While here drop old comment that doesn't really apply.

MFC after:	1 month
Discussed with:	gleb
2013-07-25 19:37:49 +00:00
Pedro F. Giffuni
0b54fe540c ext2fs: Drop a check that wan't supposed to be in r253651.
MFC after:	1 month
2013-07-25 16:04:55 +00:00
Pedro F. Giffuni
78d912bbc3 ext2fs: Don't assume that on-disk format of a directory is the same
as in <sys/dirent.h>

ext2_readdir() has always been very fs specific and different
with respect to its ufs_ counterpart. Recent changes from UFS
have made it possible to share more closely the implementation.

MFUFS r252438:
Always start parsing at DIRBLKSIZ aligned offset, skip first entries if
uio_offset is not DIRBLKSIZ aligned. Return EINVAL if buffer is too
small for single entry.

Preallocate buffer for cookies.

Skip entries with zero inode number.

Reviewed by:	gleb, Zheng Liu
MFC after:	1 month
2013-07-25 15:34:20 +00:00
Pedro F. Giffuni
7d20a270cc fuse: revert kernel_header update.
It seems to be causing problems due to the lack of the new features.

Found by:	bapt
Pointed hat:	pfg
2013-07-24 20:21:29 +00:00
Nathan Whitehorn
59169d9156 tmpfs works perfectly fine with -o union -- there is no reason to exclude it
from the list of options.
2013-07-23 14:48:37 +00:00
Rick Macklem
a36b76a787 The NFSv4 server incorrectly assumed that the high order words of
the attribute bitmap argument would be non-zero. This caused an
interoperability problem for a recent patch to the Linux NFSv4 client.
The Linux folks have changed their patch to avoid this, but this
patch fixes the problem on the server.

Reported and tested by:	Andre Heider (a.heider@gmail.com)
MFC after:	3 days
2013-07-20 22:35:32 +00:00