Commit Graph

242410 Commits

Author SHA1 Message Date
Alan Somers
5993fa5582 open(2): fix the description of O_FSYNC
The man page claims that with O_FSYNC (aka O_SYNC) the kernel will not cache
written data. However, that's not true. Nor does POSIX require it.
Perhaps it was true when that section of the man page was written in r69336
(I haven't checked). But it's not true now.  Now the effect is simply that
writes are sent to disk immediately and synchronously, but they're still
cached.

See also: https://pubs.opengroup.org/onlinepubs/9699919799/
See also: ffs_write in sys/ufs/ffs/ffs_vnops.c

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20641
2019-06-14 20:35:37 +00:00
Alexander Motin
3bae917061 Minimize aggsum_compare(&arc_size, arc_c) calls.
For busy ARC situation when arc_size close to arc_c is desired.  But
then it is quite likely that aggsum_compare(&arc_size, arc_c) will need
to flush per-CPU buckets to find exact comparison result.  Doing that
often in a hot path penalizes whole idea of aggsum usage there, since it
replaces few simple atomic additions with dozens of lock acquisitions.

Replacing aggsum_compare() with aggsum_upper_bound() in code increasing
arc_p when ARC is growing (arc_size < arc_c) according to PMC profiles
allows to save ~5% of CPU time in aggsum code during sequential write
to 12 ZVOLs with 16KB block size on large dual-socket system.

I suppose there some minor arc_p behavior change due to lower precision
of the new code, but I don't think it is a big deal, since it should
affect only very small window in time (aggsum buckets are flushed every
second) and in ARC size (buckets are limited to 10 average ARC blocks
per CPU).

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-06-14 20:04:28 +00:00
Alexander Motin
b3b3aa2e29 Alike to ZoL disable metaslab allocation tracing code.
It is too generous to collect in production debug traces that can only
be read with kernel debugger.  Illumos includes special code in their
mdb debugger to read it, we don't.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-06-14 19:57:32 +00:00
Alexander Motin
284e53a401 Properly align struct multilist_sublist to cache line.
Manual Illumos alignment does not fit us due to different kmutex_t size.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-06-14 17:09:39 +00:00
Kirk McKusick
e924cb6ff9 Clarify that the -r and -R options can be used only to create level 0 dumps.
Suggested by: phk
MFC after:    3 days
2019-06-14 06:10:15 +00:00
Alan Cox
aa4a1d3a55 Change the arm64 pmap so that updates to the global count of wired pages are
not performed directly by the pmap.  Instead, they are performed by
vm_page_free_pages_toq().  (This is the same approach that we use on x86.)

Reviewed by:	kib, markj
MFC after:	10 days
Differential Revision:	https://reviews.freebsd.org/D20627
2019-06-14 04:01:08 +00:00
Doug Moore
771315283b Avoid using the prev field of vm_map_entry_t in two functions that
iterate over consecutive vm_map entries, and that can easily just
'remember' the prev value instead of looking it up.

Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D20628
2019-06-14 03:15:54 +00:00
Alexander Motin
c80038a0a7 Update td_runtime of running thread on each statclock().
Normally td_runtime is updated on context switch, but there are some kernel
threads that due to high absolute priority may run for many seconds without
context switches (yes, that is bad, but that is true), which means their
td_runtime was not updated all that time, that made them invisible for top
other then as some general CPU usage.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-06-14 01:09:10 +00:00
Warner Losh
1682a3ab4b Add opt_cam.h so we can build this outside of a kernel build. 2019-06-13 22:03:53 +00:00
Doug Moore
af1d6d6a11 Create a function for creating objects to back map entries, and one
for giving cred to a map entry backed by an object, and use them
instead of the code duplicated inline now.

Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D20370
2019-06-13 20:09:07 +00:00
Vincenzo Maffione
4f7c3b7be5 bhyve: move common code to net_utils.c
Both virtio_net and e82545 network frontends have code to validate and
generate MAC addresses. These functionalities are replicated in the two
files, so we move them in a separate compilation unit.

Reviewed by:	rgrimes, bryanv, imp, kevans
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20626
2019-06-13 17:39:32 +00:00
Warner Losh
2e3d6d0f5f Don't print the request we may be aborting in ciss_notify_abort as
part of ciss_detach. It's a left-over debug that isn't needed and also
discloses a kernel address. Only root could provoke as part of a
devctl or kldunload.

Submitted by: Fuqian Huang
MFC After: 1 week
2019-06-13 05:19:42 +00:00
Warner Losh
07810343ee Increase the timeout for READ NATIVE MAX
READ NATIVE MAX can take longer than a second if the queued NCQ I/Os
take longer than a second to drain.
2019-06-13 05:19:36 +00:00
Alan Somers
38edd595df Add test cases for epair
Implements the missing test cases for epair in a similar fashion to the
existing tests. Fixes shared abstractions to work with epair tests.

Submitted by:	Ryan Moeller <ryan@freqlabs.com>
Reviewed by:	asomers
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D20498
2019-06-13 05:05:58 +00:00
Li-Wen Hsu
3aa9258286 Fix gcc build by removing redeclaration
Reviewed by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20622
2019-06-13 03:48:36 +00:00
Alexander Motin
913095dc56 Move write aggregation memory copy out of vq_lock.
Memory copy is too heavy operation to do under the congested lock.
Moving it out reduces congestion by many times to almost invisible.
Since the original zio removed from the queue, and the child zio is
not executed yet, I don't see why would the copy need protection.
My guess it just remained like this from the time when lock was not
dropped here, which was added later to fix lock ordering issue.

Multi-threaded sequential write tests with both HDD and SSD pools
with ZVOL block sizes of 4KB, 16KB, 64KB and 128KB all show major
reduction of lock congestion, saving from 15% to 35% of CPU time
and increasing throughput from 10% to 40%.

Reviewed by:	ahrens, behlendorf, ryao
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-06-13 01:21:32 +00:00
Bryan Drewery
61b54f34a3 Don't delete .depend files outside of cleandepend.
Sponsored by:	DellEMC
2019-06-12 23:09:10 +00:00
Dimitry Andric
efc5c4420a Upgrade our copies of clang, llvm, lld, lldb, compiler-rt, libc++,
libunwind and openmp to the upstream release_80 branch r363030
(effectively, 8.0.1 rc2).  The 8.0.1 release should follow this within a
week or so.

MFC after:	2 weeks
2019-06-12 21:10:37 +00:00
Alan Cox
a04cd5cdee Change pmap_demote_l2_locked() so that it removes the superpage mapping on a
demotion failure.  Otherwise, some callers to pmap_demote_l2_locked(), such
as pmap_protect(), may leave an incorrect mapping in place on a demotion
failure.

Change pmap_demote_l2_locked() so that it handles addresses that are not
superpage aligned.  Some callers to pmap_demote_l2_locked(), such as
pmap_protect(), may not pass a superpage aligned address.

Change pmap_enter_l2() so that it correctly calls vm_page_free_pages_toq().
The arm64 pmap is updating the count of wired pages when freeing page table
pages, so pmap_enter_l2() should pass false to vm_page_free_pages_toq().

Optimize TLB invalidation in pmap_remove_l2().

Reviewed by:	kib, markj (an earlier version)
Discussed with:	andrew
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D20585
2019-06-12 20:38:49 +00:00
Mariusz Zaborski
4b3f792718 fileargs: add wrapping/unwrapping functions
Those function may be useful to pass fileargs connections around.
2019-06-12 19:31:26 +00:00
Mariusz Zaborski
a802439365 geli: style nits 2019-06-12 19:29:48 +00:00
Mariusz Zaborski
e7630efbe6 geli: partially revert r348709
Let's change the unsigned arguments to the signed one, but let's don't
change pointers to the array notation.

Requested by:	pjd
2019-06-12 19:29:12 +00:00
Stephen Hurd
705aad98c6 Some devices take undesired actions when RTS and DTR are
asserted. Some development boards for example will reset on DTR,
and some radio interfaces will transmit on RTS.

This patch allows "stty -f /dev/ttyu9.init -rtsdtr" to prevent
RTS and DTR from being asserted on open(), allowing these devices
to be used without problems.

Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D20031
2019-06-12 18:07:04 +00:00
John Baldwin
0026d8ccb7 Remove a spurious break when setting up a 64-bit memory BAR.
This was causing 'enbit' to not be initialized in this case.

CID:		1401924
Reported by:	Coverity
MFC after:	1 week
2019-06-12 16:49:01 +00:00
Jonathan T. Looney
1524298754 The current IPMI KCS code is waiting 100us for all transitions (roughly
between each byte either sent or received). However, most transitions
actually complete in 2-3 microseconds.

By polling the status register with a delay of 4us with exponential
backoff, the performance of most IPMI operations is significantly
improved:
  - A BMC update on a Supermicro x9 or x11 motherboard goes from ~1 hour
    to ~6-8 minutes.
  - An ipmitool sensor list time improves by a factor of 4.

Testing showed no significant improvements on a modern server by using
a lower delay.

The changes should also generally reduce the total amount of CPU or
I/O bandwidth used for a given IPMI operation.

Submitted by:	Loic Prylli <lprylli@netflix.com>
Reviewed by:	jhb
MFC after:	2 weeks
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20527
2019-06-12 16:06:31 +00:00
Ian Lepore
3a3ab50916 Don't attempt to include hwpmc support for armv6, we're missing some of the
necessary support functions in cpu-v6.h, and it may be that the only armv6
platform we support (RPi, the bcm2835 SOC) is incapable of supporting hwpmc.

Reported by:	dim@
2019-06-12 16:05:20 +00:00
Brandon Bergren
664104b4af Fix PPC970 boot after r348783
r348783 changed the behavior of the kernel mappings and broke booting on G5.

- Split the kernel mapping logic out so that the case where we are
running from the wrong memory space is handled using identity
mappings, and the case where we are not using a DMAP is handled by
forcibly mapping the kernel into the dmap range as intended by
r348783.

Reported by:	Mikael Urankar
Reviewed by:	luporl
Approved by:	jhibbits (mentor)
Differential Revision:	https://reviews.freebsd.org/D20608
2019-06-12 15:58:11 +00:00
Martin Matuska
fae5c36e4c MFV r348971,r348977:
Sync libarchive with vendor.

Relevant vendor changes:
  - check_symlinks_fsobj() without chdir() and fchdir()
  - bsdtar.1 manpage fixes
  - patches from OpenBSD to libarchive_fe/passphrase.c
  - version bumped to 3.4.0

MFC after:	2 weeks
2019-06-12 13:34:12 +00:00
Emmanuel Vadot
2e01fdb266 mtree: Restore mode for /var/spool/lock and opielocks
mode was dropped unintentionally in r348981 for those two directories.

Reviewed by:	rgrimes
MFC after:	1 month
X-MFC-With:	r348981
2019-06-12 11:34:16 +00:00
Cy Schubert
6ee97dd9a3 Whitespace adjustments replacing spaces with tabs.
MFC after:	1 month
X-MFC with:	r348987
2019-06-12 11:18:11 +00:00
Cy Schubert
394fa2b515 Resolve IPv6 checksum errors with stateful inspection. According to
PR/203585 this appears to have been broken by r235959, which predates
the ipfilter 5.1.2 import into FreeBSD.

The IPv6 checksum calculation is incorrect. To resolve this we call
in6_cksum() to do the the heavy lifting for us, through a new function
ipf_pcksum6(). Should we need to revisit this area again, a DTrace probe
is added to aid with future debugging.

PR:		203275, 203585
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20583
2019-06-12 11:06:58 +00:00
Cy Schubert
6000630b72 Register pfil hooks when VNET != vnet0. r302298, which virtualized ipf,
assumed the pfil hook registration performed in ipf_modload() would take
care of this. However ipf_modload() is only called when the ipl kld is
loaded or when ipfilter is first called when it is statically linked
into the kernel at build time.

Prior to this, even though r302298 has been in the tree for a while, it
has never been used. So, r302298 in reality begins now.

PR:		212000
Reported by:	ahsanb@
MFC after:	1 month
2019-06-12 11:06:54 +00:00
Cy Schubert
61208bb681 Enclose a long multi-line single conditional statement in braces to
improve legibility and aesthetics.

MFC after:	1 week
2019-06-12 11:06:51 +00:00
Emmanuel Vadot
1577943439 pkgbase: Add some tags to files installed in distribution target
Add the MK_MAIL dependant file to the runtime package as well as the
MK_KERBEROS ones the empty locate database, the FreeBSD copyright file
and the GENERIC.hints.
Tag the unbound link from /etc to /var to belong in the unbound package.

Reviewed by:	bapt
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20607
2019-06-12 09:18:23 +00:00
Emmanuel Vadot
24e1c3e196 pkgbase: Add the correct package for sendmail configuration files
Reviewed by:	bapt
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20606
2019-06-12 09:17:32 +00:00
Emmanuel Vadot
7155299509 pkgbase: Set a default package=runtime for var directories
This way every directory is at least present in packages.
While here tag some directory from being in sendmail or dma

Reviewed by:	bapt
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20605
2019-06-12 09:16:42 +00:00
Bryan Drewery
f68801df10 Stop using .OODATE for extracting firmware.
This fixes META_MODE rebuilding since it assumes that it this is
a non-consistent build command. These are always unencoded consistently
though and do not need to use the .OODATE/$? mechanism.

MFC after:	2 weeks
Reported by:	npn
Sponsored by:	DellEMC
2019-06-12 00:03:00 +00:00
Martin Matuska
67e87fe612 Update vendor/libarchive/dist to git 91b5c59ada211293bd3d9fd6e803ebfc07085c04
- cosmetic changes only
2019-06-11 23:43:29 +00:00
Bryan Drewery
c40ecff845 Add missing DPSRCS entry for assym.inc.
This brings in various CLEANFILES/DEPENDOBJS handling.

MFC after:	2 weeks
Sponsored by:	DellEMC
2019-06-11 23:35:49 +00:00
Bryan Drewery
9d2b4c4fa3 Restore genassym.o to CLEANFILES.
This was lost in r335910 for some reason.

This also fixes a META_MODE rebuild issue in some modules [1].

MFC after:	2 weeks
Reported by:	npn [1]
Sponsored by:	DellEMC
2019-06-11 23:35:34 +00:00
John Baldwin
a0c4047d4d Move declaration of warninterval out from under COMPAT_FREEBSD32.
This fixes builds of kernels without COMPAT_FREEBSD32.

Reported by:	tinderbox
MFC after:	1 month
2019-06-11 23:28:07 +00:00
Martin Matuska
e3586989c2 Update vendor/libarchive/dist to git 614110e76d9dbb9ed3e159a71cbd75fa3b23efe3
Relevant vendor changes (release 3.4.0):
  - check_symlinks_fsobj() without chdir() and fchdir()
  - bsdtar.1 manpage fixes
  - patches from OpenBSD to libarchive_fe/passphrase.c
2019-06-11 23:16:13 +00:00
John Baldwin
0f70218343 Make the warning intervals for deprecated crypto algorithms tunable.
New sysctl/tunables can now set the interval (in seconds) between
rate-limited crypto warnings.  The new sysctls are:
- kern.cryptodev_warn_interval for /dev/crypto
- net.inet.ipsec.crypto_warn_interval for IPsec
- kern.kgssapi_warn_interval for KGSSAPI

Reviewed by:	cem
MFC after:	1 month
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D20555
2019-06-11 23:00:55 +00:00
John Baldwin
574b98cbd9 Document sysctl nodes that translate their values.
This documents the behavior of sysctl_msec_to_ticks and
SYSCTL_{ADD,}_SBINTIME_[UM]SEC.

Reviewed by:	cem
MFC after:	1 week
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D20596
2019-06-11 22:57:25 +00:00
Doug Moore
e65d58a0fe To test to see if a free space is big enough compare the required
length to the difference of the two offsets that define the gap, to
avoid overflow, rather that adding the length to an offset and
comparing that to another offset.

This addresses an overflow issue reported by Peter Holm on i386.

Reported by: pho
Tested by: pho
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D20594
2019-06-11 22:41:39 +00:00
Xin LI
bf6ed41692 Avoid out of boundary access when checking invalid long filenames.
Obtained from:	OpenBSD (dir.c,v 1.25)
MFC after:	3 days
2019-06-11 22:21:29 +00:00
John Baldwin
77a0144145 Sort opt_foo.h #includes and add a missing blank line in ip_output(). 2019-06-11 22:07:39 +00:00
John Baldwin
dbdfb1fd19 Add M_NOFREE to M_FLAG_BITS. 2019-06-11 22:06:31 +00:00
John Baldwin
19c40c76e1 Trim an extra space. 2019-06-11 22:06:05 +00:00
Warner Losh
15865ae73d Minor white space changes.
Remove trailing white space that's crept into this file.
2019-06-11 20:48:19 +00:00