Commit Graph

244413 Commits

Author SHA1 Message Date
Hiroki Sato
740a5434e9 Add a workaround for servers which respond RPC_PROGNOTREGISTERED
to a clnt_create() call even when it is actually a program
version mismatch.

Normally the server is supposed to return RPC_PROGVERSMISMATCH
when it supports the specified program but not support
the specified version.  Some filers return RPC_PROGNOTREGISTERED
to RQUOTA v2 calls and FreeBSD does not retry with the old
v1 calls.  This change fixes this failure scenario.

Submitted by:	Jian-Bo Liao
PR:		236179
2019-09-21 00:17:40 +00:00
Kyle Evans
5fdac75222 msdosfs: do not deget unlinked denodes
When a file is unlinked, the denode is not reclaimed until the last
reference is dropped, but the directory entry is immediately up for reuse.
This is a problem later when createde goes to grab a denode for the newly
created entry -- we search the hash and find a dead denode, then return that
without even bumping the reference count and the data later gets truncated
when the the last reference to the unlinked file is dropped.

This manifested itself as a broken in-place strip(1) on msdosfs. elfcopy
will do a sequence incredibly roughly like this:

open("/mnt/foo", ...) => fd 3
mmap()
unlink("/mnt/foo")
open("/mnt/foo", ...) => fd 4
write(4, ...)
close(4)
close(3)

and the resulting file would be truncated, but the write succeeded, as long
as a reference to the unlinked file had not been closed.

Some archaeology indicates that this bug has likely existed since msdosfs
was converted to use vfs_hash instead of a home rolled hash implementation
in r143570. Prior to that point, the hashget implementation would do a
refcnt check while searching and explicitly only return a denode with
de_refcnt != 0. vfs_hash did not yet have the callback that it does today,
so this slipped away and did not come back when it later grew that
functionality.

The comment indicating that we want to skip these denodes has been updated
to reflect where this is actually done. My repo-diving session seems to
indicate that the refcnt check was likely never actually below the comment,
to be pedantic, but instead a detail wrapped up in the hashget
implementation since the beginning of its inclusion into FreeBSD.

This bug was the cause behind the issue addressed in r352557.

Reported by:	jhibbits
Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21731
2019-09-20 20:47:10 +00:00
Kyle Evans
bb7b569d36 loader: Respect loader_color=YES for serial consoles
It's not uncommon these days for the terminals attached to serial consoles
to support ANSI escape sequences. However, we assume escape sequences may
break some serial consoles and default to not using them when boot_serial or
boot_multicons (or if console contains "comconsole" in the forth loader) for
broader compatibility. We also have loader_color which can be explicitly set
to "NO" to disable the use of ANSI escape sequences.

The problem is that loader_color=YES gets ignored when boot_serial=YES or
boot_multicons=YES (or when console contains "comconsole" in the forth
loader).

To fix, the existing default behavior remains unchanged when loader_color is
unset, loader_color=NO explicitly disables the use of ANSI escape sequences
still, and the change is that loader_color=YES can now be used to explicitly
allow ANSI escapes when a serial console is enabled.

Submitted by:	Ryan Moeller <ryan@ixsystems.com>
Reviewed by:	tsoome (forth), kevans (lua)
MFC after:	1 week
Sponsored by:	iXsystems, Inc. (Ryan)
Differential Revision:	https://reviews.freebsd.org/D21732
2019-09-20 19:43:40 +00:00
Daichi GOTO
b1de37fadd top(1): support multibyte characters in command names (ARGV array)
depending on locale.

 - add setlocale()
 - remove printable() function
 - add VIS_OCTAL and VIS_SAFE to the flag of strvisx() to display
   non-printable characters that do not use C-style backslash sequences
   in three digit octal sequence, or remove it

This change allows multibyte characters to be displayed according to
locale. If it is recognized as a non-display character according to the
locale, it is displayed in three digit octal sequence.

Reference:
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165751.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165766.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165833.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165846.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165891.html

Submitted by: hrs
Differential Revision: https://reviews.freebsd.org/D16204
2019-09-20 17:37:23 +00:00
Justin Hibbits
39404490b7 powerpc/loader: Install ubldr without stripping
Summary:
Install's strip capability, by way of strip(1), doesn't seem to work
correctly on msdosfs, and instead ends up truncating the resulting
binary to 0-length.  As a workaround, don't strip ubldr(8).  This
fixes installworld on Book-E ubldr-based platforms, which prior to this
would need to manually install ubldr separately after installworld, in
order to have a functional ubldr.

The same thing could be done on PowerNV platforms that use msdosfs /boot
volumes, since loader and loader.kboot, etc, all get truncated to 0 on
install.  However, PowerNV does not use loader, instead loading from
petitboot, so it's not really necessary at this time.

Reviewed by:	kevans
Differential Revision: https://reviews.freebsd.org/D21725
2019-09-20 13:35:28 +00:00
Hans Petter Selasky
7fca0e69f6 Add quirk for XHCI(4) controllers to support USB control transfers
above 1Kbyte.  It might look like some XHCI(4) controllers do not
support when the USB control transfer is split using a link TRB. The
next NORMAL TRB after the link TRB is simply failing with XHCI error
code 4. The quirk ensures we allocate a 64Kbyte buffer so that the
data stage TRB is not broken with a link TRB.

Found at:	EuroBSDcon 2019
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-09-20 11:28:45 +00:00
Hans Petter Selasky
4631d7f717 Increase the maximum user-space buffer size from 256kBytes to 32MBytes for
libusb. This is useful for speeding up large data transfers while reducing
the interrupt rate.

Found at:	EuroBSDcon 2019
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-09-20 11:00:02 +00:00
Hans Petter Selasky
4e792e431a The maximum TD size is 31 and not 15.
Found at:	EuroBSDcon 2019
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-09-20 10:56:13 +00:00
Baptiste Daroussin
af3ccc0740 Ensure libthr is always built before libprivatezstd when building the
startup libs

Reported by:	"Galazka, Krzysztof" <krzysztof.galazka@intel.com>
2019-09-20 09:45:38 +00:00
Andrew Gallatin
61b8a4af71 remove redundant "ktls" in KTLS thr name
This reducesthe string width of the ktls thread name
and improves "ps" output.

Glanced at by: jhb
Event: EuroBSDCon hackathon
Sponsored by:	Netflix
2019-09-20 09:36:07 +00:00
Ed Maste
8b43027381 elf_common: add ELF note names
r348628 added a definition of NT_GNU_BUILD_ID.  Some software (Valgrind)
also expects a #define for the note name (ELF_NOTE_GNU) in the case that
NT_GNU_BUILD_ID is defined.

PR:		239669
Reported by:	Yuichiro NAITO
Sponsored by:	The FreeBSD Foundation
Event:		EuroBSDCon FreeBSD DevSummit 2019
2019-09-20 09:04:52 +00:00
Michael Tuexen
e6b3bd22d8 Fix the handling of invalid parameters in ASCONF chunks.
Thanks to Mark Wodrich from Google for reproting the issue in
https://github.com/sctplab/usrsctp/issues/376
for the userland stack.

MFC after:		3 days
2019-09-20 08:20:20 +00:00
Toomas Soome
23883413d4 loader: fix typo in zalloc. 2019-09-20 05:22:34 +00:00
Alexander Motin
657dc81d90 Improve ioat(4) NUMA-awareness.
Allocate ioat->ring memory from the device domain.
Schedule ioat->poll_timer to the first CPU of the device domain.

According to pcm-numa tool from intel-pcm port, this reduces number of
remote DRAM accesses while copying data by 75%.  And unless it is a noise,
I've noticed some speed improvement when copying data to other domain.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-09-19 22:15:57 +00:00
Mateusz Guzik
b488246b45 vfs: group fields used for per-cpu ops in one cacheline
Sponsored by:	The FreeBSD Foundation
2019-09-19 21:23:14 +00:00
Michael Gmelin
12294db4ac Fix src component detection
Reviewed by:	emaste
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D21579
2019-09-19 21:13:51 +00:00
Cy Schubert
20674acb70 Follow up on r352304 which disabled default mlockall() at startup.
Unfortunately though the original tarball supports this in ./configure
(for Linux), to fully support disabling of mlockall() by default requires
a little extra help otherwise the following is logged in syslog:

	Cannot set RLIMIT_MEMLOCK: Operation not permitted

MFC after:	2 weeks
X-MFC with:	r352304
2019-09-19 20:16:51 +00:00
Glen Barber
5f6bb72e7f Apply r346792 (cperciva) from stable/12 to head. The original commit
message:

 On non-x86 systems, use "quarterly" packages.

 x86 architectures have "latest" package builds on stable/*, so keep using
 those (they'll get switched over to "quarterly" during releases).

The original commit was a direct commit to stable/12, as at the time it
was presumed it would not be necessary for head.  However, when it is time
to create a releng branch or switch from PRERELEASE/STABLE to BETA/RC, the
pkg(7) Makefile needs further adjusting.  This commit includes those
further adjustments, evaluating the BRANCH variable from release/Makefile
to determine the pkg(7) repository to use.

MFC after:	immediate (if possible)
Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-09-19 16:43:12 +00:00
Cy Schubert
1e8687d261 Reduce calls to close(2) at startup through the use of closefrom(2).
Submitted by:	pawel.biernacki@gmail.com
Reviewed by:	mjg, cy
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D21715
2019-09-19 14:45:04 +00:00
Li-Wen Hsu
05a42b7621 Whitespace cleanup, no functional change
Sponsored by:	The FreeBSD Foundation
2019-09-19 13:25:19 +00:00
Li-Wen Hsu
e751215341 Temporarily add test_write_filter_zstd BROKEN_TESTS as it always fails in CI
There is no trivial way to mark single libarchive test skip currently so just
add it to BROKEN_TESTS for now.

PR:		240683
Sponsored by:	The FreeBSD Foundation
2019-09-19 13:23:25 +00:00
Ed Maste
75cb6429f5 freebsd-update: make usage output consistent
Drop trailing . which appeared only on description of IDS.

Submitted by:	grembo
Event:		EuroBSDCon Norway FreeBSD DevSummit
2019-09-19 11:46:43 +00:00
Ed Maste
18a52cf418 freebsd-update.8: appease igor
igor follows American style guides in the belief that abbreviations i.e.
and e.g. are always followed by a comma.  Make that change now so that
future updates to freebsd-update.8 do not complain about this.

Submitted by:	grembo
Event:		EuroBSDCon Norway FreeBSD DevSummit
2019-09-19 11:34:35 +00:00
Michael Tuexen
dd3121a895 When the RACK stack computes the space for user data in a TCP segment,
it wasn't taking the IP level options into account. This patch fixes this.
In addition, it also corrects a KASSERT and adds protection code to assure
that the IP header chain and the TCP head fit in the first fragment as
required by RFC 7112.

Reviewed by:		rrs@
MFC after:		3 days
Sponsored by:		Nertflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D21666
2019-09-19 10:27:47 +00:00
Michael Tuexen
e7a541b0b9 When processing an incoming IPv6 packet over the loopback interface which
contains Hop-by-Hop options, the mbuf chain is potentially changed in
ip6_hopopts_input(), called by ip6_input_hbh().
This can happen, because of the the use of IP6_EXTHDR_CHECK, which might
call m_pullup().
So provide the updated pointer back to the called of ip6_input_hbh() to
avoid using a freed mbuf chain in`ip6_input()`.

Reviewed by:		markj@
MFC after:		3 days
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D21664
2019-09-19 10:22:29 +00:00
Andriy Gapon
dfb2b9a361 update zfs send usage help with r352447
MFC after:	3 days
2019-09-19 09:48:01 +00:00
Andriy Gapon
6caa629e73 fix dsl_scan_ds_clone_swapped logic
It was incorrect with respect to swapping dataset IDs both in the
on-disk ZAP object and the in-memory queue.

In both cases, if only ds1 was already present, then it would be first
replaced with ds2 and then ds2 would be replaced back with ds1.  Also,
both cases did not properly handle a situation where both ds1 and ds2
are already queued.  A duplicate insertion would be attempted and its
failure would result in a panic.

This change has also been submitted to ZoL as zfsonlinux/zfs@dd262c9

PR:		239566
Reported by:	pascal.guitierrez@gmail.com
MFC after:	4 days
Sponsored by:	CyberSecure
2019-09-19 09:43:56 +00:00
Andriy Gapon
9a2ed10014 vt: fix problems with trying to switch to a closed VT
If there is an attempt to switch from a process-owned VT to a closed VT,
then vt(4) first requests the process to release its VT and only then
realizes that the target VT is closed and, so, the switch is not
possible.  So, the driver does not actually do any switch, but at the
same time the owning process is not notified about that and it does not
re-acquire the VT.

This change adds an early check for the target VT state, so that the
switch can be refused before the process coordination dance.
On top of that, the code now checks for a failure of vt_window_switch()
and calls vt_window_postswitch() for the current VT if it is in the
process mode.

Test Plan:
- configure VT1 - VT8 (ttyv0 - ttyv7) to be text consoles (run getty)
- configure VT9 (ttyv8) to rn X server
- make sure that the X server configuration allows VT switching
- leave VT10 - VT12 unconfigured
- while in the X server press Ctrl+Alt+F10
- without the patch, observe strange screen content and problems with
  keyboard input
- with the patch, observe that nothing happens

The problem has been observed and the fix has been tested with an nVidia
graphics card and the proprietary nvidia driver.
Not sure if that matters.

Reviewed by:	ray
MFC after:	2 weeks
Differential Revision: https://reviews.freebsd.org/D21704
2019-09-19 09:22:45 +00:00
Allan Jude
4a9c211af5 sys/vm/vm_glue.c: Incorrect function name in panic string
Use __func__ to avoid this issue in the future.

Submitted by:	Wuyang Chung <wuyang.chung1@gmail.com>
Reviewed by:	markj, emaste
Obtained from:	https://github.com/freebsd/freebsd/pull/410
2019-09-19 07:28:24 +00:00
Jilles Tjoelker
416e2de337 Add some tests for page fault signals and codes
It is useful to have some tests for page fault signals.

More tests would be useful but creating the conditions (such as various
kinds of running out of memory and I/O errors) is more complicated.

The tests page_fault_signal__bus_objerr_1 and
page_fault_signal__bus_objerr_2 depend on https://reviews.freebsd.org/D21566
before they can pass.

PR:		211924
Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D21624
2019-09-18 21:00:32 +00:00
Alexander Motin
f4897c94dd Fix typo, setting hidden flag instead of reparse.
Submitted by:	Ryan Moeller <ryan@ixsystems.com>
MFC after:	3 days
Sponsored by:	iXsystems, Inc.
2019-09-18 19:33:08 +00:00
Konstantin Belousov
43ce0d902c truss: decode sysctl names.
Submitted by:	Pawel Biernacki
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21688
2019-09-18 16:15:05 +00:00
Ruslan Bukin
230754ccca Add support for BERI statcounters.
BERI stands for Bluespec Extensible RISC Implementation, based on MIPS.

BERI has not implemented standard MIPS perfomance monitoring counters,
instead it provides statistical counters.

BERI statcounters have a several limitations:
- They can't be written
- They don't support start/stop operation
- None of hardware interrupt is provided on a counter overflow.

So make it separate to hwpmc_mips module and support process/system
counting mode only.

Sponsored by:	DARPA, AFRL
2019-09-18 16:13:50 +00:00
Konstantin Belousov
382e01c8dc sysctl: use names instead of magic numbers.
Replace magic numbers with symbols for internal sysctl operations.
Convert in-kernel and libc consumers.

Submitted by:	Pawel Biernacki
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21693
2019-09-18 16:13:10 +00:00
Baptiste Daroussin
85686f3425 Add the missing bits for LIBADD to properly function now that
libarchive is linked to libzstd

Pointy hat:	bapt
Reported by:	antoine
2019-09-18 08:02:03 +00:00
Baptiste Daroussin
d567f909ef Add native support for zstd to libarchive
Note that old pkg will failed to build after this. A recent ports tree (one
providing pkg 1.12+) is required to build. Older already built pkg, should
continue working as expected

PR:		238797
Exp run by:	antoine
Reviewed by:	cem
Approved by:	cem
Differential Revision:	https://reviews.freebsd.org/D20752
2019-09-18 07:57:56 +00:00
Kyle Evans
05e08cbdac src.conf(5): regenerate after r352465, r352466
These changed the defaults for the GOOGLETEST knob and added a description
for WITH_GOOGLETEST.
2019-09-18 02:04:41 +00:00
Kyle Evans
4759088665 Add description for WITH_GOOGLETEST
This is the logical negation of WITHOUT_GOOGLETEST, and helpful to have as
we now have different per-arch defaults for this option.
2019-09-18 02:03:39 +00:00
Kyle Evans
15b53426e8 googletest: default-disable on all of MIPS for now
Parts of the fusefs tests trigger a bug in current versions of llvm: IR
representation of some routine for the MIPS targets is a function with a
large number of arguments. This then leads the compiler on an hour+ long
goose chase, which is OK if you build the current tree but less-so if you're
trying external toolchain or doing a universe build involving mips when it
eventually gets switched over to LLVM.

Better, accurate details can be found in LLVM PR43263.
2019-09-18 01:58:56 +00:00
Kyle Evans
8b2195605b mips: ubldr: use truncated load address for mips32
BFD appears to silently truncate 0xffffffff80800000 when it processes the
ldscript for 32-bit mips, but LLD chokes on it as the linker script tries to
place elements above 32-bit range. It's unclear to me if silent truncation
is kosher or not and whether this patch is really what we want to do, but it
is one approach at least.

Reviewed by:	imp, mizhka
Differential Revision:	https://reviews.freebsd.org/D21487
2019-09-18 01:33:17 +00:00
Li-Wen Hsu
ebcb81079e Temporarily skip sys.netpfil.common.tos.pf_tos on i386 CI as it always fails
PR:		240086
Sponsored by:	The FreeBSD Foundation
2019-09-17 22:09:14 +00:00
Li-Wen Hsu
ae92090ae0 Temporarily skip sys.netpfil.common.forward.pf_v4 on i386 CI as it always fails
PR:		240085
Sponsored by:	The FreeBSD Foundation
2019-09-17 22:08:16 +00:00
Steve Wills
6cc4a3c970 Use correct filename in newsyslog.conf
Approved by:		bapt (implicit)
Differential Revision:	https://reviews.freebsd.org/D21561
2019-09-17 20:05:06 +00:00
Steve Wills
cd4b2a3c08 log daemon.info to /var/log/daemon.log by default
log daemon facility now that daemon(8) has syslog support which defaults to
daemon facility, info priority

Reviewed by:		bapt
Approved by:		bapt
Differential Revision:	https://reviews.freebsd.org/D21561
2019-09-17 20:03:20 +00:00
Konstantin Belousov
093aa3e311 ifconfig: add report of the string from SIOCGIFDOWNREASON.
Sample output:
# ifconfig mce0
mce0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=3ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,TXRTLMT,HWRXTSTMP>
        ether e4:1d:2d:e7:10:0a
        media: Ethernet autoselect <full-duplex,rxpause,txpause>
        status: no carrier (Negotiation failure)
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Reviewed by:	hselasky, rrs
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21527
2019-09-17 18:51:10 +00:00
Konstantin Belousov
247cf5664e Add SIOCGIFDOWNREASON.
The ioctl(2) is intended to provide more details about the cause of
the down for the link.

Eventually we might define a comprehensive list of codes for the
situations.  But interface also allows the driver to provide free-form
null-terminated ASCII string to provide arbitrary non-formalized
information.  Sample implementation exists for mlx5(4), where the
string is fetched from firmware controlling the port.

Reviewed by:	hselasky, rrs
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21527
2019-09-17 18:49:13 +00:00
Konstantin Belousov
6fd583583b Further refine r352393, only call vnode_pager_setsize() outside the
node lock when shrinking.

This is similar to r252528, applied to the above commit.

Apparently there is a race which makes necessary at least to keep the
n_size and pager size consistent when extending.  Current suspect is
that iod threads perform vnode_pager_setsize() without taking the
vnode lock, which corrupts the file content.

Reported and tested by:	Masachika ISHIZUKA <ish@amail.plala.or.jp>
Discussed with:	rmacklem (related issues)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-09-17 18:41:39 +00:00
Konstantin Belousov
0edc114ac0 realloc(x, 0) should not return NULL.
See http://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm#dr_400.
Upstream jemalloc issue is opened by emaste at
https://github.com/jemalloc/jemalloc/issues/1629.

Reviewed by:	emaste
PR:	240456
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
DIfferential revision:	https://reviews.freebsd.org/D21632
2019-09-17 18:36:29 +00:00
Konstantin Belousov
55894117b1 Return EISDIR when directory is opened with O_CREAT without O_DIRECTORY.
Reviewed by:	bcr (man page), emaste (previous version)
PR:	240452
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
DIfferential revision:	https://reviews.freebsd.org/D21634
2019-09-17 18:32:18 +00:00
Toomas Soome
68861a62f5 loader: provide u> and xemit words if needed
We have external interpreter (userboot.so) which may be lagging behind
with updates and may be missing u> xemit words.
2019-09-17 18:05:33 +00:00