Commit Graph

20295 Commits

Author SHA1 Message Date
Xin LI
f89d207279 Separate kernel crc32() implementation to its own header (gsb_crc32.h) and
rename the source to gsb_crc32.c.

This is a prerequisite of unifying kernel zlib instances.

PR:		229763
Submitted by:	Yoshihiro Ota <ota at j.email.ne.jp>
Differential Revision:	https://reviews.freebsd.org/D20193
2019-06-17 19:49:08 +00:00
Martin Matuska
7d8ec1b79d MFV r349134:
Sync libarchive with vendor.

Relevant vendor changes:
  PR #1212: RAR5 reader - window_mask was not updated correctly
            (OSS-Fuzz 15278)
  OSS-Fuzz 15120: RAR reader - extend use after free bugfix

MFC after:	1 week (together with r348993)
2019-06-17 11:46:37 +00:00
Bryan Drewery
dcf1f83876 Support reading in .depend files.
This is for an upcoming change that fixes .depend handling in here.
It will cause some duplicate sources which need to be trimmed out.

MFC after:	2 weeks
Sponsored by:	DellEMC
2019-06-15 17:08:32 +00:00
Bryan Drewery
c63c5ab001 Fix .depend files to work for build tools.
This is somewhat of a follow-up to r335746.

MFC after:	2 weeks
Sponsored by:	DellEMC
2019-06-15 17:08:13 +00:00
Alan Somers
5993fa5582 open(2): fix the description of O_FSYNC
The man page claims that with O_FSYNC (aka O_SYNC) the kernel will not cache
written data. However, that's not true. Nor does POSIX require it.
Perhaps it was true when that section of the man page was written in r69336
(I haven't checked). But it's not true now.  Now the effect is simply that
writes are sent to disk immediately and synchronously, but they're still
cached.

See also: https://pubs.opengroup.org/onlinepubs/9699919799/
See also: ffs_write in sys/ufs/ffs/ffs_vnops.c

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20641
2019-06-14 20:35:37 +00:00
Dimitry Andric
efc5c4420a Upgrade our copies of clang, llvm, lld, lldb, compiler-rt, libc++,
libunwind and openmp to the upstream release_80 branch r363030
(effectively, 8.0.1 rc2).  The 8.0.1 release should follow this within a
week or so.

MFC after:	2 weeks
2019-06-12 21:10:37 +00:00
Mariusz Zaborski
4b3f792718 fileargs: add wrapping/unwrapping functions
Those function may be useful to pass fileargs connections around.
2019-06-12 19:31:26 +00:00
Martin Matuska
fae5c36e4c MFV r348971,r348977:
Sync libarchive with vendor.

Relevant vendor changes:
  - check_symlinks_fsobj() without chdir() and fchdir()
  - bsdtar.1 manpage fixes
  - patches from OpenBSD to libarchive_fe/passphrase.c
  - version bumped to 3.4.0

MFC after:	2 weeks
2019-06-12 13:34:12 +00:00
Dimitry Andric
90579156da Vendor import of compiler-rt release_80 branch r363030:
https://llvm.org/svn/llvm-project/compiler-rt/branches/release_80@363030
2019-06-11 18:16:59 +00:00
Dimitry Andric
70198750b9 Vendor import of clang release_80 branch r363030:
https://llvm.org/svn/llvm-project/cfe/branches/release_80@363030
2019-06-11 18:16:50 +00:00
Dimitry Andric
687a64222b Vendor import of llvm release_80 branch r363030:
https://llvm.org/svn/llvm-project/llvm/branches/release_80@363030
2019-06-11 18:16:27 +00:00
Hans Petter Selasky
08216d1854 Fix for reading the configuration descriptor in libusb. Catch invalid
configuration descriptor reads early on to avoid issues with devices
that don't check for a valid USB configuration read request.

Submitted by:	takahiro.kurosawa@gmail.com
PR:		238412
MFC after:	3 days
2019-06-08 09:34:02 +00:00
Mariusz Zaborski
5c816e43b4 unlink: add missing function to unlink.2 man page 2019-06-05 22:36:19 +00:00
Allan Jude
87e34f6cf4 sha.3: clarify admonition against use in NEW signature schemes
Reported by:	cem, cperciva (grammar)
2019-06-05 20:18:08 +00:00
Bjoern A. Zeeb
96b3894360 Rather than using the legacy IP struct fields in the union for the
port number, properly access them by their IPv6 names.
This will make it easier to slice up and compile out address families
in the future.

No functional change intended.

MFC after:	6 weeks
2019-06-04 20:53:35 +00:00
Ed Maste
9c954a4843 build llvm-ar and llvm-nm with Clang (promote out of CLANG_EXTRAS)
To facilitate experimentation with LTO we require an ar that supports
LLVM IR, and to a lesser degree also an nm.  As a first step always
install llvm-ar and llvm-nm.

Sponsored by:	The FreeBSD Foundation
2019-06-04 13:00:49 +00:00
Mark Johnston
8e7299c08b Typo.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-06-03 16:47:00 +00:00
Kyle Evans
067498ed56 jail_getid(3): add special-case immediate return for jid 0
As depicted in the comment: jid 0 always exists, but the lookup will fail as
it does not appear in the kernel's alljails list being a special jail. Some
callers will expect/rely on this, and we have no reason to lie because it
does always exist.

Reported by:	Stefan Hegnauer <stefan.hegnauer gmx ch>
MFC after:	soon (regression, breaks inspecting jail host bits, partial
revert)
2019-06-02 14:03:56 +00:00
Kyle Evans
71259635f7 llvm-symbolizer: Move out of CLANG_EXTRAS, into CLANG
ASAN reports become a lot more useful with llvm-symbolizer in $PATH, and the
build is not much more time-consuming. The added benefit is that the
resulting reports will actually include symbol information; without, thread
trace information includes a bunch of addresses that immediately resolve to
an inline function in
^/contrib/compiler-rt/lib/sanitizer_common/sanitizer_common.h and take a
little more effort to examine.

Reviewed by:	emaste
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20484
2019-06-02 04:04:21 +00:00
Dimitry Andric
740890be23 Partially revert r271349, which disabled the msun cexp test #4 on i386.
Ensure the expected result is stored first in a volatile variable with
the desired type.  This makes all the tests succeed.

Slightly changed from the original pull request, but functionally the
same.

Obtained from:	https://github.com/freebsd/freebsd/pull/401
Submitted by:	Moritz Buhl <gh@moritzbuhl.de>
PR:		191676
MFC after:	3 days
2019-06-01 20:08:10 +00:00
Ed Maste
15c0034ff2 libatf: remove workaround not required after atf >= 0.18 update
lib/atf/libatf-c/tests/Makefile added the -Wno-duplicate-decl-specifier
due to an issue with an old version of ATF.  ATF has long since been
updated to a version with the fix so the workaround is no longer
necessary.

Found during review for PR 236889.

PR:		236889
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2019-06-01 18:26:07 +00:00
Mark Johnston
0a8314e06c libelftc: Add MLINKS.
Reviewed by:	emaste
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20474
2019-05-30 19:18:01 +00:00
Gleb Smirnoff
bec2d7e9a2 The KVM code also needs a fix similar to r344269.
Reported by:	pho
2019-05-29 03:14:46 +00:00
Pedro F. Giffuni
ec845b07c6 typo: suppported. 2019-05-29 02:08:23 +00:00
Xin LI
a6c0c82406 Chase r261913: hardcoded default crypt(3) algorithm is SHA-512 when DES
is not available.

Submitted by:	Ali Mashtizadeh <ali mashtizadeh.com>
MFC after:	3 days
2019-05-27 06:37:23 +00:00
Li-Wen Hsu
2b32abd2b1 Correct the argument passed to g_eli_algo2str()
MFC with:	r348206
Sponsored by:	The FreeBSD Foundation
2019-05-24 06:49:47 +00:00
Kyle Evans
a9cefddd40 jail_getid(3): validate jid string input
Currently, if jail_getid(3) is passed in a numeric string, it assumes that
this is a jid string and passes it back converted to an int without checking
that it's a valid/existing jid. This breaks consumers that might use
jail_getid(3) to see if it can trivially grab a jid from a name if that name
happens to be numeric but not actually the name/jid of the jail. Instead of
returning -1 for the jail not existing, it'll return the int version of the
input and the consumer will not fallback to trying other methods.

Pass the numeric input to jail_get(2) as the jid for validation, rather than
the name. This works well- the kernel enforces that jid=name if name is
numeric, so doing the safe thing and checking numeric input as a jid will
still DTRT based on the description of jail_getid.

Reported by:	Wes Maag
Reviewed by:	jamie, Wes Maag
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D20388
2019-05-24 01:28:07 +00:00
John Baldwin
5c420aae3b Add deprecation warnings for weaker algorithms to geli(4).
- Triple DES has been formally deprecated in Kerberos (RFC 8429)
  and is soon to be deprecated in IPsec (RFC 8221).
- Blowfish is deprecated.  FreeBSD doesn't support its successor
  (Twofish).
- MD5 is generally considered a weak digest that has known attacks.

geli refuses to create new volumes using these algorithms via 'geli
init'.  It also warns when attaching to existing volumes or creating
temporary volumes via 'geli onetime' .  The plan is to fully remove
support for these algorithms in FreeBSD 13.

Note that none of these algorithms have ever been the default
algorithm used by geli(8).  Users would have had to explicitly select
these algorithms when creating volumes in the past.

Reviewed by:	cem, delphij
MFC after:	3 days
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D20344
2019-05-23 22:31:55 +00:00
Baptiste Daroussin
45a13fd899 Move back group, master.passwd and shells to etc directory
Use the .PATH mechanism instead so keep installing them from lib/libc/gen

While here revert 347961 and 347893 which are no longer needed

Discussed with:	manu
Tested by:	manu
ok manu@
2019-05-23 18:37:05 +00:00
Alan Somers
8bbd9a3839 Link fhlinkat(2) man page
Reviewed by:	kib
MFC after:	3 days
MFC-With:	r341689
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20339
2019-05-22 01:11:21 +00:00
Allan Jude
34841dd627 Add admonitions against using MD5 and SHA1 to the API man pages 2019-05-21 22:17:00 +00:00
Allan Jude
f26f4cf170 Add missing errors section to md[2-5], ripemd160, sha*, and skein* manpages
PR:		148987
Submitted by:	Dan Lukes <dan@obluda.cz> (original version)
MFC after:	1 week
Sponsored by:	Klara Systems
Event:	Waterloo Hackathon 2019
2019-05-21 22:11:53 +00:00
Ed Maste
b0ec537031 mips: remove 16-byte alignment from .init and .fini epilogues
The .init and .fini epilogues from crtn should be placed immediately
after any instructions in .init and .fini sections from the linked
objects.  Using 16-byte alignment for the epilogues on MIPS was a bug,
but it did not cause any issue with GNU ld as GNU ld (2.17.50) fills the
padding with NOPs.

Current versions of LLD fill any padding between different object files
with trap instructions.  Inserting trap padding prior to the .init/.fini
epilogue is undesriable as the resulting binary will crash at runtime.

The .init and .fini sections in object files linked between crti and
crtn must already be a multiple of the instruction size and so no
alignment directive is required in crtn.  Indeed, other architectures
(except sparc64) do not specify alignment in their crtn implementations.

Reported by:	arichardson
Reviewed by:	andrew
Event:		Waterloo Hackathon 2019
Differential Revision:	https://reviews.freebsd.org/D18291
2019-05-20 20:48:04 +00:00
Li-Wen Hsu
d08b7fa2e5 Install missing data file for
lib.libarchive.functional_test.test_read_format_zip_utf8_paths

MFC after:	2 weeks (with r347989)
Sponsored by:	The FreeBSD Foundation
2019-05-20 18:07:45 +00:00
Martin Matuska
52c2bb7516 MFV r347989:
Sync libarchive with vendor.

Relevant vendor changes:
  Issue #795: XAR - do not try to add xattrs without an allocated name
  PR #812: non-recursive option for extract and list
  PR #958: support reading metadata from compressed files
  PR #999: add --exclude-vcs option to bsdtar
  Issue #1062: treat empty archives with a GNU volume header as valid
  PR #1074: Handle ZIP files with trailing 0s in the extra fields
            (Android APK archives)
  PR #1109: Ignore padding in Zip extra field data (Android APK archives)
  PR #1167: fix problems related to unreadable directories
  Issue #1168: fix handling of strtol() and strtoul()
  PR #1172: RAR5 - fix invalid window buffer read in E8E9 filter
  PR #1174: ZIP reader - fix of MSZIP signature parsing
  PR #1175: gzip filter - fix reading files larger than 4GB from memory
  PR #1177: gzip filter - fix memory leak with repeated header reads
  PR #1180: ZIP reader - add support for Info-ZIP Unicode Path Extra Field
  PR #1181: RAR5 - fix merge_block() recursion
            (OSS-Fuzz 12999, 13029, 13144, 13478, 13490)
  PR #1183: fix memory leak when decompressing ZIP files with LZMA
  PR #1184: fix RAR5 OSS-Fuzz issues 12466, 14490, 14491, 12817
    OSS-Fuzz 12466: RAR5 - fix buffer overflow when parsing huffman tables
    OSS-Fuzz 14490, 14491: RAR5 - fix bad shift-left operations
    OSS-Fuzz 12817: RAR5 - handle a case with truncated huffman tables
  PR #1186: RAR5 - fix invalid type used for dictionary size mask
            (OSS-Fuzz 14537)
  PR #1187: RAR5 - fix integer overflow (OSS-Fuzz 14555)
  PR #1190: RAR5 - RAR5 don't try to unpack entries marked as directories
            (OSS-Fuzz 14574)
  PR #1196: RAR5 - fix a potential SIGSEGV on 32-bit builds
  OSS-Fuzz 2582: RAR - fix use after free if there is an invalid entry
  OSS-Fuzz 14331: RAR5 - fix maximum owner name length
  OSS-Fuzz 13965: RAR5 - use unsigned int for volume number + range check

  Additional RAR5 reader changes:
    - support symlinks, hardlinks, file owner, file group, versioned files
    - change ARCHIVE_FORMAT_RAR_V5 to 0x100000
    - set correct mode for readonly directories
    - support readonly, hidden and system Windows file attributes

MFC after:	2 weeks
2019-05-20 12:57:39 +00:00
Simon J. Gerraty
980bde5834 libsecureboot: allow control of when pseudo pcr is updated
During boot we only want to measure things which *must*
be verified - this should provide more deterministic ordering.

Reviewed by:	stevek
MFC after:	1 week
Sponsored by:	Juniper Networks
Differential Revision:	https://reviews.freebsd.org/D20297
2019-05-19 20:28:49 +00:00
Simon J. Gerraty
e5ec655d67 load_key_buf do not free data from dearmor
The data returned by dearmor is referenced by the key
leave it alone!

Reviewed by:	stevek
MFC after:	2 days
2019-05-19 20:24:17 +00:00
Dimitry Andric
001ec17e68 To avoid unnecessarily modifying ports, add a -lgomp symlink, since GCC
does not ship a -lomp symlink.  Also update OptionalObsoleteFiles for
this, and add 32-bit variants while here.

Submitted by:	jbeich
PR:		237975
MFC after:	3 days
2019-05-19 20:13:55 +00:00
Konstantin Belousov
7c5a46a1bc Remove resolver_qual from DEFINE_IFUNC/DEFINE_UIFUNC macros.
In all practical situations, the resolver visibility is static.

Requested by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Approved by:	so (emaste)
Differential revision:	https://reviews.freebsd.org/D20281
2019-05-16 22:20:54 +00:00
Kyle Evans
88a950762a libbe(3): Descend into children of datasets w/ mountpoint=none
These datasets will generally be canmount=noauto,mountpoint=none (e.g.
zroot/var) but have children that may need to be mounted. Instead of
skipping that segment for no good reason, descend.

Submitted by:	Wes Maag
Reported by:	Wes Maag
MFC after:	3 days
2019-05-16 02:11:33 +00:00
Brad Davis
871b6b10ce Move master.passwd and group to lib/libc/gen/
libc was picked as the destination location for these because of the syscalls
that use these files as the lowest level place they are referenced.

Approved by:	will (mentor), rgrimes, manu
Differential Revision:	https://reviews.freebsd.org/D16728
2019-05-16 01:09:13 +00:00
Benedict Reuschling
15c9c06e6a Add small EXAMPLE section to bsearch.3.
Submitted by:		fernape (via Phabricator)
Reviewed by:		bcr, jilles, dab
Approved by:		bcr (man pages), jilles (src)
MFC after:		3 days
Differential Revision:	https://reviews.freebsd.org/D19902
2019-05-15 15:54:27 +00:00
Mark Johnston
adbb25df4b Extend the libcap_sysctl tests.
- Add some coverage for cap_sysctl(3).
- Add a test for the case where the caller wishes to find the sysctl
  output length without specifying an output buffer.

Reviewed by:	oshogbo
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17856
2019-05-13 17:53:03 +00:00
Mark Johnston
3c766430f7 Convert the libcap_sysctl test cases to ATF.
Reviewed by:	oshogbo
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17855
2019-05-13 17:51:03 +00:00
Mark Johnston
1608c46ea4 Add cap_sysctl(3) and cap_sysctlnametomib(3).
These complement cap_sysctlbyname(3) to provide a drop-in
replacement for the corresponding libc functions.

Also revise the libcap_sysctl limit interface to provide access
to sysctls by MIB, and to avoid direct manipulation of nvlists
by the caller.

Reviewed by:	oshogbo
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17854
2019-05-13 17:49:54 +00:00
Mark Johnston
54a3a11421 Provide separate accounting for user-wired pages.
Historically we have not distinguished between kernel wirings and user
wirings for accounting purposes.  User wirings (via mlock(2)) were
subject to a global limit on the number of wired pages, so if large
swaths of physical memory were wired by the kernel, as happens with
the ZFS ARC among other things, the limit could be exceeded, causing
user wirings to fail.

The change adds a new counter, v_user_wire_count, which counts the
number of virtual pages wired by user processes via mlock(2) and
mlockall(2).  Only user-wired pages are subject to the system-wide
limit which helps provide some safety against deadlocks.  In
particular, while sources of kernel wirings typically support some
backpressure mechanism, there is no way to reclaim user-wired pages
shorting of killing the wiring process.  The limit is exported as
vm.max_user_wired, renamed from vm.max_wired, and changed from u_int
to u_long.

The choice to count virtual user-wired pages rather than physical
pages was done for simplicity.  There are mechanisms that can cause
user-wired mappings to be destroyed while maintaining a wiring of
the backing physical page; these make it difficult to accurately
track user wirings at the physical page layer.

The change also closes some holes which allowed user wirings to succeed
even when they would cause the system limit to be exceeded.  For
instance, mmap() may now fail with ENOMEM in a process that has called
mlockall(MCL_FUTURE) if the new mapping would cause the user wiring
limit to be exceeded.

Note that bhyve -S is subject to the user wiring limit, which defaults
to 1/3 of physical RAM.  Users that wish to exceed the limit must tune
vm.max_user_wired.

Reviewed by:	kib, ngie (mlock() test changes)
Tested by:	pho (earlier version)
MFC after:	45 days
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D19908
2019-05-13 16:38:48 +00:00
Justin Hibbits
2f420a7c7f revert r346588 for now
The rewrite of strcmp in assembly uses an instruction added in PowerISA
2.05, making it SIGILL on CPUs older than the POWER6, such as the PPC970 in
the PowerMac G5.  Revert this until we get clang+lld, or retire the in-tree
binutils in favor of newer binutils with IFUNC support, whichever comes
first.
2019-05-11 15:17:42 +00:00
Mark Johnston
7f75bbd066 Atomically update the global gMsgId in libnetgraph.
Otherwise concurrently running threads may inadvertently use the same
token for different messages.

Preserve the behaviour of disallowing negative message tokens, but allow
a message token value of zero since this simplifies the code a bit and
tokens are documented to be non-negative.

PR:		234442
Reported and tested by:	eugen
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2019-05-10 16:43:47 +00:00
Simon J. Gerraty
9bee6a6083 libsecureboot: make it easier to customize trust anchors
Avoid making hash self-tests depend on X.509 certs.
Include OpenPGP keys in trust store count.

Reviewed by:	stevek
MFC after:	1 week
Sponsored by:	Juniper Networks
Differential Revision:	https://reviews.freebsd.org/D20208
2019-05-09 22:25:12 +00:00
Xin LI
c9083b850a Move contrib/zlib to sys/contrib/zlib so that we can use it in kernel.
This is a prerequisite of unifying kernel zlib instances.

Submitted by:	Yoshihiro Ota <ota at j.email.ne.jp>
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20191
2019-05-08 08:43:15 +00:00