Commit Graph

19935 Commits

Author SHA1 Message Date
Yuri Pankov
547bc083d6 regcomp: reduce size of bitmap for multibyte locales
This fixes the obscure endless loop seen with case-insensitive
patterns containing characters in 128-255 range;  originally
found running GNU grep test suite.

Our regex implementation being kludgy translates the characters
in case-insensitive pattern to bracket expression containing both
cases for the character and doesn't correctly handle the case when
original character is in bitmap and the other case is not, falling
into the endless loop going through in p_bracket(), ordinary(),
and bothcases().

Reducing the bitmap to 0-127 range for multibyte locales solves this
as none of these characters have other case mapping outside of bitmap.
We are also safe in the case when the original character outside of
bitmap has other case mapping in the bitmap (there are several of those
in our current ctype maps having unidirectional mapping into bitmap).

Reviewed by:	bapt, kevans, pfg
Differential revision:	https://reviews.freebsd.org/D18302
2018-12-12 04:23:00 +00:00
Kirk McKusick
8f829a5cf0 Continuing efforts to provide hardening of FFS. This change adds a
check hash to the filesystem inodes. Access attempts to files
associated with an inode with an invalid check hash will fail with
EINVAL (Invalid argument). Access is reestablished after an fsck
is run to find and validate the inodes with invalid check-hashes.
This check avoids a class of filesystem panics related to corrupted
inodes. The hash is done using crc32c.

Note this check-hash is for the inode itself and not any of its
indirect blocks. Check-hash validation may be extended to also
cover indirect block pointers, but that will be a separate (and
more costly) feature.

Check hashes are added only to UFS2 and not to UFS1 as UFS1 is
primarily used in embedded systems with small memories and low-powered
processors which need as light-weight a filesystem as possible.

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-12-11 22:14:37 +00:00
Dimitry Andric
959530cc41 Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
the upstream release_70 branch r348686 (effectively, 7.0.1 rc3).  The
release will follow very soon, but no more functional changes are
expected.

Release notes for llvm, clang and lld 7.0.0 are available here:
<http://releases.llvm.org/7.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/lld/docs/ReleaseNotes.html>

PR:		230240, 230355
Relnotes:	yes
MFC after:	2 months
2018-12-11 19:05:28 +00:00
Dimitry Andric
893405b45f Merge ^/head r341764 through r341812. 2018-12-11 06:47:04 +00:00
Konstantin Belousov
eba8ab0e3e Remove special case handling for getfhat(fd, NULL, handle).
There is no reason for it to behave differently from openat(fd, NULL).
Also the handling did not worked because the substituted path was from
the system address space, causing EFAULT.

Submitted by:	Jack Halford <jack@gandi.net>
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D18501
2018-12-11 02:48:49 +00:00
Dimitry Andric
67350cb56a Merge ^/head r340918 through r341763. 2018-12-09 11:39:45 +00:00
Dimitry Andric
0b9890fcbf Merge llvm, clang, lld, lldb, compiler-rt and libc++ release_70 branch
r348686 (effectively 7.0.1 rc3), resolve conflicts, and bump version
numbers.

PR:		230240, 230355
2018-12-09 11:36:04 +00:00
Dimitry Andric
52c5eb8567 Vendor import of clang release_70 branch r348686:
https://llvm.org/svn/llvm-project/cfe/branches/release_70@348686
2018-12-08 14:31:58 +00:00
Dimitry Andric
cfd4990248 Vendor import of llvm release_70 branch r348686:
https://llvm.org/svn/llvm-project/llvm/branches/release_70@348686
2018-12-08 14:31:49 +00:00
Mark Johnston
08c4a937a6 Let the cap_syslog capability inherit stdio descriptors.
Otherwise cap_openlog(LOG_PERROR) doesn't work.

Reviewed by:	oshogbo
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18457
2018-12-07 15:52:50 +00:00
Konstantin Belousov
d1fd400a80 Add new file handle system calls.
Namely, getfhat(2), fhlink(2), fhlinkat(2), fhreadlink(2).  The
syscalls are provided for a NFS userspace server (nfs-ganesha).

Submitted by:	Jack Halford <jack@gandi.net>
Sponsored by:	Gandi.net
Tested by:	pho
Feedback from:	brooks, markj
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D18359
2018-12-07 15:17:29 +00:00
Mark Johnston
8e2a46c8bd Add zdopen(3) to complement zopen(3).
zdopen() can be used in capability mode.  Update zopen.3 accordingly
and fix some grammar nits while I'm here.

Reviewed by:	delphij
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18456
2018-12-06 20:03:06 +00:00
Yuri Pankov
16e2120af9 syslog(3): stdarg.h is needed only for vsyslog()
Reported by:	Brian Wood <woodbrian77@gmail.com>
Reviewed by:	asomers, ngie
Differential revision:	https://reviews.freebsd.org/D18334
2018-12-06 18:03:18 +00:00
Kirk McKusick
fb14e73cb4 Normally when an attempt is made to mount a UFS/FFS filesystem whose
superblock has a check-hash error, an error message noting the
superblock check-hash failure is printed and the mount fails. The
administrator then runs fsck to repair the filesystem and when
successful, the filesystem can once again be mounted.

This approach fails if the filesystem in question is a root filesystem
from which you are trying to boot. Here, the loader fails when trying
to access the filesystem to get the kernel to boot. So it is necessary
to allow the loader to ignore the superblock check-hash error and make
a best effort to read the kernel. The filesystem may be suffiently
corrupted that the read attempt fails, but there is no harm in trying
since the loader makes no attempt to write to the filesystem.

Once the kernel is loaded and starts to run, it attempts to mount its
root filesystem. Once again, failure means that it breaks to its prompt
to ask where to get its root filesystem. Unless you have an alternate
root filesystem, you are stuck.

Since the root filesystem is initially mounted read-only, it is
safe to make an attempt to mount the root filesystem with the failed
superblock check-hash. Thus, when asked to mount a root filesystem
with a failed superblock check-hash, the kernel prints a warning
message that the root filesystem superblock check-hash needs repair,
but notes that it is ignoring the error and proceeding. It does
mark the filesystem as needing an fsck which prevents it from being
enabled for writing until fsck has been run on it. The net effect
is that the reboot fails to single user, but at least at that point
the administrator has the tools at hand to fix the problem.

Reported by:    Rick Macklem (rmacklem@)
Discussed with: Warner Losh (imp@)
Sponsored by:   Netflix
2018-12-06 00:09:39 +00:00
Alan Somers
006678fd05 stat(2): clarify which syscalls modify file timestamps
The list of syscalls that modify st_atim, st_mtim, and st_ctim was quite out
of date and probably not accurate to begin with.  Update it, and make it
clear that the list is open-ended.

Reviewed by:	kib
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D18410
2018-12-05 17:28:40 +00:00
Alex Richardson
abc73e04c2 Allow bootstrapping libopenbsd on Linux
The getdtablecount.c file won't compile on Linux but it seems like none of
the bootstrap tools actually need it.

Reviewed By:	emaste, brooks
Differential Revision: https://reviews.freebsd.org/D14244
2018-12-05 10:58:02 +00:00
Brooks Davis
db19a093bb Remove MD __sys_* private symbols.
No references to any of these exist in the tree. The list was also
erratic with different architectures exporting different things
(arm64 and riscv exported none).

Reviewed by:	kib
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18425
2018-12-05 00:46:09 +00:00
Sean Bruno
8dfd9dcd55 Revert r340997 at the request of multiple users.
- breaks ports-mgmt/pkg build for mips64, powerpc64 and i386 for some users.

--- pkg-static ---
/usr/lib/liblzma.a(stream_encoder_mt.o): In function `mythread_cond_init':
/usr/local/poudriere/jails/ppc64/usr/src/contrib/xz/src/common/mythread.h:230:
undefined reference to `pthread_condattr_init'

Reported by:	jhibbits zeising
2018-12-04 03:23:14 +00:00
Konstantin Belousov
a66732de4f Print type designator 'D' for the KF_TYPE_DEV files.
No type-specific data is provided by the kernel.

Sponsored by:	Mellanox Technologies
MFC after:	1 week
2018-12-03 23:42:04 +00:00
Brooks Davis
52af996782 Remove declarations of syscalls not used in libthr.
Reviewed by:	kib
Sponsored by:	DARPA, AFRL
2018-12-03 22:09:23 +00:00
Ed Maste
0e450664ad disable BIND_NOW in libc, libthr, and rtld
An issue remains with BIND_NOW and processes using threads.  For now,
restore libc's BIND_NOW disable, and also disable BIND_NOW in rtld and
libthr.

A patch is in review (D18400) that likely fixes this issue, but just
disable BIND_NOW pending further testing after it is committed.

PR:		233333
Sponsored by:	The FreeBSD Foundation
2018-12-03 15:59:46 +00:00
Justin Hibbits
0f3a4adcb2 Fix PowerPC64 ELFv1-specific problem in __elf_phdr_match_addr() leading to crash
in threaded programs that unload libraries.

Summary:
The GNOME update to 3.28 exposed a bug in __elf_phdr_match_addr(), which leads
to a crash when building devel/libsoup on powerpc64.

Due to __elf_phdr_match_addr() limiting its search to PF_X sections, on the
PPC64 ELFv1 ABI, it was never matching function pointers properly.

This meant that libthr was never cleaning up its atfork list in
__pthread_cxa_finalize(), so if a library with an atfork handler was unloaded,
libthr would crash on the next fork.

Normally, the null pointer check it does before calling the handler would avoid
this crash, but, due to PPC64 ELFv1 using function descriptors instead of raw
function pointers, a null check against the pointer itself is insufficient, as
the pointer itself was not null, it was just pointing at a function descriptor
that had been zeroed. (Which is an ABI violation.)

Calling a zeroed function descriptor on PPC64 ELFv1 causes a jump to address 0
with a zeroed r2 and r11.

Submitted by:	git_bdragon.rtk0.net
Reviewed By:	kib
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D18364
2018-12-01 20:39:20 +00:00
Dimitry Andric
6ae2bfad8a Vendor import of clang release_70 branch r348011:
https://llvm.org/svn/llvm-project/cfe/branches/release_70@348011
2018-12-01 15:41:40 +00:00
Dimitry Andric
d17fea9f41 Vendor import of llvm release_70 branch r348011:
https://llvm.org/svn/llvm-project/llvm/branches/release_70@348011
2018-12-01 15:41:24 +00:00
Mateusz Guzik
ddf6571230 amd64: align target memmove buffer to 16 bytes before using rep movs
See the review for sample test results.

Reviewed by:	kib (kernel part)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18401
2018-12-01 14:20:32 +00:00
Mateusz Guzik
94243af2da amd64: handle small memmove buffers with overlapping stores
Handling sizes of > 32 backwards will be updated later.

Reviewed by:	kib (kernel part)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18387
2018-11-30 20:58:08 +00:00
Mariusz Zaborski
77ebcc05ea libcasper: do not fail if Capsicum is not avilable
Casper service can be also used on the kernels without Capsicum support.

Reported by:	sbruno
Tested by:	sbruno
2018-11-30 19:27:14 +00:00
Eric van Gyzen
dbb1c64e82 Use _thr_isthreaded() and _thr_setthreaded() wrappers
...instead of directly using the global variable.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-11-30 03:02:49 +00:00
Eric van Gyzen
ad8c236b28 _thr_setthreaded() cannot fail; change return type to void
Also remove logic to avoid unnecessary stores to the global variable.
Thread creation and destruction are heavy enough that any supposed savings
is in the noise.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-11-30 03:01:32 +00:00
Mateusz Guzik
2847cfce54 amd64: remove stale attribution for memmove work
While the routine started as expanded bcopy, it is now entirely rewritten.

Sponsored by:	The FreeBSD Foundation
2018-11-30 00:47:36 +00:00
Mateusz Guzik
dd219e5ea5 amd64: tidy up copying backwards in memmove
For non-ERMS case the code used handle possible trailing bytes with
movsb first and then followed it up with movsq. This also happened
to alter how calculations were done for other cases.

Handle the tail with regular movs, just like when copying forward.
Use leaq to calculate the right offset from the get go, instead of
doing separate add and sub.

This adjusts the offset for non-rep cases so that they can be used
to handle the tail.

The routine is still a work in progress.

Sponsored by:	The FreeBSD Foundation
2018-11-30 00:45:10 +00:00
Dag-Erling Smørgrav
a768df3e91 When deciding whether to send the complete URL or just the document part,
we were looking at the original URL rather than the one we were currently
processing.  This meant that if we were trying to retrieve an HTTP URL but
were redirected to an HTTPS URL, and HTTPS proxying was enabled, we would
send an invalid request and most likely get garbage back.

MFC after:	3 days
2018-11-27 16:23:17 +00:00
Dag-Erling Smørgrav
cdd2df880d Add a “skip_dsn” option to g_part's bootcode verb to prevent g_part_mbr
from setting the volume serial number.  This unbreaks older boot blocks
that don't support serial numbers, and allows boot0cfg to set the serial
number itself if requested by the user.

Submitted by:	lev@, yuripv@
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D17386
2018-11-27 14:58:19 +00:00
Dag-Erling Smørgrav
ceedec4bce A few more cases where strcasecmp() is no longer required.
MFC after:	1 week
2018-11-27 11:22:19 +00:00
Dag-Erling Smørgrav
8d9de5b10a Improve URL parsing. In particular, convert scheme and host to lowercase.
MFC after:	1 week
2018-11-27 10:45:14 +00:00
Dag-Erling Smørgrav
b68815f088 Support proxying FTP over HTTPS, not just HTTP.
There is probably a PR for this, but I can't find this, or remember who
submitted it.  The patch got lost in the noise of another that wasn't
ready to commit.

MFC after:	3 days
2018-11-27 10:06:41 +00:00
Dag-Erling Smørgrav
e86caa0bd0 Style cleanup. 2018-11-27 09:41:47 +00:00
Kirk McKusick
4391a9c0e9 Bring up to date with recently added functions berase(3), getinode(3),
putinode(3), sbget(3), and sbput(3).

Sponsored by: Netflix
2018-11-26 23:09:45 +00:00
Martin Matuska
d28d5e1457 libarchive configuration changes
- move HAVE_BZLIB_H, HAVE_LIBLZMA and HAVE_LZMA_H to config_freebsd.h
- activate support for multi-threaded lzma encoding [1]

PR:		233543 [1]
Reported by:	cem
MFC after:	1 week
2018-11-26 21:45:27 +00:00
Ed Maste
cac83db9b7 revert r340640 "libc: forcibly disable BIND_NOW"
When immediate bind mode is requested, as of r340675 rtld processes
irelocs in PLT immediately after other PLT relocs.  That addresses the
libc + BIND_NOW startup crash the workaround is no longer needed.

PR:		233333
2018-11-26 13:56:19 +00:00
Baptiste Daroussin
6bbd1d19fd libedit: improve multibyte support
Until this commit libedit only supported UTF-8 for multibyte charset
Improve it to support other multibyte charsets

Tested with eucJP and SJIS charsets.
Note that this change as been review and committed in upstream libedit
as well via christos@NetBSD

Submitted by:	naito.yuichiro _at_ gmail.com
Reviewed by:	bapt, pfg, yuripv, 0mp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D17903
2018-11-26 08:16:33 +00:00
Dimitry Andric
c1532e70f8 Merge ^/head r340869 through r340917. 2018-11-25 11:15:01 +00:00
Andrew Turner
266900be14 Re-enable the dynamiclib tests. These should be fixed by r340910. 2018-11-24 18:25:00 +00:00
Andrew Turner
e5c0fd4be6 Add the missing 0 at the end of the .jcr section.
Without this the dynamic library test was failing as it was calling
_Jv_RegisterClasses multiple times.
2018-11-24 18:23:53 +00:00
Andrew Turner
ab1e0d2410 Disable the dynamiclib test until a failure can be debugged 2018-11-24 16:05:40 +00:00
Dimitry Andric
3d5db45595 Merge ^/head r340427 through r340868. 2018-11-24 14:46:06 +00:00
Andrew Turner
e1fa5626cf Mark the function called by the MIPS .init/.fini sequence with .local.
As with r328939 we need to mark local symbols as such. Without this the
assembly parser treats the symbols as global and created relocations
against these private symbols.

MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-11-23 16:33:03 +00:00
Yuri Pankov
63cbe8d1d9 regexec: fix processing multibyte strings.
Matcher function incorrectly assumed that moffset that we get from
findmust is in bytes. Fix this by introducing a stepback function,
taking short path if MB_CUR_MAX is 1, and going back byte-by-byte,
checking if we have a legal character sequence otherwise.

PR:		153502
Reviewed by:	pfg, kevans
Approved by:	kib (mentor, implicit)
Differential revision:	https://reviews.freebsd.org/D18297
2018-11-23 15:49:18 +00:00
Mateusz Piotrowski
f5c8cb4c35 Cross-reference libbe(3) and bectl(8).
Those two manual pages are already referencing each other in the HISTORY
sections, which people might skip. Mention those manual pages explicitly in
the SEE ALSO sections.  Also, remove a reference to be(1) from libbe(3).

Reviewed by:	bcr
Approved by:	krion (mentor, implicit), mat (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D18136
2018-11-21 12:46:28 +00:00
Mateusz Piotrowski
cc58f749fb libbe(3): Put each error value in separate line.
As requested by a TODO in the source code.

Reviewed by:	bcr
Approved by:	krion (mentor, implicit), mat (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D18063
2018-11-21 11:22:44 +00:00