Commit Graph

4625 Commits

Author SHA1 Message Date
Will Andrews
2728d6307d beinstall: try to save progress from pkg updates.
This is primarily aimed at failed updates due to package conflicts, and
affects treatment of failed updates.  Whereas before potentially a large
number of packages would need to be synced for each attempt, they can now
be persisted.  Requires rsync.  There may be better ways to implement this,
e.g. using secondary cache path that is only used on followup attempts and
then wiped on success, which avoids polluting current cache.
2018-12-27 23:27:48 +00:00
Vincenzo Maffione
c2231fb0f8 netmap: update nmreplay(8)
Small modifications to the nmreplay man page.
Used igor and mandoc tools to fix warnings and errors.

Reviewed by:	bcr
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D18629
2018-12-21 14:45:10 +00:00
Vincenzo Maffione
4b45250941 netmap: nmreplay: import various fixes from upstream (2704a51839906)
Changelist:
    - General reformatting
    - Fix packet duplication in cons(). Whenever cons() reached the
      burst limit it would send all pending packets without advancing
      head. This caused the last injected packet to be sent again in
      the next round.
    - Fix full-speed transmissions after first loop.

MFC after:	3 days
2018-12-21 13:56:57 +00:00
Rebecca Cran
db8b561345 Rework UEFI ESP generation
Currently, the installer uses pre-created 800KB FAT12 filesystems that
it dd's onto the ESP partition.
This changeset improves that by having the installer generate a FAT32
filesystem directly onto the ESP using newfs_msdos and then copying
loader.efi into /EFI/freebsd.
For live installs it then runs efibootmgr to add a FreeBSD boot entry
in the BIOS.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D17947
2018-12-20 19:39:37 +00:00
Alan Somers
b1ce931bd8 Conditionalize installtion audit(4) tests on MK_AUDIT
MK_AUDIT already controls auditd(8), praudit(1), etc.  It should also control
the audit test suite.

Submitted by:	ngie
MFC after:	2 weeks
Pull Request:	https://github.com/freebsd/freebsd/pull/240
2018-12-17 18:35:32 +00:00
Alan Somers
314ce98edf OptionalObsoleteFiles: Fix deleting usr/tests/usr.sbin/sa
It's a directory, not a file.

Reported by:	ngie
MFC after:	2 weeks
X-MFC-With:	300938
2018-12-16 23:45:46 +00:00
Alan Somers
738ea87196 Conditionally install /etc/rc.d/audit* based on ${MK_AUDIT}
/usr/sbin/audit(dist)?d are only installed if ${MK_AUDIT} == yes. Their
supporting scripts should only be installed in those instances as well.

Submitted by:	ngie
Reviewed by:	emaste
MFC after:	2 weeks
Pull Request:	https://github.com/freebsd/freebsd/pull/242
2018-12-16 23:38:46 +00:00
Cy Schubert
11783f005c Repair build post-r342139 2018-12-15 22:02:02 +00:00
George V. Neville-Neil
53fc043d41 Remove, the now very outdated, timed.
Submitted by:	Kyle Spiers ksspiers at gmail
Reviewed by:	bcr,brooks,bz,sbruno
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D18505
2018-12-15 21:34:40 +00:00
Stefan Eßer
385e98080c Remove CTM from 13-CURRENT after the release of FreeBSD-12.0.
The removal (and creation of a port) has been pre-announced in UPDATING
1 month ago. Packages are available for all supported FreeBSD vesions.

I did not think that another entry in UPDATING is required to note the
actual removal.

No MFC is planned - CTM shall be kept in base for all releases up to 12.x.

Reviewed by:	rgrimes
Approved by:	imp, bcr (manpages)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D17935
2018-12-15 16:53:15 +00:00
Bjoern A. Zeeb
7984cba7d7 Remove a dead file. CVS was removed in r251794. 2018-12-11 21:16:09 +00:00
Dimitry Andric
67350cb56a Merge ^/head r340918 through r341763. 2018-12-09 11:39:45 +00:00
Vincenzo Maffione
9e53f3bd67 tools: netmap: pkt-gen: check packet length against interface MTU
Validate the value of the -l argument (packet length) against the MTU of the netmap port.
In case the netmap port does not refer to a physical interface (e.g. VALE port or pipe), then
the netmap buffer size is used as MTU.
This change also sets a better default value for the -M option, so that pkt-gen uses
the largest possible fragments in case of multi-slot packets.

Differential Revision:	https://reviews.freebsd.org/D18436
2018-12-08 12:52:09 +00:00
Brooks Davis
1b5dbfa8a0 Remove test for KSE (removed in 2008).
Approved by:	julian
2018-12-04 00:22:08 +00:00
Yuri Pankov
ea22b5bd75 Add ga_IE.UTF-8 locale.
PR:		228587
Submitted by:	Micil <micheal.maclochlainn@oegaillimh.ie> (LC_TIME)
Reviewed by:	bapt
Approved by:	kib (mentor, implicit)
MFC after:	3 days
Differential revision:	https://reviews.freebsd.org/D17997
2018-11-26 19:39:49 +00:00
Ed Maste
eda7d3bec4 Do not install GNU ld if lld is /usr/bin/ld
GNU binutils ld.bfd 2.17.50 does not support ifuncs and produces broken
binaries when ifuncs are in use.  When LLD_IS_LD is default we have an
ifunc-capable system linker and can just avoid installing ld.bfd.

Reported by:	theraven
Reviewed by:	bz
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18340
2018-11-26 17:07:35 +00:00
Dimitry Andric
3d5db45595 Merge ^/head r340427 through r340868. 2018-11-24 14:46:06 +00:00
Ed Maste
0161256ccc Add some non-default src.conf(5) knob descriptions
Some WITH_/WITHOUT_ defaults will likey change in the future (e.g. as we
migrate to copyfree base system components).

Add non-default descriptions for the benefit of WIP branches.
2018-11-21 14:50:45 +00:00
Ben Widawsky
4a88e7dc00 git-svn-init: Add docs to the choice of repos 2018-11-20 23:27:15 +00:00
Alex Richardson
3b7e9bfef4 Add capsicum_helpers.h to -legacy if needed
This fixes bootstrap of capsicumized strings on FreeBSD 11.

Reviewed By:	oshogbo, bdrewery
Differential Revision: https://reviews.freebsd.org/D17971
2018-11-19 18:58:34 +00:00
Yuri Pankov
dbcf594ac9 Use UnicodeData.txt to create UTF-8 ctype map.
This should provide more complete coverage of currently defined Unicode
characters as compared to manually assembled one we use currently.

Comparison of original and new UTF-8 ctype maps by character class:

TYPE    ORIG    NEW
alnum   94229   126029
alpha   93557   125419
blank   4       2
cntrl   73      137685
digit   469     622
graph   109615  137203
lower   1478    2145
print   109641  137222
punct   3428    797
rune    110481  274907
space   33      24
upper   983     1781
xdigit  469     622

Large number of added cntrl definitions is due to the fact that private-use
planes are currently defined as such, this can change in the future.

Discussed with:	bapt
Approved by:	kib (mentor, implicit)
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D17842
2018-11-17 10:36:00 +00:00
Dimitry Andric
6149ed01a1 Merge ^/head r340368 through r340426. 2018-11-14 06:46:44 +00:00
Kirk McKusick
9fc5d538fc In preparation for adding inode check-hashes, clean up and
document the libufs interface for fetching and storing inodes.
The undocumented getino / putino interface has been replaced
with a new getinode / putinode interface.

Convert the utilities that had been using the undocumented
interface to use the new documented interface.

No functional change (as for now the libufs library does not
do inode check-hashes).

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-11-13 21:40:56 +00:00
Dimitry Andric
9b5cb2f639 Merge ^/head r340235 through r340367. 2018-11-12 16:41:41 +00:00
Vincenzo Maffione
80ad548dca netmap: pkt-gen: several updates from upstream
Various improvements to the netmap pkt-gen program:

 - indentation fixes
 - support for IPV6
 - fixes to checksum computation
 - support for NS_MOREFRAG
 - rate limiting in ping mode

Reviewed by:	bcr, 0mp
Approved by:	gnn (mentor)
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D17698
2018-11-10 15:10:22 +00:00
Ed Maste
e8bd280446 Move llvm-profdata build into MK_LLVM_COV block
llvm-profdata is used with llvm-cov for code coverage (although llvm-cov
can also operate independently in a gcov-compatible mode).
Although llvm-profdata can be used independently of llvm-cov it makes
sense to group these under one option.

Also handle these in OptionalObsoleteFiles.inc while here.

Sponsored by:	The FreeBSD Foundation
2018-11-09 18:35:09 +00:00
Ed Maste
9f7331ad2c llvm-cov: also install as gcov (if GNU gcov is disabled)
llvm-cov provides a gcov-compatible interface when invoked as gcov.

Reviewed by:	dim, markj
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17923
2018-11-09 17:43:19 +00:00
Ed Maste
2f02c9d6d5 Consolidate gcov entries in OptionalObsoleteFiles
Sponsored by:	The FreeBSD Foundation
2018-11-09 15:27:41 +00:00
Vincenzo Maffione
689f146bdd netmap: add load balancer program
Add the lb program, which is able to load-balance input traffic
received from a netmap port over M groups, with N netmap pipes in
each group. Each received packet is forwarded to one of the pipes
chosen from each group (using an L3/L4 connection-consistent hash function).
This also adds a man page for lb and some cross-references in related
man pages.

Reviewed by:	bcr, 0mp
Approved by:	gnn (mentor)
Differential Revision:	https://reviews.freebsd.org/D17735
2018-11-09 08:43:40 +00:00
Dimitry Andric
8c1538224a Merge ^/head r340213 through r340234. 2018-11-07 18:54:24 +00:00
Dimitry Andric
c06e7b66a1 Merge ^/head r340126 through r340212. 2018-11-07 18:52:28 +00:00
Edward Tomasz Napierala
768893cdb5 Add benchmarks for lstat(2) and readlink(2).
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-11-07 11:46:12 +00:00
Yuri Pankov
2350a089e7 Cleanup locale tools:
- Simplify the source dir specification, and update README
  appropriately
- Drop the LC (doonly) processing, it's broken, and even if fixed, not
  really useful
- Don't remove the target directories while installing new data as it
  removes Makefile.depend which we don't manage; only rm the files we
  are going to add/replace/delete instead
- Restrict adding bsd.endian.mk to colldef and ctypedef Makefiles, it's
  not needed in other (text-only) categories
- GC unused scripts; they don't seem to be particularly helpful standalone
  as well

Reviewed by:	bapt
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17858
2018-11-06 21:49:50 +00:00
Ed Maste
bf81eb3ca9 Add a WITH_BIND_NOW build knob
The linker's -z now flag sets the DF_BIND_NOW flag, which signals to the
runtime loader that all relocation processing should be performed at
process startup rather than on demand.  In combination with lld's
default of enabling relro this causes the GOT to be made read-only when
the process starts, preventing straightforward GOT overwrite attacks.

Shawn Webb discovered a failure on HardenedBSD with BIND_NOW and ifunc
use, which resulted in my rtld fix in r340137.  Add a BIND_NOW knob as
it is trivial to do so and is a useful ELF hardening feature.  This
change is equivalent to HardenedBSD's but not identical as there are
other diffs/conflicts nearby.

Note that our ELF Tool Chain readelf does not currently decode the
DF_BIND_NOW flag - see PR232983.

Reviewed by:	brooks
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17846
2018-11-06 15:52:49 +00:00
Alex Richardson
b25c717954 Fix -DNO_CLEAN build after r340157
Approved By:	jhb (mentor)
2018-11-05 21:30:00 +00:00
Alex Richardson
8f62bca488 Allow building world without inheriting $PATH
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.

All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.

Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.

Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
2018-11-05 19:51:10 +00:00
Ed Maste
7ae5db1f37 Expand WITH_LLD_BOOTSTRAP description
Reported by:	bz
2018-11-05 15:22:41 +00:00
Yuri Pankov
dd7c41a378 Add hybrid C.UTF-8 locale being identical to default C locale except
that it uses the same ctype maps and functions as other UTF-8 locales.

Reviewed by:	bapt, cem, eadler
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17833
2018-11-04 22:13:22 +00:00
Conrad Meyer
d83db3fb6a Drop ed(1) "crypto"
You should not be using DES.  You should not have been using DES for the
past 30 years.

The ed DES-CBC scheme lacked several desirable properties of a sealed
document system, even ignoring DES itself.  In particular, it did not
provide the "integrity" cryptographic property (detection of tampering), and
it treated ASCII passwords as 64-bit keys (instead of using a KDF like
scrypt or PBKDF2).

Some general approaches ed(1) users might consider to replace the removed
DES mode:

1. Full disk encryption with something like AES-XTS.  This is easy to
conceptualize, design, and implement, and it provides confidentiality for
data at rest.  Like CBC, it lacks tampering protection.  Examples include
GELI, LUKS, FileVault2.

2. Encrypted overlay ("stackable") filesystems (EncFS, PEFS?, CryptoFS,
others).

3. Native encryption at the filesystem layer.  Ext4/F2FS, ZFS, APFS, and
NTFS all have some flavor of this.

4. Storing your files unencrypted.  It's not like DES was doing you much
good.

If you have DES-CBC scrambled files produced by ed(1) prior to this change,
you may decrypt them with:

  openssl des-cbc -d -iv 0 -K <key in hex> -in <inputfile> -out <plaintext>

Reviewed by:	allanjude, bapt, emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D17829
2018-11-04 17:56:16 +00:00
Dimitry Andric
2a22df74e9 Merge ^/head r339813 through r340125. 2018-11-04 15:49:06 +00:00
Dimitry Andric
689486003b Merge llvm, clang, lld, lldb, compiler-rt and libc++ release_70 branch
r346007 (effectively 7.0.1 rc2), resolve conflicts, and bump version
numbers.

PR:		230240, 230355
2018-11-04 15:46:30 +00:00
Dimitry Andric
86392292ee Vendor import of llvm release_70 branch r346007:
https://llvm.org/svn/llvm-project/llvm/branches/release_70@346007
2018-11-04 14:41:47 +00:00
Yuri Pankov
1ade1c4113 Update to CLDR 34 and UNICODE 11.
Discussed with:	bapt
Approved by:	kib (mentor)
MFC after:	1 month
2018-11-03 22:02:10 +00:00
Ed Maste
4db78cacde Remove apparently unused 0-byte files that cause grief on Windows
r235274 added a sort regression test (it operates by comparing output
against GNU sort).  The commit included a number of 0-byte files, one
of which ends in a trailing . which reportedly breaks svn/git checkouts
on Windows.

It appears these were added accidentally, so just remove them.

PR:		232479
MFC after:	1 month
2018-11-03 01:53:26 +00:00
John Baldwin
921540789b Compile cryptocheck with in-tree openssl again.
Reviewed by:	cem
MFC after:	2 months
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D17767
2018-11-01 21:25:26 +00:00
Bjoern A. Zeeb
43f75d57a2 Introduce an EXPERIMENTAL option for both src.conf(5) and the kernel.
In the last decade(s) we have seen both short term or long term projects
committed to the tree which were considered or even marked "experimental".
While out-of-tree development has become easier than it used to be in
CVS times, there still is a need to have the code shipping with HEAD but
not enabled by default.

While people may think about VIMAGE as one of the recent larger, long term
projects, early protocol implementations (before they are standardised)
are others.  (Free)BSD historically was one of the operating systems
which would have running code at early stages and help develop and
influence standardisation and the industry.

Give developers an opportunity to be more pro-active for early adoption
or running large scale code changes stumbling over each others but not
the user's feet.  I have not added the option to NOTES in order to avoid
breaking supported option builds, which require constant compile testing.

Discussed with:	people in the corridor
2018-10-30 15:46:30 +00:00
Dimitry Andric
fda9adafa3 Merge ^/head r339670 through r339812. 2018-10-27 14:44:39 +00:00
Warner Losh
24b6d87155 Redo r339563: Remove joy(4) driver.
This driver was marked as gone in 12. We're at 13 now. Remove it.
Data from nycbug's dmesg cache shows only one potential user,
suggesting it never was used much. However, even though this device
has been obsolete for 15 years at least, sys/joystick.h is included in
a number of graphics packages still, so that remains. A full exprun
is needed before that can be removed.

RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D17629
2018-10-26 16:03:30 +00:00
Warner Losh
7c320a22df Revert r339563.
I held the mistaken belief this was completely unused. While the
driver is unused and likely not relevant for a long time,
sys/joystick.h lives on in maybe half a dozen ports, even though
hardware to use it hasn't been widely used in maybe 15 years.
2018-10-26 04:10:32 +00:00
Andrew Turner
31d62a73c2 Implement a BSD licensed crtbegin/crtend
These are needed for .ctors/.dtors and .jcr handling. The former needs
all the function pointers to be called in the correct order from the
.init/.fini section. The latter just needs to call a gcj specific function
if it exists with a pointer to the start of the .jcr section.

This is currently disabled until __dso_handle support is added.

Reviewed by:	emaste
MFC after:	1 month
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17587
2018-10-25 17:39:41 +00:00