Commit Graph

4722 Commits

Author SHA1 Message Date
gnn
222efc509d Remove, the now very outdated, timed.
Submitted by:	Kyle Spiers ksspiers at gmail
Reviewed by:	bcr,brooks,bz,sbruno
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D18505
2018-12-15 21:34:40 +00:00
se
6126d79b84 Remove CTM from 13-CURRENT after the release of FreeBSD-12.0.
The removal (and creation of a port) has been pre-announced in UPDATING
1 month ago. Packages are available for all supported FreeBSD vesions.

I did not think that another entry in UPDATING is required to note the
actual removal.

No MFC is planned - CTM shall be kept in base for all releases up to 12.x.

Reviewed by:	rgrimes
Approved by:	imp, bcr (manpages)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D17935
2018-12-15 16:53:15 +00:00
bz
40f5fbc906 Remove a dead file. CVS was removed in r251794. 2018-12-11 21:16:09 +00:00
dim
07b9c9ba27 Merge ^/head r340918 through r341763. 2018-12-09 11:39:45 +00:00
vmaffione
43ef1e7712 tools: netmap: pkt-gen: check packet length against interface MTU
Validate the value of the -l argument (packet length) against the MTU of the netmap port.
In case the netmap port does not refer to a physical interface (e.g. VALE port or pipe), then
the netmap buffer size is used as MTU.
This change also sets a better default value for the -M option, so that pkt-gen uses
the largest possible fragments in case of multi-slot packets.

Differential Revision:	https://reviews.freebsd.org/D18436
2018-12-08 12:52:09 +00:00
brooks
060a1f4e9d Remove test for KSE (removed in 2008).
Approved by:	julian
2018-12-04 00:22:08 +00:00
yuripv
8e370b212f Add ga_IE.UTF-8 locale.
PR:		228587
Submitted by:	Micil <micheal.maclochlainn@oegaillimh.ie> (LC_TIME)
Reviewed by:	bapt
Approved by:	kib (mentor, implicit)
MFC after:	3 days
Differential revision:	https://reviews.freebsd.org/D17997
2018-11-26 19:39:49 +00:00
emaste
52f4b8d04a Do not install GNU ld if lld is /usr/bin/ld
GNU binutils ld.bfd 2.17.50 does not support ifuncs and produces broken
binaries when ifuncs are in use.  When LLD_IS_LD is default we have an
ifunc-capable system linker and can just avoid installing ld.bfd.

Reported by:	theraven
Reviewed by:	bz
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18340
2018-11-26 17:07:35 +00:00
dim
3aa6bcc5fe Merge ^/head r340427 through r340868. 2018-11-24 14:46:06 +00:00
emaste
6dc34b206c Add some non-default src.conf(5) knob descriptions
Some WITH_/WITHOUT_ defaults will likey change in the future (e.g. as we
migrate to copyfree base system components).

Add non-default descriptions for the benefit of WIP branches.
2018-11-21 14:50:45 +00:00
bwidawsk
f9c018a835 git-svn-init: Add docs to the choice of repos 2018-11-20 23:27:15 +00:00
arichardson
bbcf733800 Add capsicum_helpers.h to -legacy if needed
This fixes bootstrap of capsicumized strings on FreeBSD 11.

Reviewed By:	oshogbo, bdrewery
Differential Revision: https://reviews.freebsd.org/D17971
2018-11-19 18:58:34 +00:00
yuripv
c6e4d24106 Use UnicodeData.txt to create UTF-8 ctype map.
This should provide more complete coverage of currently defined Unicode
characters as compared to manually assembled one we use currently.

Comparison of original and new UTF-8 ctype maps by character class:

TYPE    ORIG    NEW
alnum   94229   126029
alpha   93557   125419
blank   4       2
cntrl   73      137685
digit   469     622
graph   109615  137203
lower   1478    2145
print   109641  137222
punct   3428    797
rune    110481  274907
space   33      24
upper   983     1781
xdigit  469     622

Large number of added cntrl definitions is due to the fact that private-use
planes are currently defined as such, this can change in the future.

Discussed with:	bapt
Approved by:	kib (mentor, implicit)
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D17842
2018-11-17 10:36:00 +00:00
dim
b4fff6918d Merge ^/head r340368 through r340426. 2018-11-14 06:46:44 +00:00
mckusick
80199cdcb4 In preparation for adding inode check-hashes, clean up and
document the libufs interface for fetching and storing inodes.
The undocumented getino / putino interface has been replaced
with a new getinode / putinode interface.

Convert the utilities that had been using the undocumented
interface to use the new documented interface.

No functional change (as for now the libufs library does not
do inode check-hashes).

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-11-13 21:40:56 +00:00
dim
0838f433ff Merge ^/head r340235 through r340367. 2018-11-12 16:41:41 +00:00
vmaffione
04d6605ef4 netmap: pkt-gen: several updates from upstream
Various improvements to the netmap pkt-gen program:

 - indentation fixes
 - support for IPV6
 - fixes to checksum computation
 - support for NS_MOREFRAG
 - rate limiting in ping mode

Reviewed by:	bcr, 0mp
Approved by:	gnn (mentor)
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D17698
2018-11-10 15:10:22 +00:00
emaste
9ee3eadfc8 Move llvm-profdata build into MK_LLVM_COV block
llvm-profdata is used with llvm-cov for code coverage (although llvm-cov
can also operate independently in a gcov-compatible mode).
Although llvm-profdata can be used independently of llvm-cov it makes
sense to group these under one option.

Also handle these in OptionalObsoleteFiles.inc while here.

Sponsored by:	The FreeBSD Foundation
2018-11-09 18:35:09 +00:00
emaste
d4894c17be llvm-cov: also install as gcov (if GNU gcov is disabled)
llvm-cov provides a gcov-compatible interface when invoked as gcov.

Reviewed by:	dim, markj
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17923
2018-11-09 17:43:19 +00:00
emaste
8d29055105 Consolidate gcov entries in OptionalObsoleteFiles
Sponsored by:	The FreeBSD Foundation
2018-11-09 15:27:41 +00:00
vmaffione
93a7ff50c0 netmap: add load balancer program
Add the lb program, which is able to load-balance input traffic
received from a netmap port over M groups, with N netmap pipes in
each group. Each received packet is forwarded to one of the pipes
chosen from each group (using an L3/L4 connection-consistent hash function).
This also adds a man page for lb and some cross-references in related
man pages.

Reviewed by:	bcr, 0mp
Approved by:	gnn (mentor)
Differential Revision:	https://reviews.freebsd.org/D17735
2018-11-09 08:43:40 +00:00
dim
07eb34a317 Merge ^/head r340213 through r340234. 2018-11-07 18:54:24 +00:00
dim
b45b4d8aa3 Merge ^/head r340126 through r340212. 2018-11-07 18:52:28 +00:00
trasz
78c7b0dbc0 Add benchmarks for lstat(2) and readlink(2).
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-11-07 11:46:12 +00:00
yuripv
96845afdec Cleanup locale tools:
- Simplify the source dir specification, and update README
  appropriately
- Drop the LC (doonly) processing, it's broken, and even if fixed, not
  really useful
- Don't remove the target directories while installing new data as it
  removes Makefile.depend which we don't manage; only rm the files we
  are going to add/replace/delete instead
- Restrict adding bsd.endian.mk to colldef and ctypedef Makefiles, it's
  not needed in other (text-only) categories
- GC unused scripts; they don't seem to be particularly helpful standalone
  as well

Reviewed by:	bapt
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17858
2018-11-06 21:49:50 +00:00
emaste
7b4e55f191 Add a WITH_BIND_NOW build knob
The linker's -z now flag sets the DF_BIND_NOW flag, which signals to the
runtime loader that all relocation processing should be performed at
process startup rather than on demand.  In combination with lld's
default of enabling relro this causes the GOT to be made read-only when
the process starts, preventing straightforward GOT overwrite attacks.

Shawn Webb discovered a failure on HardenedBSD with BIND_NOW and ifunc
use, which resulted in my rtld fix in r340137.  Add a BIND_NOW knob as
it is trivial to do so and is a useful ELF hardening feature.  This
change is equivalent to HardenedBSD's but not identical as there are
other diffs/conflicts nearby.

Note that our ELF Tool Chain readelf does not currently decode the
DF_BIND_NOW flag - see PR232983.

Reviewed by:	brooks
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17846
2018-11-06 15:52:49 +00:00
arichardson
2a1d6c1ec0 Fix -DNO_CLEAN build after r340157
Approved By:	jhb (mentor)
2018-11-05 21:30:00 +00:00
arichardson
fd5d87df03 Allow building world without inheriting $PATH
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.

All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.

Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.

Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
2018-11-05 19:51:10 +00:00
emaste
c35aa5d4d8 Expand WITH_LLD_BOOTSTRAP description
Reported by:	bz
2018-11-05 15:22:41 +00:00
yuripv
b6fca3ee80 Add hybrid C.UTF-8 locale being identical to default C locale except
that it uses the same ctype maps and functions as other UTF-8 locales.

Reviewed by:	bapt, cem, eadler
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17833
2018-11-04 22:13:22 +00:00
cem
9278c98eeb Drop ed(1) "crypto"
You should not be using DES.  You should not have been using DES for the
past 30 years.

The ed DES-CBC scheme lacked several desirable properties of a sealed
document system, even ignoring DES itself.  In particular, it did not
provide the "integrity" cryptographic property (detection of tampering), and
it treated ASCII passwords as 64-bit keys (instead of using a KDF like
scrypt or PBKDF2).

Some general approaches ed(1) users might consider to replace the removed
DES mode:

1. Full disk encryption with something like AES-XTS.  This is easy to
conceptualize, design, and implement, and it provides confidentiality for
data at rest.  Like CBC, it lacks tampering protection.  Examples include
GELI, LUKS, FileVault2.

2. Encrypted overlay ("stackable") filesystems (EncFS, PEFS?, CryptoFS,
others).

3. Native encryption at the filesystem layer.  Ext4/F2FS, ZFS, APFS, and
NTFS all have some flavor of this.

4. Storing your files unencrypted.  It's not like DES was doing you much
good.

If you have DES-CBC scrambled files produced by ed(1) prior to this change,
you may decrypt them with:

  openssl des-cbc -d -iv 0 -K <key in hex> -in <inputfile> -out <plaintext>

Reviewed by:	allanjude, bapt, emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D17829
2018-11-04 17:56:16 +00:00
dim
4b4bc3c457 Merge ^/head r339813 through r340125. 2018-11-04 15:49:06 +00:00
dim
3d9ebb9be0 Merge llvm, clang, lld, lldb, compiler-rt and libc++ release_70 branch
r346007 (effectively 7.0.1 rc2), resolve conflicts, and bump version
numbers.

PR:		230240, 230355
2018-11-04 15:46:30 +00:00
yuripv
85d52a2654 Update to CLDR 34 and UNICODE 11.
Discussed with:	bapt
Approved by:	kib (mentor)
MFC after:	1 month
2018-11-03 22:02:10 +00:00
emaste
c7c12cac6e Remove apparently unused 0-byte files that cause grief on Windows
r235274 added a sort regression test (it operates by comparing output
against GNU sort).  The commit included a number of 0-byte files, one
of which ends in a trailing . which reportedly breaks svn/git checkouts
on Windows.

It appears these were added accidentally, so just remove them.

PR:		232479
MFC after:	1 month
2018-11-03 01:53:26 +00:00
jhb
c9ddd1f763 Compile cryptocheck with in-tree openssl again.
Reviewed by:	cem
MFC after:	2 months
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D17767
2018-11-01 21:25:26 +00:00
bz
11c8430e5c Introduce an EXPERIMENTAL option for both src.conf(5) and the kernel.
In the last decade(s) we have seen both short term or long term projects
committed to the tree which were considered or even marked "experimental".
While out-of-tree development has become easier than it used to be in
CVS times, there still is a need to have the code shipping with HEAD but
not enabled by default.

While people may think about VIMAGE as one of the recent larger, long term
projects, early protocol implementations (before they are standardised)
are others.  (Free)BSD historically was one of the operating systems
which would have running code at early stages and help develop and
influence standardisation and the industry.

Give developers an opportunity to be more pro-active for early adoption
or running large scale code changes stumbling over each others but not
the user's feet.  I have not added the option to NOTES in order to avoid
breaking supported option builds, which require constant compile testing.

Discussed with:	people in the corridor
2018-10-30 15:46:30 +00:00
dim
10311b69d6 Merge ^/head r339670 through r339812. 2018-10-27 14:44:39 +00:00
imp
4f1c28c8e1 Redo r339563: Remove joy(4) driver.
This driver was marked as gone in 12. We're at 13 now. Remove it.
Data from nycbug's dmesg cache shows only one potential user,
suggesting it never was used much. However, even though this device
has been obsolete for 15 years at least, sys/joystick.h is included in
a number of graphics packages still, so that remains. A full exprun
is needed before that can be removed.

RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D17629
2018-10-26 16:03:30 +00:00
imp
83bf1fdc83 Revert r339563.
I held the mistaken belief this was completely unused. While the
driver is unused and likely not relevant for a long time,
sys/joystick.h lives on in maybe half a dozen ports, even though
hardware to use it hasn't been widely used in maybe 15 years.
2018-10-26 04:10:32 +00:00
andrew
5ca9ee52a8 Implement a BSD licensed crtbegin/crtend
These are needed for .ctors/.dtors and .jcr handling. The former needs
all the function pointers to be called in the correct order from the
.init/.fini section. The latter just needs to call a gcj specific function
if it exists with a pointer to the start of the .jcr section.

This is currently disabled until __dso_handle support is added.

Reviewed by:	emaste
MFC after:	1 month
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17587
2018-10-25 17:39:41 +00:00
kevans
4d21c09b0e Update lualoader test script a little bit
Use userboot.so from the test directory if possible, fall back to .OBJDIR.
This avoids a problem that we've had since userboot coexistence was added,
where userboot.so alone no longer exists in the .OBJDIR but is instead just
a link installed later.
2018-10-25 02:04:01 +00:00
vmaffione
fb28719dd4 netmap: add man page for the vale-ctl program
Added man page for vale-ctl program.
Small fixes to vale-ctl, including the support for -m option
(to specify the netmap memory allocator id).

Reviewed by:	0mp
Approved by:	gnn (mentor)
Differential Revision:	https://reviews.freebsd.org/D17683
2018-10-24 16:01:56 +00:00
dim
6c8ec16fbe Merge ^/head r339015 through r339669. 2018-10-23 21:09:37 +00:00
vmaffione
c41c1dd360 netmap: add man page for the bridge program
Added bridge(8).
Also, minor fixes to the netmap "bridge" application:
 - indentation fixes and code cleanup
 - better usage description
 - better processing of netmap flags

Reviewed by:	0mp
Approved by:	gnn (mentor)
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D17664
2018-10-23 15:34:43 +00:00
imp
e4026c4f30 Remove mse(4) from tree
Remove mse and all support for bus and inport devices from the tree.
Data from nycbug's dmesg database shows the last sighting of this
driver was in 4.10 on only one machine.

Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D17628
2018-10-22 02:34:10 +00:00
imp
ba47cdd36c Remove joy(4) driver.
This driver was marked as gone in 12. We're at 13 now. Remove it.
Data from nycbug's dmesg cache shows only one potential user,
suggesting it never was used much.

RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D17629
2018-10-22 02:34:00 +00:00
imp
ae8394a166 Add missing options.
WITHOUT_LOADER_LUA is only needed since we turned it off by default on
powerpc and sparc64 in r338203. Same with
WITHOUT_LOADER_GEIL. WITH_NVME, WITHOUT_NVME, WITH_LOADER_FORCE_LE
have been needed since they were added.
2018-10-21 06:33:35 +00:00
emaste
a2e03c847c Introduce src.conf knob to build userland with retpoline
WITH_RETPOLINE enables -mretpoline vulnerability mitigation in userland
for CVE-2017-5715.

Reported by:	Peter Malcom
Reviewed by:	markj
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17421
2018-10-21 00:27:59 +00:00
yuripv
d641b0caff Add -b/-l options to localedef(1) to specify output endianness and use
it appropriately when building share/ctypedef and share/colldef.

This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
build and in the native EB build match.  Revert the changes done to libc
in r308170 as they are no longer needed.

PR:		231965
Reviewed by:	bapt, emaste, sbruno, 0mp
Approved by:	kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D17603
2018-10-20 20:51:05 +00:00
yuripv
a7a80f58ca Restore some of the ctype definitions reported in the PR from pre-CLDR
data, namely 0xE000-0xF8FF private use area, and 0xFF00-0xFFF half- and
fullwidth punctuation.

While here, update tools/tools/locale/README based on my experience
rebuilding the locale data.

PR:		225692
Reviewed by:	bapt, cem (previous version)
Approved by:	re (gjb), kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D17471
2018-10-11 18:30:12 +00:00
dim
3e583736c9 Merge ^/head r338731 through r338987. 2018-09-27 20:00:07 +00:00
emaste
a1844f4acd Do not remove ld man page if lld is enabled and binutils is not
Reported by:	Mark Millard
Approved by:	re (gjb)
Sponsored by:	The FreeBSD Foundation
2018-09-25 18:54:18 +00:00
brooks
bce0771d86 Only enable HyperV support by default on x86.
Without this we get spurious output during boot as we try to run
nonexistant HyperV scripts on non-x86 models.

Reviewed by:	kib
Approved by:	re (gjb)
Differential Revision:	https://reviews.freebsd.org/D17211
2018-09-18 16:06:24 +00:00
dim
8244f4a107 Merge ^/head r338595 through r338689, and resolve conflicts. 2018-09-14 19:50:36 +00:00
emaste
0d0ccdbf0d Add WITHOUT_REPRODUCIBLE_BUILD description
Approved by:	re (gjb)
Sponsored by:	The FreeBSD Foundation
2018-09-13 14:53:42 +00:00
dim
1116024d04 Merge ^/head r338392 through r338594. 2018-09-11 18:41:00 +00:00
emaste
6ec1bc7064 Remove strings from OptionalObsoleteFiles.inc
Since r326030 strings is installed unconditionally so should not be
removed when WITHOUT_TOOLCHAIN is set.

Reported by:	Dan McGregor
Approved by:	re (kib)
Sponsored by:	The FreeBSD Foundation
2018-09-07 23:12:17 +00:00
rgrimes
a9dd919a3f Increase size of working imgfile from 250MB to 4GB
Submitted by:	dexter
Approved by:	re(gjb), phk (mentor)
MFC after:	3 days
2018-09-07 15:52:20 +00:00
rgrimes
d32e021025 Remove rsync to phk's server
Submitted by:	dexter
Approved by:	re(gjb), phk (mentor)
MFC after:	3 days
2018-09-07 15:48:01 +00:00
dim
05bc8dc0eb Merge ^/head r338298 through r338391. 2018-08-29 20:54:56 +00:00
imp
55f80ecc5b Put building of drm and drm2 modules behind options.
Make the building of drm dependent on MK_MODULE_DRM and the building
of module drm2 on MK_MODULE_DRM2. The defaults are unchanged.

Approved by: re@ (gjb)
Differential Review: https://reviews.freebsd.org/D16894
2018-08-28 14:46:49 +00:00
kevans
ae68fe8b6e Fix bsdbox build WITH_OFED
hostapd requires libpcap, which links against libmlx5 and libibverbs when
building WITH_OFED. These were not pulled in to bsdbox and most bsdbox
builds were WITHOUT_OFED up until recently, so it was not noticed.

Approved by:	re (gjb)
2018-08-27 19:34:50 +00:00
markm
d8723e8b03 Remove the Yarrow PRNG algorithm option in accordance with due notice
given in random(4).

This includes updating of the relevant man pages, and no-longer-used
harvesting parameters.

Ensure that the pseudo-unit-test still does something useful, now also
with the "other" algorithm instead of Yarrow.

PR:		230870
Reviewed by:	cem
Approved by:	so(delphij,gtetlow)
Approved by:	re(marius)
Differential Revision:	https://reviews.freebsd.org/D16898
2018-08-26 12:51:46 +00:00
kevans
51650c5969 libbe(3)/bectl(8): Make consistent with beadm
vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
   missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
   the active BE can't be renamed. It doesn't make sense in our context to
   *not* use -u, so use it.

Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.

A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.

Approved by:	re (kib)
2018-08-24 20:44:58 +00:00
dim
317f4cf38a Merge ^/head r338026 through r338297, and resolve conflicts. 2018-08-24 18:09:23 +00:00
imp
ae526637c4 Revert drm2 removal.
Revert r338177, r338176, r338175, r338174, r338172

After long consultations with re@, core members and mmacy, revert
these changes. Followup changes will be made to mark them as
deprecated and prent a message about where to find the up-to-date
driver.  Followup commits will be made to make this clear in the
installer. Followup commits to reduce POLA in ways we're still
exploring.

It's anticipated that after the freeze, this will be removed in
13-current (with the residual of the drm2 code copied to
sys/arm/dev/drm2 for the TEGRA port's use w/o the intel or
radeon drivers).

Due to the impending freeze, there was no formal core vote for
this. I've been talking to different core members all day, as well as
Matt Macey and Glen Barber. Nobody is completely happy, all are
grudgingly going along with this. Work is in progress to mitigate
the negative effects as much as possible.

Requested by: re@ (gjb, rgrimes)
2018-08-24 00:02:00 +00:00
marius
9f1edbb1c3 Following r335259, don't copy boot1 from the running system for sparc64
either.
2018-08-23 22:57:42 +00:00
kevans
424f5df483 Remove hyper-v leftovers when WITHOUT_HYPERV is set
hv_vss_daemon was missed.

Submitted by:	Oliver Pinter
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D16811
2018-08-23 18:06:31 +00:00
kevans
79b73c529b Remove ZFS leftovers when WITHOUT_ZFS is set
Submitted by:	Oliver Pinter
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D16810
2018-08-23 02:14:39 +00:00
mmacy
0878bfd030 r338172 followup - removal of stray drm references 2018-08-22 03:04:43 +00:00
mmacy
786caced2e r338172 follow - remove firmwares 2018-08-22 02:53:35 +00:00
arichardson
7d8e282839 Relax the check added in 338096
Checking for any include below ${SRCTOP}/sys is too strict and breaks
e.g. mkimg which includes sys/sys/disk. ABI issues will only be caused
by including headers in sys/sys since they might not match the host.

Approved By:	jhb (mentor)
Suggested By:	imp
2018-08-21 16:52:14 +00:00
arichardson
31faac20c0 Fix -DWITHOUT_AUTO_OBJ build of rescue after r338096
Approved By:	jhb (mentor)
2018-08-21 09:35:56 +00:00
mckusick
94e9fbbd05 Fix incorrect output when printing block lists for files small enough
to fit in only direct blocks whose size is exactly a multiple of the
filesystem block size.

Reported by:  Peter Holm
Tested by:    Peter Holm
Sponsored by: Netflix
2018-08-20 20:44:11 +00:00
arichardson
1d83ea4474 Don't create directories in ${WORLDTMP}/legacy with mtree
This has two advantages:
1) We no longer create lots of empty directories that are not needed
2) This is a requirement for building on non-FreeBSD hosts since mtree will
only exist after the bootstrap-tools phase there.

Aproved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D16773
2018-08-20 10:39:53 +00:00
arichardson
8030fe55e3 Avoid depending on system headers from the source tree during bootstrap
This can cause surprising errors if the build tools is built against
headers that don't match the host system. It is also required in order
to allow building on non-FreeBSD systems where the headers in
/usr/include/sys are usually completely incompatible with those in the
source tree.

I added an error to Makefile.boot if this is done and found this was
only the case in libnv. With this error in the Makefile ABI breakages
such as r336019 should no longer be possible.

Reviewed By:	bdrewery, kevans
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D16186
2018-08-20 10:39:42 +00:00
dim
0906e40a51 Merge ^/head r337646 through r338014. 2018-08-18 12:15:54 +00:00
imp
beac22c3b6 Specify DB_FROM_SRC=yes when doing any installation target.
We want to use the UIDs from the installed system, not the host
system, when installing things.

Sponsored by: Netflix
2018-08-16 22:13:43 +00:00
dim
baaa59a288 Merge ^/head r337286 through r337585. 2018-08-10 21:02:28 +00:00
phk
3cc7e39010 Make distribution now happens from top of source tree.
Silence debugging output
2018-08-10 06:28:25 +00:00
dim
020056252b Add optional LLVM BPF target support
BPF (eBPF) is an independent instruction set architecture which is
introduced in Linux a few years ago. Originally, eBPF execute
environment was only inside Linux kernel. However, recent years there
are some user space implementation (https://github.com/iovisor/ubpf,
https://doc.dpdk.org/guides/prog_guide/bpf_lib.html) and kernel space
implementation for FreeBSD is going on
(https://github.com/YutaroHayakawa/generic-ebpf).

The BPF target support can be enabled using WITH_LLVM_TARGET_BPF, as it
is not built by default.

Submitted by:	Yutaro Hayakawa <yhayakawa3720@gmail.com>
Reviewed by:	dim, bdrewery
Differential Revision: https://reviews.freebsd.org/D16033
2018-08-09 21:28:31 +00:00
asomers
dae9290985 Switch the default pager for most commands to less
Finally, a pager for the nineties.

MFC after:	Never
Relnotes:	Yes
Differential Revision:	https://reviews.freebsd.org/D13465
Poll:		https://reviews.freebsd.org/V7
2018-08-08 19:24:20 +00:00
bwidawsk
beb0f2d23d git-svn-init: Clarify branch creation commands
Suggested by:	eadler
Reviewed by:	eadler
Approved by:	emaste (mentor)
Differential Revision:	https://reviews.freebsd.org/D16593
2018-08-06 01:08:43 +00:00
dim
39295a824e Merge ^/head r336870 through r337285, and resolve conflicts. 2018-08-04 11:53:41 +00:00
dim
a6287287d4 Update lib/libclang_rt for compiler-rt trunk r338150. 2018-08-03 15:30:32 +00:00
bwidawsk
65623ffcf1 tools: Add a git-svn bootstrap script
codified version of https://wiki.freebsd.org/GitWorkflow/GitSvn#Using_git-svn

Approved by:	emaste (mentor)
Suggested by:	Warner Losh (imp)
Differential Revision:  https://reviews.freebsd.org/D16528
2018-08-02 18:28:02 +00:00
dim
2a7ef54bf4 Update libc++ and libc++experimental Makefiles, add libc++fs Makefile,
and update OptionalObsoleteFiles.inc for new additions.
2018-08-02 18:24:03 +00:00
dim
8d520fd045 Update build blue to make WITH_CLANG_EXTRAS and WITH_CLANG_FULL work. 2018-08-01 20:46:06 +00:00
dim
46fae36835 Update OptionalObsoleteFiles.inc for new clang intrinsics headers. 2018-07-31 21:09:32 +00:00
dim
5fa2d7e5b2 Update llvm/clang version numbers in various files. 2018-07-31 18:13:44 +00:00
asomers
b3776cb8de Make timespecadd(3) and friends public
The timespecadd(3) family of macros were imported from NetBSD back in
r35029. However, they were initially guarded by #ifdef _KERNEL. In the
meantime, we have grown at least 28 syscalls that use timespecs in some
way, leading many programs both inside and outside of the base system to
redefine those macros. It's better just to make the definitions public.

Our kernel currently defines two-argument versions of timespecadd and
timespecsub.  NetBSD, OpenBSD, and FreeDesktop.org's libbsd, however, define
three-argument versions.  Solaris also defines a three-argument version, but
only in its kernel.  This revision changes our definition to match the
common three-argument version.

Bump _FreeBSD_version due to the breaking KPI change.

Discussed with:	cem, jilles, ian, bde
Differential Revision:	https://reviews.freebsd.org/D14725
2018-07-30 15:46:40 +00:00
will
02808a1ffa beinstall: perform pre-installworld steps.
Since all post-installkernel steps are assumed to operate in the updated
installation, it's necessary to chroot all of the followup steps in the new
boot environment.  Set up and mount the source and object directories at the
same paths inside the BE root, and clean up to the extent changes were made.
This commit fixes upgrading using beinstall past the new ntpd user change.

Improve testability of changes to this script while I'm here.

Reported by:	rpokala (earlier patch)
2018-07-29 01:44:26 +00:00
eadler
cdcdf90685 tools/build/beinstall.sh: Use some slightly better shell syntax; reduce duplication
This is is a first pass at improving the be installer.

Reviewed By: will
Differential Revision: https://reviews.freebsd.org/D16081
2018-07-28 00:33:40 +00:00
imp
4f0eddfd84 also set multicons for tests 2018-07-27 21:50:10 +00:00
kib
8e3e425dcb Move mostly useless examples binaries from OFED, as well as the Subnet
Manager, under the new option WITH_OFED_EXTRA, disabled by default.

Reviewed by:	bdrewery, hselasky (previous version)
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D16230
2018-07-20 23:52:11 +00:00
imp
f37de65a68 Remove three stray instances of zfsloader. 2018-07-20 05:26:14 +00:00
cem
68ad9dcc43 cryptocheck: Correct mismatch between OpenSSL use and OCF Blake2
This corrects a mistake introduced to the cryptocheck tool in r331418.

Our CRYPTO_BLAKE2B and CRYPTO_BLAKE2S algorithms refer to either the plain,
unkeyed hashes (specified with cri_klen = 0), or a Blake2-specific keyed MAC
(when a cri_key is provided).

In contrast, OpenSSL's Blake2 algorithms only provide the plain hash.
Cryptocheck's T_HMAC corresponds to OpenSSL's HMAC() routine, which is the
ordinary HMAC construction applied to any plain, unkeyed hash.  We don't
have any HMAC-Blake2 cipher modes in OCF, so fix the test to only test
Blake2 as a plain hash.

(Ideally we would test keyed Blake2 as well, but that is left as future
work.)

PR:		229795
2018-07-20 01:51:05 +00:00
asomers
6228520fc6 Fix the build of tools/regression/sockets/unix_cmsg
MFC after:	2 weeks
2018-07-19 00:22:18 +00:00
arichardson
69ce0c26e7 Fix buildworld on FreeBSD 10
Since r336126 we depend on explicit_bzero() for the libmd
bootstrap. Add it to -legacy if it is not found in /usr/include/strings.h.

Reviewed By:	ian
Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16245
2018-07-16 11:03:05 +00:00
arichardson
aafd1cf423 No longer install sys/nv.h and sys/cnv.h in lib/libnv/Makefile
Use tools/build/Makefile to install the headers into ${WORLDTMP}/legacy
instead. Compared to r336026 this has the minor advantage that it avoids
unncessary header installation when building the non-bootstrap libnv.

Reviewed By:	bdrewery, kevans
Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16187
2018-07-16 10:57:26 +00:00
ian
5c2f1c9aef Make generated scripts executable. 2018-07-13 16:37:42 +00:00
ian
4bac36e501 More little fixes... fix a function name typo (eps vs esp), and cope with
newer versions of gpart that show mbr efi partition types with the name
'efi' rather than as '!239'.
2018-07-13 16:33:21 +00:00
ian
6eba8a1886 Revert r336240, which contained unrelated changes accidentally committed. 2018-07-13 16:32:31 +00:00
ian
41fc000bdf More little fixes... fix a function name typo (eps vs esp), and cope with
newer versions of gpart that show mbr efi partition types with the name
'efi' rather than as '!239'.
2018-07-13 16:30:54 +00:00
ian
feb71042ff Wee fix: $dev is passed in to the function with the /dev prefix already in
place, don't add it.
2018-07-12 17:22:25 +00:00
imp
cf3c1524b3 Small refactor of install-boot
Use newfs_msdos to create the filesystem for the partition requested
and use loader.efi instead of boot1.efi.
2018-07-11 21:53:04 +00:00
imp
cc4c060dea Remove gateworks nanobsd support.
A quick test of this shows multiple problems. Rather than fix the
problems, just retire this board's support. It's for a 12 year old
board that's been out of production for at least 7 years and generally
lacks the memory to run even a stripped down NanoBSD image well. It's
not really relevant anymore.
2018-07-09 18:02:35 +00:00
imp
3b95f93142 Remove armeb from the list of architectures we build for the boot
laoder testing scripts. It just adds to the build time, and will be
retired soon anyway, per discussions in freebsd-arch@ last month.
2018-07-09 18:02:23 +00:00
cem
e2ca0f334d OCF: Add plain hash modes
In part, to support OpenSSL's use of cryptodev, which puts the HMAC pieces
in software and only offloads the raw hash primitive.

The following cryptodev identifiers are added:

 * CRYPTO_RIPEMD160 (not hooked up)
 * CRYPTO_SHA2_224
 * CRYPTO_SHA2_256
 * CRYPTO_SHA2_384
 * CRYPTO_SHA2_512

The plain SHA1 and 2 hashes are plumbed through cryptodev (feels like there
is a lot of redundancy here...) and cryptosoft.

This adds new auth_hash implementations for the plain hashes, as well as
SHA1 (which had a cryptodev.h identifier, but no implementation).

Add plain SHA 1 and 2 hash tests to the cryptocheck tool.

Motivation stems from John Baldwin's earlier OCF email,
https://lists.freebsd.org/pipermail/freebsd-arch/2018-January/018835.html .
2018-07-09 07:28:13 +00:00
cem
21f8de74b4 OCF: Add CRYPTO_SHA2_224_HMAC mode
Round out the complete set of basic SHA2 HMAC modes with SHA2-224.

Support is added to the cryptocheck test tool.
2018-07-09 07:26:12 +00:00
sbruno
f5a583ad15 WITHOUT_SERVICESDB:
Add src.conf knob to disable the installation of /var/db/services.db

Default to leaving services.db in place, but allow the removal of the
file and its creation with a src.conf knob.

This file ends up being 2MB in size.  For small systems this is a waste
of space but its a tradeoff.

Reviewed by:	bdrewery
Differential Revision:	https://reviews.freebsd.org/D9655
2018-07-04 17:18:35 +00:00
trasz
fafaf5a282 Add a semaphore ping benchmark.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 14:20:19 +00:00
trasz
47c25991b6 Add a very simple memcpy benchmark.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 14:18:12 +00:00
trasz
13876a8cd2 Sort the benchmarks even more.
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 14:15:48 +00:00
trasz
66c318023b Rename "dup" and "fstat_shmfd" to "shmfd_dup" and "shmfd_fstat" for consistency.
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 14:14:00 +00:00
trasz
2456108592 Sort the benchmarks; no functional changes intended.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 14:11:49 +00:00
trasz
c6f32f0c3a Disable pipepingtd benchmarks, to make it not depend on libthr.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 13:52:13 +00:00
trasz
4286b395ad Make the pipeping benchmarks work with larger buffer sizes.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 13:45:29 +00:00
trasz
277872b727 Add threaded pipe ping benchmark.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 13:39:48 +00:00
trasz
d5c96360f0 Add a trivial "pipe ping" (two processes) benchmark.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 13:34:43 +00:00
trasz
127c1e47d5 Make syscall_timing(1) build with WARNS=6.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 13:31:55 +00:00
trasz
fbbdd10ded Add getprogname(3) (ie null) benchmark.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 13:28:43 +00:00
trasz
d68c5e4624 Use a macro for iteration.
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-07-04 13:26:47 +00:00
ian
7cecfc62d4 Fix the fstab entries for tests that have an efi partition in slice 1 and
put the rootfs on s2.
2018-07-03 16:07:04 +00:00
ian
043b125ccd Doh! Commit the proper default values, not my local tweaks. 2018-07-02 22:59:29 +00:00
ian
87936c96ec Move the passphrase and iteration count into variables.
This is purely to make it easier to tweak them locally; the machine I have
for testing takes forever to do 50,000 pw strengthening iterations, and
we're not testing the strength of geli's anti-pw-guessing logic here
(especially given that our test passphrase is "passphrase", except that
I tend to tweak that also, to 'x', because typing is hard).

Some day these should be settable as cmdline args.  But then, some day this
whole script should probably get a rewrite. :)
2018-07-02 22:58:04 +00:00
ian
e5cefdc937 Allow making just one flavor of boot-test image based on cmdline args. 2018-07-02 22:51:48 +00:00
will
6dd6f3918a beinstall: enable use with git worktrees. 2018-07-01 01:09:52 +00:00
dim
9c9640b12e More follow-up to r335799 (llvm/clang 6.0.1 update), where I forgot to
update mtree files, ObsoleteFiles and a number of other paths.  Sorry
about all the breakage.

Pointy hat to:	me
MFC after:	2 weeks
X-MFC-With:	r335799
2018-06-30 15:03:22 +00:00
bdrewery
b1324a4ff9 Push users towards LLVM_TARGET_ALL.
MFC after:	1 week
2018-06-27 17:13:36 +00:00
bdrewery
9b41f30c42 Clang: Only build needed target for bootstrap compiler.
This will disable the new LLVM_TARGET_ALL option which will only
enable the required target.

This only impacts the bootstrap compiler in WORLDTMP, not the target compiler
that will be installed.

MFC after:	2 weeks
Reviewed by:	sbruno, dim (earlier version)
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D16021
2018-06-27 16:57:56 +00:00
bdrewery
dabae8d650 Add LLVM_TARGET_ALL option.
LLVM_TARGET_* will auto be set based on LLVM_TARGET_ALL and MK_CLANG.

If LLVM_TARGET_ALL is disabled, during a cross-build, then SYSTEM_COMPILER
and SYSTEM_LINKER are auto disabled.

This option should be used by users rather than the per-arch LLVM_TARGET
options as it is simpler to maintain for them should the supported
target list change.

MFC after:	2 weeks
Reviewed by:	sbruno, dim
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D16020
2018-06-27 16:57:51 +00:00
dim
6ee1d5b597 Add support for selectively enabling LLVM targets
This makes it possible, through src.conf(5) settings, to select which
LLVM targets you want to build during buildworld.  The current list is:

* (WITH|WITHOUT)_LLVM_TARGET_AARCH64
* (WITH|WITHOUT)_LLVM_TARGET_ARM
* (WITH|WITHOUT)_LLVM_TARGET_MIPS
* (WITH|WITHOUT)_LLVM_TARGET_POWERPC
* (WITH|WITHOUT)_LLVM_TARGET_SPARC
* (WITH|WITHOUT)_LLVM_TARGET_X86

To not influence anything right now, all of these are on by default, in
situations where clang is enabled.

Selectively turning a few targets off manually should work.  Turning on
only one target should work too, even if that target does not correspond
to the build architecture.  (In that case, LLVM_NATIVE_ARCH will not be
defined, and you can only use the resulting clang executable for
cross-compiling.)

I performed a few measurements on one of the FreeBSD.org reference
machines, building clang from scratch, with all targets enabled, and
with only the x86 target enabled.  The latter was ~12% faster in real
time (on a 32-core box), and ~14% faster in user time.  For a full
buildworld the difference will probably be less pronounced, though.

Reviewed by:	bdrewery
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D11077
2018-06-22 15:00:00 +00:00
kevans
c0fa49d516 Don't remove loader.conf(5) when built WITHOUT_FORTH
The new stand/ structure installs loader.conf(5) and defaults/loader.conf
regardless of interpreter. The only thing gating installation now is
MK_BOOT.

Reported by:	eadler
2018-06-21 05:28:00 +00:00
bdrewery
dc29454920 Add WITH_SYSTEM_LINKER, on by default, that avoids building lld when possible.
This works similar to WITH_SYSTEM_COMPILER added in r300354.  It only
supports lld via WITH_LLD_BOOTSTRAP.

When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools.  If either check fails though then
libclang is built.

The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".

-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.

A new 'make test-system-linker' target is added to see the logic results.

Makefile.inc1:
  CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
  could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
  logic as done with SYSTEM_COMPILER.  This also required moving where
  bsd.linker.mk was read since XLD needs to be set before parsing it.  This
  creates a situation where src.opts.mk can not test LINKER_FEATURES or
  add LLD_BOOTSTAP to BROKEN_OPTIONS.

Reviewed by:	emaste (earlier version)
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D15894
2018-06-20 16:10:14 +00:00
imp
03a02e0dc1 Fix typo noticed by pstef@. 2018-06-19 21:58:04 +00:00
sbruno
a5206a2a79 MK_EFI - Add uefisign and friends to this knob and ensure that we don't
try to build them if MK_OPENSSL is unset.

Reviewed by:	emaste imp kevans
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15211
2018-06-19 21:07:25 +00:00
imp
553ccdaf2e Add my script for coping with git-svn and the need to rebase
changes for different branches. It's a bit rough right now,
but should be good enough for most people to try to use. It's
definitely 'tools' tree quality.
2018-06-19 00:27:30 +00:00
imp
0485d02349 Switch to creating UEFI ESPs using loader.efi instead of boot1.efi.
Cope for the fact that laoder.efi, not being boot1, doesn't read
/boot.config by setting boot_serial and force the serial console.

Also add sysctl so we can display the boot method.

Provide a variable, do_boot1_efi, if you want to use boot1 for
testing. But since it's transient, it's just a variable and not
available on the command line.
2018-06-18 23:16:47 +00:00
allanjude
75f082b990 rootgen.sh: complete all profiles except GELI+MBR (not supported)
This extends the test suite to generate images for every combination of:
amd64: mbr/gpt geli/nogeli ufs/zfs legacy/uefi/both

Except for mbr+geli, which is not currently possible.

Reviewed by:	imp (previous version)
Sponsored by:	Klara Systems
Differential Revision:	https://reviews.freebsd.org/D15846
2018-06-17 05:55:31 +00:00
imp
5c704454d3 FreeBSD/amd64 requires 256MiB to boot UEFI, 128MB simply doesn't work.
128MiB still works for Legacy booting, however. Go ahead and do 256MiB
for all amd64 boxes, since the number of such boxes < 256MiB is
vanishingly small.
2018-06-17 02:49:42 +00:00
eadler
eb49fcf599 netrate: build with WARNS=6
Tested with amd64, i386
2018-06-16 23:49:22 +00:00
allanjude
c32fa4828d rootgen.sh: Don't copy various bits of the bootcode from the running system
We want to use the versions of the bootcode we just built, rather than
ones from whatever happens to be in /boot on the test machine

These were incorrectly added by me in r334888
2018-06-16 17:45:44 +00:00
emaste
478781829c Remove objcopy from BINUTILS option descriptions
As of r306649 objcopy is always ELF Tool Chain's elfcopy; binutils
objcopy is never used.

PR:		229046
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-06-15 16:18:39 +00:00
sbruno
14d40a95d8 WITHOUT_NLS cleanup of more empty dirs.
Reviewed by:	emaste
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15626
2018-06-12 19:26:25 +00:00
allanjude
c7a005a657 Add ZFS+GPT EFI and LEGACY+EFI to rootgen.sh and installboot.sh
Reviewed by:	imp (maintainer)
Sponsored by:	Klara Systems
Differential Revision:	https://reviews.freebsd.org/D15700
2018-06-09 17:49:43 +00:00
allanjude
50466e9a5d Change rootgen.sh to use the default ZFS boot environment name
Reviewed by:	imp (maintainer)
Sponsored by:	Klara Systems
Differential Revision:	https://reviews.freebsd.org/D15699
2018-06-09 17:47:56 +00:00
dim
9fb785480a Add missed libc++ entries to (Optional)ObsoleteFiles.inc
Some of these were removed during the libc++ 5.0.0 import, others were
added in the libc++ 6.0.0 import.

MFC after:	3 days
2018-06-09 17:40:31 +00:00
bapt
83eb83a386 Remove NLS support from BSD grep
GNU grep as in actually in base does not have any translations support
compiled in, so no functionnality loss.

We do support 193 locales in base, we will never catch up on that number of
translation with bsd grep.

Removing NLS support make bsd grep consistent with the other binaries in base
which are not translated, and also reduce a little bit the code.

Reviewed by:	kevans
Approved by:	kevans
Discussed with:	kevans @BSDCan
Differential Revision:	https://reviews.freebsd.org/D15682
2018-06-06 23:12:35 +00:00
emaste
85e5468fdc makeroot.sh: allow duplicate entries even with -f <filelist>
makefs disallows duplicate entries unless the -D option is specified.

Previously makeroot.sh enabled -D unless a filelist was provided via
the -f options.  The filelist logic creates an mtree manifest from the
METALOG and the provided filelist by passing them through `sort -u`,
so duplicates were not expected.  However, duplicates can still occur
when a directory appears in multiple packages -- for example,

./etc/pam.d type=dir uname=root gname=wheel mode=0755
./etc/pam.d type=dir mode=0755 tags=package=runtime
./etc/pam.d type=dir mode=0755 tags=package=at
./etc/pam.d type=dir mode=0755 tags=package=ftp
./etc/pam.d type=dir mode=0755 tags=package=telnet

For the purposes of makefs these directory entries are identical, but
are of course not identical for sort -u.

For now just leave the allow duplicates -D flag enabled.

PR:		228606
Sponsored by:	The FreeBSD Foundation
2018-05-30 13:51:00 +00:00
eadler
348b3a8006 src.conf: use more natural language for @generated
Requested by:	emaste
2018-05-26 21:14:49 +00:00
bcran
c4676ab29a Remove extra space before parenthesis in src.conf(5)
Reviewed by:	eadler
Differential Revision:	https://reviews.freebsd.org/D15528
2018-05-25 01:38:59 +00:00
eadler
68c7cdc07b Add missing file for WITH{OUT}_BSDINSTAL
PR:		227805
Submitted by:	Dmitry Wagin <dmitry.wagin@ya.ru>
2018-05-23 09:02:31 +00:00
eadler
8113741924 Add the text '@generated' to src.conf.5
This is a cross-tool approach to identifying generated code. Some tools,
notably phabricator, handle this marker specially.  See
https://reviews.freebsd.org/differential/diff/42870/ for such an
example.
2018-05-23 03:41:22 +00:00
fabient
97541ca804 Add a SPD cache to speed up lookups.
When large SPDs are used, we face two problems:

- too many CPU cycles are spent during the linear searches in the SPD
  for each packet
- too much contention on multi socket systems, since we use a single
  shared lock.

Main changes:

- added the sysctl tree 'net.key.spdcache' to control the SPD cache
  (disabled by default).
- cache the sp indexes that are used to perform SP lookups.
- use a range of dedicated mutexes to protect the cache lines.

Submitted by: Emeric Poupon <emeric.poupon@stormshield.eu>
Reviewed by: ae
Sponsored by:	Stormshield
Differential Revision: https://reviews.freebsd.org/D15050
2018-05-22 15:54:25 +00:00
emaste
c9429b411f intel-ucode-split: add -n flag to skip creating output files
Sponsored by:	The FreeBSD Foundation
2018-05-22 14:35:33 +00:00
emaste
d8bfe26bdf intel-ucode-split: incorporate review feedback, using asprintf
As reported by delphij in review D15443 asprintf cleans this up a little
by avoiding hardcoded buffer sizes.

Reported by:	delphij
2018-05-17 17:45:47 +00:00
sbruno
ff192f8aa2 Retire vxge(4).
This driver was merged to HEAD one week prior to Exar publicly announcing they
had left the Ethernet market. It is not known to be used and has various code
quality issues spotted by Brooks and Hiren. Retire it in preparation for
FreeBSD 12.0.

Submitted by:	kbowling
Reviewed by:	brooks imp
Relnotes:	yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15442
2018-05-17 14:55:41 +00:00
emaste
cd894f4017 Add intel-ucode-split to tools README 2018-05-16 03:08:06 +00:00
emaste
08c74c8316 intel-ucode-split: list platform ids based on processor_flags
The Intel CPU "Platform Id" is a 3-bit integer reported by a given MSR.
Intel microcode updates have an 8-bit field to indicate Platform Id
compatibility - one bit in the mask for each of the possible Platform Id
values.  To simplify interpretation, report the Platform Id mask also as
a list.
2018-05-16 01:55:52 +00:00
emaste
2c2b0d59be intel-ucode-split: exit on unknown ucode header version 2018-05-16 01:41:36 +00:00
emaste
c05bf438ef intel-ucode-split: add a -v verbose flag
And be quiet by default.

Sponsored by:	The FreeBSD Foundation
2018-05-16 01:08:11 +00:00
emaste
f8a2c6dc6f Add a tool to split Intel microcode into one file per Platform Id
Intel now releases microcode updates in files named after
<family>-<model>-<stepping>.  In some cases a single file may include
microcode for multiple Platform Ids for the same family, model, and
stepping.  Our current microcode update tooling (/usr/sbin/cpucontrol)
only processes the first microcode update in the file.

This tool splits combined files into individual files with one microcode
update each, named as
<family>-<model>-<stepping>.<platform_id_mask>.

Adding this to tools/ for experimentation and testing.  In the future
we'll want to have cpucontrol or other tooling work directly with the
Intel-provided microcode files.

Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15433
2018-05-15 21:51:29 +00:00
sevan
af2f38f3af Typo
Submitted by:	jrm@
Approved by:	bcr (mentor)
Differential Revision:	https://reviews.freebsd.org/D14836
2018-05-13 22:58:40 +00:00
des
bb2118ef40 Rename all Unbound binaries and man pages from unbound* to local-unbound*.
PR:		222902
2018-05-12 17:10:36 +00:00
kib
27836e8997 Add a test for vm86(2), simple to use and diagnose.
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-05-12 11:53:49 +00:00
kib
2f759f25d2 Add the test program to examine CPU behaviour for pop ss issue
CVE-2018-8897.

Requested by:	emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2018-05-10 13:52:52 +00:00
imp
401def6ae8 Remove 'All Rights Reserved'
Remove this from some of the iXsystems stuff I did. OK'd by kmoore
at iXsystems
2018-05-09 20:41:03 +00:00
sbruno
9e75a9c23d nxge(4):
Remove nxge(4) and associated man page and tools in FreeBSD 12.0.

Submitted by:	kbowling
Reviewed by:	brooks
Relnotes:	yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D1529
2018-05-08 21:14:29 +00:00
kevans
da653eef52 bsdgrep: annihilate our in-tree TRE, previously disabled by default
It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).

To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.
2018-05-04 03:13:25 +00:00
sbruno
3bb0eb8b79 Retire ixgb(4)
This driver was for an early and uncommon legacy PCI 10GbE for a single
ASIC, Intel 82597EX. Intel quickly shifted to the long lived ixgbe family.

Submitted by:	kbowling
Reviewed by:	brooks imp jeffrey.e.pieper@intel.com
Relnotes:	yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15234
2018-05-02 15:59:15 +00:00
emaste
a3eb1e9afb Retire lmc(4)
This driver supports legacy, 32-bit PCI devices, and had an ambiguous
license.  Supported devices were already reported to be rare in 2003
(when an earlier version of the driver was removed in r123201).

Reviewed by:	rgrimes
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15245
2018-05-01 16:30:48 +00:00
imp
4056e4727b Use M. Warner Losh everywhere on my copyrights.
Remove 'All Rights Reserved' where I can.
2018-05-01 16:29:22 +00:00
bapt
2f45771b4b Use a script wrapper for <compress>grep
Import the wrapper script from zstdgrep (written by wiz@netbsd.org)

Modify it to support more than just zstd (adding support for gzip,
lzma, xz and bzip2)

Write a simple manpage dedicated for it.

Only use that new wrapper both for gnu grep and bsd grep

Next step will be removing code related to compression format from bsdgrep

Reviewed by:	kevans
Approved by:	kevans
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D15193
2018-04-25 13:23:58 +00:00
mckusick
c2d88526f7 Fix incorrect output for a file consisting of a single full-size
block followed by a discontiguous fragment.

Add checks for unallocated inodes and inodes with unknown mode
types.

Cleanup variable declarations by changing from type `int' to types
like ufs_lbn_t, ufs2_daddr_t, etc.

Reported by: bde
2018-04-19 22:43:55 +00:00
brooks
93bd79f30a Add sortbench.
This is a set of benchmarks of qsort, mergesort, heapsort, and
optionally wikisort and a script to run them.

Submitted by:	Miles Fertel <milesfertel@college.harvard.edu>
Sponsored by:	Google Summer of Code 2017
Differential Revision:	https://reviews.freebsd.org/D12677
2018-04-19 21:53:57 +00:00
mckusick
3e8df1eea5 In addition to the existing argument format:
prtblknos filesystem_device inode ...

add an additional argument format:

	prtblknos file

which is more convenient than figuring out the filesystem
and inode number for "file".

When given a list of multiple inodes, rather than exiting
the program on an error with one of them, skip over it and
continue with the next one.

Submitted by: bde
2018-04-18 23:08:10 +00:00
mckusick
5e674c7664 Check the inode type and only attempt to print block lists for
regular files, directories, and symbolic links that require
external storage.

Correct the handling of files with holes and files that have
one or more large blocks and end with a fragment.

Reported by: bde
2018-04-18 22:24:44 +00:00
emaste
cb5abd6e17 Remove mention of tools/recoverdisk, now in sbin
PR:		227570
MFC after:	3 days
2018-04-18 14:02:37 +00:00
mckusick
e9a893e557 Add (intmax_t) type casts to some printf parameters to keep i386 happy. 2018-04-08 07:18:29 +00:00
mckusick
1c8afb9a0e Split tools/diag/prtblknos into two parts:
main.c - opens disk and processes the argument list
		 of inodes to be printed
	prtblknos.c - prints out the list of blocks used by an inode

This change allows the fsdb program to import prtblknos() to use when
printing out the set of blocks used by an inode.

This program was switched to using the libufs library to ease its
integration with fsdb and any other filesystem utility that might
want to use it in the future.
2018-04-08 06:52:58 +00:00
cem
5cded89b02 cryptocheck: Add Chacha20 cipher to tool 2018-03-29 04:14:37 +00:00
cem
e89a3c0e55 cryptotest: Add chacha20 cipher 2018-03-29 04:07:53 +00:00
cem
4c8d0980b7 cryptotest: Add blake2 hashes to tool
Sponsored by:	Dell EMC Isilon
2018-03-27 17:34:28 +00:00
jhb
fabaeee1e4 Update the license to note my work on cryptocheck was sponsored.
Sponsored by:	Chelsio Communications
2018-03-26 22:30:17 +00:00
kevans
503b2fb104 lua-lint: Hint that try_include is now an accepted global
try_include was added in r331563.
2018-03-26 19:08:19 +00:00
cem
54a580348e cryptocheck: Add support for Blake2{B,S} hashes
Since they are not yet present in the version of openssl in base, this will
require installing the ports openssl.

Sponsored by:	Dell EMC Isilon
2018-03-23 05:37:18 +00:00
cem
cfffe134ad Bring in JHB's cryptocheck tool
It can be used to validate basic algorithm correctness on a variety of inputs,
by comarison to openssl.

While here, add some sanity to the crypto/Makefile.

The tool may not be perfect, but getting it in tree where collaboration can
happen is a nice first step.  The pace of development outside of svn seems
to have slowed down mid-2017.

Obtained from:	github bsdjhb/freebsd:cryptocheck
Sponsored by:	Dell EMC Isilon
2018-03-23 04:31:19 +00:00
hselasky
d1efa6e93f Add kernel and userspace code to dump the firmware state of supported
ConnectX-4/5 devices in mlx5core.

The dump is obtained by reading a predefined register map from the
non-destructive crspace, accessible by the vendor-specific PCIe
capability (VSC). The dump is stored in preallocated kernel memory and
managed by the mlx5tool(8), which communicates with the driver using a
character device node.

The utility allows to store the dump in format
    <address> <value>
into a file, to reset the dump content, and to manually initiate the
dump.

A call to mlx5_fwdump() should be added at the places where a dump
must be fetched automatically. The most likely place is right before a
firmware reset request.

Submitted by:	kib@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-08 15:21:56 +00:00
kevans
a02d0ad25e lua-lint: Whitelist cli_execute_unparsed as a global 2018-03-07 18:41:16 +00:00
kevans
3449d2e6d5 Add --globals printc to lua-lint
We provide a global 'printc' as of r330282; make luacheck aware of this fact
so that we can use it without complaints.
2018-03-02 16:51:43 +00:00
kevans
292576b24d Add descriptions for recently added loader options 2018-03-01 22:00:38 +00:00
kevans
34466a0e55 Back out r330252; will reapply with an actual commit message 2018-03-01 21:59:30 +00:00
kevans
5d08418b50 MFC after: 1 week 2018-03-01 21:57:08 +00:00
kevans
6eeb735b44 Remove LOADER_EFI description files
LOADER_EFI functionality got folded into EFI as of r330248.
2018-03-01 21:51:20 +00:00
kevans
8dc36d5c2c Add missing punctuation to *_LOADER_EFI descriptions... 2018-02-28 15:47:24 +00:00
kevans
1f54682dce Some missing LOADER_EFI build option descriptions 2018-02-28 15:28:04 +00:00
kevans
898476b1a8 Add missing WITH_BSD_GREP_FASTMATCH description
MFC after:	3 days
2018-02-28 15:27:36 +00:00
emaste
33289f97a0 Add kernel retpoline option for amd64
Retpoline is a compiler-based mitigation for CVE-2017-5715, also known
as Spectre V2, that protects against speculative execution branch target
injection attacks.

In this commit it is disabled by default, but will be changed in a
followup commit.

Reviewed by:	bdrewery (previous version)
MFC after:	3 days
Security:	CVE-2017-5715
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14242
2018-02-28 14:57:45 +00:00