Commit Graph

448 Commits

Author SHA1 Message Date
Bryan Drewery
9f665e12ce Redo r288270: Hookup mkcsmapper_static and mkesdb_static for 'make clean'
These are only built as part of the top-level 'build-tools' call for
'make buildworld'.  They still need to be cleaned during the 'make clean'
treewalks though.

Reported by:	markj
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-05-01 19:34:15 +00:00
Baptiste Daroussin
a17665bcef Replace again GNU diff with BSD diff
After a firts failed attempt, BSD diff is now good enough to replace
GNU diff.

Relnotes:	yes
2017-04-20 19:24:51 +00:00
Baptiste Daroussin
e229090553 Import zstandard 1.1.4 in base
zstandard is a new compression library/tool which is very fast at
compression/decompression

For now import as a private library
2017-04-15 20:05:22 +00:00
Eric van Gyzen
f5b9907c86 Port the getaddrinfo(1) utility from NetBSD
Submitted by:	Lohith Bellad <lohithbsd@gmail.com>
Reviewed by:	hiren (earlier rev), ae
Obtained from:	NetBSD
MFC after:	1 week
Relnotes:	yes
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D9365
2017-03-20 16:44:55 +00:00
Baptiste Daroussin
2c5ddcf8b6 Temporary readd GNU diff
etcupdate requires --change-group-format it is not easy to implement
in bsd diff so for now readd GNU diff
2017-03-19 17:19:59 +00:00
Baptiste Daroussin
97ab006d51 Remove the WITHOUT_MANDOCDB option
mandoc database is activated since FreeBSD 11.0, let's remove the previous
database format for FreeBSD 12.0
2017-03-11 06:51:21 +00:00
Baptiste Daroussin
3bbe3f672e Import diff from OpenBSD and remove GNU diff
Some of the modifications from the previous summer of code has been integrated
Modification for compatibility with GNU diff output has been added

Main difference with OpenBSD:
Implement multiple GNU diff options:
* --ignore-file-name-case
* --no-ignore-file-name-case
* --normal
* --tabsize
* --strip-trailing-cr
Make diff -p compatible with GNU diff
Implement diff -l
Make diff -r compatible with GNU diff

Capsicumize diffing 2 regular files
Add a simple test suite

Approved by:	AsiaBSDcon devsummit
Obtained from:	OpenBSD, GSoC
Relnotes:	yes
2017-03-11 05:01:29 +00:00
Allan Jude
39f8282b48 Remove bdes(1)
The use of DES for anything is discouraged, especially with a static IV of 0

If you still need bdes(1) to decrypt Kirk's video lectures, see
security/bdes in ports.

This commit brought to you by the FOSDEM DevSummit and the
"remove unneeded dependancies on openssl in base" working group

Reviewed by:	bapt, brnrd
Relnotes:	yes
Sponsored by:	FOSDEM DevSummit
Differential Revision:	https://reviews.freebsd.org/D9424
2017-02-06 08:27:19 +00:00
Ed Maste
f987297fc9 Add a WITHOUT_DIALOG src.conf(5) knob
It also turns off dependencies (bsdinstall, bsdconfig, dpv, tzsetup).

Reviewed by:	dteske
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D7969
2016-09-27 18:08:38 +00:00
Konstantin Belousov
7402f93ef2 Provide proccontrol(1), an utility to control processes behaviour, related
to procctl(2).

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-09-23 12:34:54 +00:00
Ed Maste
d74808dd1a users: don't build when we don't have a C++ toolchain
(On RISC-V MK_CXX is in BROKEN_OPTIONS, so users remains skipped there.)

Reviewed by:	bdrewery
Differential Revision:	https://reviews.freebsd.org/D7725
2016-08-31 21:12:50 +00:00
Andrey A. Chernov
e8ab43489a Continuation lines with comments badly affects gprof, it is excluded from
build on amd64 f.e.
2016-07-20 15:59:37 +00:00
Martin Matuska
cdf63a700c MFV r299425:
Update libarchive to 3.2.0

New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
  similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive

Patched files (fixed compiler warnings):

contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)

MFC after:	1 month
Relnotes:	yes
2016-05-12 10:16:16 +00:00
Baptiste Daroussin
13b5b54865 import sdiff(1) from GSoC 2012
Import sdiff(1) from the diff version written by Raymond Lai,
improved during GSoC 2012 by Jesse Hagewood.

Compared to the version done in during that summer of code:
- Remove the zlib frontend: zsdiff
- Compatible output (column size and separators) with GNU sdiff

Compared to GNU sdiff in ports:
- The only difference is padding using spaces vs tabs

Compared to OpenBSD and NetBSD import:
- Implement missing options (including long options) from GNU sdiff
- Improved support for the edition mode (signal handling)
- Output visually compatible with GNU sdiff: size of columns

While here import regression tests from NetBSD adapted to fit the output as
expected by GNU sdiff

Reviewed by:	emaste (in part)
Obtained from:	OpenBSD, NetBSD, GSoC 2012
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D5981
Differential Revision:	https://reviews.freebsd.org/D6032 (diff with NetBSD version)
Differential Revision:	https://reviews.freebsd.org/D6033 (diff with OpenBSD version)
2016-04-29 23:27:15 +00:00
Conrad Meyer
3d222369ac Add a small tool, resizewin(1), to query terminal for window size
Submitted by:	Daniel O'Connor
Reviewed by:	kan, wblock, cem
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D4438
2016-04-13 00:30:42 +00:00
Maxim Sobolev
5497acc527 Obsolete mkulzma(8) and geom_uncompress(4), their functionality
is now provided by mkuzip(8) and geom_uzip(4) respectively.

MFC after:	1 month
2016-02-24 00:39:36 +00:00
Ruslan Bukin
1c052a8c31 Disable gprof and users on RISC-V, they don't build. 2016-01-25 16:49:39 +00:00
Baptiste Daroussin
dee69ea53c Reintegrate colldef(1) and mklocale(1)
While those tools are not needed anymore they are necessary to build FreeBSD 9
and 10. it does not hurt to keep those tools around until both 9 and 10 branch
become EOLed.

Modify colldef(1) to build after the change in the collation header, and ensure
it does produce the same collation definition it used to generate for 9 and 10

Reported by:	Oliver Pinter
2015-11-20 23:15:05 +00:00
Baptiste Daroussin
3c3feed41a Merge from head 2015-11-01 21:17:38 +00:00
David Chisnall
a0706eb457 Lots of improvements to the BSD-licensed dtc
- Various fixes to includes (including recursive includes)
- Lots of testing that the output exactly matches GPL'd dtc
- Lots of bug fixes to merging
- Fix incorrect mmap usage
- Ad-hoc memory management replaced with C++11 unique_ptr and similar

Patrick Wildt has successfully run many (all?) of the GPL dtc test suite.
2015-10-25 14:52:16 +00:00
Bryan Drewery
58144364e9 dtc needs to be installed if MK_GPL_DTC is not set, which doesn't fit into the
SUBDIR.yes pattern well.

This fixes installation of the BSDL dtc after r288904.

Reported by:	jhibbits
Discussed with:	imp
Sponsored by:	EMC / Isilon Storage Division
2015-10-23 17:24:34 +00:00
Baptiste Daroussin
becbad1f6e Merge from head 2015-10-13 19:44:36 +00:00
Warner Losh
3a845236df Start using the fact that SUBDIR.yes is added to SUBDIR
and move from the pattern of:

.if ${MK_FOO} != "no"
SUBDIR+= bar
.endif

to

SUBDIR.${MK_FOO}+= bar

since we know that MK_FOO is always either yes or no and the latter
form is easier to follow and much shorter. Various exception to this
pattern dealt with on an ah-hoc basis.

Discussed on arch@ a while ago.
2015-10-05 21:41:55 +00:00
Konstantin Belousov
5821213b4c Add aarch64 support to truss(1).
Reviewed by:	jhb
Discussed with:	emaste (license)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3750
2015-10-02 13:30:56 +00:00
Colin Percival
6ae1554a5d Final step of eliminating the "games" distribution: Merge src/games
(or what's left of it, at least) into src/usr.bin.

This change will not be MFCed.

Discussed at:   EuroBSDCon 2014
Committed from: EuroBSDCon 2015
2015-10-02 10:08:11 +00:00
Conrad Meyer
0ceadbf05d Revert r288270 to fix the build
Submitted by:	bdrewery
Sponsored by:	EMC / Isilon Storage Division
2015-09-26 20:26:05 +00:00
Bryan Drewery
cfac01438a Hookup mkcsmapper_static and mkesdb_static for all but install.
These are only handled as 'build-tools' in Makefile.inc1.  This causes
'make clean' from the top of the tree to not clean the directories.  It also
effectively has kept them disconnected and risks them bitrotting.  The
buildworld process never cleans them either.

Connect them so they will always be built, cleaned, etc, but never installed.

Discussed with:	imp (briefly)
Sponsored by:	EMC / Isilon Storage Division
2015-09-26 14:28:04 +00:00
Baptiste Daroussin
23a32822d2 Merge from HEAD 2015-08-25 20:14:50 +00:00
Ed Maste
1bc28ffccd Roll WITHOUT_ELFTOOLCHAIN_TOOLS into WITHOUT_TOOLCHAIN
The option was added only to ease the transition from GNU Binutils to
ELF Tool Chain tools, and that process is now complete (for the viable
replacements). Noting the removal in UPDATING is sufficient as we have
not shipped a release with the option.

Reviewed by:	brooks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3240
2015-08-13 17:50:47 +00:00
Ed Maste
728fb54dde Use consistent style for optional subdirectories
Reviewed by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3356
2015-08-10 20:19:10 +00:00
Ed Maste
9138b6e04a Remove ${_subdir}s that are never set
The MK_NIS subdirs were leftovers from r212558 and kdump/truss were
introduced in r240404.

Sponsored by:	The FreeBSD Foundation
2015-08-10 19:41:31 +00:00
Baptiste Daroussin
1fbfaeef7e Remove colldef which is not needed anymore 2015-08-08 20:08:09 +00:00
Baptiste Daroussin
c2d5d37b67 Remove mklocale which is not needed anymore 2015-08-08 20:07:32 +00:00
Baptiste Daroussin
057ca2d437 Add localedef(1), a locale definition generator tool
The localedef tool can read entire (and unmodified) CLDR posix definition
files, and generate all 6 LC categories: LC_COLLATE, LC_CTYPE, LC_TIME,
LC_NUMERIC, LC_MONETARY and LC_MESSAGES.

This tool has a long history with Solaris.  The Nexenta developers
modified it to read CLDR files and created the much richer collation
formats.  The libc collation functions have to be modified to read the
new format (called "BSD-1.0") and to handle the new data structures.

The result will be that locale-sensitive tools and functions will now
properly sort multibyte and unicode strings.

Obtained from:	Dragonfly
2015-08-07 23:53:31 +00:00
Baptiste Daroussin
c048a83f51 Replace GNU RCS ident with a BSD license ident
Rationale: ident(1) is useful out of RCS, lot of scripts are using ident(1) and
failing when base is built WITHOUT_RCS.

This version is:
- fully compatible with RCS 5.7 ident.
- fully compatible with RCS 5.9 ident.
- passes all ident test from GNU RCS 5.9 test suite

This version has support for: svn extension for the Keyword id (double colon and
# before last $)

Différences with GNU RCS ident:
- no long options as found in GNU RCS 5.9 (but not commented there).
- '-V' reports nothing but has been added for compatibility.

Differential Revision:	https://reviews.freebsd.org/D3200
Reviewed by:	pfg
2015-07-26 11:21:36 +00:00
Adrian Chadd
6520495abc Add an initial NUMA affinity/policy configuration for threads and processes.
This is based on work done by jeff@ and jhb@, as well as the numa.diff
patch that has been circulating when someone asks for first-touch NUMA
on -10 or -11.

* Introduce a simple set of VM policy and iterator types.
* tie the policy types into the vm_phys path for now, mirroring how
  the initial first-touch allocation work was enabled.
* add syscalls to control changing thread and process defaults.
* add a global NUMA VM domain policy.
* implement a simple cascade policy order - if a thread policy exists, use it;
  if a process policy exists, use it; use the default policy.
* processes inherit policies from their parent processes, threads inherit
  policies from their parent threads.
* add a simple tool (numactl) to query and modify default thread/process
  policities.
* add documentation for the new syscalls, for numa and for numactl.
* re-enable first touch NUMA again by default, as now policies can be
  set in a variety of methods.

This is only relevant for very specific workloads.

This doesn't pretend to be a final NUMA solution.

The previous defaults in -HEAD (with MAXMEMDOM set) can be achieved by
'sysctl vm.default_policy=rr'.

This is only relevant if MAXMEMDOM is set to something other than 1.
Ie, if you're using GENERIC or a modified kernel with non-NUMA, then
this is a glorified no-op for you.

Thank you to Norse Corp for giving me access to rather large
(for FreeBSD!) NUMA machines in order to develop and verify this.

Thank you to Dell for providing me with dual socket sandybridge
and westmere v3 hardware to do NUMA development with.

Thank you to Scott Long at Netflix for providing me with access
to the two-socket, four-domain haswell v3 hardware.

Thank you to Peter Holm for running the stress testing suite
against the NUMA branch during various stages of development!

Tested:

* MIPS (regression testing; non-NUMA)
* i386 (regression testing; non-NUMA GENERIC)
* amd64 (regression testing; non-NUMA GENERIC)
* westmere, 2 socket (thankyou norse!)
* sandy bridge, 2 socket (thankyou dell!)
* ivy bridge, 2 socket (thankyou norse!)
* westmere-EX, 4 socket / 1TB RAM (thankyou norse!)
* haswell, 2 socket (thankyou norse!)
* haswell v3, 2 socket (thankyou dell)
* haswell v3, 2x18 core (thankyou scott long / netflix!)

* Peter Holm ran a stress test suite on this work and found one
  issue, but has not been able to verify it (it doesn't look NUMA
  related, and he only saw it once over many testing runs.)

* I've tested bhyve instances running in fixed NUMA domains and cpusets;
  all seems to work correctly.

Verified:

* intel-pcm - pcm-numa.x and pcm-memory.x, whilst selecting different
  NUMA policies for processes under test.

Review:

This was reviewed through phabricator (https://reviews.freebsd.org/D2559)
as well as privately and via emails to freebsd-arch@.  The git history
with specific attributes is available at https://github.com/erikarn/freebsd/
in the NUMA branch (https://github.com/erikarn/freebsd/compare/local/adrian_numa_policy).

This has been reviewed by a number of people (stas, rpaulo, kib, ngie,
wblock) but not achieved a clear consensus.  My hope is that with further
exposure and testing more functionality can be implemented and evaluated.

Notes:

* The VM doesn't handle unbalanced domains very well, and if you have an overly
  unbalanced memory setup whilst under high memory pressure, VM page allocation
  may fail leading to a kernel panic.  This was a problem in the past, but it's
  much more easily triggered now with these tools.

* This work only controls the path through vm_phys; it doesn't yet strongly/predictably
  affect contigmalloc, KVA placement, UMA, etc.  So, driver placement of memory
  isn't really guaranteed in any way.  That's next on my plate.

Sponsored by:	Norse Corp, Inc.; Dell
2015-07-11 15:21:37 +00:00
Warner Losh
32c88b5252 Remove old fmake. It wasn't built by default for some time. Users that
really need it can find it in the devel/fmake port or pkg install fmake.
Note: This commit is orthogonal to the question 'can we fmake buildworld'.

Differential Revision: https://reviews.freebsd.org/D2840
2015-06-16 20:58:33 +00:00
Baptiste Daroussin
8faa40d1a0 Switch to mandoc's version of makewhatis(8), whatis(1), apropos(1) utilities.
This change among other things improve search capabilities over the manpages
allowing fine grain query.

A new build option WITHOUT_MANDOCDB has been added to keep the ancient version
of the database and the tools. The plan is to entirely remove this option before
11.0-RELEASE.

Differential Revision:	https://reviews.freebsd.org/D2603
2015-05-30 17:41:37 +00:00
Baptiste Daroussin
7b0896d882 Remove demandoc(1) from the build 2015-05-21 13:02:43 +00:00
Baptiste Daroussin
363da13804 Replace groff's soelim by soeliminate(1) renamed soelim(1) 2015-05-01 20:08:25 +00:00
Ed Maste
0a06cffc15 Add ELF Tool Chain's c++filt to the build
Differential Revision:	https://reviews.freebsd.org/D2408
Reviewed by:	brooks, imp
Sponsored by:	The FreeBSD Foundation
2015-04-30 21:30:33 +00:00
Andrew Turner
da713b9ed2 Disable truss, gprof, and lint on arm64, they don't build.
Sponsored by:	The FreeBSD Foundation
2015-04-14 10:15:58 +00:00
Ed Maste
6744fea6b3 Switch to ELF toolchain readelf(1)
ELF toolchain readelf lacked some functionality at the time other tools
(like size, strip, nm, etc.) were switched over to the ELF toolchain
versions. That has been addressed as of the last update, so we can add
it to the list.

PR:		198950 [exp-run]
Reviewed by:	bapt, imp, rpaulo
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D2156
2015-03-30 17:38:25 +00:00
Warner Losh
2aa222b153 These local variables are unused. gc them. 2015-03-12 08:32:22 +00:00
Enji Cooper
c7b6816fa9 Add MK_FILE to control whether or not to build file(1), libmagic(3), etc
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-02-04 10:24:40 +00:00
Enji Cooper
11981695fc Add the following options to enable/disable several features in the base system
WITHOUT_BOOTPARAMD - bootparamd
WITHOUT_BOOTPD - bootpd
WITHOUT_FINGER - finger, fingerd
WITHOUT_FTP - ftp, ftpd
WITHOUT_INETD - inetd
WITHOUT_RBOOTD - rbootd
WITHOUT_TCP_WRAPPERS - tcpd, et al
WITHOUT_TFTP - tftp, tftp-server
WITHOUT_TIMED - timed

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-02-04 10:19:32 +00:00
Enji Cooper
3f802165ba Add MK_TALK knob for building the talk and talkd
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-25 04:37:44 +00:00
Enji Cooper
b29d6977f3 Add MK_ISCSI knob for building the iscsi initiator, iscsi daemon, kernel
modules, etc

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-25 04:20:11 +00:00
Enji Cooper
79a86dafb7 Add MK_EE knob to control installing edit, ee, etc
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-25 00:03:44 +00:00
Baptiste Daroussin
1fb816da82 Update to mandoc cvs version as of 20141201
- Compatiblity with existing manpages has been improved
- Now support ".so" directive with compressed manpages (which fixes a regression
we have since we have new man(1))
2014-12-02 23:24:57 +00:00