12267 Commits

Author SHA1 Message Date
bofh
1485828b08 Add bofh@ in calendar.freebsd 2016-04-10 16:48:21 +00:00
fanf
3389187e4b Handle whois referrals between RIRs
This is rather ugly, because the RIPE and APNIC whois servers do
not provide referrals for address blocks that they do not manage.
However ARIN is usually the right place or knows the right place
so we try there.

The particular instance which clued me in to this bug is U.Mich.
141.211.0.0/16 for which the referral chain should be IANA ->
RIPE -> ARIN. RIPE's RDAP does provide useful redirects (for
example try `curl -I http://rdap.db.ripe.net/ip/141.211.0.0)
so maybe their whois server can be improved.

AfriNIC's whois server gives more direct referrals, but they are
designed to be human-readable. Ugly, but we can manage.

The issue of referrals between RIRs is likely to become more important
in the future whith the increasing number of cross-region IP address
block transfers increases.
2016-04-08 11:43:22 +00:00
gahr
eb925cdf00 Enhance uuencode with a -r option to produce raw output.
This matches with uudecode's -r option to decode raw data without initial and
final framing lines.

$ echo Test | uuencode -mr - | uudecode -mr
Test

Approved by:	cognet
MFC after:		1 week
2016-04-07 16:12:38 +00:00
trasz
825d80e01c Add four new RCTL resources - readbps, readiops, writebps and writeiops,
for limiting disk (actually filesystem) IO.

Note that in some cases these limits are not quite precise. It's ok,
as long as it's within some reasonable bounds.

Testing - and review of the code, in particular the VFS and VM parts - is
very welcome.

MFC after:	1 month
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5080
2016-04-07 04:23:25 +00:00
pfg
753486d87a Fix sed functions 'i' and 'a' from discarding leading white space.
This appears to be implementation dependent but convenient and makes
our sed behave more like GNU sed.

Given that it is not the historic behavior, bump FreeBSD_version
should userland/ports somehow depend on it.

Obtained from:	NetBSD (bin/49872)

Reviewed by:	bdrewery
PR:		208554
Merge after:	NEVER
2016-04-06 00:55:39 +00:00
brooks
f447eee830 Add a cross reference to ktrace(2).
Obtained from:	CheriBSD (9cb420d6b7f04c1b7d2006180b80932e5d3fe50e)
MFC after:	1 week
Sponsored by:	DARPA, AFRL
2016-03-31 23:57:03 +00:00
fanf
b951056e8c Fix whois queries for ARIN AS numbers.
The ARIN whois server likes AS number queries to be in the form
"a 8075" rather than "as7085".
2016-03-31 12:13:01 +00:00
trasz
13af8fa343 Add phttpget(8) .Xr to fetch(1).
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-03-29 11:56:00 +00:00
pfg
a7966dbd97 netstat: avoid returning uninitialized value in p_sockaddr().
In the case the width is less than 0, we are returning an uninitialized
value. For practical purposes the return value is ignored but initialize
it to avoid trouble.

CID:	1341619
2016-03-27 20:02:21 +00:00
ed
dfa2acdc82 Replace the CloudABI system call table by a machine generated version.
The type definitions and constants that were used by COMPAT_CLOUDABI64
are a literal copy of some headers stored inside of CloudABI's C
library, cloudlibc. What is annoying is that we can't make use of
cloudlibc's system call list, as the format is completely different and
doesn't provide enough information. It had to be synced in manually.

We recently decided to solve this (and some other problems) by moving
the ABI definitions into a separate file:

	https://github.com/NuxiNL/cloudabi/blob/master/cloudabi.txt

This file is processed by a pile of Python scripts to generate the
header files like before, documentation (markdown), but in our case more
importantly: a FreeBSD system call table.

This change discards the old files in sys/contrib/cloudabi and replaces
them by the latest copies, which requires some minor changes here and
there. Because cloudabi.txt also enforces consistent names of the system
call arguments, we have to patch up a small number of system call
implementations to use the new argument names.

The new header files can also be included directly in FreeBSD kernel
space without needing any includes/defines, so we can now remove
cloudabi_syscalldefs.h and cloudabi64_syscalldefs.h. Patch up the
sources to include the definitions directly from sys/contrib/cloudabi
instead.
2016-03-24 21:47:15 +00:00
trasz
6fee958d28 Make the autofs(5) -hosts map more robust, primarily to make it correctly
handle NFS shares containing whitespace. This also adds the -E parameter
to showmount(8).

Reviewed by:	emaste@, jhibbits@, wblock@
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D5649
2016-03-23 12:13:53 +00:00
pfg
34220e27d8 localedef(1): minor sorting to match Illumos.
Illumos recently included space in 'print' class. We already had
this but the code had slight sorting differences. Move it some
lines up to reduce diffs with Illumos.

No functional change.

Reference:
https://illumos.org/issues/5227
2016-03-20 03:27:06 +00:00
grembo
9533d74078 Update fetch.1 and fetch.3 to reflect libfetch's actual use of CA bundles
Reviewed by:	wblock
Approved by:	wblock
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D5558
2016-03-19 11:55:21 +00:00
sjg
44636d2ebb Update to bmake-20160315
Archive member handling works again
meta mode, treat missing reads as for writes.

Update dirdeps.mk - much improved startup time.
Update meta.stage.mk - avoid ln when chmod required.
2016-03-18 20:03:09 +00:00
sjg
4ebe624823 xo_config.h no longer in contrib, so -I's needed
PR:		/homes/sjg/commit-logs/freebsd/libxo/xo_config.diff
Reviewed by:	jkim
2016-03-17 04:21:57 +00:00
emaste
9bc25213ea vtfontcvt: support .hex fonts with characters beyond the Unicode BMP
This is already supported by the vt(4) vfnt format mapping from code
points to glyphs. Update the .hex font parser to accept up to six hex
digits.
2016-03-15 21:32:46 +00:00
glebius
c39b2fd5d1 Print running TCP connection counts with TCP statistics. 2016-03-15 00:19:30 +00:00
lidl
33eb7d8e7f Do not truncate office phones in finger's summary listing
When finger is invoked as as "finger username", it produces the
long listing by default, and phones numbers are pretty-printed
by the prphone() function. When invoked as just "finger", the
same pretty-printing happens, but is truncated at 9 characters.
Given the summary listing is already greater than 80 columns,
making it even wider is of no harm.

Approved by:	rpaulo (mentor)
Differential Revision:	https://reviews.freebsd.org/D5638
2016-03-14 22:20:22 +00:00
sobomax
bd7cbc7f20 In the de-duplication mode, when found matching md5 checksum also read
back block and compare actual content. Just output original block
instead of back reference in the unlikely event of collision.
2016-03-13 21:09:08 +00:00
ian
5d2092c908 Fix a bug in bsdgrep that caused the program to hang in a tight loop for
some combinations of command line options and search patterns.  The code was
examining regexec flags looking for a regcomp flag value.  The fix is to
look in the struct field where the decoded regcomp flag was stored when the
regex was compiled.

With this fix, it's possible to build WITHOUT_GNU_GREP_COMPAT and
WITH_BSDGREP and have a usable GPL-free grep (which of course lacks gnugrep
extensions).  It now passes the kyua tests except for one test that requires
the -z/--null-data gnu extension, and one test involving outputting context
lines across multiple files which appears to sometimes output an extra
delimiter line ("--") between matches (a rather obscure failure of a rather
obscure feature, so bsdgrep should be generally usable now).
2016-03-13 14:53:12 +00:00
kib
c259148074 Fix handling of umtxp resource limit in sh(1)/ulimit(1), limits(1), add
login.conf(5) support.

Reviewed by:	jilles
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D5610
2016-03-12 14:54:34 +00:00
bdrewery
749836195e Fix upgrade of bmake by not setting conflicting MAKE_VERSION.
This may be used in later checks, such as in bsd.dep.mk, to
enable features that rely on the built-in value.

Sponsored by:	EMC / Isilon Storage Division
2016-03-11 04:09:50 +00:00
sjg
218c7825fe Merge bmake-20160307 2016-03-11 01:35:39 +00:00
sobomax
0e0b4ac1f0 When -S is specified dump summary to stdout, not stderr, so it's
easier to capture and process it with external tools via pipe.
2016-03-10 23:19:35 +00:00
sobomax
773a64f003 Add -S option to print out summary after compression has been
completed.

MFC after: 	2 weeks
2016-03-10 21:36:24 +00:00
bdrewery
aab40fdc3d DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
jhb
5ca736c477 Use ptrace(2) LWP events to track threads reliably in truss.
- truss can now log the system call invoked by a thread during a
  voluntary process exit.  No return value is logged, but the value passed
  to exit() is included in the trace output.  Arguments passed to thread
  exit system calls such as thr_exit() are not logged as voluntary thread
  exits cannot be distinguished from involuntary thread exits during a
  system call.
- New events are now reported for thread births and exits similar to the
  recently added events for new child processes when following forks.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D5561
2016-03-09 18:45:41 +00:00
jhb
e641458f70 Fix reporting of the CloudABI ABI in kdump.
- Advertise the word size for CloudABI ABIs via the SV_LP64 flag.  All of
  the other ABIs include either SV_ILP32 or SV_LP64.
- Fix kdump to not assume a 32-bit ABI if the ABI flags field is non-zero
  but SV_LP64 isn't set.  Instead, only assume a 32-bit ABI if SV_ILP32 is
  set and fallback to the unknown value of "00" if neither SV_LP64 nor
  SV_ILP32 is set.

Reviewed by:	kib, ed
Differential Revision:	https://reviews.freebsd.org/D5560
2016-03-09 18:38:30 +00:00
bdrewery
ca7bcc42dd DIRDEPS_BUILD: Update clang dependencies after r296417.
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 21:26:44 +00:00
bdrewery
1d619dcb69 Record command exit status in the typescript file when running simple commands.
Also capitalize 'command:'.

Relnotes:	yes
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 18:05:23 +00:00
bdrewery
49a860c3b1 Just exit in the child if execve(2) fails.
No functional change.

This is mostly addressing a false-positive from the clang static
analyzer due to it thinking that done() was being called with freed
memory, however the kill(0, SIGTERM) made the done() never reached.
It doesn't make sense to the show the footer from the child anyhow, nor
does it make sense to kill the process group here since the execve(2)
failed in the child.  This code was leftover from many years of refactoring.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 18:05:20 +00:00
bdrewery
f234bc3e54 Filemon: Attach from the child to avoid racing with the parent attach.
This is the same as how the bmake filemon usage works.

This also fixes failed attach not properly flushing the TTY.

MFC after:	1 week
Relnotes:	yes
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 18:05:02 +00:00
dim
e9ad9d5ff7 Upgrade our copies of clang, llvm, lldb and compiler-rt to 3.8.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang will soon be available here:
<http://llvm.org/releases/3.8.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.8.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Davide Italiano and Antoine Brodin
for their help.

Relnotes:	yes
2016-03-05 20:24:31 +00:00
bapt
338423c265 Fix "files" arguments 2016-03-05 15:53:06 +00:00
dim
9ac7ec8827 Merge ^/head r296007 through r296368. 2016-03-03 23:15:46 +00:00
dim
d50a252468 Regenerate llvm and clang manpages. 2016-03-03 23:09:57 +00:00
kib
e76eb4255b Implement process-shared locks support for libthr.so.3, without
breaking the ABI.  Special value is stored in the lock pointer to
indicate shared lock, and offline page in the shared memory is
allocated to store the actual lock.

Reviewed by:	vangyzen (previous version)
Discussed with:	deischen, emaste, jhb, rwatson,
	Martin Simmons <martin@lispworks.com>
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
bdrewery
eb3588c7b6 Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-02-26 22:14:15 +00:00
pfg
0c969eabba talk(1): Replace select(2) with poll(2)
Hinted by:	OpenBSD and NetBSD
2016-02-25 19:06:44 +00:00
oshogbo
023f14d65b Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with:		pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by:	drysdale@google.com, bdrewery
Approved by:		pjd (mentor)
Differential Revision:	https://reviews.freebsd.org/D4277
2016-02-25 18:23:40 +00:00
jhb
85f8ae969b Add simple support for CloudABI processes to kdump(1).
This does not decode arguments to system calls but should properly
decode system call names and error return values.

Reviewed by:	ed
Differential Revision:	https://reviews.freebsd.org/D5412
2016-02-25 17:43:32 +00:00
dim
7ef26dcde6 Merge ^/head r295902 through r296006. 2016-02-24 21:38:51 +00:00
bdrewery
2a891f1feb DIRDEPS_BUILD: Regenerate without local dependencies.
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:20:11 +00:00
bdrewery
b6e8f9c3c9 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:18:35 +00:00
bdrewery
56e259a8db Properly fix these builds by adding NO_WMISSING_VARIABLE_DECLARATIONS from r249657.
This reverts r284374.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:18:31 +00:00
sobomax
85ce861e46 Obsolete mkulzma(8) and geom_uncompress(4), their functionality
is now provided by mkuzip(8) and geom_uzip(4) respectively.

MFC after:	1 month
2016-02-24 00:39:36 +00:00
sobomax
8abe971b5e Improve mkuzip(8) and geom_uzip(4), merge in LZMA support from mkulzma(8)
and geom_uncompress(4):

1. mkuzip(8):

 - Proper support for eliminating all-zero blocks when compressing an
   image. This feature is already supported by the geom_uzip(4) module
   and CLOOP format in general, so it's just a matter of making mkuzip(8)
   match. It should be noted, however that this feature while it sounds
   great, results in very slight improvement in the overall compression
   ratio, since compressing default 16k all-zero block produces only 39
   bytes compressed output block, which is 99.8% compression ratio. With
   typical average compression ratio of amd64 binaries and data being
   around 60-70% the difference between 99.8% and 100.0% is not that
   great further diluted by the ratio of number of zero blocks in the
   uncompressed image to the overall number of blocks being less than
   0.5 (typically). However, this may be important from performance
   standpoint, so that kernel are not spinning its wheels decompressing
   those empty blocks every time this zero region is read. It could also
   be important when you create huge image mostly filled with zero
   blocks for testing purposes.

 - New feature allowing to de-duplicate output image. It turns out that
   if you twist CLOOP format a bit you can do that as well. And unlike
   zero-blocks elimination, this gives a noticeable improvement in the
   overall compression ratio, reducing output image by something like
   3-4% on my test UFS2 3GB image consisting of full FreeBSD base system
   plus some of the packages (openjdk, apache etc), about 2.3GB worth of
   file data (800+MB compressed). The only caveat is that images created
   with this feature "on" would not work on older versions of FeeBSDxi
   kernel, hence it's turned off by default.

 - provide options to control both features and document them in manual
   page.

 - merge in all relevant LZMA compression support from the mkulzma(8),
   add new option to select between both.

 - switch license from ad-hoc beerware into standard 2-clause BSD.

2. geom_uzip(4):

 - implement support for de-duplicated images;

 - optimize some code paths to handle "all-zero" blocks without reading
   any compressed data;

 - beef up manual page to explain that geom_uzip(4) is not limited only
   to md(4) images. The compressed data can be written to the block
   device and accessed directly via magic of GEOM(4) and devfs(4),
   including to mount root fs from a compressed drive.

 - convert debug log code from being compiled in conditionally into
   being present all the time and provide two sysctls to turn it on or
   off. Due to intended use of the module, it can be used in
   environments where there may not be a luxury to put new kernel with
   debug code enabled. Having those options handy allows debug issues
   without as much problem by just having access to serial console or
   network shell access to a box/appliance. The resulting additional
   CPU cycles are just few int comparisons and branches, and those are
   minuscule when compared to data decompression which is the main
   feature of the module.

 - hopefully improve robustness and resiliency of the geom_uzip(4) by
   performing some of the data validation / range checking on the TOC
   entries and rejecting to attach to an image if those checks fail.

 - merge in all relevant LZMA decompression support from the
   geom_uncompress(4), enable automatically when appropriate format is
   indicated in the header.

 - move compilation work into its own worker thread so that it does not
   clog g_up. This allows multiple instances work in parallel utilizing
   smp cores.

 - document new knobs in the manual page.

Reviewed by:		adrian
MFC after:		1 month
Differential Revision:	https://reviews.freebsd.org/D5333
2016-02-23 23:59:08 +00:00
jhb
ca28498877 Add handling for non-native error values to libsysdecode.
Add two new functions, sysdecode_abi_to_freebsd_errno() and
sysdecode_freebsd_to_abi_errno(), which convert errno values between
the native FreeBSD ABI and other supported ABIs. Note that the
mappings are not necessarily perfect meaning in some cases multiple
errors in one ABI might map to a single error in another ABI. In that
case, the reverse mapping will return one of the errors that maps, but
which error is non-deterministic.

Change truss to always report the raw error value to the user but
use libsysdecode to map it to a native errno value that can be used
with strerror() to generate a description. Previously truss reported
the "converted" error value. Now the user will always see the exact
error value that the application sees.

Change kdump to report the truly raw error value to the user. Previously
kdump would report the absolute value of the raw error value (so for
Linux binaries it didn't output the FreeBSD error value, but the positive
value of the Linux error). Now it reports the real (i.e. negative) error
value for Linux binaries. Also, use libsysdecode to convert the native
FreeBSD error reported in the ktrace record to the raw error used by the
ABI. This means that the Linux ABI can now be handled directly in
ktrsysret() and removes the need for linux_ktrsysret().

Reviewed by:	bdrewery, kib
Helpful notes:	wblock (manpage)
Differential Revision:	https://reviews.freebsd.org/D5314
2016-02-23 20:00:55 +00:00
jhb
dbabd32553 Add support for displaying thread IDs to truss(1).
- Consolidate duplicate code for printing the metadata at the start of
  each line into a shared function.
- Add an -H option which will log the thread ID of the relevant thread
  for each event.

While here, remove some extraneous calls to clock_gettime() in
print_syscall() and print_syscall_ret().  The caller of print_syscall_ret()
always updates the current thread's "after" time before it is called.

Reviewed by:	kib
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D5363
2016-02-23 19:56:29 +00:00
dim
2e1a0cbbd8 Merge ^/head r295601 through r295844. 2016-02-21 13:49:26 +00:00