Commit Graph

12191 Commits

Author SHA1 Message Date
Bryan Drewery
80a5408c49 Fix upgrade of bmake by not setting conflicting MAKE_VERSION.
This may be used in later checks, such as in bsd.dep.mk, to
enable features that rely on the built-in value.

Sponsored by:	EMC / Isilon Storage Division
2016-03-11 04:09:50 +00:00
Simon J. Gerraty
be19d90b61 Merge bmake-20160307 2016-03-11 01:35:39 +00:00
Maxim Sobolev
62ee4b69cd When -S is specified dump summary to stdout, not stderr, so it's
easier to capture and process it with external tools via pipe.
2016-03-10 23:19:35 +00:00
Maxim Sobolev
d83e07789d Add -S option to print out summary after compression has been
completed.

MFC after: 	2 weeks
2016-03-10 21:36:24 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
John Baldwin
b9befd33b4 Use ptrace(2) LWP events to track threads reliably in truss.
- truss can now log the system call invoked by a thread during a
  voluntary process exit.  No return value is logged, but the value passed
  to exit() is included in the trace output.  Arguments passed to thread
  exit system calls such as thr_exit() are not logged as voluntary thread
  exits cannot be distinguished from involuntary thread exits during a
  system call.
- New events are now reported for thread births and exits similar to the
  recently added events for new child processes when following forks.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D5561
2016-03-09 18:45:41 +00:00
John Baldwin
6fc8053f1a Fix reporting of the CloudABI ABI in kdump.
- Advertise the word size for CloudABI ABIs via the SV_LP64 flag.  All of
  the other ABIs include either SV_ILP32 or SV_LP64.
- Fix kdump to not assume a 32-bit ABI if the ABI flags field is non-zero
  but SV_LP64 isn't set.  Instead, only assume a 32-bit ABI if SV_ILP32 is
  set and fallback to the unknown value of "00" if neither SV_LP64 nor
  SV_ILP32 is set.

Reviewed by:	kib, ed
Differential Revision:	https://reviews.freebsd.org/D5560
2016-03-09 18:38:30 +00:00
Bryan Drewery
f51eac06ce DIRDEPS_BUILD: Update clang dependencies after r296417.
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 21:26:44 +00:00
Bryan Drewery
66d93438b1 Record command exit status in the typescript file when running simple commands.
Also capitalize 'command:'.

Relnotes:	yes
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 18:05:23 +00:00
Bryan Drewery
95d13d1b02 Just exit in the child if execve(2) fails.
No functional change.

This is mostly addressing a false-positive from the clang static
analyzer due to it thinking that done() was being called with freed
memory, however the kill(0, SIGTERM) made the done() never reached.
It doesn't make sense to the show the footer from the child anyhow, nor
does it make sense to kill the process group here since the execve(2)
failed in the child.  This code was leftover from many years of refactoring.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 18:05:20 +00:00
Bryan Drewery
7b245cb846 Filemon: Attach from the child to avoid racing with the parent attach.
This is the same as how the bmake filemon usage works.

This also fixes failed attach not properly flushing the TTY.

MFC after:	1 week
Relnotes:	yes
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 18:05:02 +00:00
Dimitry Andric
e74e149f78 Upgrade our copies of clang, llvm, lldb and compiler-rt to 3.8.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang will soon be available here:
<http://llvm.org/releases/3.8.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.8.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Davide Italiano and Antoine Brodin
for their help.

Relnotes:	yes
2016-03-05 20:24:31 +00:00
Baptiste Daroussin
99ed5c6e34 Fix "files" arguments 2016-03-05 15:53:06 +00:00
Dimitry Andric
82aa34e6fa Merge ^/head r296007 through r296368. 2016-03-03 23:15:46 +00:00
Dimitry Andric
ada554fc20 Regenerate llvm and clang manpages. 2016-03-03 23:09:57 +00:00
Konstantin Belousov
1bdbd70599 Implement process-shared locks support for libthr.so.3, without
breaking the ABI.  Special value is stored in the lock pointer to
indicate shared lock, and offline page in the shared memory is
allocated to store the actual lock.

Reviewed by:	vangyzen (previous version)
Discussed with:	deischen, emaste, jhb, rwatson,
	Martin Simmons <martin@lispworks.com>
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
Bryan Drewery
3055a28d44 Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-02-26 22:14:15 +00:00
Pedro F. Giffuni
7880fc113f talk(1): Replace select(2) with poll(2)
Hinted by:	OpenBSD and NetBSD
2016-02-25 19:06:44 +00:00
Mariusz Zaborski
c501d73c7e Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with:		pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by:	drysdale@google.com, bdrewery
Approved by:		pjd (mentor)
Differential Revision:	https://reviews.freebsd.org/D4277
2016-02-25 18:23:40 +00:00
John Baldwin
6fb8946b7f Add simple support for CloudABI processes to kdump(1).
This does not decode arguments to system calls but should properly
decode system call names and error return values.

Reviewed by:	ed
Differential Revision:	https://reviews.freebsd.org/D5412
2016-02-25 17:43:32 +00:00
Dimitry Andric
14e9c9161c Merge ^/head r295902 through r296006. 2016-02-24 21:38:51 +00:00
Bryan Drewery
bd18fd57db DIRDEPS_BUILD: Regenerate without local dependencies.
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:20:11 +00:00
Bryan Drewery
137e183ff3 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:18:35 +00:00
Bryan Drewery
9f93cfc34b Properly fix these builds by adding NO_WMISSING_VARIABLE_DECLARATIONS from r249657.
This reverts r284374.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:18:31 +00:00
Maxim Sobolev
5497acc527 Obsolete mkulzma(8) and geom_uncompress(4), their functionality
is now provided by mkuzip(8) and geom_uzip(4) respectively.

MFC after:	1 month
2016-02-24 00:39:36 +00:00
Maxim Sobolev
8f8cb840b0 Improve mkuzip(8) and geom_uzip(4), merge in LZMA support from mkulzma(8)
and geom_uncompress(4):

1. mkuzip(8):

 - Proper support for eliminating all-zero blocks when compressing an
   image. This feature is already supported by the geom_uzip(4) module
   and CLOOP format in general, so it's just a matter of making mkuzip(8)
   match. It should be noted, however that this feature while it sounds
   great, results in very slight improvement in the overall compression
   ratio, since compressing default 16k all-zero block produces only 39
   bytes compressed output block, which is 99.8% compression ratio. With
   typical average compression ratio of amd64 binaries and data being
   around 60-70% the difference between 99.8% and 100.0% is not that
   great further diluted by the ratio of number of zero blocks in the
   uncompressed image to the overall number of blocks being less than
   0.5 (typically). However, this may be important from performance
   standpoint, so that kernel are not spinning its wheels decompressing
   those empty blocks every time this zero region is read. It could also
   be important when you create huge image mostly filled with zero
   blocks for testing purposes.

 - New feature allowing to de-duplicate output image. It turns out that
   if you twist CLOOP format a bit you can do that as well. And unlike
   zero-blocks elimination, this gives a noticeable improvement in the
   overall compression ratio, reducing output image by something like
   3-4% on my test UFS2 3GB image consisting of full FreeBSD base system
   plus some of the packages (openjdk, apache etc), about 2.3GB worth of
   file data (800+MB compressed). The only caveat is that images created
   with this feature "on" would not work on older versions of FeeBSDxi
   kernel, hence it's turned off by default.

 - provide options to control both features and document them in manual
   page.

 - merge in all relevant LZMA compression support from the mkulzma(8),
   add new option to select between both.

 - switch license from ad-hoc beerware into standard 2-clause BSD.

2. geom_uzip(4):

 - implement support for de-duplicated images;

 - optimize some code paths to handle "all-zero" blocks without reading
   any compressed data;

 - beef up manual page to explain that geom_uzip(4) is not limited only
   to md(4) images. The compressed data can be written to the block
   device and accessed directly via magic of GEOM(4) and devfs(4),
   including to mount root fs from a compressed drive.

 - convert debug log code from being compiled in conditionally into
   being present all the time and provide two sysctls to turn it on or
   off. Due to intended use of the module, it can be used in
   environments where there may not be a luxury to put new kernel with
   debug code enabled. Having those options handy allows debug issues
   without as much problem by just having access to serial console or
   network shell access to a box/appliance. The resulting additional
   CPU cycles are just few int comparisons and branches, and those are
   minuscule when compared to data decompression which is the main
   feature of the module.

 - hopefully improve robustness and resiliency of the geom_uzip(4) by
   performing some of the data validation / range checking on the TOC
   entries and rejecting to attach to an image if those checks fail.

 - merge in all relevant LZMA decompression support from the
   geom_uncompress(4), enable automatically when appropriate format is
   indicated in the header.

 - move compilation work into its own worker thread so that it does not
   clog g_up. This allows multiple instances work in parallel utilizing
   smp cores.

 - document new knobs in the manual page.

Reviewed by:		adrian
MFC after:		1 month
Differential Revision:	https://reviews.freebsd.org/D5333
2016-02-23 23:59:08 +00:00
John Baldwin
287b96dd25 Add handling for non-native error values to libsysdecode.
Add two new functions, sysdecode_abi_to_freebsd_errno() and
sysdecode_freebsd_to_abi_errno(), which convert errno values between
the native FreeBSD ABI and other supported ABIs. Note that the
mappings are not necessarily perfect meaning in some cases multiple
errors in one ABI might map to a single error in another ABI. In that
case, the reverse mapping will return one of the errors that maps, but
which error is non-deterministic.

Change truss to always report the raw error value to the user but
use libsysdecode to map it to a native errno value that can be used
with strerror() to generate a description. Previously truss reported
the "converted" error value. Now the user will always see the exact
error value that the application sees.

Change kdump to report the truly raw error value to the user. Previously
kdump would report the absolute value of the raw error value (so for
Linux binaries it didn't output the FreeBSD error value, but the positive
value of the Linux error). Now it reports the real (i.e. negative) error
value for Linux binaries. Also, use libsysdecode to convert the native
FreeBSD error reported in the ktrace record to the raw error used by the
ABI. This means that the Linux ABI can now be handled directly in
ktrsysret() and removes the need for linux_ktrsysret().

Reviewed by:	bdrewery, kib
Helpful notes:	wblock (manpage)
Differential Revision:	https://reviews.freebsd.org/D5314
2016-02-23 20:00:55 +00:00
John Baldwin
d70876fd7e Add support for displaying thread IDs to truss(1).
- Consolidate duplicate code for printing the metadata at the start of
  each line into a shared function.
- Add an -H option which will log the thread ID of the relevant thread
  for each event.

While here, remove some extraneous calls to clock_gettime() in
print_syscall() and print_syscall_ret().  The caller of print_syscall_ret()
always updates the current thread's "after" time before it is called.

Reviewed by:	kib
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D5363
2016-02-23 19:56:29 +00:00
Dimitry Andric
9893f787ec Merge ^/head r295601 through r295844. 2016-02-21 13:49:26 +00:00
Stefan Eßer
2919c53c63 Remove O_SYNC from the options passed to dbmopen().
The output file is created as a temporary file that is moved over the
existing file after completion. Thus there is no need to immediately
flush all created db records to the temporary file.

This speeds up creation of the termcap db by a factor of 40 on my
ZFS based /etc filesytem (from 25 seconds to 0.6 seconds).

I have compared multiple output files created with and without O_SYNC
and they came out identical each time.  Nonetheless it might be best
to MFC this change and the similar one for services_mkdb (r295465) at
the same time when the changes to hash.c in review D5186 are merged.

MFC:	 1 week
2016-02-19 08:42:13 +00:00
Bryan Drewery
af96ced5c5 DIRDEPS_BUILD: Add dependencies somehow missed in r295681.
Sponsored by:	EMC / Isilon Storage Division
2016-02-19 00:41:38 +00:00
Bryan Drewery
24e9c818fa Fix installation of makewhatis.local(1) since r283777.
The wrapper script has moved to libexec/makewhatis.local since it is not
directlry related to the older makewhatis(1) utility that has been replaced
by the usr.bin/mandoc version.

Reported by:	vangyzen
2016-02-18 00:26:21 +00:00
Bryan Drewery
6b128c1707 DIRDEPS_BUILD: Hookup CLANG_EXTRAS.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 23:11:09 +00:00
John Baldwin
f8ed4050d5 Remove an unused variable that snuck into the previous revision. 2016-02-16 22:13:25 +00:00
John Baldwin
00ddbdf2ee Fetch the current thread and it's syscall state from the trussinfo object
instead of passing some of that state as arguments to print_syscall() and
print_syscallret().  This just makes the calls of these functions shorter
and easier to read.
2016-02-16 22:00:01 +00:00
Pedro F. Giffuni
0bfdca975c Fix naive use of ftell(3).
Secure coding practices, FIO19-C.
2016-02-16 21:39:19 +00:00
Olivier Cochard
f53fab99ae Add myself as ports committer (including calendar)
Approved by:	jadawin
Differential Revision:	https://reviews.freebsd.org/D5295
2016-02-16 10:33:45 +00:00
Bryan Drewery
9d8a81b4e4 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 02:14:30 +00:00
Bryan Drewery
d3157f0915 Test directories can build in parallel fine.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 02:13:59 +00:00
John Baldwin
623277619d Correct the ABI name for Linux/i386 binaries under FreeBSD/i386.
This allows truss to work for these binaries again after r288424.

MFC after:	3 days
2016-02-15 20:27:16 +00:00
John Baldwin
142482cda6 Sign extend the error value for failing Linux/i386 system calls. This
restores the mapping of Linux errors to native FreeBSD errno values after
the refactoring in r288424.
2016-02-15 20:26:40 +00:00
Kevin Lo
daa1a379ce Fix a bug that caused nothing to be skipped when skipping exactly the
number of bytes present in a regular file was requested.

Obtained from:	OpenBSD
2016-02-14 14:23:56 +00:00
Dimitry Andric
90a4726375 Merge ^/head r295544 through r295600. 2016-02-13 16:02:12 +00:00
Ed Maste
715f438b42 elfcopy: enable PE and EFI support
Sponsored by:	The FreeBSD Foundation
2016-02-12 21:15:31 +00:00
Dimitry Andric
4156ce4fed Merge ^/head r295351 through r295543. 2016-02-11 20:07:09 +00:00
Konstantin Belousov
78b7ef01a8 Rename variable to reflect the condition.
Suggested by:	jhb
Sponsored by:	The FreeBSD Foundation
2016-02-09 18:35:37 +00:00
Konstantin Belousov
db57c70a5b Rename P_KTHREAD struct proc p_flag to P_KPROC.
I left as is an apparent bug in ntoskrnl_var.h:AT_PASSIVE_LEVEL()
definition.

Suggested by:	jhb
Sponsored by:	The FreeBSD Foundation
2016-02-09 16:30:16 +00:00
Dimitry Andric
a49d8b6ecc Merge ^/head r294961 through r295350. 2016-02-06 14:07:17 +00:00
Alfred Perlstein
7325dfbb59 Increase max allowed backlog for listen sockets
from short to int.

PR: 203922
Submitted by: White Knight <white_knight@2ch.net>
MFC After: 4 weeks
2016-02-02 05:57:59 +00:00
John Baldwin
5842bd683f Add a SYSDECODE_ABI_ prefix to the ABI enums to avoid potential collisions.
Suggested by:	jmallett
Reviewed by:	bdrewery, jmallett
Differential Revision:	https://reviews.freebsd.org/D5123
2016-01-30 01:00:54 +00:00