Commit Graph

1601 Commits

Author SHA1 Message Date
gjb
086e6f562f MFH
Sponsored by:	The FreeBSD Foundation
2016-03-14 18:54:29 +00:00
bdrewery
88db8d87c1 META_MODE: Simplify the META_COOKIE handling to use .USE/.USEBEFORE.
Extend it to other cases of meta mode cookies so they get the proper rm
cookie behavior when a .meta file detects it needs to rebuild and fails.

Sponsored by:	EMC / Isilon Storage Division
2016-03-11 23:45:28 +00:00
bdrewery
3e01f13c50 DIRDEPS_BUILD: None of this is needed anymore.
This file is using stage-install, so all of the .dirdep files
are properly handled.  The cookie handling also properly
handles rebuilds with .meta files.  DESTDIR from bsd.sys.mk is also
respected for staging.  This logic came in r239572.

Sponsored by:	EMC / Isilon Storage Division
2016-03-11 23:44:56 +00:00
bdrewery
24b7e3c54f DIRDEPS_BUILD: Remove the cookie when target is out-of-date.
The meta file may decide the target is out of date but nothing
ensures that the *next* build will build this target if it
fails this time for some reason; it is still out-of-date
until it succeeds.

Convert the include/ cookie usage to the global versions.

Sponsored by:	EMC / Isilon Storage Division
2016-03-11 23:44:27 +00:00
gjb
f4bd2481be MFH
Sponsored by:	The FreeBSD Foundation
2016-03-06 04:13:17 +00:00
pfg
fe88d86157 xdr: Fix xdr_rpc* defines.
The defines for xdr_rpc* in xdr.h are wrong. It could be
very well that Solaris did strip the '_t' from xdr_u_int32_t,
but Solaris has a xdr_u_int32 function, we don't have this.
So all of this defines will lead to an unresolved symbol.

This explains why we do not use these functions in FreeBSD
while they are used in Illumos/Solaris.

Obtained from:	linux libtirpc (git 7864122e61ffe4db1aa8ace89117358a1e3a391b)
MFC after:	3 weeks
2016-03-04 22:03:38 +00:00
pfg
d07921367d Add sunrpc compat define for xp_sock.
SunRPC is using xp_sock in SVCXPRT, while TI-RPC is using
xp_fd. Add a compatibility define.

Illumos has something similar for the non-kernel case.

Obtained from: linux-nfs project (git 0d94036c3a0d4c24d22bf6a8c40ac6625d972c29)
2016-03-03 14:44:30 +00:00
gjb
955ce29ea3 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
jasone
ac01d0e42d Update jemalloc to 4.1.0.
Add missing Symbol.map entry for __aligned_alloc.

Add weak-->strong symbol binding for
{malloc_stats_print,mallctl,mallctlnametomib,mallctlbymib} -->
{__malloc_stats_print,__mallctl,__mallctlnametomib,__mallctlbymib}.  These
bindings complete the set necessary to allow applications to replace all
malloc-related symbols.
2016-02-29 19:10:32 +00:00
kib
e76eb4255b Implement process-shared locks support for libthr.so.3, without
breaking the ABI.  Special value is stored in the lock pointer to
indicate shared lock, and offline page in the shared memory is
allocated to store the actual lock.

Reviewed by:	vangyzen (previous version)
Discussed with:	deischen, emaste, jhb, rwatson,
	Martin Simmons <martin@lispworks.com>
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2016-02-28 17:52:33 +00:00
pfg
d7f6887db1 RPC: update the getrpcbyname() definition to include a const qualifier.
Add const qualifier making getrpcbyname() and getrpcbyname_r()
prototypes match those used in latest Sun RPC code (TI-RPC 2.3).

Obtained from:	NetBSD
2016-02-26 23:25:21 +00:00
gjb
64f34d0d71 MFH
Sponsored by:	The FreeBSD Foundation
2016-02-15 21:58:52 +00:00
kib
3275ec6409 Hide ucontext_t and mcontext_t when neither POSIX nor XSI features are
enabled in the compilation environment, i.e. for ANSI C use of
#include <signal.h>.

Requested and reviewed by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	13 days
2016-02-13 17:23:55 +00:00
kib
a05a278552 POSIX states that #include <signal.h> shall make both mcontext_t and
ucontext_t available.  Our code even has XXX comment about this.

Add a bit of compliance by moving struct __ucontext definition into
sys/_ucontext.h and including it into signal.h and sys/ucontext.h.

Several machine/ucontext.h headers were changed to use namespace-safe
types (like uint64_t->__uint64_t) to not depend on sys/types.h.
struct __stack_t from sys/signal.h is made always visible in private
namespace to satisfy sys/_ucontext.h requirements.

Apparently mips _types.h pollutes global namespace with f_register_t
type definition.  This commit does not try to fix the issue.

PR:	207079
Reported and tested by:	Ting-Wei Lan <lantw44@gmail.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2016-02-12 07:38:19 +00:00
gjb
225942335d Ensure include/ is properly tagged in the METALOG.
Noticed by:	des
Sponsored by:	The FreeBSD Foundation
2016-02-08 20:21:07 +00:00
glebius
40ba1ae95a Provide new socket option TCP_CCALGOOPT, which stands for TCP congestion
control algorithm options.  The argument is variable length and is opaque
to TCP, forwarded directly to the algorithm's ctl_output method.

Provide new includes directory netinet/cc, where algorithm specific
headers can be installed.

The new API doesn't yet have any in tree consumers.

The original code written by lstewart.
Reviewed by:	rrs, emax
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D711
2016-01-22 02:07:48 +00:00
kevlo
b67974b0be - Add the definition of CHARCLASS_NAME_MAX, as per POSIX.1-2001.
- Avoid namespace pollution and move definitions of _POSIX2_CHARCLASS_NAME_MAX
  and _POSIX2_COLL_WEIGHTS_MAX into the .2001 section.
  With input from bde.

Submitted by	bde
2016-01-09 14:53:23 +00:00
jilles
14b7155b88 Add sbin and /usr/local directories to _PATH_DEFPATH.
Set _PATH_DEFPATH to
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin. This is the
path in the default class in the default /etc/login.conf,
excluding ~/bin which would not be expanded properly in a string
constant.

For normal logins, _PATH_DEFPATH is overridden by /etc/login.conf,
~/.login_conf or shell startup files. _PATH_DEFPATH is still used as a
default by execlp(), execvp(), posix_spawnp() and sh if PATH is not set, and
by cron. Especially the latter is a common trap (most recently in PR
204813).

PR:		204813
Reviewed by:	secteam (delphij), alfred
2016-01-05 16:21:20 +00:00
ed
f846ec41eb Replace implementation of hsearch() by one that scales.
Traditionally the hcreate() function creates a hash table that uses
chaining, using a fixed user-provided size. The problem with this
approach is that this often either wastes memory (table too big) or
yields bad performance (table too small). For applications it may not
always be easy to estimate the right hash table size. A fixed number
only increases performance compared to a linked list by a constant
factor.

This problem can be solved easily by dynamically resizing the hash
table. If the size of the hash table is at least doubled, this has no
negative on the running time complexity. If a dynamically sized hash
table is used, we can also switch to using open addressing instead of
chaining, which has the advantage of just using a single allocation for
the entire table, instead of allocating many small objects.

Finally, a problem with the existing implementation is that its
deterministic algorithm for hashing makes it possible to come up with
fixed patterns to trigger an excessive number of collisions. We can
easily solve this by using FNV-1a as a hashing algorithm in combination
with a randomly generated offset basis.

Measurements have shown that this implementation is about 20-25% faster
than the existing implementation (even if the existing implementation is
given an excessive number of buckets). Though it allocates more memory
through malloc() than the old implementation (between 4-8 pointers per
used entry instead of 3), process memory use is similar to the old
implementation as if the estimated size was underestimated by a factor
10. This is due to the fact that malloc() needs to perform less
bookkeeping.

Reviewed by:	jilles, pfg
Obtained from:	https://github.com/NuxiNL/cloudlibc
Differential Revision:	https://reviews.freebsd.org/D4644
2015-12-27 07:50:11 +00:00
ed
4fec3a8161 Let tsearch()/tdelete() use an AVL tree.
The existing implementations of POSIX tsearch() and tdelete() don't
attempt to perform any balancing at all. Testing reveals that inserting
100k nodes into a tree sequentially takes approximately one minute on my
system.

Though most other BSDs also don't use any balanced tree internally, C
libraries like glibc and musl do provide better implementations. glibc
uses a red-black tree and musl uses an AVL tree.

Red-black trees have the advantage over AVL trees that they only require
O(1) rotations after insertion and deletion, but have the disadvantage
that the tree has a maximum depth of 2*log2(n) instead of 1.44*log2(n).
My take is that it's better to focus on having a lower maximum depth,
for the reason that in the case of tsearch() the invocation of the
comparator likely dominates the running time.

This change replaces the tsearch() and tdelete() functions by versions
that create an AVL tree. Compared to musl's implementation, this version
is different in two different ways:

- We don't keep track of heights; just balances. This is sufficient.
  This has the advantage that it reduces the number of nodes that are
  being accessed. Storing heights requires us to also access all of the
  siblings along the path.

- Don't use any recursion at all. We know that the tree cannot 2^64
  elements in size, so the height of the tree can never be larger than
  96. Use a 128-bit bitmask to keep track of the path that is computed.
  This allows us to iterate over the same path twice, meaning we can
  apply rotations from top to bottom.

Inserting 100k nodes into a tree now only takes 0.015 seconds. Insertion
seems to be twice as fast as glibc, whereas deletion has about the same
performance. Unlike glibc, it uses a fixed amount of memory.

I also experimented with both recursive and iterative bottom-up
implementations of the same algorithm. This iterative top-down version
performs similar to the recursive bottom-up version in terms of speed
and code size.

For some reason, the iterative bottom-up algorithm was actually 30%
faster for deletion, but has a quadratic memory complexity to keep track
of all the parent pointers.

Reviewed by:	jilles
Obtained from:	https://github.com/NuxiNL/cloudlibc
Differential Revision:	https://reviews.freebsd.org/D4412
2015-12-22 18:12:11 +00:00
ume
33741c0d9a addrinfo.ai_family is an address family, not a protocol family.
PR:		162434
MFC after:	1 week
2015-12-20 15:18:50 +00:00
ume
658c0fc856 Add AI_V4MAPPED and AI_ALL support for getaddrinfo(3).
We need to change netdb.h to make it actually enabled.

PR:		198092
MFC after:	1 week
2015-12-18 18:08:53 +00:00
vangyzen
7a758347d4 resolver: preserve binary compatibility; reduce header pollution
In r289315, I added new fields to res_state.  This broke binary
backward compatibility.  It also broke some ports (and possibly
other code) by requiring the definition of time_t and struct timespec.

Fix these problems by moving the new fields into __res_state_ext.

Suggested by:	ume
Reviewed by:	ume
MFC after:	3 days
Sponsored by:	Dell Inc.
Differential Revision:	https://reviews.freebsd.org/D4472
2015-12-14 17:21:06 +00:00
vangyzen
81148bc373 resolver: fix the build of some ports, broken by r289315
r289315 required time_t and struct timespec to be defined before
including <resolv.h>.  This broke the build of net-mgmt/sx, at least.

Include <sys/timespec.h> in resolv.h to fix this with minimal pollution.

Reported by:	Raphael Kubo da Costa <rakuco>
MFC after:	3 days
Sponsored by:	Dell Inc.
2015-12-08 16:09:48 +00:00
bdrewery
38174a7667 Move obscure lib/ installation of /usr/lib/include symlink to include/.
This avoids the need for an afterinstall: hook and a check for LIBRARIES_ONLY.
It also now respects INCLUDEDIR.

This came in r249484.

Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:18:02 +00:00
bdrewery
4316a3fffd Replace ln -s calls with INSTALL_SYMLINK
Sponsored by:	EMC / Isilon Storage Division
2015-12-04 03:17:14 +00:00
bdrewery
b0f65ffcc8 Stop building vers.c in include/ and only build the needed osreldate.h.
Because of how osreldate.h was being built with newvers.sh, which always
spat out a vers.c dependent on SVN or git, the meta mode build was
considering osreldate.h to depend on the current git or SVN index.  This
would lead to entire tree rebuilds when modifying git's index.  There's
no reason to be generating vers.c here so just skip it.

While here, in mk-osreldate.sh rename PARAM_H to proper PARAMFILE (which
newvers.sh already has a default for) and remove unneeded export.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:10:59 +00:00
bdrewery
e891ac8c59 META MODE: Fix changing what "MACHINE=host" means when computing dirdeps for include/.
The _SKIP_BUILD is used while computing DIRDEPS.  If MACHINE=host is passed in
then this logic was replacing 'MACHINE' with a literal value of the host arch,
which then caused the dirdeps graph to be wrong since it no longer had the
literal 'host' for any of include's dependencies.

This is a NOP currently since include/ is not usually built with MACHINE=host.

Sponsored by:	EMC / Isilon Storage Division
2015-11-18 21:39:58 +00:00
sjg
206f180a1a Rename META_MODE option to DIRDEPS_BUILD
This allows META_FILES option to be renamed META_MODE.
Also add META_COOKIE_TOUCH for use in targets that can benefit
from a cookie when in meta mode.

Differential Revision:	https://reviews.freebsd.org/D4153
Reviewed by:	bdrewery
2015-11-14 03:24:48 +00:00
bapt
41e30bab0f Merge from head r290483 2015-11-07 11:02:33 +00:00
pfg
e73a610f27 Rename __sentinel to __null_sentinel
GCC 5 uses a conflicting __sentinel definition in include/c++/bits/stl_algo.h

Reported by:	matteo
2015-11-05 14:55:58 +00:00
bapt
1477bec6db Merge from head 2015-11-01 21:17:38 +00:00
ache
64dac23af8 Add _flags2 per jhb@ suggestion since no room left in _flags.
Rewrite O_APPEND flag checking using new __S2OAP flag.

MFC after:      3 weeks
2015-10-28 14:40:02 +00:00
bapt
d93a511528 Merge from head 2015-10-19 11:51:10 +00:00
bdrewery
153205e33f Add more SUBDIR_PARALLEL.
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:55:08 +00:00
vangyzen
60458e70da resolver: automatically reload /etc/resolv.conf
On each resolver query, use stat(2) to see if the modification time
of /etc/resolv.conf has changed.  If so, reload the file and reinitialize
the resolver library.  However, only call stat(2) if at least two seconds
have passed since the last call to stat(2), since calling it on every
query could kill performance.

This new behavior is enabled by default.  Add a "reload-period" option
to disable it or change the period of the test.

Document this behavior and option in resolv.conf(5).

Polish the man page just enough to appease igor.

https://lists.freebsd.org/pipermail/freebsd-arch/2015-October/017342.html

Reviewed by:	kp, wblock
Discussed with:	jilles, imp, alfred
MFC after:	1 month
Relnotes:	yes
Sponsored by:	Dell Inc.
Differential Revision:	https://reviews.freebsd.org/D3867
2015-10-14 14:26:44 +00:00
bapt
979669d923 Commit log from Dragonfly:
FreeBSD extended ctypes to include numbers (e.g. isnumber()) but never
actually implemented it.  The isnumber() function was equivalent to the
isdigit() function in every case.

Now that DragonFly's ctype source files have number definitions, the
number ctype can finally be implemented.  It's given a new flag _CTYPE_N.
The isalnum() and iswalnum() functions have been changed to use this
flag rather than the _CTYPE_D digit flag.

While isalnum(), isnumber(), and their wide equivalents now return
different values in locale cases, the ishexnumber() and iswhexnumber()
functions are unchanged.  They are still aliases for isxdigit() and
iswxdigit().

Also change ctype.h for isdigit and isxdigit to use sbistype like the
other functions.

Obtained from:	dragonfly
2015-10-13 20:43:49 +00:00
bapt
c8d6d4a785 Merge from head 2015-10-13 19:44:36 +00:00
bdrewery
454d8f80ce Mute this cookie as well 2015-10-03 17:28:46 +00:00
bdrewery
134cb4f827 Avoid make compatibility mode issues with creating cookies from r287844 and r287848.
Also hide the cookie creation.

Suggested by:	imp, Daniel O'Connor
2015-10-03 16:09:55 +00:00
bdrewery
28bd003b30 META_MODE: Avoid command changing in 2nd build.
If the command to be ran changes then a rebuild is caused. Checking
exists(${DESTDIR}...) from make results in this on the 2nd and
possibly subsequent builds due to staging during build.  Avoid this
by always running the existence check in the make sh command.

Sponsored by:	EMC / Isilon Storage Division
2015-09-18 21:36:29 +00:00
bdrewery
fa4fb8209b Similar to r287844, create 'symlinks' cookie in proper place with -j and META_MODE.
Sponsored by:	EMC / Isilon Storage Division
2015-09-16 04:27:12 +00:00
bdrewery
36977b7e7a Create 'copies' cookie in proper place in META_MODE.
With -j the cookie would be created in CURDIR/sys/teken rather than OBJDIR.

Sponsored by:	EMC / Isilon Storage Division
2015-09-16 04:07:39 +00:00
bapt
2a77c3b71d Merge from HEAD 2015-08-25 20:14:50 +00:00
jasone
2dee86e95f Use bool rather than _Bool for C++ compatibility.
Submitted by:	Nikolai Lifanov
2015-08-19 18:32:12 +00:00
jasone
73461f2594 Fix minor malloc regressions.
- Use _Bool rather than bool to resolve missing type errors in malloc_np.h.
- Fix malloc manual page #include documentation.
- Add *allocm manual pages to obsolete files.

Submitted by:	jbeich
2015-08-19 00:06:46 +00:00
jasone
cbb776d46e Update jemalloc to version 4.0.0. 2015-08-18 00:21:25 +00:00
bapt
2c81d16589 Merge from HEAD 2015-08-09 00:15:17 +00:00
pfg
81778d2549 Use the __sentinel attribute.
Start using the gcc sentinel attribute, which can be used to
mark varargs function that need a NULL pointer to mark argument
termination, like execl(3).

Relnotes:	yes
2015-07-08 16:21:10 +00:00
oshogbo
e1ffe0f912 Add fdclose(3) function.
This function is equivalent to fclose(3) function except that it
does not close the underlying file descriptor.
fdclose(3) is step forward to make FILE structure private.

Reviewed by:	wblock, jilles, jhb, pjd
Approved by:	pjd (mentor)
Differential Revision:	https://reviews.freebsd.org/D2697
2015-07-04 16:42:14 +00:00