aligned on a int64_t boundary. However, when we allocate the array of
these structures, we use ALIGNBYTES which defaults to sizeof(int) on
arm, i386 and others. The i386 stuff can handle unaligned accesses
seemlessly. However, arm cannot. Take this into account when creating
the array of FILEs, and add some comments about why.
Differential Revision: https://reviews.freebsd.org/D4708
cperciva's libmd implementation is 5-30% faster
The same was done for SHA256 previously in r263218
cperciva's implementation was lacking SHA-384 which I implemented, validated against OpenSSL and the NIST documentation
Extend sbin/md5 to create sha384(1)
Chase dependancies on sys/crypto/sha2/sha2.{c,h} and replace them with sha512{c.c,.h}
Reviewed by: cperciva, des, delphij
Approved by: secteam, bapt (mentor)
MFC after: 2 weeks
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3929
(CLOCK_REALTIME case) system calls is non negative.
This commit hides a kernel panic in atrtc_settime() as the clock_ts_to_ct()
does not properly convert negative tv_sec.
ps. in my opinion clock_ts_to_ct() should be rewritten to properly handle
negative tv_sec values.
Differential Revision: https://reviews.freebsd.org/D4714
Reviewed by: kib
MFC after: 1 week
Traditionally the hcreate() function creates a hash table that uses
chaining, using a fixed user-provided size. The problem with this
approach is that this often either wastes memory (table too big) or
yields bad performance (table too small). For applications it may not
always be easy to estimate the right hash table size. A fixed number
only increases performance compared to a linked list by a constant
factor.
This problem can be solved easily by dynamically resizing the hash
table. If the size of the hash table is at least doubled, this has no
negative on the running time complexity. If a dynamically sized hash
table is used, we can also switch to using open addressing instead of
chaining, which has the advantage of just using a single allocation for
the entire table, instead of allocating many small objects.
Finally, a problem with the existing implementation is that its
deterministic algorithm for hashing makes it possible to come up with
fixed patterns to trigger an excessive number of collisions. We can
easily solve this by using FNV-1a as a hashing algorithm in combination
with a randomly generated offset basis.
Measurements have shown that this implementation is about 20-25% faster
than the existing implementation (even if the existing implementation is
given an excessive number of buckets). Though it allocates more memory
through malloc() than the old implementation (between 4-8 pointers per
used entry instead of 3), process memory use is similar to the old
implementation as if the estimated size was underestimated by a factor
10. This is due to the fact that malloc() needs to perform less
bookkeeping.
Reviewed by: jilles, pfg
Obtained from: https://github.com/NuxiNL/cloudlibc
Differential Revision: https://reviews.freebsd.org/D4644
bugfix-only release, with no new features.
Please note that from 3.5.0 onwards, clang and llvm require C++11
support to build; see UPDATING for more information.
POSIX requires for the c99 compiler.
(In fact, our c99(1) already ignores -lxnet; but our make(1) doesn't set
${CC} correctly, and our cc(1) treats xnet like any other library.)
Reviewed by: kib
450 seconds
This is required on slower network connections, and on older releases
(stable/10 seems to be slower as far as name resolution goes.. not sure
why yet).
Remove an outdated comment in the Makefile from when I was working on
this code over a year ago on github
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
i386 is the only current FreeBSD architecture that ever used a.out
format.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D4687
sysdecode_ioctlname() function. This function matches the behavior
of the truss variant in that it returns a pointer to a string description
for known ioctls. The caller is responsible for displaying unknown
ioctl requests. For kdump this meant moving the logic to handle unknown
ioctl requests out of the generated function and into an ioctlname()
function in kdump.c instead.
Differential Revision: https://reviews.freebsd.org/D4610
The existing implementations of POSIX tsearch() and tdelete() don't
attempt to perform any balancing at all. Testing reveals that inserting
100k nodes into a tree sequentially takes approximately one minute on my
system.
Though most other BSDs also don't use any balanced tree internally, C
libraries like glibc and musl do provide better implementations. glibc
uses a red-black tree and musl uses an AVL tree.
Red-black trees have the advantage over AVL trees that they only require
O(1) rotations after insertion and deletion, but have the disadvantage
that the tree has a maximum depth of 2*log2(n) instead of 1.44*log2(n).
My take is that it's better to focus on having a lower maximum depth,
for the reason that in the case of tsearch() the invocation of the
comparator likely dominates the running time.
This change replaces the tsearch() and tdelete() functions by versions
that create an AVL tree. Compared to musl's implementation, this version
is different in two different ways:
- We don't keep track of heights; just balances. This is sufficient.
This has the advantage that it reduces the number of nodes that are
being accessed. Storing heights requires us to also access all of the
siblings along the path.
- Don't use any recursion at all. We know that the tree cannot 2^64
elements in size, so the height of the tree can never be larger than
96. Use a 128-bit bitmask to keep track of the path that is computed.
This allows us to iterate over the same path twice, meaning we can
apply rotations from top to bottom.
Inserting 100k nodes into a tree now only takes 0.015 seconds. Insertion
seems to be twice as fast as glibc, whereas deletion has about the same
performance. Unlike glibc, it uses a fixed amount of memory.
I also experimented with both recursive and iterative bottom-up
implementations of the same algorithm. This iterative top-down version
performs similar to the recursive bottom-up version in terms of speed
and code size.
For some reason, the iterative bottom-up algorithm was actually 30%
faster for deletion, but has a quadratic memory complexity to keep track
of all the parent pointers.
Reviewed by: jilles
Obtained from: https://github.com/NuxiNL/cloudlibc
Differential Revision: https://reviews.freebsd.org/D4412
variables in loader(8) and other libstand applications.
Sometimes a dhcp server provides incorrect information along with the IP
address. It would be useful to have a way to override this with
locally-supplied information, such as command line parameters passed from a
prior-stage bootloader. This change allows pre-existing env vars to take
precedence over values delivered by the dhcp or bootp server.
The bootp/dhcp code in libstand automatically creates environment variables
from the data provided by the server (dhcp.root-path, dhcp.domain-name,
etc). It also transcribes the values to some global variables such as
'rootpath' and 'hostname'.
This change does two things:
When adding dhcp.* vars to the environment, don't replace existing
vars/values.
When setting the global vars rootpath and hostname, use the
dhcp.root-path and dhcp.host-name env var values if they exist.
This allows the platform-specific part of loader(8) to obtain override
values in some platform-specific way and store them in the environment
before opening the network device. The set of values that can be overriden
is currently limited to just string options. The values that are delivered
as binary data are things that probably shouldn't be overridden (IP,
netmask, gateway, etc).
The original patch this evolved from was submitted by martymac@
PR: 202098
Differential Revision: https://reviews.freebsd.org/D4559
Depending on system configuration and parameters, clock_gettime() and
gettimeofday() may not be system calls. If so, passing an invalid pointer
will cause a signal and not an [EFAULT] error.
From a standards perspective, this is OK since passing an invalid pointer is
undefined behaviour.
MFC after: 1 week
normal libc versions of these functions use asprintf and sscanf, both
of which aren't available in the libstand world. It's easier to
rewrite these than to try to claw those functions into libstand.
Also include uuid_create_nil, since it's safe to include. These were
written from scratch, except for bits of comments that I got from the
libc version, which are acknowledged in the source.
Differential Revision: https://reviews.freebsd.org/D4540
pxeboot in tftp loader mode (when built with LOADER_TFTP_SUPPORT) now
prefix all the path to open with the path obtained via the option 'root-path'
directive.
This allows to be able to use the traditional content /boot out of box. Meaning
it now works pretty much like all other loaders. It simplifies hosting hosting
multiple version of FreeBSD on a tftp server.
As a consequence, pxeboot does not look anymore for a pxeboot.4th (which was
never provided)
Note: that pxeboot in tftp loader mode is not built by default.
Reviewed by: rpokala
Relnotes: yes
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D4590
risk leaving the connection in an indeterminate state if the server fails
to send a chunk delimiter. Depending on the application and on the sizes
of the preceding chunks, the result can be anything from missing data to a
segfault. With this patch, it will be reported as a protocol error.
PR: 204771
MFC after: 1 week
into the FreeBSD test suite
There's no functional change with these testcases; they're purposely
being left in TAP format for the time being
Other testcases which crash on amd64/i386 as-is have not been
integrated yet (they need to be retested on a later version of
CURRENT, as I haven't used i386 in some time)
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
lib/libc/tests/nss
- Convert the testcases to ATF
- Do some style(9) cleanups:
-- Sort headers
-- Apply indentation fixes
-- Remove superfluous parentheses
- Explicitly print out debug printfs for use with `kyua {debug,report}`; for
items that were overly noisy, they've been put behind #ifdef DEBUG
conditionals
- Fix some format strings
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
system call information such as system call arguments. Initially this
will consist of pulling duplicated code out of truss and kdump though it
may prove useful for other utilities in the future.
This commit moves the shared utrace(2) record parser out of kdump into
the library and updates kdump and truss to use it. One difference from
the previous version is that the library version treats unknown events
that start with the "RTLD" signature as unknown events. This simplifies
the interface and allows the consumer to decide how to handle all
non-recognized events. Instead, this function only generates a string
description for known malloc() and RTLD records.
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D4537
In r289315, I added new fields to res_state. This broke binary
backward compatibility. It also broke some ports (and possibly
other code) by requiring the definition of time_t and struct timespec.
Fix these problems by moving the new fields into __res_state_ext.
Suggested by: ume
Reviewed by: ume
MFC after: 3 days
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D4472
RISC-V is a new ISA designed to support computer research and education, and
is now become a standard open architecture for industry implementations.
This is a minimal set of changes required to run 'make kernel-toolchain'
using external (GNU) toolchain.
The FreeBSD/RISC-V project home: https://wiki.freebsd.org/riscv.
Reviewed by: andrew, bdrewery, emaste, imp
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
Differential Revision: https://reviews.freebsd.org/D4445
When accepting a connection on underlying tcp socket rpc vc
transporter fails to populate local address. This change rectifies
this problem by modifying rendezvous_request() to fill out the xptr
structure member with appropriate information.
Submitted by: Alex Burlyga <alex.burlyga.ietf@gmail.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D4206
revision 1.13
date: 2015/12/09 11:54:12; author: tb; state: Exp; lines: +2 -2
Add a cast to silence a compiler warning by clang on FreeBSD.
From Craig Rodrigues.
ok tedu@
revision 1.12
date: 2015/12/05 13:06:52; author: claudio; state: Exp; lines: +4 -6
Do not loop on EAGAIN in imsg_read(). Better to return the error to the
caller and let him do another poll loop. This fixes spinning relayd
processes seen on busy TLS relays. OK benno@ henning@
The requirement is for a GCC-compatible compiler and not necessarily
GCC itself. However, we currently expect any compiler used for building
the whole of FreeBSD to be GCC-compatible and many things will break if
not; there's no longer a need to have an explicit test for this in csu.
Sponsored by: The FreeBSD Foundation
bsd.lib.mk and bsd.prog.mk already depend all objs on headers in SRCS if
there is not yet a depend file. The headers in SRCS are never built or
installed. After 'make depend' the header was already added as a proper
dependency on the objects where needed.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
clock_gettime(2) on ARMv7 and ARMv8 systems which have architectural
generic timer hardware. It is similar how the RDTSC timer is used in
userspace on x86.
Fix a permission problem where generic timer access from EL0 (or
userspace on v7) was not properly initialized on APs.
For ARMv7, mark the stack non-executable. The shared page is added for
all arms (including ARMv8 64bit), and the signal trampoline code is
moved to the page.
Reviewed by: andrew
Discussed with: emaste, mmel
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4209
The normal LIBADD is ssp_nonshared. This also had a DPADD on LIBSSP which
does not actually exist, it is blank.
Sponsored by: EMC / Isilon Storage Division
- Create automatically generated include header for split.c
main.c:
- Use function definitions from debug.ih and split.ih instead of externs
Sponsored by: EMC / Isilon Storage Division
This avoids the need for an afterinstall: hook and a check for LIBRARIES_ONLY.
It also now respects INCLUDEDIR.
This came in r249484.
Sponsored by: EMC / Isilon Storage Division
kyua 0.12 has fix for https://github.com/jmmv/kyua/pull/148
which eliminates invalid XML characters from being written to test reports
with "kyua report-junit".
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.
Currently rescue/rescue and sys/modules are still not connected.
Sponsored by: EMC / Isilon Storage Division
- Support more of the toolchain from TOOLSDIR.
- This also improves 'make bootstrap-tools' to pass, for example,
AS=/usr/bin/as to Makefile.inc1, which will tell cross-tools to use
external toolchain support and avoid building things we won't be using
in the build.
- Always set the PATH to contain the staged TOOLSDIR directories when
not building the bootstrap targets.
The previous version was only setting this at MAKE.LEVEL==0 and if the
TOOLSDIR existed. Both of these prevented using staged tools that were
built during the build though as DIRDEPS with .host dependencies, such
as the fix for needing usr.bin/localedef.host in r291311.
This is not a common tool so we must build and use it during the build,
and need to be prepared to change PATH as soon as it appears.
This should also fix the issue of host dependencies disappearing from
Makefile.depend and then reappearing due to the start of the fresh build not
having the directory yet, resulting in the tools that were built not actually
being used.
- Only use LEGACY_TOOLS while building in Makefile.inc1. After r291317
and r291546 there is no need to add LEGACY_TOOLS into the PATH for
the pseudo/targets/toolchain build.
- Because the pseudo/targets/toolchain will now build its own
[clang-]tblgen, the special logic in clang.build.mk is no longer needed.
- LEGACY_TOOLS is no longer used outside of targets/pseudo/bootstrap-tools
so is no longer passed into the environment in its build.
Sponsored by: EMC / Isilon Storage Division
staged.
None of usr.bin/clang/clang-tblgen or its dependencies need
lib/clang/include, so there is no cyclic dependency here to worry about.
The issue came about because of workarounds to dependencies on clang
being optional.
Without this, the clang-tblgen called during the build would change
after it was staged for the host. This would cause lib/clang/include to
rebuild due to changed build commands.
Sponsored by: EMC / Isilon Storage Division
There seems to be no cyclic dependency here. This partially reverts r283595
which had no explanation for the change.
Sponsored by: EMC / Isilon Storage Division
tested on the Broadwell-Xeon with a hacked up version of pmcstudy -T. I still need
to circle back and add in to pmcstudy all the new tests from the Broadwell Vtune
guide (for the hacked up version I just made it so I could run the -T option). The
Skylake CPU is not yet available (even though Intel is advertising it .. imagine that).
The Skylake PMC's will need to be tested once we can get a sample skylake CPU :-)
Sponsored by: Netflix Inc.
routines to interface with net80211.
This is all from the ifconfig program; the duplicate code from ifconfig
will be removed when it starts using this API.
Differential Revision: https://reviews.freebsd.org/D4290
- Add a kvaddr_type to represent kernel virtual addresses instead of
unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles(). It drops the
unused 'swapfile' argument and adds a new function pointer argument for
a symbol resolving function. Native kernels still use _fdnlist() from
libc to resolve symbols if a resolver function is not supplied, but cross
kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
Each backend is responsible for implementing kvm_read2() for a given
vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
powerpc cores).
- Add internal helper routines for the common page offset hash table used
by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
to be cross-friendly by using private constants instead of ones that
vary by platform (e.g. PAGE_SIZE). Static assertions are present when
a given backend is compiled natively to ensure the private constants
match the real ones.
- Enable all of the existing vmcore backends on all platforms. This means
that libkvm on any platform should be able to perform KVA translation
and read data from a vmcore of any platform.
Tested on: amd64, i386, sparc64 (marius)
Differential Revision: https://reviews.freebsd.org/D3341
The dependency for ypldap is only with WITH_NIS, but libevent is small
enough to not warrant the extra logic and maintenance cost here so just
always build it.
Sponsored by: EMC / Isilon Storage Division
This is so that 'make depend' is not a required build step in these
files.
DPSRCS is overall unneeded. DPSRCS already contains SRCS, so anything
which can safely be in SRCS should be. DPSRCS is mostly just a way to
generate files that should not be linked into the final PROG/LIB. For
headers and grammars it is safe for them to be in SRCS since they will
be excluded during linking and installation.
The only remaining uses of DPSRCS are for generating .c or .o files that
must be built before 'make depend' can run 'mkdep' on the SRCS c files
list. A semi-proper example is in tests/sys/kern/acct/Makefile where a
checked-in .c file has an #include on a generated .c file. The
generated .c file should not be linked into the final PROG though since
it is #include'd. The more proper way here is just to build/link it in
though without DPSRCS. Another example is in sys/modules/linux/Makefile
where a shell script runs to parse a DPSRCS .o file that should not be
linked into the module. Beyond those, the need for DPSRCS is largely
unneeded, redundant, and forces 'make depend' to be ran. Generally,
these Makefiles should avoid the need for DPSRCS and define proper
dependencies for their files as well.
An example of an improper usage and why this matters is in usr.bin/netstat.
nl_defs.h was only in DPSRCS and so was not generated during 'make all',
but only during 'make depend'. The files including it lacked proper
depenencies on it, which forced running 'make depend' to workaround that
bug. The 'make depend' target should mostly be used for incremental build
help, not to produce a working build. This specific example was broken in
the meta build until r287905 since it does not run 'make depend'.
The gnu/lib/libreadline/readline case is fine since bsd.lib.mk has 'OBJS:
SRCS:M*.h' when there is no .depend file.
Sponsored by: EMC / Isilon Storage Division
MFC after: 1 week
This avoids using the staged headers for sys/crypto/sha2/*.h, such as sha256.h,
which added an unneeded pre-build dependency on libmd to libcrypt. This
header is an INCS in lib/libmd, but found via .PATH in sys/crypto/sha2.
Since the libcrypt build was already using the in-src libmd headers
directly, just teach it how to find the sha256.h header as well.
Sponsored by: EMC / Isilon Storage Division
The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.
This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.
For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds. This has not yet been a problem due
to readelf not being built as a host tool in buildworld. This is possible
in the meta build though when building the toolchain.
Sponsored by: EMC / Isilon Storage Division
Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file. There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.
Sponsored by: EMC / Isilon Storage Division
librtld_db only needs libutil.h to build, not the libproc library. So
it can safely use its header and allow libproc to depend on librtld_rb
to be built first to link. This is required after fixing ld --sysroot
in r291226.
Sponsored by: EMC / Isilon Storage Division
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.
Sponsored by: EMC / Isilon Storage Division
It included libutil.h for setproctitle(3), which was moved from libutil to libc
in r65353 in 2000.
Reviewed by: gshapiro [sendmail change]
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D4261
(HOST_SHELL is used in NetBSD)
This fixes permission denied issues when gen_ether_subr is not executable
MFC after: 3 days
Reported by: José Pérez <fbl@aoek.com>
Suggested by: bdrewery, sjg
Move fdopen() up near other resource allocation like malloc(); do proper
deallocation on failure later on in the function.
Submitted by: Ramachandra Topannavar <rtopannavar@panasas.com>
Reviewed by: jilles
Approved by: jhb (mentor)
MFC after: 2 weeks
Sponsored by: Panasas, Inc.
Differential Revision: https://reviews.freebsd.org/D4126
M lib/libc/gen/popen.c
1024 specified on YPMAXRECORD the ypmatch can get in an infinite retry
loop when is requesting the information from the NIS server.
The ypmatch(1) will return an error until the command receives an
kill(1).
To avoid this problem, we check the MAX_RETRIES that is by default set
to 20 and avoid get in infinet loop at the client side.
NOTE: FreeBSD nis(8) server doesn't present this issue.
Submitted by: Ravi Pokala <rpokala@panasas.com>,
Lakshmi N. Sundararajan <lakshmi.n@msystechnologies.com>,
Lewis, Fred <flewis@panasas.com>,
Pushkar Kothavade <pushkar.kothavade@msystechnologies.com>
Approved by: bapt (mentor)
MFC after: 1 month
Differential Revision: D4095
They are emitting characters which are triggering
a kyua bug which causes kyua to emit invalid XML.
This invalid XML is causing false failures in Jenkins.
On a separate note, kyua needs to be fixed with this:
https://github.com/jmmv/kyua/pull/148
or something similar.
When pmcstat exits after some samples were dropped, give the user an
idea of how many were lost. (Granted, these are global numbers, but
they may still help quantify the scope of the loss.)
Differential Revision: https://reviews.freebsd.org/D4123
Approved by: gnn (mentor)
MFC after: 1 month
Sponsored by: Juniper Networks
- Split up the testcases into C locale and ja_JP.eucJP testcases.
- Avoid a segfault in the event that setlocale fails, similar to r290843
- Replace `sizeof(x) / sizeof(*x)` pattern with `nitems(x)`
MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division
Provide more meaningful diagnostic messages if LC_CTYPE can't be set properly
instead of segfaulting, because setlocale returns NULL and strcmp(NULL, b) will
always segfault
Split up the testcases so one failing (in this case en_US.ISO8859-15) won't
cause the rest of the testcases to be skipped
Remove some unused variables
MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division
immediatelly as old code does, now for append modes too.
Real use case for such fallback is impossible (unless specially crafted).
2) Remove now unneded include I forgot to remove in prev. commits.
MFC after: 1 week
The NONE:US-ASCII case isn't necessary. The "NONE:" case will handle
US-ASCII, so let's remove the redundant handling.
Submitted by: marino
Obtained from: DragonflyBSD
The US-ASCII format was getting treated identically to POSIX. It is
supposed to throw an ILSEQ errno if a value of 0x80 or greater is
encountered, so let's bring back the "ASCII" handling.
While here, change nl_codeset to return US-ASCII only when the encoding
really is "US-ASCII". Before "C" and "POSIX" encoding returned this
string, so now they return "POSIX".
Discussed with: ache
Submitted by: marino
Obtained from: DragonflyBSD
