Commit Graph

282 Commits

Author SHA1 Message Date
tuexen
ab274c9d96 armv8 has support for optional CRC32C instructions. This patch checks if they are
available and if that is true make use of them.
Thank you very much to Andrew Turner for providing help and review the patch!
Reviewed by:		andrew
MFC after:		1 week
Differential Revision:	https://reviews.freebsd.org/D10499
2017-04-27 17:53:05 +00:00
markm
bd86cc8ebf Replace the RC4 algorithm for generating in-kernel secure random
numbers with Chacha20. Keep the API, though, as that is what the
other *BSD's have done.

Use the boot-time entropy stash (if present) to bootstrap the
in-kernel entropy source.

Reviewed by: delphij,rwatson
Approved by: so(delphij)
MFC after: 2 months
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D10048
--This line, and those below, will be ignored--
> Description of fields to fill in above:                     76 columns --|
> PR:                       If and which Problem Report is related.
> Submitted by:             If someone else sent in the change.
> Reported by:              If someone else reported the issue.
> Reviewed by:              If someone else reviewed your modification.
> Approved by:              If you needed approval for this commit.
> Obtained from:            If the change is from a third party.
> MFC after:                N [day[s]|week[s]|month[s]].  Request a reminder email.
> MFH:                      Ports tree branch name.  Request approval for merge.
> Relnotes:                 Set to 'yes' for mention in release notes.
> Security:                 Vulnerability reference (one per line) or description.
> Sponsored by:             If the change was sponsored by an organization.
> Differential Revision:    https://reviews.freebsd.org/D### (*full* phabric URL needed).
> Empty fields above will be automatically removed.
2017-04-16 09:11:02 +00:00
bde
5a798b035b Use inline asm instead of unportable intrinsics for the SSE4 crc32
optimization.

This fixes building with gcc-4.2.1 (it doesn't support SSE4).
gas-2.17.50 [FreeBSD] supports SSE4 instructions, so this doesn't
need using .byte directives.

This fixes depending on host user headers in the kernel.

Fix user includes (don't depend on namespace pollution in <nmmintrin.h>
that is not included now).

The instrinsics had no advantages except to sometimes avoid compiler
pessimixations.  clang understands them a bit better than inline asm,
and generates better looking code which also runs better for cem, but
for me it just at the same speed or slower by doing excessive
unrollowing in all the wrong places.  gcc-4.2.1 also doesn't understand
what it is doing with unrolling, but with -O3 somehow it does more
unrolling that helps.

Reduce 1 of the the compiler pessimizations (copying a variable which
already satisfies an "rm" constraint in a good way by being in memory
and not used again, to different memory and accessing it there.  Force
copying it to a register instead).

Try to optimize the inner loops significantly, so as to run at full
speed on smaller inputs.  The algorithm is already very MD, and was
tuned for the throughput of 3 crc32 instructions per cycle found on
at least Sandybridge through Haswell.  Now it is even more tuned for
this, so depends more on the compiler not rearranging or unrolling
things too much.  The main inner loop for should have no difficulty
runing at full speed on these CPUs unless the compiler unrolls it too
much.  However, the main inner loop wasn't even used for buffers smaller
than 24K.  Now it is used for buffers larger than 384 bytes.  Now it
is not so long, and the main outer loop is used more.  The new
optimization is to try to arrange that the outer loop runs in parallel
with the next inner loop except for the final iteration; then reduce
the loop sizes significantly to take advantage of this.

Approved by:	cem
Not tested in production by:	bde
2017-03-26 10:31:48 +00:00
delphij
9395fdf886 Discard first 3072 bytes of RC4 keystream, this is a bandaid
that allows us to work on switching to a more modern PRNG.

Submitted by:	Steven Chamberlain <steven pyro eu org>
Approved by:	so
2017-03-14 06:00:44 +00:00
imp
9a9a206596 strstr.c was inadvertently blasted with a copy of isa_nmi.c. Revert
and remove clause 3 while I'm here.
2017-03-01 02:07:51 +00:00
imp
7e6cabd06e Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
pfg
077418d939 sys: Replace zero with NULL for pointers.
Found with:	devel/coccinelle
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D9694
2017-02-22 02:35:59 +00:00
vangyzen
f36100ef2c Remove inet_ntoa() from the kernel
inet_ntoa() cannot be used safely in a multithreaded environment
because it uses a static local buffer.  Remove it from the kernel.

Suggested by:	glebius, emaste
Reviewed by:	gnn
MFC after:	never
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D9625
2017-02-16 20:50:01 +00:00
cem
c4b27423f5 calculate_crc32c: Add SSE4.2 implementation on x86
Derived from an implementation by Mark Adler.

The fast loop performs three simultaneous CRCs over subsets of the data
before composing them.  This takes advantage of certain properties of
the CRC32 implementation in Intel hardware.  (The CRC instruction takes 1
cycle but has 2-3 cycles of latency.)

The CRC32 instruction does not manipulate FPU state.

i386 does not have the crc32q instruction, so avoid it there.  Otherwise
the implementation is identical to amd64.

Add basic userland tests to verify correctness on a variety of inputs.

PR:		216467
Reported by:	Ben RUBSON <ben.rubson at gmail.com>
Reviewed by:	kib@, markj@ (earlier version)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D9342
2017-01-31 03:26:32 +00:00
cem
8cb19b99f4 Use time_t for intermediate values to avoid overflow in clock_ts_to_ct
Add additionally safety and overflow checks to clock_ts_to_ct and the
BCD routines while we're here.

Perform a safety check in sys_clock_settime() first to avoid easy local
root panic, without having to propagate an error value back through
dozens of APIs currently lacking error returns.

PR:		211960, 214300
Submitted by:	Justin McOmie <justin.mcomie at gmail.com>, kib@
Reported by:	Tim Newsham <tim.newsham at nccgroup.trust>
Reviewed by:	kib@
Sponsored by:	Dell EMC Isilon, FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9279
2017-01-24 18:05:29 +00:00
cem
feedc874d8 libkern: Remove obsolete 'register' keyword
Sponsored by:	Dell EMC Isilon
2017-01-12 17:02:29 +00:00
fabient
d99afb2c92 Update r309143 to prevent false sharing.
Reported by:	mjg
Approved by:	so
MFC after:	1 month
2016-11-25 17:20:23 +00:00
fabient
daeb893b0f In a dual processor system (2*6 cores) during IPSec throughput tests,
we see a lot of contention on the arc4 lock, used to generate the IV
of the ESP output packets.

The idea of this patch is to split this mutex in order to reduce the
contention on this lock.

Reviewed by:	delphij, markm, ache
Approved by:	so
Obtained from: emeric.poupon@stormshield.eu
MFC after: 1 month
Sponsored by:	Stormshield
Differential Revision:	https://reviews.freebsd.org/D8130
2016-11-25 13:49:33 +00:00
pfg
e72339bbf0 sys: Make use of our rounddown() macro when sys/param.h is available.
No functional change.
2016-04-30 14:41:18 +00:00
jkim
0df04f6464 Merge ACPICA 20160422. 2016-04-27 19:09:21 +00:00
pfg
5b3421712d kern: for pointers replace 0 with NULL.
These are mostly cosmetical, no functional change.

Found with devel/coccinelle.
2016-04-15 16:10:11 +00:00
cem
07799e51a0 libkern: ffs, fls: s/4/3/ the 3rd BSD clause
Approved by:	emaste
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 21:04:47 +00:00
cem
dfc679ec8e Add libkern ffsll() for parity with flsll()
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D3962
2015-10-22 20:28:37 +00:00
andrew
f62b818736 Add the __aeabi_memclr8 symbol, clang 3.7 uses this.
MFC after:	1 week
Sponsored by:	ABT Systems Lts
2015-09-21 18:35:32 +00:00
andrew
6a4a0db965 Remove checks for __ARM_EABI__, we only build for EABI now.
Sponsored by:	ABT Systems Ltd
2015-07-09 21:02:40 +00:00
andrew
b85d394d0e Add support for __aeabi_memclr4, clang 3.7 calls it.
Sponsored by:	ABT Systems Ltd
2015-07-09 20:54:38 +00:00
markm
d586165577 Huge cleanup of random(4) code.
* GENERAL
- Update copyright.
- Make kernel options for RANDOM_YARROW and RANDOM_DUMMY. Set
  neither to ON, which means we want Fortuna
- If there is no 'device random' in the kernel, there will be NO
  random(4) device in the kernel, and the KERN_ARND sysctl will
  return nothing. With RANDOM_DUMMY there will be a random(4) that
  always blocks.
- Repair kern.arandom (KERN_ARND sysctl). The old version went
  through arc4random(9) and was a bit weird.
- Adjust arc4random stirring a bit - the existing code looks a little
  suspect.
- Fix the nasty pre- and post-read overloading by providing explictit
  functions to do these tasks.
- Redo read_random(9) so as to duplicate random(4)'s read internals.
  This makes it a first-class citizen rather than a hack.
- Move stuff out of locked regions when it does not need to be
  there.
- Trim RANDOM_DEBUG printfs. Some are excess to requirement, some
  behind boot verbose.
- Use SYSINIT to sequence the startup.
- Fix init/deinit sysctl stuff.
- Make relevant sysctls also tunables.
- Add different harvesting "styles" to allow for different requirements
  (direct, queue, fast).
- Add harvesting of FFS atime events. This needs to be checked for
  weighing down the FS code.
- Add harvesting of slab allocator events. This needs to be checked for
  weighing down the allocator code.
- Fix the random(9) manpage.
- Loadable modules are not present for now. These will be re-engineered
  when the dust settles.
- Use macros for locks.
- Fix comments.

* src/share/man/...
- Update the man pages.

* src/etc/...
- The startup/shutdown work is done in D2924.

* src/UPDATING
- Add UPDATING announcement.

* src/sys/dev/random/build.sh
- Add copyright.
- Add libz for unit tests.

* src/sys/dev/random/dummy.c
- Remove; no longer needed. Functionality incorporated into randomdev.*.

* live_entropy_sources.c live_entropy_sources.h
- Remove; content moved.
- move content to randomdev.[ch] and optimise.

* src/sys/dev/random/random_adaptors.c src/sys/dev/random/random_adaptors.h
- Remove; plugability is no longer used. Compile-time algorithm
  selection is the way to go.

* src/sys/dev/random/random_harvestq.c src/sys/dev/random/random_harvestq.h
- Add early (re)boot-time randomness caching.

* src/sys/dev/random/randomdev_soft.c src/sys/dev/random/randomdev_soft.h
- Remove; no longer needed.

* src/sys/dev/random/uint128.h
- Provide a fake uint128_t; if a real one ever arrived, we can use
  that instead. All that is needed here is N=0, N++, N==0, and some
  localised trickery is used to manufacture a 128-bit 0ULLL.

* src/sys/dev/random/unit_test.c src/sys/dev/random/unit_test.h
- Improve unit tests; previously the testing human needed clairvoyance;
  now the test will do a basic check of compressibility. Clairvoyant
  talent is still a good idea.
- This is still a long way off a proper unit test.

* src/sys/dev/random/fortuna.c src/sys/dev/random/fortuna.h
- Improve messy union to just uint128_t.
- Remove unneeded 'static struct fortuna_start_cache'.
- Tighten up up arithmetic.
- Provide a method to allow eternal junk to be introduced; harden
  it against blatant by compress/hashing.
- Assert that locks are held correctly.
- Fix the nasty pre- and post-read overloading by providing explictit
  functions to do these tasks.
- Turn into self-sufficient module (no longer requires randomdev_soft.[ch])

* src/sys/dev/random/yarrow.c src/sys/dev/random/yarrow.h
- Improve messy union to just uint128_t.
- Remove unneeded 'staic struct start_cache'.
- Tighten up up arithmetic.
- Provide a method to allow eternal junk to be introduced; harden
  it against blatant by compress/hashing.
- Assert that locks are held correctly.
- Fix the nasty pre- and post-read overloading by providing explictit
  functions to do these tasks.
- Turn into self-sufficient module (no longer requires randomdev_soft.[ch])
- Fix some magic numbers elsewhere used as FAST and SLOW.

Differential Revision: https://reviews.freebsd.org/D2025
Reviewed by: vsevolod,delphij,rwatson,trasz,jmg
Approved by: so (delphij)
2015-06-30 17:00:45 +00:00
andrew
a20d17917f Add more __aeabi_memcpy functions, later versions of clang generate calls
to these functions.
2015-05-31 07:31:20 +00:00
rodrigc
7807fcddc4 Move zlib.c from net to libkern.
It is not network-specific code and would
be better as part of libkern instead.
Move zlib.h and zutil.h from net/ to sys/
Update includes to use sys/zlib.h and sys/zutil.h instead of net/

Submitted by:		Steve Kiernan stevek@juniper.net
Obtained from:		Juniper Networks, Inc.
GitHub Pull Request:	https://github.com/freebsd/freebsd/pull/28
Relnotes:		yes
2015-04-22 14:38:58 +00:00
rstone
96dd7966e7 Implement asprintf in libkern
Differential Revision:		https://reviews.freebsd.org/D1877
Reviewed by:			pjd, jfv
MFC After:			1 month
Sponsored by:			Sandvine Inc.
2015-03-01 00:22:16 +00:00
dim
7004e07d17 Similar to r277901, fix more -Wcast-qual warnings in libkern's strtoq(),
strtoul() and strtouq(), by using __DECONST.  No functional change.

MFC after:	3 days
2015-01-29 21:54:01 +00:00
dim
5a70de8b51 Fix a -Wcast-qual warning in libkern's strtol(), by using __DECONST. No
functional change.

MFC after:	3 days
2015-01-29 21:16:45 +00:00
jmg
c3ff54cc39 Add some new modes to OpenCrypto. These modes are AES-ICM (can be used
for counter mode), and AES-GCM.  Both of these modes have been added to
the aesni module.

Included is a set of tests to validate that the software and aesni
module calculate the correct values.  These use the NIST KAT test
vectors.  To run the test, you will need to install a soon to be
committed port, nist-kat that will install the vectors.  Using a port
is necessary as the test vectors are around 25MB.

All the man pages were updated.  I have added a new man page, crypto.7,
which includes a description of how to use each mode.  All the new modes
and some other AES modes are present.  It would be good for someone
else to go through and document the other modes.

A new ioctl was added to support AEAD modes which AES-GCM is one of them.
Without this ioctl, it is not possible to test AEAD modes from userland.

Add a timing safe bcmp for use to compare MACs.  Previously we were using
bcmp which could leak timing info and result in the ability to forge
messages.

Add a minor optimization to the aesni module so that single segment
mbufs don't get copied and instead are updated in place.  The aesni
module needs to be updated to support blocked IO so segmented mbufs
don't have to be copied.

We require that the IV be specified for all calls for both GCM and ICM.
This is to ensure proper use of these functions.

Obtained from:	p4: //depot/projects/opencrypto
Relnotes:	yes
Sponsored by:	FreeBSD Foundation
Sponsored by:	NetGate
2014-12-12 19:56:36 +00:00
andrew
a0c22b9f10 Use the unified syntax in a few more assembly files
MFC after:	1 week
Sponsored by:	ABT Systems Ltd
2014-12-05 19:08:36 +00:00
delphij
8d13dca13b Sync with userland variant.
MFC after:	2 weeks
2014-11-17 21:01:35 +00:00
des
325f19fddb Add a complete implementation of MurmurHash3. Tweak both implementations
so they match the established idiom.  Document them in hash(9).

MFC after:	1 month
MFC with:	r272906
2014-10-18 22:15:11 +00:00
delphij
c27f7c8888 Remove duplicated code.
Suggested by:	jmg
2014-10-16 22:04:07 +00:00
gnn
23f601a6ca Change the PF hash from Jenkins to Murmur3. In forwarding tests
this showed a conservative 3% incrase in PPS.

Differential Revision:	https://reviews.freebsd.org/D461
Submitted by:	des
Reviewed by:	emaste
MFC after:	1 month
2014-10-10 19:26:26 +00:00
delphij
e23313b949 Add explicit_bzero(3) and its kernel counterpart.
Obtained from:	OpenBSD
MFC after:	2 weeks
2014-10-07 04:54:11 +00:00
bz
1fec365dc9 Use __DECONST to avoid compiler warnings (and thus build failures)
with gcc on sparc64, mips, and powerpc after r271173.
2014-09-08 08:12:09 +00:00
benno
b46fbbac90 Add support for gdb's memory searching capabilities to our in-kernel gdb
server.

Submitted by:	Daniel O'Connor <daniel.oconnor@isilon.com>
Reviewed by:	jhb
Sponsored by:	EMC Isilon Storage Division
2014-09-05 16:40:47 +00:00
trasz
cac9beab7d Bring in the new automounter, similar to what's provided in most other
UNIX systems, eg. MacOS X and Solaris.  It uses Sun-compatible map format,
has proper kernel support, and LDAP integration.

There are still a few outstanding problems; they will be fixed shortly.

Reviewed by:	allanjude@, emaste@, kib@, wblock@ (earlier versions)
Phabric:	D523
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
2014-08-17 09:44:42 +00:00
ian
e51357b618 Fix unwind-info errors in our hand-written arm assembler code.
We have functions nested within functions, and places where we start a
function then never end it, we just jump to the middle of something else.
We tried to express this with nested ENTRY()/END() macros (which result
in .fnstart and .fnend directives), but it turns out there's no way to
express that nesting in ARM EHABI unwind info, and newer tools treat
multiple .fnstart directives without an intervening .fnend as an error.

These changes introduce two new macros, EENTRY() and EEND().  EENTRY()
creates a global label you can call/jump to just like ENTRY(), but it
doesn't emit a .fnstart.  EEND() is a no-op that just documents the
conceptual endpoint that matches up with the same-named EENTRY().

This is based on patches submitted by Stepan Dyatkovskiy, but I made some
changes and added the EEND() stuff, so blame any problems on me.

Submitted by:	Stepan Dyatkovskiy <stpworld@narod.ru>
2014-08-01 18:24:44 +00:00
marcel
9f28abd980 Remove ia64.
This includes:
o   All directories named *ia64*
o   All files named *ia64*
o   All ia64-specific code guarded by __ia64__
o   All ia64-specific makefile logic
o   Mention of ia64 in comments and documentation

This excludes:
o   Everything under contrib/
o   Everything under crypto/
o   sys/xen/interface
o   sys/sys/elf_common.h

Discussed at: BSDcan
2014-07-07 00:27:09 +00:00
jhb
7a3f5808a0 Use strcasecmp() instead of strcmp() when checking user-supplied encoding
names so that encoding names are treated as case-insensitive.  This allows
the use of 'utf-8' instead of 'UTF-8' for example and matches the behavior
of iconv(1).

PR:		167977
Submitted by:	buganini@gmail.com
MFC after:	1 week
2014-06-09 19:27:47 +00:00
eadler
f50f4fbaba Fix typo
Reported by:	emaste
2013-11-29 20:14:26 +00:00
eadler
26e90e1c0c Fix typo
Reported by:	swildner@DragonFlyBSD.org
2013-11-29 20:12:02 +00:00
andrew
f093c57980 Fix ixp425 boot2 with ARM EABI:
- libkern is missing __aeabi_llsl, implement this by calling __ashldi3.
 - Because of how the asm entry macros are defined the boot2 code
   requires the unwind symbols to exist, include them in boot2.

Approved by:	re (marius)
2013-09-29 15:19:34 +00:00
alfred
fdd68bea0e Fix watchdog pretimeout.
The original API calls for pow2ns, however the new APIs from
Linux call for seconds.

We need to be able to convert to/from 2^Nns to seconds in both
userland and kernel to fix this and properly compare units.
2013-07-27 20:47:01 +00:00
ache
60f7807df1 Attempt to mitigate poor initialization of arc4 by one-shot
reinitialization from yarrow right after good entropy is harvested.

Approved by:    secteam (delphij)
MFC after:      1 week
2013-04-19 00:30:52 +00:00
andrew
e271e57551 Add __aeabi_memset to libkern, implemented using memset, as clang may
generate calls to it.
2013-03-16 23:11:55 +00:00
andrew
faaba46948 Add END to ARM libkern assembly functions 2013-03-16 04:08:01 +00:00
andrew
147f0b89f3 Move the __aeabi_unwind_cpp_pr{0,1,2} functions to libkern so they can be
referenced in a non-debug kernel.
2013-03-16 04:06:49 +00:00
andrew
f7c2b781b9 Add __aeabi_memcpy to libkern as clang may generate calls to it. 2013-03-10 07:55:40 +00:00
andrew
7603324a00 Add __aeabi_ulcmp to allow building of the LINT kernel. 2013-01-23 09:18:18 +00:00