Commit Graph

178 Commits

Author SHA1 Message Date
Andrew Turner
a947203221 Revert "Disable the accelerated arm64 sha25 in static libraries"
This is now unneeded as arm64 can now resolve ifuncs in static binaries

This reverts commit c81ea895b5.

Sponsored by:	The FreeBSD Foundation
2021-08-30 10:36:43 +01:00
Andrew Turner
c81ea895b5 Disable the accelerated arm64 sha25 in static libraries
We don't have ifunc support in static arm64 binaries. Until we do
disable the accelerated sha256 code in a static libmd as it uses an
ifunc.

Reported by:	brd
Sponsored by:	The FreeBSD Foundation
2021-08-19 16:48:30 +00:00
Andrew Turner
69d8dc20be Use arm64 sha256 intrinsics in libmd
Summary:
When running on a CPU that supports the arm64 sha256 intrinsics use them
to improve perfromance of sha256 calculations.

With this changethe following improvement has been seen on an Apple M1
with FreeBS running under Parallels, with similar results on a
Neoverse-N1 r3p1.

x sha256.orig
+ sha256.arm64
+--------------------------------------------------------------------+
|++                                                               x x|
|+++                                                              xxx|
||A                                                               |A||
+--------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5          3.41           3.5          3.46         3.458   0.042661458
+   5          0.47          0.54           0.5         0.504   0.027018512
Difference at 95.0% confidence
        -2.954 +/- 0.0520768
        -85.4251% +/- 0.826831%
        (Student's t, pooled s = 0.0357071)

Reviewed by:	cem
Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31284
2021-08-11 10:20:48 +00:00
Gordon Bergling
3d265fce43 Fix a few mandoc issues
- skipping paragraph macro: Pp after Sh
- sections out of conventional order: Sh EXAMPLES
- whitespace at end of input line
- normalizing date format
2020-10-09 19:12:44 +00:00
Ed Maste
da6e5efe92 libmd: add dependency workaround for r366344
r366344 fixed and reenabled the assembly optimized skein implementation,
but skein_block objects were not being rebuilt in no-clean builds. This
resulted in failing no-clean builds. SKEIN_USE_ASM controls which
routines come from C vs assembly, and with no explicit dependency
r366344's change to SKEIN_USE_ASM did not cause skein_block.{o,pico}
to be rebuilt.

Add a dependency on this Makefile for the skein_block objects. This
dependency is broader in scope than absolutely required (that is, the
skein_block objects will now be rebuilt on any change to this Makefile).
There are ways this could be addressed, but it is probably not worth the
additional effort or testing time to pursue them.

PR:		248221
Reported by:	kevans, Jeremy Faulkner
Discussed with:	kevans
Sponsored by:	The FreeBSD Foundation
2020-10-02 14:00:52 +00:00
Ed Maste
36972ee3e0 libmd: fix assembly optimized skein implementation
The assembly implementation incorrectly used logical AND instead of
bitwise AND. Fix, and re-enable in libmd.

Submitted by:	Yang Zhong <yzhong@freebsdfoundation.org>
Reviewed by:	cem (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26614
2020-10-01 21:05:50 +00:00
Ed Maste
0d2c19d05b libmd: temporarily disable optimized assembly skein1024 implementation
It is apparently broken when assembled by contemporary GNU as as well as
Clang IAS (which is used in the default configuration).

PR:		248221
Reported by:	pizzamig
Sponsored by:	The FreeBSD Foundation
2020-07-23 18:55:47 +00:00
Alex Richardson
7425ce4b54 Remove warning that is no longer accurate after r361853
We now build the skein assembly with clangs integrated assembler.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D25664
2020-07-15 17:24:34 +00:00
Ed Maste
24ed6f550a Rename skein_block_asm.s to .S and assemble using Clang IAS
Comparing the object files produced by GNU as 2.17.50 and Clang IAS
shows many immaterial changes in strtab etc., and one material change
in .text:

   1bac:  4c 8b 4f 18             mov    0x18(%rdi),%r9
   1bb0:  eb 0e                   jmp    1bc0 <Skein1024_block_loop>
-  1bb2:  66 66 2e 0f 1f 84 00    data16 nopw %cs:0x0(%rax,%rax,1)
-  1bb9:  00 00 00 00
-  1bbd:  0f 1f 00                nopl   (%rax)
+  1bb2:  66 2e 0f 1f 84 00 00    nopw   %cs:0x0(%rax,%rax,1)
+  1bb9:  00 00 00
+  1bbc:  0f 1f 40 00             nopl   0x0(%rax)

 0000000000001bc0 <Skein1024_block_loop>:
 Skein1024_block_loop():
   1bc0:  4c 8b 47 10             mov    0x10(%rdi),%r8
   1bc4:  4c 03 85 c0 00 00 00    add    0xc0(%rbp),%r8

That is, GNU as and Clang's integrated assembler use different multi-
byte NOPs for alignment (GNU as emits an 11 byte NOP + a 3 byte NOP,
while Clang IAS emits a 10 byte NOP + a 4 byte NOP).

Dependency cleanup hacks are not required, because we do not create
.depend files from GNU as.

Reviewed by:	allanjude, arichardson, cem, tsoome
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8434
2020-06-06 00:35:41 +00:00
Ed Maste
f2b8688664 Also pass SKEIN_USE_ASM to the assembler, via AFLAGS 2020-06-05 18:56:43 +00:00
Ed Maste
310e81aede Apply C SKEIN_LOOP setting only to skein_block.c
Otherwise if assembling skein_block_asm.s with Clang's integrated assembler
we can pass conflicting SKEIN_LOOP settings (via CFLAGS and ACFLAGS).
2020-06-05 17:00:38 +00:00
Ed Maste
6e46e45f97 ANSIfy and KNF function arg definitions in libmd/md4.c
Reported by:	bde, in 2017
2020-04-03 20:56:43 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Emmanuel Vadot
a7b5a3d486 pkgbase: Put a lot of binaries and lib in FreeBSD-runtime
All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
FreeBSD-runtime.

Reviewed by:    bapt, gjb
Differential Revision:  https://reviews.freebsd.org/D21503
2019-09-05 14:13:08 +00:00
Allan Jude
87e34f6cf4 sha.3: clarify admonition against use in NEW signature schemes
Reported by:	cem, cperciva (grammar)
2019-06-05 20:18:08 +00:00
Allan Jude
34841dd627 Add admonitions against using MD5 and SHA1 to the API man pages 2019-05-21 22:17:00 +00:00
Allan Jude
f26f4cf170 Add missing errors section to md[2-5], ripemd160, sha*, and skein* manpages
PR:		148987
Submitted by:	Dan Lukes <dan@obluda.cz> (original version)
MFC after:	1 week
Sponsored by:	Klara Systems
Event:	Waterloo Hackathon 2019
2019-05-21 22:11:53 +00:00
Sean Bruno
01a9c32322 r338270 had the side effect of no longer installing libmd.so into /lib.
For users who have a seperate zfs mount of /usr or /usr/lib, this will
cause dynamic loading failures when attempting to execute zfs mount on
bootup. E.g. the system won't boot.

Including <src.opts.mk> sets SHLIBDIR, so SHLIBDIR?= has no
effect.  The other lib/ Makefiles solve this problem by moving the
SHLIBDIR  assignment to before .include <src.opts.mk>.

Submitted by:	jilles
Reviewed by:	allanjude
Approved by:	re (rgrimes)
Differential Revision:	https://reviews.freebsd.org/D16910
2018-08-26 17:05:43 +00:00
Alex Richardson
c91575ceec Don't build skein_block_asm.s if we don't have an as binary
This fixes building libmd on MacOS/Linux. The real fix is probably to
build it as a .S file with $CC instead. It might also be better to just
compile the C file in userspace since the compiler can the use SSE/AVX.

Reviewed By:	emaste, brooks
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D16844
2018-08-23 18:19:33 +00:00
Alex Richardson
078018e0ac Allow bootstrapping libmd on MacOS
The assembly files use directives that only work for ELF targets so skip
them when bootstrapping on MacOS.

Reviewed By:	imp
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D14247
2018-08-20 10:39:48 +00:00
Xin LI
66bdf50fac libmd: Always erase context in _Final method, and when doing
it, consistently use explicit_bzero().

Update manual pages to match the behavior.

Reviewed by:	pfg, allanjude, jmg
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D16316
2018-07-20 07:01:28 +00:00
Conrad Meyer
8ff3cdd1b5 Integrate SHA2-224 with userspace components
The double compilation of the kernel sources in libmd and libcrypt is
baffling, but add yet another define hack to prevent duplicate symbols.

Add documentation and SHA2-224 test cases to libmd.

Integrate with the md5(1) command, document, and add more test cases;
self-tests pass.
2018-07-09 08:19:04 +00:00
Mark Johnston
2531686329 Revert r334090.
It causes the 32bit compat build of libmd to fail with:

libmd/rmd160c.c:86:9: error: 'ripemd160_block' macro redefined
#define ripemd160_block ripemd160_block_x86
        ^
libmd/ripemd.h:122:9: note: previous definition is here
#define ripemd160_block         _libmd_ripemd160_block
2018-05-23 17:01:28 +00:00
Eitan Adler
272b99f722 libmd: build with WARNS=1
- build with WARNS=1. This works without any changes
- remove two unused variables noticed at WARNS=2
2018-05-23 11:20:16 +00:00
Conrad Meyer
ee9f155f8f libmd: Remove trailing whitespace from mdXhl.c
No functional change.

Reported by:	me, in a CR from three years ago today
Sponsored by:	Dell EMC Isilon
2018-04-17 17:23:47 +00:00
Edward Tomasz Napierala
7651c3af83 Fix formatting errors that resulted in apropos(1) output looking weird.
MFC after:	2 weeks
2018-03-17 11:41:06 +00:00
Pedro F. Giffuni
62cf53fdac SPDX: some uses of the RSA-MD license. 2017-12-13 16:30:39 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Allan Jude
929b476ae6 Increase loop unrolling for skein hashes
This patch was inspired by an opposite change made to shrink the code
for the boot loader.

On my i7-4770, it increases the skein1024 speed from 470 to 550 MB/s

Reviewed by:	sbruno
MFC after:	1 month
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D7824
2017-07-01 21:18:06 +00:00
Toomas Soome
a87f43927c ANSIfy md5 functions
Update the function declarations.

Reviewed by:	dim
Differential Revision:	https://reviews.freebsd.org/D11055
2017-06-05 11:55:26 +00:00
Bryan Drewery
a1b9cad56b Fix bootstrapping libmd on older systems after r314709.
This follows another fix to bootstrap libmd after r313404.  The
MD5FileChunk prototype is needed to build libmd, but it is
only reliably in the src tree's sys/md5.h header.  Rather than
polluting the legacy build with this header for the entire build,
just symlink it in here for now as is done in the elftoolchain
build.  Libmd is already referencing other src tree headers by
its used of CFLAGS+= ${SRCTOP}/sys/crypto/sha2.  This, and
other uses of CFLAGS+= ${SRCTOP}/sys..., may later change to
be in the legacy mechanism.

Reported by:	bde, ian, sjg
Tested by:	ian
2017-03-06 21:06:55 +00:00
Enji Cooper
49233bae4d Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones
This simplifies pathing in make/displayed output

MFC after:    3 weeks
Sponsored by: Dell EMC Isilon
2017-01-20 04:43:21 +00:00
Alan Somers
8254c3c5d3 Fix C++ includability of crypto headers with static array sizes
C99 allows array function parameters to use the static keyword for their
sizes. This tells the compiler that the parameter will have at least the
specified size, and calling code will fail to compile if that guarantee is
not met. However, this syntax is not legal in C++.

This commit reverts r300824, which worked around the problem for
sys/sys/md5.h only, and introduces a new macro: min_size(). min_size(x) can
be used in headers as a static array size, but will still compile in C++
mode.

Reviewed by:	cem, ed
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D8277
2016-10-18 23:20:49 +00:00
Ed Maste
de13c2427d libmd: introduce functions that operate on an fd instead of filename
Reviewed by:	allanjude, cem
MFC after:	2 months
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8264
2016-10-17 13:47:22 +00:00
Ed Maste
60a4cd8411 libmd: strip local absolute symbols (to link with lld)
Old versions of gas produce an invalid section index. That is ignored by
old versions of ld, but prevents a link with lld.

Submitted by:	Rafael Ávila de Espíndola (earlier version)
Reviewed by:	allanjude
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6789
2016-07-21 18:47:51 +00:00
Allan Jude
b468a9ff1d Import the skein hashing algorithm, based on the threefish block cipher
Connect it to userland (libmd, libcrypt, sbin/md5) and kernel (crypto.ko)

Support for skein as a ZFS checksum algorithm was introduced in r289422
but is disconnected because FreeBSD lacked a Skein implementation.

A further commit will enable it in ZFS.

Reviewed by:	cem
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D6166
2016-05-29 01:15:36 +00:00
Allan Jude
1780e40715 Implement SHA-512 truncated (224 and 256 bits)
This implements SHA-512/256, which generates a 256 bit hash by
calculating the SHA-512 then truncating the result. A different initial
value is used, making the result different from the first 256 bits of
the SHA-512 of the same input. SHA-512 is ~50% faster than SHA-256 on
64bit platforms, so the result is a faster 256 bit hash.

The main goal of this implementation is to enable support for this
faster hashing algorithm in ZFS. The feature was introduced into ZFS
in r289422, but is disconnected because SHA-512/256 support was missing.
A further commit will enable it in ZFS.

This is the follow on to r292782

Reviewed by:	cem
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D6061
2016-05-28 16:06:07 +00:00
Conrad Meyer
c6b03be14b libmd: Work around C++'s inability to understand C
Reported by:	antoine@ (x265)
2016-05-27 05:31:14 +00:00
Glen Barber
52259a98ad MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
Bryan Drewery
bd18fd57db DIRDEPS_BUILD: Regenerate without local dependencies.
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:20:11 +00:00
Glen Barber
a70cba9582 First pass through library packaging.
Sponsored by:	The FreeBSD Foundation
2016-02-04 21:16:35 +00:00
Jonathan T. Looney
492e9ee5a9 Improvements to the MDXFileChunk() template function:
- Remove unneeded fstat()/lseek() calls.
- Return NULL and set errno to EINVAL on negative length.
- Fix small style problems and expand variable names.

After this change, it is possible to use this code for some irregular
files. For example, 'md5 /dev/md0' should now succeed.

Differential Revision:	https://reviews.freebsd.org/D4748
Suggested by:	bde
Reviewed by:	bde, allanjude, delphij
2016-01-14 21:08:23 +00:00
Bryan Drewery
e3ebb82118 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-01-07 19:58:23 +00:00
Jonathan T. Looney
8d03d93fb7 Fix a file descriptor leak in mdXhl.c (which is used by numerous hashing
algorithms.

CID:	1305669,1305611,1305663,1305603,1305584,1305639,1346865,1305601
Differential Revision:	https://reviews.freebsd.org/D4732
Reviewed by:	allanjude, delphij
MFC after:	2 weeks
Sponsored by:	Juniper Networks
2015-12-30 18:04:50 +00:00
Allan Jude
7a3f5d11fb Replace sys/crypto/sha2/sha2.c with lib/libmd/sha512c.c
cperciva's libmd implementation is 5-30% faster

The same was done for SHA256 previously in r263218

cperciva's implementation was lacking SHA-384 which I implemented, validated against OpenSSL and the NIST documentation

Extend sbin/md5 to create sha384(1)

Chase dependancies on sys/crypto/sha2/sha2.{c,h} and replace them with sha512{c.c,.h}

Reviewed by:	cperciva, des, delphij
Approved by:	secteam, bapt (mentor)
MFC after:	2 weeks
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D3929
2015-12-27 17:33:59 +00:00
Bryan Drewery
5a0d38b3aa META MODE: Remove unneeded libmd.host dependency for xinstall.host.
This is chasing r291026.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:55 +00:00
Bryan Drewery
4aa63711a7 META MODE: Fix 'make bootstrap-tools'.
The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.

This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.

For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds.  This has not yet been a problem due
to readelf not being built as a host tool in buildworld.  This is possible
in the meta build though when building the toolchain.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:51 +00:00
Bryan Drewery
7b3ea376a2 META MODE: Prefer INSTALL=tools/install.sh to lessen the need for xinstall.host.
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:10:28 +00:00
Craig Rodrigues
3a6dcb1fad Add new include path for sha256.h
This fixes the bootstrap build on FreeBSD 10.

Submitted by:	andrew
2015-07-12 03:39:36 +00:00
John-Mark Gurney
2ff9c4f915 Complete the move that was started w/ r263218.. For some reason I
didn't delete the files, so that means we need to bring the changes in
r282726 to the correct files..

make tinderbox completed with this patch...
2015-07-11 03:12:34 +00:00