[PowerPC64] Fix multiple issues in fpsetmask().

Building R exposed a problem in fpsetmask() whereby we were not properly
clamping the provided mask to the valid range.

R initilizes the mask by calling fpsetmask(~0) on FreeBSD. Since we
recently enabled precise exceptions, this was causing an immediate
SIGFPE because we were attempting to set invalid bits in the fpscr.

Properly limit the range of bits that can be set via fpsetmask().

While here, use the correct fp_except_t type instead of fp_rnd_t.

Reported by:	pkubaj (in IRC)
MFC after:	1 week
Sponsored by:	Tag1 Consulting, Inc.
This commit is contained in:
Brandon Bergren 2021-02-28 20:35:53 -06:00
parent 55eb51ab66
commit dd95b39235

View File

@ -43,11 +43,11 @@ fp_except_t
fpsetmask(fp_except_t mask)
{
u_int64_t fpscr;
fp_rnd_t old;
fp_except_t old;
__asm__("mffs %0" : "=f"(fpscr));
old = (fp_rnd_t)((fpscr >> 3) & 0x1f);
fpscr = (fpscr & 0xffffff07) | (mask << 3);
old = (fp_except_t)((fpscr >> 3) & 0x1f);
fpscr = (fpscr & 0xffffff07) | ((mask & 0x1f) << 3);
__asm__ __volatile("mtfsf 0xff,%0" :: "f"(fpscr));
return (old);
}