d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
c0cb813711
freebsd-skq/sys/i386/isa/npx.c

1513 lines
38 KiB
C
Raw Normal View History

Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*-
* Copyright (c) 1990 William Jolitz.
* Copyright (c) 1991 The Regents of the University of California.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
ALL: Removed patch kit headers and rcsid strings, add $Id$. isa.c: Removed old #ifdef notyet isa_configure code, since it will never be used, and I have done 90% of what it attempted to. Add conflict checking code that searchs back through the devtab's looking for any device that has already been found that may conflict with what we are about to probe. Checks are mode for I/O address, memory address, IRQ, and DRQ. This should stop the screwing up of any device that has alread been found by other device probes. Print out messages when we are not going to probe a device due to a conflict so the user knows WHY something was not found. For example: aha0 not probed due to irq conflict with ahb0 at 11 Now print out a message when a device is not found so the user knows that it was probed for, but could not be found. For example: ed1 not found at 0x320 For devices that have I/O address < 0x100 say that they are on the motherboard, not on isa! The 0x100 magic number is per ISA spec. It may seem funny that pc0 and sc0 report as being on the motherboard, but this is due to the fact that the I/O address used is that of the keyboard controller which IS on the motherboard. We really need to split the keyboard probe from the display probe. It is completly legal to build a pc with out one or the other, or even with out both! npx.c: Return -1 from the probe routine if we are using the Emulator so that the i/o addresses are not printed, this is the same trick used for 486's. Do not print the ``Errors reported via Exception 16'', and ``Errors reported via IRQ 13'' messages any more, since these just lead to more user confusion that anything. It still prints the message ``Error reporting broken, using 387 emulator'' so that the person is aware that there mother board is ill.
1993-10-13 15:59:30 +00:00
* from: @(#)npx.c 7.2 (Berkeley) 5/12/91
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Use __FBSDID().
2003-06-02 16:32:55 +00:00
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
#include "opt_cpu.h"
Don't include <isa/isavar.h> or compile code depending on it when isa is not configured. Including <isa/isavar.h> when it is not used is harmful as well as bogus, since it includes "isa_if.h" which is not generated when isa is not configured. This was fixed in 1999 but was broken by unconditionalizing PNPBIOS.
2002-01-30 12:41:12 +00:00
#include "opt_isa.h"
MFpc98: all changes in sys/pc98/pc98/npx.c related to FPU_ERROR_BROKEN. - 1.58 (2000/09/01; author: kato) Fixed FPU_ERROR_BROKEN code. It had old-isa code. - 1.33 (1998/03/09; author: kato) Make FPU_ERROR_BROKEN a new-style option. - 1.7 (1996/10/09; author: asami) Make sure FPU is recognized for non-Intel CPUs. The log for rev.1.7 should have said something like: Added FPU_ERROR_BROKEN option. This forces a successful probe for exception 16, so that hardware with a broken FPU error signal can sort of work.
2001-10-21 05:18:30 +00:00
#include "opt_npx.h"
convert the math emulation to use the new-style options.
1996-01-04 19:51:50 +00:00
Change all #includes to follow the current Berkeley style. Some of these ``changes'' are actually not changes at all, but CVS sometimes has trouble telling the difference. This also includes support for second-directory compiles. This is not quite complete yet, as `config' doesn't yet do the right thing. You can still make it work trivially, however, by doing the following: rm /sys/compile mkdir /usr/obj/sys/compile ln -s M-. /sys/compile cd /sys/i386/conf config MYKERNEL cd ../../compile/MYKERNEL ln -s /sys @ rm machine ln -s @/i386/include machine make depend make
1994-08-13 03:50:34 +00:00
#include <sys/param.h>
#include <sys/systm.h>
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
#include <sys/bus.h>
Sorry, the last commit screwed up for me, this is the right one (I hope!) Please refer to the previous commit message about sysctl variables.
1995-10-28 13:07:28 +00:00
#include <sys/kernel.h>
Undo part of the tangle of having sys/lock.h and sys/mutex.h included in other "system" header files. Also help the deprecation of lockmgr.h by making it a sub-include of sys/lock.h and removing sys/lockmgr.h form kernel .c files. Sort sys/*.h includes where possible in affected files. OK'ed by: bde (with reservations)
2001-05-01 08:13:21 +00:00
#include <sys/lock.h>
Only use the i586-optimized copying and zeroing functions if they are actually faster (more than 20% faster for zeroing 1 MB at boot time). This fixes pessimized copying and zeroing on K6's and perhaps on other CPUs that are misclassified as i586's.
1998-02-12 21:41:10 +00:00
#include <sys/malloc.h>
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
#include <sys/module.h>
- Make npx_intr INTR_MPSAFE and move acquiring Giant into the function itself. - Remove a hack to allow acquiring Giant from the npx asm trap vector.
2001-01-20 02:30:58 +00:00
#include <sys/mutex.h>
Undo part of the tangle of having sys/lock.h and sys/mutex.h included in other "system" header files. Also help the deprecation of lockmgr.h by making it a sub-include of sys/lock.h and removing sys/lockmgr.h form kernel .c files. Sort sys/*.h includes where possible in affected files. OK'ed by: bde (with reservations)
2001-05-01 08:13:21 +00:00
#include <sys/mutex.h>
#include <sys/proc.h>
New APIC support code: - The apic interrupt entry points have been rewritten so that each entry point can serve 32 different vectors. When the entry is executed, it uses one of the 32-bit ISR registers to determine which vector in its assigned range was triggered. Thus, the apic code can support 159 different interrupt vectors with only 5 entry points. - We now always to disable the local APIC to work around an errata in certain PPros and then re-enable it again if we decide to use the APICs to route interrupts. - We no longer map IO APICs or local APICs using special page table entries. Instead, we just use pmap_mapdev(). We also no longer export the virtual address of the local APIC as a global symbol to the rest of the system, but only in local_apic.c. To aid this, the APIC ID of each CPU is exported as a per-CPU variable. - Interrupt sources are provided for each intpin on each IO APIC. Currently, each source is given a unique interrupt vector meaning that PCI interrupts are not shared on most machines with an I/O APIC. That mapping for interrupt sources to interrupt vectors is up to the APIC enumerator driver however. - We no longer probe to see if we need to use mixed mode to route IRQ 0, instead we always use mixed mode to route IRQ 0 for now. This can be disabled via the 'NO_MIXED_MODE' kernel option. - The npx(4) driver now always probes to see if a built-in FPU is present since this test can now be performed with the new APIC code. However, an SMP kernel will panic if there is more than one CPU and a built-in FPU is not found. - PCI interrupts are now properly routed when using APICs to route interrupts, so remove the hack to psuedo-route interrupts when the intpin register was read. - The apic.h header was moved to apicreg.h and a new apicvar.h header that declares the APIs used by the new APIC code was added.
2003-11-03 21:53:38 +00:00
#include <sys/smp.h>
Undo part of the tangle of having sys/lock.h and sys/mutex.h included in other "system" header files. Also help the deprecation of lockmgr.h by making it a sub-include of sys/lock.h and removing sys/lockmgr.h form kernel .c files. Sort sys/*.h includes where possible in affected files. OK'ed by: bde (with reservations)
2001-05-01 08:13:21 +00:00
#include <sys/sysctl.h>
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
#include <machine/bus.h>
#include <sys/rman.h>
Disabled logging of masked exceptions on exit. Keep the side effect of saving the state (see rev.1.17).
1997-01-29 13:46:28 +00:00
#ifdef NPX_DEBUG
Log processes that exit with an masked npx exception that would trap with the current default exception (un)mask. There should be no such processes unless you change the mask. Someday the mask should be changed to the IEEE default of everything masked. The npx state gets saved so that it can be checked and this may have the side effect of fixing a bug that was reported for 1.1.5. (npx exceptions may sometimes leak across exits and clobber another process. I can't see how this can happen.) Get some missing/wrong declarations from headers now that the headers have them.
1994-11-14 14:59:06 +00:00
#include <sys/syslog.h>
Disabled logging of masked exceptions on exit. Keep the side effect of saving the state (see rev.1.17).
1997-01-29 13:46:28 +00:00
#endif
Log processes that exit with an masked npx exception that would trap with the current default exception (un)mask. There should be no such processes unless you change the mask. Someday the mask should be changed to the IEEE default of everything masked. The npx state gets saved so that it can be checked and this may have the side effect of fixing a bug that was reported for 1.1.5. (npx exceptions may sometimes leak across exits and clobber another process. I can't see how this can happen.) Get some missing/wrong declarations from headers now that the headers have them.
1994-11-14 14:59:06 +00:00
#include <sys/signalvar.h>
MFamd64: Explicitly initialize the mxcsr during npxinit().
2014-11-02 21:34:24 +00:00
#include <vm/uma.h>
Finished device configuration database work for all ISA devices (except `ze') and all SCSI devices (except that it's not done quite the way I want). New information added includes: - A text description of the device - A ``state''---unknown, unconfigured, idle, or busy - A generic parent device (with support in the m.i. code) - An interrupt mask type field (which will hopefully go away) so that . ``doconfig'' can be written This requires a new version of the `lsdev' program as well (next commit).
1994-10-23 21:28:03 +00:00
Make the necessary changes so that an ELF kernel can be built. I have successfully built, booted, and run a number of different ELF kernel configurations, including GENERIC. LINT also builds and links cleanly, though I have not tried to boot it. The impact on developers is virtually nil, except for two things. All linker sets that might possibly be present in the kernel must be listed in "sys/i386/i386/setdefs.h". And all C symbols that are also referenced from assembly language code must be listed in "sys/i386/include/asnames.h". It so happens that failure to do these things will have no impact on the a.out kernel. But it will break the build of the ELF kernel. The ELF bootloader works, but it is not ready to commit quite yet.
1997-04-22 06:55:47 +00:00
#include <machine/asmacros.h>
Don't include <machine/cputypes.h> or declare cputype/class interfaces in <machine/cpu.h>. Moved the declarations to <machine/cputypes.h>. Fixed style bugs in the moved code. Fixed everything that depended on the nested include. Don't include <machine/cpu.h> (in the changed files) unless something in it is used directly.
1997-10-28 11:43:57 +00:00
#include <machine/cputypes.h>
#include <machine/frame.h>
Added #include of <machine/md_var.h>. This will be needed when some declarations are moved from <machine/cpufunc.h> to better places.
1996-06-25 20:31:01 +00:00
#include <machine/md_var.h>
Include file updates.. <machine/spl.h> -> <machine/ipl.h>, add <machine/ipl.h> to those files that were depending on getting SWI_* implicitly via <machine/cpufunc.h>
1997-05-31 09:27:31 +00:00
#include <machine/pcb.h>
Don't include <machine/cputypes.h> or declare cputype/class interfaces in <machine/cpu.h>. Moved the declarations to <machine/cputypes.h>. Fixed style bugs in the moved code. Fixed everything that depended on the nested include. Don't include <machine/cpu.h> (in the changed files) unless something in it is used directly.
1997-10-28 11:43:57 +00:00
#include <machine/psl.h>
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
#include <machine/resource.h>
Change all #includes to follow the current Berkeley style. Some of these ``changes'' are actually not changes at all, but CVS sometimes has trouble telling the difference. This also includes support for second-directory compiles. This is not quite complete yet, as `config' doesn't yet do the right thing. You can still make it work trivially, however, by doing the following: rm /sys/compile mkdir /usr/obj/sys/compile ln -s M-. /sys/compile cd /sys/i386/conf config MYKERNEL cd ../../compile/MYKERNEL ln -s /sys @ rm machine ln -s @/i386/include machine make depend make
1994-08-13 03:50:34 +00:00
#include <machine/specialreg.h>
Don't include <machine/cputypes.h> or declare cputype/class interfaces in <machine/cpu.h>. Moved the declarations to <machine/cputypes.h>. Fixed style bugs in the moved code. Fixed everything that depended on the nested include. Don't include <machine/cpu.h> (in the changed files) unless something in it is used directly.
1997-10-28 11:43:57 +00:00
#include <machine/segments.h>
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
#include <machine/ucontext.h>
Finished device configuration database work for all ISA devices (except `ze') and all SCSI devices (except that it's not done quite the way I want). New information added includes: - A text description of the device - A ``state''---unknown, unconfigured, idle, or busy - A generic parent device (with support in the m.i. code) - An interrupt mask type field (which will hopefully go away) so that . ``doconfig'' can be written This requires a new version of the `lsdev' program as well (next commit).
1994-10-23 21:28:03 +00:00
New APIC support code: - The apic interrupt entry points have been rewritten so that each entry point can serve 32 different vectors. When the entry is executed, it uses one of the 32-bit ISR registers to determine which vector in its assigned range was triggered. Thus, the apic code can support 159 different interrupt vectors with only 5 entry points. - We now always to disable the local APIC to work around an errata in certain PPros and then re-enable it again if we decide to use the APICs to route interrupts. - We no longer map IO APICs or local APICs using special page table entries. Instead, we just use pmap_mapdev(). We also no longer export the virtual address of the local APIC as a global symbol to the rest of the system, but only in local_apic.c. To aid this, the APIC ID of each CPU is exported as a per-CPU variable. - Interrupt sources are provided for each intpin on each IO APIC. Currently, each source is given a unique interrupt vector meaning that PCI interrupts are not shared on most machines with an I/O APIC. That mapping for interrupt sources to interrupt vectors is up to the APIC enumerator driver however. - We no longer probe to see if we need to use mixed mode to route IRQ 0, instead we always use mixed mode to route IRQ 0 for now. This can be disabled via the 'NO_MIXED_MODE' kernel option. - The npx(4) driver now always probes to see if a built-in FPU is present since this test can now be performed with the new APIC code. However, an SMP kernel will panic if there is more than one CPU and a built-in FPU is not found. - PCI interrupts are now properly routed when using APICs to route interrupts, so remove the hack to psuedo-route interrupts when the intpin register was read. - The apic.h header was moved to apicreg.h and a new apicvar.h header that declares the APIs used by the new APIC code was added.
2003-11-03 21:53:38 +00:00
#include <machine/intr_machdep.h>
Call in to xen for fpu handling when XEN is set MFC after: 1 month
2008-08-15 21:43:38 +00:00
Don't include <isa/isavar.h> or compile code depending on it when isa is not configured. Including <isa/isavar.h> when it is not used is harmful as well as bogus, since it includes "isa_if.h" which is not generated when isa is not configured. This was fixed in 1999 but was broken by unconditionalizing PNPBIOS.
2002-01-30 12:41:12 +00:00
#ifdef DEV_ISA
Add a little do-nothing ``slopsucker'' device which gives a home to PNP0c04 (legacy ISA coprocessor support). Tourist info.
2000-05-04 23:57:32 +00:00
#include <isa/isavar.h>
Don't include <isa/isavar.h> or compile code depending on it when isa is not configured. Including <isa/isavar.h> when it is not used is harmful as well as bogus, since it includes "isa_if.h" which is not generated when isa is not configured. This was fixed in 1999 but was broken by unconditionalizing PNPBIOS.
2002-01-30 12:41:12 +00:00
#endif
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
Remove the CPU_ENABLE_SSE option from the i386 and pc98 architectures, as they are already default for I686_CPU for almost 3 years, and CPU_DISABLE_SSE always disables it. On the other hand, CPU_ENABLE_SSE does not work for I486_CPU and I586_CPU. This commit has: - Removed the option from conf/options.* - Removed the option and comments from MD NOTES files - Simplified the CPU_ENABLE_SSE ifdef's so they don't deal with CPU_ENABLE_SSE from kernel configuration. (*) For most users, this commit should be largely no-op. If you used to place CPU_ENABLE_SSE into your kernel configuration for some reason, it is time to remove it. (*) The ifdef's of CPU_ENABLE_SSE are not removed at this point, since we need to change it to !defined(CPU_DISABLE_SSE) && defined(I686_CPU), not just !defined(CPU_DISABLE_SSE), if we really want to do so. Discussed on: -arch Approved by: re (scottl)
2005-07-02 20:06:44 +00:00
#if !defined(CPU_DISABLE_SSE) && defined(I686_CPU)
Automatically enable CPU_ENABLE_SSE (detect and enable SSE instructions) if compiling with I686_CPU as a target. CPU_DISABLE_SSE will prevent this from happening and will guarantee the code is not compiled in. I am still not happy with this, but gcc is now generating code that uses these instructions if you set CPUTYPE to p3/p4 or athlon-4/mp/xp or higher.
2002-09-07 07:02:12 +00:00
#define CPU_ENABLE_SSE
#endif
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
* 387 and 287 Numeric Coprocessor Extension (NPX) Driver.
*/
netchild's mega-patch to isolate compiler dependencies into a central place. This moves the dependency on GCC's and other compiler's features into the central sys/cdefs.h file, while the individual source files can then refer to #ifdef __COMPILER_FEATURE_FOO where they by now used to refer to #if __GNUC__ > 3.1415 && __BARC__ <= 42. By now, GCC and ICC (the Intel compiler) have been actively tested on IA32 platforms by netchild. Extension to other compilers is supposed to be possible, of course. Submitted by: netchild Reviewed by: various developers on arch@, some time ago
2005-03-02 21:33:29 +00:00
#if defined(__GNUCLIKE_ASM) && !defined(lint)
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
Simplify fldcw() macro. There is no reason to use pointer here. No object file change after this commit (verified with md5).
2010-07-26 23:20:55 +00:00
#define fldcw(cw) __asm __volatile("fldcw %0" : : "m" (cw))
Reduce diff against fenv.h: Mark all inline asms as volatile for safety. No object file change after this commit (verified with md5).
2010-07-26 22:16:36 +00:00
#define fnclex() __asm __volatile("fnclex")
#define fninit() __asm __volatile("fninit")
Finish supporting compiling with `gcc -ansi'. Fix missing `volatile's in __asm() statements while I'm here.
1998-04-15 18:58:09 +00:00
#define fnsave(addr) __asm __volatile("fnsave %0" : "=m" (*(addr)))
#define fnstcw(addr) __asm __volatile("fnstcw %0" : "=m" (*(addr)))
FNSTSW instruction can use AX register as an operand. Obtained from: fenv.h
2010-07-26 21:24:52 +00:00
#define fnstsw(addr) __asm __volatile("fnstsw %0" : "=am" (*(addr)))
Reduce diff against fenv.h: Mark all inline asms as volatile for safety. No object file change after this commit (verified with md5).
2010-07-26 22:16:36 +00:00
#define fp_divide_by_0() __asm __volatile( \
"fldz; fld1; fdiv %st,%st(1); fnop")
#define frstor(addr) __asm __volatile("frstor %0" : : "m" (*(addr)))
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
#ifdef CPU_ENABLE_SSE
Reduce diff against fenv.h: Mark all inline asms as volatile for safety. No object file change after this commit (verified with md5).
2010-07-26 22:16:36 +00:00
#define fxrstor(addr) __asm __volatile("fxrstor %0" : : "m" (*(addr)))
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
#define fxsave(addr) __asm __volatile("fxsave %0" : "=m" (*(addr)))
MFamd64: Explicitly initialize the mxcsr during npxinit().
2014-11-02 21:34:24 +00:00
#define ldmxcsr(csr) __asm __volatile("ldmxcsr %0" : : "m" (csr))
MFamd64 r238597: Add stmxcsr. MFC after: 3 weeks
2012-07-21 21:39:23 +00:00
#define stmxcsr(addr) __asm __volatile("stmxcsr %0" : : "m" (*(addr)))
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
static __inline void
xrstor(char *addr, uint64_t mask)
{
uint32_t low, hi;
low = mask;
hi = mask >> 32;
__asm __volatile("xrstor %0" : : "m" (*addr), "a" (low), "d" (hi));
}
static __inline void
xsave(char *addr, uint64_t mask)
{
uint32_t low, hi;
low = mask;
hi = mask >> 32;
__asm __volatile("xsave %0" : "=m" (*addr) : "a" (low), "d" (hi) :
"memory");
}
static __inline void
xsaveopt(char *addr, uint64_t mask)
{
uint32_t low, hi;
low = mask;
hi = mask >> 32;
__asm __volatile("xsaveopt %0" : "=m" (*addr) : "a" (low), "d" (hi) :
"memory");
}
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
#endif
netchild's mega-patch to isolate compiler dependencies into a central place. This moves the dependency on GCC's and other compiler's features into the central sys/cdefs.h file, while the individual source files can then refer to #ifdef __COMPILER_FEATURE_FOO where they by now used to refer to #if __GNUC__ > 3.1415 && __BARC__ <= 42. By now, GCC and ICC (the Intel compiler) have been actively tested on IA32 platforms by netchild. Extension to other compilers is supposed to be possible, of course. Submitted by: netchild Reviewed by: various developers on arch@, some time ago
2005-03-02 21:33:29 +00:00
#else /* !(__GNUCLIKE_ASM && !lint) */
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
Simplify fldcw() macro. There is no reason to use pointer here. No object file change after this commit (verified with md5).
2010-07-26 23:20:55 +00:00
void fldcw(u_short cw);
Fixed some style bugs in the removal of __P(()). The main ones were not removing tabs before "__P((", and not outdenting continuation lines to preserve non-KNF lining up of code with parentheses. Switch to KNF formatting and/or rewrap the whole prototype in some cases.
2002-03-23 16:01:49 +00:00
void fnclex(void);
void fninit(void);
void fnsave(caddr_t addr);
void fnstcw(caddr_t addr);
void fnstsw(caddr_t addr);
void fp_divide_by_0(void);
void frstor(caddr_t addr);
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
#ifdef CPU_ENABLE_SSE
Fixed some style bugs in the removal of __P(()). The main ones were not removing tabs before "__P((", and not outdenting continuation lines to preserve non-KNF lining up of code with parentheses. Switch to KNF formatting and/or rewrap the whole prototype in some cases.
2002-03-23 16:01:49 +00:00
void fxsave(caddr_t addr);
void fxrstor(caddr_t addr);
MFamd64: Explicitly initialize the mxcsr during npxinit().
2014-11-02 21:34:24 +00:00
void ldmxcsr(u_int csr);
Change (unused) prototype for stmxcsr() to match reality. Noted by: jhb MFC after: 1 week
2012-07-30 19:26:02 +00:00
void stmxcsr(u_int *csr);
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
void xrstor(char *addr, uint64_t mask);
void xsave(char *addr, uint64_t mask);
void xsaveopt(char *addr, uint64_t mask);
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
#endif
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
netchild's mega-patch to isolate compiler dependencies into a central place. This moves the dependency on GCC's and other compiler's features into the central sys/cdefs.h file, while the individual source files can then refer to #ifdef __COMPILER_FEATURE_FOO where they by now used to refer to #if __GNUC__ > 3.1415 && __BARC__ <= 42. By now, GCC and ICC (the Intel compiler) have been actively tested on IA32 platforms by netchild. Extension to other compilers is supposed to be possible, of course. Submitted by: netchild Reviewed by: various developers on arch@, some time ago
2005-03-02 21:33:29 +00:00
#endif /* __GNUCLIKE_ASM && !lint */
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
Add a clts() wrapper around the 'clts' instruction to <machine/cpufunc.h> on x86 and use that to implement stop_emulating() in the fpu/npx code. Reimplement start_emulating() in the non-XEN case by using load_cr0() and rcr0() instead of the 'lmsw' and 'smsw' instructions. Intel explicitly discourages the use of 'lmsw' and 'smsw' on 80386 and later processors in the description of these instructions in Volume 2 of the ADM. Reviewed by: kib MFC after: 1 month
2012-07-09 20:55:39 +00:00
#define start_emulating() load_cr0(rcr0() | CR0_TS)
#define stop_emulating() clts()
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
#ifdef CPU_ENABLE_SSE
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
#define GET_FPU_CW(thread) \
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
(cpu_fxsr ? \
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
(thread)->td_pcb->pcb_save->sv_xmm.sv_env.en_cw : \
(thread)->td_pcb->pcb_save->sv_87.sv_env.en_cw)
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
#define GET_FPU_SW(thread) \
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
(cpu_fxsr ? \
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
(thread)->td_pcb->pcb_save->sv_xmm.sv_env.en_sw : \
(thread)->td_pcb->pcb_save->sv_87.sv_env.en_sw)
A better fix for handling different FPU initial control words for different ABIs: - Store the FPU initial control word in the pcb for each thread. - When first using the FPU, load the initial control word after restoring the clean state if it is not the standard control word. - Provide a correct control word for Linux/i386 binaries under FreeBSD/amd64. - Adjust the control word returned for fpugetregs()/npxgetregs() when a thread hasn't used the FPU yet to reflect the real initial control word for the current ABI. - The Linux/i386 ABI for FreeBSD/i386 now properly sets the right control word instead of trashing whatever the current state of the FPU is. Reviewed by: bde
2009-03-05 19:42:11 +00:00
#define SET_FPU_CW(savefpu, value) do { \
if (cpu_fxsr) \
(savefpu)->sv_xmm.sv_env.en_cw = (value); \
else \
(savefpu)->sv_87.sv_env.en_cw = (value); \
} while (0)
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
#else /* CPU_ENABLE_SSE */
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
#define GET_FPU_CW(thread) \
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
(thread->td_pcb->pcb_save->sv_87.sv_env.en_cw)
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
#define GET_FPU_SW(thread) \
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
(thread->td_pcb->pcb_save->sv_87.sv_env.en_sw)
A better fix for handling different FPU initial control words for different ABIs: - Store the FPU initial control word in the pcb for each thread. - When first using the FPU, load the initial control word after restoring the clean state if it is not the standard control word. - Provide a correct control word for Linux/i386 binaries under FreeBSD/amd64. - Adjust the control word returned for fpugetregs()/npxgetregs() when a thread hasn't used the FPU yet to reflect the real initial control word for the current ABI. - The Linux/i386 ABI for FreeBSD/i386 now properly sets the right control word instead of trashing whatever the current state of the FPU is. Reviewed by: bde
2009-03-05 19:42:11 +00:00
#define SET_FPU_CW(savefpu, value) \
(savefpu)->sv_87.sv_env.en_cw = (value)
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
#endif /* CPU_ENABLE_SSE */
Correct a local information leakage bug affecting AMD FPUs. Security: FreeBSD-SA-06:14.fpu
2006-04-19 07:00:19 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
CTASSERT(sizeof(union savefpu) == 512);
CTASSERT(sizeof(struct xstate_hdr) == 64);
CTASSERT(sizeof(struct savefpu_ymm) == 832);
/*
* This requirement is to make it easier for asm code to calculate
* offset of the fpu save area from the pcb address. FPU save area
* must be 64-byte aligned.
*/
CTASSERT(sizeof(struct pcb) % XSAVE_AREA_ALIGN == 0);
Improve support for XSAVE with debuggers. - Dump an NT_X86_XSTATE note if XSAVE is in use. This note is designed to match what Linux does in that 1) it dumps the entire XSAVE area including the fxsave state, and 2) it stashes a copy of the current xsave mask in the unused padding between the fxsave state and the xstate header at the same location used by Linux. - Teach readelf() to recognize NT_X86_XSTATE notes. - Change PT_GET/SETXSTATE to take the entire XSAVE state instead of only the extra portion. This avoids having to always make two ptrace() calls to get or set the full XSAVE state. - Add a PT_GET_XSTATE_INFO which returns the length of the current XSTATE save area (so the size of the buffer needed for PT_GETXSTATE) and the current XSAVE mask (%xcr0). Differential Revision: https://reviews.freebsd.org/D1193 Reviewed by: kib MFC after: 2 weeks
2014-11-21 20:53:17 +00:00
/*
* Ensure the copy of XCR0 saved in a core is contained in the padding
* area.
*/
CTASSERT(X86_XSTATE_XCR0_OFFSET >= offsetof(struct savexmm, sv_pad) &&
X86_XSTATE_XCR0_OFFSET + sizeof(uint64_t) <= sizeof(struct savexmm));
Correct a local information leakage bug affecting AMD FPUs. Security: FreeBSD-SA-06:14.fpu
2006-04-19 07:00:19 +00:00
static void fpu_clean_state(void);
#endif
Add getcontext, setcontext, and swapcontext as system calls. Previously these were libc functions but were requested to be made into system calls for atomicity and to coalesce what might be two entrances into the kernel (signal mask setting and floating point trap) into one. A few style nits and comments from bde are also included. Tested on alpha by: gallatin
2002-11-16 06:35:53 +00:00
static void fpusave(union savefpu *);
static void fpurstor(union savefpu *);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
int hw_float;
Sorry, the last commit screwed up for me, this is the right one (I hope!) Please refer to the previous commit message about sysctl variables.
1995-10-28 13:07:28 +00:00
Some whitespace and style fixes. Submitted by: bde (partly)
2009-02-23 15:39:24 +00:00
SYSCTL_INT(_hw, HW_FLOATINGPT, floatingpoint, CTLFLAG_RD,
&hw_float, 0, "Floating point instructions executed in hardware");
Sorry, the last commit screwed up for me, this is the right one (I hope!) Please refer to the previous commit message about sysctl variables.
1995-10-28 13:07:28 +00:00
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
int use_xsave;
uint64_t xsave_mask;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
static uma_zone_t fpu_save_area_zone;
static union savefpu *npx_initialstate;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
struct xsave_area_elm_descr {
u_int offset;
u_int size;
} *xsave_area_desc;
static int use_xsaveopt;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
The "easy" fixes for compiling the kernel -Wunused: remove unreferenced static and local variables, goto labels, and functions declared but not defined.
1998-12-07 21:58:50 +00:00
static volatile u_int npx_traps_while_probing;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
Deleted most of npxprobe(), and merged npxprobe1() back into npxprobe(). Use the normal interrupt handler (npx_intr()) instead of a special probe-time interrupt handler, although this causes problems due to the bus_teardown_intr() not actually even tearing down the interrupt (these problems were avoided by doing interrupt attachment for the special interrupt handler directly). Fixed minor bitrot in comments. The reason for the npxprobe()/npxprobe1() split mostly went away at about the same time it was made (in 1992 or 1993 just before the beginning of history). 386BSD ran all probes with interrupts completely masked, and I didn't want to disturb this when I added an irq probe to npxprobe(). An irq (not necessarily npx) must be acked for at least external npx's to take the cpu out of the wait state that it enters when an npx error occurs, so the probe must be done with a suitable irq unmasked. npxprobe() went to great lengths to unmask precisely the npx irq. Running probes with all interrupts masked was never really needed in FreeBSD, since FreeBSD always masked interrupts well enough using splhigh(), but it wasn't until rev.1.48 (1995/12/12) of autoconf.c that all probes were run with CPU interrupts enabled. This permits npxprobe() to probe its irq using normal interrupt resources. Note that most drivers still can't depend on this. It depends on the interrupt handler being fast and the irq not being shared.
2001-10-16 14:12:35 +00:00
alias_for_inthand_t probetrap;
Support compiling with `gcc -pedantic' (don't use hard newlines in (asm) string constants or hard long long constants).
1998-04-19 15:39:26 +00:00
__asm(" \n\
.text \n\
.p2align 2,0x90 \n\
Fixed profiling of elf kernels. Made high resolution profiling compile for elf kernels (it is broken for all kernels due to lack of egcs support). Renaming of many assembler labels is avoided by declaring by declaring the labels that need to be visible to gprof as having type "function" and depending on the elf version of gprof being zealous about discarding the others. A few type declarations are still missing, mainly for SMP. PR: 9413 Submitted by: Assar Westerlund <assar@sics.se> (initial parts)
1999-05-06 09:44:57 +00:00
.type " __XSTRING(CNAME(probetrap)) ",@function \n\
Support compiling with `gcc -pedantic' (don't use hard newlines in (asm) string constants or hard long long constants).
1998-04-19 15:39:26 +00:00
" __XSTRING(CNAME(probetrap)) ": \n\
ss \n\
incl " __XSTRING(CNAME(npx_traps_while_probing)) " \n\
fnclex \n\
iret \n\
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
");
Made the SMP case ignore the possibility of an INT13 interface. This eliminates all the APIC code, and thus several routines that would otherwise need to be made MP-safe. Reviewed by: Bruce Evans <bde@zeta.org.au>
1997-07-21 07:57:50 +00:00
Add an identify method to allow npx to arrange itself to be attached to the nexus without explicit code in the nexus to do so.
1999-08-22 19:52:51 +00:00
/*
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
* Determine if an FPU is present and how to use it.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
static int
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
npx_probe(void)
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
{
Deleted most of npxprobe(), and merged npxprobe1() back into npxprobe(). Use the normal interrupt handler (npx_intr()) instead of a special probe-time interrupt handler, although this causes problems due to the bus_teardown_intr() not actually even tearing down the interrupt (these problems were avoided by doing interrupt attachment for the special interrupt handler directly). Fixed minor bitrot in comments. The reason for the npxprobe()/npxprobe1() split mostly went away at about the same time it was made (in 1992 or 1993 just before the beginning of history). 386BSD ran all probes with interrupts completely masked, and I didn't want to disturb this when I added an irq probe to npxprobe(). An irq (not necessarily npx) must be acked for at least external npx's to take the cpu out of the wait state that it enters when an npx error occurs, so the probe must be done with a suitable irq unmasked. npxprobe() went to great lengths to unmask precisely the npx irq. Running probes with all interrupts masked was never really needed in FreeBSD, since FreeBSD always masked interrupts well enough using splhigh(), but it wasn't until rev.1.48 (1995/12/12) of autoconf.c that all probes were run with CPU interrupts enabled. This permits npxprobe() to probe its irq using normal interrupt resources. Note that most drivers still can't depend on this. It depends on the interrupt handler being fast and the irq not being shared.
2001-10-16 14:12:35 +00:00
struct gate_descriptor save_idt_npxtrap;
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
u_short control, status;
Deleted most of npxprobe(), and merged npxprobe1() back into npxprobe(). Use the normal interrupt handler (npx_intr()) instead of a special probe-time interrupt handler, although this causes problems due to the bus_teardown_intr() not actually even tearing down the interrupt (these problems were avoided by doing interrupt attachment for the special interrupt handler directly). Fixed minor bitrot in comments. The reason for the npxprobe()/npxprobe1() split mostly went away at about the same time it was made (in 1992 or 1993 just before the beginning of history). 386BSD ran all probes with interrupts completely masked, and I didn't want to disturb this when I added an irq probe to npxprobe(). An irq (not necessarily npx) must be acked for at least external npx's to take the cpu out of the wait state that it enters when an npx error occurs, so the probe must be done with a suitable irq unmasked. npxprobe() went to great lengths to unmask precisely the npx irq. Running probes with all interrupts masked was never really needed in FreeBSD, since FreeBSD always masked interrupts well enough using splhigh(), but it wasn't until rev.1.48 (1995/12/12) of autoconf.c that all probes were run with CPU interrupts enabled. This permits npxprobe() to probe its irq using normal interrupt resources. Note that most drivers still can't depend on this. It depends on the interrupt handler being fast and the irq not being shared.
2001-10-16 14:12:35 +00:00
- Don't set CR0_NE and CR0_MP in npx_probe() as they are already set earlier in cpu_setregs(). - If we know this CPU has a FPU via cpuid, then just assume the INT16 interface and make the npx device quiet to not clutter the dmesg. This is true for all Pentium and later CPUs and even some of the later 486dx CPUs. Reviewed by: bde Tested by: ps MFC after: 1 week
2006-04-06 17:17:45 +00:00
/*
* Modern CPUs all have an FPU that uses the INT16 interface
* and provide a simple way to verify that, so handle the
* common case right away.
*/
if (cpu_feature & CPUID_FPU) {
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
hw_float = 1;
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
return (1);
- Don't set CR0_NE and CR0_MP in npx_probe() as they are already set earlier in cpu_setregs(). - If we know this CPU has a FPU via cpuid, then just assume the INT16 interface and make the npx device quiet to not clutter the dmesg. This is true for all Pentium and later CPUs and even some of the later 486dx CPUs. Reviewed by: bde Tested by: ps MFC after: 1 week
2006-04-06 17:17:45 +00:00
}
Add constants for entries in the IDT and use those instead of magic numbers.
2003-09-10 01:07:04 +00:00
save_idt_npxtrap = idt[IDT_MF];
setidt(IDT_MF, probetrap, SDT_SYS386TGT, SEL_KPL,
GSEL(GCODE_SEL, SEL_KPL));
Eleminate 3 PC98 ifdefs: o Use IP_NPX in preference to hard coded value to write 0 to clear busy# o Use md macro for a full reset of the npx o Use IRQ_NPX in preference to hard coded value for each platform. # The other two ifdefs in this file are hard to remove
2005-03-16 20:46:16 +00:00
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
- Don't set CR0_NE and CR0_MP in npx_probe() as they are already set earlier in cpu_setregs(). - If we know this CPU has a FPU via cpuid, then just assume the INT16 interface and make the npx device quiet to not clutter the dmesg. This is true for all Pentium and later CPUs and even some of the later 486dx CPUs. Reviewed by: bde Tested by: ps MFC after: 1 week
2006-04-06 17:17:45 +00:00
* Don't trap while we're probing.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
stop_emulating();
- Don't set CR0_NE and CR0_MP in npx_probe() as they are already set earlier in cpu_setregs(). - If we know this CPU has a FPU via cpuid, then just assume the INT16 interface and make the npx device quiet to not clutter the dmesg. This is true for all Pentium and later CPUs and even some of the later 486dx CPUs. Reviewed by: bde Tested by: ps MFC after: 1 week
2006-04-06 17:17:45 +00:00
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
* Finish resetting the coprocessor, if any. If there is an error
Deleted most of npxprobe(), and merged npxprobe1() back into npxprobe(). Use the normal interrupt handler (npx_intr()) instead of a special probe-time interrupt handler, although this causes problems due to the bus_teardown_intr() not actually even tearing down the interrupt (these problems were avoided by doing interrupt attachment for the special interrupt handler directly). Fixed minor bitrot in comments. The reason for the npxprobe()/npxprobe1() split mostly went away at about the same time it was made (in 1992 or 1993 just before the beginning of history). 386BSD ran all probes with interrupts completely masked, and I didn't want to disturb this when I added an irq probe to npxprobe(). An irq (not necessarily npx) must be acked for at least external npx's to take the cpu out of the wait state that it enters when an npx error occurs, so the probe must be done with a suitable irq unmasked. npxprobe() went to great lengths to unmask precisely the npx irq. Running probes with all interrupts masked was never really needed in FreeBSD, since FreeBSD always masked interrupts well enough using splhigh(), but it wasn't until rev.1.48 (1995/12/12) of autoconf.c that all probes were run with CPU interrupts enabled. This permits npxprobe() to probe its irq using normal interrupt resources. Note that most drivers still can't depend on this. It depends on the interrupt handler being fast and the irq not being shared.
2001-10-16 14:12:35 +00:00
* pending, then we may get a bogus IRQ13, but npx_intr() will handle
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
* it OK. Bogus halts have never been observed, but we enabled
* IRQ13 and cleared the BUSY# latch early to handle them anyway.
*/
fninit();
Made the SMP case ignore the possibility of an INT13 interface. This eliminates all the APIC code, and thus several routines that would otherwise need to be made MP-safe. Reviewed by: Bruce Evans <bde@zeta.org.au>
1997-07-21 07:57:50 +00:00
Submitted by: seb@erix.ericsson.se (Sebastian Strollo) Remove over-cautious early fnop() synchronization. It caused the probe to hang on systems without an FPU.
1995-02-23 17:32:38 +00:00
/*
* Don't use fwait here because it might hang.
* Don't use fnop here because it usually hangs if there is no FPU.
*/
Use sufficient parentheses in macros. Remove bogus input operands for fnsave(), fnstcw() and fnstsw(). Change all fwait's to fnop's. This might help avoid hardware bugs. Wait after fninit with an fnop. This should be safer now. Fix some spelling and formatting errors. Use natural sizes for control and status words (u_short, promotes to int). Don't clobber the SWI_CLOCK_MASK bits in npx0_imask when using IRQ13. Set the devconf state correctly (always busy, if configured). Improve code for npx_registerdev() a little (gcc can't keep id->id_unit in a register for some reason). Don't register a nonexistent npx device. Print a useful message in npxattach() again (delete references to errors and not the whole message). Don't print "387 emulator" if there is no emulator in the kernel. Use %p for pointers in error messages. Don't clobber the FPU state when there is an FPU exception. Just clear the exception flags (after saving the flags as before). This allows debuggers and SIGFPE handlers to look at the full exception state. SIGFPE handlers should normally return via longjmp(), which restores a good FPU state (as before). Returning from a SIGFPE handler may leave the FPU in the wrong state (as before). Clear the busy latch _after_ clearing the exception flags so that there is less chance of getting a bogus h/w interrupt for a control operation. Clear the saved exception status word when the next FPU instruction is excuted so that it doesn't stick around until the next exception. Clear the busy latch after fnsave() in npxsave() in case it was set when npxsave() was called.
1995-01-03 04:00:06 +00:00
DELAY(1000); /* wait for any IRQ13 */
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
#ifdef DIAGNOSTIC
if (npx_traps_while_probing != 0)
printf("fninit caused %u bogus npx trap(s)\n",
npx_traps_while_probing);
#endif
/*
* Check for a status of mostly zero.
*/
status = 0x5a5a;
fnstsw(&status);
if ((status & 0xb8ff) == 0) {
/*
* Good, now check for a proper control word.
*/
Remove trailing whitespace.
1995-05-30 08:16:23 +00:00
control = 0x5a5a;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
fnstcw(&control);
if ((control & 0x1f3f) == 0x033f) {
/*
* We have an npx, now divide by 0 to see if exception
* 16 works.
*/
control &= ~(1 << 2); /* enable divide by 0 trap */
Simplify fldcw() macro. There is no reason to use pointer here. No object file change after this commit (verified with md5).
2010-07-26 23:20:55 +00:00
fldcw(control);
MFpc98: all changes in sys/pc98/pc98/npx.c related to FPU_ERROR_BROKEN. - 1.58 (2000/09/01; author: kato) Fixed FPU_ERROR_BROKEN code. It had old-isa code. - 1.33 (1998/03/09; author: kato) Make FPU_ERROR_BROKEN a new-style option. - 1.7 (1996/10/09; author: asami) Make sure FPU is recognized for non-Intel CPUs. The log for rev.1.7 should have said something like: Added FPU_ERROR_BROKEN option. This forces a successful probe for exception 16, so that hardware with a broken FPU error signal can sort of work.
2001-10-21 05:18:30 +00:00
#ifdef FPU_ERROR_BROKEN
/*
* FPU error signal doesn't work on some CPU
* accelerator board.
*/
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
hw_float = 1;
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
return (1);
MFpc98: all changes in sys/pc98/pc98/npx.c related to FPU_ERROR_BROKEN. - 1.58 (2000/09/01; author: kato) Fixed FPU_ERROR_BROKEN code. It had old-isa code. - 1.33 (1998/03/09; author: kato) Make FPU_ERROR_BROKEN a new-style option. - 1.7 (1996/10/09; author: asami) Make sure FPU is recognized for non-Intel CPUs. The log for rev.1.7 should have said something like: Added FPU_ERROR_BROKEN option. This forces a successful probe for exception 16, so that hardware with a broken FPU error signal can sort of work.
2001-10-21 05:18:30 +00:00
#endif
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
npx_traps_while_probing = 0;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
fp_divide_by_0();
if (npx_traps_while_probing != 0) {
/*
* Good, exception 16 works.
*/
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
hw_float = 1;
goto cleanup;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
printf(
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
"FPU does not use exception 16 for error reporting\n");
goto cleanup;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
}
Some cleanups to the i386 FPU support: - Remove the control word parameter to npxinit(). It was always set to __INITIAL_NPXCW__. - Remove npx_cleanstate_ready as the cleanstate is always initalized when it is used. - Improve the handling of the case when the FPU isn't present. Now the npx0 device no longer succeeds in its probe so all of npx_attach() is skipped. Also, we allow this case with SMP (though that shouldn't actually occur as all i386 systems that support SMP have FPUs) now. SMP was only an issue back when we had an FPU emulator which was not per-CPU. - MFamd64: Clear some of the state in npx_cleanstate rather than leaving it as garbage. - MFamd64: When a user thread first uses the FPU, use npx_cleanstate for the initial FPU state. Reviewed by: bde
2009-03-05 18:32:43 +00:00
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
/*
* Probe failed. Floating point simply won't work.
* Notify user and disable FPU/MMX/SSE instruction execution.
*/
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
printf("WARNING: no FPU!\n");
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
__asm __volatile("smsw %%ax; orb %0,%%al; lmsw %%ax" : :
"n" (CR0_EM | CR0_MP) : "ax");
Some cleanups to the i386 FPU support: - Remove the control word parameter to npxinit(). It was always set to __INITIAL_NPXCW__. - Remove npx_cleanstate_ready as the cleanstate is always initalized when it is used. - Improve the handling of the case when the FPU isn't present. Now the npx0 device no longer succeeds in its probe so all of npx_attach() is skipped. Also, we allow this case with SMP (though that shouldn't actually occur as all i386 systems that support SMP have FPUs) now. SMP was only an issue back when we had an FPU emulator which was not per-CPU. - MFamd64: Clear some of the state in npx_cleanstate rather than leaving it as garbage. - MFamd64: When a user thread first uses the FPU, use npx_cleanstate for the initial FPU state. Reviewed by: bde
2009-03-05 18:32:43 +00:00
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
cleanup:
Add constants for entries in the IDT and use those instead of magic numbers.
2003-09-10 01:07:04 +00:00
idt[IDT_MF] = save_idt_npxtrap;
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
return (hw_float);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
* Enable XSAVE if supported and allowed by user.
* Calculate the xsave_mask.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
static void
npxinit_bsp1(void)
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
{
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
u_int cp[4];
uint64_t xsave_mask_user;
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (cpu_fxsr && (cpu_feature2 & CPUID2_XSAVE) != 0) {
use_xsave = 1;
TUNABLE_INT_FETCH("hw.use_xsave", &use_xsave);
}
if (!use_xsave)
return;
Replaced I586_OPTIMIZED_BCOPY and I586_OPTIMIZED_BZERO with boot-time negative-logic flags (flags 0x01 and 0x02 for npx0, defaulting to unset = on). This changes the default from off to on. The options have been in current for several months with no problems reported. Added a boot-time negative-logic flag for the old I5886_FAST_BCOPY option which went away too soon (flag 0x04 for npx0, defaulting to unset = on). Added a boot-time way to set the memory size (iosiz in config, iosize in userconfig for npx0). LINT: Removed old options. Documented npx0's flags and iosiz. options.i386: Removed old options. identcpu.c: Don't set the function pointers here. Setting them has to be delayed until after userconfig has had a chance to disable them and until after a good npx0 has been detected. machdep.c: Use npx0's iosize instead of MAXMEM if it is nonzero. support.s: Added vectors and glue code for copyin() and copyout(). Fixed ifdefs for i586_bzero(). Added ifdefs for i586_bcopy(). npx.c: Set the function pointers here. Clear hw_float when an npx exists but is too broken to use. Restored style from a year or three ago in npxattach().
1996-11-11 20:39:03 +00:00
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
cpuid_count(0xd, 0x0, cp);
xsave_mask = XFEATURE_ENABLED_X87 | XFEATURE_ENABLED_SSE;
if ((cp[0] & xsave_mask) != xsave_mask)
panic("CPU0 does not support X87 or SSE: %x", cp[0]);
xsave_mask = ((uint64_t)cp[3] << 32) | cp[0];
xsave_mask_user = xsave_mask;
TUNABLE_QUAD_FETCH("hw.xsave_mask", &xsave_mask_user);
xsave_mask_user |= XFEATURE_ENABLED_X87 | XFEATURE_ENABLED_SSE;
xsave_mask &= xsave_mask_user;
if ((xsave_mask & XFEATURE_AVX512) != XFEATURE_AVX512)
xsave_mask &= ~XFEATURE_AVX512;
if ((xsave_mask & XFEATURE_MPX) != XFEATURE_MPX)
xsave_mask &= ~XFEATURE_MPX;
cpuid_count(0xd, 0x1, cp);
if ((cp[0] & CPUID_EXTSTATE_XSAVEOPT) != 0)
use_xsaveopt = 1;
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
/*
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
* Calculate the fpu save area size.
*/
static void
npxinit_bsp2(void)
{
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
u_int cp[4];
if (use_xsave) {
cpuid_count(0xd, 0x0, cp);
cpu_max_ext_state_size = cp[1];
/*
* Reload the cpu_feature2, since we enabled OSXSAVE.
*/
do_cpuid(1, cp);
cpu_feature2 = cp[2];
} else
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
cpu_max_ext_state_size = sizeof(union savefpu);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
/*
* Initialize floating point unit.
*/
void
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
npxinit(bool bsp)
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
{
The per-cpu temporary buffers are not needed since the pcb_save areas have the proper alignment. Change dummy variable in npxinit from stack to bss to ensure proper alignment. Reviewed by: bde
2001-07-17 13:06:47 +00:00
static union savefpu dummy;
Use 'saveintr' instead of 'savecrit' or 'eflags' to hold the state returned by intr_disable(). Requested by: bde
2010-10-25 15:31:13 +00:00
register_t saveintr;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
u_int mxcsr;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
Some cleanups to the i386 FPU support: - Remove the control word parameter to npxinit(). It was always set to __INITIAL_NPXCW__. - Remove npx_cleanstate_ready as the cleanstate is always initalized when it is used. - Improve the handling of the case when the FPU isn't present. Now the npx0 device no longer succeeds in its probe so all of npx_attach() is skipped. Also, we allow this case with SMP (though that shouldn't actually occur as all i386 systems that support SMP have FPUs) now. SMP was only an issue back when we had an FPU emulator which was not per-CPU. - MFamd64: Clear some of the state in npx_cleanstate rather than leaving it as garbage. - MFamd64: When a user thread first uses the FPU, use npx_cleanstate for the initial FPU state. Reviewed by: bde
2009-03-05 18:32:43 +00:00
u_short control;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (bsp) {
if (!npx_probe())
return;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
npxinit_bsp1();
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (use_xsave) {
load_cr4(rcr4() | CR4_XSAVE);
load_xcr(XCR0, xsave_mask);
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
/*
* XCR0 shall be set up before CPU can report the save area size.
*/
if (bsp)
npxinit_bsp2();
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
* fninit has the same h/w bugs as fnsave. Use the detoxified
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
* fnsave to throw away any junk in the fpu. fpusave() initializes
* the fpu.
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
*
* It is too early for critical_enter() to work on AP.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Use 'saveintr' instead of 'savecrit' or 'eflags' to hold the state returned by intr_disable(). Requested by: bde
2010-10-25 15:31:13 +00:00
saveintr = intr_disable();
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
stop_emulating();
Explicitly initialize the fpu when SSE is enabled since this no longer happens as a side effect of calling npxsave. Reviewed by: peter, bde
2001-10-15 20:18:06 +00:00
#ifdef CPU_ENABLE_SSE
if (cpu_fxsr)
fninit();
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
else
Explicitly initialize the fpu when SSE is enabled since this no longer happens as a side effect of calling npxsave. Reviewed by: peter, bde
2001-10-15 20:18:06 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
fnsave(&dummy);
Some cleanups to the i386 FPU support: - Remove the control word parameter to npxinit(). It was always set to __INITIAL_NPXCW__. - Remove npx_cleanstate_ready as the cleanstate is always initalized when it is used. - Improve the handling of the case when the FPU isn't present. Now the npx0 device no longer succeeds in its probe so all of npx_attach() is skipped. Also, we allow this case with SMP (though that shouldn't actually occur as all i386 systems that support SMP have FPUs) now. SMP was only an issue back when we had an FPU emulator which was not per-CPU. - MFamd64: Clear some of the state in npx_cleanstate rather than leaving it as garbage. - MFamd64: When a user thread first uses the FPU, use npx_cleanstate for the initial FPU state. Reviewed by: bde
2009-03-05 18:32:43 +00:00
control = __INITIAL_NPXCW__;
Simplify fldcw() macro. There is no reason to use pointer here. No object file change after this commit (verified with md5).
2010-07-26 23:20:55 +00:00
fldcw(control);
MFamd64: Explicitly initialize the mxcsr during npxinit().
2014-11-02 21:34:24 +00:00
#ifdef CPU_ENABLE_SSE
if (cpu_fxsr) {
mxcsr = __INITIAL_MXCSR__;
ldmxcsr(mxcsr);
}
#endif
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
start_emulating();
Use 'saveintr' instead of 'savecrit' or 'eflags' to hold the state returned by intr_disable(). Requested by: bde
2010-10-25 15:31:13 +00:00
intr_restore(saveintr);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
/*
* On the boot CPU we generate a clean state that is used to
* initialize the floating point unit when it is first used by a
* process.
*/
static void
npxinitstate(void *arg __unused)
{
register_t saveintr;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
int cp[4], i, max_ext_n;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (!hw_float)
return;
npx_initialstate = malloc(cpu_max_ext_state_size, M_DEVBUF,
M_WAITOK | M_ZERO);
saveintr = intr_disable();
stop_emulating();
fpusave(npx_initialstate);
#ifdef CPU_ENABLE_SSE
if (cpu_fxsr) {
if (npx_initialstate->sv_xmm.sv_env.en_mxcsr_mask)
cpu_mxcsr_mask =
npx_initialstate->sv_xmm.sv_env.en_mxcsr_mask;
else
cpu_mxcsr_mask = 0xFFBF;
/*
* The fninit instruction does not modify XMM
* registers. The fpusave call dumped the garbage
* contained in the registers after reset to the
* initial state saved. Clear XMM registers file
* image to make the startup program state and signal
* handler XMM register content predictable.
*/
bzero(npx_initialstate->sv_xmm.sv_fp,
sizeof(npx_initialstate->sv_xmm.sv_fp));
bzero(npx_initialstate->sv_xmm.sv_xmm,
sizeof(npx_initialstate->sv_xmm.sv_xmm));
} else
#endif
bzero(npx_initialstate->sv_87.sv_ac,
sizeof(npx_initialstate->sv_87.sv_ac));
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
/*
* Create a table describing the layout of the CPU Extended
* Save Area.
*/
if (use_xsave) {
if (xsave_mask >> 32 != 0)
max_ext_n = fls(xsave_mask >> 32) + 32;
else
max_ext_n = fls(xsave_mask);
xsave_area_desc = malloc(max_ext_n * sizeof(struct
xsave_area_elm_descr), M_DEVBUF, M_WAITOK | M_ZERO);
/* x87 state */
xsave_area_desc[0].offset = 0;
xsave_area_desc[0].size = 160;
/* XMM */
xsave_area_desc[1].offset = 160;
xsave_area_desc[1].size = 288 - 160;
for (i = 2; i < max_ext_n; i++) {
cpuid_count(0xd, i, cp);
xsave_area_desc[i].offset = cp[1];
xsave_area_desc[i].size = cp[0];
}
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
fpu_save_area_zone = uma_zcreate("FPU_save_area",
cpu_max_ext_state_size, NULL, NULL, NULL, NULL,
XSAVE_AREA_ALIGN - 1, 0);
start_emulating();
intr_restore(saveintr);
}
SYSINIT(npxinitstate, SI_SUB_DRIVERS, SI_ORDER_ANY, npxinitstate, NULL);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
* Free coprocessor (if we have it).
*/
void
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
npxexit(td)
struct thread *td;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
{
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_enter();
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (curthread == PCPU_GET(fpcurthread)) {
stop_emulating();
fpusave(curpcb->pcb_save);
start_emulating();
PCPU_SET(fpcurthread, NULL);
}
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_exit();
Disabled logging of masked exceptions on exit. Keep the side effect of saving the state (see rev.1.17).
1997-01-29 13:46:28 +00:00
#ifdef NPX_DEBUG
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
if (hw_float) {
Log processes that exit with an masked npx exception that would trap with the current default exception (un)mask. There should be no such processes unless you change the mask. Someday the mask should be changed to the IEEE default of everything masked. The npx state gets saved so that it can be checked and this may have the side effect of fixing a bug that was reported for 1.1.5. (npx exceptions may sometimes leak across exits and clobber another process. I can't see how this can happen.) Get some missing/wrong declarations from headers now that the headers have them.
1994-11-14 14:59:06 +00:00
u_int masked_exceptions;
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
masked_exceptions = GET_FPU_CW(td) & GET_FPU_SW(td) & 0x7f;
Log processes that exit with an masked npx exception that would trap with the current default exception (un)mask. There should be no such processes unless you change the mask. Someday the mask should be changed to the IEEE default of everything masked. The npx state gets saved so that it can be checked and this may have the side effect of fixing a bug that was reported for 1.1.5. (npx exceptions may sometimes leak across exits and clobber another process. I can't see how this can happen.) Get some missing/wrong declarations from headers now that the headers have them.
1994-11-14 14:59:06 +00:00
/*
Disabled logging of masked exceptions on exit. Keep the side effect of saving the state (see rev.1.17).
1997-01-29 13:46:28 +00:00
* Log exceptions that would have trapped with the old
* control word (overflow, divide by 0, and invalid operand).
Log processes that exit with an masked npx exception that would trap with the current default exception (un)mask. There should be no such processes unless you change the mask. Someday the mask should be changed to the IEEE default of everything masked. The npx state gets saved so that it can be checked and this may have the side effect of fixing a bug that was reported for 1.1.5. (npx exceptions may sometimes leak across exits and clobber another process. I can't see how this can happen.) Get some missing/wrong declarations from headers now that the headers have them.
1994-11-14 14:59:06 +00:00
*/
if (masked_exceptions & 0x0d)
log(LOG_ERR,
Whitespace fixes.
2001-09-18 21:05:04 +00:00
"pid %d (%s) exited with masked floating point exceptions 0x%02x\n",
td->td_proc->p_pid, td->td_proc->p_comm,
masked_exceptions);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
Disabled logging of masked exceptions on exit. Keep the side effect of saving the state (see rev.1.17).
1997-01-29 13:46:28 +00:00
#endif
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
int
npxformat()
{
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
if (!hw_float)
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
return (_MC_FPFMT_NODEV);
#ifdef CPU_ENABLE_SSE
if (cpu_fxsr)
return (_MC_FPFMT_XMM);
#endif
return (_MC_FPFMT_387);
}
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
/*
* The following mechanism is used to ensure that the FPE_... value
* that is passed as a trapcode to the signal handler of the user
* process does not have more than one bit set.
*
* Multiple bits may be set if the user process modifies the control
Various formatting fixes on my FPE trapcode commit. Submitted by: BDE
1999-07-26 05:47:31 +00:00
* word while a status word bit is already set. While this is a sign
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
* of bad coding, we have no choise than to narrow them down to one
* bit, since we must not send a trapcode that is not exactly one of
* the FPE_ macros.
*
Various formatting fixes on my FPE trapcode commit. Submitted by: BDE
1999-07-26 05:47:31 +00:00
* The mechanism has a static table with 127 entries. Each combination
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
* of the 7 FPU status word exception bits directly translates to a
* position in this table, where a single FPE_... value is stored.
* This FPE_... value stored there is considered the "most important"
Various formatting fixes on my FPE trapcode commit. Submitted by: BDE
1999-07-26 05:47:31 +00:00
* of the exception bits and will be sent as the signal code. The
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
* precedence of the bits is based upon Intel Document "Numerical
* Applications", Chapter "Special Computational Situations".
*
* The macro to choose one of these values does these steps: 1) Throw
Various formatting fixes on my FPE trapcode commit. Submitted by: BDE
1999-07-26 05:47:31 +00:00
* away status word bits that cannot be masked. 2) Throw away the bits
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
* currently masked in the control word, assuming the user isn't
Various formatting fixes on my FPE trapcode commit. Submitted by: BDE
1999-07-26 05:47:31 +00:00
* interested in them anymore. 3) Reinsert status word bit 7 (stack
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
* fault) if it is set, which cannot be masked but must be presered.
* 4) Use the remaining bits to point into the trapcode table.
*
* The 6 maskable bits in order of their preference, as stated in the
* above referenced Intel manual:
* 1 Invalid operation (FP_X_INV)
* 1a Stack underflow
* 1b Stack overflow
* 1c Operand of unsupported format
* 1d SNaN operand.
* 2 QNaN operand (not an exception, irrelavant here)
* 3 Any other invalid-operation not mentioned above or zero divide
* (FP_X_INV, FP_X_DZ)
* 4 Denormal operand (FP_X_DNML)
* 5 Numeric over/underflow (FP_X_OFL, FP_X_UFL)
Various formatting fixes on my FPE trapcode commit. Submitted by: BDE
1999-07-26 05:47:31 +00:00
* 6 Inexact result (FP_X_IMP)
*/
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
static char fpetable[128] = {
0,
Various formatting fixes on my FPE trapcode commit. Submitted by: BDE
1999-07-26 05:47:31 +00:00
FPE_FLTINV, /* 1 - INV */
FPE_FLTUND, /* 2 - DNML */
FPE_FLTINV, /* 3 - INV | DNML */
FPE_FLTDIV, /* 4 - DZ */
FPE_FLTINV, /* 5 - INV | DZ */
FPE_FLTDIV, /* 6 - DNML | DZ */
FPE_FLTINV, /* 7 - INV | DNML | DZ */
FPE_FLTOVF, /* 8 - OFL */
FPE_FLTINV, /* 9 - INV | OFL */
FPE_FLTUND, /* A - DNML | OFL */
FPE_FLTINV, /* B - INV | DNML | OFL */
FPE_FLTDIV, /* C - DZ | OFL */
FPE_FLTINV, /* D - INV | DZ | OFL */
FPE_FLTDIV, /* E - DNML | DZ | OFL */
FPE_FLTINV, /* F - INV | DNML | DZ | OFL */
FPE_FLTUND, /* 10 - UFL */
FPE_FLTINV, /* 11 - INV | UFL */
FPE_FLTUND, /* 12 - DNML | UFL */
FPE_FLTINV, /* 13 - INV | DNML | UFL */
FPE_FLTDIV, /* 14 - DZ | UFL */
FPE_FLTINV, /* 15 - INV | DZ | UFL */
FPE_FLTDIV, /* 16 - DNML | DZ | UFL */
FPE_FLTINV, /* 17 - INV | DNML | DZ | UFL */
FPE_FLTOVF, /* 18 - OFL | UFL */
FPE_FLTINV, /* 19 - INV | OFL | UFL */
FPE_FLTUND, /* 1A - DNML | OFL | UFL */
FPE_FLTINV, /* 1B - INV | DNML | OFL | UFL */
FPE_FLTDIV, /* 1C - DZ | OFL | UFL */
FPE_FLTINV, /* 1D - INV | DZ | OFL | UFL */
FPE_FLTDIV, /* 1E - DNML | DZ | OFL | UFL */
FPE_FLTINV, /* 1F - INV | DNML | DZ | OFL | UFL */
FPE_FLTRES, /* 20 - IMP */
FPE_FLTINV, /* 21 - INV | IMP */
FPE_FLTUND, /* 22 - DNML | IMP */
FPE_FLTINV, /* 23 - INV | DNML | IMP */
FPE_FLTDIV, /* 24 - DZ | IMP */
FPE_FLTINV, /* 25 - INV | DZ | IMP */
FPE_FLTDIV, /* 26 - DNML | DZ | IMP */
FPE_FLTINV, /* 27 - INV | DNML | DZ | IMP */
FPE_FLTOVF, /* 28 - OFL | IMP */
FPE_FLTINV, /* 29 - INV | OFL | IMP */
FPE_FLTUND, /* 2A - DNML | OFL | IMP */
FPE_FLTINV, /* 2B - INV | DNML | OFL | IMP */
FPE_FLTDIV, /* 2C - DZ | OFL | IMP */
FPE_FLTINV, /* 2D - INV | DZ | OFL | IMP */
FPE_FLTDIV, /* 2E - DNML | DZ | OFL | IMP */
FPE_FLTINV, /* 2F - INV | DNML | DZ | OFL | IMP */
FPE_FLTUND, /* 30 - UFL | IMP */
FPE_FLTINV, /* 31 - INV | UFL | IMP */
FPE_FLTUND, /* 32 - DNML | UFL | IMP */
FPE_FLTINV, /* 33 - INV | DNML | UFL | IMP */
FPE_FLTDIV, /* 34 - DZ | UFL | IMP */
FPE_FLTINV, /* 35 - INV | DZ | UFL | IMP */
FPE_FLTDIV, /* 36 - DNML | DZ | UFL | IMP */
FPE_FLTINV, /* 37 - INV | DNML | DZ | UFL | IMP */
FPE_FLTOVF, /* 38 - OFL | UFL | IMP */
FPE_FLTINV, /* 39 - INV | OFL | UFL | IMP */
FPE_FLTUND, /* 3A - DNML | OFL | UFL | IMP */
FPE_FLTINV, /* 3B - INV | DNML | OFL | UFL | IMP */
FPE_FLTDIV, /* 3C - DZ | OFL | UFL | IMP */
FPE_FLTINV, /* 3D - INV | DZ | OFL | UFL | IMP */
FPE_FLTDIV, /* 3E - DNML | DZ | OFL | UFL | IMP */
FPE_FLTINV, /* 3F - INV | DNML | DZ | OFL | UFL | IMP */
FPE_FLTSUB, /* 40 - STK */
FPE_FLTSUB, /* 41 - INV | STK */
FPE_FLTUND, /* 42 - DNML | STK */
FPE_FLTSUB, /* 43 - INV | DNML | STK */
FPE_FLTDIV, /* 44 - DZ | STK */
FPE_FLTSUB, /* 45 - INV | DZ | STK */
FPE_FLTDIV, /* 46 - DNML | DZ | STK */
FPE_FLTSUB, /* 47 - INV | DNML | DZ | STK */
FPE_FLTOVF, /* 48 - OFL | STK */
FPE_FLTSUB, /* 49 - INV | OFL | STK */
FPE_FLTUND, /* 4A - DNML | OFL | STK */
FPE_FLTSUB, /* 4B - INV | DNML | OFL | STK */
FPE_FLTDIV, /* 4C - DZ | OFL | STK */
FPE_FLTSUB, /* 4D - INV | DZ | OFL | STK */
FPE_FLTDIV, /* 4E - DNML | DZ | OFL | STK */
FPE_FLTSUB, /* 4F - INV | DNML | DZ | OFL | STK */
FPE_FLTUND, /* 50 - UFL | STK */
FPE_FLTSUB, /* 51 - INV | UFL | STK */
FPE_FLTUND, /* 52 - DNML | UFL | STK */
FPE_FLTSUB, /* 53 - INV | DNML | UFL | STK */
FPE_FLTDIV, /* 54 - DZ | UFL | STK */
FPE_FLTSUB, /* 55 - INV | DZ | UFL | STK */
FPE_FLTDIV, /* 56 - DNML | DZ | UFL | STK */
FPE_FLTSUB, /* 57 - INV | DNML | DZ | UFL | STK */
FPE_FLTOVF, /* 58 - OFL | UFL | STK */
FPE_FLTSUB, /* 59 - INV | OFL | UFL | STK */
FPE_FLTUND, /* 5A - DNML | OFL | UFL | STK */
FPE_FLTSUB, /* 5B - INV | DNML | OFL | UFL | STK */
FPE_FLTDIV, /* 5C - DZ | OFL | UFL | STK */
FPE_FLTSUB, /* 5D - INV | DZ | OFL | UFL | STK */
FPE_FLTDIV, /* 5E - DNML | DZ | OFL | UFL | STK */
FPE_FLTSUB, /* 5F - INV | DNML | DZ | OFL | UFL | STK */
FPE_FLTRES, /* 60 - IMP | STK */
FPE_FLTSUB, /* 61 - INV | IMP | STK */
FPE_FLTUND, /* 62 - DNML | IMP | STK */
FPE_FLTSUB, /* 63 - INV | DNML | IMP | STK */
FPE_FLTDIV, /* 64 - DZ | IMP | STK */
FPE_FLTSUB, /* 65 - INV | DZ | IMP | STK */
FPE_FLTDIV, /* 66 - DNML | DZ | IMP | STK */
FPE_FLTSUB, /* 67 - INV | DNML | DZ | IMP | STK */
FPE_FLTOVF, /* 68 - OFL | IMP | STK */
FPE_FLTSUB, /* 69 - INV | OFL | IMP | STK */
FPE_FLTUND, /* 6A - DNML | OFL | IMP | STK */
FPE_FLTSUB, /* 6B - INV | DNML | OFL | IMP | STK */
FPE_FLTDIV, /* 6C - DZ | OFL | IMP | STK */
FPE_FLTSUB, /* 6D - INV | DZ | OFL | IMP | STK */
FPE_FLTDIV, /* 6E - DNML | DZ | OFL | IMP | STK */
FPE_FLTSUB, /* 6F - INV | DNML | DZ | OFL | IMP | STK */
FPE_FLTUND, /* 70 - UFL | IMP | STK */
FPE_FLTSUB, /* 71 - INV | UFL | IMP | STK */
FPE_FLTUND, /* 72 - DNML | UFL | IMP | STK */
FPE_FLTSUB, /* 73 - INV | DNML | UFL | IMP | STK */
FPE_FLTDIV, /* 74 - DZ | UFL | IMP | STK */
FPE_FLTSUB, /* 75 - INV | DZ | UFL | IMP | STK */
FPE_FLTDIV, /* 76 - DNML | DZ | UFL | IMP | STK */
FPE_FLTSUB, /* 77 - INV | DNML | DZ | UFL | IMP | STK */
FPE_FLTOVF, /* 78 - OFL | UFL | IMP | STK */
FPE_FLTSUB, /* 79 - INV | OFL | UFL | IMP | STK */
FPE_FLTUND, /* 7A - DNML | OFL | UFL | IMP | STK */
FPE_FLTSUB, /* 7B - INV | DNML | OFL | UFL | IMP | STK */
FPE_FLTDIV, /* 7C - DZ | OFL | UFL | IMP | STK */
FPE_FLTSUB, /* 7D - INV | DZ | OFL | UFL | IMP | STK */
FPE_FLTDIV, /* 7E - DNML | DZ | OFL | UFL | IMP | STK */
FPE_FLTSUB, /* 7F - INV | DNML | DZ | OFL | UFL | IMP | STK */
On FPU exceptions, pass a useful error code (one of the FPE_... macros) to the signal handler, for old-style BSD signal handlers as the second (int) argument, for SA_SIGINFO signal handlers as siginfo_t->si_code. This is source-compatible with Solaris, except that we have no <siginfo.h> (which isn't even mentioned in POSIX 1003.1b). An rather complete example program is at http://www3.cons.org/cracauer/freebsd-signal.c This will be added to the regression tests in src/. This commit also adds code to disable the (hardware) FPU from userconfig, so that you can use a software FP emulator on a machine that has hardware floating point. See LINT.
1999-07-25 13:16:09 +00:00
};
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
MFamd64 r238668: Stop clearing x87 exceptions in the #MF handler. Requested by: bde MFC after: 1 week
2012-07-21 21:49:05 +00:00
* Read the FP status and control words, then generate si_code value
* for SIGFPE. The error code chosen will be one of the
* FPE_... macros. It will be sent as the second argument to old
* BSD-style signal handlers and as "siginfo_t->si_code" (second
* argument) to SA_SIGINFO signal handlers.
Use sufficient parentheses in macros. Remove bogus input operands for fnsave(), fnstcw() and fnstsw(). Change all fwait's to fnop's. This might help avoid hardware bugs. Wait after fninit with an fnop. This should be safer now. Fix some spelling and formatting errors. Use natural sizes for control and status words (u_short, promotes to int). Don't clobber the SWI_CLOCK_MASK bits in npx0_imask when using IRQ13. Set the devconf state correctly (always busy, if configured). Improve code for npx_registerdev() a little (gcc can't keep id->id_unit in a register for some reason). Don't register a nonexistent npx device. Print a useful message in npxattach() again (delete references to errors and not the whole message). Don't print "387 emulator" if there is no emulator in the kernel. Use %p for pointers in error messages. Don't clobber the FPU state when there is an FPU exception. Just clear the exception flags (after saving the flags as before). This allows debuggers and SIGFPE handlers to look at the full exception state. SIGFPE handlers should normally return via longjmp(), which restores a good FPU state (as before). Returning from a SIGFPE handler may leave the FPU in the wrong state (as before). Clear the busy latch _after_ clearing the exception flags so that there is less chance of getting a bogus h/w interrupt for a control operation. Clear the saved exception status word when the next FPU instruction is excuted so that it doesn't stick around until the next exception. Clear the busy latch after fnsave() in npxsave() in case it was set when npxsave() was called.
1995-01-03 04:00:06 +00:00
*
MFamd64 r238668: Stop clearing x87 exceptions in the #MF handler. Requested by: bde MFC after: 1 week
2012-07-21 21:49:05 +00:00
* Some time ago, we cleared the x87 exceptions with FNCLEX there.
* Clearing exceptions was necessary mainly to avoid IRQ13 bugs. The
* usermode code which understands the FPU hardware enough to enable
* the exceptions, can also handle clearing the exception state in the
* handler. The only consequence of not clearing the exception is the
* rethrow of the SIGFPE on return from the signal handler and
* reexecution of the corresponding instruction.
Use sufficient parentheses in macros. Remove bogus input operands for fnsave(), fnstcw() and fnstsw(). Change all fwait's to fnop's. This might help avoid hardware bugs. Wait after fninit with an fnop. This should be safer now. Fix some spelling and formatting errors. Use natural sizes for control and status words (u_short, promotes to int). Don't clobber the SWI_CLOCK_MASK bits in npx0_imask when using IRQ13. Set the devconf state correctly (always busy, if configured). Improve code for npx_registerdev() a little (gcc can't keep id->id_unit in a register for some reason). Don't register a nonexistent npx device. Print a useful message in npxattach() again (delete references to errors and not the whole message). Don't print "387 emulator" if there is no emulator in the kernel. Use %p for pointers in error messages. Don't clobber the FPU state when there is an FPU exception. Just clear the exception flags (after saving the flags as before). This allows debuggers and SIGFPE handlers to look at the full exception state. SIGFPE handlers should normally return via longjmp(), which restores a good FPU state (as before). Returning from a SIGFPE handler may leave the FPU in the wrong state (as before). Clear the busy latch _after_ clearing the exception flags so that there is less chance of getting a bogus h/w interrupt for a control operation. Clear the saved exception status word when the next FPU instruction is excuted so that it doesn't stick around until the next exception. Clear the busy latch after fnsave() in npxsave() in case it was set when npxsave() was called.
1995-01-03 04:00:06 +00:00
*
MFamd64 r238668: Stop clearing x87 exceptions in the #MF handler. Requested by: bde MFC after: 1 week
2012-07-21 21:49:05 +00:00
* For XMM traps, the exceptions were never cleared.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
int
MFCamd64 r238598: Provide siginfo.si_code for floating point errors when error occurs using the SSE math processor. MFC after: 3 weeks
2012-07-21 21:52:48 +00:00
npxtrap_x87(void)
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
{
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
u_short control, status;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
if (!hw_float) {
MFCamd64 r238598: Provide siginfo.si_code for floating point errors when error occurs using the SSE math processor. MFC after: 3 weeks
2012-07-21 21:52:48 +00:00
printf(
"npxtrap_x87: fpcurthread = %p, curthread = %p, hw_float = %d\n",
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
PCPU_GET(fpcurthread), curthread, hw_float);
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
panic("npxtrap from nowhere");
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_enter();
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
* Interrupt handling (for another interrupt) may have pushed the
* state to memory. Fetch the relevant parts of the state from
* wherever they are.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Overhaul the per-CPU support a bit: - The MI portions of struct globaldata have been consolidated into a MI struct pcpu. The MD per-CPU data are specified via a macro defined in machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the interface would be cleaner (PCPU_GET(my_md_field) vs. PCPU_GET(md.md_my_md_field)). - All references to globaldata are changed to pcpu instead. In a UP kernel, this data was stored as global variables which is where the original name came from. In an SMP world this data is per-CPU and ideally private to each CPU outside of the context of debuggers. This also included combining machine/globaldata.h and machine/globals.h into machine/pcpu.h. - The pointer to the thread using the FPU on i386 was renamed from npxthread to fpcurthread to be identical with other architectures. - Make the show pcpu ddb command MI with a MD callout to display MD fields. - The globaldata_register() function was renamed to pcpu_init() and now init's MI fields of a struct pcpu in addition to registering it with the internal array and list. - A pcpu_destroy() function was added to remove a struct pcpu from the internal array and list. Tested on: alpha, i386 Reviewed by: peter, jake
2001-12-11 23:33:44 +00:00
if (PCPU_GET(fpcurthread) != curthread) {
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
control = GET_FPU_CW(curthread);
status = GET_FPU_SW(curthread);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
} else {
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
fnstcw(&control);
fnstsw(&status);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_exit();
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
return (fpetable[status & ((~control & 0x3f) | 0x40)]);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
MFCamd64 r238598: Provide siginfo.si_code for floating point errors when error occurs using the SSE math processor. MFC after: 3 weeks
2012-07-21 21:52:48 +00:00
#ifdef CPU_ENABLE_SSE
int
npxtrap_sse(void)
{
u_int mxcsr;
if (!hw_float) {
printf(
"npxtrap_sse: fpcurthread = %p, curthread = %p, hw_float = %d\n",
PCPU_GET(fpcurthread), curthread, hw_float);
panic("npxtrap from nowhere");
}
critical_enter();
if (PCPU_GET(fpcurthread) != curthread)
mxcsr = curthread->td_pcb->pcb_save->sv_xmm.sv_env.en_mxcsr;
else
stmxcsr(&mxcsr);
critical_exit();
return (fpetable[(mxcsr & (~mxcsr >> 7)) & 0x3f]);
}
#endif
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
* Implement device not available (DNA) exception
*
Overhaul the per-CPU support a bit: - The MI portions of struct globaldata have been consolidated into a MI struct pcpu. The MD per-CPU data are specified via a macro defined in machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the interface would be cleaner (PCPU_GET(my_md_field) vs. PCPU_GET(md.md_my_md_field)). - All references to globaldata are changed to pcpu instead. In a UP kernel, this data was stored as global variables which is where the original name came from. In an SMP world this data is per-CPU and ideally private to each CPU outside of the context of debuggers. This also included combining machine/globaldata.h and machine/globals.h into machine/pcpu.h. - The pointer to the thread using the FPU on i386 was renamed from npxthread to fpcurthread to be identical with other architectures. - Make the show pcpu ddb command MI with a MD callout to display MD fields. - The globaldata_register() function was renamed to pcpu_init() and now init's MI fields of a struct pcpu in addition to registering it with the internal array and list. - A pcpu_destroy() function was added to remove a struct pcpu from the internal array and list. Tested on: alpha, i386 Reviewed by: peter, jake
2001-12-11 23:33:44 +00:00
* It would be better to switch FP context here (if curthread != fpcurthread)
Use sufficient parentheses in macros. Remove bogus input operands for fnsave(), fnstcw() and fnstsw(). Change all fwait's to fnop's. This might help avoid hardware bugs. Wait after fninit with an fnop. This should be safer now. Fix some spelling and formatting errors. Use natural sizes for control and status words (u_short, promotes to int). Don't clobber the SWI_CLOCK_MASK bits in npx0_imask when using IRQ13. Set the devconf state correctly (always busy, if configured). Improve code for npx_registerdev() a little (gcc can't keep id->id_unit in a register for some reason). Don't register a nonexistent npx device. Print a useful message in npxattach() again (delete references to errors and not the whole message). Don't print "387 emulator" if there is no emulator in the kernel. Use %p for pointers in error messages. Don't clobber the FPU state when there is an FPU exception. Just clear the exception flags (after saving the flags as before). This allows debuggers and SIGFPE handlers to look at the full exception state. SIGFPE handlers should normally return via longjmp(), which restores a good FPU state (as before). Returning from a SIGFPE handler may leave the FPU in the wrong state (as before). Clear the busy latch _after_ clearing the exception flags so that there is less chance of getting a bogus h/w interrupt for a control operation. Clear the saved exception status word when the next FPU instruction is excuted so that it doesn't stick around until the next exception. Clear the busy latch after fnsave() in npxsave() in case it was set when npxsave() was called.
1995-01-03 04:00:06 +00:00
* and not necessarily for every context switch, but it is too hard to
* access foreign pcb's.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
static int err_count = 0;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
int
Some cleanups to the i386 FPU support: - Remove the control word parameter to npxinit(). It was always set to __INITIAL_NPXCW__. - Remove npx_cleanstate_ready as the cleanstate is always initalized when it is used. - Improve the handling of the case when the FPU isn't present. Now the npx0 device no longer succeeds in its probe so all of npx_attach() is skipped. Also, we allow this case with SMP (though that shouldn't actually occur as all i386 systems that support SMP have FPUs) now. SMP was only an issue back when we had an FPU emulator which was not per-CPU. - MFamd64: Clear some of the state in npx_cleanstate rather than leaving it as garbage. - MFamd64: When a user thread first uses the FPU, use npx_cleanstate for the initial FPU state. Reviewed by: bde
2009-03-05 18:32:43 +00:00
npxdna(void)
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
{
Implement preemptive scheduling of hardware interrupt threads. - If possible, context switch to the thread directly in sched_ithd(), rather than triggering a delayed ast reschedule. - Disable interrupts while restoring fpu state in the trap handler, in order to ensure that we are not preempted in the middle, which could cause migration to another cpu. Reviewed by: peter Tested by: peter (alpha)
2001-02-01 03:34:20 +00:00
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
if (!hw_float)
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
return (0);
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_enter();
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
if (PCPU_GET(fpcurthread) == curthread) {
printf("npxdna: fpcurthread == curthread %d times\n",
++err_count);
stop_emulating();
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_exit();
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
return (1);
}
Overhaul the per-CPU support a bit: - The MI portions of struct globaldata have been consolidated into a MI struct pcpu. The MD per-CPU data are specified via a macro defined in machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the interface would be cleaner (PCPU_GET(my_md_field) vs. PCPU_GET(md.md_my_md_field)). - All references to globaldata are changed to pcpu instead. In a UP kernel, this data was stored as global variables which is where the original name came from. In an SMP world this data is per-CPU and ideally private to each CPU outside of the context of debuggers. This also included combining machine/globaldata.h and machine/globals.h into machine/pcpu.h. - The pointer to the thread using the FPU on i386 was renamed from npxthread to fpcurthread to be identical with other architectures. - Make the show pcpu ddb command MI with a MD callout to display MD fields. - The globaldata_register() function was renamed to pcpu_init() and now init's MI fields of a struct pcpu in addition to registering it with the internal array and list. - A pcpu_destroy() function was added to remove a struct pcpu from the internal array and list. Tested on: alpha, i386 Reviewed by: peter, jake
2001-12-11 23:33:44 +00:00
if (PCPU_GET(fpcurthread) != NULL) {
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
printf("npxdna: fpcurthread = %p (%d), curthread = %p (%d)\n",
PCPU_GET(fpcurthread),
PCPU_GET(fpcurthread)->td_proc->p_pid,
curthread, curthread->td_proc->p_pid);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
panic("npxdna");
}
stop_emulating();
/*
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
* Record new context early in case frstor causes a trap.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Overhaul the per-CPU support a bit: - The MI portions of struct globaldata have been consolidated into a MI struct pcpu. The MD per-CPU data are specified via a macro defined in machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the interface would be cleaner (PCPU_GET(my_md_field) vs. PCPU_GET(md.md_my_md_field)). - All references to globaldata are changed to pcpu instead. In a UP kernel, this data was stored as global variables which is where the original name came from. In an SMP world this data is per-CPU and ideally private to each CPU outside of the context of debuggers. This also included combining machine/globaldata.h and machine/globals.h into machine/pcpu.h. - The pointer to the thread using the FPU on i386 was renamed from npxthread to fpcurthread to be identical with other architectures. - Make the show pcpu ddb command MI with a MD callout to display MD fields. - The globaldata_register() function was renamed to pcpu_init() and now init's MI fields of a struct pcpu in addition to registering it with the internal array and list. - A pcpu_destroy() function was added to remove a struct pcpu from the internal array and list. Tested on: alpha, i386 Reviewed by: peter, jake
2001-12-11 23:33:44 +00:00
PCPU_SET(fpcurthread, curthread);
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
Fix a few nits in the earlier changes to prevent local information leakage in AMD FPUs: - Do not clear the affected state in the case that the FPU registers for the thread that already owns the FPU are changed via fpu_setregs(). The only local information the thread would see is its own state in that case. - Fix a type mismatch for the dummy variable used in a "fld". It accepts a float, not a double. Reviewed by: bde Approved by: so (cperciva) MFC after: 1 month
2009-03-25 22:08:30 +00:00
#ifdef CPU_ENABLE_SSE
if (cpu_fxsr)
fpu_clean_state();
#endif
MFamd64 r238623: Introduce curpcb magic variable. Requested and reviewed by: bde MFC after: 3 weeks
2012-07-26 09:11:37 +00:00
if ((curpcb->pcb_flags & PCB_NPXINITDONE) == 0) {
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
/*
* This is the first time this thread has used the FPU or
* the PCB doesn't contain a clean FPU state. Explicitly
Rename (fpu|npx)_cleanstate to (fpu|npx)_initialstate to better reflect their purpose. Inspired by: bde MFC after: 1 month
2009-03-25 14:17:08 +00:00
* load an initial state.
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
*
* We prefer to restore the state from the actual save
* area in PCB instead of directly loading from
* npx_initialstate, to ignite the XSAVEOPT
* tracking engine.
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
*/
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
bcopy(npx_initialstate, curpcb->pcb_save, cpu_max_ext_state_size);
fpurstor(curpcb->pcb_save);
MFamd64 r238623: Introduce curpcb magic variable. Requested and reviewed by: bde MFC after: 3 weeks
2012-07-26 09:11:37 +00:00
if (curpcb->pcb_initial_npxcw != __INITIAL_NPXCW__)
fldcw(curpcb->pcb_initial_npxcw);
curpcb->pcb_flags |= PCB_NPXINITDONE;
if (PCB_USER_FPU(curpcb))
curpcb->pcb_flags |= PCB_NPXUSERINITDONE;
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
} else {
MFamd64 r238623: Introduce curpcb magic variable. Requested and reviewed by: bde MFC after: 3 weeks
2012-07-26 09:11:37 +00:00
fpurstor(curpcb->pcb_save);
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
}
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_exit();
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
return (1);
}
/*
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
* Wrapper for fpusave() called from context switch routines.
Throw away the complications in npxsave() and their infrastructure. npxsave() went to great lengths to excecute fnsave with interrupts enabled in case executing it froze the CPU. This case can't happen, at least for Intel CPU/NPX's. Spurious IRQ13's don't imply spurious freezes. Anyway, the complications were usually no-ops because IRQ13 is not used on i486's and newer CPUs, and because SMPng broke them in rev.1.84. Forcible enabling of interrupts was changed to write_eflags(old_eflags), but since SMPng usually calls npxsave() from cpu_switch() with interrupts disabled, write_eflags() usually just kept interrupts disabled.
2001-05-20 20:04:40 +00:00
*
* npxsave() must be called with interrupts disabled, so that it clears
Overhaul the per-CPU support a bit: - The MI portions of struct globaldata have been consolidated into a MI struct pcpu. The MD per-CPU data are specified via a macro defined in machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the interface would be cleaner (PCPU_GET(my_md_field) vs. PCPU_GET(md.md_my_md_field)). - All references to globaldata are changed to pcpu instead. In a UP kernel, this data was stored as global variables which is where the original name came from. In an SMP world this data is per-CPU and ideally private to each CPU outside of the context of debuggers. This also included combining machine/globaldata.h and machine/globals.h into machine/pcpu.h. - The pointer to the thread using the FPU on i386 was renamed from npxthread to fpcurthread to be identical with other architectures. - Make the show pcpu ddb command MI with a MD callout to display MD fields. - The globaldata_register() function was renamed to pcpu_init() and now init's MI fields of a struct pcpu in addition to registering it with the internal array and list. - A pcpu_destroy() function was added to remove a struct pcpu from the internal array and list. Tested on: alpha, i386 Reviewed by: peter, jake
2001-12-11 23:33:44 +00:00
* fpcurthread atomically with saving the state. We require callers to do the
Throw away the complications in npxsave() and their infrastructure. npxsave() went to great lengths to excecute fnsave with interrupts enabled in case executing it froze the CPU. This case can't happen, at least for Intel CPU/NPX's. Spurious IRQ13's don't imply spurious freezes. Anyway, the complications were usually no-ops because IRQ13 is not used on i486's and newer CPUs, and because SMPng broke them in rev.1.84. Forcible enabling of interrupts was changed to write_eflags(old_eflags), but since SMPng usually calls npxsave() from cpu_switch() with interrupts disabled, write_eflags() usually just kept interrupts disabled.
2001-05-20 20:04:40 +00:00
* disabling, since most callers need to disable interrupts anyway to call
Overhaul the per-CPU support a bit: - The MI portions of struct globaldata have been consolidated into a MI struct pcpu. The MD per-CPU data are specified via a macro defined in machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the interface would be cleaner (PCPU_GET(my_md_field) vs. PCPU_GET(md.md_my_md_field)). - All references to globaldata are changed to pcpu instead. In a UP kernel, this data was stored as global variables which is where the original name came from. In an SMP world this data is per-CPU and ideally private to each CPU outside of the context of debuggers. This also included combining machine/globaldata.h and machine/globals.h into machine/pcpu.h. - The pointer to the thread using the FPU on i386 was renamed from npxthread to fpcurthread to be identical with other architectures. - Make the show pcpu ddb command MI with a MD callout to display MD fields. - The globaldata_register() function was renamed to pcpu_init() and now init's MI fields of a struct pcpu in addition to registering it with the internal array and list. - A pcpu_destroy() function was added to remove a struct pcpu from the internal array and list. Tested on: alpha, i386 Reviewed by: peter, jake
2001-12-11 23:33:44 +00:00
* npxsave() atomically with checking fpcurthread.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
void
npxsave(addr)
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
union savefpu *addr;
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
{
Made the SMP case ignore the possibility of an INT13 interface. This eliminates all the APIC code, and thus several routines that would otherwise need to be made MP-safe. Reviewed by: Bruce Evans <bde@zeta.org.au>
1997-07-21 07:57:50 +00:00
stop_emulating();
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (use_xsaveopt)
xsaveopt((char *)addr, xsave_mask);
else
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
fpusave(addr);
Made the SMP case ignore the possibility of an INT13 interface. This eliminates all the APIC code, and thus several routines that would otherwise need to be made MP-safe. Reviewed by: Bruce Evans <bde@zeta.org.au>
1997-07-21 07:57:50 +00:00
start_emulating();
Overhaul the per-CPU support a bit: - The MI portions of struct globaldata have been consolidated into a MI struct pcpu. The MD per-CPU data are specified via a macro defined in machine/pcpu.h. A macro was chosen over a struct mdpcpu so that the interface would be cleaner (PCPU_GET(my_md_field) vs. PCPU_GET(md.md_my_md_field)). - All references to globaldata are changed to pcpu instead. In a UP kernel, this data was stored as global variables which is where the original name came from. In an SMP world this data is per-CPU and ideally private to each CPU outside of the context of debuggers. This also included combining machine/globaldata.h and machine/globals.h into machine/pcpu.h. - The pointer to the thread using the FPU on i386 was renamed from npxthread to fpcurthread to be identical with other architectures. - Make the show pcpu ddb command MI with a MD callout to display MD fields. - The globaldata_register() function was renamed to pcpu_init() and now init's MI fields of a struct pcpu in addition to registering it with the internal array and list. - A pcpu_destroy() function was added to remove a struct pcpu from the internal array and list. Tested on: alpha, i386 Reviewed by: peter, jake
2001-12-11 23:33:44 +00:00
PCPU_SET(fpcurthread, NULL);
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
}
Save and restore FPU state across suspend and resume. In earlier revisions of this patch, resumectx() called npxresume() directly, but that doesn't work because resumectx() runs with a non-standard %cs selector. Instead, all of the FPU suspend/resume handling is done in C. MFC after: 1 week
2014-08-30 17:48:38 +00:00
/*
* Unconditionally save the current co-processor state across suspend and
* resume.
*/
void
npxsuspend(union savefpu *addr)
{
register_t cr0;
if (!hw_float)
return;
if (PCPU_GET(fpcurthread) == NULL) {
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
bcopy(npx_initialstate, addr, cpu_max_ext_state_size);
Save and restore FPU state across suspend and resume. In earlier revisions of this patch, resumectx() called npxresume() directly, but that doesn't work because resumectx() runs with a non-standard %cs selector. Instead, all of the FPU suspend/resume handling is done in C. MFC after: 1 week
2014-08-30 17:48:38 +00:00
return;
}
cr0 = rcr0();
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
stop_emulating();
Save and restore FPU state across suspend and resume. In earlier revisions of this patch, resumectx() called npxresume() directly, but that doesn't work because resumectx() runs with a non-standard %cs selector. Instead, all of the FPU suspend/resume handling is done in C. MFC after: 1 week
2014-08-30 17:48:38 +00:00
fpusave(addr);
load_cr0(cr0);
}
void
npxresume(union savefpu *addr)
{
register_t cr0;
if (!hw_float)
return;
cr0 = rcr0();
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
npxinit(false);
Save and restore FPU state across suspend and resume. In earlier revisions of this patch, resumectx() called npxresume() directly, but that doesn't work because resumectx() runs with a non-standard %cs selector. Instead, all of the FPU suspend/resume handling is done in C. MFC after: 1 week
2014-08-30 17:48:38 +00:00
stop_emulating();
fpurstor(addr);
load_cr0(cr0);
}
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
void
npxdrop()
{
struct thread *td;
Fixed a panic caused by over-optimizing npxdrop() in the non-FXSR case. frstor can trap despite it being a control instruction, since it bogusly checks for pending exceptions in the state that it is overwriting. This used to be a non-problem because frstor was always paired with a previous fnsave, and fnsave does an implicit fninit so any pending exceptions only remain live in the saved state. Now frstor is sometimes paired with npxdrop() and we must do a little more than just forget that the npx was used in npxdrop() to avoid a trap later. This is a non-problem in the FXSR case because fxrstor doesn't do the bogus check. FXSR is part of SSE, and npxdrop() is only in FreeBSD-5.x, so this bug only affected old machines running FreeBSD-5.x. PR: 68058
2004-06-18 02:10:55 +00:00
/*
* Discard pending exceptions in the !cpu_fxsr case so that unmasked
* ones don't cause a panic on the next frstor.
*/
#ifdef CPU_ENABLE_SSE
if (!cpu_fxsr)
#endif
fnclex();
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
td = PCPU_GET(fpcurthread);
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
KASSERT(td == curthread, ("fpudrop: fpcurthread != curthread"));
CRITICAL_ASSERT(td);
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
PCPU_SET(fpcurthread, NULL);
td->td_pcb->pcb_flags &= ~PCB_NPXINITDONE;
start_emulating();
}
/*
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
* Get the user state of the FPU into pcb->pcb_user_save without
* dropping ownership (if possible). It returns the FPU ownership
* status.
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
*/
int
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
npxgetregs(struct thread *td)
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
{
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
struct pcb *pcb;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
uint64_t *xstate_bv, bit;
char *sa;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
int max_ext_n, i;
#endif
int owned;
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
if (!hw_float)
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
return (_MC_FPOWNED_NONE);
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb = td->td_pcb;
if ((pcb->pcb_flags & PCB_NPXINITDONE) == 0) {
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
bcopy(npx_initialstate, get_pcb_user_save_pcb(pcb),
cpu_max_ext_state_size);
SET_FPU_CW(get_pcb_user_save_pcb(pcb), pcb->pcb_initial_npxcw);
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
npxuserinited(td);
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
return (_MC_FPOWNED_PCB);
}
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_enter();
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
if (td == PCPU_GET(fpcurthread)) {
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
fpusave(get_pcb_user_save_pcb(pcb));
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
#ifdef CPU_ENABLE_SSE
if (!cpu_fxsr)
#endif
/*
* fnsave initializes the FPU and destroys whatever
* context it contains. Make sure the FPU owner
* starts with a clean state next time.
*/
npxdrop();
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
owned = _MC_FPOWNED_FPU;
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
} else {
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
owned = _MC_FPOWNED_PCB;
}
critical_exit();
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (use_xsave) {
/*
* Handle partially saved state.
*/
sa = (char *)get_pcb_user_save_pcb(pcb);
xstate_bv = (uint64_t *)(sa + sizeof(union savefpu) +
offsetof(struct xstate_hdr, xstate_bv));
if (xsave_mask >> 32 != 0)
max_ext_n = fls(xsave_mask >> 32) + 32;
else
max_ext_n = fls(xsave_mask);
for (i = 0; i < max_ext_n; i++) {
bit = 1ULL << i;
if ((xsave_mask & bit) == 0 || (*xstate_bv & bit) != 0)
continue;
bcopy((char *)npx_initialstate +
xsave_area_desc[i].offset,
sa + xsave_area_desc[i].offset,
xsave_area_desc[i].size);
*xstate_bv |= bit;
}
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
return (owned);
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
}
void
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
npxuserinited(struct thread *td)
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
{
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
struct pcb *pcb;
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb = td->td_pcb;
if (PCB_USER_FPU(pcb))
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
pcb->pcb_flags |= PCB_NPXINITDONE;
pcb->pcb_flags |= PCB_NPXUSERINITDONE;
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
int
npxsetxstate(struct thread *td, char *xfpustate, size_t xfpustate_size)
{
struct xstate_hdr *hdr, *ehdr;
size_t len, max_len;
uint64_t bv;
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
/* XXXKIB should we clear all extended state in xstate_bv instead ? */
if (xfpustate == NULL)
return (0);
if (!use_xsave)
return (EOPNOTSUPP);
len = xfpustate_size;
if (len < sizeof(struct xstate_hdr))
return (EINVAL);
max_len = cpu_max_ext_state_size - sizeof(union savefpu);
if (len > max_len)
return (EINVAL);
ehdr = (struct xstate_hdr *)xfpustate;
bv = ehdr->xstate_bv;
/*
* Avoid #gp.
*/
if (bv & ~xsave_mask)
return (EINVAL);
hdr = (struct xstate_hdr *)(get_pcb_user_save_td(td) + 1);
hdr->xstate_bv = bv;
bcopy(xfpustate + sizeof(struct xstate_hdr),
(char *)(hdr + 1), len - sizeof(struct xstate_hdr));
return (0);
}
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
int
npxsetregs(struct thread *td, union savefpu *addr, char *xfpustate,
size_t xfpustate_size)
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
{
struct pcb *pcb;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
int error;
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
Remove the support for int13 FPU exception reporting on i386. It is believed that all 486-class CPUs FreeBSD is capable to run on, either have no FPU and cannot use external coprocessor, or have FPU on the package and can use #MF. Reviewed by: bde Tested by: pho (previous version)
2010-06-23 11:12:58 +00:00
if (!hw_float)
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
return (ENXIO);
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb = td->td_pcb;
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_enter();
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
if (td == PCPU_GET(fpcurthread) && PCB_USER_FPU(pcb)) {
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
error = npxsetxstate(td, xfpustate, xfpustate_size);
if (error != 0) {
critical_exit();
return (error);
}
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
if (!cpu_fxsr)
#endif
fnclex(); /* As in npxdrop(). */
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr));
fpurstor(get_pcb_user_save_td(td));
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_exit();
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb->pcb_flags |= PCB_NPXUSERINITDONE | PCB_NPXINITDONE;
} else {
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_exit();
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
error = npxsetxstate(td, xfpustate, xfpustate_size);
if (error != 0)
return (error);
Move all of the XSAVE support under #ifdef I686_CPU and fix the build without I686_CPU.
2014-11-03 19:16:25 +00:00
#endif
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr));
Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs() functions, they are unused. Remove 'user' from npxgetuserregs() etc. names. For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU context storage. This eliminates the need for ugly copying with overwrite of the newly added and reserved fields in ucontext on i386 to satisfy alignment requirements for fpusave() and fpurstor(). pc98 version was copied from i386. Suggested and reviewed by: bde Tested by: pho (i386 and amd64) MFC after: 1 week
2010-11-26 14:50:42 +00:00
npxuserinited(td);
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
}
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
return (0);
Add kernel support needed for the KSE-aware libpthread: - Maintain fpu state across signals. - Save and restore FPU state properly in ucontext_t's. Reviewed by: bde, deischen, julian Approved by: -arch
2002-09-16 19:25:59 +00:00
}
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
static void
Use PCPU_GET(cpuid) instead of curproc->p_oncpu. Reviewed by: peter
2001-07-16 06:00:23 +00:00
fpusave(addr)
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
union savefpu *addr;
{
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (use_xsave)
xsave((char *)addr, xsave_mask);
else if (cpu_fxsr)
The per-cpu temporary buffers are not needed since the pcb_save areas have the proper alignment. Change dummy variable in npxinit from stack to bss to ensure proper alignment. Reviewed by: bde
2001-07-17 13:06:47 +00:00
fxsave(addr);
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
else
#endif
fnsave(addr);
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
}
Provide npx_get_fsave(9) and npx_set_fsave(9) functions to obtain and restore the FPU state from the format of machine FSAVE area. The intended use is for ABI emulators to provide FSAVE-formatted FPU state to usermode requiring it, while kernel could use FXSAVE due to XMM/XSAVE. The core functionality to convert from/to FXSAVE format is shared with the fill_fpregs_xmm() and set_fpregs_xmm(). Move the later functions to npx.c and rename them to npx_fill_fpregs_xmm() and npx_set_fpregs_xmm(). They differ from nptx_get/set_fsave(9) since our mcontext contains padding to be zeroed or ignored. fill_fpregs() and set_fpregs() could be converted to use the new interface, but there are small differences to handle. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-06-29 12:06:36 +00:00
#ifdef CPU_ENABLE_SSE
static void
npx_fill_fpregs_xmm1(struct savexmm *sv_xmm, struct save87 *sv_87)
{
struct env87 *penv_87;
struct envxmm *penv_xmm;
int i;
penv_87 = &sv_87->sv_env;
penv_xmm = &sv_xmm->sv_env;
/* FPU control/status */
penv_87->en_cw = penv_xmm->en_cw;
penv_87->en_sw = penv_xmm->en_sw;
penv_87->en_fip = penv_xmm->en_fip;
penv_87->en_fcs = penv_xmm->en_fcs;
penv_87->en_opcode = penv_xmm->en_opcode;
penv_87->en_foo = penv_xmm->en_foo;
penv_87->en_fos = penv_xmm->en_fos;
Convert between abridged (from FXSAVE) and unabridged (from FSAVE) versions of the x87 tags. The conversion is naive, used abridged tag is converted to valid unabridged, without additional checks for zero and special values. Noted by: bde Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-07-10 09:20:13 +00:00
/* FPU registers and tags */
penv_87->en_tw = 0xffff;
for (i = 0; i < 8; ++i) {
Provide npx_get_fsave(9) and npx_set_fsave(9) functions to obtain and restore the FPU state from the format of machine FSAVE area. The intended use is for ABI emulators to provide FSAVE-formatted FPU state to usermode requiring it, while kernel could use FXSAVE due to XMM/XSAVE. The core functionality to convert from/to FXSAVE format is shared with the fill_fpregs_xmm() and set_fpregs_xmm(). Move the later functions to npx.c and rename them to npx_fill_fpregs_xmm() and npx_set_fpregs_xmm(). They differ from nptx_get/set_fsave(9) since our mcontext contains padding to be zeroed or ignored. fill_fpregs() and set_fpregs() could be converted to use the new interface, but there are small differences to handle. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-06-29 12:06:36 +00:00
sv_87->sv_ac[i] = sv_xmm->sv_fp[i].fp_acc;
Convert between abridged (from FXSAVE) and unabridged (from FSAVE) versions of the x87 tags. The conversion is naive, used abridged tag is converted to valid unabridged, without additional checks for zero and special values. Noted by: bde Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-07-10 09:20:13 +00:00
if ((penv_xmm->en_tw & (1 << i)) != 0)
/* zero and special are set as valid */
penv_87->en_tw &= ~(3 << i);
}
Provide npx_get_fsave(9) and npx_set_fsave(9) functions to obtain and restore the FPU state from the format of machine FSAVE area. The intended use is for ABI emulators to provide FSAVE-formatted FPU state to usermode requiring it, while kernel could use FXSAVE due to XMM/XSAVE. The core functionality to convert from/to FXSAVE format is shared with the fill_fpregs_xmm() and set_fpregs_xmm(). Move the later functions to npx.c and rename them to npx_fill_fpregs_xmm() and npx_set_fpregs_xmm(). They differ from nptx_get/set_fsave(9) since our mcontext contains padding to be zeroed or ignored. fill_fpregs() and set_fpregs() could be converted to use the new interface, but there are small differences to handle. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-06-29 12:06:36 +00:00
}
void
npx_fill_fpregs_xmm(struct savexmm *sv_xmm, struct save87 *sv_87)
{
bzero(sv_87, sizeof(*sv_87));
npx_fill_fpregs_xmm1(sv_xmm, sv_87);
}
void
npx_set_fpregs_xmm(struct save87 *sv_87, struct savexmm *sv_xmm)
{
struct env87 *penv_87;
struct envxmm *penv_xmm;
int i;
penv_87 = &sv_87->sv_env;
penv_xmm = &sv_xmm->sv_env;
/* FPU control/status */
penv_xmm->en_cw = penv_87->en_cw;
penv_xmm->en_sw = penv_87->en_sw;
penv_xmm->en_fip = penv_87->en_fip;
penv_xmm->en_fcs = penv_87->en_fcs;
penv_xmm->en_opcode = penv_87->en_opcode;
penv_xmm->en_foo = penv_87->en_foo;
penv_xmm->en_fos = penv_87->en_fos;
Convert between abridged (from FXSAVE) and unabridged (from FSAVE) versions of the x87 tags. The conversion is naive, used abridged tag is converted to valid unabridged, without additional checks for zero and special values. Noted by: bde Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-07-10 09:20:13 +00:00
/* FPU registers and tags */
penv_xmm->en_tw = 0;
for (i = 0; i < 8; ++i) {
Provide npx_get_fsave(9) and npx_set_fsave(9) functions to obtain and restore the FPU state from the format of machine FSAVE area. The intended use is for ABI emulators to provide FSAVE-formatted FPU state to usermode requiring it, while kernel could use FXSAVE due to XMM/XSAVE. The core functionality to convert from/to FXSAVE format is shared with the fill_fpregs_xmm() and set_fpregs_xmm(). Move the later functions to npx.c and rename them to npx_fill_fpregs_xmm() and npx_set_fpregs_xmm(). They differ from nptx_get/set_fsave(9) since our mcontext contains padding to be zeroed or ignored. fill_fpregs() and set_fpregs() could be converted to use the new interface, but there are small differences to handle. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-06-29 12:06:36 +00:00
sv_xmm->sv_fp[i].fp_acc = sv_87->sv_ac[i];
Convert between abridged (from FXSAVE) and unabridged (from FSAVE) versions of the x87 tags. The conversion is naive, used abridged tag is converted to valid unabridged, without additional checks for zero and special values. Noted by: bde Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-07-10 09:20:13 +00:00
if ((penv_87->en_tw && (3 << i)) != (3 << i))
penv_xmm->en_tw |= 1 << i;
}
Provide npx_get_fsave(9) and npx_set_fsave(9) functions to obtain and restore the FPU state from the format of machine FSAVE area. The intended use is for ABI emulators to provide FSAVE-formatted FPU state to usermode requiring it, while kernel could use FXSAVE due to XMM/XSAVE. The core functionality to convert from/to FXSAVE format is shared with the fill_fpregs_xmm() and set_fpregs_xmm(). Move the later functions to npx.c and rename them to npx_fill_fpregs_xmm() and npx_set_fpregs_xmm(). They differ from nptx_get/set_fsave(9) since our mcontext contains padding to be zeroed or ignored. fill_fpregs() and set_fpregs() could be converted to use the new interface, but there are small differences to handle. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2015-06-29 12:06:36 +00:00
}
#endif /* CPU_ENABLE_SSE */
void
npx_get_fsave(void *addr)
{
struct thread *td;
union savefpu *sv;
td = curthread;
npxgetregs(td);
sv = get_pcb_user_save_td(td);
#ifdef CPU_ENABLE_SSE
if (cpu_fxsr)
npx_fill_fpregs_xmm1(&sv->sv_xmm, addr);
else
#endif
bcopy(sv, addr, sizeof(struct env87) +
sizeof(struct fpacc87[8]));
}
int
npx_set_fsave(void *addr)
{
union savefpu sv;
int error;
bzero(&sv, sizeof(sv));
#ifdef CPU_ENABLE_SSE
if (cpu_fxsr)
npx_set_fpregs_xmm(addr, &sv.sv_xmm);
else
#endif
bcopy(addr, &sv, sizeof(struct env87) +
sizeof(struct fpacc87[8]));
error = npxsetregs(curthread, &sv, NULL, 0);
return (error);
}
Correct a local information leakage bug affecting AMD FPUs. Security: FreeBSD-SA-06:14.fpu
2006-04-19 07:00:19 +00:00
#ifdef CPU_ENABLE_SSE
/*
* On AuthenticAMD processors, the fxrstor instruction does not restore
* the x87's stored last instruction pointer, last data pointer, and last
* opcode values, except in the rare case in which the exception summary
* (ES) bit in the x87 status word is set to 1.
*
* In order to avoid leaking this information across processes, we clean
* these values by performing a dummy load before executing fxrstor().
*/
static void
fpu_clean_state(void)
{
Fix a few nits in the earlier changes to prevent local information leakage in AMD FPUs: - Do not clear the affected state in the case that the FPU registers for the thread that already owns the FPU are changed via fpu_setregs(). The only local information the thread would see is its own state in that case. - Fix a type mismatch for the dummy variable used in a "fld". It accepts a float, not a double. Reviewed by: bde Approved by: so (cperciva) MFC after: 1 month
2009-03-25 22:08:30 +00:00
static float dummy_variable = 0.0;
Correct a local information leakage bug affecting AMD FPUs. Security: FreeBSD-SA-06:14.fpu
2006-04-19 07:00:19 +00:00
u_short status;
/*
* Clear the ES bit in the x87 status word if it is currently
* set, in order to avoid causing a fault in the upcoming load.
*/
fnstsw(&status);
if (status & 0x80)
fnclex();
/*
* Load the dummy variable into the x87 stack. This mangles
* the x87 stack, but we don't care since we're about to call
* fxrstor() anyway.
*/
Use unambiguous inline assembly to load a float variable. GNU as silently converts 'fld' to 'flds', without taking the actual variable type into account (!), but clang's integrated assembler rightfully complains about it. Discussed with: cperciva
2010-11-25 18:14:18 +00:00
__asm __volatile("ffree %%st(7); flds %0" : : "m" (dummy_variable));
Correct a local information leakage bug affecting AMD FPUs. Security: FreeBSD-SA-06:14.fpu
2006-04-19 07:00:19 +00:00
}
#endif /* CPU_ENABLE_SSE */
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
static void
Use PCPU_GET(cpuid) instead of curproc->p_oncpu. Reviewed by: peter
2001-07-16 06:00:23 +00:00
fpurstor(addr)
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
union savefpu *addr;
{
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
#ifdef CPU_ENABLE_SSE
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (use_xsave)
xrstor((char *)addr, xsave_mask);
else if (cpu_fxsr)
The per-cpu temporary buffers are not needed since the pcb_save areas have the proper alignment. Change dummy variable in npxinit from stack to bss to ensure proper alignment. Reviewed by: bde
2001-07-17 13:06:47 +00:00
fxrstor(addr);
Fix a few nits in the earlier changes to prevent local information leakage in AMD FPUs: - Do not clear the affected state in the case that the FPU registers for the thread that already owns the FPU are changed via fpu_setregs(). The only local information the thread would see is its own state in that case. - Fix a type mismatch for the dummy variable used in a "fld". It accepts a float, not a double. Reviewed by: bde Approved by: so (cperciva) MFC after: 1 month
2009-03-25 22:08:30 +00:00
else
Dont compile in SSE fxsave/fxrstor instructions if CPU_ENABLE_SSE isn't active.
2001-08-23 01:03:56 +00:00
#endif
frstor(addr);
Activate SSE/SIMD. This is the extra context switching support that we are required to do if we let user processes use the extra 128 bit registers etc. This is the base part of the diff I got from: http://www.issei.org/issei/FreeBSD/sse.html I believe this is by: Mr. SUZUKI Issei <issei@issei.org> SMP support apparently by: Takekazu KATO <kato@chino.it.okayama-u.ac.jp> Test code by: NAKAMURA Kazushi <kaz@kobe1995.net>, see http://kobe1995.net/~kaz/FreeBSD/SSE.en.html I have fixed a couple of style(9) deviations. I have some followup commits to fix a couple of non-style things.
2001-07-12 06:32:51 +00:00
}
Fixed a misplaced ifdef that prevented npx.c building without "device isa" ISA. npx has few isa dependencies, but it does unconditional outb()'s to the isa bus in the !SMP case, and it attaches to isa if "device isa" is configured in order to support PNP-ISA. The ifdef for the latter was misplaced. PR: 62595
2004-02-13 18:04:51 +00:00
#ifdef DEV_ISA
Add a little do-nothing ``slopsucker'' device which gives a home to PNP0c04 (legacy ISA coprocessor support). Tourist info.
2000-05-04 23:57:32 +00:00
/*
Add ACPI attachments.
2001-08-30 09:17:03 +00:00
* This sucks up the legacy ISA support assignments from PNPBIOS/ACPI.
Add a little do-nothing ``slopsucker'' device which gives a home to PNP0c04 (legacy ISA coprocessor support). Tourist info.
2000-05-04 23:57:32 +00:00
*/
static struct isa_pnp_id npxisa_ids[] = {
{ 0x040cd041, "Legacy ISA coprocessor support" }, /* PNP0C04 */
{ 0 }
};
static int
npxisa_probe(device_t dev)
{
Make the PnP 'slopsucker' quiet in the !bootverbose case - the real NPX probe happens much earlier, and may come to very different conclusions about the system's NPX setup.
2000-06-23 08:19:50 +00:00
int result;
if ((result = ISA_PNP_PROBE(device_get_parent(dev), dev, npxisa_ids)) <= 0) {
device_quiet(dev);
}
return(result);
Add a little do-nothing ``slopsucker'' device which gives a home to PNP0c04 (legacy ISA coprocessor support). Tourist info.
2000-05-04 23:57:32 +00:00
}
static int
npxisa_attach(device_t dev)
{
return (0);
}
static device_method_t npxisa_methods[] = {
/* Device interface */
DEVMETHOD(device_probe, npxisa_probe),
DEVMETHOD(device_attach, npxisa_attach),
DEVMETHOD(device_detach, bus_generic_detach),
DEVMETHOD(device_shutdown, bus_generic_shutdown),
DEVMETHOD(device_suspend, bus_generic_suspend),
DEVMETHOD(device_resume, bus_generic_resume),
{ 0, 0 }
};
static driver_t npxisa_driver = {
"npxisa",
npxisa_methods,
1, /* no softc */
};
static devclass_t npxisa_devclass;
DRIVER_MODULE(npxisa, isa, npxisa_driver, npxisa_devclass, 0, 0);
MFi386: - sys/pc98/pc98/npx.c 1.87 (2001/09/15; author: imp) I don't think pc98 has acpi at all, so ifdef the acpi attachments for now. This completes merging sys/pc98/pc98/npx.c into sys/i386/isa/npx.c so that the former can be removed.
2001-10-21 06:05:08 +00:00
#ifndef PC98
Add ACPI attachments.
2001-08-30 09:17:03 +00:00
DRIVER_MODULE(npxisa, acpi, npxisa_driver, npxisa_devclass, 0, 0);
MFi386: - sys/pc98/pc98/npx.c 1.87 (2001/09/15; author: imp) I don't think pc98 has acpi at all, so ifdef the acpi attachments for now. This completes merging sys/pc98/pc98/npx.c into sys/i386/isa/npx.c so that the former can be removed.
2001-10-21 06:05:08 +00:00
#endif
Don't include <isa/isavar.h> or compile code depending on it when isa is not configured. Including <isa/isavar.h> when it is not used is harmful as well as bogus, since it includes "isa_if.h" which is not generated when isa is not configured. This was fixed in 1999 but was broken by unconditionalizing PNPBIOS.
2002-01-30 12:41:12 +00:00
#endif /* DEV_ISA */
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
Add support for the extended FPU states on amd64, both for native 64bit and 32bit ABIs. As a side-effect, it enables AVX on capable CPUs. In particular: - Query the CPU support for XSAVE, list of the supported extensions and the required size of FPU save area. The hw.use_xsave tunable is provided for disabling XSAVE, and hw.xsave_mask may be used to select the enabled extensions. - Remove the FPU save area from PCB and dynamically allocate the (run-time sized) user save area on the top of the kernel stack, right above the PCB. Reorganize the thread0 PCB initialization to postpone it after BSP is queried for save area size. - The dumppcb, stoppcbs and susppcbs now do not carry the FPU state as well. FPU state is only useful for suspend, where it is saved in dynamically allocated suspfpusave area. - Use XSAVE and XRSTOR to save/restore FPU state, if supported and enabled. - Define new mcontext_t flag _MC_HASFPXSTATE, indicating that mcontext_t has a valid pointer to out-of-struct extended FPU state. Signal handlers are supplied with stack-allocated fpu state. The sigreturn(2) and setcontext(2) syscall honour the flag, allowing the signal handlers to inspect and manipilate extended state in the interrupted context. - The getcontext(2) never returns extended state, since there is no place in the fixed-sized mcontext_t to place variable-sized save area. And, since mcontext_t is embedded into ucontext_t, makes it impossible to fix in a reasonable way. Instead of extending getcontext(2) syscall, provide a sysarch(2) facility to query extended FPU state. - Add ptrace(2) support for getting and setting extended state; while there, implement missed PT_I386_{GET,SET}XMMREGS for 32bit binaries. - Change fpu_kern KPI to not expose struct fpu_kern_ctx layout to consumers, making it opaque. Internally, struct fpu_kern_ctx now contains a space for the extended state. Convert in-kernel consumers of fpu_kern KPI both on i386 and amd64. First version of the support for AVX was submitted by Tim Bird <tim.bird am sony com> on behalf of Sony. This version was written from scratch. Tested by: pho (previous version), Yamagi Burmeister <lists yamagi org> MFC after: 1 month
2012-01-21 17:45:27 +00:00
static MALLOC_DEFINE(M_FPUKERN_CTX, "fpukern_ctx",
"Kernel contexts for FPU state");
#define FPU_KERN_CTX_NPXINITDONE 0x01
Add FPU_KERN_KTHR flag to fpu_kern_enter(9), which avoids saving FPU context into memory for the kernel threads which called fpu_kern_thread(9). This allows the fpu_kern_enter() callers to not check for is_fpu_kern_thread() to get the optimization. Apply the flag to padlock(4) and aesni(4). In aesni_cipher_process(), do not leak FPU context state on error. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2014-06-23 07:37:54 +00:00
#define FPU_KERN_CTX_DUMMY 0x02
Now that aesni won't reuse fpu contexts (D3016), add seatbelts to the fpu code to prevent other reuse of the contexts in the future... Differential Revision: https://reviews.freebsd.org/D3015 Reviewed by: kib, gnn
2015-07-08 19:26:36 +00:00
#define FPU_KERN_CTX_INUSE 0x04
Add support for the extended FPU states on amd64, both for native 64bit and 32bit ABIs. As a side-effect, it enables AVX on capable CPUs. In particular: - Query the CPU support for XSAVE, list of the supported extensions and the required size of FPU save area. The hw.use_xsave tunable is provided for disabling XSAVE, and hw.xsave_mask may be used to select the enabled extensions. - Remove the FPU save area from PCB and dynamically allocate the (run-time sized) user save area on the top of the kernel stack, right above the PCB. Reorganize the thread0 PCB initialization to postpone it after BSP is queried for save area size. - The dumppcb, stoppcbs and susppcbs now do not carry the FPU state as well. FPU state is only useful for suspend, where it is saved in dynamically allocated suspfpusave area. - Use XSAVE and XRSTOR to save/restore FPU state, if supported and enabled. - Define new mcontext_t flag _MC_HASFPXSTATE, indicating that mcontext_t has a valid pointer to out-of-struct extended FPU state. Signal handlers are supplied with stack-allocated fpu state. The sigreturn(2) and setcontext(2) syscall honour the flag, allowing the signal handlers to inspect and manipilate extended state in the interrupted context. - The getcontext(2) never returns extended state, since there is no place in the fixed-sized mcontext_t to place variable-sized save area. And, since mcontext_t is embedded into ucontext_t, makes it impossible to fix in a reasonable way. Instead of extending getcontext(2) syscall, provide a sysarch(2) facility to query extended FPU state. - Add ptrace(2) support for getting and setting extended state; while there, implement missed PT_I386_{GET,SET}XMMREGS for 32bit binaries. - Change fpu_kern KPI to not expose struct fpu_kern_ctx layout to consumers, making it opaque. Internally, struct fpu_kern_ctx now contains a space for the extended state. Convert in-kernel consumers of fpu_kern KPI both on i386 and amd64. First version of the support for AVX was submitted by Tim Bird <tim.bird am sony com> on behalf of Sony. This version was written from scratch. Tested by: pho (previous version), Yamagi Burmeister <lists yamagi org> MFC after: 1 month
2012-01-21 17:45:27 +00:00
struct fpu_kern_ctx {
union savefpu *prev;
uint32_t flags;
char hwstate1[];
};
struct fpu_kern_ctx *
fpu_kern_alloc_ctx(u_int flags)
{
struct fpu_kern_ctx *res;
size_t sz;
sz = sizeof(struct fpu_kern_ctx) + XSAVE_AREA_ALIGN +
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
cpu_max_ext_state_size;
Add support for the extended FPU states on amd64, both for native 64bit and 32bit ABIs. As a side-effect, it enables AVX on capable CPUs. In particular: - Query the CPU support for XSAVE, list of the supported extensions and the required size of FPU save area. The hw.use_xsave tunable is provided for disabling XSAVE, and hw.xsave_mask may be used to select the enabled extensions. - Remove the FPU save area from PCB and dynamically allocate the (run-time sized) user save area on the top of the kernel stack, right above the PCB. Reorganize the thread0 PCB initialization to postpone it after BSP is queried for save area size. - The dumppcb, stoppcbs and susppcbs now do not carry the FPU state as well. FPU state is only useful for suspend, where it is saved in dynamically allocated suspfpusave area. - Use XSAVE and XRSTOR to save/restore FPU state, if supported and enabled. - Define new mcontext_t flag _MC_HASFPXSTATE, indicating that mcontext_t has a valid pointer to out-of-struct extended FPU state. Signal handlers are supplied with stack-allocated fpu state. The sigreturn(2) and setcontext(2) syscall honour the flag, allowing the signal handlers to inspect and manipilate extended state in the interrupted context. - The getcontext(2) never returns extended state, since there is no place in the fixed-sized mcontext_t to place variable-sized save area. And, since mcontext_t is embedded into ucontext_t, makes it impossible to fix in a reasonable way. Instead of extending getcontext(2) syscall, provide a sysarch(2) facility to query extended FPU state. - Add ptrace(2) support for getting and setting extended state; while there, implement missed PT_I386_{GET,SET}XMMREGS for 32bit binaries. - Change fpu_kern KPI to not expose struct fpu_kern_ctx layout to consumers, making it opaque. Internally, struct fpu_kern_ctx now contains a space for the extended state. Convert in-kernel consumers of fpu_kern KPI both on i386 and amd64. First version of the support for AVX was submitted by Tim Bird <tim.bird am sony com> on behalf of Sony. This version was written from scratch. Tested by: pho (previous version), Yamagi Burmeister <lists yamagi org> MFC after: 1 month
2012-01-21 17:45:27 +00:00
res = malloc(sz, M_FPUKERN_CTX, ((flags & FPU_KERN_NOWAIT) ?
M_NOWAIT : M_WAITOK) | M_ZERO);
return (res);
}
void
fpu_kern_free_ctx(struct fpu_kern_ctx *ctx)
{
Now that aesni won't reuse fpu contexts (D3016), add seatbelts to the fpu code to prevent other reuse of the contexts in the future... Differential Revision: https://reviews.freebsd.org/D3015 Reviewed by: kib, gnn
2015-07-08 19:26:36 +00:00
KASSERT((ctx->flags & FPU_KERN_CTX_INUSE) == 0, ("free'ing inuse ctx"));
Add support for the extended FPU states on amd64, both for native 64bit and 32bit ABIs. As a side-effect, it enables AVX on capable CPUs. In particular: - Query the CPU support for XSAVE, list of the supported extensions and the required size of FPU save area. The hw.use_xsave tunable is provided for disabling XSAVE, and hw.xsave_mask may be used to select the enabled extensions. - Remove the FPU save area from PCB and dynamically allocate the (run-time sized) user save area on the top of the kernel stack, right above the PCB. Reorganize the thread0 PCB initialization to postpone it after BSP is queried for save area size. - The dumppcb, stoppcbs and susppcbs now do not carry the FPU state as well. FPU state is only useful for suspend, where it is saved in dynamically allocated suspfpusave area. - Use XSAVE and XRSTOR to save/restore FPU state, if supported and enabled. - Define new mcontext_t flag _MC_HASFPXSTATE, indicating that mcontext_t has a valid pointer to out-of-struct extended FPU state. Signal handlers are supplied with stack-allocated fpu state. The sigreturn(2) and setcontext(2) syscall honour the flag, allowing the signal handlers to inspect and manipilate extended state in the interrupted context. - The getcontext(2) never returns extended state, since there is no place in the fixed-sized mcontext_t to place variable-sized save area. And, since mcontext_t is embedded into ucontext_t, makes it impossible to fix in a reasonable way. Instead of extending getcontext(2) syscall, provide a sysarch(2) facility to query extended FPU state. - Add ptrace(2) support for getting and setting extended state; while there, implement missed PT_I386_{GET,SET}XMMREGS for 32bit binaries. - Change fpu_kern KPI to not expose struct fpu_kern_ctx layout to consumers, making it opaque. Internally, struct fpu_kern_ctx now contains a space for the extended state. Convert in-kernel consumers of fpu_kern KPI both on i386 and amd64. First version of the support for AVX was submitted by Tim Bird <tim.bird am sony com> on behalf of Sony. This version was written from scratch. Tested by: pho (previous version), Yamagi Burmeister <lists yamagi org> MFC after: 1 month
2012-01-21 17:45:27 +00:00
/* XXXKIB clear the memory ? */
free(ctx, M_FPUKERN_CTX);
}
static union savefpu *
fpu_kern_ctx_savefpu(struct fpu_kern_ctx *ctx)
{
vm_offset_t p;
p = (vm_offset_t)&ctx->hwstate1;
p = roundup2(p, XSAVE_AREA_ALIGN);
return ((union savefpu *)p);
}
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
int
fpu_kern_enter(struct thread *td, struct fpu_kern_ctx *ctx, u_int flags)
{
struct pcb *pcb;
Now that aesni won't reuse fpu contexts (D3016), add seatbelts to the fpu code to prevent other reuse of the contexts in the future... Differential Revision: https://reviews.freebsd.org/D3015 Reviewed by: kib, gnn
2015-07-08 19:26:36 +00:00
KASSERT((ctx->flags & FPU_KERN_CTX_INUSE) == 0, ("using inuse ctx"));
Add FPU_KERN_KTHR flag to fpu_kern_enter(9), which avoids saving FPU context into memory for the kernel threads which called fpu_kern_thread(9). This allows the fpu_kern_enter() callers to not check for is_fpu_kern_thread() to get the optimization. Apply the flag to padlock(4) and aesni(4). In aesni_cipher_process(), do not leak FPU context state on error. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2014-06-23 07:37:54 +00:00
if ((flags & FPU_KERN_KTHR) != 0 && is_fpu_kern_thread(0)) {
Now that aesni won't reuse fpu contexts (D3016), add seatbelts to the fpu code to prevent other reuse of the contexts in the future... Differential Revision: https://reviews.freebsd.org/D3015 Reviewed by: kib, gnn
2015-07-08 19:26:36 +00:00
ctx->flags = FPU_KERN_CTX_DUMMY | FPU_KERN_CTX_INUSE;
Add FPU_KERN_KTHR flag to fpu_kern_enter(9), which avoids saving FPU context into memory for the kernel threads which called fpu_kern_thread(9). This allows the fpu_kern_enter() callers to not check for is_fpu_kern_thread() to get the optimization. Apply the flag to padlock(4) and aesni(4). In aesni_cipher_process(), do not leak FPU context state on error. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2014-06-23 07:37:54 +00:00
return (0);
}
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb = td->td_pcb;
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
KASSERT(!PCB_USER_FPU(pcb) || pcb->pcb_save ==
get_pcb_user_save_pcb(pcb), ("mangled pcb_save"));
Now that aesni won't reuse fpu contexts (D3016), add seatbelts to the fpu code to prevent other reuse of the contexts in the future... Differential Revision: https://reviews.freebsd.org/D3015 Reviewed by: kib, gnn
2015-07-08 19:26:36 +00:00
ctx->flags = FPU_KERN_CTX_INUSE;
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
if ((pcb->pcb_flags & PCB_NPXINITDONE) != 0)
ctx->flags |= FPU_KERN_CTX_NPXINITDONE;
npxexit(td);
ctx->prev = pcb->pcb_save;
Add support for the extended FPU states on amd64, both for native 64bit and 32bit ABIs. As a side-effect, it enables AVX on capable CPUs. In particular: - Query the CPU support for XSAVE, list of the supported extensions and the required size of FPU save area. The hw.use_xsave tunable is provided for disabling XSAVE, and hw.xsave_mask may be used to select the enabled extensions. - Remove the FPU save area from PCB and dynamically allocate the (run-time sized) user save area on the top of the kernel stack, right above the PCB. Reorganize the thread0 PCB initialization to postpone it after BSP is queried for save area size. - The dumppcb, stoppcbs and susppcbs now do not carry the FPU state as well. FPU state is only useful for suspend, where it is saved in dynamically allocated suspfpusave area. - Use XSAVE and XRSTOR to save/restore FPU state, if supported and enabled. - Define new mcontext_t flag _MC_HASFPXSTATE, indicating that mcontext_t has a valid pointer to out-of-struct extended FPU state. Signal handlers are supplied with stack-allocated fpu state. The sigreturn(2) and setcontext(2) syscall honour the flag, allowing the signal handlers to inspect and manipilate extended state in the interrupted context. - The getcontext(2) never returns extended state, since there is no place in the fixed-sized mcontext_t to place variable-sized save area. And, since mcontext_t is embedded into ucontext_t, makes it impossible to fix in a reasonable way. Instead of extending getcontext(2) syscall, provide a sysarch(2) facility to query extended FPU state. - Add ptrace(2) support for getting and setting extended state; while there, implement missed PT_I386_{GET,SET}XMMREGS for 32bit binaries. - Change fpu_kern KPI to not expose struct fpu_kern_ctx layout to consumers, making it opaque. Internally, struct fpu_kern_ctx now contains a space for the extended state. Convert in-kernel consumers of fpu_kern KPI both on i386 and amd64. First version of the support for AVX was submitted by Tim Bird <tim.bird am sony com> on behalf of Sony. This version was written from scratch. Tested by: pho (previous version), Yamagi Burmeister <lists yamagi org> MFC after: 1 month
2012-01-21 17:45:27 +00:00
pcb->pcb_save = fpu_kern_ctx_savefpu(ctx);
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb->pcb_flags |= PCB_KERNNPX;
pcb->pcb_flags &= ~PCB_NPXINITDONE;
return (0);
}
int
fpu_kern_leave(struct thread *td, struct fpu_kern_ctx *ctx)
{
struct pcb *pcb;
Now that aesni won't reuse fpu contexts (D3016), add seatbelts to the fpu code to prevent other reuse of the contexts in the future... Differential Revision: https://reviews.freebsd.org/D3015 Reviewed by: kib, gnn
2015-07-08 19:26:36 +00:00
KASSERT((ctx->flags & FPU_KERN_CTX_INUSE) != 0,
("leaving not inuse ctx"));
ctx->flags &= ~FPU_KERN_CTX_INUSE;
Add FPU_KERN_KTHR flag to fpu_kern_enter(9), which avoids saving FPU context into memory for the kernel threads which called fpu_kern_thread(9). This allows the fpu_kern_enter() callers to not check for is_fpu_kern_thread() to get the optimization. Apply the flag to padlock(4) and aesni(4). In aesni_cipher_process(), do not leak FPU context state on error. Sponsored by: The FreeBSD Foundation MFC after: 1 week
2014-06-23 07:37:54 +00:00
if (is_fpu_kern_thread(0) && (ctx->flags & FPU_KERN_CTX_DUMMY) != 0)
return (0);
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb = td->td_pcb;
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_enter();
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
if (curthread == PCPU_GET(fpcurthread))
npxdrop();
After the FPU use requires #MF working due to INT13 FPU exception handling removal, MFi386 r209198: Use critical sections instead of disabling local interrupts to ensure the consistency between PCPU fpcurthread and the state of FPU. Reviewed by: bde Tested by: pho
2010-06-23 11:21:19 +00:00
critical_exit();
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
pcb->pcb_save = ctx->prev;
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
if (pcb->pcb_save == get_pcb_user_save_pcb(pcb)) {
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
if ((pcb->pcb_flags & PCB_NPXUSERINITDONE) != 0)
pcb->pcb_flags |= PCB_NPXINITDONE;
else
pcb->pcb_flags &= ~PCB_NPXINITDONE;
pcb->pcb_flags &= ~PCB_KERNNPX;
} else {
if ((ctx->flags & FPU_KERN_CTX_NPXINITDONE) != 0)
pcb->pcb_flags |= PCB_NPXINITDONE;
else
pcb->pcb_flags &= ~PCB_NPXINITDONE;
KASSERT(!PCB_USER_FPU(pcb), ("unpaired fpu_kern_leave"));
}
return (0);
}
int
fpu_kern_thread(u_int flags)
{
KASSERT((curthread->td_pflags & TDP_KTHREAD) != 0,
("Only kthread may use fpu_kern_thread"));
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
KASSERT(curpcb->pcb_save == get_pcb_user_save_pcb(curpcb),
MFamd64 r238623: Introduce curpcb magic variable. Requested and reviewed by: bde MFC after: 3 weeks
2012-07-26 09:11:37 +00:00
("mangled pcb_save"));
KASSERT(PCB_USER_FPU(curpcb), ("recursive call"));
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
MFamd64 r238623: Introduce curpcb magic variable. Requested and reviewed by: bde MFC after: 3 weeks
2012-07-26 09:11:37 +00:00
curpcb->pcb_flags |= PCB_KERNNPX;
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
return (0);
}
int
is_fpu_kern_thread(u_int flags)
{
if ((curthread->td_pflags & TDP_KTHREAD) == 0)
return (0);
MFamd64 r238623: Introduce curpcb magic variable. Requested and reviewed by: bde MFC after: 3 weeks
2012-07-26 09:11:37 +00:00
return ((curpcb->pcb_flags & PCB_KERNNPX) != 0);
Introduce the x86 kernel interfaces to allow kernel code to use FPU/SSE hardware. Caller should provide a save area that is chained into the stack of the areas; pcb save_area for usermode FPU state is on top. The pcb now contains a pointer to the current FPU saved area, used during FPUDNA handling and context switches. There is also a facility to allow the kernel thread to use pcb save_area. Change the dreaded warnings "npxdna in kernel mode!" into the panics when FPU usage is not registered. KPI discussed with: fabient Tested by: pho, fabient Hardware provided by: Sentex Communications MFC after: 1 month
2010-06-05 15:59:59 +00:00
}
MFamd64: Add support for extended FPU states on i386. This includes support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520
2014-11-02 22:58:30 +00:00
/*
* FPU save area alloc/free/init utility routines
*/
union savefpu *
fpu_save_area_alloc(void)
{
return (uma_zalloc(fpu_save_area_zone, 0));
}
void
fpu_save_area_free(union savefpu *fsa)
{
uma_zfree(fpu_save_area_zone, fsa);
}
void
fpu_save_area_reset(union savefpu *fsa)
{
bcopy(npx_initialstate, fsa, cpu_max_ext_state_size);
}
Reference in New Issue Copy Permalink