Commit Graph

3640 Commits

Author SHA1 Message Date
Kevin Lo
9823d52705 Revert previous commit...
Pointyhat to:	kevlo (myself)
2012-10-10 08:36:38 +00:00
Kevin Lo
a10cee30c9 Prefer NULL over 0 for pointers 2012-10-09 08:27:40 +00:00
Konstantin Belousov
a42fa0af44 Change (unused) prototype for stmxcsr() to match reality.
Noted by:	jhb
MFC after:	1 week
2012-07-30 19:26:02 +00:00
Konstantin Belousov
e93d0cbef1 MFamd64 r238623:
Introduce curpcb magic variable.

Requested and reviewed by:	bde
MFC after:	3 weeks
2012-07-26 09:11:37 +00:00
Konstantin Belousov
1e39a4bcee MFCamd64 r238598:
Provide siginfo.si_code for floating point errors when error occurs
using the SSE math processor.

MFC after:    3 weeks
2012-07-21 21:52:48 +00:00
Konstantin Belousov
dad46c5594 MFamd64 r238668:
Stop clearing x87 exceptions in the #MF handler.

Requested by:	bde
MFC after:	1 week
2012-07-21 21:49:05 +00:00
Konstantin Belousov
4596f12de7 MFamd64 r238597:
Add stmxcsr.

MFC after:	3 weeks
2012-07-21 21:39:23 +00:00
John Baldwin
d706ec297a Add a clts() wrapper around the 'clts' instruction to <machine/cpufunc.h>
on x86 and use that to implement stop_emulating() in the fpu/npx code.
Reimplement start_emulating() in the non-XEN case by using load_cr0() and
rcr0() instead of the 'lmsw' and 'smsw' instructions.  Intel explicitly
discourages the use of 'lmsw' and 'smsw' on 80386 and later processors in
the description of these instructions in Volume 2 of the ADM.

Reviewed by:	kib
MFC after:	1 month
2012-07-09 20:55:39 +00:00
Konstantin Belousov
8c6f8f3d5b Add support for the extended FPU states on amd64, both for native
64bit and 32bit ABIs.  As a side-effect, it enables AVX on capable
CPUs.

In particular:

- Query the CPU support for XSAVE, list of the supported extensions
  and the required size of FPU save area. The hw.use_xsave tunable is
  provided for disabling XSAVE, and hw.xsave_mask may be used to
  select the enabled extensions.

- Remove the FPU save area from PCB and dynamically allocate the
  (run-time sized) user save area on the top of the kernel stack,
  right above the PCB. Reorganize the thread0 PCB initialization to
  postpone it after BSP is queried for save area size.

- The dumppcb, stoppcbs and susppcbs now do not carry the FPU state as
  well. FPU state is only useful for suspend, where it is saved in
  dynamically allocated suspfpusave area.

- Use XSAVE and XRSTOR to save/restore FPU state, if supported and
  enabled.

- Define new mcontext_t flag _MC_HASFPXSTATE, indicating that
  mcontext_t has a valid pointer to out-of-struct extended FPU
  state. Signal handlers are supplied with stack-allocated fpu
  state. The sigreturn(2) and setcontext(2) syscall honour the flag,
  allowing the signal handlers to inspect and manipilate extended
  state in the interrupted context.

- The getcontext(2) never returns extended state, since there is no
  place in the fixed-sized mcontext_t to place variable-sized save
  area. And, since mcontext_t is embedded into ucontext_t, makes it
  impossible to fix in a reasonable way.  Instead of extending
  getcontext(2) syscall, provide a sysarch(2) facility to query
  extended FPU state.

- Add ptrace(2) support for getting and setting extended state; while
  there, implement missed PT_I386_{GET,SET}XMMREGS for 32bit binaries.

- Change fpu_kern KPI to not expose struct fpu_kern_ctx layout to
  consumers, making it opaque. Internally, struct fpu_kern_ctx now
  contains a space for the extended state. Convert in-kernel consumers
  of fpu_kern KPI both on i386 and amd64.

First version of the support for AVX was submitted by Tim Bird
<tim.bird am sony com> on behalf of Sony. This version was written
from scratch.

Tested by:	pho (previous version), Yamagi Burmeister <lists yamagi org>
MFC after:	1 month
2012-01-21 17:45:27 +00:00
Jung-uk Kim
3453537fa5 Use atomic load & store for TSC frequency. It may be overkill for amd64 but
safer for i386 because it can be easily over 4 GHz now.  More worse, it can
be easily changed by user with 'machdep.tsc_freq' tunable (directly) or
cpufreq(4) (indirectly).  Note it is intentionally not used in performance
critical paths to avoid performance regression (but we should, in theory).
Alternatively, we may add "virtual TSC" with lower frequency if maximum
frequency overflows 32 bits (and ignore possible incoherency as we do now).
2011-04-07 23:28:28 +00:00
Jung-uk Kim
bc34c87e81 Deprecate rarely used tsc_is_broken. Instead, we zero out tsc_freq because
it is almost always used with tsc_freq any way.
2011-03-10 20:02:58 +00:00
Konstantin Belousov
60c7c84e85 In fpudna()/npxdna(), mark FPU context initialized and optionally
mark user FPU context initialized, if current context is user context.
It was reversed in r215865, by inadequate change of this code fragment
to a call to fpuuserinited()/npxuserinited().

The issue is only relevant for in-kernel users of FPU.

Reported by:	Jan Henrik Sylvester <me janh de>, Mike Tancsa <mike sentex net>
Tested by:	Mike Tancsa
MFC after:	3 days
2010-12-12 16:16:39 +00:00
Konstantin Belousov
5c6eb03790 Remove npxgetregs(), npxsetregs(), fpugetregs() and fpusetregs()
functions, they are unused. Remove 'user' from npxgetuserregs()
etc. names.

For {npx,fpu}{get,set}regs(), always use pcb->pcb_user_save for FPU
context storage. This eliminates the need for ugly copying with
overwrite of the newly added and reserved fields in ucontext on i386
to satisfy alignment requirements for fpusave() and fpurstor().

pc98 version was copied from i386.

Suggested and reviewed by:	bde
Tested by:    pho (i386 and amd64)
MFC after:    1 week
2010-11-26 14:50:42 +00:00
Dimitry Andric
079d7e43ca Use unambiguous inline assembly to load a float variable. GNU as
silently converts 'fld' to 'flds', without taking the actual variable
type into account (!), but clang's integrated assembler rightfully
complains about it.

Discussed with:	cperciva
2010-11-25 18:14:18 +00:00
John Baldwin
0689bdcc19 Use 'saveintr' instead of 'savecrit' or 'eflags' to hold the state returned
by intr_disable().

Requested by:	bde
2010-10-25 15:31:13 +00:00
Jung-uk Kim
172754036a Simplify fldcw() macro. There is no reason to use pointer here. No object
file change after this commit (verified with md5).
2010-07-26 23:20:55 +00:00
Jung-uk Kim
8b019a8887 Remove an unused macro since r189418. 2010-07-26 22:55:14 +00:00
Jung-uk Kim
30402401a7 Reduce diff against fenv.h:
Mark all inline asms as volatile for safety.  No object file change after
this commit (verified with md5).
2010-07-26 22:16:36 +00:00
Jung-uk Kim
2e50fa36a5 FNSTSW instruction can use AX register as an operand.
Obtained from:	fenv.h
2010-07-26 21:24:52 +00:00
Konstantin Belousov
1060a94fb5 After the FPU use requires #MF working due to INT13 FPU exception handling
removal, MFi386 r209198:
    Use critical sections instead of disabling local interrupts to ensure
    the consistency between PCPU fpcurthread and the state of FPU.

Reviewed by:	bde
Tested by:	pho
2010-06-23 11:21:19 +00:00
Konstantin Belousov
699d648aab Remove the support for int13 FPU exception reporting on i386. It is
believed that all 486-class CPUs FreeBSD is capable to run on, either
have no FPU and cannot use external coprocessor, or have FPU on the
package and can use #MF.

Reviewed by:	bde
Tested by:	pho (previous version)
2010-06-23 11:12:58 +00:00
Konstantin Belousov
95882b9865 Remove unused i586 optimized bcopy/bzero/etc implementations that utilize
FPU registers for copying. Remove the switch table and jumps from
bcopy/bzero/... to the actual implementation.
As a side-effect, i486-optimized bzero is removed.

Reviewed by:	bde
Tested by:	pho (previous version)
2010-06-23 10:40:28 +00:00
Konstantin Belousov
6cf9a08d2c Introduce the x86 kernel interfaces to allow kernel code to use
FPU/SSE hardware. Caller should provide a save area that is chained
into the stack of the areas; pcb save_area for usermode FPU state is
on top. The pcb now contains a pointer to the current FPU saved area,
used during FPUDNA handling and context switches.  There is also a
facility to allow the kernel thread to use pcb save_area.

Change the dreaded warnings "npxdna in kernel mode!" into the panics
when FPU usage is not registered.

KPI discussed with:	fabient
Tested by:    pho, fabient
Hardware provided by:	Sentex Communications
MFC after:    1 month
2010-06-05 15:59:59 +00:00
Attilio Rao
3258030144 Introduce the new kernel sub-tree x86 which should contain all the code
shared and generalized between our current amd64, i386 and pc98.

This is just an initial step that should lead to a more complete effort.
For the moment, a very simple porting of cpufreq modules, BIOS calls and
the whole MD specific ISA bus part is added to the sub-tree but ideally
a lot of code might be added and more shared support should grow.

Sponsored by:	Sandvine Incorporated
Reviewed by:	emaste, kib, jhb, imp
Discussed on:	arch
MFC:		3 weeks
2010-02-25 14:13:39 +00:00
Attilio Rao
a7ccec946b - Allow clock subsystem to be compiled without the apic support [0]
- ATPIC, on pc98 is never defined somewhere, differently from i386.
  Turn its compilation to be conditional as i386 does. [1]

[0] Reported by:	nyan
[1] Submitted by:	nyan
2010-01-17 23:23:35 +00:00
Attilio Rao
a26cb6d547 Handling all the three clocks (hardclock, softclock, profclock) with the
LAPIC may lead to aliasing for softclock and profclock because frequencies
are sized in order to fit mainly hardclock.
atrtc used to take care of the softclock and profclock and it does still
do, if the LAPIC can't handle the clocks properly.

Revert the change when the LAPIC started taking charge of all three of
them and let atrtc handle softclock and profclock if not explicitly
requested. Such request can be made setting != 0 the new tunable
machdep.lapic_allclocks or if the new device ATPIC is not present
within the i386 kernel config (atrtc is linked to atpic presence).

Diagnosed by:	Sandvine Incorporated
Reviewed by:	jhb, emaste
Sponsored by:	Sandvine Incorporated
MFC:		3 weeks
2010-01-15 16:04:30 +00:00
Roman Divacky
68c4dfdf0c Make isa_dma functions MPSAFE by introducing its own private lock. These
functions are selfcontained (ie. they touch only isa_dma.c static variables
and hardware) so a private lock is sufficient to prevent races. This changes
only i386/amd64 while there are also isa_dma functions for ia64/sparc64.
Sparc64 are ones empty stubs and ia64 ones are unused as ia64 does not
have isa (says marcel).

This patch removes explicit locking of Giant from a few drivers (there
are some that requires this but lack ones - this patch fixes this) and
also removes the need for implicit locking of Giant from attach routines
where it's provided by newbus.

Approved by:	ed (mentor, implicit)
Reviewed by:	jhb, attilio (glanced by)
Tested by:	Giovanni Trematerra <giovanni.trematerra gmail com>
IA64 clue:	marcel
2009-11-09 20:29:10 +00:00
Xin LI
ee5e90dab2 - Teach vesa(4) and dpms(4) about x86emu. [1]
- Add vesa kernel options for amd64.
 - Connect libvgl library and splash kernel modules to amd64 build.
 - Connect manual page dpms(4) to amd64 build.
 - Remove old vesa/dpms files.

Submitted by:	paradox <ddkprog yahoo com> [1], swell k at gmail.com
		(with some minor tweaks)
2009-09-09 09:50:31 +00:00
Xin LI
f9d38c281c Partially revert 196524: this part of change should not be committed as
part of the changeset - it's an unrelated one.

Reported by:	danfe
2009-08-31 17:34:11 +00:00
Gleb Smirnoff
08cc4f3542 Fix build broken in r196524. 2009-08-25 14:08:33 +00:00
Xin LI
c5bebef869 Fix VESA modes and allow 8bit depth modes.
PR:		i386/124902
Submitted by:	paradox <ddkprog yahoo com>
MFC after:	2 months
2009-08-24 22:35:53 +00:00
John Baldwin
cebc7fb16c Improve the handling of cpuset with interrupts.
- For x86, change the interrupt source method to assign an interrupt source
  to a specific CPU to return an error value instead of void, thus allowing
  it to fail.
- If moving an interrupt to a CPU fails due to a lack of IDT vectors in the
  destination CPU, fail the request with ENOSPC rather than panicing.
- For MSI interrupts on x86 (but not MSI-X), only allow cpuset to be used
  on the first interrupt in a group.  Moving the first interrupt in a group
  moves the entire group.
- Use the icu_lock to protect intr_next_cpu() on x86 instead of the
  intr_table_lock to fix a LOR introduced in the last set of MSI changes.
- Add a new privilege PRIV_SCHED_CPUSET_INTR for using cpuset with
  interrupts.  Previously, binding an interrupt to a CPU only performed a
  privilege check if the interrupt had an interrupt thread.  Interrupts
  without a thread could be bound by non-root users as a result.
- If an interrupt event's assign_cpu method fails, then restore the original
  cpuset mask for the associated interrupt thread.

Approved by:	re (kib)
2009-07-01 17:20:07 +00:00
Alexander Motin
9f23a6caa4 Make algorithm a bit more bulletproof. 2009-06-23 23:16:37 +00:00
Alexander Motin
96c5d068d8 Rework r193814:
While general idea of patch was good, it was not working properly due the way
it was implemented. When we are using same timer interrupt for several of
hard/prof/stat purposes we should not send several IPIs same time to other
CPUs. Sending several IPIs same time leads to terrible accounting/profiling
results due to strong synchronization effect, when the second interrupt
handler accounts processing of the first one.
Interlink timer events in a such way, that no more then one IPI is sent for
any original timer interrupt.
2009-06-23 21:45:33 +00:00
Ariff Abdullah
b65cb1db3c When using i8254 as the only kernel timer source:
- Interpolate stat/prof clock using clkintr() in a similar fashion to
  local APIC timer, since statclock usually run slower.

- Liberate hardclockintr() from taking the burden of handling both stat
  and prof clock interrupt. Instead, send IPIs within clkintr() to handle
  those.
2009-06-09 07:26:52 +00:00
Xin LI
38676b52dd Add line width calculations for 15/16 and 24/32 bit modes in case
the "Get Scan Line Length" function fails, as it does in Parallels
(in Version 2.2, Build 2112 at least).

PR:		i386/127367
Obtained from:	DragonFly
Submitted by:	Pedro Giffuni
MFC after:	1 month
2009-06-09 00:54:57 +00:00
Alexander Motin
1703f2b424 Rename statclock_disable variable to atrtcclock_disable that it actually is,
and hide it inside of atrtc driver. Add new tunable hint.atrtc.0.clock
controlling it. Setting it to 0 disables using RTC clock as stat-/
profclock sources.

Teach i386 and amd64 SMP platforms to emulate stat-/profclocks using i8254
hardclock, when LAPIC and RTC clocks are disabled.

This allows to reduce global interrupt rate of idle system down to about
100 interrupts per core, permitting C3 and deeper C-states provide maximum
CPU power efficiency.
2009-05-03 17:47:21 +00:00
Alexander Motin
a40d9024df Add support for using i8254 and rtc timers as event sources for i386 SMP
system. Redistribute hard-/stat-/profclock events to other CPUs using IPI.
2009-05-02 12:59:47 +00:00
John Baldwin
b9dda9d6fe Fix a few nits in the earlier changes to prevent local information leakage
in AMD FPUs:
- Do not clear the affected state in the case that the FPU registers for
  the thread that already owns the FPU are changed via fpu_setregs().  The
  only local information the thread would see is its own state in that
  case.
- Fix a type mismatch for the dummy variable used in a "fld".  It accepts
  a float, not a double.

Reviewed by:	bde
Approved by:	so (cperciva)
MFC after:	1 month
2009-03-25 22:08:30 +00:00
John Baldwin
63de9515b7 Rename (fpu|npx)_cleanstate to (fpu|npx)_initialstate to better reflect
their purpose.

Inspired by:	bde
MFC after:	1 month
2009-03-25 14:17:08 +00:00
John Baldwin
2ee8325f42 A better fix for handling different FPU initial control words for different
ABIs:
- Store the FPU initial control word in the pcb for each thread.
- When first using the FPU, load the initial control word after restoring
  the clean state if it is not the standard control word.
- Provide a correct control word for Linux/i386 binaries under
  FreeBSD/amd64.
- Adjust the control word returned for fpugetregs()/npxgetregs() when a
  thread hasn't used the FPU yet to reflect the real initial control
  word for the current ABI.
- The Linux/i386 ABI for FreeBSD/i386 now properly sets the right control
  word instead of trashing whatever the current state of the FPU is.

Reviewed by:	bde
2009-03-05 19:42:11 +00:00
John Baldwin
20e9dede5e Some cleanups to the i386 FPU support:
- Remove the control word parameter to npxinit().  It was always set
  to __INITIAL_NPXCW__.
- Remove npx_cleanstate_ready as the cleanstate is always initalized
  when it is used.
- Improve the handling of the case when the FPU isn't present.  Now
  the npx0 device no longer succeeds in its probe so all of npx_attach()
  is skipped.  Also, we allow this case with SMP (though that shouldn't
  actually occur as all i386 systems that support SMP have FPUs) now.
  SMP was only an issue back when we had an FPU emulator which was not
  per-CPU.
- MFamd64: Clear some of the state in npx_cleanstate rather than leaving
  it as garbage.
- MFamd64: When a user thread first uses the FPU, use npx_cleanstate for
  the initial FPU state.

Reviewed by:	bde
2009-03-05 18:32:43 +00:00
Roman Divacky
af83f5d77c Change the functions to ANSI in those cases where it breaks promotion
to int rule. See ISO C Standard: SS6.7.5.3:15.

Approved by:	kib (mentor)
Reviewed by:	warner
Tested by:	silence on -current
2009-02-24 18:09:31 +00:00
John Baldwin
0b7dc0a7c6 Some whitespace and style fixes.
Submitted by:	bde (partly)
2009-02-23 15:39:24 +00:00
Kip Macy
3a6d1fcf9c merge 186535, 186537, and 186538 from releng_7_xen
Log:
 - merge in latest xenbus from dfr's xenhvm
 - fix race condition in xs_read_reply by converting tsleep to mtx_sleep

Log:
 unmask evtchn in bind_{virq, ipi}_to_irq

Log:
 - remove code for handling case of not being able to sleep
 - eliminate tsleep - make sleeps atomic
2008-12-29 06:31:03 +00:00
Warner Losh
0c3d9d1f6b style(9) nit: remove unnecessary {} pair. 2008-10-28 04:32:41 +00:00
Jung-uk Kim
29462bea1e Turn off CPU frequency change notifiers when the TSC is P-state invariant
or it is forced by setting 'kern.timecounter.invariant_tsc' tunable
to non-zero.
2008-10-21 00:38:00 +00:00
Yoshihiro Takahashi
05165e3276 - Add the i386_memio_map_load() function to load I/O address table.
- Add the bus_space_compare macro for bus_space consistency.
- Switch using the bus_space_map_load() in isa_load_resourcev().
2008-09-07 04:44:24 +00:00
John Baldwin
aa7c1c059f Add a very simple dpms(4) driver that uses the VESA BIOS DPMS calls to
turn off the external display during suspend and restore it to its
original state on resume.

MFC after:	2 weeks
2008-08-23 21:00:40 +00:00
Kip Macy
24b7d5cd1a Call in to xen for fpu handling when XEN is set
MFC after:	1 month
2008-08-15 21:43:38 +00:00