Commit Graph

47 Commits

Author SHA1 Message Date
Jung-uk Kim
fd90e2ed54 CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten
years for head.  However, it is continuously misused as the mpsafe argument
for callout_init(9).  Deprecate the flag and clean up callout_init() calls
to make them more consistent.

Differential Revision:	https://reviews.freebsd.org/D2613
Reviewed by:	jhb
MFC after:	2 weeks
2015-05-22 17:05:21 +00:00
Ruslan Bukin
a8c5ea04b4 Provide the number of interrupt resources added to the list
by using extra argument, so caller will know that.
2015-05-15 13:55:18 +00:00
Nathan Whitehorn
ad7ea69dce Convert PTE eviction lock from an RW lock to a RM lock. It is held for
writing approximately never (< 0.00000001% under heavy VM load, and it can
go for months without ever being acquired in normal operation). This
provides a 10% (2-minute) improvement in wall clock time for make -j32
buildworld on a 4-core 32-thread POWER8.
2015-03-16 16:29:33 +00:00
Nathan Whitehorn
1cd30eb6dd Deallocate any leftover page table entries in the LPAR at boot. This
prevents contamination from a previous kernel (e.g. after shutdown -r).
2015-03-13 00:08:58 +00:00
Nathan Whitehorn
8704b9e9db The H_VIO_SIGNAL hypercall only enables interrupts for future received
packets and does not schedule interrupts for any packets currently
enqueued. Close two races where enqueued packets may not ever trigger
interrupts. The first of these, at adapter initialization time, was
especially severe since a rush of enqueued packets could actually fill
the receive buffer completely, stalling the interface forever.

MFC after:	2 weeks
2015-03-12 17:01:30 +00:00
Nathan Whitehorn
827cc9b981 New pmap implementation for 64-bit PowerPC processors. The main focus of
this change is to improve concurrency:
- Drop global state stored in the shadow overflow page table (and all other
  global state)
- Remove all global locks
- Use per-PTE lock bits to allow parallel page insertion
- Reconstruct state when requested for evicted PTEs instead of buffering
  it during overflow

This drops total wall time for make buildworld on a 32-thread POWER8 system
by a factor of two and system time by a factor of three, providing performance
20% better than similarly clocked Core i7 Xeons per-core. Performance on
smaller SMP systems, where PMAP lock contention was not as much of an issue,
is nearly unchanged.

Tested on:	POWER8, POWER5+, G5 UP, G5 SMP (64-bit and 32-bit kernels)
Merged from:	user/nwhitehorn/ppc64-pmap-rework
Looked over by:	jhibbits, andreast
MFC after:	3 months
Relnotes:	yes
Sponsored by:	FreeBSD Foundation
2015-02-24 21:37:20 +00:00
Nathan Whitehorn
09ae596bf6 Fix race in interrupt handling that could cause IO to hang up under heavy
load.
2015-02-23 20:38:00 +00:00
Nathan Whitehorn
a4c6f6e512 Add error reporting to interrupt CPU binding. 2015-02-10 00:57:26 +00:00
Nathan Whitehorn
0174acd439 Distribute interrupts across multiple CPUs in SMP configurations instead of sending them
all to CPU 0.
2015-02-09 19:21:54 +00:00
Nathan Whitehorn
3b33679869 Mark invalid page table entries correctly for PMAP as well as for the
hypervisor. This prevents an infinite loop where processes with evicted
pages would page fault forever when PMAP decided the evicted pages on
which the process was faulting was actually present and did not need to
be restored.

Found while building LLVM with make -j32.

Sponsored by:	FreeBSD Foundation
2015-02-09 15:58:27 +00:00
Bjoern A. Zeeb
66fe94642d Properly hide a variable under #ifdef as it is only used inside the
specific #ifdef block otherwise leaving an unused variable and breaking
other kernel builds.
2015-02-09 11:34:45 +00:00
Nathan Whitehorn
07e765cd81 Fix typo in PTE insertion overflow handling: use the page we're actually
returning, not the one we just looked at.
2015-02-09 07:08:54 +00:00
Nathan Whitehorn
f1e4841797 Technically speaking, using one virtal processor area for all CPUs is a
violation of the spec. Make duplicate entries for each CPU.
2015-02-09 02:13:36 +00:00
Nathan Whitehorn
a840d2f5b8 Add some error checking on the supplied page size list. This makes sure
that we (a) get the correct large page size to provide to pmap and (b)
we can alert the user if running under incorrectly-configured PowerKVM
on POWER7 and POWER8 systems.

MFC after:	1 week
2015-02-08 16:50:00 +00:00
Nathan Whitehorn
c7be335e3e Fix bug in mapppings of multiple pages exposed by updates to the VSCSI
support in QEMU. Each page of a many page mapping was getting mapped to
the same physical address, which is not the desired behavior.

MFC after:	1 week
2015-01-27 07:20:00 +00:00
Nathan Whitehorn
08b96b9ff5 Restore use of ofw_bus_intr_to_rl() in the pseries vdevice driver after fixing
ofw_bus_intr_to_rl() to match the spec for unspecified interrupt-parent
properties.
2015-01-05 21:39:35 +00:00
Nathan Whitehorn
efabbd4c02 Revert r272109 locally, which is not quite equivalent in how it deals with
missing interrupt-parent properties. A better solution will come later,
but this restores pseries in QEMU for the time being.
2015-01-05 18:15:16 +00:00
Nathan Whitehorn
44d29d4762 Allow booting with both a real Open Firmware tree and a flattened version of
the Open Firmware, as provided by petitboot, for example. Note that this is
not quite complete, since RTAS instantiation still depends on callable
firmware.

MFC after:	2 weeks
2015-01-01 22:26:12 +00:00
Ian Lepore
c47d4cde39 Replace multiple nearly-identical copies of code to walk through an FDT
node's interrupts=<...> property creating resource list entries with a
single common implementation.  This change makes ofw_bus_intr_to_rl() the
one true copy of that code and removes the copies of it from other places.

This also adds handling of the interrupts-extended property, which allows
specifying multiple interrupts for a node where each interrupt can have a
separate interrupt-parent.  The bindings for this state that the property
cells contain an xref phandle to the interrupt parent followed by whatever
interrupt info that parent normally expects.  This leads to having a
variable number of icells per interrupt in the property.  For example you
could have <&intc1 1 &intc2 26 9 0 &intc3 9 4>.

Differential Revision: https://reviews.freebsd.org/D803
2014-09-25 15:02:33 +00:00
Gleb Smirnoff
e2efa70e1c Mechanically convert to if_inc_counter(). 2014-09-19 09:20:34 +00:00
Nathan Whitehorn
65d08437ef Move Open Firmware device root on PowerPC, ARM, and MIPS systems to
a sub-node of nexus (ofwbus) rather than direct attach under nexus. This
fixes FDT on x86 and will make coexistence with ACPI on ARM systems easier.
SPARC is unchanged.

Reviewed by:	imp, ian
2014-02-05 14:44:22 +00:00
Nathan Whitehorn
bbc6da03ef Open Firmware interrupt specifiers can consist of arbitrary-length byte
strings and include arbitrary information (IRQ line/domain/sense). When the
ofw_bus_map_intr() API was introduced, it assumed that, as on most systems,
these were either 1 cell, containing an interrupt line, or 2, containing
a line number plus a sense code. It turns out a non-negligible number of
ARM systems use 3 (or even 4!) cells for interrupts, so make this more
general.
2014-02-01 17:17:35 +00:00
Nathan Whitehorn
d2a94a7764 Set max_lun to zero. This field is ignored unless we are manually probing
LUNs anyway, and we certainly don't want to probe 2^32 values by hand in
that case.

MFC after:	2 weeks
2013-12-15 16:58:23 +00:00
Andreas Tobler
f5dfbe2f69 Do some cosmetic fixes. 2013-12-05 21:35:52 +00:00
Nathan Whitehorn
c1cb22d755 Rearchitect platform memory map parsing to make it less
Open Firmware-centric:
- Keep the static list of regions in platform.c instead of ofw_machdep.c
- Move various merging and sorting operations to platform.c as well
- Move apple_hacks code out of ofw_machdep.c and into platform_powermac.c,
  where it belongs
- Move CHRP-specific dynamic-reconfiguration memory parsing into
  platform_chrp.c instead of pretending it is shared code
2013-12-01 19:43:15 +00:00
Nathan Whitehorn
9b00150b2c This driver doesn't need the /options node, so don't check for it. 2013-11-27 22:02:55 +00:00
Andreas Tobler
90653c1c5c Take care to handle the full 16 byte buffer in the get/put routines. Also,
skip the VTERM header once when receiving data from the hypervisor call when
we have a HVTERMPROT connection.

MFC after:	1 week
2013-11-26 05:26:10 +00:00
Andreas Tobler
4af7c8949e Limit the large page size to 16MB for now.
MFC after:	1 week
2013-11-23 18:52:14 +00:00
Nathan Whitehorn
e941e1e94a For PCI<->PCI bridges, #address-cells may be 3. Allow this when parsing the
ibm,dma-window properties. This is especially a concern when
#ibm,dma-address-cells is not specified and we have to use the regular
#address-cells property.

MFC after:	1 week
2013-11-21 15:41:52 +00:00
Nathan Whitehorn
817ba5c0b8 Following the approach with ACPI DMAR on x86, split IOMMU handling into
a variant PCI bus instead of trying to shoehorn it into the PCI host bridge
adapter. Besides matching better the architecture on other platforms, this
also allows systems with multiple partitionable endpoints per PCI host
bridge to work correctly.
2013-11-12 16:15:09 +00:00
Nathan Whitehorn
acf9bb339a Actually add IOMMU domain to the list of known mappings. This fixes a bug
where multiple devices in the same IOMMU domain would be allocated
conflicting mappings unless they also shared a DMA tag.

MFC after:	3 days
2013-11-12 16:14:45 +00:00
Andreas Tobler
12e7de228a Ouch, do the shift properly. An uin64_t needs the right postfix when shifting. 2013-10-31 20:42:47 +00:00
Nathan Whitehorn
453319bf53 The ofw_bus() routines invoke the device's parent, not the device itself,
so cease iterating when the parent is NULL, not when the device is.

MFC after:	5 days
2013-10-29 21:08:20 +00:00
Nathan Whitehorn
ef5758fa10 Implement extended LUN support. If PIM_EXTLUNS is set by a SIM, encode
the upper 32-bits of the LUN, if possible, into the target_lun field as
passed directly from the REPORT LUNs response. This allows extended LUN
support to work for all LUNs with zeros in the lower 32-bits, which covers
most addressing modes without breaking KBI. Behavior for drivers not
setting PIM_EXTLUNS is unchanged. No user-facing interfaces are modified.

Extended LUNs are stored with swizzled 16-bit word order so that, for
devices implementing LUN addressing (like SCSI-2), the numerical
representation of the LUN is identical with and without PIM_EXTLUNS. Thus
setting PIM_EXTLUNS keeps most behavior, and user-facing LUN IDs, unchanged.
This follows the strategy used in Solaris. A macro (CAM_EXTLUN_BYTE_SWIZZLE)
is provided to transform a lun_id_t into a uint64_t ordered for the wire.

This is the second part of work for full 64-bit extended LUN support and is
designed to a bridge for stable/10 to the final 64-bit LUN code. The
third and final part will involve widening lun_id_t to 64 bits and will
not be MFCed. This third part will break the KBI but will keep the KPI
unchanged so that all drivers that will care about this can be updated now
and not require code changes between HEAD and stable/10.

Reviewed by:	scottl
MFC after:	2 weeks
2013-10-29 15:36:58 +00:00
Gleb Smirnoff
66e01d73cd - Provide necessary includes.
- Remove unnecessary includes.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2013-10-29 11:17:49 +00:00
Nathan Whitehorn
258dbffe6e Clean up missed header references. 2013-10-26 17:54:31 +00:00
Nathan Whitehorn
f214848258 Add two new interfaces to ofw_bus:
- ofw_bus_map_intr()
  Maps an (iparent, IRQ) tuple to a system-global interrupt number in some
  platform dependent way. This is meant to be implemented as a replacement
  for [FDT_]MAP_IRQ() that is an MI interface that knows about the bus
  hierarchy.
- ofw_bus_config_intr()
  Configures an interrupt (previously mapped) based on firmware sense flags.
  This replaces manual interpretation of the sense field in bus drivers and
  will, in a follow-up, allow that interpretation to be redirected to the PIC
  drivers where it belongs. This will eventually replace the tables in
  /sys/dev/fdt/fdt_ARCH.c

The PowerPC/AIM code has been converted to use these globally, with an
implementation in terms of MAP_IRQ() and powerpc_config_intr(), assuming
OpenPIC, at the bus root in nexus(4). The ofw_bus_config_intr() will shortly
be integrated into pic_if.m and bounced through nexus into the PIC tree.

FDT integration will happen significantly later due to larger testing
requirements. This patch in general also lays the groundwork for the removal
of /sys/dev/fdt/fdt_ARCH.c and machine/fdt.h.
2013-10-23 17:24:21 +00:00
Nathan Whitehorn
6c20c40f32 Add multicast filter control. 2013-10-20 14:01:09 +00:00
Nathan Whitehorn
af4c3211f8 Clean up some debug code that snuck in. Add a block comment on future work. 2013-10-20 01:40:59 +00:00
Nathan Whitehorn
757e5b2917 Add initial driver for POWER hypervisor interpartition ethernet. This is
sufficient to pass traffic but needs some more work before merging to
STABLE.
2013-10-20 01:31:09 +00:00
Nathan Whitehorn
0ef685586f Allow hypervisor calls with more than 7 arguments. 2013-10-20 01:28:39 +00:00
Nathan Whitehorn
391dff8624 Add driver for the PAPR VSCSI virtual SCSI controller. This lets FreeBSD
install directly into standard POWER LPARs, as found for example in
QEMU. The core of this device is the SCSI RDMA protocol as also found in
Infiniband. The SRP portions of the driver will be factored out and placed
/sys/cam in the future to allow them to be used for IB storage. Thanks to
Scott Long for a great deal of implementation help.

Reviewed by:	scottl
Approved by:	re (kib)
2013-09-28 15:46:03 +00:00
Nathan Whitehorn
9f70672718 Rework handling of ofw_quiesce(), making it the responsibility of the
platform modules. Whether to call this function or not is highly machine
dependent: on some systems, it is required, while on others it breaks
everything. Platform modules are in a better position to figure this
out. This is required for POWER hypervisor SCSI to work correctly. There
are no functional changes on Powermac systems.

Approved by:	re (kib)
2013-09-27 13:12:47 +00:00
Nathan Whitehorn
91a01b9161 Make sure that ref and changed bits propagate back to the VM layer
whenever pages are unmapped. The old code had several races that could
allow these to become stale.

Approved by:	re (kib)
2013-09-27 13:11:20 +00:00
Nathan Whitehorn
dbd2830c65 As idling the CPU no longer causes hangs on QEMU, reenable the pSeries
cpu_idle() routine.

Approved by:	re (delphij)
2013-09-26 21:18:46 +00:00
Nathan Whitehorn
145341e9cd Fix bug where UART unit number was not set properly, which prevented
operation on systems with multiple serial ports. Also turn on
interrupts for the UART device, which were disabled due to a
now-fixed bug in QEMU.

Approved by:	re (gjb)
2013-09-26 18:01:32 +00:00
Nathan Whitehorn
7a8d25c037 Merge in support for PAPR-compliant (Power Architecture Platform
Requirements) systems from the projects/pseries branch. This in principle
includes all IBM POWER hardware released in the last 15 years with the
exception of POWER3-based systems when run in 64-bit mode. The main
development target, however, has been the PAPR logical partition support
that is the default target in KVM on POWER and QEMU -- mileage may vary
on actual hardware at present. Much of the heavy lifting here was done
by Andreas Tobler.

Approved by:	re (kib)
2013-09-17 17:37:04 +00:00