Commit Graph

78 Commits

Author SHA1 Message Date
Tycho Nightingale
8d2a55ca67 zero inputs to vm_page_initfake() for predictable results
Reviewed by:	kib
Submitted by:	Anton Rang <rang at acm.org>
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20162
2019-05-06 00:57:05 +00:00
Tycho Nightingale
96ca24dc32 remove the 4GB boundary requirement on PCI DMA segments
Reviewed by:	kib
Discussed with:	jhb
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19867
2019-04-19 13:43:33 +00:00
Konstantin Belousov
2d8bfbdcb2 Use correct type name.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 15:31:03 +00:00
Konstantin Belousov
f9feb09189 Correct handling of RMRR during early enumeration stages.
On some machines, DMAR contexts must be created before all devices
under the scope of the corresponding DMAR unit are enumerated.
Current code has two problems with that:
- scope lookup returns NULL device_t, which causes to skip creating a
  context with RMRR, which is fatal for the affected device.
- calculation of the final pci dbsf address fails if any bridge in the
  scope is not yet enumerated, because code relies on pcib_get_bus().

Make creation of contexts work either with device_t, or with DMAR PCI
scope paths.  Scope provides enough information to infer context
address, and it is directly matched against DMAR tables scopes.

When calculating bus addresses for the scope or device, use direct
pci_cfgregread(PCIR_SECBUS_1) to get the secondary bus number, instead
of pcib_get_bus().

The issue was observed on HP Gen servers, where iLO PCI devices are
located behind south bridge switch.  Turning on translation without
satisfying RMRR requests caused iLO to mostly hang, up to the level of
being unusable to control the server.

While there, remove hw.dmar.dmar_match_verbose tunable, and make the
normal logging under bootverbose useful and sufficient to diagnose
DRHD and RMRR parsing and matching.

Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 14:18:06 +00:00
Konstantin Belousov
c07640c430 Remove witness warning. dmar_bus_dmamap_create() does not sleep.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 14:03:59 +00:00
Konstantin Belousov
1ad4a0314f Reduce verbosity, do not announce details of irte programming by default.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-04-18 14:02:33 +00:00
Tycho Nightingale
9708c3a2b8 DMAR driver assumes all physical addresses are backed by a fully
initialized struct vm_page.

Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19753
2019-04-02 18:50:49 +00:00
Tycho Nightingale
cec2287b6a Use the BUS_DMA_NOWRITE flag to expose and create the read-only VT-d
IOMMU mappings.

Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19729
2019-03-27 20:15:51 +00:00
Mark Johnston
9978bd996b Add malloc_domainset(9) and _domainset variants to other allocator KPIs.
Remove malloc_domain(9) and most other _domain KPIs added in r327900.
The new functions allow the caller to specify a general NUMA domain
selection policy, rather than specifically requesting an allocation from
a specific domain.  The latter policy tends to interact poorly with
M_WAITOK, resulting in situations where a caller is blocked indefinitely
because the specified domain is depleted.  Most existing consumers of
the _domain KPIs are converted to instead use a DOMAINSET_PREF() policy,
in which we fall back to other domains to satisfy the allocation
request.

This change also defines a set of DOMAINSET_FIXED() policies, which
only permit allocations from the specified domain.

Discussed with:	gallatin, jeff
Reported and tested by:	pho (previous version)
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17418
2018-10-30 18:26:34 +00:00
Konstantin Belousov
d12c446550 Convert x86 cache invalidation functions to ifuncs.
This simplifies the runtime logic and reduces the number of
runtime-constant branches.

Reviewed by:	alc, markj
Sponsored by:	The FreeBSD Foundation
Approved by:	re (gjb)
Differential revision:	https://reviews.freebsd.org/D16736
2018-09-19 19:35:02 +00:00
John Baldwin
fd036deac1 Dynamically allocate IRQ ranges on x86.
Previously, x86 used static ranges of IRQ values for different types
of I/O interrupts.  Interrupt pins on I/O APICs and 8259A PICs used
IRQ values from 0 to 254.  MSI interrupts used a compile-time-defined
range starting at 256, and Xen event channels used a
compile-time-defined range after MSI.  Some recent systems have more
than 255 I/O APIC interrupt pins which resulted in those IRQ values
overflowing into the MSI range triggering an assertion failure.

Replace statically assigned ranges with dynamic ranges.  Do a single
pass computing the sizes of the IRQ ranges (PICs, MSI, Xen) to
determine the total number of IRQs required.  Allocate the interrupt
source and interrupt count arrays dynamically once this pass has
completed.  To minimize runtime complexity these arrays are only sized
once during bootup.  The PIC range is determined by the PICs present
in the system.  The MSI and Xen ranges continue to use a fixed size,
though this does make it possible to turn the MSI range size into a
tunable in the future.

As a result, various places are updated to use dynamic limits instead
of constants.  In addition, the vmstat(8) utility has been taught to
understand that some kernels may treat 'intrcnt' and 'intrnames' as
pointers rather than arrays when extracting interrupt stats from a
crashdump.  This is determined by the presence (vs absence) of a
global 'nintrcnt' symbol.

This change reverts r189404 which worked around a buggy BIOS which
enumerated an I/O APIC twice (using the same memory mapped address for
both entries but using an IRQ base of 256 for one entry and a valid
IRQ base for the second entry).  Making the "base" of MSI IRQ values
dynamic avoids the panic that r189404 worked around, and there may now
be valid I/O APICs with an IRQ base above 256 which this workaround
would incorrectly skip.

If in the future the issue reported in PR 130483 reoccurs, we will
have to add a pass over the I/O APIC entries in the MADT to detect
duplicates using the memory mapped address and use some strategy to
choose the "correct" one.

While here, reserve room in intrcnts for the Hyper-V counters.

PR:		229429, 130483
Reviewed by:	kib, royger, cem
Tested by:	royger (Xen), kib (DMAR)
Approved by:	re (gjb)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D16861
2018-08-28 21:09:19 +00:00
Alan Cox
49bfa624ac Eliminate the arena parameter to kmem_free(). Implicitly this corrects an
error in the function hypercall_memfree(), where the wrong arena was being
passed to kmem_free().

Introduce a per-page flag, VPO_KMEM_EXEC, to mark physical pages that are
mapped in kmem with execute permissions.  Use this flag to determine which
arena the kmem virtual addresses are returned to.

Eliminate UMA_SLAB_KRWX.  The introduction of VPO_KMEM_EXEC makes it
redundant.

Update the nearby comment for UMA_SLAB_KERNEL.

Reviewed by:	kib, markj
Discussed with:	jeff
Approved by:	re (marius)
Differential Revision:	https://reviews.freebsd.org/D16845
2018-08-25 19:38:08 +00:00
Alan Cox
44d0efb215 Eliminate kmem_alloc_contig()'s unused arena parameter.
Reviewed by:	hselasky, kib, markj
Discussed with:	jeff
Differential Revision:	https://reviews.freebsd.org/D16799
2018-08-20 15:57:27 +00:00
Alan Somers
6040822c4e Make timespecadd(3) and friends public
The timespecadd(3) family of macros were imported from NetBSD back in
r35029. However, they were initially guarded by #ifdef _KERNEL. In the
meantime, we have grown at least 28 syscalls that use timespecs in some
way, leading many programs both inside and outside of the base system to
redefine those macros. It's better just to make the definitions public.

Our kernel currently defines two-argument versions of timespecadd and
timespecsub.  NetBSD, OpenBSD, and FreeDesktop.org's libbsd, however, define
three-argument versions.  Solaris also defines a three-argument version, but
only in its kernel.  This revision changes our definition to match the
common three-argument version.

Bump _FreeBSD_version due to the breaking KPI change.

Discussed with:	cem, jilles, ian, bde
Differential Revision:	https://reviews.freebsd.org/D14725
2018-07-30 15:46:40 +00:00
Mark Johnston
2fb9a51077 Don't include DMAR map entry zone items in kernel dumps.
Such items may be allocated in the I/O path used by the dumper,
potentially causing the dump to fail. Since there is some precedent
in the DMAR driver for avoiding this problem using _NODUMP, apply
this workaround to the zone as well.

Reported and tested by:	mmacy
Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14422
2018-02-18 16:03:50 +00:00
Jeff Roberson
6f4acaf4c9 Add support for NUMA domains to bus dma tags. This causes all memory
allocated with a tag to come from the specified domain if it meets the
other constraints provided by the tag.  Automatically create a tag at
the root of each bus specifying the domain local to that bus if
available.

Reviewed by:	jhb, kib
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D13545
2018-01-12 23:34:16 +00:00
Pedro F. Giffuni
ebf5747bdb sys/x86: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-27 15:11:47 +00:00
Konstantin Belousov
dc00696a27 Correct operators precedence.
Also keep the calculated vm_page_alloc_contig() flags in the variable
to not re-evaluate it on the loop iteration.

Noted by:	alc
Sponsored by:	The FreeBSD Foundation
2017-11-09 13:09:07 +00:00
Jeff Roberson
8d6fbbb867 Replace manyinstances of VM_WAIT with blocking page allocation flags
similar to the kernel memory allocator.

This simplifies NUMA allocation because the domain will be known at wait
time and races between failure and sleeping are eliminated.  This also
reduces boilerplate code and simplifies callers.

A wait primitive is supplied for uma zones for similar reasons.  This
eliminates some non-specific VM_WAIT calls in favor of more explicit
sleeps that may be satisfied without new pages.

Reviewed by:	alc, kib, markj
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
2017-11-08 02:39:37 +00:00
Konstantin Belousov
fd15fee1ed Use IOAPIC PCI rid as the interrupt TLP source id for DMAR interrupt
remapping.

VT-d specification requires use of PCI rid as source id for IOAPICs
enumerated by PCI bus.  The values from the DMAR ACPI table should be
only used when IOAPIC is not on PCI.

Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
Hardware provided by:	Intel
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D12205
2017-09-08 19:45:37 +00:00
Jason A. Harmening
eb36b1d0bc Clean up MD pollution of bus_dma.h:
--Remove special-case handling of sparc64 bus_dmamap* functions.
  Replace with a more generic mechanism that allows MD busdma
  implementations to generate inline mapping functions by
  defining WANT_INLINE_DMAMAP in <machine/bus_dma.h>.  This
  is currently useful for sparc64, x86, and arm64, which all
  implement non-load dmamap operations as simple wrappers
  around map objects which may be bus- or device-specific.

--Remove NULL-checked bus_dmamap macros.  Implement the
  equivalent NULL checks in the inlined x86 implementation.
  For non-x86 platforms, these checks are a minor pessimization
  as those platforms do not currently allow NULL maps.  NULL
  maps were originally allowed on arm64, which appears to have
  been the motivation behind adding arm[64]-specific barriers
  to bus_dma.h, but that support was removed in r299463.

--Simplify the internal interface used by the bus_dmamap_load*
  variants and move it to bus_dma_internal.h

--Fix some drivers that directly include sys/bus_dma.h
  despite the recommendations of bus_dma(9)

Reviewed by:	kib (previous revision), marius
Differential Revision:	https://reviews.freebsd.org/D10729
2017-07-01 05:35:29 +00:00
Konstantin Belousov
cf619a92d2 Fix batched unload for DMAR busdma in qi mode.
Do not queue dmar_map_entries with zeroed gseq to
dmar_qi_invalidate_locked().  Zero gseq stops the processing in the qi
task.  Do not assign possibly uninitialized on-stack gseq to map
entries when requeuing them on unit tlb_flush queue.  Random garbage
in gsec is interpreted as too high invalidation sequence number and
again stop the processing in the task.

Make the sequence numbers generation completely contained in
dmar_qi_invalidate_locked() and dmar_qi_emit_wait_seq().  Upper code
directly passes boolean requesting emiting wait command instead of
trying to provide hint to avoid it by passing NULL gseq pointer.

Microoptimize the requeueing to tlb_flush queue by doing it for the
whole queue.

Diagnosed and tested by:	Brett Gutstein <bgutstein@rice.edu>
Discussed with:	alc
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-06-19 21:48:52 +00:00
Konstantin Belousov
366bb48985 Correct calculation of the entry->free_down in the invariants-checking
code.

Reported by:	maxim
Found by:	PVS studio scan
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-04-14 15:16:41 +00:00
Konstantin Belousov
476358b30f Timeout DMAR commands.
Implement timeouts for register-based DMAR commands.  Tunable/sysctl
hw.dmar.timeout specifies the timeout in nanoseconds, set it to zero
to allow infinite wait.  Default is 1ms.

Runtime modification of the sysctl is not safe, it is allowed for
debugging.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-03-27 07:06:45 +00:00
Konstantin Belousov
70fd237c5d Provide less laborius way to enable busdma DMAR to only short list of devices.
Kernel environment variable hw.busdma.default can take values 'bounce'
and 'dmar' and selects corresponding busdma backend as default.
Per-device environment variable hw.busdma.pci<domain>.<bus>.<slot>.<func>
takes the same values and overrides hw.busdma.default for the given device.

Note that even with hw.busdma.default=bounce, DMA translation engines
are still started if DMARs are enabled, to disable them use
hw.dmar.dma tunable, as before.

Sponsored by:	The FreeBSD Foundation
MFC after: 1 week
2017-03-26 00:40:35 +00:00
Konstantin Belousov
3d47c58b98 Avoid leaking allocated but unused context after creation race.
As noted in the comment, nothing special needs to be done to destroy
the unneeded context after the allocation race, but the context memory
itself still should to be freed.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-03-25 10:47:35 +00:00
Konstantin Belousov
5f8e5c7fa2 Do not create RMRR entries for identity-mapped domains.
It does not make sense since identity mapping already provides the
required mapping for RMRR ranges.  More, since identity page tables do
not reflect content of map entries for id domains, creating RMRR
entries makes domain data inconsistent.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-03-25 10:45:16 +00:00
Konstantin Belousov
ad969cb1cd Slight cleanup of the comment.
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-03-25 10:42:10 +00:00
Konstantin Belousov
3dd3c4503b Release DMAR table after using it.
Reported and tested by:	hps
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-12-05 11:42:09 +00:00
Konstantin Belousov
85d99487b8 Rename fast taskqueues used by DMAR to avoid naming conflict of the
sleepable and spin mutexes created by the queues.

Reported and tested by:	hps
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-12-05 11:41:09 +00:00
John Baldwin
264cd10809 Add additional constants.
- Add constants for the fields in the root-entry table address register,
  namely the root type type (RTT) and root table address (RTA) mask.
- Add macros for the bitmask of the domain ID field in the second word
  of context table entries as well as a helper macro (DMAR_CTX2_GET_DID)
  to extract the domain ID from a context table entry.

Reviewed by:	kib
MFC after:	1 month
Sponsored by:	Chelsio Communications
2016-08-09 19:02:14 +00:00
John Baldwin
f454e7ebf5 Add __printflike() to bus_describe_intr() to enable -Wformat checks.
Fix a few places that were passing a raw string as the format to use
a "%s" format string instead.

MFC after:	2 months
2016-08-04 18:29:16 +00:00
Pedro F. Giffuni
d9c9c81c08 sys: use our roundup2/rounddown2() macros when param.h is available.
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
2016-04-21 19:57:40 +00:00
Konstantin Belousov
e164cafc69 Add hw.dmar.batch_coalesce tunable/sysctl, which specifies rate at
which queued invalidation completion interrupt is requested with
regard to the queued invalidation requests.  In other words, setting
the value of the knob to N requests completion interrupt after N items
are processed.  Existing behaviour is restored by setting
hw.dmar.batch_coalesce=1.

The knob significantly decreases the DMAR qi interrupt rate at the
cost of slightly longer DMAR map entries recycling.

Sponsored by:	The FreeBSD Foundation
2016-04-17 10:56:56 +00:00
John Baldwin
cbc4d2db75 Remove taskqueue_enqueue_fast().
taskqueue_enqueue() was changed to support both fast and non-fast
taskqueues 10 years ago in r154167.  It has been a compat shim ever
since.  It's time for the compat shim to go.

Submitted by:	Howard Su <howard0su@gmail.com>
Reviewed by:	sephe
Differential Revision:	https://reviews.freebsd.org/D5131
2016-03-01 17:47:32 +00:00
Konstantin Belousov
2fe1339ea2 Some BIOSes ACPI bytecode needs to take (sleepable) acpi mutex for
acpi_GetInteger() execution.  Intel DMAR interrupt remapping code
needs to know UID of the HPET to properly route the FSB interrupts
from the HPET, even when interrupt remapping is disabled, and the code
is executed under some non-sleepable mutexes.

Cache HPET UIDs in the device softc at the attach time and provide
lock-less method to get UID, use the method from the dmar hpet
handling code instead of calling GetInteger().

Reported and tested by:	Larry Rosenman <ler@lerctr.org>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-02-20 13:37:04 +00:00
Bjoern A. Zeeb
6b1ad46a3b dmar_ctx_dtr() does not exist since r284869. Remove the static function
declaration to avoid a cmpile time warning.
2015-09-22 16:50:59 +00:00
Konstantin Belousov
0a44024a4e Comment only change, fix grammar and somewhat clarify the action.
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2015-08-14 13:51:59 +00:00
Konstantin Belousov
9b3df93bf1 Typo in comment. 2015-07-20 19:51:41 +00:00
Konstantin Belousov
1abfd35537 Split the DMAR unit domains and contexts. Domains carry address space
and related data structures.  Contexts attach requests initiators to
domains.  There is still 1:1 correspondence between contexts and
domains on the running system, since only busdma currently allocates
them, using dmar_get_ctx_for_dev().

Large part of the change is formal rename of the ctx to domain, but
patch also reworks the context allocation and free to allow for
independent domain creation.

The helper dmar_move_ctx_to_domain() is introduced for future use, to
reassign request initiator from one domain to another.  The hard issue
which is not yet resolved with the context move is proper handling (or
reserving) RMRR entries in the destination domain as required by ACPI
DMAR table for moved context.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2015-06-26 07:01:29 +00:00
Konstantin Belousov
69baeadc31 Remove several write-only variables, all reported by the gcc 4.9
buildkernel run.

Some of them were write-only under some kernel options, e.g. variables
keeping values only used by CTR() macros.  It costs nothing to the
code readability and correctness to eliminate the warnings in those
cases too by removing the local cached values used only for
single-access.

Review:	https://reviews.freebsd.org/D2665
Reviewed by:	rodrigc
Looked at by:	bjk
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-05-29 13:24:17 +00:00
Konstantin Belousov
35725a97c4 Explicitely enable queued invalidation completion interrupt when the
queue is started, not relying on the interrupt remaping method to
happen.  Also disable interrupts when shooting down the queue.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-05-29 09:17:59 +00:00
Konstantin Belousov
5f01ba81ae Account for the offset of the page run when allocating the
dmar_map_entry.  Non-zero offset both increases the required mapping
size, which is handled in dmar_bus_dmamap_load_something1(), and makes
it possible that allocated range crosses boundary, which needs a check
in dmar_gas_match_one().

Reported and tested by:	jimharris
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-04-08 01:55:22 +00:00
Konstantin Belousov
bdf31595ff When mapping an allocated entry, use the entry size, instead of the
requested size.  If tag restrictions caused split entry, its size is
less then requsted.

Hardware provided by:	Michael Fuckner <michael@fuckner.net>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-03-24 12:48:51 +00:00
Konstantin Belousov
a3d78402d2 Assert that the mapping loop makes progress.
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-03-24 12:46:21 +00:00
Konstantin Belousov
0a110d5b17 Use VT-d interrupt remapping block (IR) to perform FSB messages
translation.  In particular, despite IO-APICs only take 8bit apic id,
IR translation structures accept 32bit APIC Id, which allows x2APIC
mode to function properly.  Extend msi_cpu of struct msi_intrsrc and
io_cpu of ioapic_intsrc to full int from one byte.

KPI of IR is isolated into the x86/iommu/iommu_intrmap.h, to avoid
bringing all dmar headers into interrupt code. The non-PCI(e) devices
which generate message interrupts on FSB require special handling. The
HPET FSB interrupts are remapped, while DMAR interrupts are not.

For each msi and ioapic interrupt source, the iommu cookie is added,
which is in fact index of the IRE (interrupt remap entry) in the IR
table. Cookie is made at the source allocation time, and then used at
the map time to fill both IRE and device registers. The MSI
address/data registers and IO-APIC redirection registers are
programmed with the special values which are recognized by IR and used
to restore the IRE index, to find proper delivery mode and target.
Map all MSI interrupts in the block when msi_map() is called.

Since an interrupt source setup and dismantle code are done in the
non-sleepable context, flushing interrupt entries cache in the IR
hardware, which is done async and ideally waits for the interrupt,
requires busy-wait for queue to drain.  The dmar_qi_wait_for_seq() is
modified to take a boolean argument requesting busy-wait for the
written sequence number instead of waiting for interrupt.

Some interrupts are configured before IR is initialized, e.g. ACPI
SCI.  Add intr_reprogram() function to reprogram all already
configured interrupts, and call it immediately before an IR unit is
enabled.  There is still a small window after the IO-APIC redirection
entry is reprogrammed with cookie but before the unit is enabled, but
to fix this properly, IR must be started much earlier.

Add workarounds for 5500 and X58 northbridges, some revisions of which
have severe flaws in handling IR.  Use the same identification methods
as employed by Linux.

Review:	https://reviews.freebsd.org/D1892
Reviewed by:	neel
Discussed with:	jhb
Tested by:	glebius, pho (previous versions)
Sponsored by:	The FreeBSD Foundation
MFC after:	3 weeks
2015-03-19 13:57:47 +00:00
Konstantin Belousov
dcc33b0a8a Provide definitions for all descriptors types in the DMAR invalidation
queue.  They are for first-level translations and device TLB.

Review:	https://reviews.freebsd.org/D1892
Reviewed by:	neel
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-03-19 13:05:55 +00:00
Konstantin Belousov
0f5830b045 Fix syntax error.
Review:	https://reviews.freebsd.org/D1892
Found by:	neel
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2015-03-19 13:03:58 +00:00
Konstantin Belousov
743ffdaf3c When initial placement of the new entry crosses the boundary,
allocator tries to move the entry up, after the boundary.  The new
location may still fail to satisfy boundary requirement, for instance,
if the boundary is set to page size, and allocation is of multiple
pages.

Recheck that boundary is not crossed after the move.  If it is
crossed, give up on allocating the whole entry and split it.

Reported by:	Michael Fuckner <michael@fuckner.net>, running nvme(4)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-03-17 22:00:11 +00:00
Konstantin Belousov
582b656248 When inserting new entry into the address map, ensure that not only
next entry does not intersect with the tail of the new entry, but also
that previous entry is also before new entry start.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-03-17 21:55:33 +00:00