Convert all remaining references to that field to "ref_count" and update
comments accordingly. No functional change intended.
Reviewed by: alc, kib
Sponsored by: Intel, Netflix
Differential Revision: https://reviews.freebsd.org/D21768
There are several mechanisms by which a vm_page reference is held,
preventing the page from being freed back to the page allocator. In
particular, holding the page's object lock is sufficient to prevent the
page from being freed; holding the busy lock or a wiring is sufficent as
well. These references are protected by the page lock, which must
therefore be acquired for many per-page operations. This results in
false sharing since the page locks are external to the vm_page
structures themselves and each lock protects multiple structures.
Transition to using an atomically updated per-page reference counter.
The object's reference is counted using a flag bit in the counter. A
second flag bit is used to atomically block new references via
pmap_extract_and_hold() while removing managed mappings of a page.
Thus, the reference count of a page is guaranteed not to increase if the
page is unbusied, unmapped, and the object's write lock is held. As
a consequence of this, the page lock no longer protects a page's
identity; operations which move pages between objects are now
synchronized solely by the objects' locks.
The vm_page_wire() and vm_page_unwire() KPIs are changed. The former
requires that either the object lock or the busy lock is held. The
latter no longer has a return value and may free the page if it releases
the last reference to that page. vm_page_unwire_noq() behaves the same
as before; the caller is responsible for checking its return value and
freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is
introduced for use in pmap_extract_and_hold(). It fails if the page is
concurrently being unmapped, typically triggering a fallback to the
fault handler. vm_page_wire() no longer requires the page lock and
vm_page_unwire() now internally acquires the page lock when releasing
the last wiring of a page (since the page lock still protects a page's
queue state). In particular, synchronization details are no longer
leaked into the caller.
The change excises the page lock from several frequently executed code
paths. In particular, vm_object_terminate() no longer bounces between
page locks as it releases an object's pages, and direct I/O and
sendfile(SF_NOCACHE) completions no longer require the page lock. In
these latter cases we now get linear scalability in the common scenario
where different threads are operating on different files.
__FreeBSD_version is bumped. The DRM ports have been updated to
accomodate the KPI changes.
Reviewed by: jeff (earlier version)
Tested by: gallatin (earlier version), pho
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20486
error in the function hypercall_memfree(), where the wrong arena was being
passed to kmem_free().
Introduce a per-page flag, VPO_KMEM_EXEC, to mark physical pages that are
mapped in kmem with execute permissions. Use this flag to determine which
arena the kmem virtual addresses are returned to.
Eliminate UMA_SLAB_KRWX. The introduction of VPO_KMEM_EXEC makes it
redundant.
Update the nearby comment for UMA_SLAB_KERNEL.
Reviewed by: kib, markj
Discussed with: jeff
Approved by: re (marius)
Differential Revision: https://reviews.freebsd.org/D16845
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
The i915 video driver doesn't depend on agp(4) anymore for Sandybridge
and later GPUs. Therefore, there is no need to attach agp(4) to those
devices.
While here, fix `agp_i965_res_spec` to include the aperture base for
GEN4 and GEN5.
Reviewed by: kib
Approved by: kib
Differential Revision: https://reviews.freebsd.org/D5586
This update brings initial support for Haswell GPUs.
Tested by: Many users of FreeBSD, PC-BSD and HardenedBSD
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5554
IGP may declare subclass as either VGA-compatible, or non-VGA. The
difference is that in the later case, IGP does not claim VGA cycles.
Other than that, the device functions normally, and agp_i810 should
attach to it.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
updating the GTT and flushing the AGP TLB by storing the GTT in
write-combining memory.
On x86 flushing the AGP TLB is done by an I/O operation or a store to a
MMIO register in uncacheable memory. Both cases imply that WC buffers are
flushed so no memory barriers are needed.
On powerpc there is no WC memory type. It maps to uncacheable memory and
two stores to uncacheable memory, such as to the GTT and then to an MMIO
register, are strongly ordered, so no memory barriers are needed either.
MFC after: 1 month
the queue where to enqueue pages that are going to be unwired.
- Add stronger checks to the enqueue/dequeue for the pagequeues when
adding and removing pages to them.
Of course, for unmanaged pages the queue parameter of vm_page_unwire() will
be ignored, just as the active parameter today.
This makes adding new pagequeues quicker.
This change effectively modifies the KPI. __FreeBSD_version will be,
however, bumped just when the full cache of free pages will be
evicted.
Sponsored by: EMC / Isilon storage division
Reviewed by: alc
Tested by: pho
shifts into the sign bit. Instead use (1U << 31) which gets the
expected result.
This fix is not ideal as it assumes a 32 bit int, but does fix the issue
for most cases.
A similar change was made in OpenBSD.
Discussed with: -arch, rdivacky
Reviewed by: cperciva
The flag was mandatory since r209792, where vm_page_grab(9) was
changed to only support the alloc retry semantic.
Suggested and reviewed by: alc
Sponsored by: The FreeBSD Foundation
future further optimizations where the vm_object lock will be held
in read mode most of the time the page cache resident pool of pages
are accessed for reading purposes.
The change is mostly mechanical but few notes are reported:
* The KPI changes as follow:
- VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK()
- VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK()
- VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK()
- VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED()
(in order to avoid visibility of implementation details)
- The read-mode operations are added:
VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(),
VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()
* The vm/vm_pager.h namespace pollution avoidance (forcing requiring
sys/mutex.h in consumers directly to cater its inlining functions
using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h
consumers now must include also sys/rwlock.h.
* zfs requires a quite convoluted fix to include FreeBSD rwlocks into
the compat layer because the name clash between FreeBSD and solaris
versions must be avoided.
At this purpose zfs redefines the vm_object locking functions
directly, isolating the FreeBSD components in specific compat stubs.
The KPI results heavilly broken by this commit. Thirdy part ports must
be updated accordingly (I can think off-hand of VirtualBox, for example).
Sponsored by: EMC / Isilon storage division
Reviewed by: jeff
Reviewed by: pjd (ZFS specific review)
Discussed with: alc
Tested by: pho
flush wait on the Gen2 chipsets. Confirmed by the inspection of the
Linux agp code.
Submitted by: Taku YAMAMOTO <taku@tackymt.homeip.net>
MFC after: 2 weeks
to pull vm_param.h was removed. Other big dependency of vm_page.h on
vm_param.h are PA_LOCK* definitions, which are only needed for
in-kernel code, because modules use KBI-safe functions to lock the
pages.
Stop including vm_param.h into vm_page.h. Include vm_param.h
explicitely for the kernel code which needs it.
Suggested and reviewed by: alc
MFC after: 2 weeks
Don't use Maxmem when the amount of memory is meant. Use realmem instead.
Maxmem is not only a MD variable, it represents the highest physical memory
address in use. On systems where memory is sparsely layed-out the highest
memory address and the amount of memory are not interchangeable. Scaling the
AGP aperture based on the actual amount of memory (= realmem) rather than
the available memory (= physmem) makes sure there's consistent behaviour
across architectures.
agp_i810.c:
While arguably the use of Maxmem can be considered correct, replace its use
with realmem anyway. agp_i810.c is specific to amd64, i386 & pc98, which
have a dense physical memory layout. Avoiding Maxmem here is done with an
eye on copy-n-paste behaviour in general and to avoid confusion caused by
using realmem in agp.c and Maxmem in agp_i810.c.
In both cases, remove the inclusion of md_var.h
operations required by GEMified i915.ko. It also attaches to SandyBridge
and IvyBridge CPU northbridges now.
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
scope of the object lock in agp_i810.c. (In this specific case, the scope
of the object lock shouldn't matter, but I don't want to create a bad
example that might be copied to a case where it did matter.)
Reviewed by: kib
Note that you need at least xf86-video-intel 2.4.3 for this to work.
The G4X doesn't put the GATT into the same area of stolen memory
as all the other chips and older versions of the driver didn't
handle that properly.
Tested by: ganbold
Approved by: kib
MFC after: 2 weeks
g33 based chips use a different method of identifying the gtt size.
g45 based chips gtt is located in a different area of stolen memory.
Approved by: jhb (mentor)
MFC after: 2 weeks
bumped to 800004 to note the change though userland apps should not be
affected since they use <sys/agpio.h> rather than the headers in
sys/dev/agp.
Discussed with: anholt
Repocopy by: simon
/dev/agpgart and agp_free_res() frees resources like the BAR for the
aperture. Splitting this up lets chipset-specific detach routines
manipulate the aperture during their detach routines without panicing.
MFC after: 1 week
Reviewed by: anholt
This adds a function to agp.c to set the aperture resource ID if it's
not the usual AGP_APBASE. Previously, agp.c had been assuming
AGP_APBASE, which resulted in incorrect agp_info, and contortions by
agp_i810.c to work around it.
This also adds functions to agp.c for default AGP_GET_APERTURE() and
AGP_SET_APERTURE(), which return the aperture resource size and disallow
aperture size changes. Moving to these for our AGP drivers will likely
result in stability improvements. This should fix 855-class aperture
size detection.
Additionally, refuse to attach agp_i810 when some RAM is above 4GB and
the GART can't reference memory that high. This should be very rare.
The correct solution would be bus_dma conversion for agp, which is
beyond the scope of this change. Other AGP drivers could likely use
this change as well.
G33/Q35/Q33 AGP support is also included, but disconnected by default
due to lack of testing.
PR: kern/109724 (855 aperture issue)
Submitted by: FUJIMOTO Kou<fujimoto@j.dendai.ac.jp>
Approved by: re (hrs)
The key problem was that the aperture size detection using the MSAC bit
doesn't work -- the bit appears to be set even when it shouldn't be. Linux
takes a different approach, testing for a bit of the GMADR (PCIR_BAR(2)) being
set. However, as I don't think that's a safe way to test aperture size, we
just allocate the resource and check its size. This also pointed out that
agp_generic_attach hadn't been allocating our aperture resource, which may
have caused problems in some cases.
Also corrected is a minor copy-and-pasteo in an error case.
PR: kern/103079
Submitted by: mnag
Tested on: i945GM, i915GM
MFC after: 2 weeks
